HowTo improve Debian packages using OBS October 23rd, 2013
This is a quick tutorial how to branch packages on OBS (hosted by Kolab Systems), for improving the Debian packages.
Please see also the step by step instructions in the Kolab documentation.
My steps show how to fix the bug 2404:
- Register a user https://obs.kolabsys.com/user/register_user
- Create a home project by clicking “Home Project” in the upper right corner.
- Go to project “Kolab Winterfell”: https://obs.kolabsys.com/project/show?project=Kolab%3AWinterfell
- Since the problem is a missing file (auth_cache.py) in package pykolab, go to package https://obs.kolabsys.com/package/show?package=pykolab&project=Kolab%3AWinterfell
- Branch the package by clicking “Branch package”. Confirm with “OK”
- Now you are redirected to your branched version of the package, in my example: https://obs.kolabsys.com/package/show?package=pykolab&project=home%3Atimotheus%3Abranches%3AKolab%3AWinterfell
- Go back to your project, https://obs.kolabsys.com/project/show?project=home%3Atimotheus%3Abranches%3AKolab%3AWinterfell; click on Repositories, and remove all distributions that are of no interest to you. That way there are no unnecessary builds on the server. I keep only Debian 7.0. If the 64 bit version is what you want, you can also disable the 32 bit build, by hovering over the green icon, and selecting “Explicitly disable”.
Also make sure that the “publish” flag is set for the distribution that you want to use. - Now go back to “Overview”, and click on package “pykolab”. You could actually edit files directly in the web interface, and that would trigger a new build. But for the issue regarding the missing file, we need to edit a file inside the debian.tar.gz, and it is easier to use the osc command line tool for that.
- Install osc on your own local machine. It is a bit like a git client. It is availabe for many distributions.
- Checkout your project to your local machine:
osc -A https://obs.kolabsys.com:444/ checkout home:timotheus:branches:Kolab:Winterfell cd home:timotheus:branches:Kolab:Winterfell/pykolab
- Now you untar the file debian.tar.gz, and edit the file
tar xzf debian.tar.g
cd debian
vi pykolab.install - check https://cgit.kolab.org/pykolab/tree/pykolab/auth/ldap for the missing files, and add new lines, and save:
usr/lib*/python*/*-packages/pykolab/auth/ldap/auth_cache.py
usr/lib*/python*/*-packages/pykolab/auth/ldap/syncrepl.py - tar the file again, and delete the debian directory:
tar czf debian.tar.gz debian
rm -Rf debian - In order for the package to be actually installed when you want to test it, and because it is good practise anyway, you have to update the files debian.changelog and pykolab.dsc as well:
pykolab.dsc: modify the line with Version, eg. Version: 0.6.8-0~kolab1 to Version: 0.6.8-0~kolab1.1
debian.changelog: copy the top lines, change the version number the same way as in pykolab.dsc, modify the description, and your author name - Before you can commit, make sure you add the following lines to your file ~/.oscrc:
[https://obs.kolabsys.com:444]
user=timotheus
pass=*** - Now commit your changes with the osc tool:
osc commit -m “adding missing files auth_cache.py and syncrepl.py, fixing bug 2404” - This will upload your changes to the OBS server, and immediately trigger a rebuild of the package pykolab.
- You can now install your packages, similar to installing the nightly packages, just replace the username in /etc/apt/sources.list.d/kolab.list
Edit: you should add the development packages too, since currently 3.1 is not stable for Debian yet:username=timotheus cat > /etc/apt/sources.list.d/kolab.list <<FINISH deb http://obs.kolabsys.com:82/Kolab:/3.1/Debian_7.0/ ./ deb http://obs.kolabsys.com:82/Kolab:/3.1:/Updates/Debian_7.0/ ./ deb http://obs.kolabsys.com:82/Kolab:/Winterfell/Debian_7.0/ ./ deb http://obs.kolabsys.com:82/home:/$username:/branches:/Kolab:/Winterfell/Debian_7.0/ ./ FINISH wget http://obs.kolabsys.com:82/Kolab:/3.1/Debian_7.0/Release.key apt-key add Release.key; rm -rf Release.key wget http://obs.kolabsys.com:82/Kolab:/3.1:/Updates/Debian_7.0/Release.key apt-key add Release.key; rm -rf Release.key wget http://obs.kolabsys.com:82/Kolab:/Winterfell/Debian_7.0/Release.key apt-key add Release.key; rm -rf Release.key wget http://obs.kolabsys.com:82/home:/$username:/branches:/Kolab:/Winterfell/Debian_7.0/Release.key apt-key add Release.key; rm -rf Release.key cat > /etc/apt/preferences.d/kolab <<FINISH Package: * Pin: origin obs.kolabsys.com Pin-Priority: 501 FINISH apt-get update apt-get install kolab
- When you have tested your packages, you can click “Submit package” on your pykolab project page: https://obs.kolabsys.com/package/show?package=pykolab&project=home%3Atimotheus%3Abranches%3AKolab%3AWinterfell to allow your fixes to go upstream again!
You can see the submit request here: https://obs.kolabsys.com/request/show/168
Update:
- To reinstall the Debian server for testing, I am using this script: reinstallKolabDebian.sh
- For compiling Debian locally with osc, first you need to install the osc and the build package:
wget http://download.opensuse.org/repositories/openSUSE:Tools/Debian_7.0/Release.key apt-key add - < Release.key echo 'deb http://download.opensuse.org/repositories/openSUSE:Tools/Debian_7.0/ /' >> /etc/apt/sources.list.d/osc.list apt-get update apt-get install osc build
Otherwise you would get this error:
Error: you need build.rpm with version 2007.3.12 or newer. See http://download.opensuse.org/repositories/openSUSE:/Tools/
The command for the local build looks like this:
osc build Debian_7.0 x86_64 roundcubemail-plugins-kolab.dsc
Nightly builds of Kolab for CentOS and Debian October 23rd, 2013
The nightly builds for Kolab have been running nicely for a while, and they are now compiled on the Kolab Systems OBS server.
Now I have added Debian packages too.
This might help other people interested in packages for Debian as an example, how to branch packages on OBS.
I have extended the documentation, and you can see a preview here:
http://download.pokorra.de/kolab/docs/developer-guide/nightly-builds/index.html
This includes documentation how to create a nightly build for CentOS or Debian, and how to install from a nightly build, for both distributions.
Unfortunately, there is still some work left to make the Debian Kolab 3.1 packages more stable. But one problem (setup-kolab not running) has been solved in this nightly build already.
Downloading from OBS repo via php proxy file October 22nd, 2013
The default port for the Open Build Service for the published repositories is 82.
If you are using the Internet through an http proxy, this can cause trouble, as it did for me when I wanted to install from the OBS run by KolabSystems, for installing Kolab from OBS: Downloading from the URL http://obs.kolabsys.com:82/Kolab:/3.1:/Updates/CentOS_6/Kolab:3.1:Updates.repo does not work.
The solution is to setup a small php script on a server somewhere on the Internet, that does not have the restrictions of port 82.
In my sample configuration, I use a Debian server with Lighttpd and PHP.
Install:
apt-get install lighttpd spawn-fcgi php5-curl php5-cgi |
changes to /etc/lighttpd/lighttpd.conf:
server.modules = (
[...]
"mod_fastcgi",
"mod_rewrite",
)
fastcgi.server = ( ".php" => ((
"bin-path" => "/usr/bin/php5-cgi",
"socket" => "/tmp/php.socket",
"max-procs" => 2,
"bin-environment" => (
"PHP_FCGI_CHILDREN" => "16",
"PHP_FCGI_MAX_REQUESTS" => "10000"
),
"bin-copy-environment" => (
"PATH", "SHELL", "USER"
),
"broken-scriptfilename" => "enable"
)))
url.rewrite-once = (
"^/obs\.kolabsys\.com/index.php" => "$0",
"^/obs\.kolabsys\.com/(.*)" => "/obs.kolabsys.com/index.php?page=$1"
) |
and in /var/www/obs.kolabsys.com/index.php:
<!--?php // it seems file_get_contents does not return the full page function curl_get_file_contents($URL) { $c = curl_init(); curl_setopt($c, CURLOPT_RETURNTRANSFER, 1); curl_setopt($c, CURLOPT_URL, str_replace('&', '&', $URL)); $contents = curl_exec($c); curl_close($c); if ($contents) return $contents; else return FALSE; } $page = $_GET['page']; $filename = basename($page); $content = curl_get_file_contents("http://obs.kolabsys.com:82/".$page); if (substr($page, -strlen("/")) === "/") { # print directory listing $content = str_replace("http://obs.kolabsys.com:82/", "http://download.pokorra.de/obs.kolabsys.com/", $content); $content = str_replace('href="/', 'href="http://download.pokorra.de/obs.kolabsys.com/', $content); echo $content; } else if (substr($filename, -strlen(".repo")) === ".repo") { header("Content-Type: plain/text"); echo str_replace("http://obs.kolabsys.com:82/", "http://download.pokorra.de/obs.kolabsys.com/", $content); } else { header("Content-Type: application/octet-stream"); header('Content-Disposition: attachment; filename="'.$filename.'"'); header("Content-Transfer-Encoding: binary\n"); echo curl_get_file_contents("http://obs.kolabsys.com:82/".$page); } ?--> |
Now it is possible to download the repo files like this:
cd /etc/yum.repos.d/ wget http://download.pokorra.de/obs.kolabsys.com/Kolab:/3.1/CentOS_6/Kolab:3.1.repo wget http://download.pokorra.de/obs.kolabsys.com/Kolab:/3.1:/Updates/CentOS_6/Kolab:3.1:Updates.repo yum install kolab |
This works for all other projects and distributions on obs.kolabsys.com too.
Feel free to use my “mirror”, but also consider to set it up for yourself, if you are using it extensively!