Kolab 16 for Fedora 25   December 31st, 2016

This is work in progress, but I just wanted to share the news: I have Kolab 16 packages for Fedora 25 (with PHP7), built on copr!

Read the rest of this entry »

Tags: , ,
Posted in Software Development | Comments Closed

There is a new release of the LightBuildServer, available at https://github.com/SolidCharity/LightBuildServer/releases

This has now support for copr!

Copr is a build service for Fedora and Epel packages: See for example my copr Kolab_16 that I have built with LBS: https://copr.fedorainfracloud.org/coprs/tpokorra/Kolab_16/

Copr is good for building single packages, but if you paste multiple packages, it does not care about build dependencies between the packages. This is where the LightBuildServer comes in, and orders copr to build the packages in the right order, and waits for packages to finish building if they are required by the next package.

The advantage of this is that you can build packages in parallel (defined by maxinstances, be nice!), and the copr build machines are quite fast! And the repository is served by copr, so less traffic for your server! And people can even more trust a copr repo, because you cannot do any magic and modify builds etc.

So how to use it? You need to get an API token from https://copr.fedorainfracloud.org/api/, and paste that into a file /etc/lightbuildserver/container/<yourusername>/<yourprojectname>/copr.

You also need to define a static build machine with type “copr”, like this in your /etc/lightbuildserver/config.yml:

       copr.fedorainfracloud.org:
           type: copr
           enabled: True
           maxinstances: 4
           static: True

Then you can refer that machine from your project, also in the file config.yml:

      :
          Machine: copr.fedorainfracloud.org
          CoprUserName: ""
          CoprProjectName: ""
Tags: , ,
Posted in Software Development | Comments Closed

This is an update to the post http://www.pokorra.de/2014/12/packaging-nsis-nullsoft-installer-for-centos-rpm/

NSIS (Nullsoft Scriptable Install System) “is a professional open source system to create Windows installers.”

NSIS 3.0 has been released on 25th July 2016: http://nsis.sourceforge.net/News

I have now packages for CentOS7 and Fedora 24: https://lbs.solidcharity.com/package/tpokorra/nsis/nsis. You can find instructions on that page how to install the rpms.

The sources for the package are available here: https://github.com/tpokorra/lbs-nsis/tree/master/nsis

I am using this for example to build nightly installers for OpenPetra standalone for Windows, and the build happens on CentOS7: https://lbs.solidcharity.com/package/tpokorra/openpetra/openpetranow-standalone-nightly. The sources for the installer can be found here: standalone.nsi

 

Tags: , , ,
Posted in Software Development | Comments Closed

After quite a lot of refactoring, the latest LightBuildServer release 0.2.2 is now available, quite cleanly packaged for Fedora 22.

The most important improvements are:

  • runs now with uwsgi server and nginx
  • uses sqlite to cope with persistent states, instead of using global variables
  • a cronjob triggers the processing of the build queue

For the OS that hosts the build containers I currently recommend CentOS7, with LXC 1.0.x

Here is a short tutorial how to setup a server that runs the LightBuildServer on Jiffybox. This should work on similar offerings like from Rackspace or DigitalOcean.

I have created a Jiffybox with CentOS 7. Make sure in the settings of the Jiffybox to change the kernel to pvgrub64 because that will come with the latest features from the CentOS7 default kernel. Otherwise creating LXC containers might not work, because the default Jiffybox kernel does not support SquashFS.

On the CentOS7 machine, I will now install the LXC scripts. These are useful scripts for creating LXC containers, supporting various guest Operating Systems like CentOS, Fedora, Ubuntu and Debian. For more details, see https://github.com/tpokorra/lxc-scripts/blob/master/Readme.md

yum install yum-utils epel-release
yum-config-manager --add-repo https://lbs.solidcharity.com/repos/tpokorra/lbs/centos/7/lbs-tpokorra-lbs.repo
yum install lxc-scripts
# setup the bridge for networking with the LXC containers
systemctl enable libvirtd
systemctl start libvirtd
# create a symbolic link in the root directory, so that you get quicker to the scripts
ln -s /usr/share/lxc-scripts scripts
cd scripts
./initIPTables.sh
./initLXC.sh
 
# we need nginx as proxy to redirect requests to the container
yum install nginx
systemctl enable nginx
systemctl start nginx
 
# make sure the firewall allows requests on port 80 (http) or 443 (https)
iptables -A IN_public_allow -p tcp -m tcp --dport 80 -m conntrack --ctstate NEW -j ACCEPT
iptables -A IN_public_allow -p tcp -m tcp --dport 443 -m conntrack --ctstate NEW -j ACCEPT
iptables-save > /etc/sysconfig/iptables

The next step is to create a Fedora 22 container, which will run the LightBuildServer control server and Web UI:

cd ~/scripts
# 50: this is container id, and will be used to generate the IP address of the container as well
./initFedora.sh 50-lbs.example.org 50
 
# configure the nginx proxy for the lbs.example.org website
# if /var/lib/certs/lbs.example.org.crt and lbs.example.org.key exist,
# it will be configured for https, otherwise just for http
./initWebproxy.sh 50 lbs.example.org
 
# start the container
lxc-start -d -n 50-lbs.example.org
# see the IP address
./listcontainers.sh
# and ssh into the container, using the password for the key you generated earlier when running initLXC.sh
ssh root@192.168.122.50

Now you can install the LightBuildServer inside the Fedora 22 container:

dnf install 'dnf-command(config-manager)'
dnf config-manager --add-repo https://lbs.solidcharity.com/repos/tpokorra/lbs/fedora/22/lbs-tpokorra-lbs.repo
dnf install lightbuildserver
# initialize the server
# this will enable and start the services nginx, uwsgi and crond
/usr/share/lightbuildserver/init.sh

The configuration of the LightBuildServer happens in the file /etc/lightbuildserver/config.yml. You can configure an SMTP account for the notification emails to be sent to you.
You should also define the LBSUrl and the DownloadUrl (probably the same) for your server.
You can also define your own Github or Gitlab account, both public and private. See https://github.com/SolidCharity/LightBuildServer/wiki/Config-Files#lbs_config_file for examples.
You can define your own projects and packages as well.

At last, you need to define the host for building your packages. We can define the CentOS7 host here. So replace build01.localhost with build01.lbs.example.org.
You need to add a line to the /etc/hosts file on the LBS container,

# on the LBS container.
# use the IP that is the gateway for the container to the host
echo "192.168.122.1  build01.lbs.example.org" >> /etc/hosts
# we changed config.yml and need to restart the LBS website:
systemctl restart uwsgi

You also need to copy the public key to the host, so that the LBS container can create build machines on the host. For production use, the LBS server should obviously not have root access to the host system. You should add another host for building.

# on the CentOS7 host.
# make sure there is a new line
echo >> /root/.ssh/authorized_keys 
cat /var/lib/lxc/50-lbs.example.org/rootfs/etc/lightbuildserver/container/container_rsa.pub >> /root/.ssh/authorized_keys

Now test inside from the LBS container if you have access to the host, and accept the host key:

# on the LBS container:
ssh -i /etc/lightbuildserver/container/container_rsa root@build01.lbs.example.org

Now you should be able to login on the webinterface, with user demo and password demo. Try building a Debian or Fedora package, or a CentOS or an Ubuntu package!

Tags: , , ,
Posted in Software Development | Comments Closed

First I want to say Thanks to Xamarin for providing packages now for Linux again. I am personally using the Mono packages from Xamarin for CentOS7, and the MonoDevelop package for Ubuntu from Xamarin.

I have decided I will not build my own MonoDevelop packages anymore, since the Xamarin packages are sufficient.
There are also some issues that I could not figure out, so at the moment my MonoDevelop packages on OBS do not build at all. I have no intention to fix them, because there is no need for them.

For example, to install the latest MonoDevelop on Ubuntu, you can follow these instructions (see also http://www.monodevelop.com/download/linux/#debian-ubuntu-and-derivatives):

apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
echo "deb http://download.mono-project.com/repo/debian wheezy main" | tee /etc/apt/sources.list.d/mono-xamarin.list
apt-get update
apt-get install monodevelop monodoc-browser

I also wanted to stop building the Mono packages for various Linux distributions, but there are issues for CentOS6 with libgdiplus and libpng15 with the Xamarin packages (http://lists.ximian.com/pipermail/mono-list/2014-October/051241.html), so I will keep providing Mono packages at the moment.

The latest Mono packages for 3.12.1 are available according to the instructions at http://software.opensuse.org/download.html?project=home%3Atpokorra%3Amono&package=mono-opt

I have added packages for Fedora 21 there recently.

I have also worked on getting Mono 3.12.1 built for CentOS5. This required to use a newer GCC than is available on CentOS5. This meant I cannot build it on OBS since it is not allowed to link in different repositories. Obviously I could try to build a newer GCC on OBS as well, but then it becomes more and more complicated, and others have done this work already (http://people.centos.org/tru/devtools-2/readme)

The solution now works with my own LightBuildServer, and the result can be seen here: https://lbs.solidcharity.com/package/tpokorra/mono/mono-opt
If you want to build Mono on your own LBS, let me know, and I will improve the installation instructions for the LBS. All the code is public already, at https://github.com/SolidCharity/LightBuildServer/
The sources for the RPM packages are maintained here: https://github.com/tpokorra/lbs-mono/tree/master/mono-opt

If you need some paid assistance with Mono packages for your distribution, please contact me via ODesk: https://www.odesk.com/o/profiles/users/_~01e826605fea50faae/

Tags: , ,
Posted in Software Development | Comments Closed

Previously, my nightly builds of Kolab have been built and hosted here:

https://obs.kolabsys.com/project/show/home:tpokorra:branches:Kolab:Development

There were 2 problems recently:

  • I was not able to add builds for CentOS7. The error message is:
    Failed to add project or repository: unable to walk on path 'CentOS:7/standard'
  • Each time there was a new patch or release on Kolab:Development, I needed to manually remove the patch from the nightly package because the patch was already part of git master. I also needed to resolve conflicts in the debian.changelog and package dsc file because the version number changed.

UPDATE: since there was an easy fix for Point 1 (just edit the Meta file of the project to add different Operating System), I also realised I could update the OBS packaging instructions the same way as I do for LBS, avoiding conflicts. So we will see if that works over the coming weeks…

After some improvements to my LightBuildServer, I am now able to build the nightly packages on my own LBS server.

This is done in two steps:

  1. Updating the package instructions: This is done by an LBS job, that basically is running this script: The package instructions are downloaded from https://obs.kolabsys.com/project/show/Kolab:Development, and the source tarballs are downloaded from https://git.kolab.org/. Then the changelog is updated, patches removed, some other patches applied… Also some conversion is done on the files to make Debian happy to build them. The result is uploaded to https://github.com/TBits/lbs-kolab-nightly
  2. Then I run different jobs on LBS, for each Operating System that I currently support: CentOS6, CentOS7, and Debian, to rebuild the selected packages with nightly tarballs.
    The result can be viewed here: https://lbs.solidcharity.com/project/tbits.net/kolab-nightly

These steps are executed each night, by a cronjob that initiates the builds on the LBS, by calling for example

tbitscredentials=tbits.net/secret
wget -O /dev/null https://lbs.solidcharity.com/triggerbuildproject/tbits.net/kolab-nightly/centos/6/amd64/$tbitscredentials

To test the nightly builds, you can install the nightly repository like this, additionally to the Kolab 3.4 and Kolab Development repo:

For CentOS6:

yum install yum-utils
yum-config-manager --add-repo https://download.solidcharity.com/repos/tbits.net/kolab-nightly/centos/6/lbs-tbits.net-kolab-nightly.repo

For CentOS7:

yum install yum-utils
yum-config-manager --add-repo https://download.solidcharity.com/repos/tbits.net/kolab-nightly/centos/7/lbs-tbits.net-kolab-nightly.repo

For Debian Wheezy:

apt-get install apt-transport-https
echo 'deb https://download.solidcharity.com/repos/tbits.net/kolab-nightly/debian/wheezy/ /' &gt;&gt; /etc/apt/sources.list
apt-get update

With the LightBuildServer I have full control over the builds, can just modify a config file to add a new target OS (if my LXC scripts support it), and can review the history of the package instructions on Github.

Of course, one disadvantage of LBS compared to OBS is: the LightBuildServer is not intended to directly support the work in a team. Team-work happens via Github (or your self hosted Gitlab), and every team member could install his own LightBuildServer.

A feature that LBS is still missing is that multiple build containers are just assigned with any job, without checking if one job should wait for another job to finish. A first step would be to distribute the jobs per Operating System to different build containers. Well, still lots of room for improvement…

Tags: , ,
Posted in Software Development | Comments Closed

TBits.net is glad to announce that nightly tests now run every night against the nightly development packages of Kolab, for both CentOS 6 and Debian Wheezy.

At the moment, the nightly tests also run for the Kolab 3.3 Updates for CentOS 6.

Through these tests, we can spot problems and obvious bugs in the code early, when they are still easy to fix. This is TBits.net’s contribution to making Kolab more stable and reliable.

As soon as Kolab 3.4 is out (expected for the middle of February 2015), we will enable nightly tests for CentOS 6 and Debian Wheezy for Kolab 3.4.

You can see the tests and the results here: https://lbs.solidcharity.com/package/tbits.net/kolab-test/kolab-test

I use the TBits KolabScripts to install the server unattended.

The TBits KolabScripts also contain some Selenium tests written in Python. These tests check if the user can change his own password, and tests creating new domains and new users, sending emails, catchall addresses, etc.

Also the TBits scripts for ISPs are tested: These scripts add domain admins functionality to give customers the option to manage their own domains, limited by a domain quota and maximum accounts number.

We use the LightBuildServer for running the tests. This software is written in Python, by Timotheus Pokorra. It is very light-weight, does not come with much code, and is easy to extend and configure. It uses LXC to create virtual machines for each test or build run. You can see it live in action for Kolab

We now also initiate the nightly development packages with the LightBuildServer: https://lbs.solidcharity.com/package/tbits.net/kolab-nightly-sync/updatecode. This job is run every night, fetches the source from git.kolab.org, modifies the package version number, and uploads the new package instructions to obs.kolabsys.com so that they can be built on the OBS server sponsored by Kolab Systems.

If you want to test the nightly packages yourself, please read the chapter in the Kolab Developer Guide: http://docs.kolab.org/developer-guide/nightly-builds/install.html

Tags: ,
Posted in Software Development | Comments Closed

For OpenPetra, I wanted to build the installer for the Windows Remote Client on CentOS.

I know that NSIS (Nullsoft Scriptable Install System) is available for Debian, but I could not find a recent RPM package of NSIS for CentOS.

So I built my own RPM package for the latest stable version of NSIS, version 2.46, using the LightBuildServer. You can see the project here: http://lbs.solidcharity.com/package/tpokorra/nsis/nsis

The sources for the package are hosted at Github: https://github.com/tpokorra/lbs-nsis/tree/master/nsis.

You can install the RPM on CentOS 6 like this:

cd /etc/yum.repos.d/
wget http://download.lbs.solidcharity.com/repos/tpokorra/nsis/centos/6/lbs-tpokorra-nsis.repo
yum install nsis

I use this NSIS RPM package to build OpenPetra in this LBS job: openpetra-server-test, and the sources are available at Github: openpetra-server-test sources.

By the way, the installer source for the OpenPetra remote client can be found here: remote.nsi

Please let me know if you want me to build NSIS for other CentOS versions too.

The source rpm is available here: http://download.lbs.solidcharity.com/repos/tpokorra/nsis/centos/6/src/

Tags: , , ,
Posted in Software Development | Comments Closed

I have just finished the work on the latest MonoDevelop packages.

Quite a bit has changed in the build process, so it needed some work to make the build from tarball finish successfully again.
See this thread on the mailinglist: http://lists.ximian.com/pipermail/monodevelop-list/2014-May/016077.html

I have updated the packages at
https://build.opensuse.org/project/monitor/home:tpokorra:mono

MonoDevelop 4.2.5:
http://software.opensuse.org/download.html?project=home%3Atpokorra%3Amono&package=monodevelop-opt

MonoDevelop 4.3.4:
http://software.opensuse.org/download.html?project=home%3Atpokorra%3Amono&package=monodevelop-beta

MonoDevelop 5.1 (nightly build from this morning):
http://software.opensuse.org/download.html?project=home%3Atpokorra%3Amono&package=monodevelop-alpha

By the way, I am working on a new project called LightBuildServer, which is a very light-weight alternative to OBS.
see http://lightbuildserver.org

Tags: , , ,
Posted in Software Development | Comments Closed