There is a new release of the LightBuildServer, available at https://github.com/SolidCharity/LightBuildServer/releases

This has now support for copr!

Copr is a build service for Fedora and Epel packages: See for example my copr Kolab_16 that I have built with LBS: https://copr.fedorainfracloud.org/coprs/tpokorra/Kolab_16/

Copr is good for building single packages, but if you paste multiple packages, it does not care about build dependencies between the packages. This is where the LightBuildServer comes in, and orders copr to build the packages in the right order, and waits for packages to finish building if they are required by the next package.

The advantage of this is that you can build packages in parallel (defined by maxinstances, be nice!), and the copr build machines are quite fast! And the repository is served by copr, so less traffic for your server! And people can even more trust a copr repo, because you cannot do any magic and modify builds etc.

So how to use it? You need to get an API token from https://copr.fedorainfracloud.org/api/, and paste that into a file /etc/lightbuildserver/container/<yourusername>/<yourprojectname>/copr.

You also need to define a static build machine with type “copr”, like this in your /etc/lightbuildserver/config.yml:

       copr.fedorainfracloud.org:
           type: copr
           enabled: True
           maxinstances: 4
           static: True

Then you can refer that machine from your project, also in the file config.yml:

      :
          Machine: copr.fedorainfracloud.org
          CoprUserName: ""
          CoprProjectName: ""
Tags: , ,
Posted in Software Development | Comments Closed

I was able to modify the existing Fedora spec file for owncloud-client, to build a package with the nextcloud theming:

Tags: , , ,
Posted in Software Development | Comments Closed

This is an update to the post http://www.pokorra.de/2014/12/packaging-nsis-nullsoft-installer-for-centos-rpm/

NSIS (Nullsoft Scriptable Install System) “is a professional open source system to create Windows installers.”

NSIS 3.0 has been released on 25th July 2016: http://nsis.sourceforge.net/News

I have now packages for CentOS7 and Fedora 24: https://lbs.solidcharity.com/package/tpokorra/nsis/nsis. You can find instructions on that page how to install the rpms.

The sources for the package are available here: https://github.com/tpokorra/lbs-nsis/tree/master/nsis

I am using this for example to build nightly installers for OpenPetra standalone for Windows, and the build happens on CentOS7: https://lbs.solidcharity.com/package/tpokorra/openpetra/openpetranow-standalone-nightly. The sources for the installer can be found here: standalone.nsi

 

Tags: , , ,
Posted in Software Development | Comments Closed

This post is about the latest Mono release, and its availability for older Linux distributions.

Thankfully, Xamarin provides Mono packages for the most current Linux distributions at http://www.mono-project.com/download/#download-lin. The Xamarin packages have not been updated yet, due to migration to a new Jenkins server.

I was not able yet to package MonoDevelop 6:

But here are the Mono 4.4.0.182 packages for CentOS and Fedora:

Here is the list of Mono 4.4.0.182 installed to /opt, for various Linux distributions:

For a summary of packages that I maintain for Mono, see also https://github.com/tpokorra/lbs-mono/wiki

Tags: , , ,
Posted in Hosting, Software Development | Comments Closed

After quite a lot of refactoring, the latest LightBuildServer release 0.2.2 is now available, quite cleanly packaged for Fedora 22.

The most important improvements are:

  • runs now with uwsgi server and nginx
  • uses sqlite to cope with persistent states, instead of using global variables
  • a cronjob triggers the processing of the build queue

For the OS that hosts the build containers I currently recommend CentOS7, with LXC 1.0.x

Here is a short tutorial how to setup a server that runs the LightBuildServer on Jiffybox. This should work on similar offerings like from Rackspace or DigitalOcean.

I have created a Jiffybox with CentOS 7. Make sure in the settings of the Jiffybox to change the kernel to pvgrub64 because that will come with the latest features from the CentOS7 default kernel. Otherwise creating LXC containers might not work, because the default Jiffybox kernel does not support SquashFS.

On the CentOS7 machine, I will now install the LXC scripts. These are useful scripts for creating LXC containers, supporting various guest Operating Systems like CentOS, Fedora, Ubuntu and Debian. For more details, see https://github.com/tpokorra/lxc-scripts/blob/master/Readme.md

yum install yum-utils epel-release
yum-config-manager --add-repo https://lbs.solidcharity.com/repos/tpokorra/lbs/centos/7/lbs-tpokorra-lbs.repo
yum install lxc-scripts
# setup the bridge for networking with the LXC containers
systemctl enable libvirtd
systemctl start libvirtd
# create a symbolic link in the root directory, so that you get quicker to the scripts
ln -s /usr/share/lxc-scripts scripts
cd scripts
./initIPTables.sh
./initLXC.sh
 
# we need nginx as proxy to redirect requests to the container
yum install nginx
systemctl enable nginx
systemctl start nginx
 
# make sure the firewall allows requests on port 80 (http) or 443 (https)
iptables -A IN_public_allow -p tcp -m tcp --dport 80 -m conntrack --ctstate NEW -j ACCEPT
iptables -A IN_public_allow -p tcp -m tcp --dport 443 -m conntrack --ctstate NEW -j ACCEPT
iptables-save > /etc/sysconfig/iptables

The next step is to create a Fedora 22 container, which will run the LightBuildServer control server and Web UI:

cd ~/scripts
# 50: this is container id, and will be used to generate the IP address of the container as well
./initFedora.sh 50-lbs.example.org 50
 
# configure the nginx proxy for the lbs.example.org website
# if /var/lib/certs/lbs.example.org.crt and lbs.example.org.key exist,
# it will be configured for https, otherwise just for http
./initWebproxy.sh 50 lbs.example.org
 
# start the container
lxc-start -d -n 50-lbs.example.org
# see the IP address
./listcontainers.sh
# and ssh into the container, using the password for the key you generated earlier when running initLXC.sh
ssh root@192.168.122.50

Now you can install the LightBuildServer inside the Fedora 22 container:

dnf install 'dnf-command(config-manager)'
dnf config-manager --add-repo https://lbs.solidcharity.com/repos/tpokorra/lbs/fedora/22/lbs-tpokorra-lbs.repo
dnf install lightbuildserver
# initialize the server
# this will enable and start the services nginx, uwsgi and crond
/usr/share/lightbuildserver/init.sh

The configuration of the LightBuildServer happens in the file /etc/lightbuildserver/config.yml. You can configure an SMTP account for the notification emails to be sent to you.
You should also define the LBSUrl and the DownloadUrl (probably the same) for your server.
You can also define your own Github or Gitlab account, both public and private. See https://github.com/SolidCharity/LightBuildServer/wiki/Config-Files#lbs_config_file for examples.
You can define your own projects and packages as well.

At last, you need to define the host for building your packages. We can define the CentOS7 host here. So replace build01.localhost with build01.lbs.example.org.
You need to add a line to the /etc/hosts file on the LBS container,

# on the LBS container.
# use the IP that is the gateway for the container to the host
echo "192.168.122.1  build01.lbs.example.org" >> /etc/hosts
# we changed config.yml and need to restart the LBS website:
systemctl restart uwsgi

You also need to copy the public key to the host, so that the LBS container can create build machines on the host. For production use, the LBS server should obviously not have root access to the host system. You should add another host for building.

# on the CentOS7 host.
# make sure there is a new line
echo >> /root/.ssh/authorized_keys 
cat /var/lib/lxc/50-lbs.example.org/rootfs/etc/lightbuildserver/container/container_rsa.pub >> /root/.ssh/authorized_keys

Now test inside from the LBS container if you have access to the host, and accept the host key:

# on the LBS container:
ssh -i /etc/lightbuildserver/container/container_rsa root@build01.lbs.example.org

Now you should be able to login on the webinterface, with user demo and password demo. Try building a Debian or Fedora package, or a CentOS or an Ubuntu package!

Tags: , , ,
Posted in Software Development | Comments Closed

This post originates in the idea from Stephen Gallagher, who is working on rolekit: “rolekit is a daemon for Linux systems providing a stable D-BUS interface to manage the deployment of [Fedora] Server Roles”.
The code of Rolekit is available here: https://github.com/sgallagher/rolekit

On his blog, Stephen stated in this post:

A few that I’d love to see (but don’t have time to start on yet):

  • A fileserver role that manages Samba and NFS file-shares (maybe [s]ftp as well).
  • A mail and/or groupware server role built atop something like Kolab
  • A backup server

This made me wonder, how would that be, if Kolab became a Server Role for Fedora, and could be installed from the Fedora repositories? Through my work on OpenPetra and Mono I got involved with Fedora, and noticed that the Fedora community tries out new technology, proves if it works, and then the technology will eventually end up in other distributions as well.

First steps

On IRC, we agreed that the first step would be to create a Copr repo, that contains the Kolab packages, and to write this blog post describing how to install and configure Kolab.

Creating the Copr Repo

So, here is the Copr repository for Fedora 22: https://copr.fedoraproject.org/coprs/tpokorra/kolab/

I created it by getting the src rpm packages from the Kolab repository, from 3.4 and 3.4 updates, in this order:

  • kolab-utils
  • roundcubemail-plugins-kolab
  • kolab-webadmin
  • kolab
  • pykolab
  • chwala
  • iRony
  • kolab-freebusy
  • roundcubemail-skin-chameleon
  • php-Net-LDAP3
  • roundcubemail
  • kolab-syncroton
  • roundcubemail-plugin-contextmenu
  • kolab-schema
  • kolab-autodiscover
  • python-sievelib
  • php-pear-Net-LDAP2
  • cyrus-imapd

The packages libkolab and libkolabxml and kdepim are already in Fedora, and I did not update them:

Cyrus Imapd is also in Fedora, https://admin.fedoraproject.org/pkgdb/package/cyrus-imapd/, but not on the latest version. So I used version 2.5 from Kolab.

Roundcubemail is uptodate in Fedora, https://admin.fedoraproject.org/pkgdb/package/roundcubemail, but somehow does not provide roundcubemail(core) >= 1.1 as required by some Kolab packages. So I also used the package from Kolab.

I have patched the pykolab package, and backported some features to extend the setup-kolab command so that it can be used non-interactively, which is probably required to be integrated into rolekit. In Kolab 3.5 (release planned for August 2015), those features will be included.

Installing Kolab from the Copr Repo

I have tested this with Fedora 22.

Please disable SELinux, since there isn’t a SELinux policy available yet for Kolab.
Jeroen van Meeuwen has worked on it a while ago, but it probably needs updating and testing: https://github.com/kanarip/kolab-selinux

Another thing: the server should have a FQDN, eg. kolab.example.org. See the installation instructions for details.

dnf install dnf-plugins-core
dnf copr enable tpokorra/kolab
dnf install kolab
mytz=Europe/Brussels
pwd=test
setup-kolab --default --mysqlserver=new --timezone=$mytz --directory-manager-pwd=$pwd

On my setup, I need to add this line to /etc/kolab/kolab.conf, in section [kolab-wap], because I am running it inside an LXC container with an iptables tunnel for port 80, and the Kolab webadmin does not calculate the url for the API properly:

api_url = http://localhost/kolab-webadmin/api

You also need to add these lines to /etc/roundcubemail/config.inc.php (this will be fixed in Kolab 3.5):

    # required for php 5.6, see https://bbs.archlinux.org/viewtopic.php?id=193012 and http://php.net/manual/de/context.ssl.php
    # production environment requires real security settings!!!
    $config['imap_conn_options']=array(
            'ssl'=&gt;array(
            'verify_peer_name'=&gt;false,
            'verify_peer'=&gt;false,
            'allow_self_signed'=&gt;true));
    $config['smtp_conn_options']=array(
            'ssl'=&gt;array(
            'verify_peer_name'=&gt;false,
            'verify_peer'=&gt;false,
            'allow_self_signed'=&gt;true));

After this, the Kolab Server should run, and you can go to http://localhost/kolab-webadmin and login with the user “cn=Directory Manager” (without the quotes) and the password that you specified as parameter for setup-kolab.

The webmail runs at http://localhost/roundcubemail

Conclusion

I hope this shows the possibilities, and what amount of work still needs to be done.

I guess the existing packages in Fedora should be kept uptodate, and missing Kolab packages need to be added to Fedora as well.

Work on SELinux policy is also required (see above).

The other thing: with the server role Kolab, how much should the role define how the server is configured securely? In Kolab Upstream, we documented how to secure the server, but left it to the Sysadmin to actually enforce security, because the Kolab community cannot take responsibility for the server.

I have a number of scripts, that might be useful for rolekit: https://github.com/TBits/KolabScripts There is eg. a script for setting up a self-signed SSL Certificate, etc.

Tags: , ,
Posted in Software Development | Comments Closed

There was recently a new Mono version made available: Cycle 5 Service Release 1 (4.0.1.44)

You can install it from OBS: have a look at the instructions here: https://software.opensuse.org/download.html?project=home%3Atpokorra%3Amono&package=mono-opt

Please note, that for CentOS 6 and CentOS7, Fedora 20 and Fedora 21, OpenSUSE 13.1 and OpenSUSE 13.2 these packages are available:

yum install mono-opt mono-opt-devel monodevelop-opt mod_mono-opt mono-basic-opt mono-nant-opt mono-tools-opt mono-uncrustify-opt mono-xsp-opt nunit-opt nuget-opt

For Debian 7 and Debian 8, and various Ubuntu releases, these packages are available:

apt-get install mono-opt monodevelop-opt modmono-opt mono-basic-opt mono-nant-opt mono-tools-opt mono-uncrustify-opt mono-xsp-opt nunit-opt nuget-opt

Yes, monodevelop-opt is back! I had to build nuget-opt and nunit-opt, and now it works again.

Mono 3.12.x:

I have created a new repository for those that still need mono-opt for version 3.12.x:
Have a look at the instructions here:
https://software.opensuse.org/download.html?project=home%3Atpokorra%3Amono-3.12&package=mono-opt

Tags: ,
Posted in Software Development | Comments Closed

Previously, my nightly builds of Kolab have been built and hosted here:

https://obs.kolabsys.com/project/show/home:tpokorra:branches:Kolab:Development

There were 2 problems recently:

  • I was not able to add builds for CentOS7. The error message is:
    Failed to add project or repository: unable to walk on path 'CentOS:7/standard'
  • Each time there was a new patch or release on Kolab:Development, I needed to manually remove the patch from the nightly package because the patch was already part of git master. I also needed to resolve conflicts in the debian.changelog and package dsc file because the version number changed.

UPDATE: since there was an easy fix for Point 1 (just edit the Meta file of the project to add different Operating System), I also realised I could update the OBS packaging instructions the same way as I do for LBS, avoiding conflicts. So we will see if that works over the coming weeks…

After some improvements to my LightBuildServer, I am now able to build the nightly packages on my own LBS server.

This is done in two steps:

  1. Updating the package instructions: This is done by an LBS job, that basically is running this script: The package instructions are downloaded from https://obs.kolabsys.com/project/show/Kolab:Development, and the source tarballs are downloaded from https://git.kolab.org/. Then the changelog is updated, patches removed, some other patches applied… Also some conversion is done on the files to make Debian happy to build them. The result is uploaded to https://github.com/TBits/lbs-kolab-nightly
  2. Then I run different jobs on LBS, for each Operating System that I currently support: CentOS6, CentOS7, and Debian, to rebuild the selected packages with nightly tarballs.
    The result can be viewed here: https://lbs.solidcharity.com/project/tbits.net/kolab-nightly

These steps are executed each night, by a cronjob that initiates the builds on the LBS, by calling for example

tbitscredentials=tbits.net/secret
wget -O /dev/null https://lbs.solidcharity.com/triggerbuildproject/tbits.net/kolab-nightly/centos/6/amd64/$tbitscredentials

To test the nightly builds, you can install the nightly repository like this, additionally to the Kolab 3.4 and Kolab Development repo:

For CentOS6:

yum install yum-utils
yum-config-manager --add-repo https://download.solidcharity.com/repos/tbits.net/kolab-nightly/centos/6/lbs-tbits.net-kolab-nightly.repo

For CentOS7:

yum install yum-utils
yum-config-manager --add-repo https://download.solidcharity.com/repos/tbits.net/kolab-nightly/centos/7/lbs-tbits.net-kolab-nightly.repo

For Debian Wheezy:

apt-get install apt-transport-https
echo 'deb https://download.solidcharity.com/repos/tbits.net/kolab-nightly/debian/wheezy/ /' &gt;&gt; /etc/apt/sources.list
apt-get update

With the LightBuildServer I have full control over the builds, can just modify a config file to add a new target OS (if my LXC scripts support it), and can review the history of the package instructions on Github.

Of course, one disadvantage of LBS compared to OBS is: the LightBuildServer is not intended to directly support the work in a team. Team-work happens via Github (or your self hosted Gitlab), and every team member could install his own LightBuildServer.

A feature that LBS is still missing is that multiple build containers are just assigned with any job, without checking if one job should wait for another job to finish. A first step would be to distribute the jobs per Operating System to different build containers. Well, still lots of room for improvement…

Tags: , ,
Posted in Software Development | Comments Closed

Using Flockport with Jiffybox   December 18th, 2014

I am interested in the idea of Flockport: providing ready built LXC containers for download. So I wanted to try to see how I can actually download a Flockport container and install it on a Jiffybox (the German equivalent to the American Linode…)

See also my old post: LXC Linux containers on JiffyBox running CentOS on Ubuntu

So, I install a Jiffybox with Ubuntu 14.04 64 bit operating system, 2 GB RAM.

Install lxc (currently 1.0.6) from the default Ubuntu repositories:

apt-get install lxc

Now to Flockport: Check out these pages, my next steps are based on the instructions there:

I chose the WordPress container as an example. See http://www.flockport.com/apps/wordpress/

I wanted to download the Debian Wheezy 64 Bit package called wordpress.tar.xz, but that does not work like that, because you need to be logged in.

Here comes the Flockport utility into play. It is currently in Alpha, but it works ok, even on Ubuntu, though it is only advertised for Debian:

To install the Flock utility, follow these steps:

wget http://repo.flockport.com/flockport.gpg.key
apt-key add flockport.gpg.key
echo "deb http://repo.flockport.com/debian wheezy main" > /etc/apt/sources.list.d/flockport.list
apt-get update
apt-get install flockport

Note that I am not installing lxc from the Flockport repository, but only the Flockport utility.

Some useful commands with the Flockport utility:

#shows all Flockport containers available
flockport list 
 
# login with your username and password for http://www.flockport.com/
flockport login
 
# download a container, the names were displayed by the list command above
flockport get wordpress

The Flockport utility will download the container, and extract it to /var/lib/lxc.

# shows the new container
lxc-ls -f
 
# start the container
lxc-start -d -n wordpress 
 
# now show the running container, and the currently used IP address:
lxc-ls -f 
 
#output:
#NAME      STATE   IPV4        IPV6  AUTOSTART
#-----------------------------------------------
#wordpress RUNNING 10.0.3.227  -     NO

To make this container accessible to the outside, you can use iptables:

containerIP=`lxc-ls -f -F name,ipv4 | grep wordpress | awk '{ print $2 }'`
interface=`cat /etc/network/interfaces | grep "auto" | grep -v "auto lo" | awk '{ print $2 }'`
HostIP=`ifconfig ${interface} | grep "inet addr" | awk '{ print $2 }' | awk -F ':' '{ print $2 }'`
iptables -t nat -A PREROUTING -d ${HostIP}/32 -i ${interface} -p tcp -m tcp --dport 80 -j DNAT --to-destination ${containerIP}:80
echo "make sure that mywordpress.org resolves to this HostIP: " ${HostIP}

If mywordpress.org resolves to the IP of your Jiffybox, then you can visit the WordPress installation by browsing to http://mywordpress.org

To change the domain name from mywordpress.org to the actual domain name that you want to use, you have to first go into http://mywordpress.org/wp-login.php, login with username admin and password flockport, change the password, and change in General Settings the WordPress URL and the Site URL to your desired domain name, eg. helloworld.org

You also have to change the Nginx configuration inside the container to replace the domain name with your actual domain name for your website:

# switch inside the container
lxc-attach -n wordpress
 
# unfortunately no vi available, but nano will do as well:
nano /etc/nginx/sites-available/mywordpress.org
 
# add helloworld.org or your domain name to the 4th line:
#    server_name mywordpress.org www.mywordpress.org helloworld.org;
# leave nano with Ctrl-X, and don't forget to save...
 
# reload nginx for the change of configuration to take effect
service nginx reload

Now you can reach the server on your own domain name, that points to your Jiffybox!

Tags: , ,
Posted in Software Development | Comments Closed