In 2014, I migrated the OpenPetra forum from phpBB to Discourse. The reason was that Discourse had a fresh look on the way how forums can be done, it had a nice design, and easy and modern user interface. Here is the post about the details: https://www.pokorra.de/2014/09/migrate-phpbb-forum-to-discourse-using-the-ubuntu-packages-from-packager-io/

Unfortunately, the requirements of Discourse for RAM is quite high, and you cannot just install it inside a shared hosting environment, because it is recommended to be run inside a docker container. It uses technology that I just don’t know very much, with Ruby on Rails and Redis etc.

Then I discovered Flarum, which is written in PHP, but looks as nice and feels as modern as Discourse. It does not have exotic requirements, it just works. It is helpful to have an SSH account, to install the dependencies with composer, but that is available at Hostsharing for my shared hosting anyway.
Flarum is still work in progress, and officially in Beta testing. But you can run your forums in production with the latest Beta already, if you know what you are doing.

So now the issue is, how to migrate your forum from Discourse to Flarum.

At TBits.net, we worked on a migration tool, based on the scripts https://github.com/robrotheram/phpbb_to_flarumhttps://github.com/viruxe/phpbb_to_flarum, and https://github.com/Reflic/phpbb_to_flarum.

We did not cover all aspects, but only those issues we needed for our OpenPetra forum.

Things we covered were:

  • Format text properly, code, lists, http links, etc.
  • convert categories to tags
  • keep the users email addresses

We did not cover:

  • migration of uploaded content
  • we did not convert the user passwords (each user must use the resend password functionality of Flarum)
  • and probably more…

So this blog post has the goal to make our script public, and to encourage you to use it and improve it! Pull Requests are welcome, or just fork it and use it as you like!

We started to host the script first at https://github.com/Crusader99/discourse_to_flarum, then at https://github.com/The-CJ/discourse-flarum, and now the official home for the script is:

https://github.com/TBits/discourse_to_flarum

Of course, in your real life scenario, you will have a database already. But for contributing to this script, we might need a test database, for reproducing specific situations. See my other post for details how to create a test database with dummy data for Discourse!

Setting up Flarum is quite easy if you can access your webspace with SSH and you can run Composer. See the german instructions how to install Flarum inside your Hostsharing account.

If you are working with a virtual machine and CentOS7, this Ansible script might be useful:
https://github.com/TBits/discourse_to_flarum/blob/master/ansible/flarum.yml

You can run the playbook like this (assuming your test server or container has the IP 192.168.122.52, and you can access it as root via SSH and via HTTP):

ansible-playbook flarum.yml -u root -e working_host=192.168.122.52

For running the script discourse_to_flarum.php, you also need a PostgreSQL database inside that container. This Ansible playbook will setup PostgreSQL, and will load the Discourse Backup into the PostgreSQL database. It will also install pgadmin and phpMyAdmin, which helps with debugging of the conversion routines.
You can run the playbook like this:

ansible-playbook migration.yml -u root -e working_host=192.168.122.52

After this, you can access phpMyAdmin on the url http://192.168.122.52/phpMyAdmin/, the user is flarum and the password is flarum. You can access phpPgAdmin on the url http://192.168.122.52/phpPgAdmin/, the user is discourse, and the password is discourse.
And on http://192.168.122.52/, your Flarum forum is running.

Now you can actually run the conversion script:

cd /root/flarummigration
composer update
cp migrate-example.yaml migrate.yaml
# perhaps you need to modify the database credentials in migrate.yaml if you have your own setup
# if your prefix is not fl_ for the flarum tables
sed -i "s/fl_//g" migrate.yaml
php discourse_to_flarum.php

to reset the flarum database for a rerun:

zcat ../flarum.sql.gz | mysql -u flarum flarum -p
# vi discourse_flarum.php, or vi migrate.yaml
php discourse_to_flarum.php

Here are two screenshots of the sample posts in Discourse and in Flarum after the migration (mind you, some of the BBCodes also don’t work in Discourse…)

       

             

 

So there is probably still quite some work to do. Pull requests are welcome!

Tags: , , , , ,
Posted in Hosting, Software Development | Comments Closed

Seit Juli 2017 unterstützt Hostsharing die automatische Verwaltung von Let’s Encrypt Zertifikaten für einzelne Domains und auch Subdomains.

In diesem Post soll es um die Wildcard Zertifikate gehen, die Let’s Encrypt seit März 2018 ausstellt. In meinem Fall brauche ich ein Wildcard Zertifikat für das Massenhosting von OpenPetra. Die Domain wird bei Hostsharing verwaltet, aber die Server laufen bei TBits.net, also muss ich die Zertifikate nicht bei Hostsharing einrichten. Wildcard Zertifikate werden nicht über HTTP authentifiziert, sondern über DNS. Daher muss ich also bei Hostsharing die Einträge im DNS vornehmen.

Ich habe nach einem leichtgewichtigen ACME Client gesucht, der Wildcard Domains unterstützt. Dabei bin ich auf acme.sh gestoßen, das als Shell Script (bash, dash, sh kompatibel) geschrieben ist, und auf Github gehostet ist.

Installiert wird es mit diesem Befehl:

curl https://get.acme.sh | sh

Es wird dadurch ein Verzeichnis ~/.acme.sh angelegt und mit entsprechenden Dateien gefüllt, weniger als ein MB.

Hier ist eine Anleitung für manuelle Konfiguration über DNS: https://github.com/Neilpang/acme.sh/wiki/dns-manual-mode

Der erste Aufruf sieht so aus, für die Domain openpetra.com:

~/.acme.sh/acme.sh  --issue  -d '*.openpetra.com'  --dns --yes-I-know-dns-manual-mode-enough-go-ahead-please
Die Ausgabe sieht dann in etwa so aus:
[Tue Sep 25 23:08:42 CEST 2018] Creating domain key
[Tue Sep 25 23:08:42 CEST 2018] The domain key is here: /home/pacs/tim00/users/.../.acme.sh/*.openpetra.com/*.openpetra.com.key
[Tue Sep 25 23:08:42 CEST 2018] Single domain='*.openpetra.com'
[Tue Sep 25 23:08:42 CEST 2018] Getting domain auth token for each domain
[Tue Sep 25 23:08:43 CEST 2018] Getting webroot for domain='*.openpetra.com'
[Tue Sep 25 23:08:43 CEST 2018] Add the following TXT record:
[Tue Sep 25 23:08:43 CEST 2018] Domain: '_acme-challenge.openpetra.com'
[Tue Sep 25 23:08:43 CEST 2018] TXT value: 'nrsGiN7Yb4R8eYxx7RNRAP8x6MA2zTrgbCLemD1t7aB'
[Tue Sep 25 23:08:43 CEST 2018] Please be aware that you prepend _acme-challenge. before your domain
[Tue Sep 25 23:08:43 CEST 2018] so the resulting subdomain will be: _acme-challenge.openpetra.com
[Tue Sep 25 23:08:43 CEST 2018] Please add the TXT records to the domains, and re-run with --renew.
[Tue Sep 25 23:08:43 CEST 2018] Please add '--debug' or '--log' to check more details.
[Tue Sep 25 23:08:43 CEST 2018] See: https://github.com/Neilpang/acme.sh/wiki/How-to-debug-acme.sh

Nun muss ich diesen TXT record für die Domain openpetra.com einrichten.

Ich muss zuerst eine Datei im Verzeichnis ~/doms/openpetra.com/etc einrichten, mit dem Namen pri.openpetra.com. Der Inhalt sieht in etwa so aus:

{DEFAULT_ZONEFILE}
$TTL 60M
 
_acme-challenge.openpetra.com.  3600  IN  TXT  "nrsGiN7Yb4R8eYxx7RNRAP8x6MA2zTrgbCLemD1t7aB" 
 
www       IN  A {DOM_IPNUMBER}
op012345  IN  A 178.123.12.123

Nach etwa 3 Minuten sieht man in der Datei /var/log/named/named.log, dass die eigene Domain aktualisiert wurde.

Dann kann der zweite Schritt mit acme.sh ausgeführt werden:

~/.acme.sh/acme.sh  --issue  -d '*.openpetra.com'  --dns --yes-I-know-dns-manual-mode-enough-go-ahead-please --renew

Die Ausgabe sieht wie folgt aus:

[Tue Sep 25 23:09:54 CEST 2018] Renew: '*.openpetra.com'
[Tue Sep 25 23:09:55 CEST 2018] Single domain='*.openpetra.com'
[Tue Sep 25 23:09:55 CEST 2018] Getting domain auth token for each domain
[Tue Sep 25 23:09:55 CEST 2018] Verifying:*.openpetra.com
[Tue Sep 25 23:09:58 CEST 2018] Success
[Tue Sep 25 23:09:58 CEST 2018] Verify finished, start to sign.
[Tue Sep 25 23:10:00 CEST 2018] Cert success.
-----BEGIN CERTIFICATE-----
...........entfernt.................
-----END CERTIFICATE-----
[Tue Sep 25 23:10:00 CEST 2018] Your cert is in  /home/pacs/tim00/users/..../.acme.sh/*.openpetra.com/*.openpetra.com.cer 
[Tue Sep 25 23:10:00 CEST 2018] Your cert key is in  /home/pacs/tim00/users/..../.acme.sh/*.openpetra.com/*.openpetra.com.key 
[Tue Sep 25 23:10:00 CEST 2018] The intermediate CA cert is in  /home/pacs/tim00/users/..../.acme.sh/*.openpetra.com/ca.cer 
[Tue Sep 25 23:10:00 CEST 2018] And the full chain certs is there:  /home/pacs/tim00/users/..../.acme.sh/*.openpetra.com/fullchain.cer 
[Tue Sep 25 23:10:01 CEST 2018] It seems that you are using dns manual mode. please take care: The dns manual mode can not renew automatically, you must issue it again manually. You'd better use the other modes instead.
[Tue Sep 25 23:10:01 CEST 2018] Call hook error.

Ich kopiere nun die Dateien *.openpetra.com.key nach wildcard.openpetra.com.key und fullchain.cer nach wildcard.openpetra.com.crt auf den Produktionsserver, und binde sie im Nginx ein:

    ssl_certificate /var/lib/certs/wildcard.openpetra.com.crt;  
    ssl_certificate_key /var/lib/certs/wildcard.openpetra.com.key;

Da das Zertifikat nur für 3 Monate gültig ist, habe ich eine automatische Erinnerung
in meinem Kalender eingerichtet, die mich alle 2 Monate daran erinnert, das Wildcard Zertifikat zu erneuern.

Tags: , , , ,
Posted in Hosting | Comments Closed

Ansible with su instead of sudo   October 13th, 2017

At TBits.net, we have decided to use Ansible for setting up our servers.

The most documented way of installing something on a machine via Ansible is using sudo: you create a normal user (eg. called deploy), that you can use with SSH to login to the machine, and if that user has sudo permissions (eg. part of the group wheel in CentOS), then you can install software with root privileges.

The call is quite easy:

ansible-playbook myplaybook.yaml --user=deploy --ask-become-pass

Now we wanted to limit access only to users who have the actual password for root.

Finally, this worked on the command line:

ansible-playbook myplaybook.yaml --user=deploy --become --become-method=su --ask-become-pass

Now, I wanted to specify these parameters in my ansible.cfg file. It took me a while to find out how to do this. I found https://github.com/ansible/ansible/blob/devel/lib/ansible/config/base.yml which was helpful.

[defaults]
remote_user=deploy
 
[privilege_escalation]
become = true
become_method = su
become_ask_pass = true

Two pitfalls that are solved by this:

  • You need to specify the become settings in section privilege_escalation, not just in defaults.
  • The command line parameter ask-become-pass becomes become_ask_pass in the config file.

This works with Ansible 2.3.2 on CentOS 7.4.

Tags: ,
Posted in Hosting, Software Development | Comments Closed

There has been a new release of Roundcube: https://roundcube.net/news/2015/09/14/updates-1.1.3-and-1.0.7-released/

I noticed that because Epel already has version 1.1.3, but Kolab 3.4 Updates still has 1.1.2. Now there is an installation conflict, because yum wants to use the epel version, but that leads to other conflicts.

A temporary solution is to exclude all roundcubemail* packages in the epel repo file:

sed -i "s#enabled=1#enabled=1\nexclude=roundcubemail*#g" /etc/yum.repos.d/epel.repo

The proper solution is to upgrade the roundcubemail package for Kolab 3.4 Updates on OBS.

I was slightly confused which tarball to use, and Daniel Hoffend aka dhoffend helped me out:

  1. Go to https://github.com/roundcube/roundcubemail/releases
  2. Get the commit id for release 1.1.3: 357cd5103d1c27f8416ef316c4a4c31588db45b8
  3. git clone https://github.com/roundcube/roundcubemail
    cd roundcubemail
    git checkout -b newrelease 357cd5103d1c27f8416ef316c4a4c31588db45b8
    git archive --prefix=roundcubemail-1.1.3/ HEAD | gzip -c > ../roundcubemail-1.1.3.tar.gz

To test the new package, download this repo file:

yum install yum-utils
yum-config-manager --add-repo https://obs.kolabsys.com/repositories/home:/tpokorra:/branches:/Kolab:/3.4:/Updates/CentOS_7/home:tpokorra:branches:Kolab:3.4:Updates.repo
yum update

The new updated package will hopefully arrive in Kolab 3.4 Updates within the next days.

Tags: , ,
Posted in Software Development | Comments Closed

Recently at TBits.net, we wanted to even better secure the authentication to a web application that we provide to our customers.

There are several options to extend authentication:

We decided for partial passwords: You have your username and your first password, you login with that, and if that worked, you are asked about some specific letters from your second password: Please give us the third and 5th character of your second password!

See this paper called “Give Me Letters 2, 3 and 6!”: Partial Password Implementations & Attacks that describes Partial Password implementations and attacks in a detailed study.

The next question is, how to store the information about the partial password: you cannot hash it, because you will have to verify single characters. The solution is to use a secret sharing scheme, as described in Partial Passwords – How. It uses the Shamir’s Secret Sharing algorithm.

I have now implemented this idea, processing partial passwords with Shamir’s secret sharing scheme, in PHP. You can find the code licensed under the MIT at: https://github.com/TBits/partialPasswordShamirsSecret. There is an example included.

The Wikipedia article on Lagrange polynomial has been helpful as well in properly implementing the algorithm in PHP.

Tags: , ,
Posted in Software Development | Comments Closed