In 2014, I migrated the OpenPetra forum from phpBB to Discourse. The reason was that Discourse had a fresh look on the way how forums can be done, it had a nice design, and easy and modern user interface. Here is the post about the details: https://www.pokorra.de/2014/09/migrate-phpbb-forum-to-discourse-using-the-ubuntu-packages-from-packager-io/

Unfortunately, the requirements of Discourse for RAM is quite high, and you cannot just install it inside a shared hosting environment, because it is recommended to be run inside a docker container. It uses technology that I just don’t know very much, with Ruby on Rails and Redis etc.

Then I discovered Flarum, which is written in PHP, but looks as nice and feels as modern as Discourse. It does not have exotic requirements, it just works. It is helpful to have an SSH account, to install the dependencies with composer, but that is available at Hostsharing for my shared hosting anyway.
Flarum is still work in progress, and officially in Beta testing. But you can run your forums in production with the latest Beta already, if you know what you are doing.

So now the issue is, how to migrate your forum from Discourse to Flarum.

At TBits.net, we worked on a migration tool, based on the scripts https://github.com/robrotheram/phpbb_to_flarumhttps://github.com/viruxe/phpbb_to_flarum, and https://github.com/Reflic/phpbb_to_flarum.

We did not cover all aspects, but only those issues we needed for our OpenPetra forum.

Things we covered were:

  • Format text properly, code, lists, http links, etc.
  • convert categories to tags
  • keep the users email addresses

We did not cover:

  • migration of uploaded content
  • we did not convert the user passwords (each user must use the resend password functionality of Flarum)
  • and probably more…

So this blog post has the goal to make our script public, and to encourage you to use it and improve it! Pull Requests are welcome, or just fork it and use it as you like!

We started to host the script first at https://github.com/Crusader99/discourse_to_flarum, then at https://github.com/The-CJ/discourse-flarum, and now the official home for the script is:

https://github.com/TBits/discourse_to_flarum

Of course, in your real life scenario, you will have a database already. But for contributing to this script, we might need a test database, for reproducing specific situations. See my other post for details how to create a test database with dummy data for Discourse!

Setting up Flarum is quite easy if you can access your webspace with SSH and you can run Composer. See the german instructions how to install Flarum inside your Hostsharing account.

If you are working with a virtual machine and CentOS7, this Ansible script might be useful:
https://github.com/TBits/discourse_to_flarum/blob/master/ansible/flarum.yml

You can run the playbook like this (assuming your test server or container has the IP 192.168.122.52, and you can access it as root via SSH and via HTTP):

ansible-playbook flarum.yml -u root -e working_host=192.168.122.52

For running the script discourse_to_flarum.php, you also need a PostgreSQL database inside that container. This Ansible playbook will setup PostgreSQL, and will load the Discourse Backup into the PostgreSQL database. It will also install pgadmin and phpMyAdmin, which helps with debugging of the conversion routines.
You can run the playbook like this:

ansible-playbook migration.yml -u root -e working_host=192.168.122.52

After this, you can access phpMyAdmin on the url http://192.168.122.52/phpMyAdmin/, the user is flarum and the password is flarum. You can access phpPgAdmin on the url http://192.168.122.52/phpPgAdmin/, the user is discourse, and the password is discourse.
And on http://192.168.122.52/, your Flarum forum is running.

Now you can actually run the conversion script:

cd /root/flarummigration
composer update
cp migrate-example.yaml migrate.yaml
# perhaps you need to modify the database credentials in migrate.yaml if you have your own setup
# if your prefix is not fl_ for the flarum tables
sed -i "s/fl_//g" migrate.yaml
php discourse_to_flarum.php

to reset the flarum database for a rerun:

zcat ../flarum.sql.gz | mysql -u flarum flarum -p
# vi discourse_flarum.php, or vi migrate.yaml
php discourse_to_flarum.php

Here are two screenshots of the sample posts in Discourse and in Flarum after the migration (mind you, some of the BBCodes also don’t work in Discourse…)

       

             

 

So there is probably still quite some work to do. Pull requests are welcome!

Tags: , , , , ,
Posted in Hosting, Software Development | Comments Closed

So it seems that http://www.discourse.org/ is the new way that forums should work.

Discourse uses Ruby on Rails, Redis and Postgresql on the server side. This means it is quite different than the LAMP stack that you might be used to.

For that reason, the recommended type of installation is via Docker to hide the complexity from the administrator: https://github.com/discourse/discourse/blob/master/docs/INSTALL-digital-ocean.md

I have tried that, but I must say that docker is intended for different kind of applications, if I understand correctly: for deployment of many applications in the cloud, and the data is stored not inside the Docker container but somewhere else.

I personally think that LXC containers are much more suitable for this kind of task. One aspect is that the container feels like a virtual machine, and I can control the IP address of the container using my lxc scripts: https://github.com/tpokorra/lxc-scripts

But installing all the requirements of Discourse manually is not an option either. Then I found the Debian and Ubuntu packages provided by packager.io: https://packager.io/gh/pkgr/discourse

So I followed the instructions at https://packager.io/gh/pkgr/discourse/install?bid=41#trusty to install Discourse on my Ubuntu 14.04 trusty container.

I use nginx in the host machine to manage SSL for me. I only need to set in the file /opt/discourse/config/site_settings.yml:

use_https: true

My nginx file looks like this: https://github.com/tpokorra/lxc-scripts/blob/master/nginx.sslconf.tpl

Now we get to the point where I want to migrate the existing phpBB forum to Discourse:

See this thread about the phpBB importer: https://meta.discourse.org/t/importing-phpbb-into-discourse/7956/27

I need to install mysql inside the container, and import the sql data:

apt-get install mysql-server mysql-client libmysqlclient-dev make
/etc/init.d/mysql start
mysql
  create database phpbb; 
  exit;
mysql phpbb < backup-db-phpbb.sql

Before I actually import the phpBB data, I disable sending of all emails in Discourse, in Settings/EMail, disable sending of all emails.

Now I need to load the environment variables, so that I use the correct version of Ruby and gems etc.
I have got those commands from the script /usr/bin/discourse:

. /etc/default/discourse
for file in /etc/discourse/conf.d/*; do   if [ -f $file ]; then . $file; fi; done
for file in /opt/discourse/.profile.d/*.sh; do   if [ -f $file ]; then HOME=/opt/discourse . $file; fi; done
 
# install mysql2 gem
gem install mysql2
 
#now the import call:
cd /opt/discourse/script/import_scripts
# PATH is not actually passed? therefore specifying path for ruby
RAILS_ENV=production sudo -E -u discourse /opt/discourse/bin/ruby phpbb3.rb

Update November 11th 2014:
I learnt now that you can use this command that will run the upgrade script in the right environment (Thank you Cyril Rohr from packager.io!):

discourse run gem install mysql2
discourse run ruby ./script/import_scripts/phpbb3.rb

To deactivate all users that never have posted to the forum, you can do this in Postgresql:

su - postgres
 psql discourse 
   update users set active=false where last_posted_at is null;
   \q
 exit

Now you can enable sending of emails again.

You could actually create a backup of Discourse, reinstall the container, to get rid of the Mysql database, and import the backup.

Update November 11th 2014:
To upgrade the installation to a newer version of Discourse, you need to run as root:

apt-get update && apt-get upgrade
discourse run rake db:migrate
Tags: ,
Posted in Software Development | Comments Closed

So, now Sourceforge has ended their Hosted Apps for good: http://sourceforge.net/p/forge/community-docs/Hosted%20Apps%20Retirement/

Here is the migration path, how I got the OpenPetra wiki and forum to install on my own webspace.

My task was to install them readonly, as we want to do the proper migration later, perhaps to other software, who knows.

I got the backup through the shell access, but it seems now that all backups have been placed in the project’s home directory.

MediaWiki:

I have installed the recent version 1.23.0.

  1. restore the database from the sql backup file
  2. extract the file mediawiki-1.23.0.tar.gz
  3. extract the files from the file backup file, into images.
    mv home/.../path/images/* images
  4. run the installer of MediaWiki in the webbrowser. It will automatically upgrade the database fine.
  5. modify LocalSettings.php, so that noone can register and even registered users cannot edit:
    $wgGroupPermissions['*']['createaccount'] = false;
    $wgGroupPermissions['*']['edit'] = false;
    $wgGroupPermissions['user']['edit'] = false;

 phpBB:

I have installed the recent stable version phpBB-3.0.12.

The version that Sourceforge has used was 3.0.5, see in the database, table phpbb_config, column version.

  1. restore the database from sql backup file (prefix is phpbb_)
  2. extract the phpBB source code from the zip file
  3. extract the files from the file backup file, into files and images
  4. run the installer of phpBB. It does not work to install into existing tables, therefore use prefix phpbb3_
  5. Then I modified a couple of settings in the phpbb_config table:
    UPDATE phpbb_config SET `config_value` = '1' WHERE `phpbb_config`.`config_name` = 'default_style';
    UPDATE phpbb_config SET `config_value` = 'forum.openpetra.org' WHERE `phpbb_config`.`config_name` = 'server_name';
    UPDATE phpbb_config SET `config_value` = 'files' WHERE `phpbb_config`.`config_name` = 'upload_path';
    UPDATE phpbb_config SET `config_value` = 'forum.openpetra.org' WHERE `phpbb_config`.`config_name` = 'cookie_domain';
    UPDATE phpbb_config SET `config_value` = '/' WHERE `phpbb_config`.`config_name` = 'cookie_path';
    UPDATE phpbb_config SET `config_value` = 'db' WHERE `phpbb_config`.`config_name` = 'auth_method';
    UPDATE phpbb_config SET `config_value` = '/' WHERE `phpbb_config`.`config_name` = 'script_path';
    UPDATE phpbb_config SET `config_value` = 'http://' WHERE `phpbb_config`.`config_name` = 'server_protocol';
    UPDATE phpbb_config SET `config_value` = '80' WHERE `phpbb_config`.`config_name` = 'server_port';
  6. I use the style from the original database, replacing the sourceforge style:
    ALTER TABLE phpbb_styles RENAME phpbb35_styles;
    ALTER TABLE phpbb_styles_imageset RENAME phpbb35_styles_imageset;
    ALTER TABLE phpbb_styles_imageset_data RENAME phpbb35_styles_imageset_data;
    ALTER TABLE phpbb_styles_template_data RENAME phpbb35_styles_template_data;
    ALTER TABLE phpbb_styles_template RENAME phpbb35_styles_template;
    ALTER TABLE phpbb_styles_theme RENAME phpbb35_styles_theme;ALTER TABLE phpbb3_styles RENAME phpbb_styles;
    ALTER TABLE phpbb3_styles_imageset RENAME phpbb_styles_imageset;
    ALTER TABLE phpbb3_styles_imageset_data RENAME phpbb_styles_imageset_data;
    ALTER TABLE phpbb3_styles_template_data RENAME phpbb_styles_template_data;
    ALTER TABLE phpbb3_styles_template RENAME phpbb_styles_template;
    ALTER TABLE phpbb3_styles_theme RENAME phpbb_styles_theme;
  7. Then I removed all the tables that are not needed anymore:
    SELECT CONCAT( 'DROP TABLE ', GROUP_CONCAT(TABLE_NAME) , ';' )
    AS statement FROM information_schema.tables
    WHERE table_schema = 'tim00_op_sf_readonly' AND TABLE_NAME LIKE 'phpbb3%';

    Copy the result and paste the drop command into your phpMyAdmin or mysql console client…
    You might have to run that twice, it might only return the first tables in the first call.

  8. Then change in the config.php the table prefix back to “phpbb_”
  9. Then run these commands, to reset the cache after the config changes, and to activate the forum:
    rm -Rf install
    rm cache/*.php

 

Tags: , ,
Posted in Software Development | Comments Closed

Digest for phpBB Forum without plugin   November 13th, 2010

I am very glad for the hosted apps that Sourceforge provides. It is excellent to have only one login, and not to need to worry about updating the apps.

The problem with the phpBB forum is that the notification emails, when you watch a forum, do not include the message. On Sourceforge you are also not able to install a digest modification, eg. http://phpbbservices.com/digests/
Therefore I thought it would be good to have an external script, that regularly checks for new posts and sends an email with the contents.
In the end, it was not too hard.
This only works for a public forum, where all posts are accessible by non-registered users.
There is the page search.php (eg.http://sourceforge.net/apps/phpbb/openpetraorg/search.php?&sr=topics&search_id=newposts) that shows all new posts.
I store each post that has been sent as an email with the post number in a text file.
Then the script fetches the page of the new posts, and checks each post if it has been sent already.
The script sends HTML emails, and even quoting blocks appear quite alright in my email client (google mail).
The script is also able to parse the posts of a thread that spreads across several pages.
You also might find the functions interesting to get the HTML code and to parse for code between the beginning and the end tag.
Feel free to use the script and to modify it.