Homepage of Timotheus Pokorra

In the past days, I have upgraded my forums to the latest Beta release of Flarum, 0.1.0-beta.9.

I just want to say how glad I am with using Flarum. It does not need much resources, runs fine in shared hosting at Hostsharing, and is continually improved. Even though it is not a stable version yet, the upgrades work alright.

Here is a summary of contributions I am trying to give back to Flarum:

• my first extension: Post Notifications
• Migration from Discourse to Flarum: Blog Post
• Report of our OpenPetra forum in the show case section: Forum for OpenPetra community
• Report of our Nazareth forum in the show case section: Private Forum for Nazareth Charity
• German instructions how to install and maintain Flarum in the shared hosting environment at Hostsharing: Flarum installieren

Here are now some links and notes about developing my own extension, Post Notifications:

• Description of the extension: Post Notifications
• The source of the extension: https://github.com/tpokorra/flarum-ext-post-notification
• The versioning for Composer is done with the Git tags.
• Documentation about Flarum extensions: https://flarum.org/docs/extend

For the frontend for the admin settings, I need to transpile the code (see Frontend Development: Transpilation). I don’t do that directly on Hostsharing, because I don’t have npm there, and don’t want to use up the webspace. Therefore I have the development environment on my local Ubuntu.

cd flarum-ext-post-notification
cd js
npm install webpack
npm install
npm run build
git add dist
cd ..
rsync -zvhr --exclude node_modules --exclude .git . xyz00-myuser@mydomain.de:doms/testflarum.mydomain.de/htdocs-ssl/vendor/tpokorra/flarum-ext-post-notification

It seems for upgrading the extension across minor versions, I need to run composer require again, since composer update only updates between releases? Or perhaps I was too quick, and the new tag did not arrive at Packagist yet?

Our customer receives only spam from a certain Top Level Domain, in our case it was .icu. The customer assumes they will never receive legitimate E-Mails from this top level domain. They want all emails from this top level domain to be blacklisted.

According to the MailWatch FAQ (https://docs.mailwatch.org/using/faq.html), question “Can I use wildcards when using the Blacklist/Whitelist (SQLBlackWhiteList)?”, matching top level domains is not currently supported.

So we changed our file /usr/lib/MailScanner/MailScanner/CustomFunctions/SQLBlackWhiteList.pm

In sub LookupList, add $fromtld to the local variables, and use a regular expression to get the fromtld from the message->fromdomain.

Then we added some lines like $BlackWhite->{$to}{$fromtld}; and we are done.

#
# Based on the address it is going to, choose the right spam white/blacklist.
# Return 1 if the "from" address is white/blacklisted, 0 if not.
#
sub LookupList {
  my($message, $BlackWhite) = @_;
 
  return 0 unless $message; # Sanity check the input
 
  # Find the "from" address and the first "to" address
  my($from, $fromdomain, @todomain, $todomain, @to, $to, $ip, $fromtld);
  $from       = $message->{from};
  $fromdomain = $message->{fromdomain};
  @todomain   = @{$message->{todomain}};
  $todomain   = $todomain[0];
  @to         = @{$message->{to}};
  $to         = $to[0];
  $ip         = $message->{clientip};
  $fromtld    = $message->{fromdomain};
  ($fromtld)  = $fromtld =~ /(\.\w+)$/;
 
  # It is in the list if either the exact address is listed,
  # or the domain is listed
  return 1 if $BlackWhite->{$to}{$from};
  return 1 if $BlackWhite->{$to}{$fromdomain};
  return 1 if $BlackWhite->{$to}{$fromtld};
  return 1 if $BlackWhite->{$to}{$ip};
  return 1 if $BlackWhite->{$to}{'default'};
  return 1 if $BlackWhite->{$todomain}{$from};
  return 1 if $BlackWhite->{$todomain}{$fromdomain};
  return 1 if $BlackWhite->{$todomain}{$fromtld};
  return 1 if $BlackWhite->{$todomain}{$ip};
  return 1 if $BlackWhite->{$todomain}{'default'};
  return 1 if $BlackWhite->{'default'}{$from};
  return 1 if $BlackWhite->{'default'}{$fromdomain};
  return 1 if $BlackWhite->{'default'}{$fromtld};
  return 1 if $BlackWhite->{'default'}{$ip};
 
  # It is not in the list
  return 0;
}

The status of Mono in Fedora is sad. We cannot upgrade to the latest Mono versions because we require binary reference assemblies for the build, as described in https://bugzilla.redhat.com/show_bug.cgi?id=1436896#c91

My hope is that when .NET Core and msbuild become part of official Fedora, we can create the various packages for the binary reference assemblies.

For the moment, I create a copr for each Mono version, where we build Mono “the Fedora way”, but with Bootstrap mode always enabled.

So here is the current release Mono 5.18.0.225 for Fedora 29 and Epel 7:
https://copr.fedorainfracloud.org/coprs/tpokorra/mono-5.18/

The sources for the spec file and the patches are here:
https://github.com/tpokorra/mono-5.x-fedora/tree/master/mono-5.18

In 2014, I migrated the OpenPetra forum from phpBB to Discourse. The reason was that Discourse had a fresh look on the way how forums can be done, it had a nice design, and easy and modern user interface. Here is the post about the details: https://www.pokorra.de/2014/09/migrate-phpbb-forum-to-discourse-using-the-ubuntu-packages-from-packager-io/

Unfortunately, the requirements of Discourse for RAM is quite high, and you cannot just install it inside a shared hosting environment, because it is recommended to be run inside a docker container. It uses technology that I just don’t know very much, with Ruby on Rails and Redis etc.

Then I discovered Flarum, which is written in PHP, but looks as nice and feels as modern as Discourse. It does not have exotic requirements, it just works. It is helpful to have an SSH account, to install the dependencies with composer, but that is available at Hostsharing for my shared hosting anyway.
Flarum is still work in progress, and officially in Beta testing. But you can run your forums in production with the latest Beta already, if you know what you are doing.

So now the issue is, how to migrate your forum from Discourse to Flarum.

At TBits.net, we worked on a migration tool, based on the scripts https://github.com/robrotheram/phpbb_to_flarum,  https://github.com/viruxe/phpbb_to_flarum, and https://github.com/Reflic/phpbb_to_flarum.

We did not cover all aspects, but only those issues we needed for our OpenPetra forum.

Things we covered were:

• Format text properly, code, lists, http links, etc.
• convert categories to tags
• keep the users email addresses

We did not cover:

• migration of uploaded content
• we did not convert the user passwords (each user must use the resend password functionality of Flarum)
• and probably more…

So this blog post has the goal to make our script public, and to encourage you to use it and improve it! Pull Requests are welcome, or just fork it and use it as you like!

We started to host the script first at https://github.com/Crusader99/discourse_to_flarum, then at https://github.com/The-CJ/discourse-flarum, and now the official home for the script is:

https://github.com/TBits/discourse_to_flarum

Of course, in your real life scenario, you will have a database already. But for contributing to this script, we might need a test database, for reproducing specific situations. See my other post for details how to create a test database with dummy data for Discourse!

Setting up Flarum is quite easy if you can access your webspace with SSH and you can run Composer. See the german instructions how to install Flarum inside your Hostsharing account.

If you are working with a virtual machine and CentOS7, this Ansible script might be useful:
https://github.com/TBits/discourse_to_flarum/blob/master/ansible/flarum.yml

You can run the playbook like this (assuming your test server or container has the IP 192.168.122.52, and you can access it as root via SSH and via HTTP):

ansible-playbook flarum.yml -u root -e working_host=192.168.122.52

For running the script discourse_to_flarum.php, you also need a PostgreSQL database inside that container. This Ansible playbook will setup PostgreSQL, and will load the Discourse Backup into the PostgreSQL database. It will also install pgadmin and phpMyAdmin, which helps with debugging of the conversion routines.
You can run the playbook like this:

ansible-playbook migration.yml -u root -e working_host=192.168.122.52

After this, you can access phpMyAdmin on the url http://192.168.122.52/phpMyAdmin/, the user is flarum and the password is flarum. You can access phpPgAdmin on the url http://192.168.122.52/phpPgAdmin/, the user is discourse, and the password is discourse.
And on http://192.168.122.52/, your Flarum forum is running.

Now you can actually run the conversion script:

cd /root/flarummigration
composer update
cp migrate-example.yaml migrate.yaml
# perhaps you need to modify the database credentials in migrate.yaml if you have your own setup
# if your prefix is not fl_ for the flarum tables
sed -i "s/fl_//g" migrate.yaml
php discourse_to_flarum.php

to reset the flarum database for a rerun:

zcat ../flarum.sql.gz | mysql -u flarum flarum -p
# vi discourse_flarum.php, or vi migrate.yaml
php discourse_to_flarum.php

Here are two screenshots of the sample posts in Discourse and in Flarum after the migration (mind you, some of the BBCodes also don’t work in Discourse…)

So there is probably still quite some work to do. Pull requests are welcome!

During our project with Dynamics NAV 2018, we needed to consume a web service from CleverReach (see documentation of their REST API: https://rest.cleverreach.com/howto/), so that we can synchronize the recipients of the newsletters.

Here are now procedures that can be universally useful for using web services from within Dynamics NAV:

See the code at https://github.com/TBits/Al-examples/blob/master/HTTPRequests/cod50131.HttpRequest.al

How to do a simple GET http request:

procedure LoadWebpage(url: Text) result: Text;

How to do a simple POST http request:

procedure PostRequest(url: Text; post_names: List of [Text]; post_values: List of [Text]) result: Text;

And then how to do a POST http request, that is behind Basic Authentication (eg. done with htaccess/htpasswd):

procedure PostRequestWithBasicAuth(
        url: Text;
        post_names: List of [Text];
        post_values: List of [Text];
        username: Text;
        password: Text) result: Text;

I have prepared some web services for testing, see the code in https://github.com/TBits/Al-examples/blob/master/HTTPRequests/pag50130.HttpRequest.al for how to call them.

By the way, this example also shows how to use lists in AL, which is a feature that can hardly be found in the documentation:

var
    post_names: List of [Text];
    i: Integer;
begin
    post_names.Add('test1');
    post_names.Add('test2');
    for i := 1 to post_names.Count do
    begin
        message(post_names.Get(i));
    end;
end;

Today, we had an issue that a web service returned a timestamp in unixtime format, ie. all seconds that have passed since January 1st 1970.

We would like to convert that timestamp to a DateTime variable in AL Code.

We finally came up with these functions: UnixTimeToDateTime (see https://github.com/TBits/Al-examples/blob/master/ALBasics/cod50101.AlBasics.al#L8) and DateTimeToUnixTime (see https://github.com/TBits/Al-examples/blob/master/ALBasics/cod50101.AlBasics.al#L36).

See also the example calls at https://github.com/TBits/Al-examples/blob/master/ALBasics/pag50101.ALBasics.al

procedure TestUnixTime();
var
	myCodeUnit: Codeunit myLibrary;
	nowUnixTime: BigInteger;
	FormatString: Text;
	nowDateTime: DateTime;
begin
	nowUnixTime := myCodeUnit.DateTimeToUnixTime(CurrentDateTime);
	Message('to unixtime: ' + Format(nowUnixTime));
	nowDateTime := myCodeUnit.UnixTimeToDateTime(nowUnixTime);
	// formatting the date time to string
	FormatString := '<Day,2>.<Month,2>.<Year4> <Hours24,2>:<Minutes,2>:<Seconds,2>.<Thousands,3>';
	Message('from unixtime: ' + Format(nowDateTime, 0, FormatString));
end;

Also note how you can specify the formatting of a date with the keywords Day, Month, Year4, Hours24, Minutes.

In the previous post, Setting up Dynamics NAV 2018 for Testing and Development in Docker we have covered how to setup a development and testing environment, and had our first steps with Visual Studio Code to publish our first hello world application.

Now I want to show you some AL language thing that might be useful. You can see them in action in the example app ALBasics: https://github.com/TBits/Al-examples/tree/master/ALBasics

There are test procedures illustrating how to:

• Working with basic constructs like “if”
• Format various types of variables to string (known as Text), using Format(), or StrSubstNo().
• Parse Integer to String with Evaluate()
• Working with Date and DateTime, getting the current Now with CurrentDateTime or Today with Today. Comparing dates, even by a given difference in for example days, with CalcDate(). Converting between DateTime and Date, with DT2Time(), and setting a date with DMY2DATE().
• How to work with options: calling a procedure with an option parameter, and the issues you can get if you have different definitions of the options.
• How to call procedures with reference parameters (keyword var), and how to return a result either by a named variable or with exit(<value>)
• How to call a procedure that is defined inside a code unit.

In the past months, I have learned a lot about Dynamics NAV 2018. There are some examples available on the net, but not enough. So I will provide some examples on my blog.

Let’s start with setting up the free Docker container on Windows of Dynamics NAV for testing and development purposes.

My first obstacle was the requirements to the CPU.

First, you need a CPU with SLAT for running Hyper-V on Windows 10. See also https://social.technet.microsoft.com/wiki/contents/articles/1401.hyper-v-list-of-slat-capable-cpus-for-hosts.aspx which states: “Every Core i3, i5 and i7 supports SLAT. You can look up the supported AMD versions at KB article on AMD“.
So I bought a refurbished HP ProBook 6560b with a Intel Core i5-2520M CPU, but that did not work. It does not support docker images with more than 2 GB of RAM. But you need 4 GB to run Dynamics NAV 2018 in a container. See this bug for more details: https://github.com/docker/for-win/issues/1094.
With the coreinfo tool (download here for free: https://docs.microsoft.com/en-us/sysinternals/downloads/coreinfo), you can test if your CPU is supported:

Coreinfo.exe -f

If you see a star, then you should be fine.

PAGE1GB * Supports 1 GB large pages

If you see a hyphen, then you cannot use that computer:

PAGE1GB - Supports 1 GB large pages

You can install Docker for Windows from https://www.docker.com/docker-windows. You need to register in the Docker Hub for free. You install it, and it will ask you to restart the computer, the latest version enables Hyper-V, and it asks if you want to use Linux or Windows containers. Let the installer prepare your computer for Windows containers.

When you start Docker, and right click on the symbol, you see Kitematic. That is a useful tool to manage your containers graphically. Click on it, and it will download a zip file. Extract the contents to C:\Program Files\Docker\Kitematic.

The docker images from Microsoft are listed here: https://hub.docker.com/r/microsoft/dynamics-nav/

As administrator you open a PowerShell, and run these commands:

# this should pull the latest dynamics-nav, 2018 at the moment
docker pull microsoft/dynamics-nav
# now start a container
docker run -e accept_eula=Y -e ClickOnce=Y -e UseSSL=N -m 4G microsoft/dynamics-nav

I chose the setting ClickOnce=Y because I wanted to test not only the web client, but also the Windows Client.

I chose the UseSSL=N setting because I had issues with Visual Studio Code to download the symbols or to upload my extension if it Dynamics NAV was installed with a self-signed certificate. I got the message:

Die zugrunde liegende Verbindung wurde geschlossen: Für den geschützten SSL/TLS-Kanal konnte keine Vertrauensstellung hergestellt werden..
Das Remotezertifikat ist laut Validierungsverfahren ungültig.

You probably could download the certificate from http://YOURCONTAINERIP:8080/certificate.cer and install it into your “Trusted Root Certification Authorities” but I did not want to do that just for testing purposes.

This is the output of the docker run command:

PS C:\WINDOWS\system32> docker run -e accept_eula=Y -e ClickOnce=Y -e UseSSL=N -m 4G microsoft/dynamics-nav
Initializing...
Starting Container
Hostname is c63405985172
PublicDnsName is c63405985172
Using NavUserPassword Authentication
Starting Local SQL Server
Starting Internet Information Server
Creating Self Signed Certificate
Self Signed Certificate Thumbprint 5D0E6813F13AA90A59328F196F9594CCEF3F1DF9
Modifying Service Tier Config File with Instance Specific Settings
Starting NAV Service Tier
Creating DotNetCore Web Server Instance
Creating http download site
Creating Windows user admin
Setting SA Password and enabling SA
Creating admin as SQL User and add to sysadmin
Creating SUPER user
Creating ClickOnce Manifest
Container IP Address: 172.17.43.185
Container Hostname  : c63405985172
Container Dns Name  : c63405985172
Web Client          : http://c63405985172/NAV/
NAV Admin Username  : admin
NAV Admin Password  : Qake5494
Dev. Server         : http://c63405985172
Dev. ServerInstance : NAV
ClickOnce Manifest  : http://c63405985172:8080/NAV

Files:
http://c63405985172:8080/al-0.12.25327.vsix

Initialization took 114 seconds
Ready for connections!

Now how to test your Dynamics NAV instance:

• • Web Client: you see the URL in the output above. It will be different in your installation…
  • Windows Client: it does not work in Firefox, to download the ClickOnce manifest from the URL mentioned above, installing it gives the error: System.Deployment.Application.InvalidDeploymentException (Zone)
    – Die Bereitstellung und die Anwendung haben keine übereinstimmenden Sicherheitszonen.
    But it works in Edge or Internet Explorer just fine. You use the NAV Admin Username and Password as specified in the docker output.
  • With Visual Studio Code: Install the AL Language extension from http://c63405985172:8080/, the file is called like al-0.12.25327.vsix. In Visual Studio Code, go to Extension, select the hamburger menu at the top, and choose “Install from VSIX file”. Installing the AL Language extension from the Marketplace would install a newer version of the extension, but it probably causes issues with your server because it is not compatible.

First steps with Visual Studio Code: Create a sample application: Ctrl-Shift-P, search for AL:Go!, and run it. This will ask you where you want the project stored, and then create a hello world example.

You need to modify the file launch.json, and enter for server the URL which was in the docker output labelled as “Dev. Server”, in my case “http://c63405985172”

Then you can press Ctrl-Shift-P, search for AL: Download Symbols, and run that.

After the symbols have been downloaded, run AL: Publish, which should first ask your for your username and password (use admin and the password from the docker output), then it will upload your extension to the NAV server, and VS Code will start the Web client with the page Contacts, because that page is the Object ID 22, which is defined in startupObjectId in launch.json. You will see the message box.

I have submitted this basic extension on Github: https://github.com/TBits/Al-examples/tree/master/HelloWorld

In the next post in this series, we will discover the AL language in more detail: Getting used to the AL language for Dynamics NAV 2018

In the past weeks, I have been working on a deployment of Dynamics NAV 2018. This is quite a change for me, to work again on a Windows machine, and with closed source software. Fortunately, there are already some resources on the web, eg. Powershell scripts by Waldo: https://github.com/waldo1001/Cloud.Ready.Software.PowerShell/tree/master/PSScripts

Today, I want to publish some basic Batch scripts that help our team to start our development environment. In our setup, we have a normal unprivileged Windows user. But we need a privileged user for access to the Dynamics NAV Server. Lets assume that the privileged user has a suffix called -p for privileged.

startDynamicsNAVClient.bat

This script will just start the Dynamics NAV 2018 client, running as the privileged user.

@echo off
runas /user:"MYDOMAIN\%USERNAME%-p" "C:\Program Files (x86)\Microsoft Dynamics NAV\110\RoleTailored Client\Microsoft.Dynamics.Nav.Client.exe -settings:\\files\ERP\RTCCONFIG\DEV110_NST01_V103.config"

startDynamicsNAVIDE.bat

This script will start the Dynamics NAV IDE, which we use mainly for the Object Designer. We need the parameter generatesymbolreference, so that the symbols will be generated when we compile the C/AL code, so that the symbols are available for AL development in VS Code.

It seems we have some issue with localization, some users need Ja (german culture), others need Yes as the value for generatesymbolreference.

@echo off
REM runs the Dynamics NAV IDE (Object Designer)
REM some users have to use generatesymbolreference=Ja others need generatesymbolreference=Yes
REM we try to determine the language settings by the variable HomePath: Users vs Benutzer
set "generatesymbolreference=Ja"
if "%ProgramFiles%" equ "C:\Program Files" (SET /A generatesymbolreference=Yes)
runas /user:"MYDOMAIN\%USERNAME%-p" "C:\Program Files (x86)\Microsoft Dynamics NAV\110\RoleTailored Client\finsql.exe generatesymbolreference=%generatesymbolreference%"

startDynamicsNAVShell.bat

This script starts a PowerShell with the right startup parameters, so that some specific modules will be preloaded for development of Dynamics NAV. This seems to be quite tricky, to start PowerShell with another user. So we call the Batch script again as the different user, in order to be able to start PowerShell as that user.

@echo off
REM start the DynamicsNAV Development PowerShell
SET mypath=%~dp0
echo %mypath%
IF "%USERNAME:~-2%" neq "-p" (
  runas /user:"MYDOMAIN\%USERNAME%-p" "%mypath%\startDynamicsNAVShell.bat"
) else (
  cd C:\Users\%USERNAME%\DEV
  C:\Windows\system32\WindowsPowerShell\v1.0\PowerShell.exe  -NoExit -ExecutionPolicy RemoteSigned " & ' C:\Program Files (x86)\Microsoft Dynamics NAV\110\RoleTailored Client\NavModelTools.ps1 ' "
)

startVSCode.bat

Last, but not least: we start VSCode as a different user.

@echo off
start runas /user:"MYDOMAIN\%USERNAME%-p" "C:\Program Files\Microsoft VS Code\Code.exe"

Here comes a quick overview on recent updates to Kolab 16 packages.

Please note: I am only using public information. I have no background knowledge about the releases.

In the past days, the following packages have been updated:

• kolab-freebusy: https://obs.kolabsys.com/request/show/2183, Repack of tagged version
• guam: https://obs.kolabsys.com/package/rdiff/Kolab:16/guam?linkrev=base&rev=20, Allow empty lines in commands