XLP DevOps Process

Step 1

The goals of this process is to upgrade the current docker container from Mediawiki Version 1.35 to version 1.37

Upgrade current container to Mediawiki 1.35 to latest stable version [1.37.1]

First attempt, is to update the container image to use available docker image in Mediawiki container in Docker Hub, version 1.37.1; and adjust/upgrade all the related extensions to the version that support Mediawiki 1.37.1. The major extension that is need to be adjusted is Semantic MediaWiki (SMW), and MediaWiki Language Extension Bundle (MLEB). The motivation to upgrade to Mediawiki 1.37.1 is to enable the Content Translation Feature, to enable translation process using automatic translation function, the goals can be see in Content Translation Demo page.

Platform compatibility and release status

SMW	PHP	Media Wiki	Released	Status
4.1.X	7.4.0 - 7.4.x	1.35.0 - 1.37.x	-	Development
4.0.X	7.3.0 - 7.4.x	1.35.0 - 1.37.x	2021-01-18	Stable release
3.2.x	7.1.0 - 7.4.x	1.31.0 - 1.35.x	2020-09-07	Obsolete

Based on table above, SMW Version 4.0.X and MediaWiki 1.37.X, runs on PHP 7.4.X is the version combination that we are going to use, and written into DockerFile configuration. To minimized the complexity, decided to remove all other extensions to ensure both SMW and MLEB is running correctly, thus we can localized if we found any of the extensions is having trouble.

Adjustment made on composer.local.json, as shown below

{
    "require": {
        "rht/merkle-tree": "dev-master",
        "mediawiki/maps": "9.0",
        "mediawiki/semantic-media-wiki":"~4.0",
        "mediawiki/semantic-result-formats": "~3.2",
        "mediawiki/semantic-cite": "~2.1",
        "mediawiki/sub-page-list": "~1.4",
        "mediawiki/semantic-scribunto": "~2.2"
    },
    "extra": { 
	 	"merge-plugin": { 
			"include": [ 
				"extensions/OpenIDConnect/composer.json" 
			] 
		}
	}
}

As the docker build process is runned, it shows incompatibility error between Semantic Media Wiki 4.0, Semantic Result Format 3.2, and Semantic Cite 2.1; To make the process running, decided to remove both of extension and the build process is running well. The proceed to add MLEB extension into the container.

Error creating thumbnail: Unable to save thumbnail to destination

Docker container build error

First attempt on installing the MLEB is through the git, using below command, and copy the result into the container. Not by running the code inside the docker build.

for ext in Babel cldr CleanChanges LocalisationUpdate Translate UniversalLanguageSelector
do
  if [ ! -d "$ext" ]
  then
    git clone https://gerrit.wikimedia.org/r/mediawiki/extensions/$ext.git
  fi
  cd $ext; git fetch --tags; git checkout 2022.01; cd ..
done

Then, the container is pulled to test server, and proceed with Semantic Media Wiki, Translation functionality. The test result is failed on the Preparation Step. It supposed to show top link of "Mark this page for translation", while other functionality accessed in Special Pages is shown and accessible.
To further investigate the error, debug feature is turned on and inspected. Inspection from SQL debug and debug log did not shown any error, however one of the query to revpage table did not show any result. It seems that the record that is supposed to be inserted into revpage table is failed, which causing the translation marking link cant be shown on top of the page. In attempt to fix the error, also tried to change the extension source-code from official released, instead of git hub source code found in this link. After replacing the extension source code and rebuild the container image, the Mediawiki shows similar result.

Next step to resolve the issue is to have clean docker image from official Mediawiki Docker Hub without any extension, and try to run the clean version. Once clean version is tested, each of extension MLEB and SMW is installed one-by-one and tested. Once both of the extension is running. All other extension is installed and tested. If this method is not working, then we will proceed to next paragraph.

Next step to attempt to resolve the issue is to install another source of Mediawiki installation from official released, instead of using the base container image from docker hub, the official release which found in this page. The target is to have correct functionality on host server, instead inside docker container. Once the correct functionality is found, then the installation can be dockerized while controlling the proces to ensure similar result.

Update
Docker container configuration used, after done testing, is based on Mediawiki Official Docker Images. Can be found in Docker Hub in following [links]. Based on this container configuration, SMW and MLEB extensions is installed and added. Functionality testing shows the all the feature related to SMW and MLEB is running well. Based on this particular configuration, the process is continued.

Step 2

Moving forward, is the effort to include the new docker container into installation process and run the installation into Cloud Server.

Implement the container into Devops Servers

Below are the target servers for PKC Installation

1. pkc-ops.org: Fully functional, latest stable version of Devops progress is installed in this instance
2. pkc-dev.org: Development and incremental progress is implemented in this server
3. pkc-back.org: Backup data from pkc-ops.org are stored in this instance

The complete description of installation process, in Cloud server through ansible, please see PKC Complete Installation Process

Step 3

This section is the documentation of developing Ansible Playbook to perform backup-restore process between PKC Server Instances and Backup Storage. Please refer to below diagram on how the ansible process work.

Ansible Backup and Restore Design

Develop ansible backup-restore process

The target of this ansible script is to provide complete ansible script to perform backup-restore process from ansible agent connected to two PKC Instance. For further detail, please refer to Backup and Restore Loop

Import content from pkc.pub

To import content from pkc.pub, we are going to use Backup and Restore Loop using ansible script provided in Backup and Restore Loop, with pkc.pub as source server and pkc-ops.org as destination server.