Difference between revisions of "Backup and Restore"

From PKC
Jump to navigation Jump to search
Line 13: Line 13:
* 1 Offsite storage location
* 1 Offsite storage location
There are other practices, which can be found here<ref>Mark Cambell, Why 3-2-1 Backup Sucks, https://www.unitrends.com/blog/3-2-1-backup-sucks, last accessed: August 1st 2021</ref>.
There are other practices, which can be found here<ref>Mark Cambell, Why 3-2-1 Backup Sucks, https://www.unitrends.com/blog/3-2-1-backup-sucks, last accessed: August 1st 2021</ref>.
==Importance==
To enable proper data security, convenient backup and restore procedure would be the first layer.
This MediaWiki instance already stores all data under [[mountPoint/images]] and [[mountPoint/mariadb]], these two directories store
all the data relevant to MediaWiki, including its uploaded media files, and textual data stored in the [[MariaDB]] database.
==Implementation==
The default [[up.sh]] shell script is a way to allow users to kickoff the program. This process does the following tasks:
# Before shutting down existing instance of docker services, it first asks the existing instance to backup the data.
# Then, use [[docker-compose down --volumes]] to shut down all the docker services.
# If the [[mountPoint/]] directory does not exist, yet, decompress [[InitialDataPackage.tar.gz]] to the mountPoint/ directory.
# When the mountPoint directory has the initial data content, start docker services, using docker-compose up -d
# The docker-compose program reads [[docker-compose.yml]] file to configure docker services.
# When the docker services start running, take the existing data in the directory, if [[images/UploadedFiles/]] directory or [[XLPLATEST.xml]] are available, these data elements will be imported to the database.
==Instructions==
There are two main parts of Backing up MediaWiki Data, the MySQL(MariabDB) database, and uploaded Media Files.
'''We strongly advise administrators to first restore the image files, then, load the Database content.'''
This way, all the images would be referenced correctly. This is based on operational experience.
===Media File Backup===
{{:Backup MediaWiki MediaFiles}}


For more details, see [[MediaWiki's Installation, Backup, Restore, and Recovery]].
For more details, see [[MediaWiki's Installation, Backup, Restore, and Recovery]].
Line 42: Line 18:
{{:Backup MediaWiki Database}}
{{:Backup MediaWiki Database}}


==Restoring SQL Data==
===Media File Backup===
The following instruction should be launched in the host (through docker exec or kubectl exec -it command) of the container that hosts the mariadb/mysql service.
{{:Backup MediaWiki MediaFiles}}
 
mysql -u $DATABASE_USER -p $DATABASE_NAME < BACKUP_DATA.sql
 


==Restoring Binary Files==
==Restoring Binary Files==
Line 63: Line 36:
For more information, please refer to MediaWiki's documentation on [[MW:Manual:rebuildImages.php|Manual:rebuildImages.php]].
For more information, please refer to MediaWiki's documentation on [[MW:Manual:rebuildImages.php|Manual:rebuildImages.php]].


=Backup and Restore Kubernetes=
==Restoring SQL Data==
Please refer to '''Cloud Native DevOps with Kubernetes'''<ref>{{:BOOK/Cloud Native DevOps with Kubernetes/Backing Up Cluster State}}</ref>. Particularly looking at [[Velero]].
The following instruction should be launched in the host (through docker exec or kubectl exec -it command) of the container that hosts the mariadb/mysql service.
mysql -u $DATABASE_USER -p $DATABASE_NAME < BACKUP_DATA.sql


=References=
=References=
<references />
<references />

Revision as of 13:01, 12 January 2022

Introduction

To ensure this MediaWiki's content will not be lost, we created a set of script and put it in $wgResourceBase's extensions/BackupAndRestore directory.

The main challenge is to ensure both textual data and binary files are backed up and restored. The most reliable ways are the two following actions:

  1. Official Database Backup Tools
  2. #backup and Restore Using maintenance scripts

Alternatively, the data set could be exported to a SQL file. Since that is running in a separate Docker service, the process will be discussed in a different page.

Data backup practice should follow something like 3-2-1 backup principle:

  • 3 Copies of Data
  • 2 Types of Storage
  • 1 Offsite storage location

There are other practices, which can be found here[1].

For more details, see MediaWiki's Installation, Backup, Restore, and Recovery.

Database Backup

For textual data backup, the fastest way is to use "mysqldump". The more detailed instructions can be found in the following link: [2]

To backup all the uploaded files, such as images, pdf files, and other binary files, you can reference the following Stack Overflow answer[3]

In the PKC docker-compose configuration, the backup file should be dumped to /var/lib/mysql for convenient file transfer on the host machine of Docker runtime. Example of the command to run on the Linux/UNIX shell:

mysqldump -h hostname -u userid -p --default-character-set=whatever dbname > backup.sql

For running this command in PKC's docker implementation, one needs to get into the Docker instance using something like:

docker exec -it pkc-mediawiki-1 /bin/bash (pkc-mediawiki-1 may be replace by xlp_mediawiki)

Whem running this command on the actual database host machine, hostname can be omitted, and the rest of the parameters are explained below:

mysqldump -u wikiuser -pPASSWORD_FOR_YOUR_DATABASE my_wiki > backup.sql
(note that you should NOT leave a space between -p and the passoword data)

Substituting hostname, userid, whatever, and dbname as appropriate. All four may be found in your LocalSettings.php (LSP) file. hostname may be found under $wgDBserver; by default it is localhost. userid may be found under $wgDBuser, whatever may be found under $wgDBTableOptions, where it is listed after DEFAULT CHARSET=. If whatever is not specified mysqldump will likely use the default of utf8, or if using an older version of MySQL, latin1. While dbname may be found under $wgDBname. After running this line from the command line mysqldump will prompt for the server password (which may be found under Manual:$wgDBpassword in LSP).

For your convenience, the following instruction will compress the file as it is being dumped out.

mysqldump -h hostname -u userid -p dbname | gzip > backup.sql.gz

Media File Backup

The most essential command is the following:

First, create a temporary working directory: mkdir /tmp/workingBackupMediaFiles It is common to have files not being dumped out, due to errors caused by escape characters in File names. This will be resolved in the future. You must first go to the proper directory, in the case of standard PKC configuration, you must make sure you launch the following command at this location /var/www/html:

php maintenance/dumpUploads.php \
   | sed 's~mwstore://local-backend/local-public~./images~' \
   | xargs cp -t /tmp/MediaFiles

Then, compress the file in a zip directory. zip -r ~/Mediafiles.zip /tmp/MediaFiles

Remember to remove the temporary files and its directory. rm -r /tmp/workingBackupMediaFiles


For more information: Reference the following Stack Overflow answer:

  1. Exporting and importing images in MediaWiki
  2. MediaWiki data backup
  3. MediaWiki Backup and Restore Bash Scripts

Restoring Binary Files

The process of restoring binary files, such as images, PDF, and other binary format data, should refer to Restore.sh.

Instruction for restoring Binary Files to MediaWiki

Loading binary files to MediaWiki, one must use a maintenance script in the /maintenance directory. This is the command line information. It needs to be launched in the container that runs MediaWiki instance.

Load images from the UploadedFiles location. In most cases, the variable $ResourceBasePath string can be replaced by /var/www/html.

cd $ResourceBasePath
php $ResourceBasePath/maintenance/importImages.php $ResourceBasePath/images/UploadedFiles/

After all files are uploaded, one should try to run a maintenance scrip on the server that serves Mediawiki service:

php $ResourceBasePath/maintenance/rebuildImages.php

For more information, please refer to MediaWiki's documentation on Manual:rebuildImages.php.

Restoring SQL Data

The following instruction should be launched in the host (through docker exec or kubectl exec -it command) of the container that hosts the mariadb/mysql service.

mysql -u $DATABASE_USER -p $DATABASE_NAME < BACKUP_DATA.sql

References