PKC Complete Installation Process

Introduction

This page is written to give complete documentation on how the PKC is installed, description of all the related services, and the concept behind the architecture design. PKC Architecture is implemented at several docker container that runs its own service on same virtual network. PKC can be installed on a Virtual Machine or Bare-Metal machine.

Pre-requisite

You must already have git installed on your computer, and please go to the Github PKC Repository to download the source code.

Docker Concept

Docker is an containerized application runs on top of host operating system. It is some form of more lightweight implementation of virtualization. All we need to do to deploy and shipped new application is to containerized all our executable code and all its dependencies inside one docker container, and we are ready to delivered the application.

• Docker is a tools, to develop and run container
• Container is the packed executable and application code.

In order to run container, we need some kind of tools, which in this case are using Docker. For further readings, please visit this page for comprehensive explanation of Docker by Bibin Wilson.
Below is the simple diagram on how container is deployed into one's computer, to give better understanding on high-level concept of docker container.

Simple diagram on How Docker is Deployed

On host Operating System, administrator only need to install and configured Docker or other software with the capability to run container, and installed all the application container within the operating system. Them all the service will be ready to go without need to install all the code and dependecies.

Ansible Concept

Ansible is open-source tools that provide automation tools for cross-platform computer support. Enabled developer or DevOps to perform various task of installation, configuration, and provisioning computer and/or server. Basically everything that system administrator does in routine. And it does not depend on any agent software nor additional security layers. So its easy to use and deploy. Below are the simple schematic diagram to better give high-level understanding of ansible.

Simple diagram on How Ansible is Work

The Ansile script, only need to be deployed on one ansible agent with connection to the servers. Then the task that has already defined in ansible agent will be executed on target computer and/or server.

PKC Architecture Review

Please find below are the diagram of PKC Implementation on micro-services and containerization.

PKC Implementation Overview

PKC consist of below containers

1. XLP_Mediawiki: Mediawiki service, this is where all the content of XLP can be accessed, and the entrypoint for the implementation.
2. XLP_Gitea: Gitea service, for storing files and source-code.
3. XLP_Keycloak: Identity service provider for managing users
4. XLP_Matomo: Web Analytic Service, to monitor the usage and
5. XLP_VS: Web-Based Visual Studio Code, to manage source-code.
6. phpmyadmin: Tools for managing the databaser server.
7. XLP_MariaDB: mySQL RDBMS Provider
8. NGINX: Installed on host server, to manage the connectivity between incoming traffic and microservices.

All the binds folder is under the folder of $HOME/cs/mountpoint, below are the list of folder bind into containers

Folder/Filename	Container	Container's Folder
backup_restore/mariadb	xlp_mariadb	/mnt/backup_restore/mariadb
backup_restore/mediawiki	xlp_mediawiki	/mnt/backup_restore/mediawiki
gitea	gitea	/data
images	xlp_mediawiki	/var/www/html/images
LocalSettings.php	xlp_mediawiki	/var/www/html/LocalSettings.php
mariadb	xlp_mariadb	/var/lib/mysql
matomo	xlp_matomo	/bitnami/matomo
vs	xlp_vs	/config
mysql-init	xlp_mariadb	/docker-entrypoint-initdb.d

NGINX Configuration is stored on host folder /etc/nginx/sites-enabled, which forward traffic from outside into each container's port. below are the table of default PKC Configurations.

Container	Port#	Subdomain	Remarks
xlp_mariadb	N/A	N/A	Not exposed
phpmyadmin	32040	pma
xlp_keycloak	32060	kck
xlp_matomo	32010	mtm
xlp_mediawiki	32001	[Default]
xlp_gitea	32030	git
xlp_vs	32050	code

Pre-Requisite

PKC can be installed on fresh-installation of Ubuntu Linux 20.08 or Ubuntu Linux 18.04 minimal server setup. There's no specific software that needs to be installed on target beforehand. However, there are several items that needs to be configured and installing ansibile on your local machine act as ansible agent before we can start the ansible script to setup PKC

How to provision your server

Below are the items that need to be properly setup before we can run the ansible installation script.

Install ansible on agent machine

On MacOS, type below command on terminal

brew install ansible

Notes, that you need homebrew installed beforehand.
On RHEL, type below command on terminal

sudo yum install ansible

On Ubuntu, you need to update the repository then install ansible, below are the command in terminal

sudo apt update
sudo apt install software-properties-common
sudo add-apt-repository --yes --update ppa:ansible/ansible
sudo apt install ansible 

At this moment, ansible is not supporting windows as ansible-agent, please see http://blog.rolpdog.com/2020/03/why-no-ansible-controller-for-windows.html

DNS Configuration

To enable the https:/ protocols, you will need to setup or procure domain name to point to your server's or VM's Public IP. Below are the default and subdomain configuration to setup.

Subdomain
[default] / www	Your Public IP
pma
kck
mtm
git
code
swg
qtux

Add new user

You should not install PKC using root user. In the event you only have root access, it is imperative to create non-root user on your target server. Below is the step needed to create new user. Create new user
Type in below command to create new user

sudo adduser [user-name]
ex. new user is ubuntu
sudo adduser ubuntu

and, follow the instruction on the display
Allow this user to sudo without password Type below command

sudo visudo 

and, linux system will display file to edit, please add one new line on the end of files

[user-name]  ALL=(ALL) NOPASSWD:ALL

Provide Passwordless Access

To enable ssh passwordless access, you will need to enable the private key/public key method, this is required to run the automated ansible script. On some cloud provider such as AWS or Google Cloud, the only way to access the ssh is through public key/private key method. In such case, you can directly use the provided private key. To execute the process please refer to How to create passwordless access page.

Installation Execution

This section will provide the required step to install the PKC Infrastruction on your selected server. This process assuming that you already completed all the necessary pre-requisite above in advance.

Local Installation

You can install PKC on your local machine, the procedure has been tested on MacOS. Before you can proceed with the installation process, please prepare the below files.
1. Private Key file, this are the files to enable the docker container installer to access your local system.

# type the command below on terminal
ssh-keygen -t rsa -m PEM

You will be prompted to enter a filename. By default, your keys will be saved as id_rsa and id_rsa.pub. Simply press Enter to confirm the default - there is no need to change this unless you have multiple keys! (Note: if you would like to change the default filename, you'll need to include the complete file path). Please noted where you save this file, you will need it to execute the installation command

2. Localhost inventory file Please create a new file containing below text, please have a note on where you save the file, you will need it to execute the installation command

host.docker.internal ansible_connection=ssh ansible_ssh_private_key_file=~/.ssh/local-key.pem ansible_user=emhavis domain=pkc.local default_transport=http pkc_install_root_dir=/users/your-folder/pkc-local email=your-email-address

1. only edit the pkc_install_root_dir, and put your chosen installation folder, on MacOS machine, please choose the folder inside your user's folder, e.g /Users/[your-user-name]/[your-chosen-folder]
2. save your file
   

3. Installation execution Please prepare the below one-liner command and change the directory to your chosen folder. The installation process will create a new folder called 'cs' inside your folder and copy the system into it.

It is important to note that emhavis/pkc_agent:sha-36ef9d4 is a container address and should not be changed.

‎

docker container run --rm \
--name pkc-agent \
-v [complete-path-to-inventory-file]:/pkc-agent/resources/config/hosts \
-v [complete-path-to-public-key].pub:/root/.ssh/local-key.pem.pub \
-v [complete-path-to-private-key].pem:/root/.ssh/local-key.pem \
-it emhavis/pkc_agent:sha-36ef9d4 /pkc-agent/up.sh && \
cd ./cs && docker-compose pull && docker-compose up -d && sleep 10 && \
./update-sw.sh && open http://pkc.local

Below are the example of the complete installation command for

host inventory:
/Users/emhavis/pkc-local/hosts-localhost
public key:
/Users/emhavis/.ssh/local-key.pem.pub
private key:
/Users/emhavis/.ssh/local-key.pem

‎

docker container run --rm \
--name pkc-agent \
-v /Users/emhavis/pkc-local/hosts-localhost:/pkc-agent/resources/config/hosts \
-v /Users/emhavis/.ssh/local-key.pem.pub:/root/.ssh/local-key.pem.pub \
-v /Users/emhavis/.ssh/local-key.pem:/root/.ssh/local-key.pem \
-it emhavis/pkc_agent:sha-36ef9d4 /pkc-agent/up.sh && \
cd ./cs && docker-compose pull && docker-compose up -d && sleep 10 && \
./update-sw.sh && open http://pkc.local

Docker Method

Updated on [May-2022], to defined the target of server installation, please create hosts file in your chosen folder. Below are the example and the explanation of each field

[host-name] ansible_connection=ssh ansible_ssh_private_key_file=[private-key.pem] ansible_user=ubuntu domain=[domain-name] default_transport=[default-transport] email=emhavis@gmail.com

• host-name

Your target server as seen by ansible agent. This host also can be registered on your config file, usually located at ~/.ssh/config

• private-key.pem

Your private key to access the host target

• domain-name

Your domain name, which will be use to access the PKC

• default-transport

type of default transport, available options are http or https. If you choose https, then the certbot installation procedure will be executed. Also noted that this procedure will required that http protocol for domain to be accessible from internet. Please also note that you need dockers installed on your system.

And you can execute below one-liner code to execute PKC's installation process

docker container run --rm \
-v [path-to-host inventory-file]:/pkc-agent/resources/config/hosts \
-v [path-to-pem-file]:/root/.ssh/[your-pem-file-name] \
-it emhavis/pkc_agent:dev /pkc-agent/up.sh

Source Code

To clone the specific ansible script for this PKC installation, you need change to your chosen folde and clone git repository with branch [dev], with below command:

git clone https://github.com/xlp0/PKC.git

This command will create new folder called PKC inside the folder, then change to PKC folder by executing change folder command

cd PKC

There are two files that you will need to adjust, according to your installation target, first is .env file, first line is YOUR_DOMAIN,and second line is YOUR_EMAIL_ADDRESS.

# [P]ersonal [K]ownlegde [C]ontainer
# PKC .env configuration files
#
#
# Inform your domain here, keep at localhost for local implementation
#
YOUR_DOMAIN=[put your domain here]
# Inform your transport methods, highly recommended to use HTTPS
# 
DEFAULT_TRANSPORT=https
# Email notification for Certbot Renewal
YOUR_EMAIL_ADDRESS=[put your email address here]
#
#
# Please DO NOT Change below configuration entry
# unless you really know what you are doing
# 
# Port Number for media wiki
PORT_NUMBER=32001
# Port Number for matomo
MATOMO_PORT_NUMBER=32010
# Port Number for PHP My Admin
PHP_MA=32040
# Port Number for Gitea
GITEA_PORT_NUMBER=32030
# Port Number for vs-code
VS_PORT_NUMBER=32050
# VS Default password
VS_PASSWORD=xlp-vs-pass
# VS Sudo Password
VS_SUDO_PASSWORD=xlp-vs-sudo
# Keycloack Port Number
KCK_PORT_NUMBER=32060

YOUR_DOMAIN is the target where PKC will be installed
YOUR_EMAIL_ADDRESS is the email address will be use for registering SSL Certbot.
Second file is the hosts file. In this file, you need to inform installer how to access the server, and the respecting private key location. Below are the format of the file

[your server IP Address or entry in ~/.ssh/config] ansible_connection=[type of connection] ansible_ssh_private_key_file=[complete folder location and filename for private key] ansible_user=[server's user] domain=[your domain name]

Below are the example of the file content, once adjustment has been made

my-domain.org ansible_connection=ssh ansible_ssh_private_key_file=~/.ssh/my-private-key ansible_user=myuser domain=my-domain.org

Once both file is adjusted, then we are ready to execute the ansible script for PKC installation, run below command on terminal within the working folder.

./up.sh

The ansible script in general will perform below tasks

1. Preparing NGINX configuration files
2. Preparing Mediawiki configuration files
3. Clean-up existing Dockers container and NGINX Configuration if any, please also noted that on running servers, this task will also delete existing running container and NGINX Configuration
4. Installing NGINX (if not exists)
5. Installing Software Dependencies, which are 'apt-transport-https', 'ca-certificates ', 'curl', 'software-properties-common', 'python3-pip', 'virtualenv', 'python3-setuptools', and 'unzip'
6. Installing Docker and docker-compose
7. Uploading supporting shell scripts, which reside in folder ./scripts
8. Uploading docker-compose and .env files
9. Bring docker-compose up
10. Install and Configuring SSL using Cerbot

Related Pages

Explaining LocalSettings.php
Managing PKC Users