PKC Complete Installation Process
Introduction
This page is written to give complete documentation on how the PKC is installed, description of all the related services, and the concept behind the architecture design. PKC Architecture is implemented at several docker container that runs its own service on same virtual network. PKC can be installed on a Virtual Machine or Bare-Metal machine.
Docker Concept
Docker is an containerized application runs on top of host operating system. It is some form of more lightweight implementation of virtualization. All we need to do to deploy and shipped new application is to containerized all our executable code and all its dependencies inside one docker container, and we are ready to delivered the application.
- Docker is a tools, to develop and run container
- Container is the packed executable and application code.
In order to run container, we need some kind of tools, which in this case are using Docker. For further readings, please visit this page for comprehensive explanation of Docker by Bibin Wilson.[1]
Below is the simple diagram on how container is deployed into one's computer, to give better understanding on high-level concept of docker container.
On host Operating System, administrator only need to install and configured Docker or other software with the capability to run container, and installed all the application container within the operating system. Them all the service will be ready to go without need to install all the code and dependecies.
Ansible Concept
Ansible is open-source tools that provide automation tools for cross-platform computer support. Enabled developer or DevOps to perform various task of installation, configuration, and provisioning computer and/or server. Basically everything that system administrator does in routine. And it does not depend on any agent software nor additional security layers. So its easy to use and deploy. Below are the simple schematic diagram to better give high-level understanding of ansible.
The Ansile script, only need to be deployed on one ansible agent with connection to the servers. Then the task that has already defined in ansible agent will be executed on target computer and/or server.
PKC Architecture Review
Please find below are the diagram of PKC Implementation on micro-services and containerization.
PKC consist of below containers
- XLP_Mediawiki: Mediawiki service, this is where all the content of XLP can be accessed, and the entrypoint for the implementation.
- XLP_Gitea: Gitea service, for storing files and source-code.
- XLP_Keycloak: Identity service provider for managing users
- XLP_Matomo: Web Analytic Service, to monitor the usage and
- XLP_VS: Web-Based Visual Studio Code, to manage source-code.
- phpmyadmin: Tools for managing the databaser server.
- XLP_MariaDB: mySQL RDBMS Provider
- NGINX: Installed on host server, to manage the connectivity between incoming traffic and microservices.
All the binds folder is under the folder of $HOME/cs/mountpoint, below are the list of folder bind into containers
| Folder/Filename | Container | Container's Folder |
|---|---|---|
| backup_restore/mariadb | xlp_mariadb | /mnt/backup_restore/mariadb |
| backup_restore/mediawiki | xlp_mediawiki | /mnt/backup_restore/mediawiki |
| gitea | gitea | /data |
| images | xlp_mediawiki | /var/www/html/images |
| LocalSettings.php | xlp_mediawiki | /var/www/html/LocalSettings.php |
| mariadb | xlp_mariadb | /var/lib/mysql |
| matomo | xlp_matomo | /bitnami/matomo |
| vs | xlp_vs | /config |
| mysql-init | xlp_mariadb | /docker-entrypoint-initdb.d |
NGINX Configuration is stored on host folder /etc/nginx/sites-enabled, which forward traffic from outside into each container's port. below are the table of default PKC Configurations.
| Container | Port# | Subdomain | Remarks |
|---|---|---|---|
| xlp_mariadb | N/A | N/A | Not exposed |
| phpmyadmin | 32040 | pma | |
| xlp_keycloak | 32060 | kck | |
| xlp_matomo | 32010 | mtm | |
| xlp_mediawiki | 32001 | [Default] | |
| xlp_gitea | 32030 | git | |
| xlp_vs | 32050 | code |
Pre-Requisite
PKC can be installed on fresh-installation of Ubuntu Linux 20.08 or Ubuntu Linux 18.04 minimal server setup. There's no specific software that needs to be installed on target beforehand. However, there are several items that needs to be configured and installing ansibile on your local machine act as ansible agent before we can start the ansible script to setup PKC
How to provision your server
Below are the items that need to be properly setup before we can run the ansible installation script.
Install ansible on agent machine
On MacOS, type below command on terminal
brew install ansible
Notes, that you need homebrew installed beforehand.
On RHEL, type below command on terminal
sudo yum install ansible
On Ubuntu, you need to update the repository then install ansible, below are the command in terminal
sudo apt update sudo apt install software-properties-common sudo add-apt-repository --yes --update ppa:ansible/ansible sudo apt install ansible
At this moment, ansible is not supporting windows as ansible-agent, please see http://blog.rolpdog.com/2020/03/why-no-ansible-controller-for-windows.html
DNS Configuration
To enable the https:/ protocols, you will need to setup or procure domain name to point to your server's or VM's Public IP. Below are the default and subdomain configuration to setup.
| Subdomain | |
|---|---|
| [default] / www | Your Public IP |
| pma | |
| kck | |
| mtm | |
| git | |
| code | |
| swg | |
| qtux |
Add new user
You should not install PKC using root user. In the event you only have root access, it is imperative to create non-root user on your target server. Below is the step needed to create new user.
Create new user
Type in below command to create new user
sudo adduser [user-name] ex. new user is ubuntu sudo adduser ubuntu
and, follow the instruction on the display
Allow this user to sudo without password
Type below command
sudo visudo
and, linux system will display file to edit, please add one new line on the end of files
[user-name] ALL=(ALL) NOPASSWD:ALL
Provide Passwordless Access
To enable ssh passwordless access, you will need to enable the private key/public key method, this is required to run the automated ansible script. On some cloud provider such as AWS or Google Cloud, the only way to access the ssh is through public key/private key method. In such case, you can directly use the provided private key. To execute the process please refer to How to create passwordless access page.
Installation Execution
This section will provide the required step to install the PKC Infrastruction on your selected server. This process assuming that you already completed all the necessary pre-requisite above in advance.
Local Installation
You can install PKC on your local machine, the procedure has been tested on MacOS. Before you can proceed with the installation process, please prepare the below files.
1. Private Key file, this are the files to enable the docker container installer to access your local system.
# type the command below on terminal ssh-keygen -b 4096 -t rsa
You will be prompted to enter a filename. By default, your keys will be saved as id_rsa and id_rsa.pub. Simply press Enter to confirm the default - there is no need to change this unless you have multiple keys! (Note: if you would like to change the default filename, you'll need to include the complete file path). Please noted where you save this file, you will need it to execute the installation command
2. Localhost inventory file Please create a new file containing below text, please have a note on where you save the file, you will need it to execute the installation command
host.docker.internal ansible_connection=ssh ansible_ssh_private_key_file=~/.ssh/local-key.pem ansible_user=emhavis domain=pkc.local default_transport=http pkc_install_root_dir=/users/your-folder/pkc-local email=your-email-address
- only edit the pkc_install_root_dir, and put your chosen installation folder, on MacOS machine, please choose the folder inside your user's folder, e.g /Users/[your-user-name]/[your-chosen-folder]
- save your file
3. Installation execution Please prepare the below one-liner command and change the directory to your chosen folder. The installation process will create a new folder called 'cs' inside your folder and copy the system into it.
docker container run --rm \
--name pkc-agent \
-v [complete-path-to-inventory-file]:/pkc-agent/resources/config/hosts \
-v [complete-path-to-public-key].pub:/root/.ssh/local-key.pem.pub \
-v [complete-path-to-private-key].pem:/root/.ssh/local-key.pem \
-it emhavis/pkc_agent:sha-36ef9d4 /pkc-agent/up.sh && \
cd ./cs && docker-compose pull && docker-compose up -d && sleep 10 && \
./update-sw.sh && open http://pkc.local
Below are the example of the complete installation command for
host inventory: /Users/emhavis/pkc-local/hosts-localhost public key: /Users/emhavis/.ssh/local-key.pem.pub private key: /Users/emhavis/.ssh/local-key.pem
docker container run --rm \
--name pkc-agent \
-v /Users/emhavis/pkc-local/hosts-localhost:/pkc-agent/resources/config/hosts \
-v /Users/emhavis/.ssh/local-key.pem.pub:/root/.ssh/local-key.pem.pub \
-v /Users/emhavis/.ssh/local-key.pem:/root/.ssh/local-key.pem \
-it emhavis/pkc_agent:sha-36ef9d4 /pkc-agent/up.sh && \
cd ./cs && docker-compose pull && docker-compose up -d && sleep 10 && \
./update-sw.sh && open http://pkc.local
Docker Method
Updated on [May-2022], to defined the target of server installation, please create hosts file in your chosen folder. Below are the example and the explanation of each field
[host-name] ansible_connection=ssh ansible_ssh_private_key_file=[private-key.pem] ansible_user=ubuntu domain=[domain-name] default_transport=[default-transport] email=emhavis@gmail.com- host-name
Your target server as seen by ansible agent. This host also can be registered on your config file, usually located at ~/.ssh/config
- private-key.pem
Your private key to access the host target
- domain-name
Your domain name, which will be use to access the PKC
- default-transport
type of default transport, available options are http or https. If you choose https, then the certbot installation procedure will be executed. Also noted that this procedure will required that http protocol for domain to be accessible from internet. Please also note that you need dockers installed on your system.
And you can execute below one-liner code to execute PKC's installation process
docker container run --rm \
-v [path-to-host inventory-file]:/pkc-agent/resources/config/hosts \
-v [path-to-pem-file]:/root/.ssh/[your-pem-file-name] \
-it emhavis/pkc_agent:dev /pkc-agent/up.sh
Source Code
To clone the specific ansible script for this PKC installation, you need change to your chosen folde and clone git repository with branch [dev], with below command:
git clone https://github.com/xlp0/PKC.git
This command will create new folder called PKC inside the folder, then change to PKC folder by executing change folder command
cd PKC
There are two files that you will need to adjust, according to your installation target, first is .env file, first line is YOUR_DOMAIN,and second line is YOUR_EMAIL_ADDRESS.
# [P]ersonal [K]ownlegde [C]ontainer # PKC .env configuration files # # # Inform your domain here, keep at localhost for local implementation # YOUR_DOMAIN=[put your domain here] # Inform your transport methods, highly recommended to use HTTPS # DEFAULT_TRANSPORT=https # Email notification for Certbot Renewal YOUR_EMAIL_ADDRESS=[put your email address here] # # # Please DO NOT Change below configuration entry # unless you really know what you are doing # # Port Number for media wiki PORT_NUMBER=32001 # Port Number for matomo MATOMO_PORT_NUMBER=32010 # Port Number for PHP My Admin PHP_MA=32040 # Port Number for Gitea GITEA_PORT_NUMBER=32030 # Port Number for vs-code VS_PORT_NUMBER=32050 # VS Default password VS_PASSWORD=xlp-vs-pass # VS Sudo Password VS_SUDO_PASSWORD=xlp-vs-sudo # Keycloack Port Number KCK_PORT_NUMBER=32060
YOUR_DOMAIN is the target where PKC will be installed
YOUR_EMAIL_ADDRESS is the email address will be use for registering SSL Certbot.
Second file is the hosts file. In this file, you need to inform installer how to access the server, and the respecting private key location. Below are the format of the file
[your server IP Address or entry in ~/.ssh/config] ansible_connection=[type of connection] ansible_ssh_private_key_file=[complete folder location and filename for private key] ansible_user=[server's user] domain=[your domain name]
Below are the example of the file content, once adjustment has been made
my-domain.org ansible_connection=ssh ansible_ssh_private_key_file=~/.ssh/my-private-key ansible_user=myuser domain=my-domain.org
Once both file is adjusted, then we are ready to execute the ansible script for PKC installation, run below command on terminal within the working folder.
./up.sh
The ansible script in general will perform below tasks
- Preparing NGINX configuration files
- Preparing Mediawiki configuration files
- Clean-up existing Dockers container and NGINX Configuration if any, please also noted that on running servers, this task will also delete existing running container and NGINX Configuration
- Installing NGINX (if not exists)
- Installing Software Dependencies, which are 'apt-transport-https', 'ca-certificates ', 'curl', 'software-properties-common', 'python3-pip', 'virtualenv', 'python3-setuptools', and 'unzip'
- Installing Docker and docker-compose
- Uploading supporting shell scripts, which reside in folder ./scripts
- Uploading docker-compose and .env files
- Bring docker-compose up
- Install and Configuring SSL using Cerbot
Related Pages
Explaining LocalSettings.php
Managing PKC Users