Reinstall procedures

Last modified by Hostkey Admin on 2022/07/14 11:32

We are using Foreman to deploy servers via PXE network boot. The installation process goes in several stages:

Create reinstall stage key (optional);
Create PXE config with parameters;
Change boot order for the server to start with PXE;
Reboot the server;
The server will catch the PXE bootloader and OS installation will go on;
OS will report installation stages during install progress:
1. 1st stage: OS installation was started;
2. 2nd stage: base OS was installed, proceed with post-install tasks;
3. 3rd stage: Post-install tasks are completed, running final tasks;
4. 4th stage: Install is completed, restarting the server;
5. 5th stage: The server is online;
Change boot order back to HDD;
Run extra Jenkins task on a server (i.e. install GPU drivers);
Remove PXE config to avoid a sudden reinstall.

Reinstall procedures data:

Resource	Action	Description
eq.php	reinstall	Create master key for reinstall
os.php	list	Get a list of OS for a server
jenkins.php	get_tasks	Retrieve available Ansible tasks
eq.php	create_pxe	Create PXE config for OS install
eq.php	boot_dev	Set boot order for VM or bare-metal server
eq.php	reboot	Reboot server

Stage 0: Create an optional reinstall key

If we need to automate the whole reinstall process, we will need a master key to keep track of different actions. All stage alerts from the OS installer go to this key. As the OS install goes on, it will report stages via async responses via this key. This key will be required for eq/create_pxe call.

Eq/Reinstall

HTTP Method - POST

Parameter	Required	Type	Value/default	Description
action	*	string	reinstall	Main action - Create master key for reinstall
token	*	string		Your session token
id	*	int		Your server ID

curl -s "https://invapi.hostkey.com/eq.php" -X POST \
--data "action=reinstall" \
--data "token=" \
--data "id=" \

Example response (async):

Stage 1: Create PXE config

Before we start with reinstall, some mandatory information should be collected:

Os/list

HTTP Method - POST

Parameter	Required	Type	Value/default	Description
action	*	string	list	Main action - Get a list of OS for a server
token	*	string		Your session token
id	*	int		Your server ID

Retrieve JSON with a suitable OS list for a specific server with server_id (25250 for example). The only applicable OSes will be listed.

Notable tags will be provided for each OS to generate proper UI:

bm - this OS could run on a bare-metal server;
gpu - OS could run on GPU server;
vgpu - OS could run on vGPU server (VM with GPU in PCIe pass-through mode);
vm - OS could run on a virtual machine (VPS);
min_ram - minimal requirements for RAM in Gb;
min_hdd - minimal requirements for HDD in Gb;
price_per_core_EUR - price per core for MS SPLA licenses in EUR without applicable VAT. Minimal 8 licenses are required with increment van 2;

curl -s "https://invapi.hostkey.com/os.php" -X POST \
--data "action=list" \
--data "token=" \
--data "id=" \

Example response:

The user should select hostname and password.

Disk partitioning scheme:

According to the number of storage devices in the server, the user should decide which partitioning will be used:

LVM_RAID0 - create stripe across all disks via LVM;
LVM_HBA - install OS on the smallest device, let the rest untouched, applicable if there are more than 1 storage device;
LVM_RAID1 - create disk mirror via LVM, applicable if there are two similar storage devices;
LVM_RAID10 - create LVM RAID10 across all storage device pairs, minimum 2 pairs of devices required;
LVM_RAID10S - create LVM RAID1 on smallest storage devices pairs for OS, create LVM RAID10 on rest of storage devices pars.

Public ssh key - if the customer wants to install one for the administrative user, he could provide it for configuration request.
Post-install task - standardized Ansible tasks which could be run on the server after installation via Jenkins.

You could retrieve applicable tasks via jenkins/list call:

Jenkins/get_tasks

This will retrieve all possible Ansible tasks with its tags. You could load this data in JS variables and show it for appropriate servers - checking by tags.

HTTP Method - POST

Parameter	Required	Type	Value/default	Description
action	*	string	get_tasks	Main action - Retrieve available Ansible tasks

Possible tags:

gpu - task is suitable for GPU servers;
bm - task is suitable for bare-metal server;
vm - task is suitable for VM (VPS);
vgpu - task is suitable for vGPU server;
default - default value.

curl -s "https://invapi.hostkey.com/jenkins.php" -X POST \
--data "action=get_tasks" \

Example response:

Post-install script - the piece of bash/PS code to be executed on the first run in fresh OS. Useful for automation to include a new server to the managed infrastructure.

Post-install callback URL - URL which will be called from the fresh server after installation. Useful for billing purposes.

As we have all data ready, we could try to create a PXE config for OS install

Eq/Create_pxe

HTTP Method - POST

Parameter	Required	Type	Value/default	Description
action	*	string	create_pxe	Main action - Create PXE config for OS install
token	*	string		Your session token
id	*	int		Your server ID
pin	*	int		Your PIN code
os_id	*	int		Selected OS id from os/list
root_pass	*	string		password for the administrative user. length and content should meet security policy - min 8 symbols, with some numbers and letters in caps.
hostname	*	string		Desired hostname for the server
ssh_key		string		Public ssh key
post_install_callback		string		URL to call after install is completed
post_install_script		string		Post-install script to run
reinstall_key		string		optional key from eq/reinstall to keep track of reinstall stages. All reinstall stages will be reported from this key via eq_callback.php?action=check&key=reinstall_key_value

curl -s "https://invapi.hostkey.com/eq.php" -X POST \
--data "action=create_pxe" \
--data "token=" \
--data "email=" \
--data "pin=" \
--data "id=" \
--data "os_id=" \
--data "root_pass=" \
--data "hostname=" \
--data "ssh_key=" \
--data "post_install_callback=" \
--data "post_install_script=" \
--data "reinstall_key=" \

Example response (async):

This action has asynchronous response

Example response (async):

If there was no errors, we need to set server's boot order to network boot via PXE.

Eq/Boot_dev

HTTP Method - POST

Parameter	Required	Type	Value/default	Description
action	*	string	boot_dev	Main action - Set boot order for VM or bare-metal server
token	*	string		Your session token
pin	*	int		Your PIN code
id	*	int		Your server ID
media - pxe		int		for PXE boot, disk for local storage device boot, cd for mounted CD boot (usually via virtual media via IPMI)

curl -s "https://invapi.hostkey.com/eq.php" -X POST \
--data "action=boot_dev" \
--data "token=" \
--data "id=" \
--data "pin=" \

Example response:

This action has asynchronous response

Example response (async):

After successful change of boot order we should reboot server.

Eq/Reboot

HTTP Method - POST

Parameter	Required	Type	Value/default	Description
action	*	string	reboot	Main action - Reboot server
token	*	string		Your session token
pin	*	int		Your PIN code
id	*	int		Your server ID

curl -s "https://invapi.hostkey.com/eq.php" -X POST \
--data "action=reboot" \
--data "id=" \
--data "token=" \
--data "pin=" \

Example response:

This action has asynchronous response

Example response (async):

The system will reboot and start loading from PXE. After the OS installer is loaded, you will see stage notifications - it could be retrieved with the main reinstall key from eq/reinstall

curl -s "https://invapi.hostkey.com/eq_callback.php" -X POST \
--data "action=check" \
--data "key=" \

Example response:

When "result" key is "Stage" - this means its informative message about status progress. "scope" key contains user-readable information. "context" key helps us to keep track of installation data, it contains main action, server id and it's location. "key" is master reinstall key.

There will be 5 such a notifications, last one will be.

Example response:

If we have got "result" OK, this means server is online and reinstall is completed. After we will retrieve this value, reinstall key will be removed from database.

Post-install tasks and cleanup:

Eq/boot_dev

HTTP Method - POST

Parameter	Required	Type	Value/default	Description
action	*	string	boot_dev	Main action - Switch boot order back to disk
token	*	string		Your session token
id	*	int		Your server ID
media	*	int		Switch boot order back to disk
pin	*	int		Your PIN code

curl -s "https://invapi.hostkey.com/eq.php" -X POST \
--data "action=boot_dev" \
--data "token=" \
--data "id=" \
--data "media=" \
--data "pin=" \

This action has asynchronous response

Async response:

Eq/clear_pxe

Сlear PXE config to avoid sudden reinstalls.

HTTP Method - POST

Parameter	Required	Type	Value/default	Description
action	*	string	clear_pxe	Main action - Clear PXE config to avoid sudden reinstalls
token	*	string		Your session token
id	*	int		Your server ID
hostname	*	string		Your server name

curl -s "https://invapi.hostkey.com/eq.php" -X POST \
--data "action=clear_pxe" \
--data "token=" \
--data "id=" \
--data "hostname=" \

Example response:

This action has asynchronous response

Async response:

Tags:

Reinstall procedures

Stage 0: Create an optional reinstall key

Eq/Reinstall

Stage 1: Create PXE config

Os/list

Disk partitioning scheme:

Jenkins/get_tasks

Eq/Create_pxe

Eq/Boot_dev

Eq/Reboot

Post-install tasks and cleanup:

Eq/boot_dev

Eq/clear_pxe

Navigation