> For the complete documentation index, see [llms.txt](https://docs.mayaprotocol.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.mayaprotocol.com/node-docs/mayanodes/kubernetes/setup-hetzner-bare-metal.md).

# Setup - Hetzner Bare Metal

{% hint style="danger" %}
This guide for Hetzner Bare Metal is WIP and not currently recommended. Proceed with caution until an update is released and this warning removed.
{% endhint %}

> Checkout the repository [source](https://github.com/delphidigital/bare-metal-cluster-manager) to manage a cluster of dedicated servers on Hetzner.

The scripts in this repository will set up and maintain one or more [kubernetes](https://kubernetes.io) clusters consisting of dedicated [Hetzner](https://www.hetzner.com) servers. Each cluster will also be provisioned to operate as a node in the [MAYACHain](https://mayachain.org) network.

Executing the scripts in combination with some manual procedures will get you highly available, secure clusters with the following features on bare metal.

* [Kubespray](https://kubespray.io/) (based)
* Internal NVMe storage ([Ceph](https://ceph.io)/[Rook](https://rook.io))
* Virtual LAN (also over multiple locations) ([Calico](https://www.projectcalico.org))
* Load Balancing ([MetalLB](https://metallb.universe.tf))

## Preparations

### Servers

Acquire a couple of [servers](https://www.hetzner.com/dedicated-rootserver/matrix-ax) as the basis for a cluster (`AX41-NVME`'s are working well, for instance). Visit the [admin panel](https://robot.your-server.de/server) and name the servers appropriately.

```
ma-k8s-node1
ma-k8s-node2
ma-k8s-node3
...

ma-k8s-master1
ma-k8s-master2
ma-k8s-worker1
ma-k8s-worker2
ma-k8s-worker3
...
```

Refer to the [reset procedure](#resetting-the-bare-metal-servers) to initialize them properly.

### vSwitch

Create a [vSwitch](https://robot.your-server.de/vswitch/index) and order an appropriate subnet (it may take a while to show up after the order). Give the vSwitch a name (i.e. `ma-k8s-net`) and assign this vSwitch to the servers.

Check out the [docs](https://docs.hetzner.com/robot/dedicated-server/network/vswitch) for help.

## Usage

Clone this repository `cd` into it and download kubespray.

```bash
git submodule init && git submodule update
```

Create a Python virtual environment or similar.

```bash
# Optional
virtualenv -p python3 venv
```

Install dependencies required by Python and Ansible Glaxy.

```bash
pip install -r requirements.python.txt
ansible-galaxy install -r requirements.ansible.yml
```

> Note: [Mitogen](https://mitogen.readthedocs.io/en/python3/ansible.html) does not work with ansible collections and the strategy must be changed (i.e. `strategy: linear`).

### Provisioning

Create a deployment environment inventory file for each cluster you want to manage.

```bash
cp hosts.example inventory/production.yml
cp hosts.example inventory/test.yml
cp hosts.example inventory/environment.yml
...

cp hosts.example inventory/production-01.yml
cp hosts.example inventory/production-02.yml
...

cp hosts.example inventory/production-helsinki.yml
cp hosts.example inventory/whatever.yml
```

Edit the inventory file with your server ip's and network information and customize everything to your needs.

```bash
# Manage a cluster
ansible-playbook cluster.init.yml -i inventory/environment.yml
ansible-playbook --become --become-user=root kubespray/cluster.yml -i inventory/environment.yml
ansible-playbook cluster.finish.yml -i inventory/environment.yml

# Run custom playbooks
ansible-playbook private-cluster.yml -i inventory/environment.yml
ansible-playbook private-test-cluster.yml -i inventory/environment.yml
ansible-playbook private-whatever-cluster.yml -i inventory/environment.yml
```

> Check [this](https://kubespray.io/) out for more playbooks on cluster management.

### MAYAChain

For the cluster to operate as a node in the MAYACHain network, deploy as instructed [here](https://docs.mayachain.org/mayanodes/kubernetes/deploying). You can also refer to the [node-launcher repository](https://gitlab.com/mayachain/devops/node-launcher), if necessary, or the MAYAChain [documentation](https://docs.mayachain.org) as a whole.

## Resetting the bare metal servers

This will install and use Ubuntu 20.04 on only one of the two internal NVMe drives. The unused ones will be used for persistent storage with ceph/rook. You can check the internal drive setup with `lsblk`. Change it accordingly in the command shown above when necessary.

### Manually

Visit the [console](https://robot.your-server.de/server) and put each server of the cluster into rescue mode. Then execute the following script.

```bash
installimage -a -r no -i images/Ubuntu-2004-focal-64-minimal.tar.gz -p /:ext4:all -d nvme0n1 -f yes -t yes -n hostname
```

### Automatically

Create a pristine state by running the playbooks in sequence.

```bash
ansible-playbook server.rescue.yml -i inventory/environment.yml
ansible-playbook server.bootstrap.yml -i inventory/environment.yml
```

### Instantiation

Instantiate the servers.

```bash
ansible-playbook server.instantiate.yml -i inventory/environment.yml
```


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.mayaprotocol.com/node-docs/mayanodes/kubernetes/setup-hetzner-bare-metal.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.