Running Kubernetes on Photon OS ----------------------------------------------------- **Table of Contents** - [Prerequisites](#prerequisites) - [Instructions](#instructions) ## Prerequisites * You need two or more machines with the 1.0 general availability or later version of Photon OS installed. ## Instructions This document gets you started using Kubernetes with Photon OS. The instructions present a manual configuration that gets one worker node running to help you understand the underlying packages, services, ports, and so forth. The Kubernetes package provides several services: kube-apiserver, kube-scheduler, kube-controller-manager, kubelet, kube-proxy. These services are managed by systemd. Their configuration resides in a central location: /etc/kubernetes. The following instructions break the services up between the hosts. The first host, `photon-master`, will be the Kubernetes master. This host will run the kube-apiserver, kube-controller-manager, and kube-scheduler. In addition, the master will also run `etcd`. Although `etcd` is not needed on the master if `etcd` runs on a different host, this guide assumes that `etcd` and the Kubernetes master run on the same host. The remaining host, `photon-node`, will be the node; it will run kubelet, proxy, and docker. **System Information** Hosts: ``` photon-master = 192.168.121.9 photon-node = 192.168.121.65 ``` **Prepare the hosts** The following packages should already be installed on the full version of Photon OS, but you might have to install them on the minimal version of Photon OS. If the `tdnf` command returns "Nothing to do," the package is already installed. * Install Kubernetes on all hosts--both `photon-master` and `photon-node`. ```sh tdnf install kubernetes ``` * Install iptables on photon-master: ```sh tdnf install iptables ``` * Install Docker on photon-node: ```sh tdnf install docker ``` * Add master and node to /etc/hosts on all machines (not needed if the hostnames are already in DNS). Make sure that communication works between photon-master and photon-node by using a utility such as ping. ```sh echo "192.168.121.9 photon-master 192.168.121.65 photon-node" >> /etc/hosts ``` * Edit /etc/kubernetes/config, which will be the same on all the hosts (master and node), so that it contains the following lines: ```sh # Comma separated list of nodes in the etcd cluster KUBE_MASTER="--master=http://photon-master:8080" # logging to stderr routes it to the systemd journal KUBE_LOGTOSTDERR="--logtostderr=true" # journal message level, 0 is debug KUBE_LOG_LEVEL="--v=0" # Should this cluster be allowed to run privileged docker containers KUBE_ALLOW_PRIV="--allow_privileged=false" ``` **Configure the Kubernetes services on the master** * Edit /etc/kubernetes/apiserver to appear as such. The service_cluster_ip_range IP addresses must be an unused block of addresses, not used anywhere else. They do not need to be routed or assigned to anything. ```sh # The address on the local server to listen to. KUBE_API_ADDRESS="--address=0.0.0.0" # Comma separated list of nodes in the etcd cluster KUBE_ETCD_SERVERS="--etcd_servers=http://127.0.0.1:4001" # Address range to use for services KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16" # Add your own KUBE_API_ARGS="" ``` * Start the appropriate services on master: ```sh for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler; do systemctl restart $SERVICES systemctl enable $SERVICES systemctl status $SERVICES done ``` * To add the other node, create the following node.json file on the Kubernetes master node: ```json { "apiVersion": "v1", "kind": "Node", "metadata": { "name": "photon-node", "labels":{ "name": "photon-node-label"} }, "spec": { "externalID": "photon-node" } } ``` Now create a node object internally in your Kubernetes cluster by running the following command: ```console $ kubectl create -f ./node.json $ kubectl get nodes NAME LABELS STATUS photon-node name=photon-node-label Unknown ``` Please note that in the above example, it only creates a representation for the node _photon-node_ internally. It does not provision the actual _photon-node_. Also, it is assumed that _photon-node_ (as specified in `name`) can be resolved and is reachable from the Kubernetes master node. How to provision a Kubernetes node (photon-node) is shown in a later section. **Configure the Kubernetes services on the node** You configure the kubelet on the node as follows. * Edit /etc/kubernetes/kubelet to appear like this: ```sh ### # Kubernetes kubelet (node) config # The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces) KUBELET_ADDRESS="--address=0.0.0.0" # You may leave this blank to use the actual hostname KUBELET_HOSTNAME="--hostname_override=photon-node" # location of the api-server KUBELET_API_SERVER="--api_servers=http://photon-master:8080" # Add your own #KUBELET_ARGS="" ``` * Start the appropriate services on the node (photon-node): ```sh for SERVICES in kube-proxy kubelet docker; do systemctl restart $SERVICES systemctl enable $SERVICES systemctl status $SERVICES done ``` * Check to make sure that the cluster can now see the photon-node on photon-master and that its status changes to _Ready_. ```console kubectl get nodes NAME LABELS STATUS photon-node name=photon-node-label Ready ``` If the node status is `NotReady`, verify that the firewall rules are permissive for Kubernetes. * Deletion of nodes: To delete _photon-node_ from your Kubernetes cluster, one should run the following on photon-master (please do not do it, it is just for information): ```sh kubectl delete -f ./node.json ``` That's it. You should have a functional cluster. You can now launch a test pod. Check out [Kubernetes 101](http://kubernetes.io/docs/user-guide/walkthrough/) for an introduction to working with Kubernetes.