Running Kubernetes on Photon OS

 -----------------------------------------------------
 
 **Table of Contents**
 
 - [Prerequisites](#prerequisites)
 - [Instructions](#instructions)
 
 ## Prerequisites
 

 * You need two or more machines with the 1.0 general availability or later version of Photon OS installed.

 
 ## Instructions
 

 This document gets you started using Kubernetes with Photon OS. The instructions present a manual configuration that gets one worker node running to help you understand  the underlying packages, services, ports, and so forth. 

 The Kubernetes package provides several services: kube-apiserver, kube-scheduler, kube-controller-manager, kubelet, kube-proxy.  These services are managed by systemd. Their configuration resides in a central location: /etc/kubernetes.  

 The following instructions break the services up between the hosts.  The first host, `photon-master`, will be the Kubernetes master.  This host will run the kube-apiserver, kube-controller-manager, and kube-scheduler.  In addition, the master will also run `etcd`. Although `etcd` is not needed on the master if `etcd` runs on a different host, this guide assumes that `etcd` and the Kubernetes master run on the same host.  The remaining host, `photon-node`, will be the node; it will run kubelet, proxy, and docker.

 **System Information**

 
 Hosts:
 
 ```
 photon-master = 192.168.121.9
 photon-node = 192.168.121.65
 ```
 

 **Prepare the hosts**
 
 The following packages should already be installed on the full version of Photon OS, but you might have to install them on the minimal version of Photon OS. If the `tdnf` command returns "Nothing to do," the package is already installed.

 * Install Kubernetes on all hosts--both `photon-master` and `photon-node`.

 
 ```sh
 tdnf install kubernetes

 ``` 

 * Install iptables on photon-master and photon-node:

 
 ```sh

 tdnf install iptables

 ```
 

 * Open the tcp port 8080 (api service) on the photon-master in the firewall
 
 ```sh
 iptables -A INPUT -p tcp --dport 8080 -j ACCEPT
 ```
 
 * Open the tcp port 10250 (api service) on the photon-node in the firewall
 
 ```sh
 iptables -A INPUT -p tcp --dport 10250 -j ACCEPT
 ```
 
 

 * Install Docker on photon-node:

 
 ```sh
 tdnf install docker
 ```
 

 * Add master and node to /etc/hosts on all machines (not needed if the hostnames are already in DNS). Make sure that communication works between photon-master and photon-node by using a utility such as ping.

 
 ```sh
 echo "192.168.121.9	photon-master
 192.168.121.65	photon-node" >> /etc/hosts
 ```
 

 * Edit /etc/kubernetes/config, which will be the same on all the hosts (master and node), so that it contains the following lines:

 
 ```sh
 # Comma separated list of nodes in the etcd cluster
 KUBE_MASTER="--master=http://photon-master:8080"
 

 # logging to stderr routes it to the systemd journal

 KUBE_LOGTOSTDERR="--logtostderr=true"
 
 # journal message level, 0 is debug
 KUBE_LOG_LEVEL="--v=0"
 
 # Should this cluster be allowed to run privileged docker containers
 KUBE_ALLOW_PRIV="--allow_privileged=false"
 ```
 

 **Configure the Kubernetes services on the master**

 
 * Edit /etc/kubernetes/apiserver to appear as such.  The service_cluster_ip_range IP addresses must be an unused block of addresses, not used anywhere else.  They do not need to be routed or assigned to anything.
 
 ```sh
 # The address on the local server to listen to.
 KUBE_API_ADDRESS="--address=0.0.0.0"
 
 # Comma separated list of nodes in the etcd cluster
 KUBE_ETCD_SERVERS="--etcd_servers=http://127.0.0.1:4001"
 
 # Address range to use for services
 KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"
 

 # Add your own

 KUBE_API_ARGS=""
 ```
 
 * Start the appropriate services on master:
 
 ```sh
 for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler; do
 	systemctl restart $SERVICES
 	systemctl enable $SERVICES
 	systemctl status $SERVICES
 done
 ```
 

 * To add the other node, create the following node.json file on the Kubernetes master node:

 
 ```json
 {
     "apiVersion": "v1",
     "kind": "Node",
     "metadata": {
         "name": "photon-node",
         "labels":{ "name": "photon-node-label"}
     },
     "spec": {
         "externalID": "photon-node"
     }
 }
 ```
 

 Now create a node object internally in your Kubernetes cluster by running the following command:

 
 ```console
 $ kubectl create -f ./node.json
 
 $ kubectl get nodes
 NAME                LABELS              STATUS
 photon-node         name=photon-node-label     Unknown
 ```
 

 Please note that in the above example, it only creates a representation for the node

 _photon-node_ internally. It does not provision the actual _photon-node_. Also, it
 is assumed that _photon-node_ (as specified in `name`) can be resolved and is

 reachable from the Kubernetes master node. How to provision
 a Kubernetes node (photon-node) is shown in a later section.

 **Configure the Kubernetes services on the node**

 You configure the kubelet on the node as follows. 

 * Edit /etc/kubernetes/kubelet to appear like this:

 
 ```sh
 ###
 # Kubernetes kubelet (node) config
 
 # The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
 KUBELET_ADDRESS="--address=0.0.0.0"
 
 # You may leave this blank to use the actual hostname
 KUBELET_HOSTNAME="--hostname_override=photon-node"
 
 # location of the api-server
 KUBELET_API_SERVER="--api_servers=http://photon-master:8080"
 

 # Add your own

 #KUBELET_ARGS=""
 ```
 

 * Start the appropriate services on the node (photon-node):

 
 ```sh
 for SERVICES in kube-proxy kubelet docker; do 
     systemctl restart $SERVICES
     systemctl enable $SERVICES
     systemctl status $SERVICES 
 done
 ```
 

 * Check to make sure that the cluster can now see the photon-node on photon-master and that its status changes to _Ready_.

 
 ```console
 kubectl get nodes
 NAME                LABELS              STATUS
 photon-node          name=photon-node-label     Ready
 ```
 

 If the node status is `NotReady`, verify that the firewall rules are permissive for Kubernetes.  
 

 * Deletion of nodes: To delete _photon-node_ from your Kubernetes cluster, one should run the following on photon-master (please do not do it, it is just for information):

 
 ```sh
 kubectl delete -f ./node.json
 ```
 

 That's it. You should have a functional cluster. You can now launch a test pod. Check out [Kubernetes 101](http://kubernetes.io/docs/user-guide/walkthrough/) for an introduction to working with Kubernetes.