Edit This Page

CoreOS on Multinode Cluster

Use the master.yaml and node.yaml cloud-configs to provision a multi-node Kubernetes cluster.

Attention: This requires at least CoreOS version 695.0.0, which includes etcd2.

• AWS
  • Provision the Master
  • Capture the private IP address
  • Edit node.yaml
  • Provision worker nodes
• Google Compute Engine (GCE)
  • Provision the Master
  • Capture the private IP address
  • Edit node.yaml
  • Provision worker nodes
  • Establish network connectivity
• OpenStack
  • Make sure you can connect with OpenStack
  • Get a Suitable CoreOS Image
  • Create security group
  • Provision the Master
  • Provision Worker Nodes

AWS

Attention: Replace <ami_image_id> below for a suitable version of CoreOS image for AWS.

Provision the Master

aws ec2 create-security-group --group-name kubernetes --description "Kubernetes Security Group"
aws ec2 authorize-security-group-ingress --group-name kubernetes --protocol tcp --port 22 --cidr 0.0.0.0/0
aws ec2 authorize-security-group-ingress --group-name kubernetes --protocol tcp --port 80 --cidr 0.0.0.0/0
aws ec2 authorize-security-group-ingress --group-name kubernetes --source-security-group-name kubernetes

aws ec2 run-instances \
--image-id <ami_image_id> \
--key-name <keypair> \
--region us-west-2 \
--security-groups kubernetes \
--instance-type m3.medium \
--user-data file://master.yaml

Capture the private IP address

aws ec2 describe-instances --instance-id <master-instance-id>

Edit node.yaml

Edit node.yaml and replace all instances of <master-private-ip> with the private IP address of the master node.

Provision worker nodes

aws ec2 run-instances \
--count 1 \
--image-id <ami_image_id> \
--key-name <keypair> \
--region us-west-2 \
--security-groups kubernetes \
--instance-type m3.medium \
--user-data file://node.yaml

Google Compute Engine (GCE)

Attention: Replace <gce_image_id> below for a suitable version of CoreOS image for Google Compute Engine.

Provision the Master

gcloud compute instances create master \
--image-project coreos-cloud \
--image <gce_image_id> \
--boot-disk-size 200GB \
--machine-type n1-standard-1 \
--zone us-central1-a \
--metadata-from-file user-data=master.yaml

Capture the private IP address

gcloud compute instances list

Edit node.yaml

Edit node.yaml and replace all instances of <master-private-ip> with the private IP address of the master node.

Provision worker nodes

gcloud compute instances create node1 \
--image-project coreos-cloud \
--image <gce_image_id> \
--boot-disk-size 200GB \
--machine-type n1-standard-1 \
--zone us-central1-a \
--metadata-from-file user-data=node.yaml

Establish network connectivity

Next, setup an ssh tunnel to the master so you can run kubectl from your local host. In one terminal, run gcloud compute ssh master --ssh-flag="-L 8080:127.0.0.1:8080" and in a second run gcloud compute ssh master --ssh-flag="-R 8080:127.0.0.1:8080".

OpenStack

These instructions are for running on the command line. Most of this you can also do through the Horizon dashboard. These instructions were tested on the Ice House release on a Metacloud distribution of OpenStack but should be similar if not the same across other versions/distributions of OpenStack.

Make sure you can connect with OpenStack

Make sure the environment variables are set for OpenStack such as:

OS_TENANT_ID
OS_PASSWORD
OS_AUTH_URL
OS_USERNAME
OS_TENANT_NAME

Test this works with something like:

nova list

Get a Suitable CoreOS Image

You’ll need a suitable version of CoreOS image for OpenStack Once you download that, upload it to glance. An example is shown below:

glance image-create --name CoreOS723 \
--container-format bare --disk-format qcow2 \
--file coreos_production_openstack_image.img \
--is-public True

Create security group

nova secgroup-create kubernetes "Kubernetes Security Group"
nova secgroup-add-rule kubernetes tcp 22 22   0.0.0.0/0
nova secgroup-add-rule kubernetes tcp 80 80   0.0.0.0/0

Provision the Master

nova boot \
--image <image_name> \
--key-name <my_key> \
--flavor <flavor id> \
--security-group kubernetes \
--user-data files/master.yaml \
kube-master

<image_name> is the CoreOS image name. In our example we can use the image we created in the previous step and put in ‘CoreOS723’

<my_key> is the keypair name that you already generated to access the instance.

<flavor_id> is the flavor ID you use to size the instance. Run nova flavor-list to get the IDs. 3 on the system this was tested with gives the m1.large size.

The important part is to ensure you have the files/master.yml as this is what will do all the post boot configuration. This path is relevant so we are assuming in this example that you are running the nova command in a directory where there is a subdirectory called files that has the master.yml file in it. Absolute paths also work.

Next, assign it a public IP address:

nova floating-ip-list

Get an IP address that’s free and run:

nova floating-ip-associate kube-master <ip address>

where <ip address> is the IP address that was available from the nova floating-ip-list command.

Provision Worker Nodes

Edit node.yaml and replace all instances of <master-private-ip> with the private IP address of the master node. You can get this by running nova show kube-master assuming you named your instance kube master. This is not the floating IP address you just assigned it.

nova boot \
--image <image_name> \
--key-name <my_key> \
--flavor <flavor id> \
--security-group kubernetes \
--user-data files/node.yaml \
minion01

This is basically the same as the master nodes but with the node.yaml post-boot script instead of the master.