Docker on vCHS (with 2 API Calls)

Jul 10, 2014 · 9 min read ·

In this article I am going to show how you can start using Docker on top of the VMware vCloud Hybrid Service. I am going to show you how to do that in different ways so that you can choose your own method based on the mechanisms you are more familiar with (e.g. UIs or APIs).

Docker is getting so much buzz these days that I am not going to spend time describing what it is. For reference, if you are new to Docker, you can read what is docker and docker basics. This is also a good read on common Docker misconceptions.

Most likely, if you are reading this article, you are a developer that ended up finding this blog post searching for "Docker vCHS" on Google. So let's try to make this readable to you.

vCHS Networking Setup

VMware introduced the vCHS service initially as a fit for existing vSphere customers to extend into the cloud. Because of this some of the networking configurations will appear more familiar to people with an Enterprise IT background. If you are a developer familiar with AWS you can think of the current vCHS networking layout as similar to an AWS VPC with private subnets.

If you are a developer you are likely in either one of these two situations: 1) you are consuming vCHS from the office of an Enterprise organization or 2) you are consuming vCHS from your own ADSL at home (I am exaggerating, but you get the point).

A picture is worth 1000 words:

If you are doing #1 (going through the red path) then I will assume you are part of a larger hybrid cloud setup where someone else has created a VPN between the data center you are in and the vCHS virtual data center. Long story short you will be able to spin up a Docker server (whose IP may be 192.168.109.4) and connect to it directly and transparently from your laptop (whose IP may be 192.168.0.103). No big deal.

If you are doing #2 (going through the yellow path) then you may need extra steps to expose the Docker server that you will be setting up to the Internet so that you can connect to it. In particular there is a DNAT and a SNAT rule that needs to be configured on the Edge GW:

The first rule makes sure all workloads on the 192.168.109.0 network can go out and reach the Internet.

The second rule makes sure that the public external IP of my Edge GW is mapped to my Docker VM. Note you will have to create this rule after you have created the actual Docker VM in order to know its private IP address. In this case I am doing an any:any port mapping but you can also do specific port mappings if you need or want to.

Last but not least you will need to configure the proper firewall rules to allow both all outbound traffic as well as well as inbound traffic to reach the Docker VM (if you need to SSH into it for example). Note, for simplicity, I am opening up everything inbound here for this quick test.

This is NOT a best practice. Only open the ports or the range of ports you need to open.

That is, at the high level, how you'd need to configure the networking plumbings. If you want to know more details about this check out the official vCHS Networking Guide.

Getting Docker Up and Running

In order to get Docker up and running in vCHS it is going to be as easy as deploying a CentOS template from the VMware catalog and run a few commands. In the main virtual data center view go to the Virtual Machines tab, click Add One and then select the CentOS 6.4 64bit image (6.4 is the latest CentOS image available in the VMware catalog at the time of this writing). Note that CentOS 6.4 doesn't come with the minimum kernel version suggested in the docker documentation but it works just fine for the purpose of this quick how-to-guide.

You will then be presented with this screen:

Adjust the details as you wish and connect the VM to the "routed network" that is proposed when you select the network manually. This will plumb the VM on the default 192.168.109.0 network.

Once you have done this you will need to look into the properties of the deployed VM to take note of its IP address and configure the appropriate Firewall and NAT rules (as described above). This is required if you want to reach the Docker server from the Internet (yellow path in the first picture). If you are in an Enterprise hybrid deployment it's likely that you can connect directly to the Docker VM IP as found in the vCHS portal (e.g. 192.168.109.4).

Now that you have your CentOS VM up and running you need to access it to complete the Docker setup.

You can use the Remote Console feature available in the vCHS portal to do this. This is interesting because it's completely out-of-band and, as such, it is not dependent on the in-bound Firewall and NAT rules you have configured above.

However, it may give you a hard time to login with the auto-generated password (that needs to be changed at first login) if you use an international keyboard.

I usually often suggest to SSH into the instance by pointing to the public IP you used to create the DNAT rule above. Once you type the auto-generated password (available in the VM properties in the vCHS portal) you will have to change it and you are good to go.

Now you can type these 3 commands:

1rpm -iUvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
2
3yum -y install docker-io
4
5/etc/init.d/docker start

Done.

You can test Docker started correctly by installing (for example) the Tutum WordPress package available on the Docker Hub:

1docker run -d -p 80 tutum/wordpress /run.sh

You can then run the docker ps command to find out which port the WordPress container has been mapped to:

If you connect via a web browser to the NATted IP and the port listed in the docker ps command, you should see the WordPress application user interface:

A Smarter Way to Get Docker Up and Running on vCHS

vCHS includes very powerful Guest OS customizations features that will allow you to run commands inside the guest operating system at deployment time. If you want to learn more about this please refer to this blog post (consider that blog post a must read if you want to practice for real what I am going to show below).

In this case I will take advantage of being able to run a script inside the guest when I power on the VM. Those guest customization features are only exposed in the vCD user interface for the moment so you have to use that UI to tell vCHS what script to run.

When you have deployed the CentOS VM from the VMware maintained templates, you can open its properties and click on the Guest OS customization tab. Here you can set the password policy of your choice as well as copy and paste the script to run at first power on. This is how I configured my test VM:

In this example I have configured a custom password and I didn't set the bit to force the change at first login (I am lazy). You can do what you feel it's more appropriate for you here.

Note also the script I have pasted in this screen. That's essentially the sequence of commands I ran when I did everything manually above. The Guest OS customization engine runs this script twice (before and after the actual Guest OS customization). Since we want to run those commands at the very end of the OS customization process, we structured the script to check that. See here for more details on how to customize the script with pre and post customization variables.

For your convenience this is the complete script I pasted there:

 1#!/bin/sh
 2if [ x$1 == x”precustomization” ]; then
 3echo Do Nothing
 4elif [ x$1 == x”postcustomization” ]; then
 5touch /tmp/dockersetup.txt
 6rpm -iUvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm >> /tmp/dockersetup.txt
 7yum -y install docker-io >> /tmp/dockersetup.txt
 8/etc/init.d/docker start >> /tmp/dockersetup.txt
 9docker run -d -p 80 tutum/wordpress /run.sh >> /tmp/dockersetup.txt
10fi

Once the VM has finished booting up, if you run the docker ps command or if you connect to http://23.92.224.73:49153/ you will essentially see the same results as above when you did everything manually.

The Best (and Coolest) Way to Get Docker Up and Running on vCHS

Even better, you can automate all of the above via APIs. I have discussed the basics of how you interface with vCHS through REST APIs in a couple of blog posts here and here.

For this exercise I am going to steal (and slightly adapt) the sample API call I used in the second blog post to instantiate a VM from the VMware catalog. In red the parts you may need or want to customize for your own setup.

 1Method:             POST
 2
 3URL:                 https://p3v4-vcd.vchs.vmware.com:443/api/vdc/d050eb91-d821-4121-9f1d-83615e6c0875/action/instantiateVAppTemplate
 4
 5Content-Type:   application/vnd.vmware.vcloud.instantiateVAppTemplateParams+xml
 6
 7Body:
 8
 9<?xml version=”1.0″ encoding=”UTF-8″?>
10<InstantiateVAppTemplateParams
11xmlns=”http://www.vmware.com/vcloud/v1.5″
12name=”Docker”
13deploy=”false”
14powerOn=”false”
15xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance”
16xmlns:ovf=”http://schemas.dmtf.org/ovf/envelope/1″>
17<Description>Docker vApp</Description>
18<InstantiationParams>
19<NetworkConfigSection>
20<ovf:Info>Configuration parameters for logical networks</ovf:Info>
21<NetworkConfig
22networkName=”ortles-default-routed“>
23<Configuration>
24<ParentNetwork
25href=”https://p3v4-vcd.vchs.vmware.com:443/api/network/140c5249-72e3-4dcf-853b-28c362575045” />
26<FenceMode>bridged</FenceMode>
27</Configuration>
28</NetworkConfig>
29</NetworkConfigSection>
30<LeaseSettingsSection
31type=”application/vnd.vmware.vcloud.leaseSettingsSection+xml”>
32<ovf:Info>Lease Settings</ovf:Info>
33<StorageLeaseInSeconds>172800</StorageLeaseInSeconds>
34<StorageLeaseExpiration>2010-04-11T08:08:16.438-07:00</StorageLeaseExpiration>
35</LeaseSettingsSection>
36</InstantiationParams>
37<Source
38href=”https://p3v4-vcd.vchs.vmware.com:443/api/vAppTemplate/vappTemplate-3a21102e-b8ac-455a-826b-bf6023d60f93” />
39<AllEULAsAccepted>true</AllEULAsAccepted>
40</InstantiateVAppTemplateParams>

Once we have instantiate the template we need a second API call to customize the VM.

In the next API call I am going to set a proper VM name (Docker), a proper guest OS computername (docker), connect the VM to the routed network we declared above and, more importantly, inject the customization script that will do the magic.

Note that I decided the root password to be auto-generated.

 1Method:             POST
 2
 3URL:                 https://p3v4-vcd.vchs.vmware.com:443/api/vApp/vm-1efceda0-53e2-4ec6-b5d7-2a3a228be618/action/reconfigureVm
 4
 5Content-Type:   application/vnd.vmware.vcloud.vm+xml
 6
 7Body:
 8
 9<?xml version=”1.0″ encoding=”UTF-8″?>
10<Vm
11xmlns=”http://www.vmware.com/vcloud/v1.5″
12nestedHypervisorEnabled=”false” needsCustomization=”true” deployed=”true” status=”8″ name=”Docker” id=”urn:vcloud:vm:1efceda0-53e2-4ec6-b5d7-2a3a228be618” href=   “https://p3v4-vcd.vchs.vmware.com:443/api/vApp/vm-1efceda0-53e2-4ec6-b5d7-2a3a228be618“>
13<NetworkConnectionSection
14type=”application/vnd.vmware.vcloud.networkConnectionSection+xml”
15xmlns=”http://www.vmware.com/vcloud/v1.5″
16xmlns:ovf=”http://schemas.dmtf.org/ovf/envelope/1″>
17<ovf:Info>Firewall allows access to this address.</ovf:Info>
18<PrimaryNetworkConnectionIndex>0</PrimaryNetworkConnectionIndex>
19<NetworkConnection
20network=”ortles-default-routed“>
21<NetworkConnectionIndex>0</NetworkConnectionIndex>
22<IpAddress />
23<IsConnected>true</IsConnected>
24<MACAddress>00:50:56:01:01:49</MACAddress>
25<IpAddressAllocationMode>POOL</IpAddressAllocationMode>
26</NetworkConnection>
27</NetworkConnectionSection>
28<GuestCustomizationSection
29xmlns=”http://www.vmware.com/vcloud/v1.5″
30xmlns:ovf=”http://schemas.dmtf.org/ovf/envelope/1″
31ovf:required=”false”>
32<ovf:Info>Specifies Guest OS Customization Settings</ovf:Info>
33<Enabled>true</Enabled>
34<ChangeSid>false</ChangeSid>
35<VirtualMachineId />
36<JoinDomainEnabled>false</JoinDomainEnabled>
37<UseOrgSettings>false</UseOrgSettings>
38<AdminPasswordEnabled>true</AdminPasswordEnabled>
39<AdminPasswordAuto>true</AdminPasswordAuto>
40<ResetPasswordRequired>false</ResetPasswordRequired>
41<CustomizationScript>
42#!/bin/sh
43if [ x$1 == x”precustomization” ]; then
44echo Do Nothing
45elif [ x$1 == x”postcustomization” ]; then
46touch /tmp/dockersetup.txt
47rpm -iUvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm >> /tmp/dockersetup.txt
48yum -y install docker-io >> /tmp/dockersetup.txt
49/etc/init.d/docker start >> /tmp/dockersetup.txt
50docker run -d -p 80 tutum/wordpress /run.sh >> /tmp/dockersetup.txt
51fi
52</CustomizationScript>
53<ComputerName>docker</ComputerName>
54</GuestCustomizationSection>
55</Vm>

If you now power on the VM (either via the portal or through another API call) the customization script will run and it will install Docker as well as the WordPress test application.

Massimo.