vCloud Director Networking for Dummies

During the Beta phase of vCloud Director (aka Redwood) I put together a small deck called “Redwood Networking for Dummies”. I have received a number of positive feedbacks so I decided to turn that document into a blog post. Networking in vCloud Director is certainly a controversial matter. I believe it is fair to describe it both complex and rich at the same time. There have been many attempts lately to describe it from the like of Duncan Epping and Hany Michael on their own Blogs. They have done a great job in getting into the details. However I’d like to try to give a different perspective on the subject. While I won’t be able to avoid all of the technicalities, I’d like to give you a sense of the philosophy behind what we have built into the product. Last but not least note that there are a couple of approaches to describe networking in vCloud Director. The first approach starts with the cloud end-user in mind and describes how networking works in support of certain application deployments use cases. From there you can walk all the way down to describe what happens at the vSphere platform level. The second approach starts with the vSphere administrator in mind and describes how networking works building up from the vSphere constructs, all the way to what gets exposed to the cloud end-user.

In this post I am going to use the second approach. This is not because I believe it is the right one but simply because it is the one I am more comfortable with and the one that may serve better the readers of this blog. So let’s get started.

Introduction to vCloud Director Networking

Before we get into the matter, you need to step back and think about the vCloud Director philosophy for a moment. Cloud is all about giving the end-users an unprecedented level of flexibility that allows them to do things that were only available to vSphere administrators before. In a way you can think of vCloud Director as an interface (or a proxy) into the virtual infrastructure. This allows vSphere administrators to give end-users a lot more flexibility, but at the very same time it allows them to keep full control of what end-users can do.

Achieving this level of cooperation and flexibility in the networking subsystem is no trivial task. Think about how it is difficult to implement something that allows an end-user to create, in self-service mode, separate layer 2 network segments, define custom layer 3 IP policies, configure services such as DHCP, NAT and Firewall… all without having to ask the vSphere / cloud administrator to do that for you, all without messing up with the cloud-wide setup, all without causing conflicts with the other tenants on the cloud. This is a titanic effort, believe me.

Explaining how networking in vCloud Director (vCD from now on) works is really like pealing an onion. If I was to explain it with the cloud end-user in mind I’d start from the outer part getting into the middle of it. Since I am going to explain it from the vSphere administrator point of view, I will have to start from the inner part of the onion building up the abstraction levels that the end-user will see in the end. This document will try to explain the three major networking levels within vCD. They are External Networks, Organization Networks and vApp Networks. These are in fact the type of networks you can instantiate.

Before we start discussing these three network layers we need to introduce another concept that is of paramount importance for vCloud Director operations: Network Pools. Think about it for a minute. How can we give an end-user a controlled way to deploy layer 2 networks? Layer 2 networks are usually vSphere PortGroups with an associated VLAN. How can you keep control of that? How can you let different tenants deploy these PortGroups keeping track of what’s going on in the cloud (and in turn on the vSphere layer) and, in doing so, avoiding conflicts with similar deployments in other tenants (aka Organizations)? vCloud Director solves this problem using what we call Network Pools. A Network Pool is in fact a set of layer 2 networks that the cloud administrator has declared as “available”. Think about it: in the old days when the end-user needed something like this he/she had to go to the vSphere admin which would in turns look into his/her VLAN CMDB (typically an EXCEL spreadsheet :-) ), he/she would chose an available VLAN and would create a PortGroup based on that. He/she would then connect the vNIC of this end-user’s VM to the newly created PortGroup and advise the end-user that the change was made to the VM. In a cloud self-service model it doesn’t work like this. If the end-user can deploy layer 2 networks there must be a CMDB that can be programmatically be accessed under the covers (by the end-user). Yes Network Pools are, in a way, that CMDB. More on this later.

External Networks

The vCD inner networking component is called External Networks. If you want your Organization (and in turns your vApps) to have connectivity to the external world you need to have External Networks. As the word implies, these are networks that are managed by someone that is typically external to the vCD environment and are identified by a vSphere PortGroup. That’s in fact what you do when you create a vCD External Network: you point to an existing vSphere PortGroup. Essentially you are telling vCloud Director that there is a PortGroup that is able to provide external connectivity to your cloud environment. The typical example is a PortGroup with VLAN 233 (for instance) which can support native Internet traffic. For naming convention you will be calling this External Network something like Internet or Ext-Net-Internet. I usually suggest to name the vCD External Network after the vSphere PortGroup for ease of tracking. This is a picture that shows what it is. It’s easy:

One of the most confusing points about the creation of the External Network is that the wizard asks for some layer 3 configuration parameters. In particular the wizard asks for a subnet mask, a default gateway, a DNS address and a pool of IP addresses. What are these parameters for? Well, remember that we said External Networks are networks that are built and maintained by an external entity; we are just “registering” these networks into vCD. What we are doing while filling this wizard is essentially telling vCD what layer 3 information to use when VMs will be connected to this network directly. In particular the IP pool that you need to configure is a pool of IP addresses that vCD will use to distribute IP addresses (and related layer 3 info) to VMs connecting to this network. So how do you fill that pool? You have to turn to the folks that administer that specific network segment and you need to ask them something like “can you reserve me a set of IP addresses that no one else will be using on your network and that I can dedicate to the vApps I will be instantiating on vCD?”. In other words, how would you be able to instantiate vApps directly onto that network if you don’t know which IP address to use? That’s what that Static IP pool is. This doesn’t tell the whole story on how VMs get their IPs when deployed. For this you need to be patient. We will get there.

Organization Networks

External Networks are easy. With Organization Networks things start to become more “interesting”. In the previous section we have created cloud-wide external connectivity (i.e. External Networks). Now we are zooming inside an Organization. An Organization (or Org) is a logical construct within vCD that describes a tenant or a customer. Cloud end-users are defined inside each Organization. Each tenant can have three type of networks configured as you can depict from the picture below (you may not immediately get some of the acronyms and colored labels – no worries – it will be more clear later):

The first network is called External Organization Network (Direct Connect) and it’s the simplest way to connect to the external world through an External Network. In this case nothing happens at the vSphere layer, this type of Org Network is a logical construct created inside vCD but doesn’t really have any counterpart in the vSphere world: if you connect a vApp to this Org Network, the vNIC gets configured to connect to the Internet PortGroup (VLAN 233) in the example above. Not a big deal.

The second network is called External Organization Network (NAT / Routed). This network really represents a dedicated layer 2 segment that has its own private IP schema that the Organization can chose arbitrarily (for example 192.168.x.x). This private network is then routed to the External Network I have chosen to route to. Note that in this case you are still asked for those layer 3 IP information, however this time you can create them based on your specific needs because this segment is private and its layer 3 info are not going to overlap nor to be shared with anything else anyway. So how is this implemented at the vSphere layer? When you create such a network a good learning exercise is to switch to the vSphere client interface. There you will see a number of things happening: first a new PortGroup is deployed on-the-fly; this is the layer 2 dedicated segment that will support my Org Network. Consequently a new vShield Edge appliance is automatically deployed by vShield Manager. This appliance is effectively the routing device connecting your dedicated layer 2 network to the External Network (see picture). The Edge device is then configured with the appropriate layer 3 info you have filled in the wizard when creating this Organization Network. The Edge license provided with vCD supports NAT, Firewall and DHCP functionalities to protect and serve this dedicated layer 2 segment. At this point you may wonder how vCD and vSphere can “deploy a new PortGroup on the fly” to back this dedicated layer 2 network we need to create. They come from the Network Pools we have briefly mentioned above. When creating this network in fact the wizard will ask you for the layer 3 private schema as well as the Network Pool where to grab an available layer 2 network (think of it as an available VLAN for the moment).

The third network that it is possible to created within an Organization is called Internal Organization Network. As the name implies this network is only available internally to the Organization. vApps that are deployed to connect to this network cannot go outside through the External Network. In fact this type of network is similar to the External Organization Network (NAT / Routed) with the only exception that it doesn’t connect to the external world. At this point you may wonder why there is an Edge deployed onto that PortGroup since there is no need to do routing. Remember that Edge also provides DHCP service to that segment so that’s why Edge is optionally used if the Organization Administrator decides to enable DHCP on that segment (note DHCP is disabled by default).

So, in summary, this is how you can connect your VM vNIC:

Note that, for simplicity, the picture shows a VM that can connect to different Organization Networks. Most of the time VMs will have only one vNIC connected to either one of the Org Networks. However it is possible for a VM to have two or more vNICs. Also consider that vCD treats everything as a vApp. A single VM is in fact a vApp with one VM in it. Sometimes you will be using more VMs in a single vApp. Which brings us to the third type of network.

vApp Networks

So far we have seen cloud-wide networks (aka External Networks), Organization-wise networks (aka Organization Networks) and now we are going to investigate what we call vApp Networks which are, guess what, networks that are only available within a single vApp. This is something that you may want to do to either create and support secure n-tier applications deployments or to fence a vApp to an Organization Network. Fencing a vApp allows you to instantiate many times the same vApp onto an Organization Network preserving layer 2 and layer 3 information. In a way, fencing is a shortcut given to end-users in the vCD user interface to achieve transparently this cloning operation. From a vSphere perspective creating a vApp Network explicitly or taking the “fence shortcut” in the UI translates into the deployment of Edge devices as well as separate layer 2 networks from a vCD pre-defined Network Pool. Note I am oversimplifying a matter that is more complex than what I am trying to picture. That’s because, right now, I am focusing more on what happens at the vSphere layer rather than focusing on the different end-user options vApp Networks and Fenced vApps have to offer.

While the configuration wizards may seem to be slightly different, note that the relationship between vApp Networks and Organization Networks is somewhat similar compared to the relationship between Organization Networks and External Networks. By this I mean that vApp Networks can connect directly to an Org Network (in which case the VM connects to the Organization Network PortGroup), the vApp Network can connect using NAT technologies to the Org Network (in which case a new layer 2 network is being deployed from a specified Network Pool and a new Edge is instantiated to connect to the Organization Network) or the vApp Network can be left isolated from the rest of the world (in which case a new layer 2 network is being deployed from a specified Network Pool and a new Edge is instantiated only if DHCP gets enabled). This sounds familiar if you think at the different Organization Network options. As a matter of fact we are effectively creating a similar stack at the vApp level and we could then plug this stack on top of the other stacks we created at the Org level. You remember the onion?

The picture below shows a VM that connects to a vApp Network where DHCP was enabled (note the presence of the Edge device):

This picture below, on the other hand, shows a VM connected to a vApp Network with external connectivity. Don’t be confused by this picture: an Edge can Route/NAT to one and only one network at any point in time. In fact the Edge system vm always have a maximum of two vNICs: one that connects to the network to be protected and the other one connected to the network it needs to route/NAT too. The picture below shows all the possible configurations for the second Edge vNIC: External Org Net (Direct Connect), External Org Net (NAT/Routed) and Internal Org Net. Again: only one of these three connections can be active at any point in time. Note how a VM can be potentially NATted twice if connected to a NATted vApp Networks which in turns connect to a NATted Organization Network.

It may be interesting to call out that there are a few philosophical differences between how you create, configure and deploy vApp Networks compared to Organization Networks. Org Networks are created by the cloud administrator (on behalf of the Organization administrator) and when the cloud admin starts the creation process the wizard asks interactively for “which Network Pool to use to grab an available layer 2 segment”. We do not want to expose that question to the end-user when he/she creates a vApp Network. After all this end-user may not even have a clue what a Network Pool is and perhaps it may not even know what a layer 2 network is. To overcome this we associate a network pool to the Organization vDC. In this case when a user creates a vApp Network a layer 2 network is grabbed from the Network Pool associated to the Organization vDC the user is deploying the vApp to. This also comes handy to keep control and keep track of layer 2 network usage. When you associated a Network Pool to an Organization vDC you can set a limit on the number of segments any user in that organization can grab. In fact if you associate a Network Pool with 100 networks in it, you don’t want someone creating 100 vApp Networks in half a day and consume the entire Network Pool immediately. This is helpful to set limits on what an Organization can do (and possibly charge accordingly).

I am not going to cover use cases of where and how to use combinations of vApp and Org Networks to create secure deployments because in this post I wanted to give you more the sense of what happens from a vSphere and cloud administrator perspective rather than from a cloud end-user perspective.

Network Pools

At this point you may have an overall understanding of what a Network Pool is and why it is used. In summary it is a small CMDB that contains layer 2 segments available to vCD administrators and end-users. Note Network Pools need to be created before we start deploying the actual networks we have described above (with the exception of the External Networks because they don’t use Networks Pools).

So far we kept referring to a “layer 2 segment” as a PortGroup with an associated VLAN id. This is correct but it doesn’t tell the whole story. There are really three different type of Network Pools one can create.

VLAN-backed Network Pools: this is the easiest to get. You can, for example, create a Network Pool and give it a range of VLAN ID 100 to 199. Whenever you grab one of these IDs because you need to deploy a new layer 2 segment, vCD will tell vCenter “please create on the fly a PortGroup, and give it VLAN ID 100″. The next time there is a need for another layer 2 segment vCD will tell vCenter “please create on the fly a PortGroup, and give it VLAN ID 101″. And so on. Of course if one of these networks is destroyed during the lifecycle of the cloud, the corresponding VLAN ID gets put back into the pool of available networks to be deployed.

PortGroup-backed Network Pools: it is similar to the VLAN-backed. The difference is that the PortGroups need to be pre-provisioned on the vSphere infrastructure and they need to be imported into vCloud Director. So vCD won’t tell vCenter to create these on the fly, they are already there pre-provisioned. Why using this? Well there are some circumstances where vCenter cannot easily (programmatically) create PortGroups on the fly. This is the case when you use vSphere Standard Switches (as opposed to Distributed Switches) or when you use the Nexus 1000v (at the moment vCD cannot manipulate programmatically Port Profiles).

vCloud Director Network Isolation Network Pools: This is when things start to get interesting (again). We use a technique called Mac-in-Mac to create layer 2 separated networks without using VLANs. Yeah that’s right. This is extremely useful for big environments where VLAN management is problematic, either because there is a limited number of VLANs available or because keeping track of VLANs is a big management overhead (especially if you use an excel spreadsheet to do that :-) ). When you create such a Network Pool you only specify how many of these layer 2 networks you want this Network Pool to have and you are done. When vCD starts to deploy PortGroups from this Network Pool you won’t see any VLAN associated to them but they are indeed different layer 2 segments.

Now the acronym VCD-NI  and the labels Preprovisioned and Created-on-the-fly in the pictures above should make more sense to you. Try to go back and have a look at them again.

Virtual Machines IP management

First of all note you cannot connect a vNIC to an External Network directly. You can however connect the vNIC to either an Organization Network or a vApp Network.

Now the question is: what happens when you connect a vNIC to either an Organization Network or a vApp Network? How do you control the layer 3 behavior? As we said, you have a choice of connecting each vNIC of the VM to an Organization Network, a vApp Network or leave the vNIC not connected. In the example below I have connected it to a vApp Network as you can depict from the name (vAppInternal). If you chose to connect it to a network you have three choices on how to get an IP. See the “IP Mode” drop-down in the picture:

Static IP Pool: this is the pool of IP addresses that you have configured when you created the network you are connecting to. This is the private IP Pool range you had to configure when creating a vApp Network, an External Organization Network Routed/NAT or an Internal Organization Network. In case of an External Organization Network Direct Connect the IP Pool range configured when creating the External Network it connects to will be used. It is important to understand that from a VM perspective this is considered a Static IP Address, it only happens to come from a pool that vCD controls. The first IP available in the Static IP Pool gets “plugged” into the VM (as a static address) at Guest Customization time.

DHCP: I guess this is self-explanatory. In that case the vNIC will search for a DHCP lease on the network it connects to. If it’s a vApp Network, an External Organization Network Routed/NAT or an Internal Organization Network, this will have to come from the Edge DHCP service. If it’s an External Organization Network Direct Connect it will have to be a DHCP that is available on that PortGroup associated to the External Network (in which case this would be out of the scope of vCloud Director).

Static Manual: This is used in those situations where you do not want or cannot use either one of the two above. You have to manually enter the IP address into the vCD interface and make sure it is the same you have entered into the Guest OS of the VM you are working on. It goes without saying that this manual IP address cannot fall into the same range of the DHCP scope nor the Static IP Pool if you want to avoid potential IP conflicts.

Conclusions

In conclusion I hope I managed to give you a different perspective on how vCD networking works and especially the logic behind it. I covered the three main network layers and I have then focused a bit on the concept of the Network Pools and how Virtual Machines can be configured to connect to the available networks inside the Organization (including vApp Networks). Remember that complexity, in this case, is directly proportional to the richness of configurations and options available to the cloud end-user to consume “self-service”.

Massimo.

59 comments to vCloud Director Networking for Dummies

  • [...] The networking part of the presentation I’ve mentioned above has been re-written by the master at this link. Another MUST-READ. Posted in Diagrams, vCloud Director | Tags: diagrams, vCloud Director, [...]

  • Naresh

    your presentation is great and detailed. I was trying to setup the cloud and i am stuck at a point where i am not able to allocate network pool to a Organization vDC, even though there are networks available in the Network Pools at the cloud level.

    On the screen in the Org vDC propertiesm where you can select network pool, it is none and doesn’t list the networks that were created at the cloud level.

    Any help is appreciated.

    • Massimo

      Hi Naresh. The only thing I can think of is that you created Network Pools off a vDS that is not available in the cluster where your Oragnization vDC (hence the Provider vDC) originates from. This may be the case if you have two vCenter servers driving different clusters and you created a Network Pools leveraging a switch on the first vCenter but the RP backing the Org vDC comes from a different vCenter.
      HTH. Massimo.

  • sanjai

    Thanks for your effort in explaining the vCD Network concepts.

    I still can not differentiate between Static IP Pool and DHCP as both looks same in Virtual Machines IP management.

    Please shed some light …

    • Massimo

      Hi Sanjai. From a VM IP stack perspective using an IP from the static IP Pool turns into a static IP address at the VM level (vCD will chose it from the Pool). If you select DHCP you are basically setting the VM to get a dynamic address on the Network. If the vNIC connects to an External Org Network (Direct Connect) the DHCP service needs to be available on the External Network. If the vNIC connect to an External Org Network (NAT/Routed) or to an Internal Org Network then you need to turn on the DHCP service at the Edge device (or have a DHCP server on the same network as a vApp for that matter).

      HTH. Massimo.

  • Naresh

    Thanks Massimo , You helped me think differently and could resolve the issue and proceed further. I didn’t had two vcenter server but i had to reconfigure my Vsphere environemnt and create a new cluster. I think the issue was that the resources were not available to all the ESX hosts. Any way thanks for the help i have to overcome some more issues in setting this monster up :)

  • Great post Massimo! Your style is perfect for spoon feeding. I came back several times.
    I have a pretty good understanding of vCD networking now, thanks to you.

    Cheers,
    Brad

  • [...] – Networking part 1 – Intro Creating a VMware vCloud Director Cluster vCloud Director Architecture vCloud Director Networking for Dummies Automating vCloud Director and Oracle DB [...]

  • Eric

    Hi all

    I’m a vShield beginner and I need some help.

    For 2 given customers using an overlapping IP subnet to assign their VMs, is it possible to configure 2 Vshield zones based on the same IP Subnet via VShield Manager ?

    How is the isolation performed between the 2 customers.

    Best Regards.
    Eric

    • Massimo

      Eric,

      this is not supported today. vShield Zones/App wouldn’t work with overlapping IP schemas across different tenants. This is something we are looking at for future versions.

      Massimo.

  • [...] Networking part 1 – IntroCreating a VMware vCloud Director ClustervCloud Director ArchitecturevCloud Director Networking for DummiesAutomating vCloud Director and Oracle DB InstallationDiagram: The VMware vCloud Director Cell [...]

  • wrong- sorry

    this is all very wrong sorry , there is no single ‘edge’ device like in your pictures, vshield-edge device (VM) is created for every port-group that use it , meaning multiple ‘edge’ devices for the same ORG (per port-group) , also ‘isolated’ to define VCDNI is wrong , look for lab-manager encapsulation to see why it can not be isolated. ‘internal networks’ are not internal at all , they are exactly the same as ‘external direct’ , just a vlan on a portgroup sent to external switch when moving between ESX servers. you are making it difficult when it is actually very clear ;-)

    • Massimo

      I have approved the above comment but to be honest I am not sure whether it’s spam or what. I am not sure what you are talking about. Who ever said you can only have a “single” edge per Org? Also “Isolated” may have different meanings to a lot of different people. For someone not even a dedicated port on a shared switch is “isolated” (go figure VLANs or MAC-in-Mac). I am glad it’s so “clear” in your mind… however stating that an Internal network and External Direct networks are exactly the same… makes me wonder…

      Massimo.

  • wrong- sorry

    also – when you use vshiled edge in vcloud , you are forced to NAT your servers , VSE is NOT a router, so all NAT caveat apply here.
    also – VSE as a FW ? it is not even certified as a fire-wall so why call it so ? look for the capabilities of VSE, compare it to a 50$ cheep FW and tell who is better in terms of functionality and features ;-)

    • Massimo

      The features will improve for sure down the road but the value is not in the features themselves. It’s rather in the integration of all of them (CPU/MEMORY/DISK/NETWORK) into something that an end-user can easily consume from a portal or from an API. Perhaps your 50$ supermarket firewall will have more features than the Edge today … but it doesn’t fit into the picture above.

      Massimo.

  • covertp

    As an example of the information that you will need to fill out for an “External Network”–is this public IP info given to you by your ISP?

    • Massimo

      It depends on the context. If you are an Enterprise deploying a Private Cloud and you want to give your Orgs access to the Internet (via the Edge security gateway) then yes those would be the public IPs your internet provider gave you (unless you have some sort of additional NATting at the perimiter of your physical network in which case it would be the NATted addresses that your perimeter physical device is exposing internally). Note that the External Network could also be an internal company-wide backbone where you want to attach Orgs and keep them protected (from other internal workloads) via the same Edge device.

  • Alex

    Hi, very helpful article. I still have a question about the network pools. Am I to assume that for any of the 3 types, the physical network switches on the network need to be configured for the relevant VLANs?

    I have tried both VLAN backed, and Cloud Isolated pools, and as soon as I put 2 VMs in the same org network on different ESX hosts, I lose connectivity…

    • Massimo

      Alex, for the VLAN backed and the PG-backed yes the VLANs need to be defined/preprovisioned on the switch ports (basically you need to create a trunk). For the VCD-NI pool it depends if you specify a VLAN ID for the pool. If you do, then yes that VLAN need to be defined on the ports.

      Hope this helps.

      Massimo.

  • Alex

    Hi Massimo,

    Thanks for your response, that’s very helpful. I’m still having connectivity problems, but I can’t tell whether it’s just me misunderstanding the product. I have a VCD-NI pool configured for 50 networks, without a VLAN ID configured. The port group gets created on the dvswitch, but VMs on that internal org network cannot see each other unless they are residing on the same ESX host. Do I need a VLAN configured so that the port group on the dvswitch can be shared amongst the ESX hosts?

    Thanks!

  • Boris

    Wow, thanks a lot!
    Helped me understand the inner workings a bit more.

    Thanks you!
    What about some post about how to set up properly organizations from scratch (including all the PortGroup and Network Pool configured)? It would be really helpful, I suppose.

    Thanks a lot!

  • here is a suggestion guys : build a common 2-tier ‘VDC’ or any common ‘VDC’ , try to figure out how many VMs was created to be used as ‘gateway’ inside ESX hosts. then figure out your multiple layer 2 ‘hops’ and layer 3 ‘hops’ in your hosts connected to each other. try to look for HA scenarios and SLA requirements and scale requirements etc … then give this implementation to your trusted networking guy to comment on … in my case it turned out to be just a picture on the wall, not real production implementation for my customers

    • Massimo

      Thanks “Miki” (or is that Koren again under a different name?).

      We owe you a beer for spending so much time on our products and providing valuable feedbacks to make them better.

      Massimo.

  • Rajesh

    Excellent article Massimo,
    I really liked the way you explained all the networking layers with excellent screenshots.
    Hope to see more of them coming in future on cloud.

  • Jeff

    Thank you this was extremely helpful.

  • Hoa

    Thanks so much for your post. It’s really helpful. I will also need to read it again and again to understand vCD networking.

  • Excellent, you provided a clean simplified picture of how networking is configured in vCD. I appreciate the fact that you kept it simple, it makes it much easier to digest. Thank you!

  • Aryan

    Hi massimo
    I have three esxis. my vCD is running in 192.168.0.33 my vcenter is running in 192.168.0.31 and i have some vms running in in 192.168.0.32. I have two problems, first problem is that, after i attach my first vcenter, i can see that there is written “syncing inventory” continuesly under vcenter in (Monitor &…) web. And 2nd problem is at It cannot find the resource pool which is in host 192.168.0.32. Thanks in advance.

  • coolguy

    To use vShield Edge’s DHCP service, Do I need to place vShield Edge device and vm in the same Host (ESX server)?

    • Massimo

      No you don’t. The Edge device is connected to a network (VLAN or VCDNI) that spans hosts so Edge and the VMs it serves can be hosted on different ESXi servers.

  • coolguy

    Thanks for quick reply, Massimo. My vm can’t get the ip from Edge device if my vm and Edge device are on the different Host. My vm can get the ip when they are on the same host. May I know, why?

    • Massimo

      This may happen if the trunking at the ESXi / physical network layer isn’t setup correctly. if your VM and Edge (which is a VM in the end) are connected to an Org Network that is backed by a specific VLAN but that VLAN isn’t trunked on all hosts appropriately they may not be able to talk to each others. The easiest way to check this is to bind quickly two manuall created vSphere VMs on the same PortGroup that vCD created (and that is backing that Org Network), put the VMs on different hosts and see if they are able to ping each others (over layer 2 so this assumes same IP schema for the two test VMs). If they can’t talk then it’s a VLAN problem. If it’s not a VLAN problem I’d open an incident so that GSS can validate your config.

      Massimo.

  • coolguy

    Thanks you so much for your information. Really appreciate it.
    I’ll check it on Thursday and reply back.

  • coolguy

    I think something wrong with my configuration. I did what you said, those two vms can ping each other. I have three networks inside my vcloud. One is Directly connect with external and the rest are routed network. If I choose Directly connect with external, my vm can get DHCP from external dhcp server even different host. If I choose the rest interfaces which are routed network, I can not get IP from vshield edge device. Thanks for your time.

    • Massimo

      Stupid question but … is the DHCP service enabled on the Edge? Point to the Routed Network, right click “Configure Services” and DHCP tab. If it’s enabled it should deliver IP addresses from the pool configured on the secured network behind the Edge.

  • coolguy

    yes, it is enable. I want to ask you one question. Do I need to install vshield vApps? I haven’t installed the vApps yet.

  • Very useful article Massimo, thx. This is my first foray into the complexities of vCD but it looks like it would solve my problem. I’m actually after the vApp isolation functionality more than the self service. Could be mimicked with a ‘norma’l vDS and vShield Edge I.e. no vCD?

    • Massimo

      Hi Ed. Thanks. It depends on what you need. If you are looking for a way to provide security zones backed by a security device loaded in a software instance (Edge) yes you can do that without vCD. vShield Edge is a standalone product (that vCD leverages to create what I described in the article and adds on top additional abstractions). One thing to notice is that the VLAN virtualization technology I mentioned in the article (VCDNI / VCNI in the pictures) is only available via vCD.

  • Godfrey

    Goodmorning and thanks for this wonderful website, like I said before I’m new to vCD, I will like to ask the following question

    vCD I know provides Infrastructire as a service, can it be used for the following:

    1) Provision windows workstations for hundreds of end users for day to day work working at home or in the office
    2) If users are working from home what will be the best practice way to connect i.e via VPN etc
    3) Can vmware view work along side vCD for Desktop virtualisation

    • Massimo

      As of today View doesn’t integrate well with vCD. Both products (vCD and View) talk to vSphere but they are sort of parallel. With vCD you can indeed instantiate desktop OSes and you can configure VPN access but I doubt that vCD alone could fulfill the requirements you have.

      • Godfrey

        Massimo, thanks for your prompt reply it makes thinks clearer now, I will like to ask another question, I have read all your labs as well as vCD admin guide and vCAT 2.0 downloaded from vmware, I gave a lot of my time to the Network aspect of vCD as I believe it to be most crucial, I’m going to be setting up my private lab this weekend, here goes my question.
        In my lab I have vDS and 3 ESXi 5.0 HA/DRS Clusters with 2 port groups for External connectivity to the outside world and the other port group for my Local LAN, Microsoft TMG is used for proxing and firewall both outwards and inwards. My internal LAN is on a 192.168.2.0/24 network, using vCD I felt my best plan of action will be to use the PORT GROUP BACKED NETWORK POOL, when I create a Network Pool in vCD considering internally in my vsphere environment my LAN address is 192.168.2.0/24 Network, will my vCD Nework pool address reserved have to be using the same Network as my internal address or can I say create a pool from say 192.168.3.0/24 and if yes what will be the default gateway and will it be possible to have communication between both environments

        • Massimo

          Not sure I am following you here. A diagram would be best to describe what you would like to achieve. One thing to note though is that a vCD “Network Pool” is really a collection of layer 2 networks (either created using vCDNI technology or regular VLANs) that you declare. For example when you create a VLAN-backed Network Pool you are essentially telling vCD that your vDS switch (and ESXi hosts) have specific VLANs available to use (eg 3000 to 3100). Only afterwards when you create Org Networks or vApp Networks these VLANs get used to create a PG and you define a subnet. In essence a Network Pool has nothing to do with a class of IPs.

          HTH

          Massimo.

  • Prasad

    IT gave our team one subnet with 254 IP’s
    I am trying to POC vcloud, but I do not have VLAN’s created in my switch and almost all the ports are allocated. I cannot create VLAN’s now.
    I created external network with VLAN as none and it got created.
    I am getting error when trying to create sphere port group-backed network pool, error is: “Port group “dvPortGroup2″ has a conflicting VLAN with another port group that is currently being used ”
    I also tried to check the option in administration–> general
    Allow Overlapping External networks.
    MY BIG QUESTION IS,
    Can we create vcloud infrastructure without VLANs ?
    This is just POC, IT takes 8 to 12 weeks to get our team a new subnet with VLAN’s
    Please help.

  • Jonathan

    Massimo
    Very useful, cleared up a lot of concepts for me.

    I am using VCD 1.5

    How much has changed in your blog post, since this posting was written in 2010?

    • Massimo

      Jonathan, this “basic part” hasn’t changed dramatically in vCD 1.5. What’s changed is the services on top of it. For example we added self service VPN in the Edge configuration (not available with vCD 1.0). We have also added the possibility to do Static Routing on top of NAT for the Edge devices (in vCD 1.0 NATting was the only options).

      Massimo.

  • Ben

    This all makes sense but i am wondering if a IP conflict can happen at the external vApp IP level, when i create a template from my vapp the new vApps i deploy from this template “Cannot Start” the app will not fire up. i’ve tried disabeling the nics and building a new template from that but it still has the same outcome.

    Ben

    • Massimo

      Hi Ben. If you make an identical copy the assumption is that you won’t do guest OS customization ie you are retaining the VM personality (guest OS names, IP address etc). If that’s what you are doing you need to “fence” the vApp (which is essentially the automatic process of fronting that vApp with an Edge device to retain the IP/personality and NAT it to the shared Org Network to avoid conflicts).

      You could also flag Guest Customization for this newly created VM, in that case it will get a new IP from the pool (and its personality will change). If that’s what you want to do I suggest you don’t put it into the catalog with “make identical copy”. That way it will go into the catalog and at check out it will automatically get re-personalized.

      Helps?

  • George

    Thank you for the insightful article Massimo,

    I wonder if you would recommend the vCIM on top of vCD for easier administration?

    I found this guide very helpful and I’m in the process of setting up my PoC environment based on

    http://www.vmware.com/pdf/vCloudIntegrationManager10_Cloud_Guidelines.pdf

    Are you familiar with this guide, and is there any discussion forum for a couple questions i had?

    Thanks and please keep up the great work.

  • [...] things. Also on the Trainsignal video David mentions a very thorough walk through by Massimo called vCloud Networking for Dummies over on it20.info – highly [...]

  • Marcus

    Hi, I am currently setting up a vcloud director 5.5 test environment to get a feel of things. Because we are going to replace Lab Manager with VCD, I am trying to setup networking so that the IPs of VMs within a vApp are same as production VMs (VMs outside of Cloud). So lets my production VMs are on vlan100, I require my VMs in the vApp to be on vlan100 aswell with the same IPs. To achieve this I have created a PortGroup-backed network pool and also created a Org VCD network with ip range in vlan100. So now when I deploy a vApp, which contains my Domain Controller, I lose connection to my Domain Controller outside of VCD environment. Its effectively like two machine with same IP. How can I achieve IPs? With Lab Manager it was quite simple and it worked.
    Thanks for your assistance. Marcus

Leave a Reply