Virtualization Vs Abstraction (in Cloud Networking)

Warning: yes it is one of those highly philosophical posts.

I spent the last 10 months or so “playing around” the vCloud Hybrid Service (aka vCHS). And I spent time “playing around” AWS (and Azure) too, for obvious reasons.

There are a few things that are tangibly different between vCHS and “the others”. I have already argued that it’s not just about the “common tools” mantra VMware tend to often mention. It is more about the operational model, as I tried to depict in this blog post.

However, the more I am playing with the networking services of these different clouds, the more I think there is something else I can’t easily and entirely describe. The best description I can give to this feeling is that “in vCHS the network is virtualized whereas in AWS (and Azure) the network is abstracted”. Let me try to explain this.

I have been exposed (lightly) to networking and security in my career in the last 20 years or so. There are some hard core concepts such as Layer 2 networks (that I picture in my head as a line in a Visio diagram) or Layer 3-7 services (that I picture in my head as a box in a Visio diagram). That’s how I visualize data center networking.

If I look at vCHS all of those networking hard core constructs are there. And you can easily locate them inside the vCHS interface (or via APIs).

These are Layer 2 Networks in my virtual data center, for example:

And this is the Gateway that provides all Layer 3-7 services available in vCHS. These services include NAT, Firewall, Load Balancing, VPN and DHCP functionalities:

If you want another example of how you can visualize and locate these networking constructs, see this sample utility.

This isn’t rocket science. This is how it’s been working for the last 30 years in the data centers: a bunch of network cables that get wired into physical device(s) where the magic happens. The difference, 30 years later, is that those networks today are VXLAN virtual segments and those physical boxes are either vSwitches or virtual (Edge) Gateways. This is somewhat similar to the concept of virtual machines that represents physical machines.

So, in a way, vCHS provides a virtualized experience of what actually happens in a physical data center. At this point I’d like say “SDDC” but I will refrain, no worries.

That’s why, in my opinion, vCHS is very intuitive. Because you end up dealing with the same “constructs” you have been dealing with for the last 30 years. So, in a sense, it is logically the same thing. With the only difference that procuring, provisioning and cabling physical boxes used to take months in the old world, whereas here it takes seconds.

“Right click” -> “Provision a new Gateway”.

Or “Right click” -> “Provision a new Network”.

Virtual machines anyone?

Whenever I play with AWS (and Azure) I often have a hard time to relate what (little) I know about networking to how things happen there. Sure you can configure Firewall, NAT, Load Balancing, VPN services but what’s missing (most of the time) is the “object” you are setting those rules on. Sometimes those rules come out of the blue in a magic wizard that allows you to say, for example, “server 1 can connect to server 2 on port xyz” but that rule is so abstracted that you lose sight.

Another good example is Load Balancing. I have been playing around with the vCHS Load Balancing service a few weeks ago and I found it pretty easy to use and consume. This is probably because my background is traditional networking and I have done a couple of those basic configurations on an F5 in the last few years. I found the logical layout of the objects and constructs to be pretty much similar (granted that with the F5 you can do more things, obviously). You have stuff on a network, other stuff on another network, you create a pool, you create a VIP, etc etc. Fairly traditional stuff.

At the end of the day you know that the magic happens inside that Layer3-7 device you can visualize (regardless whether it is a physical F5 box or a vCHS Gateway).

As a personal exercise, I tried to create a similar load balancing configuration (as described in the blog post) with AWS.

I have to admit that it was so much abstracted from the actual implementation in the back-end infrastructure (which we obviously don’t know how it’s been done) that it was a challenge. Shame on me!

But don’t get me wrong. By no means I am trying to say that virtualization is good and abstraction is bad. As a matter of fact I have been lobbing forever to get more and more networking and security abstractions into vCD in the last few years. I also suggested ways and built prototypes to demonstrate the advantages.

This all adds up very nicely (in my head, at least). In fact I can trace this thinking back to 2011 when I presented at VMworld 2011 the following slide (that was trying to convey the message I am trying to convey here):

In this slide I described the Layer 3-7 “virtualized” vShield Edge Gateway as a legacy security model (top – left in the quadrant).

I described the “abstracted” vShield App as an innovative model (top – right in the quadrant).

And, of course, with (heavy) innovation comes disruption. And with legacy and traditional practices comes comfort. The trick is to always  find the right balance. Which is the point I am trying to make in this post.

There is no question that abstraction can help efficiency and potentially can make consumption easier (unless you carry a heavy background of traditional networking). The risk, as always, is that by abstracting innovating too much customers that have a slow technology adoption pace may lose sight of you if you run too fast (as a technology vendor or service provider).

While I am writing this long blog post, I now realize the challenge I was facing while building the AWS and vCHS parallel presentation I did at VMworld 2013.

What I have just realized is that it was uber easy to draw the vCHS networking slide because you could relate it to both how a physical data center works today and how the traffic flows between the constructs:

What I have also just realized is that the counterpart AWS networking slide was a mess (my fault, not theirs). It was a mess because I was trying to draw it with a traditional networking background where the traffic flow as well as the networking constructs are typically central in how you draw it. This is the result:

Pretty complex uh? Surely the best way to describe how the “abstracted” AWS networking and security work would be with an “abstracted” view in a slide. Something I still need to figure out how to do.

And, as I said, there isn’t a “this is good and that is bad” hidden message in this post. Different ways of doing similar things. One may appeal a group of users, the other may appeal another group of users.

Interestingly enough VMware seems to be on a similar “abstraction” trajectory with NSX. This great post from the great Brad Hedlund describes in details this transition from virtualization to abstraction. In particular I like how Brad describes the ESR (Edge Services Router) which happens to be very similar to the Edge Gateway we use in vCHS:

“The ESR is a router in a VM (it also does other L4-L7 services like FW, LB, NAT, VPN, if you want). Both the control and data plane of the ESR router are in the VM. This VM establishes routing protocol sessions with other routers and all of the traffic flows through this VM. It’s like a router, but in a VM. This should be straight forward, not requiring much explanation”. 

“This should be straight forward, not requiring much explanation”. That is it. That is its value! On top, obviously, of it being virtual and not physical!

For the records, the other option in NSX is DLR (or Distributed Logical Router) which requires a couple of pages of explanation in Brad’s blog post. Very powerful (and efficient!), no question, but it requires a steep learning curve.

Admittedly that learning curve is steeper for cloud providers implementing it than for customers consuming it but, still, the level of abstraction it provides could be mind blowing for consumers as well.

There is little doubt, in my opinion, that the future is in the abstraction approach. The virtualization of the physical constructs is just a leg of the journey towards that end-state (assuming that that is the end-state in 5+ years).

Considering that the world is still fairly much on physical constructs (Cisco is leading today that space with a 50 B$ annual revenue, give or take a few B$), picturing virtualization as legacy is a bit of a joke. There are, in fact, a few million IT people out there for which networking constructs virtualization is “the new thing”. After all, we all know the world is moving at different (IT) paces.

Perhaps the most perfect cloud is the cloud that has a switch (that users can toggle) between traditional networking and abstracted networking that would allow them to consume networking and security the way they like, depending on their (IT) adoption curve and ability to digest innovation. Or perhaps this is just a crazy idea.

Massimo.

9 comments to Virtualization Vs Abstraction (in Cloud Networking)

  • Abstraction is the way to go. You are pointing out the limitations from years of learning and understanding complex networking semantics. The network will never improve until the semantics are removed. Just virtualizing all of the network complexity is no real gain. You are just moving complexity from one platform to another or duplicating it in two places. We network people are our own worst enemies.

    • Massimo

      I agree Todd. No question.

      The problem, as I said, it’s the pace.

      I disagree that by moving from the physical to virtual there is nothing to gain if you keep the semantics.

      If you think about that, a VM is nothing more than the representation of a physical server with all the ramifications of that (sprawls to name just one). However, the operational advantages that VMs (vs physical servers) have introduced are huge. Surely the idea of having a single platform / OS where to collapse all applications is in theory a lot more appealing… but that is still fiction in 2013.

      One may also argue that the whole notion of an IP address is a legacy approach (subnets? Layer 2? Layer 3? What?).

      We will get there, but it can’t be an overnight effort. That was my point.

      Thanks for chiming in.

      • Virtualization makes sense with servers although not in many cases and I think LXC is much more interesting. I am saying that virtualizing the network makes less sense. Abstraction is better. I agree it won’t happen overnight. I do think that NFV makes sense especially with DPI type devcies (Security, ADC) that are mostly x86 appliances now anyway.

  • “Perhaps the most perfect cloud is the cloud that has a switch (that users can toggle) between traditional networking and abstracted networking that would allow them to consume networking and security the way they like, depending on their (IT) adoption curve and ability to digest innovation. Or perhaps this is just a crazy idea.”

    I think that these are layers and the layer at which the consumer wants to engage depends upon how closely each layer matches their current needs or problems. Over the longer term view, as more and more of what IT provides becomes viewed through the lens of the business purposes being served, more and more business purposes will be layered on top of simpler abstractions. Until then, consumers of cloud services will first gravitate towards solutions that allow them to model things closest to the way they do things today. Thanks for pushing these distinctions as they will help more and more customers think about what they need out of the cloud environment.

  • Paul

    Interesting article Massimo. I googled the definitions of abstraction and virtualization as I tend to use the terms interchangeably – which is ok in some contexts and valid but not in others.

    Is this next statement correct?

    It seems virtualization would be easier for most experienced users if we duplicated the exact UIs and that abstraction of a L2 networks (for example) may only have a subset of the features to meet the intent of the implementation – hence VPC at AWS versus L2 networks?

    Got me thinking. thanks!

    • Massimo

      Funny, I often use virtualization and abstraction interchangeably as well. I have probably said a million times that a VM is an abstraction of a physical server :)

      In the context of this post I have used “virtualization” to describe representation of the software instantiation of a physical object. And I have used “abstraction” to represent the function/service delivered by such object (be it physical or virtual).

      Perhaps I should have used the word “policy” and not “abstraction”? Well, hopefully people get the sense.

      This is truly work-in-progress thinking for me too.

  • Good points, Massimo. Virtualization, as described in your post, is helpful for infrastructure admins, which seems to be who vCHS is targeting right now. Abstraction, in the AWS context, is helpful for developers which fits the early target market for AWS. Developers don’t need the same operational details as an on-ramp; they need a simple way to get the same services, customized for their applications. My hope is we get beyond all these models.

    Right now, vCHS is like the Amazon shopping cart where you don’t have to go to an actual bookstore but you you still need to input shipping and billing information. AWS is more like Amazon one-click shopping where you just choose what you want to buy and all the underling steps to bill you and to ship to you is done automatically. Eventually, I think we’ll get to a world where we will be able to just declare what is needed for an application and have the infrastructure underneath automatically apply the necessary policies and create the applicable services. Using the online shopping analogy again, think about being able to declare that you want to buy a particular book, within a certain price range, with lowest cost preferred and next-day shipping as a requirement and having the purchase and the shipping done automatically with no human input beyond setting the policies and declaring the end-state.

Leave a Reply