Virtualization Vs Abstraction (in Cloud Networking)

Warning: yes it is one of those highly philosophical posts.

I spent the last 10 months or so "playing around" the vCloud Hybrid Service (aka vCHS). And I spent time "playing around" AWS (and Azure) too, for obvious reasons.

There are a few things that are tangibly different between vCHS and "the others". I have already argued that it's not just about the "common tools" mantra VMware tend to often mention. It is more about the operational model, as I tried to depict in this blog post.

However, the more I am playing with the networking services of these different clouds, the more I think there is something else I can't easily and entirely describe. The best description I can give to this feeling is that "in vCHS the network is virtualized whereas in AWS (and Azure) the network is abstracted". Let me try to explain this.

I have been exposed (lightly) to networking and security in my career in the last 20 years or so. There are some hard core concepts such as Layer 2 networks (that I picture in my head as a line in a Visio diagram) or Layer 3-7 services (that I picture in my head as a box in a Visio diagram). That's how I visualize data center networking.

If I look at vCHS all of those networking hard core constructs are there. And you can easily locate them inside the vCHS interface (or via APIs).

These are Layer 2 Networks in my virtual data center, for example:

And this is the Gateway that provides all Layer 3-7 services available in vCHS. These services include NAT, Firewall, Load Balancing, VPN and DHCP functionalities:

If you want another example of how you can visualize and locate these networking constructs, see this sample utility.

This isn't rocket science. This is how it's been working for the last 30 years in the data centers: a bunch of network cables that get wired into physical device(s) where the magic happens. The difference, 30 years later, is that those networks today are VXLAN virtual segments and those physical boxes are either vSwitches or virtual (Edge) Gateways. This is somewhat similar to the concept of virtual machines that represents physical machines.

So, in a way, vCHS provides a virtualized experience of what actually happens in a physical data center. At this point I'd like say "SDDC" but I will refrain, no worries.

That's why, in my opinion, vCHS is very intuitive. Because you end up dealing with the same "constructs" you have been dealing with for the last 30 years. So, in a sense, it is logically the same thing. With the only difference that procuring, provisioning and cabling physical boxes used to take months in the old world, whereas here it takes seconds.

"Right click" -> "Provision a new Gateway".

Or "Right click" -> "Provision a new Network".

Virtual machines anyone?

Whenever I play with AWS (and Azure) I often have a hard time to relate what (little) I know about networking to how things happen there. Sure you can configure Firewall, NAT, Load Balancing, VPN services but what's missing (most of the time) is the "object" you are setting those rules on. Sometimes those rules come out of the blue in a magic wizard that allows you to say, for example, "server 1 can connect to server 2 on port xyz" but that rule is so abstracted that you lose sight.

Another good example is Load Balancing. I have been playing around with the vCHS Load Balancing service a few weeks ago and I found it pretty easy to use and consume. This is probably because my background is traditional networking and I have done a couple of those basic configurations on an F5 in the last few years. I found the logical layout of the objects and constructs to be pretty much similar (granted that with the F5 you can do more things, obviously). You have stuff on a network, other stuff on another network, you create a pool, you create a VIP, etc etc. Fairly traditional stuff.

At the end of the day you know that the magic happens inside that Layer3-7 device you can visualize (regardless whether it is a physical F5 box or a vCHS Gateway).

As a personal exercise, I tried to create a similar load balancing configuration (as described in the blog post) with AWS.

I have to admit that it was so much abstracted from the actual implementation in the back-end infrastructure (which we obviously don't know how it's been done) that it was a challenge. Shame on me!

But don't get me wrong. By no means I am trying to say that virtualization is good and abstraction is bad. As a matter of fact I have been lobbing forever to get more and more networking and security abstractions into vCD in the last few years. I also suggested ways and built prototypes to demonstrate the advantages.

This all adds up very nicely (in my head, at least). In fact I can trace this thinking back to 2011 when I presented at VMworld 2011 the following slide (that was trying to convey the message I am trying to convey here):

In this slide I described the Layer 3-7 "virtualized" vShield Edge Gateway as a legacy security model (top - left in the quadrant).

I described the "abstracted" vShield App as an innovative model (top - right in the quadrant).

And, of course, with (heavy) innovation comes disruption. And with legacy and traditional practices comes comfort. The trick is to always find the right balance. Which is the point I am trying to make in this post.

There is no question that abstraction can help efficiency and potentially can make consumption easier (unless you carry a heavy background of traditional networking). The risk, as always, is that by abstracting innovating too much customers that have a slow technology adoption pace may lose sight of you if you run too fast (as a technology vendor or service provider).

While I am writing this long blog post, I now realize the challenge I was facing while building the AWS and vCHS parallel presentation I did at VMworld 2013.

What I have just realized is that it was uber easy to draw the vCHS networking slide because you could relate it to both how a physical data center works today and how the traffic flows between the constructs:

What I have also just realized is that the counterpart AWS networking slide was a mess (my fault, not theirs). It was a mess because I was trying to draw it with a traditional networking background where the traffic flow as well as the networking constructs are typically central in how you draw it. This is the result:

Pretty complex uh? Surely the best way to describe how the "abstracted" AWS networking and security work would be with an "abstracted" view in a slide. Something I still need to figure out how to do.

And, as I said, there isn't a "this is good and that is bad" hidden message in this post. Different ways of doing similar things. One may appeal a group of users, the other may appeal another group of users.

Interestingly enough VMware seems to be on a similar "abstraction" trajectory with NSX. This great post from the great Brad Hedlund describes in details this transition from virtualization to abstraction. In particular I like how Brad describes the ESR (Edge Services Router) which happens to be very similar to the Edge Gateway we use in vCHS:

"The ESR is a router in a VM (it also does other L4-L7 services like FW, LB, NAT, VPN, if you want). Both the control and data plane of the ESR router are in the VM. This VM establishes routing protocol sessions with other routers and all of the traffic flows through this VM. It’s like a router, but in a VM. This should be straight forward, not requiring much explanation".

"This should be straight forward, not requiring much explanation". That is it. That is its value! On top, obviously, of it being virtual and not physical!

For the records, the other option in NSX is DLR (or Distributed Logical Router) which requires a couple of pages of explanation in Brad's blog post. Very powerful (and efficient!), no question, but it requires a steep learning curve.

Admittedly that learning curve is steeper for cloud providers implementing it than for customers consuming it but, still, the level of abstraction it provides could be mind blowing for consumers as well.

There is little doubt, in my opinion, that the future is in the abstraction approach. The virtualization of the physical constructs is just a leg of the journey towards that end-state (assuming that that is the end-state in 5+ years).

Considering that the world is still fairly much on physical constructs (Cisco is leading today that space with a 50 B$ annual revenue, give or take a few B$), picturing virtualization as legacy is a bit of a joke. There are, in fact, a few million IT people out there for which networking constructs virtualization is "the new thing". After all, we all know the world is moving at different (IT) paces.

Perhaps the most perfect cloud is the cloud that has a switch (that users can toggle) between traditional networking and abstracted networking that would allow them to consume networking and security the way they like, depending on their (IT) adoption curve and ability to digest innovation. Or perhaps this is just a crazy idea.

Massimo.