IT 2.0

Will we need a C for Nicira? God forbid!

Massimo — Thu, 09 Feb 2012 14:30:46 +0000

This morning I was on the phone with Ivan Pepelnjak (@ioshints) to decipher some of the paragraphs in one of his latest posts on Nicira Open vSwitch inside vSphere. He always has to bear with my stupid questions so I can see him (virtually), from time to time, facepalming some of my questions. Long story short we cleared a few doubts I had on his write up and I decided to ask him yet another border line question. The question sounded like this:

“Ivan, I see Nicira (and many others) are using extensively the word open. I also see a lot of excitement from people that point to Nicira as a cross-hypervisor vendor thus giving this idea of openness and good feeling of not being locked-in. However I believe this problem is multidimensional: if people consider vSphere a lock-in for the traditional virtualization space, why aren’t’ people considering Nicira proprietary for what they call the network virtualization? In the final analysis, why would one want to have 3 vendors to virtualize servers and 1 vendor to virtualize the network? What’s your thought?”.

At that point Ivan laughed out loudly and I was sure my question was another facepalm. Oh well. But before we get there, let me show you a picture of what I had in my head while asking that question and that demonstrates why I thought that vSphere isn’t that different from Nicira NVP (from a lock-in vs openness perspective):

If you segment vSphere as a mere compute virtualization layer (we need to talk about this by the way, maybe in another post) and Nicira as a network virtualization layer they both are pretty much “open” in terms of objects they support. They just happen to be different objects because we segmented them into different categories. This doesn’t mean that one compute virtualization product is a “lock-in” whereas one network virtualization product isn’t a “lock-in”. In other words, if customers are strategically looking at different hypervisors (are they?) for not being locked-in… why shouldn’t they look at different network virtualization products for not being locked-in?

I am looking forward to the day when a C comes in and say “oh wait, now you have Nicira and Pokera (a name I’ve just made up, don’t bother googling it)…. let me manage them both for you in a single pain of glass”. God forbid! My suggestion? Run! Run! Run!

And this is where the next massive mess in the compute era is going to begin, all over again, forgetting about the most important cloud principle above all: economy of scale through simplification. Amazon docet.

I don’t envy you Mr. customer: you have the choice of either being “inevitably locked-in” or die under a ton of scripts (or under a ton of expensive consultants writing them for you for that matter). I don’t honestly see a third way.

But wait a moment, we left Ivan laughing and forgot about him! Perhaps he thinks that this is all wrong. Perhaps OpenFlow is so open that you can interchange vendors at will and avoid that lock-in everybody is concerned about. Well it turned out, much to my surprise, that Ivan was laughing because he linked my very own article “The ABC of Lock-in” in a comment of a blog post published on PacketPushers that was talking about this very same problem. Read it yourself here. While there is admittedly some level of (theoretical?) interoperability between some of the components in an OpenFlow deployment, network professionals don’t seem to be so positive and I’d be interested myself to see a real life homogenous production network built with multi-vendor technologies. Mine isn’t an academic question: we know everything is possible in a demo or better in a power point deck. Mine is more of a practical question for real customers running real businesses. After all having an A and a B interoperate with each other wouldn’t be easier, in my opinion, than having a C homogenizing an A and a B.

Bear with me please. I may not understand a lot about networking (admittedly) but I have been around enough to see “the big picture” (hopefully).

I have just came to the conclusion that, perhaps, open is an abused word. What do you think?

Massimo.

The ABC of Lock-In

Massimo — Thu, 02 Feb 2012 17:03:18 +0000

There have been a lot of discussions lately about a topic I find extremely interesting: vendor lock-in.

Multi-hypervisor is a discipline where you can apply the high level ranting below but you can really apply it to pretty much everything in IT.

I started this blog post writing a couple of pages (as usual) and then I thought no one would care to read it (how can I blame you?). So I summarized it in a few pictures. A picture is worth a thousands words. Always.

So the story goes like… you (the customer) start with A and you build or buy an ecosystem of people, tools, knowledge, programs, scripts (yeah A has APIs) and a lot of other things you need to do to fully exploit the value of A.

You (the customer) are happy but then comes vendor C to your door and tells you that you are locked in into A. “It isn’t so easy to move away from it given all the investments you have done” he says. “Imagine if A was to apply a vTax at some point: God forbid!” C goes on. C tells you there is B now which is good and cheap and you can adopt both A and B so you are not locked in into either. “Let C manage them for you transparently” he says. And this is what happens (in theory):

Yeah, all of a sudden you (the customer) find out that (2 years and 2M$ of professional services later) you are… locked in into C. Imagine now if C was to apply a cTax…. God forbid! You would need to move to D which is cheaper and the story goes on and on. What’s your business? Bank transactions? Shoemaker? Doh I thought you wanted the infrastructure to disappear not become your core attention.

If you thought that this was the end of a sad story there is more. Actually it gets a lot worse than this. It turns out that (2 years and 2M$ of professional services later) you can actually only send “heterogenous” alerts (such as ) to operators in the middle of the night and perhaps present a web interface to a user to power on and off a VM on both platform A and B. Oh and did I mention that when A and B delivers a new version of their platforms you need to give C another good 2 years and 2M$ to “adapt it”? Ok now I told you.

You thought this was the end didn’t you? Well not quite, there is even more:

Since you can only send “the disk is full” type of alerts and provision a VM from a portal (which is neither multi-hypervisor management nor IaaS cloud by the way) you have to build another ecosystem for B similar to what you built for A, essentially doubling your past efforts (which is the reasons for which many people argue that a multi-hypervisor strategy is inefficient).

Can it get any worse than this? I can’t think how.. however if it can, it will. Be sure.

Tip1: I have seen these things. First hand. You have full rights to not trust me and think I am biased now though. That’s ok.

Tip2: In the interest of time (I’ve got work to do too) I exaggerated to make a point. Apply your common sense. Look at the forest and not at the tree in this post. I was also having some fun with some of you. You know who you are.

Discuss below if you want. I am running out of time.

Massimo.

Update: reading the comments below I am starting to realize there is a chance this post gets misread and misunderstood. I genuinelly believe there is a difference between “being able to use both A + B as loosely coupled platforms” and “using C to avoid lock-in and managing multiple platforms as one”. This post was meant to say that the former is doable but can be inefficient, while the latter is just a unicorn thing. More in the discussions underneath.

Virtualization Costs, Virtualization Advantages and the Case for Multi-Hypervisors

Massimo — Mon, 09 Jan 2012 15:41:17 +0000

Last week I came across an interesting blog post from Mark Thiele. The idea of the article is that, as virtualization becomes a relevant cost for IT, it becomes a target for savings. I tried to engage with Mark on twitter but discussing a matter like this in 140 chars becomes a bit frustrating. So I decided to share my thoughts in a more structured way in this (hopefully) brief post.

Mark posted these two tables to demonstrate his theory:

His theory is that, as virtualization now accounts for roughly 30% of the entire IT budget, it becomes a target for cost reduction within organizations. Perhaps I am reading too much into what Mark wrote but my understanding is that he is pointing fingers towards VMware for that “virtualization cost” and, while he is not calling this out specifically, he is alluding to the usage of competitor products. Perhaps in a mixed environment. Mark is welcome to chime in and set the record straight if that is not correct. However I’ll just go ahead and assume that. There are a lot of people thinking along these lines anyway.

I believe the numbers are plain wrong, the premises are plain wrong and, subsequently, the conclusions are wrong. The following is a list of counter arguments to this theory I’d like to throw onto the table.

Wrong numbers

I am wondering what that 30% of virtualization cost includes. If one thing is sure that is NOT the cost of the virtualization licenses alone. I used to work for a hardware vendor and when we were selling 10K$ / 15K$ worth of hardware for a new SMB virtualization project that would have been paired with a 3K$ VMware Essentials Plus license. And that 15K$ for the hardware was just a fraction of the entire IT budget. SAP or Oracle anyone? While I am not going to disclose anything particularly sensitive let’s just say that, on average, an Enterprise buying “a few M$ worth of VMware ELA” usually has an IT budget that is in the ballpark of “a few hundreds M$ in total”. I guess it is somewhat fair to say that the entire IT budget of an organization is roughly two orders of magnitude bigger than the VMware virtualization license costs. Either that 30% is a typo (perhaps it should be 3%) or there is a 27% additional hidden cost when you deploy a virtualization solution? As usual “in medio stat virtus”. More on this later.

Wrong premise

In Mark’s theory, if you adopt virtualization your bottom line remains the same. You are basically shifting costs. If you used to spend “100″ a few years ago, you are now spending “100″ if you sum up the virtualization costs with the savings in the other areas. My first reaction was “why would you want to do that then?”. My second reaction was “this is plain wrong”. I have been working with customers implementing virtualization solutions for the last 10 years and all of them told me that the savings are enormous and many times the ROI associated to implementing virtualization is measured in months, not even in years. Once you reached that milestone, it’s all savings from that point on. Unfortunately I can’t quantify what’s the bottom line “after virtualization” but my gut feeling is that:

it’s less (far?) than 100
the virtualization cost is still peanuts compared to many other areas of the IT budget.

In Mark’s table the “virtualization cost” is twice as much as the cost of the “people”. Really? That is beyond me. We must be kidding.

Wrong metric

Or at least partially wrong metric I should say. You can virtualize for many reasons. One is to lower IT costs (not shifting them). Another one is to achieve what you cannot achieve without virtualization. More agility and more business alignment someone would say. I’d like to stick on practical examples and I’ll say better DR and High Availability for your legacy applications.

Or, for example, how much ($) can you associate to the ability to deploy an application in a matter of minutes Vs a matter of weeks / months? I’ll give credit to Mark to recognize this when he says “Now, please don’t read this the wrong way, I’m not an advocate of the thinking that IT is merely a place that helps us cut the cost of IT”.

Multi-hypervisors

A lot of people think that a proper multi-hypervisor strategy would help to lower the cost of virtualization. This is a very important matter and one that would require a very detailed analysis. Not something I am going to do in this blog post anyway. “Multi-hypervisor” may mean a lot of things to different people as there are a lot of layers where you can integrate different stacks. People sometimes trivialize this complexity.

I am not conceptually against the theory of multi-hypervisors. I find however weird the idea that a multi-hypervisor strategy could save you on license costs. There are situations where a multi-hypervisor strategy may make sense (I may end up writing something about it) but for the majority of the Enterprise organizations out there it just makes little sense. In my opinion at least.

This ties back to the numbers we have discussed at the beginning. If we all agree that virtualization license costs are in the range of 3 to 5 % (or less?) of the total IT budget than it doesn’t make any sense to target that as an opportunity for savings. On the other hand I can see that the “virtualization cost” category doesn’t only account for the license costs but associated training, tooling and skills that manage the solution you are building with those licenses.

Now, I still believe that these hidden costs aren’t 27% of the whole IT budget (they could be another good 3% to 5% perhaps) but the point is that the higher this latest number is, the more expensive it becomes for an organization to have multiple hypervisors and virtualization stacks deployed to manage. This usually means duplicating tools, skills and, in the final analysis, duplicating efforts and costs.

In Conclusion…

As you can see it’s easy to make up numbers and draw wrong conclusions from them. I have tried to give you a slightly different perspective assuming different numbers and different premises. Run your own numbers and feelings against this and Mark’s blog posts and come up with your own conclusion as whether you should actually lower those costs.

My way to look at this is that reducing the cost of virtualization in an organization is like trying to save on a 3% cost of the total cost of IT and, in doing so, potentially implementing something technically inferior that will drive up management costs and will lower the business advantages you have achieved. At the end of the day what you are buying is not licenses but “value for the money” and if many people are still buying VMware solutions in bulk numbers it may mean that people are not interested in saving 1% of the IT budget by dumping an excellent infrastructure solution that is delivering so much for them.

You have a right to disagree. I’d love to continue this discussion in the comments section if you want, certainly there is a lot left to say and argue over these numbers.

Massimo.

vCD Custom Portals and Backend Integrations in a Service Provider Environment

Massimo — Thu, 01 Dec 2011 09:24:13 +0000

This article was originally posted on the VMware vCloud corporate blog. I am re-posting here for the convenience of the readers of my personal blog.

This topic is (rightly so) coming up a lot lately with the Service Providers (SPs) I am working with so I thought I’d share some high level ideas on how we are engineering those clouds. This short article is meant to share some guiding principles on how to engineering custom portals and backend integrations for SPs that are adopting vCloud Director. Please note that this is a very broad topic and if we were to get into all of the details and potential ramifications we would need a book and not a blog post to describe this.

So what does it make it so unique? SPs have been building portals and integrations forever. Why would a vCD based solution be any different? Well, let’s make a step back. There are two main reasons why Service Providers want to use vCloud Director:

Avoid reinventing the wheel and use an out-of-the-box product that delivers the cloud backbone (RBAC, virtual data centers, security, multitenancy etc) on top of which they can create their own solution and value.
Exposing the native vCloud APIs to enable federation with customers that are using VMware technologies (either vSphere or vCloud Director in so called “private cloud” deployments).

The next picture shows, at the very high level, the vCD architecture. A more detailed description can be found here if you are interested.

APIs. APIs. APIs. If there is anything that matters in the cloud that is the APIs. In other words a programmable infrastructure. If you are a Service Provider interested in vCloud Director you are probably interested in the vCloud APIs because that means that, as we mentioned above, you can reach out to a vast amount of VMware customers allowing them to connect to an “on line compatible infrastructure”. You can read more of this hybrid cloud opportunity here and this is a high level representation of this concept:

Browser based access to the cloud is a no brainer. You can read more here about how to use vCC (vCloud Connector) to connect to a public cloud. You can read more here if you are interested in connecting your vCO (vCenter Orchestrator) instance to a VMware cloud. These are just two examples that describe how the end-user can leverage a vCD based public cloud. VMware, and the ecosystem as a whole, is coming out with a number of tools that interact with the vCloud APIs natively. VMware vFabric AppDirector is another good example of these tools consuming these programmable interfaces. I encourage you to have a look at the brief demo video available here.

If it isn’t clear yet, this is the reason for which developing a ton of logic right above the vCloud APIs isn’t a good strategy if SPs want to offer a VMware compatible cloud service. You want the vCloud APIs to be widely available and well exposed. Not obscured by “a ton of scripts and workflows”. That is to say that building something that look like the following picture may not be a good idea if you want to be part of what I call the vCloud bus:

Do not do that. Please.

Having this said, let’s dig into what the SPs need and what their requirements are. An oversimplification of what they would like to achieve can be summarized as follows:

They want to have a customized portal where they can keep their own traditional look and feel and potentially expose additional services.
They need to integrate into their backend systems through a mix of business and technical orchestration tools.

So let’s try to take this apart and start with the first requirement. Ideally the SP would need to build a brand new portal (the out of the box vCloud Director web portal cannot be customized) or reuse an existing portal that they want to complement with the new vCloud Director based IaaS cloud services. As you can see this allows the SP to mesh vCD native services with other services that need to be exposed. These could be other VMware services that are not yet integrated into the vCloud API framework (VMware Chargeback or vShield App come to mind) or totally different services that the SP would like to make available to external customers.

There is only one principle that the SP needs to be conscious of when building this custom portal: the additional services exposed in the custom portal needs to be loosely coupled from the vCloud Director services. In other words the architect designing this needs to make sure that accessing vCD services through the native APIs doesn’t break the consistency. Basically the custom portal cannot inhibit users to access vCD through the out of the box UI or the native vCloud APIs if basic native functionalities is what the users need to access. Putting it in (yet) another way, accessing the cloud via the native vCloud APIs / UIs shouldn’t break the consistency of the whole solution but only limit the users in what they can do (as opposed to accessing a custom portal that has more advanced functionalities).

This is, in essence, the reason for which we removed the “Orchestration / Logic” from the top of the vCloud APIs. Should the SP build the logic on top of those APIs they are essentially obscuring them. In fact, allowing a user to access obscured vCloud APIs would lead to bypassing the logic which in turns would make the whole solution inconsistent.

So what do we do to satisfy the SPs requirements of synchronizing the backend according to events that may occur at the vCloud Director level? The typical example SPs usually refer to is a scenario where an end-user deploys a new vApp and there must be some logic (somewhere) that intercepts this event to update a CMDB with the relevant information. Now, we can spend the remaining of this post discussing the value of capturing a self-service vApp deployment in the cloud into such CMDB but we will leave this discussion for another post. The question is: if we can’t put this logic between the user and the vCloud APIs to intercept this event, how can the SP know what happened to track it properly (the CMDB is just an example, it could be any backend system such as ticketing or anything really).

In vCD 1.5 VMware introduced a new feature called “vCloud Messages” also known as “notifications” or “call-outs”. Essentially vCloud Director 1.5 is able to track internal events and notify them via an AMQP message bus for an external module to consume these information. The picture below shows the flow where vCloud Director informs the AMQP bus that an event has occurred and the Orchestrator will take the proper action to update the backend systems:

In this example a vApp is deployed using the vCloud APIs, vCloud Director puts a message on the AMQP bus that the vApp has been created, the orchestrator module reads this message and it then updates the CMDB. Note that the module where the logic is implemented connects to basically all modules in the infrastructure since the notification may require actions that go beyond those of updating a back-end system.

It is also important to note that the diagram above is a logical representation. The “Additional Cloud Services” illustrated above can either be delivered via the Orchestration / Logic components or by totally different subsystems that are available in the Service Provider infrastructure. In other words there should also be a virtual link from the Custom Portal to the Orchestrator / Logic components. The very same principles discussed above apply here as well. Exposing additional services (made available by the orchestration layer) shouldn’t inhibit and limit end-users from accessing their resources via the native vCloud APIs (or UI for that matter).

Perhaps it is worth spending a minute to better characterize the Orchestration / Logic brick. In a complex organization like a Service Provider this may be comprised potentially of multiple modules and products. Usually there are at least a couple of components inside that brick and they are what I refer to as a Business Orchestrator and a Technical Orchestrator. The former is responsible for interacting with the back-end systems (it may even be considered part of the back-end systems) whereas the latter is responsible for interacting with the actual infrastructure components and modules. Graphically, it means this:

One of the reasons for this split is because the business orchestrator module plays a key role in the governance of the solution but doesn’t usually have the full range of adapters and connectors to talk to the infrastructure modules. Because of this it leverages a technical orchestrator module to deal with that part. In most situations the Service Provider already have such a business orchestrator in place. Most of the time though, based on my experience, what’s missing is a more technical orchestrator module that interacts with the lower level infrastructure components. This leads to lots of extra in-house development that is expensive, time consuming and hard to maintain.

This is where vCenter Orchestrator comes in. We have previously mentioned, at the beginning of this post, you can use vCO as a cloud end-user tool to consume the vCloud APIs but where vCO really shines is as a technical orchestrator acting in the back of the cloud to pull all the infrastructure pieces together. There is also a nice article that talks about how to extend vCloud Director capabilities using vCenter Orchestrator (this ties back to the concept that additional cloud services exposed in the custom portal could be delivered by the orchestrator directly).

Note that what I have discussed here so far is the logical high level architecture of the solution. Different modules do not necessarily mean different products (although they often do). For example there may be situations where a single product could deliver both a portal and business orchestration modules. VMware Service Manager is an example of these products. As I said big Service Providers often have this part historically covered already anyway.

In conclusion, it is advisable (if not imperative) for Service Providers to be able to expose the native vCloud APIs to maximize market opportunities and value to existing VMware customers. In order to do so SPs need to follow proper design principles for backend integration and custom portals design. This brief blog post is only meant to be a starting point for outlining the criticalities associated.

Massimo.

Amazon, Netflix, Standard Cloud APIs and the Inevitable Lock-in

Massimo — Wed, 14 Sep 2011 14:27:33 +0000

A few weeks ago Adrian Cockcroft (Cloud Architect @ Netflix) wrote another very interesting post on his blog. Adrian warms up the discussion sharing his experience about the reasons for which you may want to use public cloud services. While there are a lot of people (including myself) sometimes advocating about these concepts, there isn’t anything like hearing this first hand from the people that are actually running a business out of this model. I like to hear/read Adrian for this reason. It’s no secret that Netflix uses Amazon AWS to run their business and this is the second part of Adrian’s post. Admittedly the part that intrigued me the most.

The remaining part of his post is basically a public ask (or hope) to see AWS API compatible clouds (or clones), possibly built around the OpenStack stack (no pun intended). He doesn’t seem to be shy about sharing his pessimism about OpenStack success (correct me if I am wrong Adrian) but this isn’t going to be the core of the post I am writing . Only time will tell who will be successful in doing what.

Going back to Adrian’s “ask” I believe there are a number of reasons why he would like to see an AWS clone. Again Adrian is welcome to set the record straight if I got the wrong understanding.

One of the reasons is somewhat logical and it boils down to: risk mitigation, additional resiliency and problem avoidance. I came to learn from another very interesting piece by Adrian that Netflix has a number of policies for backup and data retention. This includes backing up data on S3, copying them in different AWS availability zones, and eventually replicating them in different AWS regions. It only makes perfect sense for Netflix to go a step further duplicating these data at different service providers for an additional level of risk mitigation. This is after all what this slide was trying to convey in his interesting pitch (highly recommended if you haven’t watched it yet):

I’d speculate that another good reason for which Adrian would like to see alternative public clouds based on clones of the AWS APIs is this: Netflix would like to have choices. Simple. What’s wrong with that? I wouldn’t expect anything less if I was them. Someone would try to argue that Netflix doesn’t want to be locked-in into Amazon. I think the matter is a lot more complex and, in fact, I am not sure I agree (entirely) with that. I don’t even know if avoiding a certain level of lock-in is even possible at all anyway (more on this later).

Warning: I am not trying to sell vCloud to Adrian Cockcroft or anyone else. By the way I believe Adrian knows more about vCloud than I do. .

Having this said this is a hot topic. Adrian’s blog post (along with all comments on the thread) reminded me of a couple of old blog posts I wrote last year. They are “Open standards, open source, OpenStack and the TCPIP of Cloud APIs” and “vSphere, vCloud and the Meaning of Being Open” where I was trying to describe VMware’s strategy in terms of API standardization and choice of service providers. This is an oversimplified picture, from one of those blog posts, that focuses on the point I am trying to make: a common API that works across different service providers.

This picture primarily shows access to different service providers using the same interface but the story doesn’t stop here. Since vCloud Director is a product you can buy, you can even build your own private cloud if you want to. I regularly use, as a consumer of cloud services, a couple of internal labs (that mimic private clouds) as well as the public Stratogen cloud and another public cloud I am piloting with another big telco in Europe. I do have my choices.

Here I am not specifically talking about the effort of making the vCloud APIs an industry standard. Lately, I came to the (personal) conclusion that a standard API is a function of its adoption and not a function of a theoretical agreement. I am instead talking about the choice of service providers the vCloud stack would be able to guarantee to consumers. After all, it’s one stack instantiated many times by different organizations (either private or public). I am not sure if it’s a standard (yet), certainly it is very consistent. And this is where I can hear you claiming. “it’s a lock-in“. And this is where I would argue: “is a certain minimum level of lock-in avoidable anyway?”

Let’s try to get into a bit more details and explore the options this industry (more particularly consumers and providers of cloud services) have.

API lock-in

First of all, what on earth is a lock-in. How do you define it? A lock-in, to me at least, is a function of the time it takes to move to an alternative solution. In the context we are discussing here a lock-in is a function of how much time and effort it would take to rewrite your software (for example the Netflix software) to talk to a different cloud interface. Adrian at some point says it wouldn’t be (too) difficult for Netflix to do that but the mere reasons for which he is looking for an AWS clone is telling me he doesn’t want to get to that point (my speculation).

At this point, does it make any difference if the APIs you are writing your solution against are the vCloud APIs, the AWS APIs or the future OpenStack native APIs (these are APIs that exposes the OpenStack personality, not the AWS clone interface). I don’t think so. Lock-in isn’t so much what you are writing against (be it the vCloud APIs, the OpenStack APIs, or the Amazon AWS APIs), it is rather how difficult it is to move away from it.

At the end of the day, as a consumer, you don’t have control on any of those anyway. So it doesn’t make any difference at all.

If you are a service provider you are pretty much in the same situation if you intend to use vCloud Director or OpenStack. Unless you decide to take OpenStack, fork it and do with it whatever you want. In that case it’s a different kind of lock-in, and not necessarily a better one. Good luck with that.

Sure if you are big enough you may be able to contribute to the main OpenStack project and see what you need / want implemented sooner rather than later but, frankly, if you are an organization of such a size, chances are that you have a word on the roadmap of a proprietary product too. I have seen that first hand.

All in all using available third party software products (be them vCloud Director or OpenStack) to build clouds has the advantage of allowing consumers to connect to different service providers. Having this said, if users decide to consume services from these service providers, they are essentially locking themselves into that specific interface/API. Whatever that interface is.

I am not getting into the federation and hybrid cloud discussion here because it would only be useful to discuss why choosing one interface over the other could be better. Not the point of this post anyway.

Service Provider lock-in

The other option to see more openness (or the perception thereof) would be to keep Amazon AWS as your “gold standard” and pray for other service providers to implement a clone of their APIs (using OpenStack or any other tool). This is, to me, the worst of both worlds since both consumers and providers have certainly no control whatsoever on the AWS APIs (similarly to how you’d have no control over the vCloud APIs or the potential OpenStack native APIs). In addition to that you’d have to deal with the complexity of creating and consuming APIs whose clone is fundamentally a reverse engineering hack which will suffer the generic problems of copying someone else’s interfaces.

This is especially true when these interfaces are changing at the speed of light (given the pace Amazon is innovating introducing new cloud services) and also given the fact that the AWS interfaces appear to be pretty complex to track.

In reality, Adrian was asking for cloning only a subset of the features provided by AWS but, based on my past experience working for a company that was trying to be the overlay interface to everything, typically the only thing that works (somewhat) well across different virtualized platforms and interfaces is turn on and off virtual machines. I bet Netflix needs something more compelling than that to consider another service provider that claims to be compatible with the Amazon APIs. OK I am exaggerating but you see (hopefully) my point. If Amazon was to facilitate this cloning process or better yet if Amazon was to provide (read: sell) to service providers its own technology enablement stack the story would be very different but I don’t think any service provider will be successful in implementing an AWS clone if Amazon doesn’t want that to happen.

If I was evaluating this option, as a consumer, I would just give up with the idea of consuming a clone of Amazon…and I would just consume native Amazon AWS resources. Sure you are limiting yourself to a single service provider (AWS) but I think it is better to be locked-in into Amazon than having choices… that don’t work very well. Because, at the end, we all need to be pragmatic don’t we?

Conclusions

In conclusion I just want to reiterate that it’s just a bet you are making and you can’t really avoid a certain level of lock-in. It’s just a fact of (IT) life. In the last 15 years I came across a lot of vendors that were selling openness and freedom of choice. At the end of the day they were just trying to sell another control point. They don’t call it a lock-in as it makes the whole sales process a bit harder but it is what it is.

This post is not meant to bash Amazon or OpenStack. As a matter of fact I am bashing at least as much vCloud. It’s just a reality check of what’s going on and how I see these things progressing going forward for both consumers and providers of (IaaS) cloud services.

My message? Make your bet and keep your fingers crossed.

Perhaps I will be proven wrong. Oh well, it’s just my usual (less than) 2 cents

Massimo.

vCloud Director 1.0.1: Networking Samples

Massimo — Wed, 29 Jun 2011 14:50:50 +0000

My old vCloud Director Networking for Dummies post is still going strong according to my blog statistics. I believe this is an indicator that people are looking for more information about this topic so I thought I’d give it a little bit more color and create a few real life examples on how that theory works in practice. I suggest you read the Networking for Dummies post linked above before you dive into this one.

Note also that the other post as well as this one are based on vCloud Director 1.0.1 which is the latest release available as of June 2011. Things may change in the future so, if the vCD release you are using at the time you read this is above 1.0.1, chances are that things could be slightly different. I can’t really say more than this at this point.

Last but not least, everything I will be doing below can be done as a cloud consumer in self service mode. As a matter of fact I will be doing everything as an Org Admin.

Introduction

To walk through an actual implementation of the networking stack I’ll use my IT20 organization hosted in the Stratogen cloud. This discussion starts with the description of the networking plumbing in my vCloud organization. From the vCD UI it looks like this:

From a logical perspective it looks like this:

My Org has four public Internet addresses that Stratogen associated to my “Routed Network” when they created the tenant. For security reasons I am not going to widely advertise them in this post.

You can see these assigned addresses if you right-click on the Routed Network and select “Configure Services“:

The last piece of the puzzle is three vApps I have created in this Org and that we are going to connect to the various networks you have seen above. This is supposed to give you a practical idea on how things can be configured. The names of the vApps should be self-explanatory.

Direct Internet Connection

Let’s start with the most simple of the networking scenarios. Note there is a vApp called “Turnkey_Internet” which is comprised of a single VM. That VM is connected to the “Direct Internet” connection available in my Org. I have only one comment for this example: scaring! Never do this because you are in fact plugging your VM directly into the Internet without any level of protection (other than what you could have inside the Guest OS of course).

This is how my VM is configured:

And this is how the VM fits into the logical network view:

The way this works is pretty straightforward and, if you read the vCloud Director Networking for Dummies post, it should be explained there. Basically the cloud administrator has configured a pool of available IP addresses for this “External Network” (since this is a vSphere PortGroup with native Internet connectivity this pool will contain native Internet IP addresses). Since the Direct Internet connection in my Org is nothing more than a pointer to this vCD External Network which in turns is a pointer (with metadata) to the PortGroup backing it, the result is that the vNIC of my VM gets connected directly to this PortGroup. vCD assigns the (vNIC) an IP in the pool.

I am glad Stratogen configured this network for me – as it is handy if you are experimenting with vCD networking – but in a real life scenario you would never want to connect VMs to a connection like this (directly connected to the Internet). However this may become pretty interesting if you, as an Enterprise, are using virtual data centers hosted in a cloud where the Service Provider has configured an MPLS connectivity back to your headquarter. Something like this:

It goes without saying that, doing so, you are effectively dedicating an External Network (and in turn a PortGroup) to the IT20 Org. If for any reason you give access to another Org to the same External Network (either or – see next section) you are essentially giving the other Org access to the IT20 MPLS network.

Routed Network – single-tier vApp

This is where things start to become more interesting, slightly more difficult to explain and very reach at the same time. I have another vApp that is called “Turnkey-Routed”. It contains a single VM which is connected to the Routed Network available in the IT20 organization. You can imagine this Routed Network as a dedicated layer 2 segment protected by a firewall device (vShield Edge). For more information on how this work from a vSphere perspective read the vCloud Director Networking for Dummies post. Essentially the VM in this vApp gets assigned an IP address available in the pool defined for this layer 2 segment. This is how vCD shows the details of the Hardware Properties for this virtual machine:

And this is how it logically fits into our diagram:

Note that in the diagram above we went a couple of steps forward. Not only we are protecting the VM with the Edge: I have also configured the Edge to NAT the private IP. To do so I have created a one-to-one mapping rule to one of the four Internet addresses Stratogen assigned to me. I have also configured a firewall rule to only allow traffic on port 12320 to reach the VM (this is because the Turnkey appliance uses particular ports to get access to SSH and web admin interfaces). How did I do this? Move onto the Routed Network and right-click on Configure Services. Point to the “External IP Mapping” tab and configure the NAT rule:

You would then point to the “Firewall” tab where you can configure the firewall rule I have described above (as an example).

I have just blocked all traffic coming into this VM except for traffic directed to port 12320. As easy as it is.

Routed Network – multi-tier vApp

The single-tier vApp is still pretty simple. Let’s now focus on the third vApp I have mentioned. This is the “2Tiers” vApp which is comprised of a front-end Windows VM (Win-Web) and a back-end Linux VM (REHL-DB). The idea is to provide IT20 customers with access to this application protected by multiple levels of security. The first step is to connect the front-end to the Routed Network in the Org and NAT it. This is similar to what we have already done with the single-tier vApp discussed above. I am not going to show screenshots of the NAT and Firewall configurations because the steps are very similar. It goes without saying that the Win-Web VM has a different private IP and I will be using another public IP to create the DNAT rule. This is how the logical layout looks like for this specific vApp. I am opening port 80 for this example:

As you can see the back-end VM is not yet connected to any network. As I said we want to provide an additional level of security for that VM and we don’t want to connect it “directly” to the Org network. How do we do this? This is where the so called “vApp Networks” come into place. You can imagine vApp Networks as layer 2 network segments dedicated (and only available) to the specific vApp they have been created for. In other words a vApp Network created for one vApp cannot be used by any other vApp. If you want to know more about this concept please refer again to the vCloud Director Networking for Dummies post.

You can create vApp Networks in multiple ways but the easiest one is to click on the “Add Network” choice in the drop-down menu for the vNIC connectivity available in the Hardware Properties of the VM:

Selecting it kicks off a brief wizard that asks you the very basic metadata to create a new network (Subnet Mask, Default Gateway, IP Pool etc). You can then select whether you want to protect this dedicated vApp Network with NAT and Firewall functionalities. You can do this in the Networking tab when you “Open” the vApp:

Let’s pause for a second here (too many screenshots to digest).

Don’t be fooled. What we are trying to do is to create a logical layout like the one depicted below:

In a way we are applying to this vApp Network the same NAT and Firewall principles that we applied to the Routed Network at the organization level. Where do you configure these rules for the Edge device that is backing this vApp Network? Easy. Look at the latest screenshots above and click Details. Done.

This is the tab where you configure the NAT rule so that the DB private IP gets mapped to the Routed Network in the organization:

Below is the tab where you configure the Firewall rule to allow DB traffic only (this rule is just an example):

Conclusions

Let’s now try to put all these piece together and look how the logical layout of the workloads running in the organization looks like as a whole:

As you can see the self-service networking stack in vCloud Director is pretty powerful and flexible although there are certainly things that could (and should) be done better. For example you may argue there is a lot of NATting going on (and I would have a problem arguing the opposite). But, as we said, this post is based on the 1.0.1 version of the product and things may change in the future.

Note that we haven’t covered any example on how to use the “Internal Network” since it should be pretty straightforward. It’s basically a flat layer 2 network that doesn’t go anywhere and only allows VMs attached to it to communicate to each others.

I hope you found this post useful. I’d like to get your feedbacks.

Massimo.

The Cloud and the Sunset of the GHz-based CPU Metric

Massimo — Fri, 24 Jun 2011 08:47:14 +0000

We have known this for years but it’s only when you get a slap on your face that you understand what’s going on for real: the GHz metric is useless these days. I was experimenting with vCloud Director the other day and I was checking out from the catalog my Turnkey Linux Core virtual machine (I use that because it’s small and I can check it in and out from the catalog very quickly – it’s also a very nice distro!). This instance was launched in a cloud PoC I have recently started working on for a big SP and I noted it took quite some to boot, at least more than what it usually takes which is around 40-60 seconds. Similarly the user experience once booted was not optimal compared to what I am used to. While I haven’t done any serious analysis of the problem, I am going to take a stab at what I believe it was happening behind the scene.

A little background first. This service provider opted to use some quite old IBM x86 servers to run this PoC. Since the PoC, for the moment, is focusing on functionalities – rather than performance and scaling – we thought it was ok to use these servers. For the records they are IBM System x 3850 (8863-Z1S). These are single-core 3.66GHz servers with 4 sockets. Admittedly, pretty old kits. This is how they show up in vCenter:

This is technology from 2004/2005 if memory serves me well. Consider that, while they be 64-bit servers (I’d need to double check – can’t bother) they certainly do not even have the CPU virtualization extensions – required in the latest vSphere releases – to support 64-bit guest OS’es. We found this out at the beginning trying to instantiate a VM of that class. They have been working fine anyway and are serving our needs pretty well for what we need to test.

Back to the performance issue I was describing now. You should know that when vCloud Director assigns to an Organization a vDC using the PAYG model, it sets a certain “value” for the vCPU. You can think – roughly and conceptually – about this value as something similar to the AWS ECU (Elastic Compute Unit). This is a good thing to do because it provides a mechanism for the cloud administrator to normalize the capacity of a vCPU. It also allows the provider to create a mechanism to cap the workload (as you probably don’t want a consumer to stuck an entire core). For the records vCD can also reserve part of that “speed” for the VM so that it can guarantee that these reserved resources are always available. The picture below shows the screen where you set this value when creating an Organization vDC (these are all the default values).

Note that the default “speed” value for a vCPU in the PAYG model is 0,26GHz (or 260MHz if you will). This means that, when you deploy a VM in this vDC, vCloud Director configures a limit on the vCPU with that value. I am not sure how Amazon enforces the ECU on their infrastructure (or if they enforce it at all) but this is how vCD and vSphere cooperatively do it:

To the point now. Everybody knows that x86 boxes scaled CPU capacity exponentially in the last few years. Today, a last generation 4-socket server can have a ridiculous amount of cores (up to 80). That’s one dimension of the scalability Intel and AMD have achieved. Another dimension is that the core itself has gone through some very profound technology enhancements and got better and better. Let’s try to do some math and find out how much better.

To do this I am not going to do a scientific comparison (I wish I had the time). I am going to quickly leverage a couple of benchmarks to find out the different efficiency between the old and the new cores. I am going to use the TPC-C benchmark – which is a simulated OLTP workload – that may not be always relevant but it’s known to be CPU bound – although it does require a couple of hundreds thousands of disk spindles to not be bottleneck on the disk subsystem (which means: don’t bother trying it at home). Long story short I took a TPC-C benchmark of an IBM server equipped with the same CPUs that we are using in this cloud PoC and I compared it to a benchmark of one of the IBM servers that supports the latest generation of Intel Xeon processors:

Old Benchmark: 150,000 tpmC (4 sockets, 4 cores, 3.66Ghz)

New Benchmark: 2,300,000 tpmC (4 sockets, 32 cores, 2.26Ghz)

We are not interested in the metric (tpmC = transactions per minute C-workload) in absolute terms because we are using this metric just to compare the CPUs. So for the two systems the math would (more or less) look like this:

Old server: 150K transactions on 4 cores makes roughly 38000K transactions per 3.66Ghz core which means roughly 10 transactions per MHz
New server: 2.3M transactions on a 32 cores make 72K transactions per 2.26Ghz core which means roughly 32 transactions per MHz

I didn’t have time to triangulate with more benchmarks so will stick with this one and we will claim that a single MHz of a new core is worth about three MHz of the old core we are using in the PoC.

Now I guess you have an idea why talking about MHz is meaningless at this point. I guess you also see why assigning “260MHz” to the CPU tells half of the story (the other half being.. ok but of which core?). Yet there still are a lot of people out there that think that a 3Ghz processor is faster than a 2.26GHz processors. I believe you also have an idea now why Amazon and VMware introduced these different metrics: it’s basically a way for the provider of resources to normalize the actual capacity of the CPUs underneath to overcome the variance we have seen above). My initial performance problem was in fact solved raising the value of the “vCPU speed” in vCloud Director: I assigned more GHz to the vCPU to off-set the poor quality of the core.

Let me change gear here now. What we have discussed so far is fine when you are dealing with VMs since you can easily use a technique to buffer this variance (the “vCPU speed” or the Amazon “ECU”). However this becomes a little bit trickier when you start dealing with virtual data center capacity. How do you normalize that? The easiest (and more user-friendly) way to do this is to expose directly the capacity expressed in terms of GHz, which is what vCloud Director does today when configuring Organization vDCs in reservation or allocation mode.

So what do we do? We all agree that 10GHz is no longer meaningful but what is the other option? You may argue that in a cloud environment you shouldn’t bother about the low level hardware implementation details because the whole purpose of cloud is to hide them right? On the other hand we are talking about IaaS type of cloud here so a much higher level metric such as “application response time” wouldn’t be applicable as vCloud Director doesn’t really manage the middleware and application part of the stack; that would be out of its control.

GHz may sound like the right thing to expose when you are providing virtual hardware capacity in an IaaS cloud but yet the metric would need to be consistent across different providers (and we have seen this may not be the case if different providers are using different hardware technologies). An option would be to try to normalize this value similarly to how the CPU in the VM gets normalized. Sure but how? With which metric? In the VM based model you can expose a very well known metric / object: the vCPU). In that case you can pass onto the consumer the key to decrypt the amount of compute capacity of that object similarly to how Amazon does it with ECU : “One EC2 Compute Unit provides the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor. This is also the equivalent to an early-2006 1.7 GHz Xeon processor“. The keyword here being “equivalent”. This means you can use any CPU technology you want, someone will tweak a parameter so that your performance experience will always be the same. Which is hat I have done above to fix my performance problem in the PoC for example.

The vCloud Director challenge is slightly different than the Amazon challenge though in the sense that vCD is a technology that enables service providers to stand up cloud backbones quickly and efficiently… whereas Amazon is indeed a Service Provider. So coordination and standardization for them isn’t an issue (as a matter of fact ECU doesn’t really have any industry-wide meaning outside of the context of the AWS platform).

Similarly to how the industry is looking for standard APIs to consume cloud resources across different providers, I believe there is a need to standardize the metrics that describe the capacity that those resources can deliver. Can this metric be an industry standard benchmark like TPC-C (for example)? Or should it be more like a brand new synthetic value that combines a number of benchmarks covering a wider range of workload patterns? Or should it just be a normalized GHz number? Ironically enough this problem can only be worse in a PaaS context because it is supposed to be playing at a level of the stack where the hardware infrastructure is completely hidden (and exposing GHz wouldn’t make any sense at all). However PaaS doesn’t expand its reach to a level where higher level of metrics (such as application response time) can be used because the application code falls into the consumer responsibility and not in the PaaS provider set of responsibilities. Which means you could have a PaaS layer that “screams” but an application layered on top of it that is a piece of junk (performance wise).

I’d like to point out also that you may consider having an end-to-end governance of your entire stack where you can monitor high-level metrics for the services / applications and you let the “governance system” deal with the monitoring and capacity planning of the virtual hardware and all other layers above it. While I admit this would be a desirable state we are not quite there at the moment.

However, if you think at the separation of duties and roles that this multi-layer cloud stack brings in – a stack made of many different services interfaces each of which has a provider and a consumer – we also need to make sure each of these interfaces has a way to be measured consistently. In other words, it may not be a human being having to deal with the measurement of these IaaS metrics, it may be a “governance system” that automatically does that for the human being, but yet we need to instrument these interfaces so that the “governance system” can deal with them.

Imagine for example a situations where a SaaS provider may be the consumer of external IaaS resources, this end-to-end monitoring becomes difficult to achieve hence the need to create more detailed SLA metrics between the various layers and their interfaces in the stack. In this specific example how is the SaaS provider subscribing IaaS resources? How are these virtual hardware resources going to be measured, monitored and enforced from a performance perspective by the two parties when the two parties are separate entities with separate duties? How do you define those boundaries from an SLAs perspective? That’s what we are debating here.

To GHz or not to GHz? That is the problem. All in all, the GHz based capacity planning and monitoring is dead. However it seems we are still flooded with IT tools that are leveraging it pretty heavily.

I’d like to hear what you think and if you have any opinion on how to address this problem.

Massimo.

The Italian Elections and the Case for Cloudburst

Massimo — Thu, 02 Jun 2011 12:11:32 +0000

A few days ago we had a big election day in Italy for renewing a good part of the public local administration. For and in itself this wasn’t a big deal and something that wouldn’t have generated a lot of attention among the 60M people living here. However, without getting into a lot of details, suffice to say that this turned into yet another “do you like Mr. Berlusconi? Yes or No?” type of referendum. And this of course generated a lot of curiosity around the results. So what do most of the people working in an office do at 3PM when they close the voting? They connect to their favorite on-line news web sites and have a look at the exit polls statistics. And I am no different: killed by curiosity, I opened another tab on my browser and quickly pointed it to “http://corriere.it” the internet home of the most important Italian newspaper: Il Corriere della Sera. What does this have to do with cloudburst you may be wondering? Well it does have to do with cloudburst because, after waiting a minute or so on the “waiting for corriere.it“, this is what I was able to get (red emphasis and sketched question mark is mine):

How disappointed?! By the way what I have experienced personally is not anything new. It happens very often and all the times “something interesting” happens. Look at what happened when Michael Jackson passed away for example.

Now, I know that there have been a lot of bashing regarding the concept of being able to “cloudburst” into the cloud. I believe this is due to the fact people tend to jump to extreme use cases. If you associate the term cloudburst to what it actually means in a non-IT world then yes you may think about a particular use case where your IT infrastructure may need to react in nano-seconds to an unanticipated, and somewhat catastrophic, event that may last 30 seconds or so. From Wikipedia:

“Cloudbursts descend from very high clouds, sometimes with tops above 15 kilometers…Meteorologists say the rain from a cloudburst is usually of the shower type with a fall rate equal to or greater than 100mm (3.94 inches) per hour… During a cloudburst, more than 2 cm of rain may fall in a few minutes. When there are instances of cloudbursts, the results can be disastrous.”

In a world where people think that the Facebook and Google infrastructures are the norm, it doesn’t surprise me that someone may also think that that’s the only cloudburst scenario (and no, I am not referring to Chris Hoff here, in fact I think he is one of the most realistic people I follow on twitter).

With the “Google and Facebook are the norm” mind-set, of course you are mentally led to think that an IT cloudburst is a fully automated process where your application can immediately react upon a sudden spike of connections and it goes out, automagically, deploying on-the-fly new instances of the application, modifying load balancer configurations to grab new traffic immediately. Oh, and of course it wouldn’t be a “real” IT cloudburst if, after a few sub-seconds of idle time, all of these extra resources are decommissioned, automagically, and everything returns back to normal. Yeah, dream on. I bet you are saying it’s “marketing bollocks”! Of course it is!

So, if that is the meaning you are associating to the term, I agree that all this IT cloudburst talking in the industry is, most of the time, just a bunch of marketing stuff (for the moment at least). Certainly I am not selling the idea that your site could go from 300 front end servers to 1,400 in a matter of milliseconds and contract back to 300 after a 18 seconds surge. All automagically. I am not that stupid.

My concept of cloudburst is a little bit different. And practical. I am a simple guy and a pragmatic person. I don’t want to re-architect applications for failure or create a chaos-monkey programs that kill production workloads to test their resiliency. Call me an old-school IT boy, but I just want to be able to see the exit polls on http://corriere.it next time.

There are a couple of concepts regarding cloudburst that we should all consider. They are the reaction time and whether the triggering event is anticipated or not. The extreme example above obviously assume a near real-time reaction triggered by an unexpected event. What I have in mind is a much longer lead time to allow you to scale the resources associated to an event you can anticipate. Something like…. an election day for example (or a programmed marketing campaign for that matter or anything that come to your mind that has those two characteristics)!

No, we are not talking about doing this with “a spacecraft equipped with a warp drive that may travel at velocities greater than that of light by many orders of magnitude, while circumventing the relativistic problem of time dilation”. We are not talking about micro-seconds response times. We are not even talking about a 4M$ titanic orchestrator product (that happens to come with an 8M$ bill worth of professional services to implement it). I am talking about deploying (yes even manually! How old school am I?) a few additional web servers in the cloud to cope with the anticipated demand so that I can look at my damn exit polls!

Let me state it very clear. I have no idea about the back-end infrastructure that Il Corriere della Sera is running. The only thing I know is that they are running Apache on Linux according to netcraft.com. I can also imagine that they are using a traditional SQL backend database (SQL Server? Oracle? MySQL? DB2? Who knows!?).

I am not even sure whether they are a VMware customer (but one can always hope). Let me speculate they have 6 load balanced web servers (just off the top of my head). They could be 3 or 9, it doesn’t make any difference when they are exhausted anyway! Whether they are virtual or physical I don’t care; does it make a difference in the end? I don’t think so. An exhausted set of virtual resources produce the same result compared to an exhausted set of physical resources in the end: a browser error! Last but not least I’d speculate that, given the nature of the on-line service they offer, the presentation tier (web along with any application logic that goes with it) is the bottleneck rather than the backend data repository or the network. So let’s say their deployment looks similar to this:

So, if my assumptions are correct (yet to be demonstrated), why not “cloudbursting”? Not in the extreme Star Trek sense above. But rather in a more pragmatic sense where you could ideally sign up with a public IaaS on-line service provider to get access to Pay-As-You-Go resources and double your front end-access just before you need it. Ideally this cloudburst should happen in a public cloud because the whole idea of this extra spike is that it is going to use resources that you don’t have in house. Why not? Well, because most customers (if not all) out there are not Google nor Facebook and they may not have 40 spare servers in house at any point in time for capacity overflow! These customers may not have a critical mass of resources in house to deal with these peaks? You are not Google? You are no one! Come on, wake up folks! Life in the real data centers is not as fun as in the Google and Facebook

What I have in mind to make http://corriere.it more scalable is actually fairly simple and doesn’t require them to turn into a new Google. While the scenario below could easily be implemented using different technologies (such as for example an on-premise physical deployment extended to AWS virtual servers) I am going to describe, at a high level, what you’d need to do if you were a VMware customer with a vSphere deployment on premise extending to a vCloud public Service Provider. I am just more familiar with this stack, that’s why I am describing it:

A few days upfront “the event” you subscribe with a vCloud Service Provider. This entitles you to access public resources with a PAYG model. It shouldn’t take more than 104 minutes.
Depending on the nature of the connections and related security you need to have, you can ask the provider to setup a VPN between your remote virtual data center and your own on-premise vSphere deployment. For more background on how this could be done you can read this. This may be required if the web/application servers need to connect to a back-end database that is not reachable from outside the organization firewall (very likely).
You can then deploy new web/application server instances from vCenter; this may be as easy as a clone of an existing web/application instance or it may require slightly more manual work like if you were to start from a generic OS template . This isn’t very different from what you’d need to do if you were to deploy a new on-premise instance of the same web/application server.
When you are done with that additional virtual server deployments you can then easily either manually export/import these instances from the on-premise vSphere deployment into your remote virtual data center, or you can use vCloud Connector to move these workloads if you are a GUI aficionados. Moving stuff around compatible infrastructures is obviously a huge plus in this case as it simplify a lot of the work. This may not be the case if your source is a physical environment and/or your target is AWS.
Last but not least you need to reconfigure the load balancer to include these new front-end instances to make them part of the http://corriere.it site.
When “the event” is gone and the traffic is notably back to normal you can decide to reconfigure the load balancer and decommission these additional front-end instances in your remote virtual data center in the public cloud to avoid incurring into additional charges due to the PAYG model.

This is not rocket science. You can even try to optimize the process above a little bit so that if you have 3 or 4 or 7 of these “events” in a year you can commission and decommission these workloads a little bit more efficiently. This can be done either manually or with a little bit of simple scripting, still not having to spend a 4M$ tax for a “cloud” orchestrator (i.e. shooting a fly with a bazooka). Same thing for the load balancing. No I am not talking about a “global super fancy” load balancer that can provide workload balancing across sites with built-in DR algorithms, locality optimizations and the like. I am talking about the same load balancer you used to use that is now also pointing to the remote web servers via the established VPN tunnel. Sexy? Not at all but remember we are not talking about optimizing the datacenter and/or the network. We are just trying to fix a problem that is server capacity exhaustion. If your problem is network related (latency and bandwidth) then you may want to do something else (perhaps using a global super fancy load balancer, why not?). So what I am suggesting is to extend the infrastructure like this:

Note we have always been talking about events that can be anticipated and that can give to Il Corriere della Sera IT administrators the lead time required to provision new cloud resources. The same concept could apply for unanticipated events provided that the duration of “the event” is long enough to make the provisioning worth it. If an unanticipated big event generates insane amount of traffic for a few days (and you have a clean and structured semi-automated provisioning methodology) you may think about cloudbursting. If, on the other hand, an unanticipated event generates a lot of traffic for about 36 hours (and your provisioning is going to be manual with some lengthy customization work) it may not make a lot of sense to cloudburst. Another picture to fix the concept in your mind (hopefully).

Is this the sexy sub-second type of IT cloudburst most Google-minded people think about? Probably not! But, hell, at least I will be able to see the exit polls next time. This is in fact a relatively low investment, of money and time, to produce (potentially) a significantly better on-line service. But if you are in the business of over-engineering things you may perhaps disagree with my point of view.

My next action item is to go and talk to Il Corriere della Sera now to see if this makes any sense to them! Which is the only thing that matters at this point!

Massimo.

TCP-clouds, UDP-clouds, “design for fail” and AWS

Massimo — Wed, 27 Apr 2011 14:05:21 +0000

An entire Amazon AWS Region was recently down for four days. Everyone has got to blog something about it and this is my attempt. Just as a warning: this post may be highly controversial.

There has been a litany of tweets pontificating how applications on AWS should be deployed in a certain way to achieve the maximum level of availability and how applications need to be “re-architected” to properly fit into the new cloud paradigm. Basically the idea is that your application should be thought, designed, architected, developed and deployed with failure in mind. Many call it “design for fail“. That is to say: software architects and developers should never assume that any given piece of the infrastructure is reliable.

I beg to differ. I don’t like this idea even though some of you will be thinking I am a bit archaic.

George Reese wrote a great blog post titled The AWS Outage: The Cloud’s Shining Moment outlining the differences between the “design for fail” model and the “traditional” model. The traditional model, among other things, has high-availability and DR characteristics built right into the infrastructure and these features are typically application-agnostic (a couple of years ago I wrote a big document on the various alternatives for HA and DR of virtual infrastructures if you are interested). George nailed down the story very well and the story is that there are a couple of different philosophies at play here. I don’t call these two models “design for fail” and “traditional” though. I call them TCP-clouds and UDP-clouds. Let’s look at a summary of the characteristics of these two protocols.

In the context of cloud resiliency this is what that means:

AWS uses a UDP-cloud model because it doesn’t guarantee reliability at the infrastructure level. AWS essentially offers an efficient distributed computing platform that doesn’t have any built-in high availability services. The notion of Availability Zones and Regions is often misunderstood since the name may imply there is high availability built into the EC2 service. That’s not the case: AWS suggests to deploy in multiple Availability Zones simply to avoid concurrent failures. It’s mere statistic. In other words, if you deploy your application in a given Availability Zone, there is nothing that will “fail it over” to another Availability Zone as part of the AWS service (RDS is a vertical example that does that for MySQL but I am instead talking about an application-agnostic service that does that for every application regardless of the nature).

Since I am not able at the moment to write a structured thought around this complex matter, let me write down mixed and random thoughts, opinions and questions to try to make you think. I am giving you some food for thoughts. As far as answers, call me when you find them please.

Isn’t this “design for fail” theory a step back?

What we have seen in the last decade was a trend where we were able to remove the non-functional requirements complexity from within the traditional OS and put them down into the “virtual infrastructure” (arguably the backbone of any IaaS cloud). This is the point I was trying to come across during this VMworld 2007 breakout session 4 years ago. And what we are saying now is that we should put that logic back into the application (not even the Guest OS)? I thought the trend I have just described was quite successful and one of the many reasons of the success of virtualization deployments. Are we now questioning it? My idea is fairly simple although I am open to be challenged: developers focus on functional requirements, IT focuses on non-functional requirements (which includes resiliency and reliability among other aspects). If interested, you can download the full deck here. Note I did that presentation before joining VMware so, if you think I am biased, well I am biased just because I bought into that school of thought long before I was on the VMware’s payroll system.

Excuse me? What did you say? NoSQL… to whom?

In his post George suggested exploring NoSQL solutions. Not a bad idea however, other than the risk of losing transactions that he was mentioning, I’d say 95% of the customers I have been working with so far would look at me strangely and they’d ask: “what do you e x a c t l y mean by NoSQL? Is it a bad word?”. Let’s be honest folks: this is not mainstream. If we want to create a cloud for an elite of people I am fine with that. However I am convinced one of the key values of an IaaS infrastructure is, among others, providing a cloud-like experience (pay-as-you-go, elasticity, etc) to traditional workloads. I am not philosophically against the idea of re-architecting applications, however I am also convinced that, for one person thinking about writing a brand new Ruby application for a UDP-cloud leveraging NoSQL (pardon me?)… there are at least 1.000 poor sysadmins trying to figure out how to live with their traditional applications.

Can you afford a personal Chaos Monkey?

Some of the AWS customers developed tools to test the resiliency of their applications. Do you remember the old good HA and DR plans? IT people would walk into the server room to power-off servers and eventually the entire datacenter to simulate a failure and see if their HA and DR policies were working properly. If everything was good applications could survive the failure (more or less) transparently. This is what a Chaos Monkey tool does, but with a different perspective: these are software programs that are designed to break things randomly (on purpose) in order to see if the application itself is robust enough to survive those artificially created infrastructure issues in the cloud. In a TCP-cloud it would be the cloud provider to run traditional tests to make sure the infrastructure could self-recover. In a UDP-cloud it is the developer to run these Chaos Monkey tests to make sure the application could self-recover since it’s been “designed for fail“. Now, my take is that if you are Netflix or the like of Nasa and JPMorgan (these two are just examples of big organizations – not even sure if they are on Amazon) then you may have enough motivation and business reasons to re-architect your application for a UDP-Cloud and create your own Chaos Monkey to test your “design for fail” deployment. Certainly at Netflix they know what they are doing and in fact they seem to not have been impacted by this AWS outage. But if you are these guys do you think you have bandwidth, knowledge and time to re-architect the application and test it for failure? That AWS forum discussion showed up during the 4 days debacle and it deserves a proper copy and paste just in case it gets lost:

< Sorry, I could not get through in any other way. We are a monitoring company and are monitoring hundreds of cardiac patients at home. We were unable to see their ECG signals since 21st of April.

> Man mission critical systems should never be ran in the cloud. Just because AWS is HIPPA certified doesn’t mean it won’t go down for 48+ hours in a row.

< Well, it is supposed to be reliable…Anyway, I am begging anyone from Amazon team to contact us directly.

This is shocking isn’t it? Try to argue with them about NoSQL and “design for fail“. They barely probably understand the notion of Availability Zones and Regions. Don’t get me wrong. It’s not these people’s fault. They are not in the business to re-architect an application to be written with reliability in mind, they are in the business of helping their patients. Sure you can argue with them that it was their fault if they failed. But the net of this story is that they are not going to re-architect anything nor write a Chaos Monkey. When they realize what happened, they will look for a TCP-Cloud.

Design for fail: philosophy or necessity?

I hope you’ve got at least to this point because this is my biggest struggle at the moment. The more I read about suggestions to design applications for fail the more I miss whether these suggestions are tactical or strategic. In other words, are you suggesting to design for fail simply because that’s the way Amazon AWS works today (but you’d rather use an Amazon TCP-cloud if that was available)? Or are you suggesting that, in any case, you should design an application for fail because you are happy to deal with a UDP-cloud and that’s how every cloud should behave? Are we saying that it’s strategically and philosophically better to have developers deal with application high availability and disaster tolerance because that’s what makes sense to do? Or are we saying we need to do this because that’s the only option we have on Amazon AWS (today) and there is no other choice? I know it may sound like a rhetoric question but it’s actually not. Perhaps we need both models?

You don’t like the noise coming from the other apartments? Buy the entire building!

This isn’t related to the outage and the resiliency of the cloud but it relates to the overall TCP-cloud Vs UDP-cloud discussion. Similar to the “design for fail” there is the “deploy for performance” thread going on. In a multi-tenant environment (a must-have to achieve economy of scale and elasticity) there is obviously contention of resources. In an ideal world I’d like to be able to buy virtual capacity for what I need and have a certain level of guarantee that that capacity (or at least a contracted part of it) is always available for me. There are of course circumstances where I can trade-off performance and availability of capacity for a lower cost, but there are other situations where I cannot trade that off. A TCP-cloud should (ideally) be able to deliver that guarantee. A UDP-cloud works in best-effort mode and typically leverages statistical law to fight contention. This is the statistical assumption: not all users running on a shared infrastructure will be pushing like hell at the same time (one would hope – finger crossed).

So what do you have to do if you are running on a UDP-cloud? You keep the other people out of your garden.

I think Adrian is a genius but I don’t agree with his point of view :

“…you cannot control who you are sharing with and some of the time you will be impacted by the other tenants, increasing variance within each EC2 instance. You can minimize the variance by running on the biggest instance type, e.g. m1.xlarge, or m2.4xlarge. In this case there isn’t room for another big tenant, so you get as much as possible of the disk space and network bandwidth to yourself.”

“…busy client can slow down other clients that share the same EBS service resources. EBS volumes are between 1GB and 1TB in size. If you allocate a 1TB volume, you reduce the amount of multi-tenant sharing that is going on for the resources you use, and you get more consistent performance. Netflix uses this technique, our high traffic EBS volumes are mostly 1TB, although we don’t need that much space.”

“If you ever see public benchmarks of AWS that only use m1.small, they are useless, it shows that the people running the benchmark either didn’t know what they were doing or are deliberately trying to make some other system look better.”

The last sentence is like saying that, if you buy a new apartment and then complain about the big noise coming from other apartments, it’s your fault: you should have bought the entire building and enjoyed the silence! Hell Adrian, I say no! There must be a better way.

I think there must be rules in place to keep the noise at an acceptable level and if there is someone trying to scream all the time someone should “enforce” silence without having you to buy an entire building to cook and sleep in peace. That’s how it works in real life, that’s how it should work in the cloud. In my opinion at least.

In cloud terms I’d be ok if what I was buying always delivers a contracted baseline as a guarantee and then can burst (I said burst Beaker, not cloudburst) to higher throughput if there isn’t contention. What I would NOT be ok with is no baseline at all so what I get is no predictable performance all times. BTW note that Amazon made a step forward in the right direction a few weeks ago announcing the availability of what they call dedicated instances. This is an attempt to solve the noisy neighbors problem. However in doing so they did trade off multi-tenancy (hence the higher cost of such a service).

For the records I have to say that I don’t think there is a single public cloud at the moment delivering such a fine grained QoS across all subsystems on rented resources. This is a generic discussion about TCP-clouds and UDP-clouds and if you interpreted it like a vCloud Vs AWS shootout you are mistaken. In fact I think George gave vCloud too much credit in his blog associating it to the “traditional” datacenter model. There is a gap between what we can deliver, in terms of non-functional requirements, with a raw vSphere deployments and what we can deliver with a vCloud Director 1.x implementation. I am not hiding this by any means, in fact you can read here (the post but more importantly the comments) what I had to say about this. Having this said I believe VMware has a vision to fill that gap and create a true TCP-cloud. Last but not least I don’t see why a VMware service provider partner shouldn’t be able to implement a vCloud-powered UDP-cloud if need be.

PaaS and Design for fail?

If I struggle with IaaS clouds (and I do), go figure with PaaS clouds. To me PaaS is all about moving the level of abstraction at a higher level. IaaS is all about hiding infrastructure details. PaaS is all about hiding infrastructure and middleware details. In a PaaS you can upload your WAR file and that’s it. It’s the PaaS cloud provider that is going to deal with the complexity of setting up, managing and maintaining the middleware stack that can interpret that WAR file (for example). Fundamentally the developer should focus (even more than with IaaS) on the functional requirements of the application and let the cloud provider deal with the non-functional requirements aspect of it. Last time I checked HA and DR were still part of the non-.functional requirements domain. Note that, ironically, it may be easier for a PaaS cloud provider to build out-of-the-box resiliency given the nature of the interfaces they are exposing. Amazon is half way through that already with their RDS “My-SQL as a service”: they already offer automatic failover across Availability Zones and they would just need to extend this failover support across regions (this would have helped with the recent failure by the way). So, if my theory is sound, that means that if you are architecting your application for PaaS you shouldn’t design for fail. Upload your WARs, create a db instance on the fly and you are done. The cloud provider will figure out how to failover to the next server, to the next datacenter room or to another geography should a problem occur at any of the given levels.

So why isn’t Amazon offering resiliency and reliability as part of their cloud services in the end?

After all they offer other non-functional requirements such as automatic scaling of applications through tools such as Autoscaling. So why would Amazon offer auto-scale services and shouldn’t offer an automatic, agnostic, infrastructure-level recovery service across Availability Zones (or even better across Regions)? Guess what. It is at least two order of magnitude easier to instantiate a new web server and add an IP to a load balancer than implementing a (reasonably performant) backend traditional database that can geographically fail over without losing transactions in case of a disaster. Dealing with stateless objects is a piece of cake. Try to deal with statefull objects if you can.

I am sure Amazon doesn’t think that dealing with autoscaling is something the cloud should do for developers whereas dealing with reliability and DR is something a developer should do on his/her own. What do you think? My speculation is that they are simply not there yet. As easy as it sounds. But don’t be fooled. Amazon is full of smart people and I think they are looking into this as we speak. While we are suggesting (to an elite of programmers) to design for fail, they are thinking how to auto-recovery their infrastructure from a failure (for the masses). I bet we will see more failure recovery across AZs and Regions type of services in one form or another from AWS. I believe they want to implement a TCP-cloud in the long run since the UDP-cloud is not going to serve the majority of the users out there. Mark my words. I’ll have to link to this blog post once this happens and I’ll have to say “I told you” (I hate this). And that is only going to be a good thing because developers will start again to focus on functionalities and IT the cloud will continue to focus on making sure those functionalities are (highly) available.

As I said, just food for thoughts. If you find definitive answers, please let me know.

Last but not least this is a good time to remind the disclosure of my blog (courtesy of a big copy and paste from the Sam Johnston‘s blog): “The views expressed on these pages are mine alone and not (necessarily) those of any current, future or former client or employer. As I reserve the right to review my position based on future evidence, they may not even reflect my own views by the time you read them. Protip: If in doubt, ask.”

Massimo.

The 93.000 Firewall Rules Problem and Why Cloud is Not Just Orchestration

Massimo — Wed, 30 Mar 2011 20:14:19 +0000

A few days ago I was in a very interesting meeting with a big Service Provider in Europe and I heard a lot of interesting comments. I’d like to quote the best that I heard which was “Oh a portal? Oh not another one… we have many of them already!” but this will open up a different can of worms so I am not going to talk about this now. What I am going to talk about relates to another comment someone made in the middle of the meeting which was “…there is a firewall with 93.000 rules configured“.

I can’t say to be a security expert by any stretch, however they sound a lot to me. This was confirmed by someone with a lot of background in this area saying that “… they are a lot but the record is a Cisco device (somewhere on this earth) with 750.000 rules“. Suddenly someone else jumped into the discussion asking “…and what happens when you fat finger rule #457.986?“. I thought this was a joke (however I am not sure).

Before we make any step further, let’s try to dump, in a picture, the layout of this scenario (at a very high level):

Basically the idea, pretty common these days, is that you have a multi-tenant virtual infrastructure with a number of VMs running on top of it. These VMs belong to different customers and, by means of standard layer 2 segregations (VLANs if you will), you keep them separate. The big (BIG) firewall at the bottom of the picture is the one that is holding the 93.000 rules that govern how these workloads talk to each others. By the way this doesn’t appear obvious in the picture but each customer could (and will!) have more than one single VLAN because that’s how it works in this world (see below). So 93.000 firewall rules is just the tip of the iceberg… there are other problems these Service Providers are dealing with which are, for example, the sprawl of VLANs – along with all sort of issues associated with that.

So why is this a problem for an IaaS cloud? I think there are at least a couple of dimensions to this problem.

Manageability, serviceability and scalability

The first dimension relates to “how on earth can you deal with such a beast?”. How do you manage this firewall but, even more importantly, how do you troubleshoot it? That’s why I am not sure that the person that referred to the “fat finger” problem was really joking. Again, my background is not security so bear with me and please advice where I am missing something. However, whenever I mention situations like these to people that do have a security background their typical reaction is:

they laugh first….
…and scratch their head then.

So there must be something wrong somewhere, I think.

For sake of clarity, I am not bashing the firewall administrator that configured 93.000 in that box. I think that the problem is how networks (and related security) have been working until now and the associated “best practices” we built in the last 10 years. One could write a book on this but, in a nutshell, the way it works is that, to secure “services”, you need to create layer two domains (aka VLANs) that you connect by means of a firewall. Depending on what you need you may have to create subnet-based rules and/or IP-based rules. Take this approach and apply it to a Service Provider with thousands of customers each with a certain amount of “services” deployed, and before you realize what’s going on you get to thousands of firewall rules in a blink of an eye.

End-user self-service

The other dimension of the problems we are discussing strictly pertains to self-service, a key concept of paramount importance in all cloud related discussions. This is a pattern I have seen over and over again at every single Service Provider I have met so far: the usage of a central monolithic firewall to serve multiple different tenants doesn’t allow the SP to create (easily) a self-service experience for the user. Why? Simply because the more complex and more critical the object (whose functionalities you want to expose to the end-user) becomes, the more complex and critical the tool that mediates its access needs to be. You could solve this problem by using a dedicated physical firewall per each of the customers the SP is hosting. That would reduce the complexity and the criticality to a level for which the effort of the SP would be as low as telling the customer “Here is how to access the device as root“. Between the lines you could read “Screw it up and only your own organization will be screwed up, I don’t care“. It sounds great but this isn’t very scalable nor manageable obviously. Do you deploy a new physical firewall every time you get a new customer? Not the promise of cloud I’d say if cloud is really about agility, scalability, pay-per-use and the list of attributes goes on. These attributes have, in fact, very little to do with the option of deploying a new physical device on-the-fly when needed.

So what did all these SPs do when they stood up their so called… “clouds“? They created a portal (probably one of those many we were talking about at the beginning) where they gave some self-service capabilities to do basic and simple stuff (such as VMs provisioning) and they implemented a ticket system for more advanced stuff (such as creating network security rules for the workloads they were provisioning). Not very different from how you’d do it with a traditional hosting solution you may think. Well that’s one of the reasons many people refer to this practice as “lipstick on a pig” (i.e. take a hosting solution, put a cloud label on it and sell it as if it was a cloud).

The role of the orchestrator

I always say that orchestration is not cloud but cloud needs orchestration. Will orchestration alone help solving the problems we are discussing here? I don’t personally think so. I see orchestrators more like tools that are supposed to solve operational issues (especially at the level of scaling a cloud infrastructure requires) not like tools that can fix broken architectures. If you take a stone and clean it, it doesn’t become a gold nugget automagically. It becomes a cleaned stone. Same thing goes for cloud. If you take a “junk architecture” and you orchestrate it, does it become a “great architecture“? No, it becomes an “orchestrated junk architecture“. Better than having to deal with it manually… but still “junk“.

Don’t get me wrong, I do think that orchestration is key and you can’t have a cloud without (at least a certain degree of) orchestration. However don’t think that a properly architected cloud is just your “legacy” stuff with an additional kilo of orchestration workflows and a nice new portal (“Oh a portal? Oh not another one… we have many of them already!“).

Is there a way out?

Yes there is (I think). I believe there is a shared feeling in the industry, at this point, that an architecture as shown in the picture below is the way to go forward. So what is that vFW (aka virtual Firewall) below? At VMware we call it vShield Edge. Other vendors may call it differently. Other vendors don’t have anything like this today (so expect some level of bashing from their sales rep in the field) but they may end-up having it down the road (expect some level of embarassement from the same sales rep that bashed this approach in the past). We started shipping vShield Edge less than a year ago but we have seen a huge number of people experimenting with an approach like this for years. Just recently I have met another SP that said that 2 years ago they started looking into something like this using virtual appliances from Vyatta. Just recently I wrote about a small business partner getting into the “cloud” from a provider perspective and using the same model/architecture without anyone telling them this was “the right” model: they figured this out themselves based on the challenges they were dealing with! And if this isn’t enough to convince you that there is a trend here, look at what Amazon has started to pitch a couple of weeks ago.

So what’s so neat about this model? The idea is pretty simple: instead of using a monolithic physical firewall outside of the virtual infrastructure domain, you can deploy different virtualization-aware firewalls that are essentially backing the same VLAN(s) but do that in a more flexible and agile way. Other than simplifying the complexity of a single object configuration (the “93.000 rules” problem) you also gain easy self-service through administration delegation. As we have said at the beginning it is difficult to get controlled access to a shared device. However if you create a virtual device that is only supposed to “rule” access to given VLANs dedicated to a customer… you can easily delegate full access for that virtual device to that specific customer. This is at at the core of the vCloud Director self-service capabilities. In many cases you’d still want to have the traditional physical device for data center level protection against external attacks and advanced firewall features that these virtual firewall may be missing today. However the complexity of its configuration would be drastically reduced because the workloads security rules would be managed directly on the virtual firewall devices.

Can we do even better?

We could do something better, yes! What we have been talking about so far is, basically, all about keeping the very same number of VLANs and firewall rules.. and spread these rules across virtual firewalls. This solves a lot of problems when it comes to self-service for example (delegation of the entire device) and scalability (just deploy another virtual appliance when there is a new customer) but it doesn’t really solve itself the problem of VLAN sprawl and the 93.000 firewall rules (although they are now segmented in different and dedicated security domains per each customer). VMware has other technologies that may help to address these other problems.

The first one is called vCloud Director Network Isolation (vCDNI) in vCloud parlance or vShield PortGroup Isolation (PGI) in vShield parlance. It’s, basically, a technology that allows you to virtualize a VLAN. This allows different customers to be assigned dedicated vDS PortGroups that represent separate layer 2 domains… yet sharing the same VLAN ID. We use a technique called MAC-in-MAC to implement this. Kamau just posted a very interesting blog on how this works. You can read more here if you are interested. This technology is already available and fully integrated in vCloud Director so you can use it today if you want to.

There is another elegant method to solve the VLAN sprawl problem and, more specifically, the proliferation of rules you have to create in the firewall(s). This can be achieved with another vShield technology called vShield App. Think of vShield App as a vDS port-based firewall where you can say “this vNic can talk to this other vNic over this particular port“. The vNics in question are connected to the same vDS PortGroup (i.e in essence one single layer 2 domain). So imagine having a single network segment where you can create rules that mimic the deployment of a DMZ, an Application security zone, a Database security zone, etc etc. Instead of using three VLANs (in this example) you could use one and have this segmentation happening at the vDS layer via vShield App rules. The cool thing about App, in my opinion at least, is that it supports both the typical 5-tuple firewall rules as well as it works with traditional vSphere constructs such as datacenters, clusters, resource pools and things like that. So that you can say that all VMs that are in this “container” can only communicate with VMs that are in this other “container” over a specific port. This way you can change IPs, add/remove VMs from the containers and the security policies will still apply simplifying and reducing the “93.000 rules problem“. For sake of clarity this vShield technology (App) isn’t integrated (today) with vCloud Director but I hope you see a trend here.

Now imagine combining vCDNI with vShield App. You could – potentially – use one single VLAN to support multiple tenants, and within each “virtual VLAN” you can create rules that represent multiple security zones effectively mimicking DMZ’s, back-end’s etc.

Conclusions

While I focused a lot on the products I am working with at the moment, the message that I wanted to pass along with this post is that the current network security model seems to be broken, in a big way. Especially if you think about it in the scope of cloud-like deployments where agility and self-service are big mantras. There are alternative architectures that are proving to be better in this context and there is a range of products that can implement that new architecture. I mentioned vShield and vCloud Director but you can use other products if you want… as long as you fix that junk! The other point I was trying to make in this post is that orchestration itself cannot fix a bad architecture and these two topics (architecture and orchestration) should really be considered two separate workstreams when you design your cloud infrastructure. Once again, orchestration is not the means by which you can fix a bad architecture layout.

Now I talk like if I knew what I was saying. Funny.

Massimo.