The incestuous relations among containers orchestration tools

This is going to be a short and (somewhat) visual blog post where I want to discuss the absolute madness that is going on in “container land” (for lack of a better characterization).

This time I am going to try to use quotes, tweets, slide screenshots as much as possible and avoid my usual boring text rants. I believe you can draw your own conclusions in the end (but I’ll give you a hint).

If you thought this previous post of mine was a mess, wait to read watch this.

First off I’d like to thank Ken for suggesting a proper title for this post to avoid me sounding like a pervert:

Then I’d like to quote what Google’s own Kubernetes master Jedi Kelsey Hightower thinks about the container management war that is going on:

And yes, we are still in the early days of this gold rush, just in case you were wondering.

Last but not least another tweet that nailed it (with a funny joke fact):

Now you may think that the problem we are facing is the proliferation of container management solutions to pick from?

You wish it was that easy.

It’s way worse than what you think: it’s getting “incestuous”.

Container management vendors (or projects) are taking an interesting path these days.

Instead of trying to position themselves as the best and most viable containers orchestration solution, they are starting to position themselves as the foundational orchestration solution on top of which other container management solutions could run.

Yes, you read it right. The containers management industry complexity just got squared!

Instead of having to pick among 25 different alternatives, you now have a choice of (25 x 24 =) 600 permutations to choose from! How fun?!

But seriously, this is a game being played primarily by the 3 or 4 most visible vendors/projects (namely Docker, Mesos, Kubernetes and CloudFoundry) so the good news is that the permutations are much less than 600.

What a pity.

Some of them are more “serious” than others when it comes to “I want to run all the other container managers, and make donuts while I am at it”.

I say Mesos is king here. They would like to be the center of the universe. A few examples below.

They want to run Docker Swarm on top (of Mesos):

http://www.slideshare.net/Docker/building-web-scale-apps-with-docker-and-mesos (slide 23)

They want to run Kubernetes on top (of Mesos):

http://www.slideshare.net/sttts/kubernetes-on-top-of-mesos-on-top-of-dcos (slide 8)

They (also) want to run CloudFoundry on top (of Mesos):

"The way CloudFoundry-Mesos works right now—in its very early stages—is to replace the native Cloud Foundry Diego scheduler with a Mesos framework, CloudFoundry-Mesos. Doing this does not affect the user experience or performance of other Cloud Foundry components, but would let Cloud Foundry applications share a cluster with other DCOS services without worrying about resource contention."

https://mesosphere.com/blog/2015/12/15/cloud-foundry-dcos/

(https://github.com/mesos/cloudfoundry-mesos)

I have just had a shudder.

Interestingly, when Docker itself presents at MesosCon they are ok with Docker Swarm being “boxed and limited” to one of the many Mesos frameworks:

https://www.youtube.com/watch?v=qUViQAu2Bw0

This is a common pattern in the industry these days (and a good filter to use when in doubt).

Vendor A and Vendor B overlaps. When Vendor A gets a slot at a Vendor B event, Vendor A concedes Vendor B to “have control” (in this case “having control” means being the foundational element of the stack).

Vice versa when When Vendor B gets a slot at a Vendor A event, Vendor B concedes Vendor A to “have control” (if and where applicable of course).

For instance, the example above is not what Docker advertise when they are in charge of the message.

When they are (in charge of the message) what they say is the exact opposite (that is: Mesos-Marathon on top of Docker Swarm):

"This project contains Docker Compose files used to easily deploy distributed containerized applications. Currently the project contains Docker Compose files for Kubernetes and Mesos-Marathon... The rationale behind this is that Swarm is lightweight enough to deploy additional orchestration tools on top."

(https://github.com/docker/swarm-frontends)

But it’s getting even more complex and sophisticated than that. To the point that Docker is trying to “steal” historical Mesos frameworks. See (and read!) this:

(https://blog.docker.com/2016/03/docker-welcomes-aurora-project-creators/)

Translation: “Now let’s get rid of Mesos entirely and just run Mesos frameworks directly on Docker Swarm!”

I even attempted to build a table to summarize what you could run on what.

Warning: it’s more of a joke than anything just to point out the level of ridiculous madness we are at.

	K8s on	Mesos on	Swarm on	CloudFoundry on
K8s	N.A.	No	No	No
Mesos	Yes	N.A.	Yes	Yes
Swarm	Yes	Yes	N.A.	No
CloudFoundry	No	No	No	N.A.

Interestingly, it shows who is leading this confusing “game” (i.e. Mesos and Docker) and who is being pulled into this “game” (K8s and CloudFoundry).

In conclusion, if you are an average person (like me) trying to figure out what’s going on, good luck.

Please come back in 5 years when (perhaps) the dust has settled a bit. Right now, it’s just pure madness that only makes sense to a (limited) bunch of people.

What do YOU think?

Massimo.