Cloud Native Options – Personal experience when moving from Serverless to Containers

Introduction

Over the last months I have migrated VMchooser from a purely serverless implementation towards a container based one. The main reason for me to do this, was (like always) the learning effect that comes with such a refactoring. This post will run you through the various aspects that I encountered and hopefully give you a better understanding between both implementation options of a CloudNative architecture. I will divert a bit from the pure “X vs Y” comparison and also touch topics that typically come with the business discussions when thinking about both in terms of a strategic approach.

 

CloudNative?

First of all… In terms of semantics, there are some definitions floating around. When you look at the CNCF, then it seems to solely revolve around containers. Though when you look at Azure, the definition broadens up a bit… In reality, it depends a lot on the context of the person/organization you are speaking too. So do not be surprised if for some organizations the scope of CloudNative is “limited” to containers. Where for others this might be about leveraging the PaaS cloud services in a “Serverless” manner.

 

It kinda makes me think about the following “cartoon” (Source ; Simon Wardley) from a few years ago ;

Anyhow, be aware that there are different views on the semantics of “Cloud Native” and be crisp on your own when making conscious decisions.

 

Strategic Design Principle ; Portability

The design principle of software portability is high on the radar with about every organization I talk to. Here I always highlight two dimensions to take into consideration ;

• Startup Costs vs Exit Costs => Look at TCO

The first aspect is about the TCO. A very common discussion to have when talking about Serverless (think things like Functions-as-a-Service for instance) is that people are afraid to go this way due to a possible vendor lock-in. Despite that you could even run for instance Azure Functions inside of a container, and virtually anywhere for free, is already a hint of the opposite… I do relate to the sentiment.

Now if we put the technical aspects aside for a minute, and look at the business dimensions. In a typical software project, and if we look at it from a 10-mile high view, then we have several phases. The premise of Serverless is that you can reduce your start-up time/cost by a lot, and immediately gain business value. The downside being that in case you are fed up with the service, you need invest in terms of exit project. Where the true cost of the exit strategy is actually a part of the startup cost of a typical project. You will now need to foresee things which the platform took care off.

If we take a look at a platform agnostic approach (for example containers), you will have a startup cost before you are reaching any business value. This comes at the gain of having the potential of a much more portable system. This reducing the potential exit cost you have when moving platforms. In the very simplistic visualization above you can see a bit the difference between both. Where I am ignoring the potential differences in implementation and operations. Though I guess by now you get the gist of the mindset behind looking at the TCO. As people often focus on the exit part, and “forget” the business impact on the other phases.

 

• Portability is not limited to containers. Think in terms of endpoints (interfaces) of your components.

The second aspect is thinking in terms of endpoints. Sometimes I encounter organizations who swear to avoid anything PaaS-related. One example I recently encountered was an organization which swears by doing everything on IaaS. Like for instance running their own container platform on top of Azure. Where I have respect for organizations who want to do this, one must admit that for about 95% of the organizations there is zero business value in this. The “endpoint” here can be seen as a “Certified Kubernetes“, and focus on the cross platform/service features in this. Another way is to look at the endpoint like a MSSQL, MySQL, MongoDB, MariaDB, Postgresql, etc connection string. If you run any of those on IaaS yourself, or if you take a managed service… You have a portable endpoint. In essence, every cloud provider has a managed offering in this and you can run it On Premises too. So you avoid the “lock-in”, where you gain the advantages of having the managed service.

The only thing I would like to highlight here is that all cloud-native concepts can adhere to the principle of portability. As always, please focus on the business gain and do not be over zealous on the technical side when designing your business workloads.

 

VMchooser Cloud Native Architecture ; before and now

As mentioned earlier on I migrated VMchooser from a pure serverless implementation towards a container based implementation ;

• Serverless ; This architecture consisted of leveraging Azure API Management & Azure Functions for my backend, and using Azure Frontdoor and Azure Storage for my frontend (static website => Single Page Application), where the data resided in Azure CosmosDB (MongoDB API).
• Containers ; With the new architecture I moved the Azure Functions inside of a container, and the same for my frontend. The code remained the same, though packaged inside of a container. In terms of a container platform, I am leveraging Azure Kubernetes Services (and Azure Container Registry). The data still resides in Azure CosmosDB.

You might now ask… Why didn’t you put the data into the container platform too? My personal opinion is that data persistence is kept outside of a container platform. Again, this is for most of the organizations. If this is key to your business, and your maturity level in terms of container platforms is excellent, then I can follow your reasoning in doing the opposite. Though most organizations are still struggling with the basics of container platforms… So let us not start with doing the most advanced things first. Next to that, in terms of endpoints, I still adhere to the portability principle as I can run a MongoDB instance virtually anywhere. Now it is hosted for me and there are a lot of aspects I simply do not need to care about.

 

Join the Self Support Group, or “What I learned in migrating”

Right off the bat… I wanted to do things “the right way”. This meaning that I wanted to have a deployment that I considered “enterprise grade”. Meaning that I introduced the following things ;

• Fully Automated CI/CD
  • Infrastructure ; All services are managed from a Terraform powered pipeline in Azure Devops
  • Frontend / Backend ; There services are build, tested, packaged (Helm), published (ACR) and deployed (AKS) from Azure Devops.
  • MongoDB ; A daily ETL flow is also run powered by a git based repository on a daily/weekly basis.
• Service Mesh with Istio
  • In all honesty, the value of a service mesh took me quite a bit of time to fully grasp. This moved helped me a lot to understand the value. Where I was actually spoiled that the typical Azure services actually took care of several of the capabilities delivered by Istio. For example ; CORS… When moving the functions to a container, suddenly I needed to take care of things like CORS, SSL, Vhosts, etc. This is where Istio helps a lot, as it pushes these concerns outside of the application workload. Which suites the typical enterprise organization more.
  • Though this does at A LOT of complexity. Do not over estimate how easy things like Azure Functions, Azure App Service, Azure Frontdoor, etc make your life. With the gained control, you also get a lot of additional ownership on your hands. In order to succeed here, you need to incorporate a given learning curve in managing this. Which does not help in your TCO view of things.
  • The observer-abilities added by Istio with things like Jaeger and Kiali are awesome however. Though again, the simplicity of Azure Application Insights outmatches these immediately.
• Blue / Green deployments on both Infrastructure & Application level
  • Doing blue/green on application level is “easy”. Though I wanted to be as flexible on the infrastructure layer too… I was able to manage this in the end, though it was not as straight forward as you would want.
  • Next to that, Istio helps with the blue/green on application level. Though there I actually went for a rolling upgrade for existing clusters, and blue-green over clusters. I could have gone for the blue/green on the cluster level too. Though this would have made the capacity management harder.
•  Helm Packages
  • The best thing I discovered was that Azure Container Registry also supports helm charts!
  • That being said… Starting off with Helm charts also has a given learning curve. It is a very straight forward configuration process. Though when thinking in terms of automating it, you really have to master it. Just like with Istio, it adds to the learning curve!
• Scalable CosmosDB
  • The tough part of CosmosDB is that you need to consider sharding from day 1. Though the fun part is… if you did this, then scalability is virtually limitless.
  • In the ETL flow, at the start we scale up CosmosDB to improve the duration of the ETL job. And at the end we scale it back down. From a cost optimization aspect this is awesome!
• Costs…
  • Serverless wins hands down from containers. Where I used to run the environment on less than 100$ per month, this has now surged to about 8x as much. Here I hopefully still have some optimizations to go. Though from a 1:1 comparison, the costs are 8x the ones I had before. Where I see this as a technical learning path for myself, it is also very good to be aware of this for your overall TCO view.
  • vs Capabilities ; One thing to note here however, running Azure Functions for a very complex micro-services architecture will be a challenge. Things like Kubernetes, spring cloud and service fabric will outmatch Azure Functions from a given complexity. There are some other traits/effects you need to consider, like ; Cold Start during scale-out, Designed for Event Driven, …

 

Looking for some examples?

Want to see how I put all the pieces of the puzzle together? Check out VMchooser’s github repositories!

 

Container platforms everywhere!

There are a lot of different container platforms out there. They all promise the world to you… My personal advice in this space is twofold ;

• Managed Platforms ; Some people see a Kubernetes platform as the next hyper-visor. Which till a given extent is actually true… That begin said, Kubernetes is a very complex thing. Really… And if we are being very blunt about this, there is -zero- business value in managing such a platform (unless you are a very specific ISV / cloud provider). Please do not manage your own container platform (either on IaaS or with things like OpenStack). Invest your time on the business side that actually delivers value to your organization!
• Kubernetes Certified ; For those who experienced the Java J2EE times, this might sound very familiar… This also promised portability across platforms. Though one deviation from the common J2EE specification got you “locked-in” to that specific J2EE application server. The same logic applies to Kubernetes! Try to stay within the boundaries of the Kubernetes Certified distributions. Otherwise you will not have the full portability you would have hoped for.

 

Closing Thoughts

The migration project as -very- insightful for me. In essence when thinking about portability, you need to think about the TCO from a business side and consider the endpoint/interface dimension from a technical side. Do not underestimate the “invisible value” serverless brings to the table. It really helps you kick-start things without losing time to implement these common (across organizations/industries) traits. Where you might think that I was a bit harsh on the container side, it is a technology that has helps A LOT on the infrastructure side. It has become the default/common specification to package workloads and make them more portable. Containers have shown a way to tackle the whole premise of “It runs on my machine”. Which is awesome! My biggest advice in terms of containers, is to leverage a managed container platform.