Serverless On-Demand Scaling : Pushing the pedal when you need it…

Introduction

A lot of workloads are driven by peak consumption. From my experience, there aren’t the amount of workloads that have a constant performance need are in the minority. Now here comes the interesting opportunity when leveraging serverless architectures… Here you only pay for your actual consumption. So if you tweak your architecture to leverage this, then you can get huge gains!

For today’s post, I’ll be using VMchooser once again as an example. A lot has changed since the last post on the anatomy of this application. Here is an updated drawing of the high level architecture ;

Underneath you can see the flow that’ll be used when doing a “Bulk Mapping” (aka “CSV Upload”). The webapp (“frontend”) will store the CSV as a blob on the storage account. Once a new blob arrives, a function will be triggered that will examine the CSV file and put every entry onto a queue. Once a message is published onto the queue, another function will start processing this message. By using this pattern, I’m transforming this job into parallel processing job where each entry is handled (about) simultaneously. The downside of this, is that there will be contention/competition for the back-end resources (being the data store). Luckily, CosmosDB can scale on the fly too… We can adapt the request units as needed; up or down! So let’s do a small PoC and see who this could work…

Continue reading “Serverless On-Demand Scaling : Pushing the pedal when you need it…”

Advertisements

XenData : When a Windows Volume suddenly gets hybrid towards an Azure Storage Account

Introduction

Today I took the Xendata Cloud File Gateway out for a spin… Why? This little piece of software allows a windows volume to be extended by an Azure Storage Account. And from a technical level, we are talking about blob storage here. So you can leverage hot & cold storage, and even archive storage in the long-term. Imagine that huge exploding file server? Suddenly we can extend our typical Windows File Server with an seamlessly unlimited cloud tier. Whoppah!

Let’s take a look shall we!

Continue reading “XenData : When a Windows Volume suddenly gets hybrid towards an Azure Storage Account”

Azure : Finding out the available metrics for autoscaling virtual machine scale sets

For those who have been test driving the autoscale on the virtual machines scale sets… You probably have run into the situation where you wanted to go beyond the quickstart examples!

A quick tip on how to find out which Metrics are available for your autoscaling ;

So now you have the list of metrics which you can use to tweak the vmss-autoscale templates (for example ; https://github.com/Azure/azure-quickstart-templates/blob/master/201-vmss-ubuntu-autoscale/azuredeploy.json )

Autoscalling on Azure : Scale Up … & Down!

Introduction

Today we’ll be building further on the steps we took in the previous post ; “Autoscaling Docker hosts on Azure with Virtual Machine Scale Sets & Rancher“.

 

What will we be doing?

  • Deploy our VMSS with a scale-up & a scale-down configuration
  • Stress our VMSS so it will scale-up
  • Remove the stress so our VMSS will scale-down

Continue reading “Autoscalling on Azure : Scale Up … & Down!”

Autoscaling Docker hosts on Azure with Virtual Machine Scale Sets & Rancher

Introduction

A while back Mark Russinovich announced the public preview of the “Virtual Machine Scale Sets“;

VM Scale Sets are an Azure Compute resource you can use to deploy and manage a collection of virtual machines as a set. Scale sets are well suited for building large-scale services targeting big compute, big data, and containerized workloads – all of which are increasing in significance as cloud computing continues to evolve. Scale set VMs are configured identically, you just choose how many you need, which enables them to scale out and in rapidly and automatically.

07ed41cb-fa01-409f-b84b-b912b13253bf

So here we have a cloud service that would enable us to autoscale our hosts in terms of the load of the underlying systems. Now imaging combining this feature with Docker… I don’t know about your, but I’m excited about this premise! When combining this with Rancher, you could make your own Containers-as-a-Service (CaaS)! Today we’ll be delving into the matter to see how to implement this…

 

The Design

A quick extract from the ARM Resource Visualizer… when loading the ARM Template I have prepared for this deep dive.

2016-03-04 14_39_53-Azure Resource Visualizer

Continue reading “Autoscaling Docker hosts on Azure with Virtual Machine Scale Sets & Rancher”