Serverless On-Demand Scaling : Pushing the pedal when you need it…

Introduction

A lot of workloads are driven by peak consumption. From my experience, there aren’t the amount of workloads that have a constant performance need are in the minority. Now here comes the interesting opportunity when leveraging serverless architectures… Here you only pay for your actual consumption. So if you tweak your architecture to leverage this, then you can get huge gains!

For today’s post, I’ll be using VMchooser once again as an example. A lot has changed since the last post on the anatomy of this application. Here is an updated drawing of the high level architecture ;

Underneath you can see the flow that’ll be used when doing a “Bulk Mapping” (aka “CSV Upload”). The webapp (“frontend”) will store the CSV as a blob on the storage account. Once a new blob arrives, a function will be triggered that will examine the CSV file and put every entry onto a queue. Once a message is published onto the queue, another function will start processing this message. By using this pattern, I’m transforming this job into parallel processing job where each entry is handled (about) simultaneously. The downside of this, is that there will be contention/competition for the back-end resources (being the data store). Luckily, CosmosDB can scale on the fly too… We can adapt the request units as needed; up or down! So let’s do a small PoC and see who this could work…

Continue reading “Serverless On-Demand Scaling : Pushing the pedal when you need it…”

Advertisements

Rancher End-to-End Service Example using an Owncloud-plus-mysql Deployment

Introduction

So what will we be doing today? We are going to leverage the power of the combination between docker containers & the rancher eco system. As a demonstration, we’ll be publishing “Owncloud” with a “mysql” backend. As we tend to like it a bit more secure, we’ll introduce a loadbalancer service as SSL termination. This as we want to keep our “Owncloud” as “vanilla” as possible. We’ll be pointing that service towards the outside world and will make it accessible via the “external dns”.

RancherOwncloud

What can we optimize further about the design? (but is out-of-scope for today)

  • Add sidekick containers for backup purposes
  • Add data volume containers
  • Introduce scalable worker containers (“Owncloud”)
  • Introduce convoy for our data containers

Continue reading “Rancher End-to-End Service Example using an Owncloud-plus-mysql Deployment”

Rancher : Docker Lifecycle Management – Or how to upgrade containers?

Introduction

It’s all fun & games to create & deploy containers. And the “pets vs cattle” thingie is also cool… Though what about the lifecycle management? That’s something we’ll be handling today!

What will we be doing today?

  • Create a small dummy container
  • Setup a source respository (at BitBucket) for that dummy container
  • Setup an automated build (linked to the source repository) on your docker hub respository
  • Deploy a service on rancher
  • Update the source
  • Upgrade the service to the latest version
  • Enjoy life even more!

What will already need to be setup?

Continue reading “Rancher : Docker Lifecycle Management – Or how to upgrade containers?”

Xmas Tech Cookbook : Docker Swarm & Rancher Walkthrough

Objectives of this post

  • Install Docker on all machines
  • Setup a Docker Swarm
  • Setup Rancher to manage the lot

docker-swarm-020-5-638.jpg

Test Scenario

For this walkthrough I’ll be using 4 x Azure A0 Machines with Ubuntu 14.04TLS on them. Three of those will serve as docker hosts and one will be my Rancher management tooling. The docker hosts will be put into a swarm. For easy reference (and as a basic enterprise simulation), I’ve setup my docker hosts in a seperate subnet compared to the rancher.

Continue reading “Xmas Tech Cookbook : Docker Swarm & Rancher Walkthrough”