Azure Container Service : Using the Azure File Storage as a persistent (kubernetes) volume

Introduction

Today’s post is a brief one… Though it packs some punch! In the past I talked about storage patterns for docker/containers. Today we’ll touch how you can leverage the Azure File Storage as a shared & persistent storage for your container deployments. Kubernetes has been gaining a lot of traction, and that one has support for the Azure File Storage as a persistent volume too.

 

Demo Files

Want to run this yourself? Check out the following GitHub repository!

Continue reading “Azure Container Service : Using the Azure File Storage as a persistent (kubernetes) volume”

Advertisements

Extending a Storage Spaces Direct pool on Azure

Introduction

Yesterday we talked about the combination of Azure+S2D+SOFS+MSSQL. Here we had a cluster where each node had two P20 disks. What if at a given point we would need more than 1TB of disk space? We’ll be extending the pool (and virtual disk etc). So let’s take a look what that would look like?

 

Adding the disks

First part… Let’s add the disks (note : even entire hosts is possible!). Browse to both VMs and press “attach new” in the disks section ;

2017-02-01-15_57_08-disks-microsoft-azure

Continue reading “Extending a Storage Spaces Direct pool on Azure”

Ever tried the mix of Azure, SQL Server, Storage Spaces Direct & Scale Out File Server?

Introduction

A while back I posted a blog post how to setup a High Available SQL cluster on Azure using SIOS Datakeeper. As I’m an avid believer of storage spaces, I was looking for a moment to test drive “storage spaces direct” on Azure. The blog post of today will cover that journey…

UPDATE (01/02/2017) ; At this point, there is no official support for this solution. So do not implement it for production at this point. As soon as this changes, I’ll update this post accordingly!

UPDATE (08/02/2017) ; New official documentation has been released. Though I cannot find official support statements.

UPDATE (30/03/2017) ; A few days after the previous update, the following post was made => Deploying IaaS VM Guest Clusters in Microsoft Azure

 

Solution Blueprint

What do we want to build today?

  • A two node cluster which will be used as a Failover Cluster Instance for MSSQL.
  • As a quorum, we’ll be using the cloud witness feature of Windows 2016 in combination with an Azure storage account.
  • In regards to storage, we’ll create a Scale Out File Server setup which will leverage the local disks of the two servers via Storage Spaces Direct.
  • To achieve a “floating IP”, we’ll be using the Azure LoadBalancer setup (as we did in the last post).

kvaes-sql-cluster-s2d-sofs-azure

 

Continue reading “Ever tried the mix of Azure, SQL Server, Storage Spaces Direct & Scale Out File Server?”

Basic Azure IoT Flow : From Event Hub via Stream Analytics to Power Bi

Introduction

A few weeks back I posted a blog post on how you can leverage “serverless” components for IoT. Today I’ll show you what it would mean if we replace the Azure Functions component in that post by Azure Stream Analytics.

 

Flow

So the flow between device and event hub is untouched. Though we’ll replace the functions part with Azure Stream Analytics. So how will the end result look?

2017-01-03-11_11_49-job-diagram-microsoft-azure

We’ll be using one Stream Analytics job to trigger three flows. One will store the data into an Azure Table Storage, another on will store it as a JSON file on to Azure Blob Storage and another one will stream it directly into a PowerBi dataset.

So let’s take a look at all the components from within this Stream Analytics Flow we’ll be using…

2017-01-03-11_23_08-inputs-microsoft-azure

Continue reading “Basic Azure IoT Flow : From Event Hub via Stream Analytics to Power Bi”

Azure IoT : From RaspberryPi with Sensor to Azure Storage Table by using a serverless architecture

Introduction

A few days ago my connectors arrived for my latest PoC on Azure. So today I’m writing about my experience in using a RaspberryPi with a temperature & humidity sensor and to save the telemetry data in Azure. For this we’ll be using Azure Event Hub as an ingress mechanism, and Azure Functions to storage the events towards an Azure Storage Account. My next venture will be to use this data to create reports and maybe (on the long run) do some machine learning. For the latter, I’m pondering about linking this system to my ebus system of my heating system. That way I could correlate the data from the various censors (RPi, Thermostat & outside sensor) in combination with the heater information & heating schedules. Basically… creating my own  Google Nest. 🙂

 

Sensor : Physical Connection (I2C)

The guys from ThingTank had a spare sensor lying around, which they lend to me for my PoC… This was a “Grove – Temperature&Humidity Sensor (High-Accuracy & Mini)“. As you can see in the picture, underneath, this one has an I2C connector. We see four connections ; GND, VCC, SDA & SCL.

grove-tem-hum-accuracy-mini_01

Continue reading “Azure IoT : From RaspberryPi with Sensor to Azure Storage Table by using a serverless architecture”

Azure : Benchmarking SQL Database Setups – To measure is to know, and being able to improve…

Introduction

To measure is to know. If you can not measure it, you cannot improve it!

Today’s post will go more in-depth on what performance to expect from different SQL implementations in Azure. We’ll be focussing on two kind of benchmarks ; the storage subsystem and an industry benchmark for SQL. This so that we can compare the different scenario’s to each other in the most neutral way possible.

to-measure-is-to-know-storage-database-performance-kvaes

Test Setup

As a test bed I started from one of my previous posts

kvaes-azure-sql-cluster-sios-datakeeper-high-availability-ha

The machines I used were DS1 v2 machines when using single disks and a DS2 v2 machines when using multiple disks. In terms of OS, I’ll be using Windows 2012 R2 and MSSQL 2014 (12.04100.1) as database.

Continue reading “Azure : Benchmarking SQL Database Setups – To measure is to know, and being able to improve…”

Azure : Performance limits when using MSSQL datafiles directly on an Storage Account

Introduction

In a previous post I explained how you are able to integrate MSSQL with Azure storage by directly storing the data files on the storage account.

2016-04-22 19_41_15-kvaessql21 - 104.40.158.231_3389 - Remote Desktop Connection

Now this made me wondering what the performance limitations would be of this setup? After doing some research, the basic rule is that the same logic applies to “virtual disks”, as to the “data files”… Why is this? They are both “blobs” ; the virtual disk is a blob called “disk” and the data files will be “page blobs”.

2016-04-25 09_35_42-Pricing - Cloud Storage _ Microsoft Azure

Continue reading “Azure : Performance limits when using MSSQL datafiles directly on an Storage Account”