Hardening your storage account with Private Link / Endpoint

Introduction

Earlier this week, a new capability called “Azure Private Link” (and also “Azure Private Endpoint”) went into public preview. As a nice copy & past from the documentation page ;

Azure Private Link enables you to access Azure PaaS Services (for example, Azure Storage and SQL Database) and Azure hosted customer/partner services over a Private Endpoint in your virtual network. Traffic between your virtual network and the service traverses over the Microsoft backbone network, eliminating exposure from the public Internet. You can also create your own Private Link Service in your virtual network (VNet) and deliver it privately to your customers. The setup and consumption experience using Azure Private Link is consistent across Azure PaaS, customer-owned, and shared partner services.

As always, we’ll take this one out for a spin! For this we’ll see if we can access a storage account privately (from a virtual machine) over the VNET.

 

Taking the user delegation SAS tokens for a spin

Introduction

A few days ago the preview for the “User delegation SAS token” has seen the light. In today’s post, we’ll take a first glance on this new capability! Though why should we care about this feature? You can now create SAS tokens based on the scoped permissions of an AAD user, instead of linked towards the storage account key. From a security perspective this is REALLY awesome, cause you can harden the scope of a possible even more.

 

Bibliography

• https://azure.microsoft.com/en-us/blog/announcing-user-delegation-sas-tokens-preview-for-azure-storage-blobs/
• https://docs.microsoft.com/en-us/rest/api/storageservices/create-user-delegation-sas
• https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blob-user-delegation-sas-create-cli
• https://docs.microsoft.com/en-us/cli/azure/?view=azure-cli-latest

Writing straight to the Azure Storage Access Tier you want (with AzCopy)

Introduction

With the “Set Blob Tier” operation you can set the Access Tier of the blob object of a storage account. Now at times you know a certain object will go to tier that’s not your default access tier. Or you want to write immediately to archive. The cool thing is that AzCopy can assist you in this!

 

AzCopy Flags

Take a look at the AzCopy flags… Here you’ll notice the “–block-blob-tier” flag. This is the one that’ll help you on writing directly to the access tier you want.

Quick Demo

I’ve created a storage account in mint condition, where the default access tier is set to “Hot”.

How to talk to the Azure Storage APIs from a Single Page Webapplication (NuxtJS/VueJS) by using AAD (Oauth2 Implicit Flow)

Introduction

In today’s post we’ll go through how you can setup an SPA (Single Page Application) to access the data pane of an Azure Storage account. For this I’ll be using NuxtJS (a Vue.js framework) for my boiler plating, and will rely on the its generic Oauth2 authentication library. The awesomeness here is that the bearer tokens will automatically be injected in your API calls (Axios). Though you can use this post as inspiration on how to get things working in another framework too!

Why this post? It has has been on my personal todo list for a while due to many reasons. One of which is to remove the barrier of using Azure Storage in this scenario. I find that it makes a lot of sense in way too many use cases. Though I must admit we’ve failed to reduce the learning curve in this scenario! You suddenly find yourself in flux between the complexities of AAD and the way how Azure Storage handles the presented JWT tokens. That being said, this is actually the only correct (secure) way of talking from an SPA to Azure Storage. DO NOT use storage keys or SAS tokens in your SPA. It’s like leaving your master keys out on the porch of your house. AAD is the correct way, and actually… With an SPA, the only correct authentication flow is actually Implicit Flow.

 

Sample Code Repository

If you are looking for the sample code used for this post ; https://github.com/kvaes/TasmanianTraders-NuxtJS-AzureStorageExample

Hardening your Azure Storage Account by using Service Endpoints

Introduction

Earlier this week I received a two folded question ; “Does a service endpoint go over internet? As when I block the storage account tags with a NSG, my connection towards the storage account stops.” Let’s look at the following illustration ;

 

The first thing to mention here is that the storage account (at this time) always listens to a public IP address. The funky thing is, that in Azure, you’ll have a capability called “Service Endpoint“, which I already covered briefly in the past. For argument’s sake, I’ve made a distinction in the above illustration between the “Azure Backbone” and the “Azure SDN”. A more correct representation might have been to have said “internal” & “external” Azure Backbone in terms of the IP address space used. So see the “Azure Backbone” in the above drawing as the public IP address space. Here all public addresses reside. Where the “Azure SDN” is the one that covers the internal flows. Also be aware that an Azure VNET can only have address spaces as described by RFC1918. So why did I depict it like this? To indicate that there are different flows;

• Connections from outside of Azure (“internet”)
• Connections from within the Microsoft backbone (“Azure Backbone”)
• Connections by leveraging a service endpoint

So how does the service endpoint work?

So to answer the question stated above ;

• Q: Does a service endpoint go over “the internet”?
• A : Define “internet”…
  • If you mean that it uses a public ip address instead of an internal one? Then yes.
  • If you mean that it leaves the Microsoft backbone? Then no.
  • If you mean that the service is accessible from the internet? Unless you open up the firewall, it won’t (by default, when having a service endpoint configured).

 

• Q: When I block the storage tag in my network security group (“NSG”), then the traffic stops. How come?
• A: The NSG is active on NIC level. The storage account, even when using a service endpoint, will still use the public IP. As this public IP is listed in the ranges that are configured in the service tag, you’ll be effectively blocking the service. This might be your objective… Though if you do this to “lock down the internet flow”, then you won’t achieve the requested you wanted. You should leverage the firewall functionality from the service for which the service endpoint was used.

 

Deep Dive

As always, let’s do a deep dive to experience this flow! So I did set up the above drawing in my personal lab ;

Aggregating Metrics from multiple Azure Storage Accounts

Introduction

When working at scale, the only way to properly handle true scale is to work with horizontal scaling options. Some services (like CosmosDBCosmosDB for instance) do this out of the box and abstract it away from the user/customer. Though sometimes this is something you need to facilitate yourself… In terms of Azure Storage, we’re very open in regards to our limitations. For example, at this point in time, we’re currently facing a maximum egress of 50Gbps per storage account. Where this is more than enough for a lot or customers, at times we need to scale beyond this. Here the solution at hand is to see the storage account as a “scale unit”, and use it for horizontal scaling. So if you need 200GBps, then you can partition your data across four storage accounts.

In today’s post, we’re going to take a look at how you can aggregate these metrics into a single pane of glass. Because, at the end of the day, your operations team does not want to have a disaggregated view of all the components in play.

 

Important Note

All Azure teams are constantly looking to evolve their services. Please note that the limits mention in this post are linked to the point in time when the article was written. As many of you know, Azure keeps evolving at vast pace, so the limits might already have been changed. If you are wondering, always check the following page for the most current limits that are linked to GA (“General Available”) services!

Taking the Azure Data Box Gateway (preview) out for a spin!

Introduction

At the last Ignite conference, three new additions joined the Data Box family. In today’s post we’ll take one of those out for a spin, being the “Data Box Gateway“. This one comes as a virtual appliance that you can run on top of your own physical hardware.

 

So where does it fit into the picture?

• Cloud archival – Copy hundreds of TBs of data to Azure storage using Data Box Gateway in a secure and efficient manner. The data can be ingested one time or an ongoing basis for archival scenarios.
• Data aggregation – Aggregate data from multiple sources into a single location in Azure Storage for data processing and analytics.
• Integration with on-premises workloads – Integrate with on-premises workloads such as backup and restore that use cloud storage and need local access for commonly used files.

 

Let’s take it for a spin!

So let’s make it a bit more tangible and see what the user experience is in setting it up & using it. Start by searching the Azure Marketplace for Data Box Gateway.

Azure Data Lake Storage (Gen2) : Exploring AAD B2B & ACL hardening

Introduction

In the summer of 2018, the 2nd generation of the Azure Data Lake Storage was announced.  In today’s post, we’ll delve into the authentication & authorization part of this service. We’re going to see how we can leverage AAD to tighten security around our Data Lake.

 

Use Case

To help us in this storyline, we’ll be looking to solve the following use case. A customer has stored a lot of data on its Data Lake, and is looking to provide a “partner” access to a subset of the data. In this use case, what would we need to to to achieve this goal?

 

Azure Data Lake Storage : Access Control Model

The first part of our puzzle is looking at the “Access Control Model“… In essence there are four ways to provide access to the data lake ;

• Shared Key ; The caller effectively gains ‘super-user’ access, meaning full access to all operations on all resources, including setting owner and changing ACLs
• SAS Tokens ; The token includes the allowed permissions as part of the token. The permissions included in the SAS token are effectively applied to all authorization decisions, but no additional ACL checks are performed.
• Azure RBAC ; Azure Role-based Access Control (RBAC) uses role assignments to effectively apply sets of permissions to users, groups, and service principals for Azure resources. Typically, those Azure resources are constrained to top-level resources (e.g., Azure Storage accounts). In the case of Azure Storage, and consequently Azure Data Lake Storage Gen2, this mechanism has been extended to the file system resource.
• ACL ; And last, but not least, we have the access control list we can apply at a more fine-grained level.

Traffic Light Protocol alike Security Reference Architecture for Azure

Introduction

The way how organizations categorize/handle classified information can vary significantly. Where it can go from about 6 categories towards a more “limited” set of 3 to 4 categories. Where you see that some government organizations have even tried to reduce this in an effort to make it more accessible.

 

So for today, we’ll be looking at how we can handle sensitive/classified information in Azure. And to ensure you that you Azure implementations can facilitate sensitive data.

 

Side Story : Security should be like a roundabout

Though I don’t remember which conference talk it was… One visual has always stuck with me when talking about security. Imagine security like road infrastructure. Having a complex situation might be needed at times, though it will increase the risk that the drivers (~users) will make mistakes.

Putting Azure FrontDoor in front of your webapp

Introduction

Today we’ll be going through the process of putting Azure FrontDoor in front (pun intended!) of a web app.

 

Step One) Adding the custom domain

The first step  is to add a new custom domain to your Frontend hosts ;

Now if you go through this, you’ll see that AFD expects you to link the custom domain to the azurefd.net domain in order to proceed.