Microsoft Azure : Benchmark Tests – Storage – How do the different series relate to each other?

Azure currently has different “series” of machines. The A-series are seen as “general purpose” machines, where the D-series are targeted towards compute optimization. In the US, the G-series have even seen the light! Today I want to know what the effect of this is on storage performance… Typical IT organizations are worried by storage performance in the cloud as their ERP/BI implementation is “quite eager” to obtain the maximum storage performance.

So what will we be covering today?

  • A bit of theory concerning the differences
  • Test Environment Explained
  • Test Method Explained
  • Display of test results
  • Conclusion / analysis of the test results

 

A bit of theory concerning the differences

What does Microsoft say about their series ;

  • General purpose compute (A) – Basic tier : An economical option for development workloads, test servers, and other applications that don’t require load balancing, auto-scaling, or memory-intensive virtual machines.
  • General purpose compute (A) – Standard tier : Offers the most flexibility. Supports all virtual machine configurations and features.
  • Optimized compute (D) : 60% faster CPUs, more memory, and local SSD – D-seres virtual machines feature solid state drives (SSDs) and 60% faster processors than the A-series and are also available for web or worker roles in Azure Cloud Services. This series is ideal for applications that demand faster CPUs, better local disk performance, or higher memories.
  • Performance optimized compute (G) : unparalleled computational performance with latest CPUs, more memory, and more local SSD – G-series virtual machines feature latest Intel® Xeon® processor E5 v3 family, two times more memory and four times more Solid State Drive storage (SSDs) than the D-series. G-series will provide unparalleled computational performance, more memory and more local SSD storage than any current VM size in the public cloud making it very ideal for your most demanding applications.

Sidenote ; Azure has also released “DS” (“Premium Storage“). We won’t be looking into this area, as it is current still under preview.

Today we’ll be checking what we can get out of those machines via benchmarking. Be aware, that Microsoft is open towards the IOPS delivered by each machine. Be sure to check out the support article “Virtual Machine and Cloud Service Sizes for Azure“. Depending on the type of virtual machine, you can attach a maximum amount of disks. Per disk, you are granted a given number of IOPS. The amount of IOPS granted differs by “series”. An “A – Basic” will be granted 300 IOPS per disk. An “A – Standard”, “D” & “G” will be granted 500 IOPS per disk, where the “DS” will be granted 1600 IOPS per disk.

 

Test Environment Explained

We’ll be creating four machines today ;

  • TEST-BSC-A1 : A1 Basic (West Europe)2015-01-27 16_13_03-Virtual machines - Windows Azure
  • TEST-STD-A1 : A1 Standard (West Europe)
    2015-01-27 16_12_48-Virtual machines - Windows Azure
  • TEST-STD-D1 : D1 Standard (West Europe)
    2015-01-27 16_13_24-Virtual machines - Windows Azure
  • TEST-STD-G1 : G1 Standard (West US*)
    2015-01-28 08_48_56-Virtual machines - Windows Azure

Each machine will be installed with Ubuntu 14.04 with the Azure image of 23/01/2015. The system will then be foreseen with two benchmarking tools ;

These packages will be installed from the Azure Ubuntu Repositories by using the following method. First make sure to uncomment all “multiverse” repositories.

sudo vi /etc/apt/sources.list

Then do an update of the packages list and install both softwares

sudo apt-get update && sudo apt-get install bonnie++ iozone3

 

(Disclaimer : For the test with the G1, I created an additional disk, as the base OS disk was not large enough to fit the test file. Bonnie++ advises to create a test file that is twice the size of the memory. This to counter caching mechanisms. / Update : One error I made, was that I the host caching is disabled by default, so some results on the G1 are not aligned with the other tests. This is only relevant towards the Bonnie++ tests, not to the IOzone tests.)

 

Test Environment / Method Explained

Now we are ready to go… On each system the following commands were executed ;

bonnie++ -d /tmp > /tmp/bonnie.txt

iozone -R -l 5 -u 5 -r 4k -s 100m -F /tmp/f1 /tmp/f2 /tmp/f3 /tmp/f4 /tmp/f5 > /tmp/iozone_results.txt

iozone -R -l 5 -u 5 -r 4k -s 100m -F /mnt/f1 /mnt/f2 /mnt/f3 /mnt/f4 /mnt/f5 > /tmp/iozone_results-mnt.txt

So what are we basically going to do? A good description about what IOzone will do can be found in the article “I Feel the Need for Speed: Linux File System Throughput Performance, Part 1” of Linux Magazine. The highlights ;

IOzone

IOzone is open-source and written in ANSI C. It is capable of single thread, multi-threaded, and multi-client testing. The basic idea behind IOzone is to break up a file of a given size into records. Records are written or read in some fashion until the file size is reached. Using this concept, IOzone has a number of tests that can be performed:

  • WriteThis is a fairly simple test that simulates writing to a new file. Because of the need to create new metadata for the file, many times the writing of a new file can be slower than rewriting to an existing file. The file is written using records of a specific length (either specified by the user or chosen automatically by IOzone) until the total file length has been reached.

  • Re-writeThis test is similar to the write test but measures the performance of writing to a file that already exists. Since the file already exists and the metadata is present, it is commonly expected for the re-write performance to be greater than the write performance. This particular test opens the file, puts the file pointer at the beginning of the file, and then writes to the open file descriptor using records of a specified length until the total file size is reached. Then it closes the file which updates the metadata./LI>

  • ReadThis test reads an existing file. It reads the entire file, one record at a time.

  • Re-readThis test reads a file that was recently read. This test is useful because operating systems and file systems will maintain parts of a recently read file in cache. Consequently, re-read performance should be better than read performance because of the cache effects. However, sometimes the cache effect can be mitigated by making the file much larger than the amount of memory in the system.

  • Random ReadThis test reads a file with the accesses being made to random locations within the file. The reads are done in record units until the total reads are the size of the file. The performance of this test is impacted by many factors including the OS cache(s), the number of disks and their configuration, disk seek latency, and disk cache among others.

  • Random WriteThe random write test measures the performance when writing a file with the accesses being made to random locations with the file. The file is opened to the total file size and then the data is written in record sizes to random locations within the file.

  • Backwards ReadThis is a unique file system test that reads a file backwards. There are several applications, notably, MSC Nastran, that read files backwards. There are some file systems and even OS’s that can detect this type of access pattern and enhance the performance of the access. In this test a file is opened and the file pointer is moved 1 record forward and then the file is read backward one record. Then the file pointer is moved 2 records backward in the file, and the process continues.

  • Record RewriteThis test measures the performance when writing and re-writing a particular spot with a file. The test is interesting because it can highlight “hot spot” capabilities within a file system and/or an OS. If the spot is small enough to fit into the various cache sizes; CPU data cache, TLB, OS cache, file system cache, etc., then the performance will be very good.

  • Strided ReadThis test reads a file in what is called a strided manner. For example, you could read data starting at a file offset of zero, for a length of 4 KB, then seek 200 KB forward, then read for 4 KB, then seek 200 KB, and so on. The constant pattern is important and the “distance” between the reads is called the stride (in this simple example it is 200 KB). This access pattern is used by many applications that are reading certain data structures. This test can highlight interesting issues in file systems and storage because the stride could cause the data to miss any striping in a RAID configuration, resulting in poor performance.

  • FwriteThis test measures the performance of writing a file using a library function “fwrite()”. It is a binary stream function (examine the man pages on your system to learn more). Equally important, the routine performs a buffered write operation. This buffer is in user space (i.e. not part of the system caches). This test is performed with a record length buffer being created in a user-space buffer and then written to the file. This is repeated until the entire file is created. This test is similar to the “write” test in that it creates a new file, possibly stressing the metadata performance.

  • FrewriteThis test is similar to the “rewrite” test but using the fwrite() library function. Ideally the performance should be better than “Fwrite” because it uses an existing file so the metadata performance is not stressed in this case.

  • FreadThis is a test that uses the fread() library function to read a file. It opens a file, and reads it in record lengths into a buffer that is in user space. This continues until the entire file is read.

  • FrereadThis test is similar to the “reread” test but uses the “fread()” library function. It reads a recently read file which may allow file system or OS cache buffers to be used, improving performance.

When taking a look at Bonnie++, check out this article by TextualityMy objective is to gain a proper insight towards the latencies with Bonnie++ and use IOzone for the actual thoughput.

 

Display of test results

Latency

2015-01-28 11_57_47-Bonnie.xlsx - Excel  2015-01-28 12_02_31-Bonnie.xlsx - Excel

Throughput

2015-01-28 12_06_15-iometer.xlsx - Excel

2015-01-28 12_01_58-iometer.xlsx - Excel

Download Raw Results Files

Conclusion / analysis of the test results

So what have we learned today?

  • The latency of the A-series is significately higher than those of the D/G-series.
  • There is a performance difference between the “Basic” and “Standard” of the A-series.
  • Whilst the D-series outperform the A-series, the G-series put all of the others in the dark.
  • There is a performance answer to all loads… Just choose wisely!

 

Advertisements

Microsoft Azure : Budget Automation for your Development / Test Environment

Billing-per-minute

What is one of the biggest business advantages of Azure? You are only charge for your actual usage per minute.  For many organizations, the cost of a development/test environment is a sore spot as this costs a handful of cash. Today will introduce you to Azure Automation, which will let you orchestrate things, as stopping/starting your environment.

What are we going to do?

  • Setup a dedicated account for our scheduled runbooks
  • Configure two runbooks ; “stop all servers” & “start all servers”
  • Schedule those runbooks

 

Setup a dedicated account for our scheduled runbooks

In my opinion, you always needs to set up dedicated accounts for services. They should not be running under anyones “personal” account. At a given point they will leave the company. At that time, if the system is still active and the user account will be decommissioned, the system will cease to halt. In addition, this will also give you a traceability of the actions of the given service.

So how do you setup a dedicated account for the scheduled runbooks? Check the following post ; Azure Automation: Authenticating to Azure using Azure Active Directory

In summary, the steps you will need to do ;

  • Create an additional user in your Azure Active Directory
    2015-01-27 08_15_14-Active Directory - Windows Azure
  • Add the user as a co-administrator to your account2015-01-27 08_13_21-Settings - Windows Azure

It’s also advised to note down both the full username (dixit, username@account.onmicrosoft.com) and the password you have assigned. After the creation, be sure to login with the account. You will be asked to change your password. If you “forget” (too lazy huh?) to do this step, you will get an authentication error when trying to use this account for your automations (So yes, I tried to be lazy too…).

 

Configure two runbooks ; “stop all servers” & “start all servers”

In this phase, we’ll do the following

  • Create the Automation account (“folder”) under the Runbooks will be stored
  • Create a “start all servers” runbook from the gallery
  • Create a “stop all servers” runbook from the gallery

 

Browse to “Automation”, select “Runbook” and then choose “From Gallery”

2015-01-27 08_21_38-Automation - Windows Azure

 

In the gallery, go to “VM Lifecycle Management”, and select “Azure Automation Workflow to Schedule starting of all Azure Virtual Machines”2015-01-27 08_22_12-Automation - Windows Azure

Press next, review the code. The code is pretty straight forward… But we’ll get into that later on.

2015-01-27 08_22_29-Automation - Windows Azure

Now enter the name of your runbook, and choose “Create a new automation account”. Give the account a name and choose your subscription & region.

2015-01-27 08_23_19-Automation - Windows Azure

Now we’ll repeat the process for the “stop all servers” runbook.

2015-01-27 08_28_22-Automation - Windows Azure 2015-01-27 08_28_37-Automation - Windows Azure 2015-01-27 08_28_49-Automation - Windows Azure

Now browse back to the “Automation” screen ;

2015-01-27 08_29_51-Automation - Windows Azure

Before we can go on with these steps, we’ll need to add our user to the “Assets” of our “Automation Account”. Browse to “Assets” and select “Add settings”.

2015-01-28 10_43_07-Automation - Windows Azure

Select “Add credential”… Then use “Windows Powershell Credential” as “Credential Type” and name the credential.

2015-01-28 10_43_34-Automation - Windows Azure

Now enter the user information you noted down earlier… and press save.

2015-01-28 10_44_32-Automation - Windows Azure

You are now good to go!

2015-01-28 10_42_48-Automation - Windows Azure

Select “Runbooks”, now you can see both runbooks we just created.

 

2015-01-27 08_30_09-Automation - Windows Azure

Select the “Stop-AllAzureVM” & adjust the two parameters and press save ;

  • -Name “username@domain.onmicrosoft.com”
  • -Subscriptionname “Subscription Name”

2015-01-27 08_30_48-Automation - Windows Azure

Select the “Start-AllAzureVM” & adjust the three parameters and press save ;

  • -Name “username@domain.onmicrosoft.com”
  • -Subscriptionname “Subscription Name”
  • -Name “Your Most Important Server”

2015-01-27 08_33_48-Automation - Windows Azure

What did we just do for both scripts? We entered the user account & subscription under which the script will be executed. This is a mandatory step and understandingly so. Now let us test the “StartAllAzureVM”-script… I’ve prepared two virtual machines, which are currently shutdown.

2015-01-27 08_34_03-Virtual machines - Windows Azure

So we’ll press “Test” on the runbook…

2015-01-27 08_34_20-Automation - Windows Azure

And yes, we are sure. Azure Automation will save the runbook one more time to be safe.

2015-01-27 08_34_33-Automation - Windows Azure

 

The output pane will show the status “starting”.

2015-01-27 08_34_52-Automation - Windows Azure

And it will change to “running” after a while.

2015-01-27 08_35_40-Automation - Windows Azure.

Once you see the code below, you will know that you have been authenticated. So all our hard work with creating the user paid off! If you do not see this, that is the part you should be debugging…

2015-01-27 08_35_56-Automation - Windows Azure

Suddenly our “most important server” will be showing the status “Starting”…

2015-01-27 08_36_31-Virtual machines - Windows Azure

 

And the output pane will verify this status!

2015-01-27 08_36_41-Automation - Windows Azure

So basically, we are safe to say that our script works. Let’s publish the runbooks so that we can schedule them later on.

2015-01-27 08_50_11-Edit Post ‹ Karim Vaes — WordPress

 

For each runbook, press the “publish”-button

2015-01-27 08_48_32-Automation - Windows Azure

We are sure, and you will see the runbook shift from “draft” to “published”.

 

2015-01-27 08_48_59-Automation - Windows Azure

Congrats so far! We are now ready to schedule those babies!

 

Schedule those runbooks

So which steps will we be doing in this phase?

  • Create two schedules ; “start of business day” & “end of business day”
  • Attach the “start” runbook to the “start of business day” schedule
  • Attach the “stop” runbook to the “end of business day” schedule

 

Let us start creating the two schedules ;

 

Go to our “Automation Account” and select “Assets”. Here you press the “Add Setting”-button.

2015-01-27 08_54_49-Automation - Windows Azure 2015-01-27 08_55_04-

Choose “Add Schedule”2015-01-27 08_55_16-Automation - Windows Azure

Enter the name…2015-01-27 08_55_28-Automation - Windows Azure

The schedule…2015-01-27 08_56_14-Automation - Windows Azure

Rince & repeat…

2015-01-27 08_58_01-Automation - Windows Azure

Now we have both schedules. One that will occur at 08:00 and another one that will occur at 17:00 (5pm). Now let’s link our runbooks…

Go to our “Automation Account”, and select “Runbooks”. Click on one of them

2015-01-27 09_01_15-Automation - Windows Azure

Go to “Schedule”, and press “Link to an existing schedule”.

2015-01-27 09_01_29-Automation - Windows Azure

Select the schedule…

2015-01-27 09_01_41-Automation - Windows Azure

And you will see the schedule attached.

 

2015-01-27 09_02_04-Automation - Windows Azure

Rince & repeat for the other one.

 

Summary

With the power of automation & a gallery of pre-made runbooks, we were able to save our business tons of money by only running the servers during the business hours. Be aware that the above example does not accompany holidays / weekends… In addition, the money saving is “limited” to the “compute”, as the storage of your devices will remain “active” (on disk).

Microsoft Azure : How-to setup a site-to-site VPN using OpenSwan (on a Telenet SOHO subscription)

Objective of the day?

We’ll be setting up an IPSec VPN tunnel between Microsoft Azure and a development/management environment using commodity internet connection of a Belgian ISP.

Azure-Site_to_Site_VPN

What will our test environment look like?

  • Private Network : 192.168.0.0/24
  • System running Openswan : 192.168.0.226
  • Private Internet Connection : 81.82.83.84
  • Azure VPN Gateway : 104.40.149.247
  • Test System on Azure : 10.0.0.4
  • Azure Network : 10.0.0.0/24

The steps we’ll be going through?

 

  • Configure Virtual Network on Azure
  • Configure VPN Gateway
  • Configure Openswan
  • Configure NAT Rules on the ISP (Telenet) Router
  • Activate IPSec VPN Tunnel
  • Test Connectivity

Continue reading “Microsoft Azure : How-to setup a site-to-site VPN using OpenSwan (on a Telenet SOHO subscription)”

Database variants explained : SQL or NoSQL? Is that really the question?

A first glance beyond the religion

When taking a look towards the landscape of databases, one can only accept that there has been a lot of commotion about “SQL vs NoSQL” in the last years. But what is it really about?

SQL, which stands for “Structured Query Language”, has been around since the seventies and is commonly used in relational databases. It consists of a data definition language to define the structure and a data manipulation language to alter the data within the structure. Therefore a RDBMS will have a defined structure and has been a common choice for the storage of information in new databases used for financial records, manufacturing and logistical information, personnel data, and other applications since the 1980s.

1401269083847

NoSQL, which stands for “Not only SQL”, departs from the standard relational model since it saw its first introduction in the nineties. The primary focus of these database was performance, or a given niche, and focus less consitency/transactions. These databases provide a mechanism for storage and retrieval of data that is modeled in means other than the tabular relations used in relational databases. Motivations for this approach include simplicity of design, horizontal scaling, and finer control over availability. The data structures used by NoSQL databases (e.g. key-value, graph, or document) differ from those used in relational databases, making some operations faster in NoSQL and others faster in relational databases. The particular suitability of a given NoSQL database depends on the problem it must solve.

So it depends on your need…

Do you want NoSQL, NoSQL, NoSQL or NoSQL?

NoSQL comes in various flavors. The most common types of NoSQL databases (as portrayed by Wikipedia) ;

There have been various approaches to classify NoSQL databases, each with different categories and subcategories. Because of the variety of approaches and overlaps it is difficult to get and maintain an overview of non-relational databases. Nevertheless, a basic classification is based on data model. A few examples in each category are:

  • Column: Accumulo, Cassandra, Druid, HBase, Vertica
  • Document: Clusterpoint, Apache CouchDB, Couchbase, MarkLogic, MongoDB, OrientDB
  • Key-value: Dynamo, FoundationDB, MemcacheDB, Redis, Riak, FairCom c-treeACE, Aerospike, OrientDB
  • Graph: Allegro, Neo4J, InfiniteGraph, OrientDB, Virtuoso, Stardog
  • Multi-model: OrientDB, FoundationDB, ArangoDB, Alchemy Database, CortexDB

Column

A column of a distributed data store is a NoSQL object of the lowest level in a keyspace. It is a tuple (a key-value pair) consisting of three elements:

  • Unique name: Used to reference the column
  • Value: The content of the column. It can have different types, like AsciiType, LongType, TimeUUIDType, UTF8Type among others.
  • Timestamp: The system timestamp used to determine the valid content.

Example

{
    street: {name: "street", value: "1234 x street", timestamp: 123456789},
    city: {name: "city", value: "san francisco", timestamp: 123456789},
    zip: {name: "zip", value: "94107", timestamp: 123456789},
}

Document

A document-oriented database is designed for storing, retrieving, and managing document-oriented information, also known as semi-structured data. The central concept of a document-oriented database is that Documents, in largely the usual English sense, contain vast amounts of data which can usefully be made available. Document-oriented database implementations differ widely in detail and functionality. Most accept documents in a variety of forms, and encapsulate them in a standardized internal format, while extracting at least some specific data items that are then associated with the document.

Example

<Article>
   <Author>
       <FirstName>Bob</FirstName>
       <Surname>Smith</Surname>
   </Author>
   <Abstract>This paper concerns....</Abstract>
   <Section n="1"><Title>Introduction</Title>
       <Para>...
   </Section>
 </Article>

Key-Value

A key-value (an associative array, map, symbol table,or dictionary) is an abstract data type composed of a collection of key/value pairs, such that each possible key appears just once in the collection.

Example

{
    "Pride and Prejudice": "Alice",
    "The Brothers Karamazov": "Pat",
    "Wuthering Heights": "Alice"
}

Graph

A graph database is a database that uses graph structures for semantic queries with nodes, edges, and properties to represent and store data. A graph database is any storage system that provides index-free adjacency. This means that every element contains a direct pointer to its adjacent elements and no index lookups are necessary. General graph databases that can store any graph are distinct from specialized graph databases such as triplestores and network databases.

Example

GraphDatabase_PropertyGraph

MultiModel

Most database management systems are organized around a single data model that determines how data can be organized, stored, and manipulated. In contrast, a multi-model database is designed to support multiple data models against a single, integrated backend. Document, graph, relational, and key-value models are examples of data models that may be supported by a multi-model database.

And what flavor do I want?

Each type and implementation has its own advantages… The following chart from Shankar Sahai provides a good overview ;

nosql-comparison-table

Any other considerations I should take into account?

Be wary that most implementations were not designed around consistency integrity and more towards performance. Transactions are referential integrity are not supported by most implementations. High availability designs (including on geographic level) are possible with some implementations, though this often implies a performance impact (as one would expect).

Also check out the research made by Altoros ;

5. Conclusion
As you can see, there is no perfect NoSQL database. Every database has its advantages and disadvantages that become more or less important depending on your preferences and the type of tasks.
For example, a database can demonstrate excellent performance, but once the amount of records exceeds a certain limit, the speed falls dramatically. It means that this particular solution can be good for moderate data loads and extremely fast computations, but it would not be suitable for jobs that require a lot of reads and writes. In addition, database performance also depends on the capacity of your hardware.

They did a very decent job in performance testing various implementations!

2015-01-21 09_08_23-A_Vendor_independent_Comparison_of_NoSQL_Databases_Cassandra_HBase_MongoDB_Riak.

Microsoft Azure : How to connect my Enterprise? Expressroute or VPN?

microsoft-azure1

Microsoft has been going at warp speed last year (and it looks this pace will be kept) with the features they have been adding to Azure. In the beginning when I came into contact with Azure, one of my first questions was ; “How can I hook up Azure in my Wide-Area-Network (WAN)?” The answer at that point was a kinda flaky VPN connection. About a half-year ago, Microsoft released “ExpressRoute”. This was the answer Enterprise customers were looking for in terms of hooking up Azure to their WAN. So let’s take a look at your options…

Basically, you have five options to connect to Azure ;

Internet (public)

  • Medium : Public
  • Network : Public
  • Capacity: No explicit cap
  • Connection Resilience : Active / Active
  • High Level Solution : Your “typical” enterprise internet
  • Typical Usage : Almost everything in Azure that isn’t linked by the underneath mentioned services.

Virtual Network – Point-to-site

  • Medium : Public
  • Network : Private
  • Capacity: Typically 100 Mbit Aggregates
  • Connection Resilience : Active / Pasive
  • High Level Solution : A point-to-site VPN also allows you to create a secure connection to your virtual network. In a point-to-site configuration, the connection is configured individually on each client computer that you want to connect to the virtual network. Point-to-site connections do not require a VPN device. They work by using a VPN client that you install on each client computer. The VPN is established by manually starting the connection from the on-premises client computer. You can also configure the VPN client to automatically restart.
  • Typical Usage : Proof-of-Concept, Prototyping, Evaluation, …

Virtual Network – Site-to-site

  • Medium : Public
  • Network : Private
  • Capacity: Typically 100 Mbit Aggregates
  • Connection Resilience : Active / Pasive
  • High Level Solution : A site-to-site VPN allows you to create a secure connection between your on-premises site and your virtual network. To create a site-to-site connection, a VPN device that is located on your on-premises network is configured to create a secure connection with the Azure Virtual Network Gateway. Once the connection is created, resources on your local network and resources located in your virtual network can communicate directly and securely. Site-to-site connections do not require you to establish a separate connection for each client computer on your local network to access resources in the virtual network.
  • Typical Usage : Small scale production workloads, development/test environments, …

ExpressRoute – Exchange Provider

  • Medium : Private
  • Network : Public
  • Capacity: up to 1Gbps
  • Connection Resilience : Active / Active (customer managed)
  • High Level Solution : Azure ExpressRoute lets you create private connections between Azure datacenters and infrastructure that’s on your premises or in a co-location environment. ExpressRoute connections do not go over the public Internet, and offer more reliability, faster speeds, lower latencies and higher security than typical connections over the Internet. In some cases, using ExpressRoute connections to transfer data between on-premises and Azure can also yield significant cost benefits. With ExpressRoute Exchange Provider, you can establish connections to Azure at an ExpressRoute location (Exchange Provider facility) clip_image002 - Exchange Provider - ExpressRoute
  • Typical Usage : Mission Critical Workloads

ExpressRoute – Network Service Provider

  • Medium : Public
  • Network : Public
  • Capacity : up to 10Gbps
  • Connection Resilience : Active / Active (telecom provider managed)
  • High Level Solution : Azure ExpressRoute lets you create private connections between Azure datacenters and infrastructure that’s on your premises or in a co-location environment. ExpressRoute connections do not go over the public Internet, and offer more reliability, faster speeds, lower latencies and higher security than typical connections over the Internet. In some cases, using ExpressRoute connections to transfer data between on-premises and Azure can also yield significant cost benefits. With ExpressRoute Service Provider, you can directly connect to Azure from your existing WAN network (such as a MPLS VPN) provided by a network service provider. clip_image002 - Network Service Provider - ExpressRoute
  • Typical Usage : Mission Critical Workloads

Network Seggregation

So if I get ExpressRoute, how will my network flows go?

clip_image001
Basically, the private solutions will ensure that your company communication will not traverse over the public internet. You can configure your service to either use the internet connect of Azure, or your own hop, to breakout towards public services. Let’s say for instance, if you want to download updates, you could set it up that those are done via Azure, instead of going back over your ExpressRoute link in order to break out from within your own premises.

Decision Chart

So what does this mean for a typical Enterprise?

It depends on your scenario…

  • Looking to get do some raw testing?
      Isolated Test : Internet only
      Integrated : Point/site-to-site vpn
  • Hook up your development/test environment in a lean manner? Site-to-site vpn
  • Azure as a Disaster Recovery location? Dependent on your size …
      Small IT Landscape : Site-to-site
    1. From a few TB : ExpressRoute
  • Azure as a Primary Datacenter : ExpressRoute Service Provider

route

References

Where can I find additional information?

Web Development : A step up with Automated Deployment

Developing a website… ; Open up “notepad++”, browse to your web server via FTP and edit the files. Then refresh to see the changes…

Sounds familiar? Probably… It’s a very straight forward and easy process. The downside however is that you have no tracking of your changes (Version Control) and that the process is pretty manual. So this becomes a problem when you aren’t the only one on the job or if something goes wrong.

So let’s step it up and introduce “version control”… Now we have an overview of all the revisions we made to our code and we are able to revert back to it. Yet suddenly, we need to do a lot more to get our code onto the web server. This brings us to the point where we want a kind of helper that does the “deployment” for us.

The basic process
automated-web-development-kvaes.be

  • Local Development : The development will happen here. Have fun… When you (think you) are happy with what you have produced, you update the files via your version system.
  • Source Repository : The source repository will contain all the versions of your code. Here you can configure it to send a notification to your deployment system whenever a new version has been introduced.
  • Deployment System : The deployment system will query the source repository and retrieve the latest code. This code will be packaged, transmitted and deployed onto the target system(s).
  • Target Systems : The systems that will actually host your code and deliver the (web) service!

Real Life Example?
Ingredients

Recipe

  • Create a private repository at BitBucket
  • Pull/push the repository between BitBucket & your local SourceTree
  • In GitHub, go to “Settings”, “Deployment Keys” and generate a key for your automation. Copy it to your clipboard…
    2015-01-12 15_33_53-kvaes _ kvaes.be - 2015 _ Admin _ Deployment keys — Bitbucket
  • In DeployHQ, go to “Settings”, “General Settings” and copy to key into the “Public Key Authentication” textbox.
    2015-01-12 15_31_19-Website 2015 - LogiTouch - Deploy
  • In DeployHQ, go to “Settings”, “Servers & Group” and create a new server.
    2015-01-12 15_36_53-Website 2015 - LogiTouch - Deploy
  • In the same screen, Enable “Auto Deploy” and copy the url hook.
    2015-01-12 15_38_19-Website 2015 - LogiTouch - Deploy
  • Now go to “Settings” in GitHub, and then “Hooks”. Add a “POST” hook containing the url hook you just copied.
    2015-01-12 15_39_11-kvaes _ kvaes.be - 2015 _ Admin _ Hooks — Bitbucket
  • Now every time you do a commit on your workstation, the code will be deployed to your server!

In fact, this is the mechanism I utilize for my own (hobby) development projects. An example of here, is my own homepage, which is deployed via the system as described above.