In an earlier blog post I discussed the decision criteria in selecting a VM. In that post I also showed a tool called “VMchooser“. Today’s post will be on the architecture I used to build this one. As you might have guessed, it’s built on Azure components. Let’s get to it and check the anatomy of this application.
High Level Architecture
VMchooser has the following high level architecture ;
- Web App : The front-end of the application is hosted on an Azure Web App.
- Azure Functions : The back-end API & batch parser are built with Azure Functions. Which unlocks insane scaling possibilities.
- Storage Account : The storage account serves as decoupled/central storage component for the batch parsing. And it could also be used for hosting the “database” (flat file).
- Application Insights : Application insights is used to have the needed insights into the usage & other metrics.
- Github : All code for this project is open-source and publically hosted. You can run your own VMchooser if you want… 😉 Every change is immediately pushed towards the front-end, back-end & database.
- API Management : As the back-end API is decoupled from the application, I’ve also linked this api with api management. This would provide me with the option to allow 3th party application integrations via an API subscription plan.
Continue reading “The anatomy of “vmchooser”… Adding some serverless into the architecture!”
When I was working on the “CSV import” of VMchooser, I noticed that long jobs sometimes had issues. After some investigation, it quickly became apparent that I was hitting the time-out. So I had a need to increase it…
Logic Apps Time-out = 2 minutes
As a bit of back story, I first started off with my async parsing flow by using the combination of logic apps & functions. Where I at first thought it was due to the time-out on the functions side, it actually appeared to be on the logic apps side. And that one cannot be changed…
Azure Functions Time-out
So I browsed the web, and encountered two variables, which I both set on my platform ;
To reach set these, click on your functions bar (“kvaesvmapi” here), then “Platform Features”, and then “Application Settings” ;
But let’s get back to the variables… From various posts I saw these two popping up. Where I set them both to be safe, I started investigating a bit. The first one ;
WEBJOBS_IDLE_TIMEOUT – Time in seconds after which we’ll abort a running triggered job’s process if it’s in idle, has no cpu time or output (Only for triggered jobs).
This is actually the one you need for your function. As it’s clearly described in the kudu docs. Where the second one apparently is only related to the deployment phase; hence the “SCM_”-prefix.
Set the “WEBJOBS_IDLE_TIMEOUT” in your “Application Settings” ;
And don’t be distracted by the “SCM_COMMAND_IDLE_TIMEOUT”. 😉
For today I’ll show you how to roll your own rss2twitter bot in about 15 minutes. What will this bot do?
- Check an RSS feed for new entries
- Parse the new entry and replace key words with hashtags
- Post it to twitter
For this we’ll be using ;
- Azure Function : as the customer code to find/replace certain words
- Logic App : as the tool to create our process via a visual tool
And in the end, we’ll see something like this ;
Continue reading “How to roll your own rss2twitter bot in 15 minutes in Azure?”
With one of my flows, I was using an Azure function to generate a filename for my Azure logic app. This name was generated based on the date…
What did I see happening…
As I the script ran just after midnight, I saw that I was getting the day before instead of the actual date.
Continue reading “Changing the timezone on your Azure Webapp / App Service / Function”
There are various articles/blogs/etc that compare logicapps vs flow vs functions vs azure automation, etc… Though there was one use case where I often struggled what to use ;
What to use when I want to retrieve a file from X on a Y timed interval?
Azure Functions? Great abstraction, though the output files have random names. Sometimes / Often I want to be able to control that.
Flow? Doesn’t allow the customization I was looking for. More to integrate existing / popular services.
Azure Automation? Very good and gets the job done. The only downside, you need to code a lot of logic yourself.
Azure Logic Apps? Shows potential, but doesn’t let you include custom functions. Or does it…?!?
You can link Azure Functions to Logic Apps and create some the flow I was looking for.
So what do I want to do?
- On a daily basis
- Retrieve content from an authenticated API
- Save the content to a Blob storage
And afterwards I’ll use other services to process that data. 🙂
What do I want to achieve? On a daily basis I want to retrieve data from a service provider that serves sports data. And if you are looking for such a thing, check out MySportsFeed! So back to our proof-of-concept; how will this look in Logic Apps?
Let’s dissect this flow…
Continue reading “Azure : Mixing LogicApps & Functions as a periodic data retriever”
A few days ago my connectors arrived for my latest PoC on Azure. So today I’m writing about my experience in using a RaspberryPi with a temperature & humidity sensor and to save the telemetry data in Azure. For this we’ll be using Azure Event Hub as an ingress mechanism, and Azure Functions to storage the events towards an Azure Storage Account. My next venture will be to use this data to create reports and maybe (on the long run) do some machine learning. For the latter, I’m pondering about linking this system to my ebus system of my heating system. That way I could correlate the data from the various censors (RPi, Thermostat & outside sensor) in combination with the heater information & heating schedules. Basically… creating my own Google Nest. 🙂
Sensor : Physical Connection (I2C)
The guys from ThingTank had a spare sensor lying around, which they lend to me for my PoC… This was a “Grove – Temperature&Humidity Sensor (High-Accuracy & Mini)“. As you can see in the picture, underneath, this one has an I2C connector. We see four connections ; GND, VCC, SDA & SCL.
Continue reading “Azure IoT : From RaspberryPi with Sensor to Azure Storage Table by using a serverless architecture”
Azure functions is the “serverless” offering on Azure. Serverless doesn’t mean there aren’t any servers, but it’s rather a platform service where you can run your own snippets of code. Each code can be triggered, where it then can have an in and/or output. In terms of billing, you pay for the amount of runtime you consume.
Imagine having a stateless web front end which publishes state onto a queue or storage account. Then Azure functions is being triggered and your business logic starts. Sounds familiar? Yes, because that’s a typical flow you normally do in your application stacks. Now you can segment that even more into services you do not need to manage.
Today I’m going to show you a very brief demo on using an Azure Event Hub as trigger/input for an Azure function. So what’s the flow we’ll be doing?
- A client will put a message on the event hub queue
- This will trigger an Azure function
- The function will save the contents of the message onto a storage account
Sound simple right? And it actually is that simple to do too!
Continue reading “Azure Functions : A quick demo when using Event Hub”