Building a reactive solution with Azure Event Grid

With Event, Grid offers a mechanism for reactive event handling. An application can subscribe to events and handle them accordingly — for instance, handle events like storage blob events, provisioning notifications in an Azure subscription, IoT device signals, or even custom event.

The concept of Azure Event Grid is simple — it is a pub-sub model. An event source pushes events to Azure Event Grid, and event handlers subscribe to events. An event source or publisher — an Azure service like Storage, IoT Hub, or a third party source emits an event, for instance, blobCreated or blobDeleted. You can send the event to a topic. Each topic can have one or multiple subscribers (event handlers). You can configure an Azure Service, if supported, as an event publisher or you create a custom Azure Event Grid Topic. Subsequently, the event handlers in Azure like Functions, WebHook, and Event Hubs can react to the events and process them.

The pub-sub model of Azure Event Grid is not similar to service bus Topics and subscriptions. Moreover, Topics and subscription are push-pull, while Azure Event Grid is push-push. Furthermore, there are other differences too, as shown in the picture below.

Consider a function listening to a blob storage container and each time a new blob is created it will send a message to a message queue. Subsequently, you can listen to the queue with another function, process the blob with the function and call two other functions (or place a queue between that function and the other two functions). The diagram below shows how pull-push would work using a Service Bus Queue.

With Event Grid, you can have a different approach as shown in the diagram below.

You can view Event Grid as the backbone of event-driven computing in Microsoft Azure — and it brings a few benefits to the enterprise. With the push-push pub-sub model you can be more efficient with resources. Event Grid delivers besides efficiency:

These benefits will become apparent in this blog post.

With Event Grid in Azure, you can use as a developer can or adopt reverse dependencies in an existing cloud solution — where services are supporting business processes by depending on each other. In this type of solution, each service has some logic to communicate with the others across the architecture. Moreover, the solution or architecture becomes complex or spaghetti of services.

However, with leveraging Event Grid, you can loosen these dependencies by making the services more event-driven. The services communicate by pushing events to a central place, i.e., Event Grid.

A typical use-case or sample of Event Grid is processing an image leveraging the Vision API. Azure Blob Storage emits an event blobCreated, which is subsequently handled by a function or Logic App subscribing to that event. In the following scenario, Event Grid plays a crucial role in handling blobCreated events where several functions subscribe to that event.

In the above scenario, a camera registers every car going through a toll booth. The picture of the vehicle includes a license plate. These images are uploaded in batches to an Azure Storage account — configured as event publisher. Every image is stored as a blob and thus results in an event being emitted to the Event Grid topic. Subsequently, several functions subscribe to the events:

The above scenario shows how a developer can build several functions to handle the events. Moreover, the developer is a part of making reactive components, i.e., functions (tasks) in this case. The process of managing events (car passing a toll booth) is through a loosely coupled, scalable, and serverless type of solution. In the following paragraphs, we will further discuss some of the critical concepts of Event Grid with the given scenario.

The events in Event Grid adhere to a predefined schema — a JSON schema with several properties. Furthermore, events are sent as a JSON array to Event Grid with a maximum size of 1 MB — each event in the array is limited to 64 KB. In case you send an array of event exceeding the limits — you will receive the HTTP response 413 Payload Too Large. Note that even a single event is sent as an array.

The following schema shows the properties of an event:

[
{
"topic": string,
"subject": string,
"id": string,
"eventType": string,
"eventTime": string,
"data":{
object-unique-to-each-publisher
},
"dataVersion": string,
"metadataVersion": string
}
]

For example, the schema published for an Azure Blob storage event in our scenario:

The image processing function in our scenario can handle the event and use the URL in the data payload to download the image from the storage account container. The Rest Call pushes the downloaded image to the Computer Vision endpoint. Finally, this function will read the JSON result and push to another Event Grid topic as an event.

The security of Event Grid consists of several parts. The first part is the Role-based Access Control (RBAC) on the Event Grid resource — where the person creating a new subscription needs to have the Microsoft.EventGrid/EventSubscriptions/Write permissions. Second, there is a validation on WebHooks — the first and foremost mechanism for listening to events — every newly registered WebHook needs to be validated by Event Grid first. This is what is implicitly happening when creating subscriptions to events in an event resource like Storage Account V2 — you can hook events to a Function or Logic App using the events feature in the storage account. In our scenario, the arrangement of subscriptions to events is implicit.

In case you want to subscribe to events using custom code, i.e., the on-premise client you will need to respond to the validation token the Event Grid will send out. The validation request Event Grid will send to the client will look like below:

[{
"id": "2d1781af-3a4c-4d7c-bd0c-e34b19da4e66",
"topic": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
"subject": "",
"data": {
"validationCode": "512d38b6-c7b8-40c8-89fe-f46f9e9622b6"
},
"eventType": "Microsoft.EventGrid.SubscriptionValidationEvent",
"eventTime": "2018-01-25T22:12:19.4556811Z",
"metadataVersion": "1",
"dataVersion": "1"
}]

You can view Event Grid as serverless that is you do not have to manage any infrastructure, you pay per operation, and Microsoft handles the scaling for you. Event Grid leverages Service Fabric under the hood and thus scales when the workload increases.

Event Hubs and Event Grid together work seamlessly in ingestion (fan-in) and push (fan-out) scenarios. Both scale easily when workloads increases and do not require you manage infrastructure. In the following situation, the capture feature will send the wind farm data (per turbine) to the event hub.

This blog discusses Event Grid scenarios where Event Grid plays a central role, and how developers can leverage this service. Another scenario showed how Event Grid could collaborate with Event Hubs in high volume ingestion and migration of event data.

Related posts:

Optimizing landing pages for conversion is a great skill that will open up your funnel’s floodgates, increasing leads at all stages. No matter what download, opportunity, engagement, webinar, or…

There was once this librarian who told his assistant to label all the books in the book shelf. The librarian told him to finish this task in two days. The assistant walked all the rows and labelled…

I have been sitting on this for several days, mostly because everything I’ve been feeling was so big and so tangled that I didn’t, and still kind of don’t, know how to write about it. And a large…