Intro
Event Grid efficiently and reliably routes events from Azure and non-Azure resources, and distributes the events to registered subscriber endpoints.
Documentation
Tips and Tidbits
Event Grid efficiently and reliably routes events from Azure and non-Azure resources, and distributes the events to registered subscriber endpoints.
Azure Event Grid is an eventing backplane that enables event-driven, reactive programming.
It uses the publish-subscribe model.
Publishers emit events, but have no expectation about how the events are handled.
Subscribers decide on which events they want to handle.
It is strongly recommended to use available messaging products and services that support a publish-subscribe model, rather than building your own.
In Azure, consider using Service Bus, Event Hubs or Event Grid.
Other technologies that can be used for pub/sub messaging include Redis, RabbitMQ, and Apache Kafka.
First, select the Azure resource you would like to subscribe to, and then give the event handler or WebHook endpoint to send the event to.
Event Grid has built-in support for events coming from Azure services, like storage blobs and resource groups.
Event Grid also has support for your own events, using custom topics.
You can use filters to route specific events to different endpoints, multicast to multiple endpoints, and make sure your events are reliably delivered.
There are five concepts in Azure Event Grid:
Events - What happened.
An event is the smallest amount of information that fully describes something that happened in the system.
Every event has common information like: source of the event, time the event took place, and unique identifier.
Every event also has specific information that is only relevant to the specific type of event.
An event of size up to 64 KB is covered by General Availability (GA) Service Level Agreement (SLA).
The support for an event of size up to 1 MB is currently in preview. Events over 64 KB are charged in 64-KB increments.
Event sources - Where the event took place.
Your application is the event source for custom events that you define.
Event sources are responsible for sending events to Event Grid.
Topics - The endpoint where publishers send events.
The publisher creates the event grid topic, and decides whether an event source needs one topic or more than one topic.
A topic is used for a collection of related events.
System topics are built-in topics provided by Azure services.
You don't see system topics in your Azure subscription because the publisher owns the topics, but you can subscribe to them.
To subscribe, you provide information about the resource you want to receive events from.
As long as you have access to the resource, you can subscribe to its events.
Custom topics are application and third-party topics.
When you create or are assigned access to a custom topic, you see that custom topic in your subscription.
Event subscriptions - The endpoint or built-in mechanism to route events, sometimes to more than one handler.
Subscriptions are also used by handlers to intelligently filter incoming events.
A subscription tells Event Grid which events on a topic you're interested in receiving.
When creating the subscription, you provide an endpoint for handling the event.
You can filter the events that are sent to the endpoint. You can filter by event type, or subject pattern.
Event handlers - The app or service reacting to the event.
From an Event Grid perspective, an event handler is the place where the event is sent.
The handler takes some further action to process the event.
You can use a supported Azure service or your own webhook as the handler.
Depending on the type of handler, Event Grid follows different mechanisms to guarantee the delivery of the event.
For HTTP webhook event handlers, the event is retried until the handler returns a status code of
200 – OK.For Azure Storage Queue, the events are retried until the Queue service successfully processes the message push into the queue.
You can authenticate clients that publish events to Azure Event Grid topics, domains, partner namespaces using access key or Shared Access Signature (SAS) token.
For more information, see Using access keys or using Shared Access Signatures (SAS).
Event schemas
Events consist of a set of four required string properties.
The properties are common to all events from any publisher.
The data object has properties that are specific to each publisher.
Event sources send events to Azure Event Grid in an array, which can have several event objects.
When posting events to an event grid topic, the array can have a total size of up to 1 MB.
Each event in the array is limited to 1 MB.
If an event or the array is greater than the size limits, you receive the response
413 Payload Too Large.Operations are charged in 64 KB increments though.
So, events over 64 KB will incur operations charges as though they were multiple events.
Event Grid sends the events to subscribers in an array that has a single event.
You can find the JSON schema for the Event Grid event and each Azure publisher's data payload in the Event Schema store
Properties that are used by all event publisher
For custom topics, the event publisher determines the data object.
[
{
"topic": string,
"subject": string,
"id": string,
"eventType": string,
"eventTime": string,
"data":{
object-unique-to-each-publisher
},
"dataVersion": string,
"metadataVersion": string
}
]
Subscribers use the subject to filter and route events.
Consider providing the path for where the event happened, so subscribers can filter by segments of that path.
if you provide a three segment path like
/A/B/Cin the subject, subscribers can filter by the first segment/Ato get a broad set of events.Those subscribers get events with subjects like
/A/B/Cor/A/D/E.
the Storage Accounts publisher provides the subject
/blobServices/default/containers/<container-name>/blobs/<file>when a file is added to a container.A subscriber could filter by the path
/blobServices/default/containers/testcontainerto get all events for that container but not other containers in the storage account
In addition to its default event schema, Azure Event Grid natively supports events in the JSON implementation of CloudEvents v1.0 and HTTP protocol binding.
CloudEvents is an open specification for describing event data.
Event Delivery Durability
Event Grid provides durable delivery.
It tries to deliver each event at least once for each matching subscription immediately.
If a subscriber's endpoint doesn't acknowledge receipt of an event or if there is a failure, Event Grid retries delivery based on a fixed retry schedule and retry policy.
By default, Event Grid delivers one event at a time to the subscriber, and the payload is an array with a single event.
Event Grid doesn't guarantee order for event delivery, so subscribers may receive them out of order.
When Event Grid receives an error for an event delivery attempt, EventGrid decides whether it should retry the delivery, dead-letter the event, or drop the event based on the type of the error.
If Dead-Letter isn't configured for an endpoint, events will be dropped when the above errors happen.
Consider configuring Dead-Letter if you don't want these kinds of events to be dropped.
Event Grid waits 30 seconds for a response after delivering a message.
After 30 seconds, if the endpoint hasn’t responded, the message is queued for retry.
Event Grid uses an exponential backoff retry policy for event delivery.
If the endpoint responds within 3 minutes, Event Grid will attempt to remove the event from the retry queue on a best effort basis but duplicates may still be received.
You can customize the retry policy when creating an event subscription by using the following two configurations.
An event will be dropped if either of the limits of the retry policy is reached.
Maximum number of attempts - The value must be an integer between 1 and 30. The default value is 30.
Event time-to-live (TTL) - The value must be an integer between 1 and 1440. The default value is 1440 minutes (24 hours)
Output batching
You can configure Event Grid to batch events for delivery for improved HTTP performance in high-throughput scenarios.
Batching is turned off by default and can be turned on per-subscription
Batched delivery has two settings:
Max events per batch - Maximum number of events Event Grid will deliver per batch. Must be between 1 and 5,000.
Preferred batch size in kilobytes - Target ceiling for batch size in kilobytes.
It's possible that a batch is larger than the preferred batch size if a single event is larger than the preferred size.
Delayed delivery
As an endpoint experiences delivery failures, Event Grid will begin to delay the delivery and retry of events to that endpoint.
For example, if the first 10 events published to an endpoint fail, Event Grid will assume that the endpoint is experiencing issues and will delay all subsequent retries, and new deliveries, for some time - in some cases up to several hours.
When Event Grid can't deliver an event within a certain time period or after trying to deliver the event a certain number of times, it can send the undelivered event to a storage account.
This process is known as dead-lettering.
Event Grid dead-letters an event when one of the following conditions is met.
Event isn't delivered within the time-to-live period.
The number of tries to deliver the event exceeds the limit.
By default, Event Grid doesn't turn on dead-lettering.
To enable it, you must specify a storage account to hold undelivered events when creating the event subscription.
Custom delivery properties
Event subscriptions allow you to set up HTTP headers that are included in delivered events.
This capability allows you to set custom headers that are required by a destination.
You can set up to 10 headers when creating an event subscription.
Each header value shouldn't be greater than 4,096 (4K) bytes.
Access Control
Azure Event Grid allows you to control the level of access given to different users to do various management operations such as list event subscriptions, create new ones, and generate keys.
Event Grid uses Azure role-based access control (Azure RBAC)
EventGrid doesn't support Azure RBAC for publishing events to Event Grid topics or domains.
Use a Shared Access Signature (SAS) key or token to authenticate clients that publish events.
If you're using an event handler that isn't a WebHook (such as an event hub or queue storage), you need write access to that resource.
This permissions check prevents an unauthorized user from sending events to your resource.
You must have the Microsoft.EventGrid/EventSubscriptions/Write permission on the resource that is the event source.
You need this permission because you're writing a new subscription at the scope of the resource.
The required resource differs based on whether you're subscribing to a system topic or custom topic
System topic: Need permission to write a new event subscription at the scope of the resource publishing the event.
The format of the resource is:
/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/{resource-provider}/{resource-type}/{resource-name}
For custom topics, you need permission to write a new event subscription at the scope of the event grid topic.
The format of the resource is:
/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/Microsoft.EventGrid/topics/{topic-name}For example, to subscribe to a custom topic named mytopic, you need the Microsoft.EventGrid/EventSubscriptions/Write permission on:
/subscriptions/####/resourceGroups/testrg/providers/Microsoft.EventGrid/topics/mytopic
Receive events by using webhooks
Webhooks are one of the many ways to receive events from Azure Event Grid.
When a new event is ready, Event Grid service POSTs an HTTP request to the configured endpoint with the event in the request body.
Event Grid requires you to prove ownership of your Webhook endpoint before it starts delivering events to that endpoint.
This requirement prevents a malicious user from flooding your endpoint with events.
When you use any of the three Azure services listed below, the Azure infrastructure automatically handles this validation:
Azure Logic Apps with Event Grid Connector
Azure Automation via webhook
Azure Functions with Event Grid Trigger
Endpoint validation with Event Grid events
If you're using any other type of endpoint, such as an HTTP trigger based Azure function, your endpoint code needs to participate in a validation handshake with Event Grid.
Event Grid supports two ways of validating the subscription.
Synchronous handshake: At the time of event subscription creation, Event Grid sends a subscription validation event to your endpoint.
The schema of this event is similar to any other Event Grid event.
The data portion of this event includes a
validationCodeproperty.Your application verifies that the validation request is for an expected event subscription, and returns the validation code in the response synchronously.
This handshake mechanism is supported in all Event Grid versions.
Asynchronous handshake: In certain cases, you can't return the ValidationCode in response synchronously.
Event Grid supports a manual validation handshake.
If you're creating an event subscription, Event Grid sends a
validationUrlproperty in the data portion of the subscription validation event.To complete the handshake, find that URL in the event data and do a GET request to it.
You can use either a REST client or your web browser.
Filter events
When creating an event subscription, you have three options for filtering:
Event types
By default, all event types for the event source are sent to the endpoint. You can decide to send only certain event types to your endpoint
"filter": { "includedEventTypes": [ "Microsoft.Resources.ResourceWriteFailure", "Microsoft.Resources.ResourceWriteSuccess" ] }
Subject begins with or ends with
"filter": { "subjectBeginsWith": "/blobServices/default/containers/mycontainer/log", "subjectEndsWith": ".jpg" }
Advanced fields and operators
To filter by values in the data fields and specify the comparison operator, use the advanced filtering option.
"filter": { "advancedFilters": [ { "operatorType": "NumberGreaterThanOrEquals", "key": "Data.Key1", "value": 5 }, { "operatorType": "StringContains", "key": "Subject", "values": ["container1", "container2"] } ] }
Comparison of services
Service | Purpose | Type | When to use |
|---|---|---|---|
Event Grid | Reactive programming | Event distribution (discrete) | React to status changes |
Event Hubs | Big data pipeline | Event streaming (series) | Telemetry and distributed data streaming |
Service Bus | High-value enterprise messaging | Message | Order processing and financial transactions |
Webhook Event delivery
Webhooks are one of the many ways to receive events from Azure Event Grid.
When a new event is ready, Event Grid service POSTs an HTTP request to the configured endpoint with the event in the request body.
Event Grid requires you to prove ownership of your Webhook endpoint before it starts delivering events to that endpoint.
Azure infrastructure automatically handles this validation for these services:
Azure Logic Apps with Event Grid Connector
Azure Automation via webhook
Azure Functions with Event Grid Trigger
If you're using any other type of endpoint, such as an HTTP trigger based Azure function, your endpoint code needs to participate in a validation handshake with Event Grid.
Synchronous handshake: the data portion of this event includes a
validationCodeproperty.Your application verifies that the validation request is for an expected event subscription, and returns the validation code in the response synchronously.
Asynchronous handshake: Event Grid supports a manual validation handshake.
Event Grid sends a
validationUrlproperty in the data portion of the subscription validation event.To complete the handshake, find that URL in the event data and do a GET request to it.
The provided URL is valid for 5 minutes. During that time, the provisioning state of the event subscription is
AwaitingManualAction.
[{
"id": "135b3c37-5c5b-4019-8509-a3b9c5bbb129",
"topic": "/subscriptions/dd4a7b94-2399-46fc-b2e9-a6ac4ad9fb56/resourceGroups/az204-evgrid-rg/providers/microsoft.eventgrid/topics/az204-egtopic-888382",
"subject": "",
"data": {
"validationCode": "3747BBE8-BEA3-452F-AA7F-16BA33288B99",
"validationUrl": "https://rp-eastus.eventgrid.azure.net:553/eventsubscriptions/az204viewersub/validate?id=3747BBE8-BEA3-452F-AA7F-16BA33288B99&t=2022-02-06T21:29:08.4932495Z&apiVersion=2020-10-15-preview&token=EpJAnYMDBQV5DBZCRdNf114A4%2fNzRfEt1jV0dkUGX8s%3d"
},
"eventType": "Microsoft.EventGrid.SubscriptionValidationEvent",
"eventTime": "2022-02-06T21:29:08.4932495Z",
"metadataVersion": "1",
"dataVersion": "2"
}]
To prove endpoint ownership, echo back the validation code in the validationResponse property, as shown in the following example:
{
"validationResponse": "3747BBE8-BEA3-452F-AA7F-16BA33288B99"
}
you can manually validate the subscription by sending a GET request to the validation URL.
The event subscription stays in a pending state until validated.
The validation Url uses port 553. If your firewall rules block port 553 then rules may need to be updated for successful manual handshake.
0 Comments