23 November 2021 –4 Reading minutes
Queues are a mechanism for disconnecting applications. This allows us to create a workflow for asynchronous processing.
Azure Queue Storage provides a simple and cost-effective solution for handling large amounts of messages. It enables asynchronous communication between different applications. It allows you to disconnect applications, build resilience and also change the scale for traffic bursts.
In this post, let’s explore how to create Azure Queue Storage and use it from the ASP NET Core application. The steps are very similar if you are looking to use it from any .NET application.
To start using the queue, let’s create it first in Azure. The easiest way is to create it using the Azure portal.
In a real-world application, you can use scripts to automate it, or, even better, use infrastructure as a code (IAC) to automate them from your build and deploy pipeline.
But for now, let’s go to the Azure portal and search from ‘Azure Storage’.
Storage as Azure lives under a Storage account in Azure. To create a queue, we must first create a storage account.
Once created, we can add queues under the storage account, as shown below.
With the creation of queue storage, we can start using it from our app. Queue storage is great when you want to disconnect long-term workloads that will occur in the background. This prevents users from waiting until the long introductions are completed.
NuGet package Azure.Storage.Queues Provides the libraries required to connect to Queue Storage. After installation, use
QueueClient Class send and receive messages.
QueueClient Requires a connection string to confirm and verify. It is available from the Azure Portal Access Keys section under the storage account (as shown below).
Azure Queue Storage Access Keys
Later in the post, I’ll show you how to connect to Queue without connecting and using Managed Identity.
Create a new ASP NET Core application using the default template. It has a WeatherForecast controller, which we can integrate with Queue Storage.
Let’s say we now need to post new weather data for our app. But adding weather data involves some processing work, and we want that to happen in the background. Then the user will simply post new weather data, and we will send this message to a queue that will be processed separately.
For now, let’s use
QueueClient And the connection string to connect to Queue Storage. Uses b
SendMessageAsync Method, we can send queue messages.
[HttpPost] public async Task Post([FromBody]WeatherForecast weatherForecast, CancellationToken cancellationToken) var connectionString = "<Connection String>" var queueName = "add-weatherdata"; var client = new QueueClient(connectionString, queueName); var message = JsonSerializer.Serialize(weatherForecast); await client.SendMessageAsync(message, cancellationToken);
POST The method adds a message to the queue and makes it available for processing to any application that listens in the queue.
Messages added to the queue have identification, insertion time, expiration time and queue count in addition to the actual message.
- Income time ← Time to insert the message in the queue
- Expiry Date ← By default, this is seven days, after which the message will automatically expire and be deleted from the queue
- Queuing → Indicates the number of times the message was read from the queue and returned
SendMessageAsync have Methods too busy, Allows you to explicitly set the expiration time and also a visibility timeout.
Visibility timeout indicates the delay after which a message appears in the queue after insertion. So if you set a value of 60 seconds, the message will appear in the queue (and will only be available for processing after 60 seconds).
To create messages that will never expire, set the
timeToLive As a negative value.
Messages that have fallen in line can be collected by another application for processing. Below I have a simple background task that runs every 10 seconds and reads a message from the queue.
protected override async Task ExecuteAsync(CancellationToken stoppingToken) while (!stoppingToken.IsCancellationRequested) _logger.LogInformation("Reading from queue"); var queueMessage = await _queueClient.ReceiveMessageAsync(); if (queueMessage.Value != null) var weatherData = JsonSerializer.Deserialize<WeatherForecast>(queueMessage.Value.MessageText); _logger.LogInformation("New Mesasge Read: weatherData", weatherData); await _queueClient.DeleteMessageAsync(queueMessage.Value.MessageId, queueMessage.Value.PopReceipt); await Task.Delay(TimeSpan.FromSeconds(10));
ReveiveMessagesAsync The method reads one or more messages from the queue. This removes the message from the queue for 30 seconds (by default or specified
Once the message has been processed, the message must be deleted from the queue so that it is not collected again for processing. If not deleted, it will be available back in the queue after the
God Azure Storage Explorer The tool enables the management of storage accounts from desktops. This will allow management of storage accounts across multiple subscribers and accounts.
When reading messages from the queue, the explorer tool uses take a peek Functionality. With Peek, the message is not removed from the queue and is still available for processing for other applications looking at the same queue.
Whenever you need to interact with the Azure Queue from the .NET app, you need an instance of
QueueClient. Instead of explicitly creating a show, it’s easier to inject dependency.
[Microsoft.Extensions.Azure](https://www.nuget.org/packages/Microsoft.Extensions.Azure/) The Nuget package makes it easy to integrate with different Azure customers .NET Dependency Injection (DI) And configuration systems.
ConfigureServices Method e
Startup.cs In class, we can register the QueueClient for the application.
services .AddAzureClients(builder => builder.AddClient<QueueClient, QueueClientOptions>((_, _, _) => var connectionString = "<Connection String>"; var queueName = "add-weatherdata"; return new QueueClient(connectionString, queueName); ); );
After registering in the DI container, we can request a instance of
QueueClient In the constructor. The .NET DI will inject it automatically while running.
public WeatherForecastController( ILogger<WeatherForecastController> logger, QueueClient queueClient) _logger = logger; _queueClient = queueClient;
The code above, when registering e
QueueClient Still using the connection string. Certificate management and ConnectionString can be a bit tricky, so it’s best to avoid having to use them in the first place. Azure Managed Identity eliminates the need for resource string authentication connections.
Want to learn more about DefaultAzureCredential?
In the past, Azure had different ways to validate with different resources. The Azure SDK brings it all under one roof and provides more unified access to developers when connecting to resources in Azure.
To enable a managed identity connection in Azure Queue, go to the Azure portal and add a new role assignment under ‘Access Control (IAM)‘ segment. You can grant access to individuals, groups, or application identities to connect to Queue Storage.
Below I have added my email address to Donates storage queue data role. It allows you to read, edit, and delete messages from the queue. When you are working on your local development, you can configure the IDE to connect and retrieve the token to use with it
The code below no longer uses a connection string to connect to Azure Queue. When running on the local computer, it can automatically pick up the user connected to Visual Studio or other supported IDEs. If the application is running on the Azure infrastructure, run Identity on the resource and grant it permissions using access control (IAM).
var credential = new DefaultAzureCredential(); var queueUri = new Uri("https://youtubestoragetest.queue.core.windows.net/myqueue"); return new QueueClient(queueUri, credential);
I hope this helps you get started with Azure Queue Storage and build offline applications.