the cloud way

Azure Storage Blobs .Net SDK v12 upgrade guide and tips

Published Reading time
image

 

Introduction

Updated 5/1/2020 with notes from the Azure SDK Team. Thanks Ted and Jon!

According to the Azure SDK announcement back in November 2019, the v12 SDK for Azure Blob Storage showed a 227% improvement in downloading 1GB blobs. With this kind of performance increase, I thought it was about time to make jump to the new SDK version.

In this post we will look specifically the Azure Blob SDK v12 change highlights from the older versions. Let's start with the basic namespace and class name changes and then highlight the changes in a few common scenarios for block blobs.

Warning

If you are not familiar with the Azure Blob Storage SDK already, read the quick start instead.

Name Changes

//Old
<PackageReference Include="Microsoft.Azure.Storage.Blob" Version="10.0.3" />
//

//New v12
<PackageReference Include="Azure.Storage.Blobs" Version="12.4.0" />
// Old
using Microsoft.Azure.Storage;
using Microsoft.Azure.Storage.Blob;
using Microsoft.Azure.Storage.Blob.Protocol;

//-------------------

// New v12
using Azure;
using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;

Package Old Microsoft.Azure.Storage.Blob v12 Azure.Storage.Blobs
Namespace Microsoft.Azure.Storage.Blob.Protocol Azure.Storage.Blobs.Models
Namespace Microsoft.Azure.Storage.Blob Azure.Storage.Blobs
Namespace Microsoft.Azure.Storage Azure
Class CloudBlobClient BlobServiceClient
Class CloudBlobContainer BlobContainerClient
Class CloudBlockBlob BlobClient or BlockBlobClient
Class StorageException RequestFailedException
Class BlobErrorCodeStrings BlobErrorCode

Service and Container Basics

Note

Notice the name changes from CloudBlobClient to BlobServiceClient and CloudBlobContainer to BlobContainerClient. Also, the method changes throughout the SDK from Reference to Client.

// Old
// Create CloudBlobClient from the connection string.
    CloudBlobClient blobClient = CloudStorageAccount.Parse("StorageConnectionString")
                                .CreateCloudBlobClient();

// Get and create the container for the blobs
    CloudBlobContainer container = blobClient.GetContainerReference("BlobContainerName");
    await container.CreateIfNotExistsAsync();

//-------------------

// New v12
// Create BlobServiceClient from the connection string.
    BlobServiceClient blobServiceClient = new BlobServiceClient("StorageConnectionString");

// Get and create the container for the blobs
    BlobContainerClient container = blobServiceClient.GetBlobContainerClient("BlobContainerName");
    await container.CreateIfNotExistsAsync();

Common Blob Operations

Now, let's dive into a few common block blob operations and highlight the code changes needed.

CloudBlockBlob vs. BlobClient

//Old CloudBlockBlob

    CloudBlockBlob blob = container.GetBlockBlobReference("BlobName");

//-------------------

//New v12 BlobClient

    BlobClient blob = container.GetBlobClient("BlobName");

//

Uploading Json Text

Simple Json text uploaded to a blob.

//Old

    string jsonEntityContent = "{ }";

    CloudBlockBlob blob = container.GetBlockBlobReference("BlobName");
    blob.Properties.ContentType = "application/json";

    await blob.UploadTextAsync(jsonEntityContent);
    await blob.SetPropertiesAsync();

//-------------------

//New v12

    string jsonEntityContent = "{ }";

    BlobClient blob = container.GetBlobClient("BlobName");
    await blob.UploadAsync(new MemoryStream(Encoding.UTF8.GetBytes(jsonEntityContent)), 
        new BlobHttpHeaders()
        {
            ContentType = "application/json"
        });

//

Download Json Text With Exception Handling

Method GetEntityBlobAsync<Entity> accepts a CloudBlockBlob or BlobClient and returns a json entity from the text downloaded from the blob. If the blob doesn't exist, the 404 error is handled and returns a null reference for the json entity.

//Old

CloudBlockBlob blobJson = container.GetBlockBlobReference("BlobName");
Entity jsonEntity = await GetEntityBlobAsync<Entity>(blobJson);

public async Task<Entity> GetEntityBlobAsync<Entity>(CloudBlockBlob blobJson)
 where Entity : class, new()
{
    try
    {
        using (Stream s = await blobJson.OpenReadAsync())
        {
            using (StreamReader sr = new StreamReader(s, Encoding.UTF8))
            {
                using (JsonReader reader = new JsonTextReader(sr))
                {
                    JsonSerializer serializer = new JsonSerializer();
                    return serializer.Deserialize<Entity>(reader);
                }
            }
        }
    }
    catch (StorageException storageEx)
        when (storageEx.RequestInformation.ErrorCode 
                == BlobErrorCodeStrings.BlobNotFound)
    {
        return null;
    }
}

//-------------------

//New v12 - Updated 5/1/2020 

BlobClient blobJson = container.GetBlobClient("BlobName");
Entity jsonEntity = await GetEntityBlobAsync<Entity>(blobJson);

public async Task<Entity> GetEntityBlobAsync<Entity>(BlobClient blobJson)
 where Entity : class, new()
{
    try
    {
        using (MemoryStream s = new MemoryStream())
        {
            await blobJson.DownloadToAsync(s);
            using (StreamReader sr = new StreamReader(s, Encoding.UTF8))
            {
                using (JsonReader reader = new JsonTextReader(sr))
                {
                    JsonSerializer serializer = new JsonSerializer();
                    return serializer.Deserialize<Entity>(reader);
                }
            }
        }
    }
    catch (RequestFailedException ex)
        when (ex.ErrorCode == BlobErrorCode.BlobNotFound)
    {
        return null;
    }
}

//
Note

Updated 5/1/2020 with notes from the Azure SDK Team.

  • Since the storage account is based on transaction volume, it is better to handle the 404 (BlobNotFound) exception because that is just one api call, as opposed to checking the BlobClient.ExistsAsync() first and then making the BlobClient.DownloadAsync() call (2 api transactions).
  • The Exists() call followed by another operation is begging to become a race condition as your application scales. Avoid the Exists() methods whenever possible. A lot of operations support ETags via the conditions parameter if you want to get fancy. - Ted @ MSFT
  • Prefer DownloadTo instead of Download. That'll run through our parallel download code path. Users can play with the transfer options if they want to carefully tune performance for their specific scenarios. - Ted @ MSFT
  • Consider upgrading to System.Text.Json while you're converting things. We're using that from all of our new wave of Azure client libraries. - Ted @ MSFT

Get All Blobs in a Container

This example shows how to page through all of the blobs in a container and return a CloudBlockBlob or BlobClient, respectively. Notice, the Container.ListBlobsSegmented() is now replaced with Container.GetBlobs() or Container.GetBlobsAsync().

//Old

public IEnumerable<CloudBlockBlob> GetAllBlobs(CloudBlobContainer container)
{
    BlobContinuationToken token = new BlobContinuationToken();
    while (token != null)
    {
        var blobSegment = container.ListBlobsSegmented(string.Empty, true, 
            BlobListingDetails.None, 100, 
            token, 
            new BlobRequestOptions() , 
            new Microsoft.Azure.Storage.OperationContext() );

        foreach (var blobItem in blobSegment.Results)
        {
            CloudBlockBlob blockBlob = blobItem as CloudBlockBlob;
            if (blockBlob != null)
            {
                yield return blockBlob;
            }
        }
        token = blobSegment.ContinuationToken;
    }
}

//-------------------

//New v12
// Updated 5/1/2020
public IEnumerable<BlobClient> GetAllBlobs(BlobContainerClient container)
{
    foreach (BlobItem blob in container.GetBlobs(BlobTraits.None, BlobStates.None, string.Empty))
    {
        yield return container.GetBlobClient(blob.Name);
    }
}

// -- or --

public async IAsyncEnumerable<BlobClient> GetAllBlobsAsync(BlobContainerClient container)
{
    await foreach (BlobItem page in container.GetBlobsAsync(BlobTraits.None, BlobStates.None, string.Empty))
    {
        yield return container.GetBlobClient(blob.Name);
    }
}

//
Warning
  • The IAsyncEnumerable interface is only available in C# 8 and higher. You must target .netstandard2.1 or .NET Core 3 or higher for this code to work. Or, enable C# 8 in the project file and target .netstandard2.0 or .NET Core 2.1 with <LangVersion>8.0</LangVersion> <PackageReference Include="Microsoft.Bcl.AsyncInterfaces" Version="1.1.0" />
  • You can use IAsyncEnumerable with older versions of C#. You just lose the new await foreach syntax. - Ted @ MSFT
  • Most people shouldn't be enumerating using explicit continuation tokens any longer. When you use our Async/Pageable<T> it's automatically doing that behind the scenes for you. Most people shouldn't even bother enumerating by page unless you want explicit control over the number of requests that are being made or you plan to pause/resume the enumeration via continuation tokens. - Ted @ MSFT

Summary

This article was intended to give the highlights of upgrading your project from Azure Storage Blob .NET SDK v11 and lower to the more performant v12. First, we reviewed naming conventions. Also, we looked at some common examples between the old v11 and lower code and new v12 code to expedite your upgrade. Finally, we pointed out a few breaking changes and fixes like:

  • Namespace and Class mapping
  • Exceptions and error handling
  • Where the commonly used classes and operations live now

Open a discussion

Open Source Projects

 
image
Project Heading

Project intro lorem ipsum dolor sit amet, consectetuer adipiscing elit. Cum sociis natoque penatibus et magnis dis parturient montes.

NuGet Badge   NuGet Badge

image
Project Heading

Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus.

NuGet Badge   NuGet Badge

image
Project Heading

Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus.

NuGet Badge

image
Project Heading

Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus.

NuGet Badge