How to use AWS Secrets Manager client-side caching in .NET

AWS Secrets Manager now has a client-side caching library for.NET that makes it easier to access secrets from .NET applications. This is in addition to client-side caching libraries for Java, JDBC, Python, and Go. These libraries help you improve availability, reduce latency, and reduce the cost of retrieving your secrets. Secrets Manager cache library does this by serving secrets out of a local cache and eliminating frequent Secrets Manager API calls.

AWS Secrets Manager enables you to automatically rotate, manage, and retrieve secrets throughout their lifecycle. Users and applications can access secrets through a call to Secrets Manager APIs, eliminating the need to hardcode sensitive information in plain text. It offers secret rotation with built-in integration for AWS services such as Amazon Relational Database Service (Amazon RDS) and Amazon Redshift, and it’s also extensible to other types of secrets. Secrets Manager enables you to control access to secrets using fine-grained permissions, and all actions on secrets, including retrievals, are traceable and auditable through AWS CloudTrail.

AWS Secrets Manager client-side caching for .NET extends benefits of the AWS Secrets Manager to wider use cases in .NET applications. These extra benefits are now available without having to spend precious time and effort on developing your own caching solution.

In this post, I’ll discuss the following topics:

  • The benefits offered by Secrets Manager client-side caching library for .NET
  • How Secrets Manager client-side caching library for .NET works
  • How to use Secrets Manager client-side library in .NET applications
  • How to extend Secrets Manager client-side library with your own custom logic

The benefits offered by Secrets Manager client-side caching library for .NET

Client-side caching is benefitial in following ways:

  • Availability: Network links sometimes suffer slowdowns or intermittent breaks. Client-side caching can significantly improve availability by eliminating a large number of API calls.
  • Latency: Retrieving secrets through API calls includes the network latency. Retrieving secrets from the local cache eliminates that latency and, therefore, improves performance.
  • Cost: Each API call to a Secrets Manager endpoint encounters a small charge. Using a local cache saves costs associated with API calls.

Using a client-side cache is a best practice; however, in the same way that you don’t want to reinvent the wheel everytime you need one, crafting your own client-side caching solution is suboptimal. The Secrets Manager client-side caching library relieves you from writing your own client-side caching solution while still giving you its benefits. Furthermore, it includes best practices such as:

  • Automatically refreshing cached secrets: the library periodically updates secrets to ensure your application gets the most recent version of a secret. You can control and change refresh intervals using configuration properties.
  • Integration with your applications: To use this library, just add the dependency to your .NET project and provide the identifier to the secret you want to access in your code.

How Secrets Manager client-side caching library for .NET works

The library is implemented in .NET Standard. This means you can reuse the same library in projects of all flavors of .NET, including .NET Framework, .NET Core, and Xamarin.

Note: Because the AWS Secrets Manager client-side caching library depends on Microsoft.Extensions.caching.memory, make sure you add it to your project dependencies.

As an extension to Secrets Manager .NET SDK, the cache library provides you an alternative to direct invocation of Secrets Manager API methods. You invoke cache library methods, and if the value doesn’t exist in the cache, the cache library invokes Secrets Manager methods on your behalf.

The default refresh interval for “current” version of secrets (the latest value stored in Secrets Manager for that secret) is 1 hour. This is because latest version may change from time to time. The library allows you to configure this frequency to higher or lower per your specific application requirements.

If you request a specific version of a secret by specifying both secret ID and secret version parameters, by default the library sets refresh interval to 48 hours. Since each version of a secret is immutable, there is no need to refresh them frequently.

You can also enable “Last known good value caching” to provide some protection in cases of transient network issues or service outages. If this is enabled, the cache will keep track of the last known good secret value, and in the event of an error occurring while refreshing a secret value from the service, the cache will return the last known good value. This feature is disabled by default, and can be enabled by setting the EnableLastKnownGoodValueCaching property of SecretsManagerCacheOptions class to true. You can pass your instance of SecretsManagerCacheOptions to SecretsManagerCache constructor.

The cache library provides a thread-safe implementation for both cache-check as well as entry populations. Therefore, simultaneous requests for a secret that is not available in the cache will result in a single API request to SecretsManager.

How to use Secrets Manager client-side caching library in .NET applications

You can add Secrets Manager client-side caching library to your projects either directly or through dependency injection. The dependency package is also available through NuGet. In this example, I use NuGet to add the library to my project. Open the NuGet Package Manager console and browse for AWSSDK.SecretsManager.Caching. Select the library and install it.
 

Figure 1: Select the AWSSDK.SecretsManager.Caching library

Figure 1: Select the AWSSDK.SecretsManager.Caching library

Before using the cache, you need to have at least one secret stored in your account using AWS Secrets Manager. To create a test secret:

  1. Go to the AWS Console, and then select AWS Secrets Manager.
  2. Select Store a new secret.
  3. For secret type, select Other type of secret, and then add three key/value pairs as shown here:
    
        {
          "Domain": "<yourDomainName>",
          "UserName": "<yourUserName>",
          "Password": "<yourPassword>"
        }  
        

  4. Next, create a cache object, and then invoke its methods with appropriate parameters. Below is a code snippet using AWS Secrets Manager client-side cache library to access our secret. Notice this snippet assumes you’ve added Newtonsoft.Json library to your project:
    
        public MyClass : IDisposable
        {
                private readonly IAmazonSecretsManager secretsManager;
                private readonly ISecretsManagerCache cache;
            
                public MyClass()
                {
                    this.secretsManager = new AmazonSecretsManagerClient();
                    this.cache = new SecretsManagerCache(this.secretsManager);
                }
        
                public void Dispose()
                {
                    this.secretsManager.Dispose();
                    this.cache.Dispose();
                }
            
                public async Task<NetworkCredential> GetNetworkCredential(string secretId)
                    {
                        var sec = await this.cache.GetSecretAsync(secretId);
                        var jo = Newtonsoft.Json.Linq.JObject.Parse(sec.SecretString);
                        return new NetworkCredential(
                            domain: jo["Domain"].ToObject<string>(),
                            userName: jo["Username"].ToObject<string>(),
                            password: jojo["Password"].ToObject<string>());
                    }
                }        
        

For ASP.NET projects, you can use the library with dependency-injection. To do this, you first have to register Secrets Manager caching to the dependency injection service collection in the Startup class of your ASP.NET project:


public class Startup
{
    public void ConfigureServices(IServiceCollection services)
    {
        services.AddSecretsManagerCaching();
    }
}

Then, you’ll be able to consume the cache using constructor injection in your classes.


public MyClass : IDisposable
    {
        private readonly ISecretsManagerCache cache;

        public MyClass(ISecretsManagerCache cache)
        {
            this.cache = cache;
        }

        public async Task<NetworkCredential> GetNetworkCredential(string secretId)
        {
            var sec = await this.cache.GetSecretAsync(secretId);
            var jo = Newtonsoft.Json.Linq.JObject.Parse(sec.SecretString);
                        return new NetworkCredential(
                        domain: jo["Domain"].ToObject<string>(),
                        userName: jo["Username"].ToObject<string>(),
                        password: jojo["Password"].ToObject<string>());
        }
    }    

How to add in-memory encryption and other custom extensions

The Secrets Manager caching library is designed to be extendable with your own custom logic. One possibility is to extend its implementation to include in-memory encryption of cached secrets to add another layer of protection on your retrieved secrets. For this purpose, you have to manually implement two of the interfaces included in the library. The library includes SecretCacheEntry class, implementing the interface ISecretCacheEntry. This is the object that stores secrets in memory. You could create another class implementing the same ISecretCacheEntry interface to add in-memory encryption/decryption.


public class EncryptedSecretCacheEntry : ISecretCacheEntry
    {
        public EncryptedSecretCacheEntry(GetSecretValueResponse response, TimeSpan expiry)
        {
            this.VersionId = response.VersionId;
            this.LastRetreived = DateTime.UtcNow;
            this.Name = response.Name;
            this.Expires = this.LastRetreived.Add(expiry);

            if (response.SecretBinary != null && response.SecretBinary.Length > 0)
            {
                using (var ms = response.SecretBinary)
                {
                    this.SecretBinary = ms.ToArray();
                }
            }
            else
            {
                this.SecretString = response.SecretString; 
            }
        }
    
        private byte[] _EncryptedSecretString;
        public string SecretString
        {
            get { return MyCustomCipherService.DecryptString(_EncryptedSecretString); }
            set { _EncryptedSecretString = MyCustomCipherService.EncryptString(value); }
        }
    
        private byte[] _EncryptedSecretBinary;
        public byte[] SecretBinary
        {
            get { return MyCustomCipherService.Decrypt(_EncryptedSecretBinary); }
            set { _EncryptedSecretBinary = MyCustomCipherService.Encrypt(value); }
        }
    
        public string VersionId { get; private set; }

        public string Name { get; private set; }

        public string LocalId => $"{this.Name}:{this.VersionId}";

        public DateTime LastRetreived { get; private set; }

        public DateTime Expires { get; private set; }
    } 

The second step is to implement the ISecretCacheEntryFactory class:


public class EncryptedSecretCacheEntryFactory : ISecretCacheEntryFactory
    {
        public ISecretCacheEntry CreateEntry(GetSecretValueResponse response, TimeSpan expiry)
        {
            return new EncryptedSecretCacheEntry(response, expiry);
        }
    }

Having these two classes, I can now modify the constructor of my SecretsUserClass to add my custom encryption logic to Secrets Manager cache library:


public SecretsUserClass()
    {
        this.secretsManager = new AmazonSecretsManagerClient();
        this.cache = new SecretsManagerCache(this.secretsManager, new   EncryptedSecretCacheEntryFactory(), new SecretsManagerCacheOptions(), null);
    }

You could even go further and fully customize the cache by implementing ISecretsManagerCache or implementing a child class that inherits functionality of SecretsManagerCache and adds new methods to it.

Conclusion

It’s critical for enterprises to protect secrets from unauthorized access and adhere to various industry or legislative compliance requirements. Mitigating the risk of compromise often involves complex techniques, significant effort, and costs, such as applying encryption, managing vaults and HSM modules, rotating secrets, audit access, and so on. Because the level of effort is high, many developers tend to use the much simpler, but substantially riskier, alternative of hard-coding secrets in application code, or simply storing secrets in plain-text format. These practices are problematic from the security and compliance point of view, but they need to be understood as symptoms of the more fundamental problem of complexity in the systems enterprises have built. To address the problem of weak security and compliance practices, you have to address the problem of complexity. Complex systems can be simplified and made more secure when they are reusable, accessible, and are automated, needing no human interaction.

In this post, I’ve shown how you can improve availability, reduce latency, and reduce the cost of using your secrets by using the Secrets Manager client-side caching library for .NET. I also showed how to extend it by implementing your own custom logic for more advanced use-cases, such as in-memory encryption of secrets.

To get started managing secrets, open the Secrets Manager console. To learn more, read
How to Store, Distribute, and Rotate Credentials Securely with Secret Manager or refer to the Secrets Manager documentation. See AWS Region Table for the list of AWS regions where Secrets Manager is available.

If you have feedback about this blog post, submit comments in the Comments section below. If you have questions about this blog post, start a new thread in the Secrets Manager forum.

Want more AWS Security how-to content, news, and feature announcements? Follow us on Twitter.

Sepehr Samiei

Sepehr is currently a Senior Solutions Architect at AWS. He started his professional career as a .Net developer, which continued for more than 10 years. Early on, he quickly became a fan of cloud computing and loves to help customers utilise the power of Microsoft tech on AWS. His wife and daughter are the most precious parts of his life, and he and his wife expect to have a son soon!