Azure Key Vault integration with .NET Web App using Managed Identity

 

Summary: 

Very good article. How to integrate web app with key vault and clear steps on how to set up permissions .

Further Reading: 

New features in .net 8 and accessing secrets 

Implementing Azure Key Vault in C# for Secure Configuration

Azure Key Vault

In this article, I will explain securing the secrets, passwords, connection strings, etc., using the Azure key vault. and integrating the same Azure key vault in the .net web API application using managed identity to access the securely stored credentials in the Azure key vault. So let's begin with what an Azure key vault means and its need.

What is Azure key vault, and why would we need it?

Secrets are anything you want to control access to, like client secrets, API keys, passwords, certificates, or cryptographic keys. Nowadays (for various reasons) everyone wants to store secrets securely where any unauthorized entity can't access them. It becomes more challenging when we have to manage secrets in the cloud. That's where the 'Azure key vault' comes to the rescue. It is one of the most popular Azure offerings used to store secrets and credentials much more securely.

We should not store our secrets and credentials in the config file as everyone can access it, and it will also be exposed to the repo level. We should store those securely.

How to access it?

To perform any operations with the key vault or if you want to access it in your application, you need to be authenticated first. This can be achieved in three ways.

1. Managed Identity

2. Using service principals and certificates.

3. Using service principal and secret.

In this article, we will see how we can connect the Azure key vault from a web app using Managed Identity, the latest, most popular, and easy way to access secrets from the key vault without writing a lot of code.

The prerequisites are:

1. .NET 6 (I have used .NET 6, but you can use .NET core 5 or .NET core 3.1)
2. Azure key vault created in Azure portal.
3. Web app created in an Azure portal where we will deploy our code.
4. Visual Studio 2022 (You can use 2019 also for earlier .NET core versions).

Access secrets from a web app using the managed identity

Managed identity is a concept in the Microsoft Azure cloud that eliminates the need for developers to manage credentials after deploying their applications. With managed identity, we can access different Azure services securely without storing credentials like connection strings or passwords.

There are two types of managed identities.

1. System assigned — it is created as a part of Azure resources like Azure app service and virtual machine. When we delete the resource, related managed identity is also deleted, which means it shares a life cycle.
2. User-assigned managed identity — created as a stand-alone Azure resource that does not share a life cycle and has to be explicitly deleted.

We will use a system-assigned managed identity for this article/implementation. Let's begin by creating a web app to host our web API created with .NET 6.

Create a Web app in the Azure portal

1. Navigate to the Azure portal's home page and click Create Resource from the left menu.

Press enter or click to view image in full size

Create a web app in the Azure portal

2. From the list, select Web app

Press enter or click to view image in full size

Selecting a web app to create

3. As shown in the below screen, provide the details.

• Subscription
• resource group
• name — the name of the web app, which should be unique
• publish — select 'Code' as we will publish our code on the web app.
• Runtime stack — .NET 6 LTS
• Operating system — Windows
• Region — Central US (you can provide others if you want).

Like this, provide the details.

Press enter or click to view image in full size

Form to create web app service in Azure

Enable Azure Managed Identity

Now we have created a web app in Azure, let's enable managed identity for that web app.

Press enter or click to view image in full size

Enable system identity for web app

As above screen,

• Go to the created web app and navigate to the Identity section from the left menu under the settings option. You will see System assigned and a user assigned, Select the system assigned and toggle that status button to 'On'.
• Once you enable the managed identity, you will see the Object ID created on the above screen; copy that, as we will need it for granting access to the key vault.

Create Azure key vault

Like the web app, When we click the Create resource button, we will see the Key Vault option below.

Press enter or click to view image in full size

Create Azure key vault

Create the key vault by providing the details.

Press enter or click to view image in full size

Form to create an Azure key vault

Once you create a key vault, you will get one key vault URI which will be used to integrate the key vault into the web app.

Press enter or click to view image in full size

Azure key vault overview dashboard with URI

We have created a web app and its managed identity, Azure key vault. The Next step is to grant access to the Azure key vault to the web app created above using the access policy.

Grant access to the secret

From the left menu of the Key Vault page, select Access Policies, and then click Create, as shown in the following picture:

Press enter or click to view image in full size

Creating an access policy for the Azure key vault

Press enter or click to view image in full size

Options for creating access policy for Azure key vault

• From the 'Configure from template' dropdown, select The 'Key, Secret & Certificate management' option.
• Then select Get and List as shown above. This will allow us to list and read the secret values stored in the Azure Key Vault instance.
• Click Next, and you will see the 'principal' tab; in the search box, provide the object ID of created managed identity of a web app. Once you enter the object ID, the web app name will get displayed; select that as shown below.

Press enter or click to view image in full size

Select the service to grant access to the Azure key vault

Click Next, Next, and finally, Create. It will create an access policy for the key vault, which tells us that the web app can now access the key vault secrets using the managed identity.

Deploy a Python API to an Azure App service with Azure container registry and Azure DevOps

 Summary : Article explain deployment of phyton api which get data from sql database . On-Prem database in a SQL server instance using the App Services’ Hybrid connection feature.

Objective

This post is to show how to publish a docker container image to an Azure Container registry, then deploy that image to Azure App Service. The web API will require a connection to an On-Prem database in a SQL server instance using the App Services’ Hybrid connection feature. This uses a very simple python API creation calling stored procedures on a SQL server that returns results in a JSON format. The repository can be found here.

Repository: https://github.com/jaya1971/Py-API

Requirements

1. Basic understanding of Azure App Services, Azure DevOps and Docker.

2. An account with Azure cloud and Azure DevOps.

Overview

1. Create an Azure container registry.

2. Configure the Dockerfile with the necessary updates and installs, along with other references to data that will be brought over from the pipeline, through the Dockerfile and into the API.

3. Create a Docker Registry service connection.

4. Create a library group in your Azure DevOps project to host your secret variables and SQL information.

5. Create a pipeline to build and push the image to the Azure Container Registry.

6. Create an App Service in Azure with predefined settings.

7. Pipeline to deploy the CR image to the App Service.

8. Configure pipeline environment variables and pass them from the pipeline to the Python API.

9. Configure the App Service Hybrid connection to interact with an On-Prem SQL server and database.

Automation Order

I would like to point out that I have two pipelines for automating this process. One for deploying an azure container registry and App Service through terraform. The other for the python API. Because the creation and deployment of an Azure container app service must occur in a certain order. I have parameterized my pipelines with Booleans to run certain parts of the automation so I can deploy resources in a specific order.

Here is the order of deployment:

1. Create an Azure Container Registry to hold your docker image (Manual or through Terraform)(I used a terraform module pipeline).

Get Alex Jaya’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

2. Build and push an image to the Azure Container Registry (I ran this from this current repository in a pipeline by setting the “ Update docker image repo” to True).

3. Create a Linux container azure app service (Manual or through Terraform)(I used a terraform module pipeline). When creating a Linux container azure app service, you must reference the image in the container registry. This is why it is done after pushing the image to the container registry.

4. Deploy image to the Linux container app service. (I ran this from this current repository in a pipeline by setting “Deploy img to app service” to True).

Walkthrough

1. I created a main YAML pipeline to call template YAML based on conditions so it would be able to deploy it to different environments. You’ll notice in my repository, that I’m deploying it to dev or prod. This YAML file is what the pipeline uses when triggered. This YAML file is parameterized. The parameters will select a template at runtime.

• In the main “azure-pipelines.yml” file lines 32–45 contain the variables and library group used to pass variables to the called YAML files.
  - Line 33,34 app: Application name.
  - Line 35,36 repo: The repository created when you pushed the image to the container registry
  - Line 39,40 docker_img_path: is comprised of “<loginServer>/<repository>:<ImageTag>
  - Lines 47–57 contain the references to the called YAML files in the ‘azurepipelines’ folder.
  - Lines 59–71 contain the stages and conditions to determine which YAML file to use.

2. Template YAML files.

• The called pipelines are broken into two jobs: “Build and Push the Image” and “Deploy Image to App Service”.
• The first job “BuildAndPushImage” contains three tasks: “printAllVariables@1”, and two “Docker@2”.
  - printAllVariables@1: lists all pipeline variables. This is great for reviewing directory and artifact locations along with variable information. Secret variable values are not displayed.
  - Docker@2: One is to Build the docker image and the other is to push the docker image to the container registry. Variables used are a hybrid of variables declared in the entry YAML template, ADO project library and project service connection.
  - The “arguments” on line 27 contain the variables from the library group that will be used in the python application.
• The second job “DeployImageToAppService” contains two tasks: “AzureWebContainer@1” and “AzureAppServiceSettings@1”.
  - AzureWebContainer@1: Will deploy the image from the container registry to the app service.
  - AzureAppServiceSettings@1: Contains additional configuration settings.

3. This takes care of the basic setup of having your API’s image in the acr and deployed to your App Service. Since we’re working with an On-Prem database we’ll need to configure a connection that will allow us to access the On-Prem SQL. If you navigate to your App Service’s Log stream (Under Monitoring), you’ll notice an error message that reads something like: “pyodbc.OperationalError: (‘HYT00’, u’[HYT00]……..”. This usually indicates that you do not have a valid connection to your data source, in our case, our On-Prem SQL server. We’ll also have to configure a way to pass the connection string information so our API will properly connect to our data source. The objective is to keep sensitive information hidden. We can do this by creating environment pipeline variables. There are various ways you can do this in Azure DevOps:

• Environment variables
  - Pipeline variables — make sure you mark sensitive values as secret. Pipeline variables are kept in your pipeline and separate from your repository.
  - Library Group variables — These are ADO’s library groups where you will be able to configure environment variables to be used by your pipelines. The library groups can be secured to all pipelines in your project or to specific ones. You can either place your variables directly in this group or sync it with a keyvault located in your Azure subscription. (Link your keyvault to your ADO Library Group)
• Once you’ve chosen how you will store your environment variables, you will want to pass these values to your API in your py app. I’ve intentionally named the variables/parameters slightly different to make it easier to distinguish which direction they are being passed. The process is to:
  - Create the reference in your pipeline: This is reference in lines 32–45 in the main YAML pipeline file.
  - Pass those variables in the template YAML files: Arguments on line 27 will pass these variables to the Dockerfile.
  - Pass those values to your Dockerfile as parameters: Dockerfile: Lines 5–12 contain the passed variables from the YAML file as “ARG”. They are converted into environment variables using the ENV tag.
  - In the py app file, you can reference the environment variables passed by the Dockerfile as “os.getenv(‘<ENV_VARIABLENAME>’)

4. Now since we’re connecting to an On-Prem SQL server, you’ll still receive an error message that references a bad connection to your data source and that’s because in this case we are trying to connect to an On-Prem data source. If your data source was an Azure SQL server, you would be able to successfully run your app service API. Let’s configure a hybrid connection to our On-Prem SQL Server. (Great article with step by step — Hybrid Connection)

• In your Azure app service, navigate to Settings >> Networking >> Hybrid connections >> Add hybrid connection >> Create new hybrid connection. Create your hybrid connection. Your endpoint should be the SQL fqdn that can be found by an On-Prem node where you will install the hybrid connection agent. Copy the “Gateway Connection String”, you’ll need it when you install the agent On-Prem.
• Download the connection manager to install on an On-Prem node. When creating the connection, select manual and paste the “Gateway Connection String” you copied in the previous step. Restart the “Azure Hybrid Connection Manager Service”. Then your hybrid connection should reflect as:

Press enter or click to view image in full size

Challenges and things learned

There were a few challenges and issues that popped up during this development. One of them being how to pass environmental variables to python, which is different depending on the method being used. The others are listed below.

1. SQL Server instances. We have MS SQL Server configured with instances. When I first configured the hybrid connection, it read successful and connected but I was still receiving the pyodbc connection error message. This was because instances use different ports rather than the default 1433. This can be found in SQL Server Configuration Manager >> SQL Server Network Configuration >> Protocols >> TCP/IP >> IP Addresses tab, under IPAll, you will find the TCP Dynamic Port being used. You’ll want to reference the instance in your ADO Pipeline as “server.domain.com,PORTNUMBER”

2. After everything was up and running, it seemed that the connection to the sql server was still busy with another query and would return an error message that read like: “Error # HY000 [Microsoft][ODBC SQL Server Driver]Connection is busy with results for another hstmt)”. I found that setting the MARS (Multiple Active Result Sets) reference in your pyodbc connection string would resolve this issue. This is referenced in Line 38 of the main.py file: “MARS_Connection=YES”

Hope you found this useful!