Data and Devops practices and tools: Angular, Microservices and Authentication with IdentityServer, MongoDB and Docker

In a previous article of mine, I talked about Microservices and how authenticate an Angular client with them using IdentityServer as authentication authority. I that case, I used the in-memory configuration to simplify the concept, but in a real application we need to save the data in a persistent storage like a database.

Since I used MongoDB in a project for some microservices, I thought it was useful to use the Mongo instance to save also the IdentityServer data, obviously in a separate database; this had interesting implications that are worth telling. Moreover, in a real development scenario, you will never have all the microservices and clients in the same repository, so working on localhost can become a problem, but it is easily solved using dockers and docker-compose. This aspect also has interesting implications because it puts you in front of some networking considerations that are certainly elusive in localhost.

IdentityServer and MongoDB

Let's start with the integration of MongoDB with IdentityServer. There are some libraries ready-to-use for this purpose, but the best ones use Entity Framework as an intermediate layer to store data on MongoDB. I don't understand the use of an ORM with a NoSql database, I think it is a nonsense, but I accepted the idea that it's a convenience to be able to change storage when I reuse something in a different project, because let's face it ... changing databases in a project that is in production is a fairy tale that is told to children and young programmers.

I reused my existing generic implementation of the Repository pattern for MongoDB in the IdentityServer template:

public interface IRepository
{
    IQueryable<T> All<T>() where T : class, new();
    IQueryable<T> Where<T>(Expression<Func<T, bool>> expression) where T : class, new();
    T Single<T>(Expression<Func<T, bool>> expression) where T : class, new();
    void Delete<T>(Expression<Func<T, bool>> expression) where T : class, new();
    void Add<T>(T item) where T : class, new();
    void Add<T>(IEnumerable<T> items) where T : class, new();
}
 
public class MongoRepository : IRepository
{
    private readonly IMongoClient _client;
    private readonly IMongoDatabase _database;
  
    public MongoRepository(string mongoConnection, string mongoDatabaseName)
    {
        _client = new MongoClient(mongoConnection);
        _database = _client.GetDatabase(mongoDatabaseName);
    }
  
    public IQueryable<T> All<T>() where T : class, new()
        => _database.GetCollection<T>(typeof(T).Name).AsQueryable();
  
    public IQueryable<T> Where<T>(System.Linq.Expressions.Expression<Func<T, bool>> expression) where T : class, new()
        => All<T>().Where(expression);
  
    public void Delete<T>(System.Linq.Expressions.Expression<Func<T, bool> predicate) where T : class, new()
        => _database.GetCollection<T>(typeof(T).Name).DeleteMany(predicate);
  
    public T Single<T>(System.Linq.Expressions.Expression<Func<T, bool>> expression) where T : class, new()
        => All<T>().Where(expression).SingleOrDefault();
  
    public void Add<T>(T item) where T : class, new()
        => _database.GetCollection<T>(typeof(T).Name).InsertOne(item);
  
    public void Add<T>(IEnumerable<T> items) where T : class, new()
        => _database.GetCollection<T>(typeof(T).Name).InsertMany(items);
}

Nothing complicated, it's a simple use of the official C# driver for MongoDB to implement the CRUD operations of the repository. In order to use the repository with IdentityServer, we must implement the IResourceStore, IPersistedGrantStore and IClientStore interfaces with which IdentityServer retrieves the Resources, Grants and Clients to be used in the authentication and token generation process. I take as example the IClientStore implementation, the remaining classes can be found on my GitHub repository:

public class RepositoryClientStore : IClientStore
{
    protected IRepository _repository;
  
    public RepositoryClientStore(IRepository repository)
        => _repository = repository;
  
    public Task<Client> FindClientByIdAsync(string clientId)
        => Task.FromResult(_repository.Single<Client>(c => c.ClientId == clientId));
}

Let's create a class of helpers to register all these interfaces in order to make the configuration of the services more fluent:

public static class RepositoryExtensions
{
    public static IIdentityServerBuilder AddMongoRepository(
        this IIdentityServerBuilder builder,
        string mongoConnection,
        string mongoDatabaseName)
    {
        builder.Services.AddTransient<IRepository, MongoRepository>(
            s => new MongoRepository(mongoConnection, mongoDatabaseName));
        return builder;
    }
  
    public static IIdentityServerBuilder AddClients(this IIdentityServerBuilder builder)
    {
        builder.Services.AddTransient<IClientStore, RepositoryClientStore>();
        builder.Services.AddTransient<ICorsPolicyService, InMemoryCorsPolicyService>();
        return builder;
    }
  
    public static IIdentityServerBuilder AddIdentityApiResources(this IIdentityServerBuilder builder)
    {
        builder.Services.AddTransient<IResourceStore, RepositoryResourceStore>();
        return builder;
    }
  
    public static IIdentityServerBuilder AddPersistedGrants(this IIdentityServerBuilder builder)
    {
        builder.Services.AddSingleton<IPersistedGrantStore, RepositoryPersistedGrantStore>();
        return builder;
    }       
}

At this point the configuration becomes trivial:

public void ConfigureServices(IServiceCollection services)
{
    ...
    var builder = services.AddIdentityServer(options =>
    {
        options.Events.RaiseErrorEvents = true;
        options.Events.RaiseInformationEvents = true;
        options.Events.RaiseFailureEvents = true;
        options.Events.RaiseSuccessEvents = true;
    })
    .AddMongoRepository(
        Configuration.GetValue<string>("MONGO_CONNECTION"),
        Configuration.GetValue<string>("MONGO_DATABASE_NAME"))
    .AddClients()
    .AddIdentityApiResources()
    .AddPersistedGrants();
  
    seedDatabase(services);
    ...
}

I also added a private seedDatabase() method that inserts into the database the data we used in memory if it detects that the respective collections are empty:

private void seedDatabase(IServiceCollection services)
{
    configureMongoDriverIgnoreExtraElements();
    var sp = services.BuildServiceProvider();
    var repository = sp.GetService<IRepository>();
  
    if (repository.All<Client>().Count() == 0)
    {
        foreach (var client in Config.Clients)
        {
            repository.Add<Client>(client);
        }
    }
    ...
}

Since we are going to save on Mongo some objects that are not ours, but of IdentityServer, we must instruct MongoDB that it will not find in these classes some extra elements that are added by the database engine during the creation of the documents, such as the _id property. To do this, I created a private method that I call before any data entry, configureMongoDriverIgnoreExtraElements():

1

2

3

4

5

6

private void configureMongoDriverIgnoreExtraElements()
{
    var pack = new ConventionPack();
    pack.Add(new IgnoreExtraElementsConvention(true));
    ConventionRegistry.Register("IdentityServer Mongo Conventions", pack, t => true);
}

In theory, we are done, let's go now to test our configuration.

Docker and Docker Compose

Now we need to run the MongoDB daemon, so we have the opportunity to start configuring our docker-compose, where at the moment we put only MongoDB. If you do not know Docker you can read a previous article of mine or my free book that talks also about Kubernetes (https://www.syncfusion.com/ebooks/using-netcore-docker-and-kubernetes-succinctly).

You can manually create the docker-compose.yml file in the project root, or use the Docker Tools for Visual Studio Code:

We modify the script to run the container with MongoDB and expose the port on localhost. In general, it is not a good thing to let Mongo save the data inside the container, but for our examples and only for the development phase, we can avoid creating a volume for them:

1

2

3

4

5

6

7

version: '3.4'
services:
 mongodb:
   image: mongo:latest
   hostname: mongodb
   ports:
     - "27017:27017"

At this point we can run the script (docker-compose up) and once up, we can launch the IdentityServer project with the classic dotnet run. If it works correctly, you will see the Mongo database collections:

Now let's create a Dockerfile to be able to containerize our IdentityServer. In the folder of the identityserver project, we create a Dockerfile file which will contain the following instructions:

FROM mcr.microsoft.com/dotnet/core/sdk:3.1 AS build
WORKDIR /app
COPY . .
RUN dotnet restore
RUN dotnet publish -c Release -o out
  
FROM mcr.microsoft.com/dotnet/core/aspnet:3.1 AS runtime
WORKDIR /app
COPY --from=build /app/out ./
ENTRYPOINT ["dotnet", "identityserver.dll"]

We also add a .Dockerignore to exclude the bin and obj folders from the copy of the files in the container:

1 2	`/bin` `/obj`

Let's add the build of the Dockerfile to our docker-compose.yml script in order to have everything ready for the client and the microservices:

version: '3.4'
services:
 identityserver:
   build: ./identityserver
   environment:
     - MONGO_CONNECTION=mongodb://mongodb
     - MONGO_DATABASE_NAME=identityserber
   ports:
     - "5000:80"
   depends_on:
     - mongodb
  
 mongodb:
   image: mongo:latest
   hostname: mongodb
   ports:
     - "27017:27017"

We do not need to expose the MongoDB port on localhost, because the identityserver container will connect to Mongo using the default network created by Docker and resolve the database address through the service name (mongodb). Despite that, I left the port forwarding for convenience, so that I can always connect with a client like Robo 3T, to inspect the collections. Launching the docker-compose up command, the image for IdentityServer is created (only the first time) and then both containers are launched, then by opening the browser at http://localhost:5000/account/login we will see IdentityServer answer correctly:

At this point we can launch the Angular client and see if we can authenticate. Let's go to the client-angular folder and run the usual ng serve command, open the browser at http:// localhost:4200 and verify that we are redirected to the authentication page:

Unfortunately although the authentication page is reachable, we get a CORS error, which would seem due to the difference between client (4200) and server (5000) port. But why in the previous article did the same example work without problems? Given that often IdentityServer responds with a CORS error also for different errors, such as a data access error, in this case the reported problem is the right one.

In the previous article, the origins of the registered Clients were automatically added to the authorized sources, therefore the localhost:4200 should be among them. However, if we consult the official documentation, in the CORS section we discover a side effect of not having used Entity Framework:

“This default CORS implementation will be in use if you are using either the “in-memory” or EF-based client configuration that we provide. If you define your own IClientStore, then you will need to implement your own custom CORS policy service (see below)”.

So let's not get discouraged and implement our CORS policy service:

public class RepositoryCorsPolicyService : ICorsPolicyService
{
    private readonly string[] _allowedOrigins;
  
    public RepositoryCorsPolicyService(IRepository repository)
    {
        _allowedOrigins = repository.All<Client>().SelectMany(x => x.AllowedCorsOrigins).ToArray();
    }
  
    public Task<bool> IsOriginAllowedAsync(string origin)
        => Task.FromResult(_allowedOrigins.Contains(origin));
}

We just have to register our implementation:

public void ConfigureServices(IServiceCollection services)
{
    ...
    var builder = services.AddIdentityServer(options =>
    {
        ...
    })
  
    ...
    services.AddSingleton<ICorsPolicyService, RepositoryCorsPolicyService>();
}

Remember to add your registration AFTER AddIdentityServer(), otherwise it will be overwritten by the standard one that will not interrogate your Clients. We force the regeneration of the Docker image using the docker-compose build --no-cache command and try again with docker-compose up:

This time everything will work correctly, giving us access to the application immediately after authentication. If you try to launch microservices too, you will see that they will respond without problems. So we finished? To complete the tour we just have to dockerize the two microservices, making the communication parameters with IdentityServer configurable, since we are no longer in localhost, but in the default Docker network:

public void ConfigureServices(IServiceCollection services)
{
    services.AddAuthentication(options =>
    {
        ...
    }).AddJwtBearer(o =>
    {
        o.Authority = Configuration.GetValue<string>("IDENTITY_AUTHORITY");
        o.Audience = Configuration.GetValue<string>("IDENTITY_AUDIENCE");
        o.RequireHttpsMetadata = Configuration.GetValue<bool>("IDENTITY_REQUIREHTTPSMETADATA");
    });
    ...
}

At this point we can use the same Dockerfile used for IdentityServer (they are both ASP.NET Core applications) in which we only change the name of the assembly to be launched on ENTRYPOINT:

1

2

3

4

5

...
ENTRYPOINT ["dotnet", "microservice1.dll"]
  
...
ENTRYPOINT ["dotnet", "microservice2.dll"]

We modify the docker-compose.yml file by adding the two services:

version: '3.4'
services:
 microservice1:
   build: ./microservice1
   environment:
     - IDENTITY_AUTHORITY=http://identityserver
   ports:
     - "5002:80"
   depends_on:
     - identityserver
  
 microservice2:
   build: ./microservice2
   environment:
     - IDENTITY_AUTHORITY=http://identityserver
   ports:
     - "5003:80"
   depends_on:
     - identityserver
  
 identityserver:
   build: ./identityserver
   ...
  
 mongodb:
   image: mongo:latest
   ...

Note that the authority address is no longer http://localhost:5000 but we set it to http://identityserver, using the name of the service to resolve the IP of the identityserver container in the default network created by Docker for us. We launch the docker-compose up command, which will take a little longer to create the images of the microservices, and test the invocations:

As you can see, we get a 401 error because our token is invalid due to the issuer value. The official documentation of the JWT standard tells us:

“The JWT MUST contain an "iss" (issuer) claim that contains a unique identifier for the entity that issued the JWT. In the absence of an application profile specifying otherwise,compliant applications MUST compare issuer values using the Simple String Comparison method defined in Section 6.2.1 of RFC39862”

Everything worked fine in localhost because IdentityServer generates this value for us, as it is clearly explicit by the official documentation:

“IssuerUri: Set the issuer name that will appear in the discovery document and the issued JWT tokens. It is recommended to not set this property, which infers the issuer name from the host name that is used by the clients.”

Therefore, when generating the token requested by our client, it used http://localhost:5000 as issuer, while our microservice contacts IdentityServer to validate the token using the http://identityserver authority that we have configured in the docker-compose script. In production we would have no problem since these would coincide, but in this hybrid case instead we have to force the hand of IdentityServer, using a fixed IssuerUri:

var builder = services.AddIdentityServer(options =>
{
options.IssuerUri = Configuration.GetValue<string>("ISSUER_URI");
...
})
.AddMongoRepository(
Configuration.GetValue<string>("MONGO_CONNECTION"),
       Configuration.GetValue<string>("MONGO_DATABASE_NAME"))
.AddClients()
.AddIdentityApiResources()
.AddPersistedGrants();

Conclusions

With this change everything works properly again! You can check it by yourself downloading and running the sources you can find here: https://github.com/apomic80/angular-microservices-identityserver.

If you want to learn more about Docker and Docker Compose, you can watch the recordings of our Antonio's live broadcasts, which every Friday deals with issues related to DevOps live on its Twitch channel: https://www.twitch.tv/turibbio.

Data and Devops practices and tools

Sunday, August 9, 2020

Angular, Microservices and Authentication with IdentityServer, MongoDB and Docker

IdentityServer and MongoDB

Docker and Docker Compose

Conclusions

No comments:

Post a Comment

Free hosting web sites and features -2024