Storage Configuration

This page explains how to configure storage options for AsyncEndpoints, including in-memory storage for development and Redis storage for production environments.

Overview

AsyncEndpoints supports multiple storage backends through the IJobStore interface. The storage layer is responsible for managing job persistence, state transitions, and job queue operations.

In-Memory Storage (Development)

Basic Setup

The in-memory store is perfect for development and single-instance deployments:

var builder = WebApplication.CreateBuilder(args);

builder.Services
    .AddAsyncEndpoints()
    .AddAsyncEndpointsInMemoryStore() // Development storage
    .AddAsyncEndpointsWorker();

var app = builder.Build();

Characteristics

Data Persistence: Volatile - data is lost when the application restarts
Concurrency: Thread-safe within a single application instance
Performance: Fast in-memory operations
Suitability: Development, testing, single-instance deployments
Limitations: No persistence across application restarts, not suitable for multi-instance deployments

In-Memory Store Implementation

The in-memory store uses concurrent collections for thread safety:

public class InMemoryJobStore : IJobStore
{
    private readonly ConcurrentDictionary<Guid, Job> _jobs = new();
    private readonly ConcurrentQueue<Guid> _queue = new();
    private readonly SemaphoreSlim _semaphore = new(1, 1);
    
    // Implementation details for job operations
}

Redis Storage (Production)

Basic Setup with Connection String

For production environments with persistence and multi-instance support:

var builder = WebApplication.CreateBuilder(args);

builder.Services
    .AddAsyncEndpoints()
    .AddAsyncEndpointsRedisStore("localhost:6379") // Redis connection string
    .AddAsyncEndpointsWorker();

var app = builder.Build();

Setup with Connection Multiplexer

For more advanced scenarios with custom connection configuration:

var connection = ConnectionMultiplexer.Connect(new ConfigurationOptions
{
    EndPoints = { "localhost:6379" },
    ConnectRetry = 3,
    ConnectTimeout = 5000,
    AbortOnConnectFail = false
});

builder.Services
    .AddAsyncEndpoints()
    .AddAsyncEndpointsRedisStore(connection)
    .AddAsyncEndpointsWorker();

Setup with Configuration Object

For maximum flexibility in configuration:

builder.Services
    .AddAsyncEndpoints()
    .AddAsyncEndpointsRedisStore(config =>
    {
        config.ConnectionString = "localhost:6379";
        // Additional Redis configuration can be set directly on the ConnectionMultiplexer if needed
    })
    .AddAsyncEndpointsWorker();

Redis Store Characteristics

Data Persistence: Persistent data storage with Redis durability options
Concurrency: Atomic operations for safe multi-instance usage
Performance: Optimized Redis operations with Lua scripts
Scalability: Supports multiple application instances
Recovery: Supports distributed job recovery (when enabled)
Monitoring: Can leverage Redis monitoring and metrics

Redis Connection Management

The Redis store includes robust connection management:

private IDatabase InitializeDatabase(string connectionString)
{
    var redis = ConnectionMultiplexer.Connect(connectionString);

    // Register for connection events to handle reconnection
    redis.ConnectionFailed += (sender, e) =>
        logger.LogError(e.Exception, "Redis connection failed: {ErrorMessage}", e.Exception?.Message);
    redis.ConnectionRestored += (sender, e) =>
        logger.LogInformation("Redis connection restored");

    return redis.GetDatabase();
}