Durable Function Patterns
When you look at Durable Functions, you learn you can program workflow and instantiate tasks in order (sequential) or simultaneously (parallel) – or you can build a monitor, or support a human interaction flow (approval workflow). The patterns you can think of are chaining functions to control your flow, fan-in and fan-out scenario’s, correlating events, flexible automation, and long-running processes, and the earlier mentioned human interaction — patterns that are hard to implement with only functions or with Logic Apps.
Chaining Functions
The most natural and straightforward pattern to implement with Durable Functions is chaining functions together. You have one orchestrator function, that calls multiple functions in the order you desire. With functions alone and using Service Bus Queues you can achieve pattern too, however, you will face some challenges:
- No visualization to show the relationship between functions and queues.
- Middle queues are an implementation detail – conceptual overhead.
- Error handling adds a lot more complexity.
Using Durable Function, you will not run into those challenges. With an orchestrator you can have a central place to set the order of calling the functions (relations), require no management of queues (under hood the Durable Functions use Storage Queues, and manage them), and central error handling (when an error occurs in one of the activity functions then the error propagates back to the orchestrator).
//calls functions in sequence
public static async Task<object> Run (DurableOrchestrationContext ctx)
{
try
{
var x = await ctx.CallFunctionAsync ("F1");
var y = await ctx.callFunctionAsync ("F2", x);
var z = await ctx.callFunctionAsync ("F3", y);
return = await ctx.CallFunctionAsync ("F4", z);
}
catch (Exception )
{
//global error handling /compensation goes here
}
}
Fan-out/Fan-in
An alternative to the previous patterns is the fan-out/fan-in pattern – one you can use when you need to execute one or more functions in parallel and based on the results you run some other tasks. With just functions, you cannot implement such a pattern. Moreover, you will also face the challenges mentioned in the previous pattern. However, with Durable Functions, you can achieve the fan-out/fan-in pattern.
public static async Task Run (Durableorchestrationcontext ctx)
{
var parallelTasks = new List<Task<int>>();
//get a list of N work items to process in parallel
object []workBatch = await ctx.CallFunctionAsync<object[]> ("F1");
for (int i = 0; i < workBatch.Length; i++)
{
Task<int> task = ctx.CallFunctionAsync <int> ("F2", workBatch [i]);
parallelTasks.Add (task);
}
await Task.WhenAll(parallelTasks);
//aggregate all N outputs and send result to F3
int sum = parallelTasks.Sum(t=> t.Result);
await ctx.CallFunctionAsync ("F3", sum);
}
See also the Call to Action section in the end with links to samples.
You can download this blog as a PDF document for offline read. Free Download Now
HTTP Async Response
With Functions, it is possible you call another API and do not know the amount of time it would take before a response is returned. Latency, volume, and other factors can lead to not understanding the time it would make the API to process the request and return a response. A function can time-out (Consumption Plan), or state needs to be maintained (undesirable for functions, as they need to be stateless). Fortunately, Durable Functions provides built-in APIs that simplify the code you write for interacting with long-running function executions. Furthermore, the state is managed by the Durable Functions run-time.
//HTTP-triggered function to start a new orchestrator function instance.
public static async Task<HttpResponseMessage> Run (
HttpReq uestMessage req, DurableOrchestrationClient starter,
string functionName,
Ilogger log)
{
//Function name comes from the request URL.
//Function input comes from the request content .
dynamic eventData await req.Content .ReadAsAsync<object>();
string instanceid = await starter.StartNewAsync (functionName , eventData);
log .Loginformation ($"Started orchestration with ID = '{instanceid} '.");
return starter.CreateCheckStatusResponse (req, instanceid);
}
See also the Call to Action section in the end with links to samples.
Actors
Another pattern is the monitor pattern – a recurring process in a workflow such as a cleanup process. You can implement this with a function. However, you will have some challenges:
- Functions are stateless and short-lived.
- Read/write access to external state needs to be carefully synchronized.
With Durable Functions, you can have flexible recurrence intervals, task lifetime management, and the ability to create multiple monitor processes from a single orchestration.
public static async Task Run(DurableOrchestrationContext ctx)
{
int counterState = ctx.Getinput<int>();
string operation = await ctx.WaitForExternalEvent<string>("operation");
if (operation == "incr")
{
counterState++;
}
else if (operation == "decr")
{
counterstate--;
}
ctx.ContinueAsNew(counterState);
}
See also the Call to Action section in the end with links to samples.
Human Interaction
Within organizations, you will face processes that require some human interaction such as approvals. Interactions like approvals require the availability of the approver, and thus the process needs to be active some for time and need a reliable mechanism when process times out. For instance, when an approval doesn’t occur within 72 hours, an escalation process must start. With Durable Functions, you can support such a scenario.
public static async Task Run(DurableOrchestrationContext ctx)
{
await ctx.CallFunctionAsync<object []>("RequestApproval");
using (var timeoutCts = new CancellationTokenSource())
{
DateTime dueTime = ctx.CurrentUtcDateTime.AddHours(72);
Task durableTimeout = ctx.CreateTimer(dueTime, 0, cts.Token);
Task<bool > approvalEvent = ctx.WaitForExternalEvent< bool>("ApprovalEvent");
if (approvalEvent == await Task .WhenAny(approvalEvent, durableTimeout ))
{
timeoutCts.Cancel();
await ctx .CallFunctionAsync("HandleApproval", approvalEvent.Result);
}
else
{
await ctx.CallFunctionAsy nc("Escalate" );
}
}
}
See also the Call to Action section in the end with links to samples.
Sample Implementation: Chaining using Azure Durable Functions
Assume you have a message resembling an order, which needs to trigger several tasks such as persisting the order in a database, store it in blob storage (archiving), and sending a notification (via a queue and Logic App). With Durable Functions, you can coordinate and control the flow of tasks (see diagram below).
The Orchestrator client is a function that can be triggered when a message is sent to the Orchestrator client. This client (function) will call the orchestrator and pass the order message.
public static async Task<HttpResponseMessage> Run (
HttpReq uestMessage req, DurableOrchestrationClient starter, string functionName,
Ilogger log)
{
//Function name comes from the request URL.
//Function input comes from the request content .
dynamic eventData await req.Content .ReadAsAsync<object>();
string instanceid = await starter.StartNewAsync ( functionName , eventData);
log .Loginformation ($"Started orchestration with ID = '{instanceid} '.");
return starter.CreateCheckStatusResponse (req, instanceid);
}
The orchestrator will on its turn receive the order and call the activity functions.
public static async Task Run(DurableOrchestrationContext context, object order, ILogger log)
{
log.LogInformation($"Data = '{order}'.");
var orderDetail = (OrderDetail) order;
try
{
bool x = await context.CallActivityAsync<bool>("WriteToDatabase", orderDetail);
log.LogInformation($"Data storage = '{x}'.");
if (x == true)
{
await context.CallActivityAsync<OrderDetail>("WriteToArchive", orderDetail);
await context.CallActivityAsync<OrderDetail>("SendNotification", orderDetail);
}
}
catch (Exception)
{
//ErrorHandling
}
}
Each of the activity functions will perform a task – in this case, store the order in a document collection in a CosmosDB instance, store (archive) the message is stored, and send a message to the queue to send out a notification via a Logic App.
Best Practices
With Durable Functions there are a few best practices you need to consider:
- Use Azure Application Insights to monitor running instances and health (this also account for using Azure Functions).
- Function App also exposes HTTP API for management (status, update, terminate), also discussed with the HTTP Async Response. With the API methods, you can influence the course of action of your Durable Functions.
- Version your durable function consciously.
- Side-by-side deployment (update the name of your task hub on deployment), (see Durable Functions Blue Green Deployment Strategies).
Call to action
Here is a list of resources you can leverage to get some hands-on experiences with Durable Functions or see some of the patterns in action:
- Durable Functions Overview
- Create your first durable function in C# (Chaining Functions)
- Azure Durable Functions Pattern: Approval Workflow with SendGrid
- Perform long-running Logic Apps tasks with Durable Functions
- Async Http APIs with Azure Durable Functions (and Polling Client)
- Durable Function Recipe — Webhooks Made Easy
- Pluralsight: Azure Durable Functions Fundamentals
Wrap up
In this blog post, we hope you have a better understanding of what implementing some of Durable Functions patterns can bring and what value it offers. Durable Functions, in the end, give you the ultimate control over a workflow not achievable with alternative technologies such as Logic Apps or Functions alone. Together with some of the best practices, we consolidated you should now be able to build sustainable solutions with Durable Functions.
No comments:
Post a Comment