.jpg)
As many systems are distributed and parallel, managing concurrency continues to be extremely important. At Flow, concurrent and distributed applications are built using Play Framework and Akka for the actor model. This post demonstrates how we use Akka Streams to handle asynchronous operations.
A common use case of Akka Streams is to consume a sequence of input, make asynchronous calls and then emit one-to-one mapped results. As stated in the Akka document, `Source` is a set of stream processing steps that has one open output. In the example below, we stream a sequence of parameters that are needed for each API call. Using `mapAsync` with a parallelism of 1, we can consume the parameters in order and one at a time. To process the parameters concurrently, we can increase the parallelism to number n, which allows us to have up to n inputs being processed in parallel. When n > 1, the event processing order will no longer be strict, even though the messages emitted downstream will follow the same order as the input.
One thing to note is that Akka Streams comes with different supervision strategies that handle exceptions, with the default strategy of stopping the actor when an exception occurs. This behavior can be customized to better cater to your needs. For example, we can configure a stream to use the `Resume` strategy, which will drop the element when the exception occurs and continue the stream processing.
Another use case of Akka Streams is to handle paginations. Have you tried to fetch all rows from a REST API that requires pagination? There are often cases where we are unable to fetch all rows in one call due to scale, but can make a sequence of calls, each specifying a limit - the maximum number of rows to fetch. A more common approach is to write a recursive function that repeatedly appends the fetched results. For example, if we want to fetch all memberships from an organization, we can start with limit = 1000 and offset = 0, writing a recursive method like:
Alternatively, we can achieve the goal by using Akka stream’s `unfoldAsync` method. This method will stream the result of a function as long as it returns a “Some”.
Similar to Scala’s `foldLeft` method, we start with the state (offset) of 0, and increment it by the number of records being fetched. The result is an Optional tuple with the first element being the new state (offset) going into the next call. After we fetch all the results, simply return `None` to complete.
Akka Streams is a powerful tool to handle both simple and complex concurrency tasks. We are continuously exploring ways we can leverage Akka Streams to improve the service performance. Hopefully this post helps you learn a bit more about how we use Akka Streams asynchronously at Flow. Want to learn more about Akka Streams? Check out the Akka documentation here.
