Two months ago I published on hex the first version of Galena, a topic-based producer-consumer library built on top of GenStage for Elixir. It was initially designed to create flows of producers, producer-consumers and consumers where messages have to be delivered as soon as possible. Each consumer or producer-consumer can receive messages from several producers and/or producer-consumers. In addition, consumers and producer-consumers can select the messages they want to receive, thanks to a topic-based approach.

This library was designed to be used by an application running on a single server, but why not try to distribute Galena’s producers, producer-consumers and consumers across several machines?

Elixir code runs on the Erlang Virtual Machine, or BEAM, and therefore it can take advantage of all its distribution features.

Erlang provides the global module, which helps us register process names globally. This means that we can interact with any globally registered process once the nodes are connected.

Taking these ideas into account, we can use Galena as a distributed producer-consumer system:

iex(test1@machine1)> MyProducer.start_link([], [name: {:global, :producer}]

iex(test2@machine2)> MyConsumer.start_link([producers_info: [{["topic"], {:global, :producer}}]], [name: {:global, :consumer}])

Check out the code on Github.

Enjoy your distributed messages!!!