Event Sourcing

Domain Driven Design, Event Sourcing and Micro-Services explained for developers

riccardone

And when I speak of the other division of the intelligible, you will understand me to speak of that other sort of knowledge which reason herself attains by the power of dialectic, using the hypotheses not as first principles, but only as hypotheses — that is to say, as steps and points of departure into a world which is above hypotheses, in order that she may soar beyond them to the first principle of the whole

Plato’s Republic

Here are my notes about Domain Driven Design, Event Sourcing and Micro-Service architecture and the relations between these concepts.

I’d like to write it down in order to offer the people that I work with a way to get on with my software approach for Distributed Systems and how to shape them out of the darkness. It’s not a technical guide “how to” build a Micro-Service but instead it’s my explanation of Domain Driven Design concepts and Event Sourcing. There are other concepts that are essentials for a Micro-Service architecture like horizontal scalability that are not discussed here.
Code example

Table of contents

What are MicroServices Bounded-Contexts Aggregates?
What are the relations between them?
What is a Process Manager and do I need it?
Micro-Services interactions with the external world
Event Sourcing in a Micro-Service architecture
Difference between internal and external events
Ubiquitous language
Behaviors vs Data structures
Keep out of the Domain the external dependencies
Stay away from Shared Libraries across components

What are MicroServices Bounded-Contexts Aggregates?

  • A Micro-Service is a process that does some work around a relevant business feature
  • A Component is a process that does some work around a relevant business feature
  • A Bounded Context is a logical view of a relevant business feature
  • An Aggregate is where the behaviours (functions) related to a relevant business feature are located.
  • A relevant Business Feature for the sake of this article is composed by one or more related tasks that execute some logic to obtain some results. From a more technical point of view, a task is a function. From a less technical point of view it could be a requirement.

What are the relations between them?

A Bounded Context is a relevant business feature. But it doesn’t represent a running process. A running process belongs to the “physical” world and can be better represented by the terms Micro-Service or Component. The Bounded Context meaning belongs to the “logical” world. These 3 terms can be confused together but they represent a different point of view of the same “relevant” business feature like “As a user I want to fill a basket with Products So I can checkout the bill and own them”
What does relevant mean then? This term is used to differentiate infrastructure services from services that the business domain experts expect and can understand. Is a proxy service that calls a third party service a relevant business feature? Probably not. It is better to define a more understandable Basket Bounded Context contained in a Micro-Service interacting with the other components.
In the terminology that I use day to day, I often mix the physical with the logical using the term “Domain Component”.

To summarize

  • I use both Micro-Service or Component terms to indicate a physical running process
  • I use Bounded Context term to indicate the boundary of a relevant business feature
  • I use the term Domain Component to indicate a relevant business feature within a running process when there is not a need to differentiate between physical and logical views

The size of a component or a Micro-Service can be narrowed down to a single task or function but there could be a wealth of behaviour if the requirement is complex. Within the same Micro-Service there is an Application Service that behave as an orchestrator deciding which tasks or functions need to be called depending on received messages.

There is a good metaphor describing this layering as an Onion Architecture.
If we want to describe on a whiteboard the Onion we can draw the external onion’s layer as an Application Service Endpoint that subscribes and listens for events and converts them to commands. The middle layer Handler handle the commands and calls the behaviours (functions) exposed by the Aggregate. The Aggregate contains the logic related to the relevant business feature.

Event Sourcing, Domain Driven Design, CQRS
Event Sourcing, Domain Driven Design, CQRS

Generally speaking it’s better to keep the size small as much as we can. When I start shaping the Domain Model (Domain Model = group of components) I probably start with grouping more than one Aggregate in the same component. After a little while from the daily discussions with other devs and business stake holder I can see if they are using the components names easily or not. This can drive me onto a more granular size and it’s easier to split a fat Bounded Context in small parts.

What is a Process Manager and do I need it?

Another important aspect of a distributed architecture is the need for some sort of ‘Process Managers’. If you are building a simple ingestion system that goes from a set of files down to the processing component and out in the report synchronizers (CQRS) then maybe you can avoid building a Process Manager.
It can be useful when there are multiple components or legacy subsystems involved and the coordination between them depends on unpredictable conditions. In that case…

  • Domain Components are responsible to process commands and raise events
  • A Process Manager listens for events and sends commands.

Consider a Process Manager as a sort of Application Service on top of the other components. It can contains one or more stateful aggregate that keep the state of a long running process depending on requirements.

Micro-Services interactions with the external world

Through messages of course. The pattern could be a simple Event Driven Pub Sub. Why not “Request Response”? We can use the Request Response pattern defining an HTTP Api but this is more suitable when we need to expose service endpoints outside our domain and not within it. Within our domain, the components can better use a Publish Subscribe mechanism where a central message broker is responsible to dispatch commands and events around the distributed system. The asynchronous nature of Pub Sub is also a good way to achieve resiliency in case our apps are not always ready to process or send messages to others.

Event Sourcing in a Micro-Service architecture

With this Event Driven Pub Sub architecture we are able to describe the interaction between components. Event-Sourcing is a pattern to describe how one component processes and stores its results.

In other words, Event-Sourcing describes something happened within a single component whether Pub Sub is a pattern used to handle interactions between components.

Moving down our focus inside a single Event Sourced bounded context we can see how it stores time’s ordered internal domain events into streams. The business results are represented by a stream of events. The stream becomes the boundary of the events correlated to a specific instance of the relevant business feature.

Event Sourcing example
Event Sourcing example

In a Distributed Architecture not all components are Event Sourced. There could be components used as proxy services to call external services. There could be components that are not required to keep track of changes across the time. In that case they can store their results updating the current state in a store like MongoDb, Neo4J or even Sql Server and publish external events to communicate with the rest of the interested components.

Difference between internal and external events

In a Distributed Architecture any component can potentially publish External Events when something has happened. Any interested component can subscribe to those events and handle them if it is required. In that scenario when I change the content of an external event I must carefully consider any breaking change that can stop other components to handle those events correctly.

In a Distributed Architecture any Event Sourced component stores its internal Domain Events in its streams. Considering one Event Sourced component, if I change the scheme of its internal events I can only break itself. In that scenario I’m more free to change the Internal Domain Events.

It can be beneficial having in place a versioning strategy for changing the External Events in order to reduce the friction with other components.

But not all changes are breaking changes. In case of breaking changes, like if you remove or change an existing field it’s easier create a new Event.

Ubiquitous language

There is the need to define an ubiquitous language when you start shaping your domain components. In my experience as a developer you can get the right terminology discussing day after day with business stakeholders and other expert devs about use cases. You can also introduce some more structured type of discussions using Event Storming techniques.

Slowly you can see the picture coming out and then just start building the components. If I dedicate to much time drawing components and interactions on the papers I don’t get it right so it’s better to balance theories with early proof of concepts.

Behaviours vs Data structures

Another difference doing DDD compared with an old school layered architecture is that I need to stop thinking about the data. The data is not important. What is important are the behaviours. Think about what you have to do with the data, what is the business result that they expect instead of which tables and relations I need to create to store the data. The behaviours that you identify will be exposed from one or more aggregates as functions.

Keep out of the Domain the external dependencies

My domain project contains one or more aggregates. It is a good approach to try to keep this project clean from any external dependencies. Expand the list of references. Can you see a data storage library or some proxy client for external services in there? Remove them and instead define an abstraction interface in the Domain project. Your external host process will be responsible to inject concrete implementation of these dependencies.

In other words everything points in to the Domain. The Domain doesn’t point out.

Stay away from Shared Libraries across components

The DRY coding principle is valid within one domain component but not between different components. This principle, Don’t Repeat Yourself, keep on the right track during development of related classes with a high level of cohesion. This is a good approach. In a distributed architecture made of multiple separate Domain Components it’s much better if you don’t apply the DRY principle across them in a way that all the components are tight to that shared library (language, version, features).

I personally broke this approach and I always regret having done that. As an example, I created a shareable infrastructure library containing a few classes to offer an easy and fast way to build up well integrated aggregates and message handlers . This library doesn’t contains any external dependency. It’s just a bunch of a few classes and functions.

Once it is stable I could probably accept to attach it to all the C# components and follow the same approach in Java. In F#, Scala or Clojure, I don’t really need it or I can provide a slightly different set of features.

Instead of sharing it, it’s better to include the code in each single Domain Component (copy and paste). Keep it simple and let each single Domain Component evolve independently.

Enjoy!

External References

Related code from my Github repository

Thank you

A big thank you to Eric Evans, Greg Young, Udi Dahan for all their articles and talks. Whatever is contained in this post is my filtered version of their ideas.