Home Posts Post Search Tag Search

Real World Event Sourcing - 10 - Modeling Discroverable Application Domains
Published on: 2026-07-30 Tags: elixir, schema, Domain, event-sourcing, software-architecture, Real World Event Sourcing, Avro, AsyncAPI, CloudEvents, Event Catalog, JSON Schema, Protocol Buffers, RDF

We have spent much of this book talking about high level overviews of events. This is not always how the real world works. We will need to get messy and deal with some uncertainty.


Let’s look at some real world examples and learn some techniques for dealing with growth and modeling.



Defining and Documenting Schemas

Right now all the schemas are part of the structure definitions. like the spawn building event.

defmodule LunarFrontiers.App.Events.BuildingSpawned.V2 do
  @derive Jason.Encoder
  defstruct [:site_id, :game_id, :site_type, :location, :player_id, :tick, :completion_ticks]
end

This isn’t strictly a schema so much as the fields for the struct. We can add some types to this as follows:

defmodule LunarFrontiers.App.Events.BuildingSpawned.V2 do
  @type t :: %__MODULE__{
    site_id: String.t,
    game_id: String.t,
    site_type:  :oxygen_generator |
                :water_generator  |
                :hq               |
                :power_generator  |
                :colonist_housing,
    location: Point.t,
    player_id: String.t,
    tick: integer,
    completion_ticks: integer
  }

  @enforce_keys [:site_id, :game_id, :site_type]

  @derive Jason.Encoder
  defstruct [
    :site_id,
    :game_id,
    :site_type,
    :location,
    :player_id,
    :tick,
    :completion_ticks
  ]
end

This will help a person who is head deep in the code know what the struct is for and what types can be used for each field, but we want a better way of documenting the code. That is where Domain Specific Languages (DSLs) come in. There are a lot but in this section we will go over one.


Modeling with JSON Schemas

A JSON Schema is a declarative language represented with JSON. This will be writing in JSON and able to be read by the same language. Here is one for the BuildingSpawned.

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://lunarfrontiers.com/schema/events/building_spawned.json",
  "title": "BuildingSpawned",
  "description": "An event indicating a building spawned",
  "type": "object",
  "properties": {
    "site_id": {
      "description": "A UUID for the site",
      "type": "string"
    },
    "site_type": {
      "description": "The type of this building",
      "type": "string",
      "enum": ["HQ", "OxygenGenerator", "WaterGenerator", "Housing"]
    },
. . .
  }
}

This is a good way to add documentation with the code and will be able to be picked to generate Markdown for documentation.


Modeling with Protocol Buffers

Protocol Buffers are a language and platform-neutral way of describing and serializing data. They might also be known as protobufs. They are written in the (.proto) file type and be transmitted the same way messages can be sent. Here is the same BuildingSpawned as before.

syntax = "proto3";

enum SiteType {
  HQ = 0;
  OXYGEN_GENERATOR = 1;
  WATER_GENERATOR = 2;
  POWER_GENERATOR = 3;
}

message Point {
  uint32 x = 1;
  uint32 y = 2;
}

message BuildingSpawned {
  string site_id = 1;
  string game_id = 2;
  SiteType site_type = 3;
  Point location = 4;
  string player_id = 5;
  uint64 ticks = 6;
  uint32 completion_ticks = 7;
}

Once you have this you will need to install protoc suitable for your OS and CPU. Once this is installed it can generate code in any number of languages. For C# code the command line might look like this.

protoc buildingspawned.proto --csharp_out=.

This will generate a lot of code so as to not have to put too much on the page here is a small snippet.

public enum SiteType {
  [pbr::OriginalName("HQ")] Hq = 0,
  [pbr::OriginalName("OXYGEN_GENERATOR")] OxygenGenerator = 1,
  [pbr::OriginalName("WATER_GENERATOR")] WaterGenerator = 2,
  [pbr::OriginalName("POWER_GENERATOR")] PowerGenerator = 3,
}

Modeling with Avro

Apache Avro is a data serialization system. You define how the data is serialized through Avro Schemas. Kafka has some streamlined Avro support. There are ways of using JSON to represent the Schema but there is also Avro’s Interface Definition Language (IDL). Here is a snippet for the same module.

enum SiteTypes {
  OXYGENERATOR, WATERGENERATOR, POWERGENERATOR, HQ, HOUSING
}

record BuildingSpawned {
  string site_id;
  string game_id;
  SiteTypes site_type;
  Point location;
  string player_id;
  int ticks;
  int completion_ticks;
}

The thing that makes this different is that it requires you to not generate code and it can be built from the spec sheet.


Examining Cloud Events

Remember the Cloud Events are designed to be defined in a very particular way. This is designed to make it so the payload can be read by many different Cloud storage sources. This does not mean that it will tell you how to implement it just that you will be able to send the information in a standard way.

{
  "specversion": "1.0",
  "id": "2a0562bb-6657-4918-ad21-bec63f38bc11",
  "type": "building_spawned",
  "source": "lunar_frontiers",
  "datacontenttype": "application/json",
  "time": "2023-07-30T18:07:40.210505Z",
  "data": {
    "game_id": "9c0562bb-6657-4918-ad21-bec63f38bc10",
    "site_id": "193e4856-3ffa-4509-b6b0-16f2d106d046",
    "site_type": "HQ",
    . . .
  }
}

Remember this is a JSON that represent the BuildingSpawned Event. These can also be represented in many other formats but the outer wrapper must be similar.


Deciding on an Event Schema Language

This will be up to you and your team but always try and work with the data base you have and find a way to integrate it into your database or code. Some will be more hands on in terms of what you will build from the Schema language, while others will simply represent the structure of the current Schemas.



Modeling Event Flows

One of the issues that you might find with everything so far is that you don’t have a way of visualizing the structure or even the payload of a given event. Let’s go over a few ways of doing that now.


Event Flows Are Directed Graphs

This is a way of representing the flow of a system with nodes and arrows. Each step can further be broken down but this will allow you to represent the state of the system or the work that is being done at any given step.


One of the downfalls of this is as you add more and more nodes to the graph and then the corresponding arrows you will quickly lose a lot of functionality. It might be better to try for overall structure with these types of models and then create smaller more niche ones for the steps within a certain node.


Specifying Systems with AsyncAPI

AsyncAPI refers to a specification and set of tools for defining asynchronous APIs. It will be made up of some of the following:

• Server—A server is a broker system responsible for connecting consumers
and producers.
• Producer—A producer is an application (AsyncAPI interprets the word
“application” broadly) that publishes messages.
• Consumer—A consumer is an application that listens for events.
• Channel—A channel is the means by which messages flow through servers
between producers and consumers. Channels in this specification can 
represent concepts like topics, queues, routes, paths, or subjects depending
on which protocols are used.
• Application—An application is a broad category that can be a program or
collection of programs. This can be a microservice, groups of microservices,
a monolith, an IoT sensor, and so on.
• Protocol—Protocols define how information flows through the system. Note
that you can still specify your system even if you don’t use any of the
predefined protocols.
• Message—A message is a unit of data transmitted from a producer to a
consumer through a server.

YAML tends to be the easiest way of structuring the AsyncAPI here is a file for a bank withdrawal event.

asyncapi: 2.5.0
info:
  title: Bank
  version: 0.1.0
channels:
  bank/funds-withdrawn:
    subscribe:
      message:
        description: An event indicating funds withdrawn from an account
        payload:
          type: object
          additionalProperties: false
          properties:
            accountNumber:
              type: string
            routing:
              type: string
            amount:
              type: integer
              minimum: 1

One of the things to get your head around for this is that you will be doing everything from the respective of the client. Meaning that you would think that the Bank would subscribe to the funds-withdrawn, but the client in this case will be subscribing to the event and you might give them permission to publish to the API if you want them to be able to do those actions.


Documenting Systems with Event Catalog

Doing any changes to the system will make all the hard work of all the diagrams no longer represent the current state of the system. Then if you are no longer maintaining the documentation you will find that all the Markdown is no longer a representation of the state of the system. Event Catalog might be the perfect solution to this problem. This is an open source code library that creates static websites. These are not ordinary sites they have 3-d interfaces and much more.


Specifying Systems with RDF

If you are not getting what you want from Event Catalog graph databases might be what you want. There is a popular one, Neo4j, that will include a lot of tools that you might want.


There are a lot of ways to represent graphs outside of all the above examples. Resource Description Framework lets you form triples from a subject, predicate and an object. Once you have these you can describe the relationship of the nodes. It can be very robust but it can be brittle and take a longer time to build these.


There is a more human friendly version of this and it’s called Turtle here is an example of a Turtle relationship.

<#green-goblin>
  rel:enemyOf <#spiderman> ;
  a foaf:Person ; # in the context of the Marvel universe
  foaf:name "Green Goblin" .

<#spiderman>
  rel:enemyOf <#green-goblin> ;
  a foaf:Person ;
  foaf:name "Spiderman" .


Modeling Case Study: Crafter Hustle

The more that you do something the better you get at it. So in this way you can take an existing event sourced application and extend and upgrade the model.


Collect the Jobs to Be Done

• Keep track of supplies needed to make pieces of art
• Keep track of the art they have on hand
• Get useful information about going to craft shows to sell their wares.
Specifically, was the profit worth the cost of being a vendor? They want
to use that to figure out which shows to do next year.
• Manage consignments to different stores for different periods of time
• Track sales of art via online and direct sale

Establish a Common Language

Here are the common ideas and vocabulary for the project.

Inventory
Inventory refers to “stuff” the crafter has made that’s in their possession
(it doesn’t matter where). These are finished products and not supplies
used to make things. Each piece of inventory is considered unique, even
if they’ve made a dozen of them.
Supplies
Supplies are consumed to make a product (which is in inventory). Some
crafters, especially depending on the type of things they make, don’t need
to keep a tight track of supplies. Supplies might be cheap or easily
obtained. In my SME’s case, some of the supplies can be so expensive
that they affect the bottom line and profit.
Show
A show is an event (not to be confused with an immutable piece of data
that occurred in the past). A number of costs are associated with a show,
including fees required to participate, travel, and materials that may only
ever be used for shows (for example, a tent to cover a crafting “booth”)
Consignment Merchant
A consignment merchant is a person or organization that takes artwork
on consignment. This means that the crafter delivers inventory to the
merchant and the merchant then sells the product. The merchant will
take a small portion of the sale price as a way of leasing the space used
by the crafter’s products.
Consignment Period
Consignments aren’t usually permanent. Most of the time, consignments
have a period or a run. During the consignment period, the crafter’s wares
are on display and sold. The consignment period ends on a specific date
or when all of the products have sold.
Custom Order
A custom order is when someone requests a certain type of craft. The 
customers can often specify details like their favorite colors or any number of
options. Custom orders _reserve_ supplies until the crafter begins work on
it, then the supplies are consumed to make a product.

Discover Flows from the User’s Perspective

This flow describes the sequence of “events” as seen through the eyes of the user:

1. Make products (consume supplies)
2. Deliver products to consignment merchant (consignment period begins)
3. Merchant makes multiple sales of products; crafter may not know the
details until the end of the period
4. Consignment period ends (merchant is out of inventory or time elapses)
5. Sales details are delivered to crafter
6. Any remaining product is returned to crafter

Iterate on the Model Document

Now that we have some ideas and concrete steps its best to iterate over what we have and then see if you have more questions. When does the order start? When does the merchant get a confirmation? Finalizing the requests is a big part of knowing what to do.


Build the App

This is the final form of the exercise. You should now know ways of building the Event Sourced application.



Wrapping Up

This chapter was mostly about the tooling and ways of making the Event-Sourced application model and documentation. Always try and make the tools you use able to flow from your own unique coding style. If you want to build more and describe less try and find tools that will build off your own code, maybe just by adding in a bit more comments into the code itself. If you like to design but not code as much, ASH and other frameworks where you define then it build the code for you will be better.

Back to posts