7. Evolving Event-Sourced Systems

The second that you deploy your code it is Legacy software, it is the past there will be new code and iterations. You will need to work within the constraints of immutable events and build from there.

---

Evolving Event Schemas

Keep this in mind while reading the chapter, _”Events Schemas Are Immutable.” This is the law.

Let’s think of an AccountCreated event stream. You start off with: UserID, Name, Address, Email, then add in Subscription Plan, then add Service Region. These all feel backwards compatible.

There are some subtle differences that we should go over here. Backwards compatible means that we can then take any AccountCreated1 and use the same schema from AccountCreated2, this means that we need to assume a lot of information about a User as we process the initial event. Do we default to a basic account if they have not upgraded to the premium account or declared at some point in the future since they have created the account with AccountCreated1? You see that we are relying on outside data to fill in the gaps. This is the anti-pattern that we should avoid.

Now let’s talk about the other way of looking at it. You will create a new schema for the AccountCreated2 and only use the schema that is associated with that version of AccountCreated*. This means that we will not always have the full information but we can rely on the information that we are seeing for the user. Remember we want to build a reusable version of the code that will always be true.

Here is some rules to follow:

The advice from this section about evolving events can be distilled into the
following items:
• A change to an event schema produces a new event schema.
• Events that are no longer used aren’t deleted; they just stop occurring.
• Avoid the backward compatibility trap by not making assumptions about
optional fields.
• We cannot change the past to accommodate new schemas or events.

---

Evolving Aggregates

So as we move forward with our code we will need to update an aggregate. They might come in the following forms:

• Change how it computes internal state
• Change the events emitted in response to commands

We must also keep in mind that the time line must be preserved and that any event of the past must remain the same, but the internal state can change based on the new aggregate.

---

Evolving Projections and Projectors

This is a very disposable part of the Event Sourcing. This will not be relied upon for any business logic, just a consumer facing projection. Thus they are to be used and then gotten rid of once the need for them has passed. There will be times where you will need to keep an older projection until you can work around the older data field.

A couple of common patterns are used when evolving projections:

• The new projector writes to a different projection, and old and new never
overlap.
• The old projection is deleted, and the new projector generates the new
projection in its place, breaking compatibility and forcing old code to
update.
• The new projector adds the new field to the old projection alongside the
old field, allowing both old and new consumers to co-exist.

You cannot use a Projectors data with other projectors.

---

Evolving Process Managers

Process Managers take events and emit commands that move a process forward. Just like everything else we need to understand that changes can’t go into the past. We also need to understand with PMs that anything you change about a PM will need to probably have a command associated with it. Thus your aggregate also needs to be updated and then even the projector might need to be updated. PMs will change but they are the most in-depth in regards to down-stream changes.

---

Evolving Lunar Frontiers

For this we will take the event cycle for the building aggregate from

SiteSpawned -> ConstructionAdvanced -> ConstructionCompleted -> (command) -> BuildingSpawned

To

BuildingSpawned -> ConstructionProgressed -> ConstructionCompleted

We do this so we can see the evolution of just one update and not try and update everything at the same time.

Looking again we had in the older version a PM that would be responsible for passing AdvanceConstruction commands and being active as long as the Construction process was active. We can now just have the aggregate take care of that now. Let’s first look at the new game_loop_manager.ex

defmodule LunarFrontiers.App.ProcessManagers.GameLoopManager do
  require Logger
  alias LunarFrontiers.App.Commands.AdvanceConstruction

  alias LunarFrontiers.App.Events.{
    GameloopAdvanced,
    GameStarted,
    BuildingSpawned.V2,
    ConstructionCompleted,
    SiteSpawned,
    GameStopped
  }

  alias __MODULE__

  use Commanded.ProcessManagers.ProcessManager,
    application: LunarFrontiers.App.Application,
    name: __MODULE__

  @derive Jason.Encoder
  defstruct [
    :current_tick,
    :buildings_under_construction,
    :game_id
  ]

  def interested?(%GameStarted{game_id: gid}),
    do: {:start, gid}

  def interested?(%V2{game_id: gid}),
    do: {:continue, gid}

  def interested?(%ConstructionCompleted{game_id: gid}),
    do: {:continue, gid}

  def interested?(%GameloopAdvanced{game_id: gid}), do: {:continue, gid}
  def interested?(%GameStopped{game_id: gid}), do: {:stop, gid}
  def interested?(_event), do: false

  def handle(%__MODULE__{} = state, %GameloopAdvanced{tick: tick}) do
    buildings = state.buildings_under_construction || []

    construction_cmds =
      buildings
      |> Enum.map(fn site_id ->
        %AdvanceConstruction{
          site_id: site_id,
          tick: tick,
          game_id: state.game_id,
          advance_ticks: 1
        }
      end)

    construction_cmds
  end

  def apply(%GameLoopManager{} = state, %GameloopAdvanced{tick: tick}) do
    %GameLoopManager{
      state
      | current_tick: tick
    }
  end

  def apply(%GameLoopManager{} = _state, %GameStarted{game_id: gid} = _evt) do
    %GameLoopManager{
      current_tick: 0,
      game_id: gid,
      buildings_under_construction: []
    }
  end

  def apply(
        %GameLoopManager{} = state,
        %V2{site_id: sid, tick: t} = _evt
      ) do
    %GameLoopManager{
      current_tick: t,
      game_id: state.game_id,
      buildings_under_construction: state.buildings_under_construction ++ [sid]
    }
  end

  def apply(
        %GameLoopManager{} = state,
        %ConstructionCompleted{site_id: sid, tick: t} = _evt
      ) do
    %GameLoopManager{
      current_tick: t,
      game_id: state.game_id,
      buildings_under_construction: state.buildings_under_construction -- [sid]
    }
  end

  # By default skip any problematic events
  def error(error, _command_or_event, _failure_context) do
    Logger.error(fn ->
      "#{__MODULE__} encountered an error: #{inspect(error)}"
    end)

    :skip
  end
end

We now are keeping track of the buildings that are under construction, and when we advance the game tick we are checking all the buildings and setting a new command to be run. Once the building is complete we remove the building from the buildings under construction.

Now let’s look at the router.ex

defmodule LunarFrontiers.App.Router do
  alias LunarFrontiers.App.Commands.{
    AdvanceGameloop,
    StartGame,
    AdvanceConstruction,
    SpawnSite,
    SpawnBuilding.V2
  }

  alias LunarFrontiers.App.Aggregates.{
    Gameloop,
    Building
  }

  use Commanded.Commands.Router

  identify(Gameloop,
    by: :game_id,
    prefix: "game-"
  )

  identify(Building,
    by: :site_id,
    prefix: "bldg-"
  )

  dispatch([AdvanceGameloop, StartGame], to: Gameloop)
  dispatch([V2, AdvanceConstruction], to: Building)
end

Now let’s look at the V2 version of the building_spawned_v2.ex

defmodule LunarFrontiers.App.Events.BuildingSpawned.V2 do
  # In this version of the event, a building is spawned with a
  # tick count indicating how many ticks until construction is completed
  @derive Jason.Encoder
  defstruct [:site_id, :game_id, :site_type, :location, :player_id,
             :tick, :completion_ticks]
end

This will now carry the amount of ticks needed to finished the building.

Lastly we will look at the supervisor.ex

defmodule LunarFrontiers.App.Supervisor do
  use Supervisor

  alias LunarFrontiers.App

  def start_link(arg) do
    Supervisor.start_link(__MODULE__, arg, name: __MODULE__)
  end

  @impl true
  def init(_arg) do
    children = [
      # Application
      App.Application,

      # Process Managers
      App.ProcessManagers.GameLoopManager,

      # Projectors (read model)
      App.Projectors.Building
    ]

    Supervisor.init(children, strategy: :one_for_one)
  end
end

So to go over we will now carry the ticks needed to finish a building, we will no longer need to dispatch a PM for building construction, and the supervisor will no longer need to keep track of the PMs for building construction.

---

Migrating Event Streams

Something to keep in mind is that if you can (in-memory) can process a new event, you might be able to just process a whole new stream. The other side is to have the events upgraded as they are processed, but that violates the idea that events are immutable.

The idea of a new stream is that you can do “what-if” scenarios to see if you like or can use the new stream all the while not effecting production.

Thinking about the Lunar Frontiers we will see that the new process needs to have the amount of ticks needed to finish the building when spawned. If we were to want to upgrade the events as they happen we would add those in but since they are already completed we would not need that information. But if they were in the process of building built when we shut down we would need that.

---

Wrapping Up

Evolution for event sourcing is the biggest hurdle that you will face when working with Event Sourcing. You need clearly defined guards and reasons for doing something in order to best upgrade a system. Do not take that for granted and you will be fine.