Home Posts Post Search Tag Search

Ash Framework 08 - Testing
Published on: 2025-10-14 Tags: elixir, Blog, Testing, LiveView, Ecto, Html/CSS, Phoenix, Ash

Chapter 7: Testing Your Application (157)


Before I start I wanted to update an issue that you might have had while using the pure code that I posted, While trying to access the artists page for any artists you might have had an issue that you couldn’t see the page without an error, this was because I added in an extra param that shouldn’t have been there within the tunez/lib/tunez_web/live/artists/show_live.ex 


def render(assigns) do
    # ...
    <.button_link navigate={~p"/artists/#{@artist.id}/albums/new"} kind="primary"
        :if={Tunez.Music.can_create_album?(@current_user)}> # Remove @artist
        New Album
    </.button_link>
end

This should solve that issue.


There are 2 main reasons to test your code:


  • To confirm our current understanding of the code. <br>
  • To protect against untended code changes. <br> For this we will use ExUnit to test in Elixir <br>

What Should We Test?


The Basic First Test


We will start with this simple test.


describe "Tunez.Music.read_artists!/0-2" do
    @tag :skip
    test "when there is no data, nothing is returned" do
        assert Music.read_artists!() == []
    end
end

Setting Up Data


For these tests we will need some data to test against. So for this we will need to create some data. There are 2 ways of doing this:


  • Setting up test data using your resource actions <br>
  • Seeding data directly via the data layer, bypassing actions <br>

Using Actions to Set Up Test Data


We will use actions to start and these will be considered a series of events to create an artist etc.


describe "Tunez.Music.search_artists!/1-3" do
    test "can find artists by partial name match" do
        artist =
            Tunez.Music.create_artist!(
            %{
                name: "The Froody Dudes",
                biography: "42 musicians all playing the same instrument (a towel)"
            },
            authorize?: false
            )

    assert %{results: [match]} = Tunez.Music.search_artists!("Frood")
    assert match.id == artist.id
end

Pro: We Are Testing Real Sequences of Events


If we change anything in the way that we are doing the creating or updating how a user will create this will fail, so we can see exactly how a user might see the results.


Con: Real Application Code Has Rules and Dependencies


There might be a reason to only make a test on Tuesdays (Weird but that might really be a thing) this test will fail. There are workarounds but it might not always be perfect for this. Here is some syntax that might solve that issue.


create :create do
    argument :validate_tuesday, :boolean, default: true, public?: false
    validate IsTuesday, where: argument_equals(:validate_tuesday, true)
end

Pro: Your Application is End-to-End Testable


This can be the best so long as you are able to ensure that you have the right steps enabled to work around the quirks of your current setup.


Seeding Data


Seeds can be used to make any or a lot of user data and artists so that you can do some tests on the data created. AshPostgres can essentially perform an INSERT. There might be an error that we want to have at least 3 sentences, We can even use AshPostgres to insert things that might not hold to the current version of what we want or choice to validate. 


validations do
    validate {SentenceCount, field: :biography, min: 3} do
        where changing(:biography)
    end
end

Let’s write a test for this.


describe "Tunez.Music.update_artist!/1-3" do
    test "when an artist's name is updated, the biography length does
                            not cause a validation error" do
        artist =
            Ash.Seed.seed!(%Tunez.Music.Artist{
            name: "The Froody Dudes",
            biography: "42 musicians all playing the same instrument (a towel)."
            })


    updated_artist = Tunez.Music.update_artist!(artist, %{name: "New Name"})
    assert updated_artist.name == "New Name"
end

Pro: Your Tests Are Faster and Simpler


This can work to set everything that you have in the exact scenario that you are trying to test.


Con: Your Tests Are Not as Realistic


This might not be the exact way that a user will be able to interact with the db or the UI.


How do I choose Between Seeds and Calling Actions?


When both will do what you need, consider what you’re trying to test. Are you testing a data condition, such as the validation example, or are you testing an event, such as running a search query? If the former, then use seeds. If the latter, use your resource actions. When in doubt, use actions.


Consolidating Test Setup Logic


Ash.Generator provides tools for dynamically generating data. It is built off of StreamData. Let’s try it out with an IEX session.


iex -S mix
ex(1)> Ash.Type.generator(:integer, min: 1, max: 100)
#StreamData<66.1229758/2 in StreamData.integer/1>
iex(2)> Ash.Type.generator(:integer, min: 1, max: 100) |> Enum.take(10)
[21, 79, 33, 16, 15, 95, 53, 27, 69, 31]

It can even do more with maps see below.


iex(1)> Ash.Type.generator(:map, fields: [
        hello: [
            type: {:array, :integer},
            constraints: [min_length: 2, items: [min: -1000, max: 1000]]
        ],
        world: [type: :uuid]
    ]) |> Enum.take(1)
[%{hello: [-98, 290], world: "2368cc8d-c5b6-46d8-97ab-1fe1d9e5178c"}]

Creating Test Data Using Ash.Generator


There are already some lines set in the file tunez/test/support/generator.ex. Just uncomment those lines and remove the raise line. Once that is done we can use that generator. 


defmodule Tunez.Accounts.UserTest do
    import Tunez.Generator

test "can create user records" do
    # Generate a user with all default data
    user = generate(user())
    # Or generate more than one user, with some specific data
    two_admins = generate_many(user(role: :admin), 2)
end

This way we will create a new artist for each instance. There is a helper once/2 to ensure only one default user is created. 


def artist(opts \\ []) do
    actor = opts[:actor] || once(:default_actor, fn ->
        generate(user(role: :admin))
    end)         
    changeset_generator(
        Tunez.Music.Artist,
        :create,
        defaults: [name: sequence(:artist_name, &"Artist #{&1}")],
        actor: actor,
        overrides: opts
    )
end

There is a way to put this all together with both albums and artists. You can also seed the data using the seeded_artist generator. 


def album(opts \\ []) do
    actor =
        opts[:actor] || once(:default_actor, fn ->
            generate(user(role: opts[:actor_role] || :editor))
        end)


artist_id =
    opts[:artist_id] || once(:default_artist_id, fn ->
        generate(artist()).id
    end)

changeset_generator(
    Tunez.Music.Album,
    :create,
    defaults: [
        name: sequence(:album_name, &"Album #{&1}"),
        year_released: StreamData.integer(1951..2024),
        artist_id: artist_id,
        cover_image_url: nil
    ],
    overrides: opts,
    actor: actor
)
end

Testing Resources


Testing Actions


The tests should follow a few guidelines:


  • Prefer to use code interfaces when calling actions <br>
  • Use the raising “bang” versions of code <br>
  • Avoid pattern matching to assert success or failure <br>
  • For asserting errors, use Ash.Test.assert_has_error or assert_false <br>
  • Test policies, calculations, aggregates, and relationships separately <br> 
    describe "Tunez.Music.search_artists/1-2" do
  defp names(page), do: Enum.map(page.results, & &1.name)

  test "can filter by partial name matches" do
      ["hello", "goodbye", "what?"]
      |> Enum.each(&generate(artist(name: &1)))

      assert Enum.sort(names(Music.search_artists!("o"))) == ["goodbye", "hello"]
      assert names(Music.search_artists!("oo")) == ["goodbye"]
      assert names(Music.search_artists!("he")) == ["hello"]
  end

  test "can sort by number of album releases" do
      generate(artist(name: "two", album_count: 2))
      generate(artist(name: "none"))
      generate(artist(name: "one", album_count: 1))
      generate(artist(name: "three", album_count: 3))

      actual =
          names(Music.search_artists!("", query: [sort_input: "-album_count"]))

      assert actual == ["three", "two", "one", "none"]
  end
end

Testing Errors


describe "Tunez.Music.create_album/1-2" do
    test "year_released must be between 1950 and next year" do
        admin = generate(user(role: :admin))
        artist = generate(artist())
        assert %{artist_id: artist.id, name: "test 2024", year_released: 2024}
                |> Music.create_album!(actor: admin)

    assert_raise Ash.Error.Invalid, ~r/must be between 1950 and next year/, fn ->
        %{artist_id: artist.id, name: "test 1925", year_released: 1925}
        |> Music.create_album!(actor: admin)
    end

    %{artist_id: artist.id, name: "test 1950", year_released: 1950}
    |> Music.create_album(actor: admin)
    |> Ash.Test.assert_has_error(Ash.Error.Invalid, fn error ->
        match?(%{message: "must be between 1950 and next year"}, error)
    end)
end

Testing Policies


describe "Music.can_create_artist?/1" do
    test "only admins can create artists" do
        admin = generate(user(role: :admin))
        assert Music.can_create_artist?(admin)
        editor = generate(user(role: :editor))
        refute Music.can_create_artist?(editor)
        user = generate(user())
        refute Music.can_create_artist?(user)
        refute Music.can_create_artist?(nil)
    end
end

policy action(:get_by_email) do
    authorize_if expr(id == ^actor(:id))
    authorize_if actor_attribute_equals(:role, :admin)
end

Testing this functionality:


test "users can only read themselves" do
    [actor, other] = generate_many(user(), 2)
    assert Accounts.can_get_user_by_email?(actor, actor.email, data: actor)
    refute Accounts.can_get_user_by_email?(actor, other.email, data: other)
end

test "admins can read all users" do
[user1, user2] = generate_many(user(), 2)
admin = generate(user(role: :admin))
assert Accounts.can_get_user_by_email?(admin, user1.email, data: user1)
assert Accounts.can_get_user_by_email?(admin, user2.email, data: user2)
end

Testing Relationships and Aggregates


test "users cannot see who created an album" do
    user = generate(user())
    album = generate(album())
    assert Ash.load!(album, :created_by, authorize?: false).created_by
    assert Ash.load!(album, :created_by, actor: user).created_by
end

Testing Calculations


calculations do
    calculate :name_length, :integer, expr(string_length(name))
end

Testing in IEx:


iex(1)> artist = %Tunez.Music.Artist{name: "Amazing!"} |> Ash.load!(:name_length)
iex(2)> artist.name_length
8
iex(3)> Ash.calculate!(Tunez.Music.Artist, :name_length, refs: %{name: "Amazing!"})
8

PostgreSQL version:


calculations do
    calculate :name_length, :integer, expr(fragment("length(?)", name))
end

iex(3)> Ash.calculate!(Tunez.Music.Artist, :name_length, refs: %{name: "Amazing!"})
SELECT (length($1))::bigint FROM (VALUES(1)) AS f0 ["Amazing!"]
8

Code interface:


resource Tunez.Music.Artist do
    define_calculation :artist_name_length, calculation: :name_length,
    args: [{:ref, :name}]
end

Test:


test "name_length shows how many characters are in the name" do
    assert Tunez.Music.artist_name_length!("fred") == 4
    assert Tunez.Music.artist_name_length!("wat") == 3
end

Unit Testing Changesets, Queries, and Other Ash Modules


test "year_released must be greater than 1950" do
    Album
    |> Ash.Changeset.for_create(:create, %{year_released: 1920})
    |> assert_has_error(fn error ->
        match?(%{message: "must be between 1950 and" <> _}, error)
    end)
end

test "previous_names store the current name when changing to a new name" do
changeset =
%Artist{name: "george", previous_names: ["fred"]}
|> Ash.Changeset.new()
|> Tunez.Music.Changes.UpdatePreviousNames.change([], %{})

assert Ash.Changeset.changing_attribute?(changeset, :previous_names)
assert {:ok, ["george", "fred"]} = Ash.Changeset.fetch_change(changeset, :previous_names)
end

Should I Actually Unit Test Every Single One of These Things?


Variations need their own test. Anything that you want to be sure always works, you should test.


Testing Interfaces


Everything so far has been about resources or actions, but there will be needs to test the interface to ensure a consistent user experience.


Testing GraphQL


Since AshGraphql is built on top of the absinthe library, we can use its great utilities for testing. It offers three different approaches: resolvers, documents, or HTTP requests.


Testing AshJsonApi


Everything previously said for testing a GraphQL API applies to testing an API built with AshJsonApi. Since we generate an OpenAPI specification for your API, you can use the same strategy for guarding against breaking changes.


Testing Phoenix LiveView


Testing user interfaces is entirely different than anything else. There are whole books dedicated solely to this topic. LiveView has many testing utilities, and often testing LiveView tests more than just the core functionality.


Fixing “pg-trgm” Error


** (Postgrex.Error) ERROR 0A000 (feature_not_supported) extension “pg-trgm” is not available


hint: The extension must first be installed on the system where PostgreSQL is running.


Want to figure out why this is not working for the tunez_test db.


Back to posts