Home Posts Tags Post Search Tag Search

Post 79

Ash Framework 08 - Testing

Published on: 2025-10-14 Tags: elixir, Blog, Testing, LiveView, Ecto, Html/CSS, Phoenix, Ash
Chapter 7: Testing Your Application (157)
    Before I start I wanted to update an issue that you might have had while using the pure code that I posted, While trying to access the artists page for any artists you might have had an issue that you couldn't see the page without an error, this was because I added in an extra param that shouldn't have been there within the tunez/lib/tunez_web/live/artists/show_live.ex 
        def render(assigns) do
            # ...
            <.button_link navigate={~p"/artists/#{@artist.id}/albums/new"} kind="primary"
                :if={Tunez.Music.can_create_album?(@current_user)}> # Remove @artist
                New Album
            </.button_link>
    This should solve that issue.

    There are 2 main reasons to test your code:
        To confirm our current understanding of the code.
        To protect against untended code changes.
        
    For this we will use ExUnit to test in Elixir

    What Should We Test?
        The Basic First Test
            We will start with this simple test. 
                describe "Tunez.Music.read_artists!/0-2" do
                    @tag :skip
                    test "when there is no data, nothing is returned" do
                        assert Music.read_artists!() == []
                    end
                end

    Setting Up Data
        For these test we will need some data to test against. So for this we will need to create some data. There is 2 ways of doing this:
            Setting up test data using your resource actions
            Seeding data directly via the data layer, bypassing actions

        Using Actions to Set Up Test Data
            We will use actions to start and these will be considered a series of events to create an artist etc. tunez/test/music/artists_test.exs
                describe "Tunez.Music.search_artists!/1-3" do
                    test "can find artists by partial name match" do
                        artist =
                            Tunez.Music.create_artist!(
                            %{
                                name: "The Froody Dudes",
                                biography: "42 musicians all playing the same instrument (a towel)"
                            },
                            authorize?: false
                            )

                        assert %{results: [match]} = Tunez.Music.search_artists!("Frood")
                        assert match.id == artist.id
                    end
                end

            As we can see here we are creating an artist and then doing the search to make sure that its in the DB. Let's talk about some pros and cons for this.

            "Pro: We Are Testing Real Sequences of Events"
                If we change anything in the way that we are doing the creating or updating how a user will create this will fail, so we can see exactly how a user might see the results.

            "Con: Real Application Code Has Rule and Dependencies"
                There might be a reason to only make a test on Tuesdays (Weird but that might really be a thing) this test will fail. There are workarounds but it might not always be perfect for this. Here is some syntax that might solve that issue.
                    create :create do
                        argument :validate_tuesday, :boolean, default: true, public?: false
                        validate IsTuesday, where: argument_equals(:validate_tuesday, true)
                    end
            
            "Pro: Your Application is End-to-End Testable"  
                This can be the best so long as you are able to ensure that you have the right steps enabled to work around the quirks of your current setup

        Seeding Data
            Seeds can be used to make any or a lot of user data and artists so that you can do some tests on the data created. AshPostgres can essentially perform an INSERT. There might be a error that we want to have at least 3 sentences, We can even use AshPostgres to insert things that might not hold to the current version of what we want or choice to validate. lib/tunez/mu 
                # The validation for needing 3 sentences might look like
                validations do
                    validate {SentenceCount, field: :biography, min: 3} do
                        where changing(:biography)
                    end
                end
            
            Let's write a test for this.
                # Demonstration test only - this validation doesn't exist in Tunez!
                describe "Tunez.Music.update_artist!/1-3" do
                test "when an artist's name is updated, the biography length does
                                        not cause a validation error" do
                artist =
                    Ash.Seed.seed!(%Tunez.Music.Artist{
                    name: "The Froody Dudes",
                    biography: "42 musicians all playing the same instrument (a towel)."
                    })

                updated_artist = Tunez.Music.update_artist!(artist, %{name: "New Name"})
                assert updated_artist.name == "New Name"
                end
            end

            "Pro: Your Tests Are Faster and Simpler"
                This can work to set everything that you have in the exact scenario that you are trying to test. 

            "Con: Your Tests Are Not as Realistic"
                This might not be the exact way that a user will be able to interact with the db or the UI

        How do I choose Between Seeds and Calling Actions?
            When both will do what you need, consider what you’re trying to test. Are you testing a data condition, such as the validation example, or are you testing an event, such as running a search query? If the former, then use seeds. If the latter, use your resource actions. When in doubt, use actions.

    Consolidating Test Setup Logic
        Ash.Generator provides tools for dynamically for generating. It is built off of the StreamData. Let's try it out with an IEX session.
            iex -S mix
            ex(1)> Ash.Type.generator(:integer, min: 1, max: 100)
            #StreamData<66.1229758/2 in StreamData.integer/1>
            iex(2)> Ash.Type.generator(:integer, min: 1, max: 100) |> Enum.take(10)
            [21, 79, 33, 16, 15, 95, 53, 27, 69, 31]

        It can even do more with maps see below.
            iex(1)> Ash.Type.generator(:map, fields: [
                    hello: [
                        type: {:array, :integer},
                        constraints: [min_length: 2, items: [min: -1000, max: 1000]]
                    ],
                    world: [type: :uuid]
                ]) |> Enum.take(1)
            [%{hello: [-98, 290], world: "2368cc8d-c5b6-46d8-97ab-1fe1d9e5178c"}]

        Creating Test Data Using Ash.Generator
            There is already set some of these set in the file tunez/test/support/generator.ex, just un comment the lines that is already there. And then remove the raise line. Once that is done we can use that generator. Head back to the file tunez/test/tunez/music/artist_test.exs
                # Demonstration test - this is only to show how to call generators!
                defmodule Tunez.Accounts.UserTest do
                    import Tunez.Generator

                    test "can create user records" do
                        # Generate a user with all default data
                        user = generate(user())
                        # Or generate more than one user, with some specific data
                        two_admins = generate_many(user(role: :admin), 2)
                    end
                end

            This way we will create a new artists for each instance of the entires there is a way to make sure that we only create a new user for each one with the once/2 helper. You would use it like this.
                def artist(opts \\ []) do
                    actor = opts[:actor] || once(:default_actor, fn ->
                        generate(user(role: :admin))
                    end)         
                    changeset_generator(
                        Tunez.Music.Artist,
                        :create,
                        defaults: [name: sequence(:artist_name, &"Artist #{&1}")],
                        actor: actor,
                        overrides: opts
                    )
                end

            There is even a way to put this all together with both albums and artists, there is also a way to seed the data with seed_generator seeded_artists generator.
                def artist(opts \\ []) do
                    actor =
                    opts[:actor] ||
                        once(:default_actor, fn ->
                        generate(user(role: :admin))
                        end)

                    changeset_generator(
                    Tunez.Music.Artist,
                    :create,
                    defaults: [name: sequence(:artist_name, &"Artist #{&1}")],
                    actor: actor,
                    overrides: opts
                    )
                end

                def album(opts \\ []) do
                    actor =
                    opts[:actor] ||
                        once(:default_actor, fn ->
                        generate(user(role: opts[:actor_role] || :editor))
                        end)

                    artist_id =
                    opts[:artist_id] ||
                        once(:default_artist_id, fn ->
                        generate(artist()).id
                        end)

                    changeset_generator(
                    Tunez.Music.Album,
                    :create,
                    defaults: [
                        name: sequence(:album_name, &"Album #{&1}"),
                        year_released: StreamData.integer(1951..2024),
                        artist_id: artist_id,
                        cover_image_url: nil
                    ],
                    overrides: opts,
                    actor: actor
                    )
                end

                # This is for when you want to seed the data not just 
                def seeded_artist(opts \\ []) do
                    actor =
                    opts[:actor] ||
                        once(:default_actor, fn ->
                        generate(user(role: :admin))
                        end)

                    seed_generator(
                    %Tunez.Music.Artist{name: sequence(:artist_name, &"Artist #{&1}")},
                    actor: actor,
                    overrides: opts
                    )
                end
    Testing Resources
        Testing Actions
            The tests should follow a few guidelines:
                Prefer to use code interfaces when calling actions
                Use the raising "bang" versions of code
                Avoid Pattern matching to assert the success of failure
                For asserting errors, use Ash.Test.assert_has_error or assert_false
                Test policies, calculations, aggregates, and relationships separately

            Lets set some more test that will work with the search functions.
                describe "Tunez.Music.search_artists/1-2" do
                    defp names(page), do: Enum.map(page.results, & &1.name)

                    test "can filter by partial name matches" do
                        ["hello", "goodbye", "what?"]
                        |> Enum.each(&generate(artist(name: &1)))

                        assert Enum.sort(names(Music.search_artists!("o"))) == ["goodbye", "hello"]
                        assert names(Music.search_artists!("oo")) == ["goodbye"]
                        assert names(Music.search_artists!("he")) == ["hello"]
                    end

                    test "can sort by number of album releases" do
                        generate(artist(name: "two", album_count: 2))
                        generate(artist(name: "none"))
                        generate(artist(name: "one", album_count: 1))
                        generate(artist(name: "three", album_count: 3))

                        actual =
                            names(Music.search_artists!("", query: [sort_input: "-album_count"]))

                        assert actual == ["three", "two", "one", "none"]
                    end
                end

        Testing Errors
            tunez/test/tunez/music/album_test.exs
                describe "Tunez.Music.create_album/1-2" do
                    test "year_released must be between 1950 and next year" do
                    admin = generate(user(role: :admin))
                    artist = generate(artist())
                    # The assertion isn't really needed here, but we want to signal to
                    # our future selves that this is part of the test, not the setup.
                    assert %{artist_id: artist.id, name: "test 2024", year_released: 2024}
                            |> Music.create_album!(actor: admin)

                    # Using `assert_raise`
                    assert_raise Ash.Error.Invalid, ~r/must be between 1950 and next year/, fn ->
                        %{artist_id: artist.id, name: "test 1925", year_released: 1925}
                        |> Music.create_album!(actor: admin)
                    end

                    # Using `assert_has_error` - note the lack of bang to return the error
                    %{artist_id: artist.id, name: "test 1950", year_released: 1950}
                    |> Music.create_album(actor: admin)
                    |> Ash.Test.assert_has_error(Ash.Error.Invalid, fn error ->
                        match?(%{message: "must be between 1950 and next year"}, error)
                    end)
                end
        
        Testing Policies
            There is a way to test even policies, the Ash.can? is the way again. tunez/test/tunez/music/artist_test.exs
                describe "Music.can_create_artist?/1" do
                    test "only admins can create artists" do
                    admin = generate(user(role: :admin))
                    assert Music.can_create_artist?(admin)
                    editor = generate(user(role: :editor))
                    refute Music.can_create_artist?(editor)
                    user = generate(user())
                    refute Music.can_create_artist?(user)
                    refute Music.can_create_artist?(nil)
                end
            end

            Lets say that you want to add a new requirement that a user should be able to look up their own records and that admins should be able to look up any record. 
                policy action(:get_by_email) do
                    authorize_if expr(id == ^actor(:id))
                    authorize_if actor_attribute_equals(:role, :admin)
                end

            Now we can test this functionality.
                # Demonstration tests only - this functionality doesn't exist in Tunez!
                test "users can only read themselves" do
                    [actor, other] = generate_many(user(), 2)
                    # this assertion would fail, because the actor *can* run the action
                    # but it *wouldn't* return the other user record
                    # refute Accounts.can_get_user_by_email?(actor, other.email)
                    assert Accounts.can_get_user_by_email?(actor, actor.email, data: actor)
                    refute Accounts.can_get_user_by_email?(actor, other.email, data: other)
                end

                test "admins can read all users" do
                    [user1, user2] = generate_many(user(), 2)
                    admin = generate(user(role: :admin))
                    assert Accounts.can_get_user_by_email?(admin, user1.email, data: user1)
                    assert Accounts.can_get_user_by_email?(admin, user2.email, data: user2)
                end

            Testing Relationships and Aggregates
                There is nothing that is needed to test Relationships or aggregates as they can just be done at any time. The thing that we can do right now is show the way you can use authorize?: false to skip the authorization. That way you can test what you want after you have tested the polices, again one at a time.
                    # Demonstration test only - this functionality doesn't exist in Tunez
                    test "users cannot see who created an album" do
                        user = generate(user())
                        album = generate(album())
                        # We *can* load the user record if we skip authorization
                        assert Ash.load!(album, :created_by, authorize?: false).created_by
                        # If this assertion fails, we know that it must be due to authorization
                        assert Ash.load!(album, :created_by, actor: user).created_by
                    end
            
            Testing Calculations
                One thing to test is the aggregates that we created so for this one we will add in a temp calculation head to tunez/lib/tunez/music/artists.ex
                    calculations do
                        calculate :name_length, :integer, expr(string_length(name))
                    end

                Then we can test it in an iex session. 
                    iex(1)> artist = %Tunez.Music.Artist{name: "Amazing!"} |>
                                    Ash.load!(:name_length)
                    #Tunez.Music.Artist<...>
                    iex(2)> artist.name_length
                    8
                    iex(30)> Ash.calculate!(Tunez.Music.Artist, :name_length,
                                    refs: %{name: "Amazing!"})
                    8

                We can then even use this for calculations that require the database. We can rewrite that same function using the PostgreSQL's length function:
                    calculations do
                        calculate :name_length, :integer, expr(fragment("length(?)", name))
                    end

                Then we can call it in an iex session:
                    iex(3)> Ash.calculate!(Tunez.Music.Artist, :name_length,
                                        refs: %{name: "Amazing!"})
                    SELECT (length($1))::bigint FROM (VALUES(1)) AS f0 ["Amazing!"]
                    8

                We can also define a code interface with define_calculation, They take in a tuple and then can use the syntax to call them later. lib/tunez/music.ex
                    resource Tunez.Music.Artist do
                        ...
                        define_calculation :artist_name_length, calculation: :name_length,
                        args: [{:ref, :name}]
                    end

                Then we can test with
                    # Demonstration test only - this function doesn't exist in Tunez!
                    test "name_length shows how many characters are in the name" do
                        assert Tunez.Music.artist_name_length!("fred") == 4
                        assert Tunez.Music.artist_name_length!("wat") == 3
                    end
                
            Unit Testing Changesets, Queries, and Other Ash Modules
                # Demonstration test only - this is covered by action tests in Tunez
                test "year_released must be greater than 1950" do
                    Album
                    |> Ash.Changeset.for_create(:create, %{year_released: 1920})
                    |> assert_has_error(fn error ->
                        match?(%{message: "must be between 1950 and" <> _}, error)
                    end)
                end

                # Demonstration test only - this is covered by action tests in Tunez
                # This won't work for logic in hook functions - only code in a change body
                test "previous_names store the current name when changing to a new name" do
                    changeset =
                        %Artist{name: "george", previous_names: ["fred"]}
                        |> Ash.Changeset.new()
                        # `opts` and `context` aren't used by this change, so we can
                        # leave them empty
                        |> Tunez.Music.Changes.UpdatePreviousNames.change([], %{})

                    assert Ash.Changeset.changing_attribute?(changeset, :previous_names)
                    assert {:ok, ["george", "fred"]} = Ash.Changeset.fetch_change(changeset,
                        :previous_names)

            Should I Actually Unit Test Every Single One of These Things?
                Variations needs there own test and anything that you want to be sure that you want to make sure will always work you should test.

    Testing Interfaces
        Everything so far has been about resources or actions, but there will be needs to test the interface. So that you know that a user will have the same experience.

        Testing GraphQL
            Since AshGraphql is built on top of the excellent absinthe library, we can use its great utilities22 for testing. It offers three different approaches for testing either resolvers, documents, or HTTP requests.

        Testing AshJsonApi
            Everything we previously said for testing a GraphQL API applies to testing an API built with AshJsonApi as well. Since we generate an OpenAPI specification for your API, you can even use the same strategy for guarding against breaking changes.

        Testing Phoenix LiveView
            Testing user interfaces is entirely different than anything else that we’ve discussed thus far. There are whole books dedicated solely to this topic. LiveView itself has many testing utilities, and often when testing LiveView, we’re testing much more than the functionality of our application core.

    Fixing "pg-trgm" Error
        ** (Postgrex.Error) ERROR 0A000 (feature_not_supported) extension "pg-trgm" is not available

        hint: The extension must first be installed on the system where PostgreSQL is running.

        Want to figure out why this is not working need to test this for the tunez_test db.