Built-in

Unit testing is built into the language and the tools, as well as the tutorial - I love this approach. When you create a new project with mix, this is the output you get:

$ mix new myproject
...
Your mix project was created successfully.
You can use mix to compile it, test it, and more:

    cd myproject
    mix compile
    mix test

Run `mix help` for more information.

The test skeleton is not beginner friendly

The skeleton unit test suffers the same problems like the Django equivalent - it is aimed at those who get unit testing already. If you are new to automated testing, seeing code like

defmodule FooTest do
  use ExUnit.Case

  test "the truth" do
    assert(true)
  end
end

won't really make you say "yes, this makes me understand and love testing!". Granted, the whole Elixir tutorial is aimed at people who can already program.

Simplified, yet flexible assertions

Unlike classic unit testing frameworks, which come bundled with numerous assertWhatever and assertNotWhatever etc. methods, (various overloads for each type), ExUnit mostly only uses assert and assert_raise (though there are some more - see ExUnit.Assertions). Rather, it relies on pattern matching and the assert macro being smart enough to figure out how to provide a good (enough) failure message - in this aspect, it reminds me of py.test

I'm curious to see whether it will stay this way - custom assertions are superb for writing DSL-like code that experts can read (e.g.: Hamcrest), and I really like the protocol-based extensibility model used in Elixir elsewhere.

Constraint of the failure messages

Your opinion of this feature might differ from mine, but it's worth pointing this out. While the assertions are pretty flexible, the actual error message will become the values on the left- and right hand side of the pattern matching. This takes away the information of exactly which method was called with what parameters, which I have grown to rely on in Python. Consider the following tests and the resulting failure messages - which one is more helpful?

Elixir:

    test "survival - a living cell with 2 or 3 neighbours survives" do
        assert Gofl.Rules.next_state(:alive, 2) == :alive
        assert Gofl.Rules.next_state(:alive, 3) == :alive
        assert Gofl.Rules.next_state(:dead, 3) == :alive
        assert Gofl.Rules.next_state(:dead, 2) == :dead
    end

  1) test survival - a living cell with 2 or 3 neighbours survives (GoflRulesTest)
     ** (ExUnit.ExpectationError)
                  expected: :alive
       to be equal to (==): :dead
     at unit_testing_with_elixir_gofl.exs:18



Finished in 0.05 seconds (0.04s on load, 0.01s on tests)
1 tests, 1 failures

Python:

    def test_survival_a_living_cell_with_2_or_3_neighbours_survives(self):
        self.assertEquals(ALIVE, cell_next_state(curr_state=ALIVE, live_neighbour_cnt=2))
        self.assertEquals(ALIVE, cell_next_state(curr_state=ALIVE, live_neighbour_cnt=3))
        self.assertEquals(ALIVE, cell_next_state(curr_state=DEAD, live_neighbour_cnt=3))
        self.assertEquals(DEAD, cell_next_state(curr_state=DEAD, live_neighbour_cnt=2))

F
======================================================================
FAIL: test_survival_a_living_cell_with_2_or_3_neighbours_survives (__main__.GoflRulesTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "unit_testing_with_elixir_gofl.py", line 22, in test_survival_a_living_cell_with_2_or_3_neighbours_survives
    self.assertEquals(DEAD, cell_next_state(curr_state=DEAD, live_neighbour_cnt=2))
AssertionError: False != True

----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (failures=1)

Granted, Python is unique in this regard, IIRC I wouldn't get such useful stacktraces in C# or Java either, but since I've been mostly working in Python lately, it certainly was a surprise - not necessarily for discouraging multiple assertions per test (it's a valid approach), but more the hiding of the call site information.

However, since assert itself is a macro, its technically possible to change the assertion - added to my list of things to try :)

:async

Test isolation is almost always hard, and these issues tend to only come out from hiding once tests need to run in parallel for speed - good things never come alone :) So having concurrent test running built in from the start is a great way to have the one big (future) problem broken into many minor inconveniences.

I also like that instead of a test runner command line switch, it is declared on the TestCase level whether it's OK to run it in parallel - giving finer control to the test author.

defmodule GoflRulesTest do
    use ExUnit.Case, async: true

I just wish it was true by default!

Cool things I would like to explore further

1. This short exploration hasn't allowed me figure what kind of

   differences there are between testing functional- and object oriented programs. My gut feeling is there shouldn't be many: black box, transformation (state) based should be the same. Interaction testing (mocking) seems an odd fit, however it might just turn out to be as easy as passing in a "mock" function as an argument.

2. Property-based testing has been on my radar for a while to be

   tried - and its use seems to be more common in functional programming, and it's appropriate I should try it in a context where it's set up to succeed. This is when I'm thankful for Elixir being built/run on top of a mature platform - otherwise there wouldn't yet be an available library.