Testing asynchronous code with Spock

Alternative title: What’s wrong with Spock?

There are some times when simple things just strike you and you have really no idea what’s wrong. This was one of these times. HOW ON EARTH CAN I CHECK IF SOME METHOD WAS CALLED ASYNCHRONOUSLY IN SPOCK?!?!?!?!?!?

I made some test. It didn’t work, so - following good rule of solving problems - I made it simpler. And simpler. Finally I’ve got:

def list = Mock(List)

def "test"() {
    when:
    new Thread({
        Thread.sleep(200)
        list.add("")
    }).start()

    then:
    Thread.sleep(300)
    1 * list.add(_)
}

It’s simple: I create a new thread, I wait 200 ms and I add item to the list. Finally, after waiting 300 ms - to make sure thread was finished - I check if add method was called. This should obviously work. And obviously it doesn’t. Surprisingly, below code (just moving Thread.sleep(300) two lines up) works:

def list = Mock(List)

def "test"() {
    when:
    new Thread({
        Thread.sleep(200)
        list.add("")
    }).start()
    Thread.sleep(300)

    then:
    1 * list.add(_)
}

Soooooo… Spock is broken, right? It seems that flow given-when-then is not called sequentially! So how should we write tests in Spock if order of lines doesn’t necessarily determine order execution?

Fortunately, it seems that in most cases it does, but that’s only small ‘fortunately’. Because unfortunately it seems that checking mock’s method execution is evaluated immediately when entering then block. It doesn’t care if there is any blocking operation (like sleep), it just goes straight to mock verification, even if there is some code before that.

When is - as you would expect - fully evaluated before moving to other block. That’s why putting Thread.sleep() inside when block works, but putting it inside then block, doesn’t.

At least that was my understanding from the tests I made. But as some other occasion I spotted interesting bit in Spock documentation:

Internally, Spock must have full information about expected interactions before they take place. So how is it possible for interactions to be declared in a then: block? The answer is that under the hood, Spock moves interactions declared in a then: block to immediately before the preceding when: block. In most cases this works out just fine, but sometimes it can lead to problems

And because there is never too much of explanation, some thoughts from Spock contributor, robfletcher:

Many people think that Spock’s mocking framework works like Mockito because it also allows to specify interactions after the fact. But in reality, it works more like EasyMock and JMock. That is, the interactions must be known beforehand. Spock achieves this by moving interactions in then-blocks before the corresponding when-block (at compile time). Compared to Mockito, this has advantages (e.g. a single interaction can be used for mocking AND stubbing) and drawbacks (if you factor out interactions into a helper method and insist on calling the helper method from a then-block, you have to wrap the call with “interaction { … }” to help Spock understand that the helper method contains interactions).

So actually it’s still true that interactions are evaluated at the beginning of then block, but it works by moving interaction definition before when block. That’s interesting.

Ok, so how can we deal with it? In this case interactions is not the solution. We still need to have workarounds and there are some ‘advised’ workarounds like Blocking Variable or something like good old CountDownLatch. They are not bad per se but in some cases - like mine case in production code - are not solutions. They assume that you pass callback that will be called asynchronously and this is not always the case.

Spock framework is just not suitable for such test. But you don’t have to give up on Spock, there are other possibilities:

use another tool, e.g. Mockito with its verification with timeout. It does exactly what we need and it is just a library so it works well with Spock
think about another way to write this test. In general it’s good practice to check what is the result of the test, not which mock was called. So if you can, try to evaluate result, what is the influence of the when block on outside world.

It took me so much time and I was really surprised this problem is not described well on the internet (or I didn’t find it). Future “me” will be very grateful when I come back to Spock.

Share on: