Two months ago new version of KotlinTest was released - brand new shiny 3.0. Even logo is new! There were some issues with previous version that I described in The best testing framework for Kotlin. Let’s give it another try.

TL;DR KotlinTest looks good!

My main issue with KotlinTest was slow development and small - but painful - bugs. KotlinTest doesn’t follow JUnit with many things, for example one instance per test. And I couldn’t even enforce this at a time - issue was lingering more than I’d like to, but finally: it’s fixed.

And not only that. I gave it a try and my experience was great. Everything simply works and support for Android project is almost flawless. Almost, because Android Instrumented Test don’t work at all, but I don’t use them often. Admittedly, integrating with Android-specific tests is not easy task.

But with support for JUnit 5 I was afraid of integrating KotlinTest with JUnit 4 and gradle-based Android projects. Luckily, no worries, normal unit tests work just fine. I created small Android project on GitHub which you can see here: kotlin-testing.

App itself doesn’t do much, but main logic is about verifying payment card number validity and recognizing who produced this card. Just so I have something to test.

Test nesting

I was really surprised how well nested tests look in Android Studio. I was trying a few different configurations for KotlinTest and one of them was many levels of nesting, this is how it looks like:

How awesome is that?

Pretty cool!

Especially as nested tests are one of the most prominent features of KotlinTest when comparing to JUnit 4. There are quite a few different styles and I’m sure everyone will find one that suits best. Even if you don’t care about nesting.

Table testing

Another nice thing is table testing. You can read official docs here but if you’ve ever used Spock, you probably know what this is and how useful it might be. If not, see below:

Normal JUnit4-style tests for recognizing card issuer:

class CardIssuerTest : StringSpec({
    "When card number starts with 34 or 37 the issuer should be American Express" {
        CardIssuer.fromCardNumber("344013826477034") shouldBe AMERICAN_EXPRESS
        CardIssuer.fromCardNumber("379237058082588") shouldBe AMERICAN_EXPRESS
    }

    "When card number starts with 5 the issuer should be Mastercard" {
        CardIssuer.fromCardNumber("5494275943345454") shouldBe MASTER_CARD
    }

    "When card number starts with 4 the issuer should be Visa" {
        CardIssuer.fromCardNumber("4716157914089748") shouldBe VISA
    }

    "When card number starts with 30 the issuer should be JCB" {
        CardIssuer.fromCardNumber("3096164799494029") shouldBe JCB
    }
})

Its equivalent using table testing:

class CardIssuerTableTest : StringSpec({
    "Correctly recognizes card issuer" {

        val cardData = table(
                headers("cardNumber", "issuer"),
                row("344013826477034", AMERICAN_EXPRESS),
                row("379237058082588", AMERICAN_EXPRESS),
                row("5494275943345454", MASTER_CARD),
                row("4716157914089748", VISA),
                row("3096164799494029", JCB)
        )

        forAll(cardData) { cardNumber, issuer ->
            CardIssuer.fromCardNumber(cardNumber) shouldBe issuer
        }
    }
})

Granted, it’s not exactly the same as name is not that precise anymore.

How awesome is that?

But you see how it might be time and space saving. And sometimes you don’t want to have separate tests for each tested value, as it might get messy.

BUT if you want separate tests, it’s worth noting there is no equivalent of @Unroll from Spock. Fear not! Because you build test cases with normal methods, you can obtain the same result with KotlinTest, generating tests in loop, see below:

class CardIssuerManualTableTest : FeatureSpec({
    feature("Card issuer recognition") {

        val cardData = mapOf(
                "344013826477034" to AMERICAN_EXPRESS,
                "379237058082588" to AMERICAN_EXPRESS,
                "5494275943345454" to MASTER_CARD,
                "4716157914089748" to VISA,
                "3096164799494029" to JCB)

        cardData.forEach { cardNumber, issuer ->
            scenario("card number $cardNumber should be recognized as $issuer") {
                CardIssuer.fromCardNumber(cardNumber) shouldBe issuer
            }
        }
    }
})

And in Android Studio it looks like this:

Property testing

It gives great flexibility when creating tests and their names. Code is the limit.

Matchers

Awesome matchers are still there. You can see full list here: Matchers. There are many of them and finally shouldBe is doing equality comparison, not instance comparison which for me is more natural. Also there are now matchers for Arrow. Very useful? Maybe not, but nice nonetheless.

More

There are more new things worth knowing about:

  • config is now done after test’s name, not after the whole test, which IMHO is much better
  • oneInstancePerTest now works, see example in my repo

And a few old things worth knowing about:

If Github stars count is any indicator, KotlinTest is getting much more popular, which is always a good sign. More importantly, Stephen Samuel (KotlinTest creator) is working on the project, a lot. Pull requests are merged, issues are discussed, new versions are released quickly.

Future for KotlinTest is bright, but we will see if it can outshine JUnit 5.