With every new language or even new environment there comes time to ask yourself very important question: How will I test that? Kotlin itselft is just awesome. But state of its test frameworks is just not awesome.
This is state for 15th January 2018. For update on current state of testing frameworks, see: Kotlin testing frameworks - revisited
To even start to consider using some framework, it needs to be popular. With popularity comes support and future development, that’s pretty obvious. Taking that into account for Kotlin currently there are 3 possibilities:
I’m not going to distinguish between JUnit and TestNG - you probably already have your favorite one, this articles focuses on new stuff for Kotlin. For the sake of simplicity I will just talk about JUnit.
Ok, let’s make it clear - current state of Kotlin test framework sucks.
I tried it in early 2017 and I failed miserably. Or maybe it was Spek that failed miserably with being user-friendly. It was before 1.0 version, so a lot could change, right? Current version is 1.1.15 so it works, right? Yeah… it doesn’t.
I was about to give Spek another chance. However, I watched talk from KotlinConf Testing Kotlin at Scale: Spek by Artem Zinnatullin. First of all, some arguments of Artem are really invalid with good programming practices, he is not good at convincing to switch. But I was eager to try Spek anyway, I hoped this video would be a nice introduction. It was a nice introduction but for NOT using Spek. Artem explicitly states that you should not use version 1.X, 2.0 is coming soon! It was about to come soon and we are over 2 months from the conference and on Spek repo:
We can keep waiting. If one of the main contributors say to wait for 2.0 release and not use older releases, I’m listening. It’s a pity we have to wait so long for something so promising. I’m just starting new project and was going to use Spek but with current state - not a chance.
TL;DR - Don’t use until 2.0
I already used Kotlintest in a project, because entry barrier is much easier than in Spek. I really enjoyed its structure - it’s lovely you can use the structure you like. Of course StringSpec FTW! Useful matchers, ease of asserting exceptions and eventually lambda make tests much more readable. I like it.
But nothing is perfect and one big imperfect thing is the development of the project. It’s slow. I really appreciate all hard work put into this, but the fact is that doing JUnit equivalent of
@After - which seems like the most basic stuff - is broken for a few months already. There is a fix for it but author of Kotlintest wants to make it to 3.0 already. And… we are still waiting for 3.0.
Also there is no way to run single test - Spek has this weird way of adding ‘f’ at the beginning of test and Kotlintest has only open issue for Intellij plugin.
Maybe these are small issues compared to all goodness coming from Kotlintest. If that’s true, feel free to use this framework. I just hope the development will speed up, because Kotlin’s version of scalatest would be amazing.
TL;DR - Cool constructs, better than Spek, can be buggy
I don’t have much experience with TestNG, but JUnit is just JUnit. Proven, everything works, stable, as future-proof as it gets. JUnit with Kotlin works great. Of course remember about naming your test with backtick and spaces
Beautiful enough <3
JUnit is framework that can be extended with use of Kotlin. Maybe you don’t need much more than that?
TL;DR JUnit is already pretty good and can be extended with Kotlin libraries
What would I choose? Definitely not Spek. I’m a fan of Kotlintest because it’s very promising but with its current state I’d lean towards JUnit with libraries. Old, boring JUnit, but refreshed thanks to Kotlin abilities.
Saying that, I root for Spek and Kotlintest to make Kotlin testing even nicer.