TestBalloon styling – have it your way

Routinely, a test framework comes with its preferred style and TestBalloon is no exception: It registers test suites and tests via testSuite() and test() invocations.

As a DSL-based Kotlin-first framework, TestBalloon is obviously very flexible, letting us structure tests in plain Kotlin with parameters, loops and whatever it takes. But how much freedom does its API support? How far can we go if we want to change its look and feel?

Let's try, starting with two popular JavaScript styles from Jest and Mocha. But we won't stop there and find out if we could suppport the Gherkin grammar (Scenario/Given/When/And/Then) made popular by Cucumber, a BDD test framework.

Can we convince TestBalloon, via its public API, to accept a totally different structure?

Jest style

Jest is a JavaScript test framework used in the React world. Like TestBalloon, Jest uses test() to register a test. For a test suite, Jest uses describe(). So we'd want to write:

describe("Integer operations") {
    test("max(5, 3) returns 5") {
        assertEquals(5, max(5, 3))
    }
}

Making TestBalloon accept that is easy. The common DSL practice of creating an extension function is all it takes:

fun TestSuiteScope.describe( // (1)!
    name: String,
    testConfig: TestConfig = TestConfig, // (2)!
    content: TestSuite.() -> Unit
) = testSuite(name, testConfig = testConfig, content = content)

1. We are extending TestSuiteScope instead of just TestSuite to make describe work everywhere, including in special-purpose scopes (like those for fixtures and Robolectric).
2. Always offering a testConfig parameter is a good practice for customizability.

Mocha style

Mocha, another popular JavaScript test framework, uses describe() for test suites like Jest, but prefers it() for tests. We'd want to write:

describe("Integer operations") {
    it("max(5, 3) returns 5") {
        assertEquals(5, max(5, 3))
    }
}

Again, a single extension function suffices:

fun TestSuiteScope.it(
    name: String,
    testConfig: TestConfig = TestConfig,
    action: suspend Test.ExecutionScope.() -> Unit
) = test(name, testConfig = testConfig, action = action)

Gherkin style

We'd like to use a simplified version of the Gherkin style, that lets us write our tests like this:

Scenario("Too much food syndrome", { Belly() }) { // (1)!
    Given("I eat 48 cucumbers") { // (2)!
        eatCucumbers(48)
    }

    When("I wait for an hour") { // (2)!
        delay(1.hours)
    }

    Then("my belly growls") { // (3)!
        assertTrue(growls())
    }
}

1. Makes the initial context Belly() available to steps in its scope via this.
2. A Gherkin step, modifying the common context (state).
3. A Gherkin step, checking a result.

Our Scenario function creates an initial context (the Belly object). Subsequent steps then operate on this context (Belly.eatCucumbers(48), Belly.growls()). The sequence of steps is called a step definition in Gherkin.

While Gherkin specifies names for step types and provides recommendations on how to use them, what happens inside a step is up to its users. Each step can modify state (the context), or it can test state, or both.

How do we make TestBalloon handle this structure?

A Scenario with a DSL scope

To make TestBalloon support a scenario, we create a function like this:

fun <Context : Any> TestSuiteScope.Scenario( // (1)!
    description: String,
    context: suspend () -> Context, // (2)!
    testConfig: TestConfig = TestConfig, // (3)!
    content: StepDefinition<Context>.() -> Unit // (4)!
) {
    testSuite("Scenario: $description", testConfig = testConfig) { // (5)!
        StepDefinition(testSuiteInScope, context).apply {
            content() // (6)!
            register() // (7)!
        }
    }
}

1. Our Scenario function is part of a test suite. We make it an extension of TestSuiteScope as shown before.
2. The lambda returns the context object used in Gherkin steps.
3. Offering a testConfig parameter is a good practice for customizability.
4. The trailing lambda operates in its StepDefinition DSL scope where steps are defined.
5. We create a test suite with a scenario description.
6. We initialize our DSL scope (more on that later), run our trailing lambda in that scope…
7. …and make the completed step definition register its tests.

1, 2 step

The StepDefinition DSL scope owns our steps. It provides those steps with our common context object and orchestrates when and how our steps perform.

The initial iteration of StepDefinition looks like this:

class StepDefinition<Context : Any>(
    override val testSuiteInScope: TestSuite, // (1)!
    private val context: suspend () -> Context // (2)!
) : TestSuiteScope { // (1)!

    private class Step<Value : Any>(val description: String, val action: suspend Value.() -> Unit) // (3)!

    private val steps = mutableListOf<Step<Context>>() // (4)!

    fun Given(description: String, action: suspend Context.() -> Unit) { // (5)!
        steps.add(Step("Given $description", action))
    }

    // ... Step functions for `When`, `And` and `Then`

    internal fun register() { // (6)!
        testFixture { context() }.asParameterForAll {
            for (step in steps) {
                test(step.description) { context ->
                    step.action(context)
                }
            }
        }
    }
}

1. To integrate properly into TestBalloon's test element hierarchy, our class is a TestSuiteScope, providing a testSuiteInScope.
2. We capture the context created by our Given function above. The context will be lazily created as needed. If our tests are not selected for execution, it will not cost us a thing.
3. A step is nothing more than a description and an action to be performed.
4. Our step definition collects a number of steps.
5. The step function defining a Given step.
6. The register function creates a linear list of tests, one per step.

With that, this is how our test results are reported:

Same same but different

What if we'd prefer our steps to form a hierarchy?

Turns out we can do that as well. The implementation is more complex, since we need to transform our linear list of steps, making each step wrap around subsequent steps. But the Kotlin language and its standard library provide folding functions to achieve just this.(1)

1. See the code examples referenced at the bottom.

Let me choose

If we want to provide our users with a choice between the linear and hierarchical styles, we can create a TestConfig extension:

fun TestConfig.behaviorStyle(value: BehaviorStyle) = 
    parameter(BehaviorStyleParameter.Key) {
        BehaviorStyleParameter(value) // (1)!
    }

enum class BehaviorStyle {
    Linear,
    Hierarchical
}

1. For brevity, the implementation of BehaviorStyleParameter and its evaluation is left out here. It is shown in the sources listed below.

With that, wherever a testConfig parameter is present, we can specify our preferred style for that part of the test element hierarchy:

testConfig = TestConfig.behaviorStyle(BehaviorStyle.Hierarchical)

Parameterization for free

With the above and a little helper function (withExamples()), we can easily parameterize our tests in the Gherkin style we just created:

data class Example(val name: String, val initial: Int, val addend: Int, val factor: Int?, val expected: Int)

withExamples(
    Example("+/+/*", initial = 1, addend = 3, factor = 10, expected = 40),
    Example("-/+/*", initial = -3, addend = 5, factor = 10, expected = 20),
    Example("+/-", initial = 10, addend = -2, factor = null, expected = 8)
) {
    Scenario(
        "Calculate $name",
        context = { Calculator() },
        testConfig = TestConfig.behaviorStyle(BehaviorStyle.Hierarchical)
    ) {
        Given("I enter $initial") {
            set(initial)
        }

        When("I add $addend") {
            add(addend)
        }

        if (factor != null) {
            And("I multiply by $factor") {
                multiply(factor)
            }
        }

        Then("the result should be $expected") {
            assertEquals(expected, result)
        }
    }
}

Show me the code

• A complete implementation for the Gherkin BDD-style grammar: testLibrary/BehaviorStyle.kt
• The above examples and more: UsingBehaviorStyle.kt

Freedom

The TestBalloon API is perfectly capable of letting us write and structure tests in ways that differ from its standard structure. As we have seen, we can get away with very small amounts of code, while also avoiding an extra dependency.

Note

Of course, a huge creative potential can be found outside TestBalloon. The Prepared test library stacks its own API on top of TestBalloon. The TestBalloon Addons library by A-SIT Plus provides a translation for Kotest's FreeSpec.

So we have a choice, not just for today, but also for future needs as they may arise. We are free to pick what fits our use case best. We are independent of other people's opinions, be it the creators of TestBalloon or other libraries.

If you ask me, at some point this can be the stuff that might save our day. Just like Kotlin.