Configuration

Introduction🔗

TestBalloon provides two mechanisms to adapt the testing process to your needs via plain Kotlin:

TestSuite extensions to register custom tests or test suites.
TestConfig, a uniform builder API to configure test elements at any level:

Single testTest suiteTest compartmentTest session

Note

TestBalloon aims to be as composable as possible with a simple but powerful API foundation. It is expected that users can more easily achieve their goals with a small amount of their own customization, rather than by using huge APIs and extension libraries.

`TestSuite` extensions🔗

To create reusable test variants, you can use extension functions on TestSuite.

A test with a timeout parameter, which also appears in the test's name:

fun TestSuite.test(
    name: String,
    timeout: Duration,
    action: suspend TestExecutionScope.() -> Unit
) = test(
    "$name (timeout: $timeout)",
    testConfig = TestConfig.testScope(false)
) {
    try {
        withTimeout(timeout) {
            action()
        }
    } catch (cancellation: TimeoutCancellationException) {
        throw AssertionError("$cancellation", cancellation)
    }
}

A reusable test series:

fun TestSuite.testSeries(
    name: String,
    iterations: Int,
    action: suspend TestExecutionScope.() -> Unit
) {
    for (iteration in 1..iterations) {
        test("$name $iteration") {
            action()
        }
    }
}

A test providing a database resource as a context:
```
@TestRegistering // (1)!
fun TestSuite.databaseTest(name: String, action: suspend Database.() -> Unit) {
    test(name) {
        Database(this).use { // (2)!
            it.action() // (3)!
        }
    }
}
```
1. This annotation makes the IDE plugin aware of the non-standard method signature.
2. Use a standard Kotlin scope function to safely close the resource after use.
3. All test actions can now directly invoke database functions via this.

Using the same technique, you can create custom test suites, or test suite series.

`TestConfig`🔗

Use the testConfig parameter in conjunction with the TestConfig builder to configure any part of the test element hierarchy – your tests, test suites, up to global settings.

testSuite(
    "let's test concurrency",
    testConfig = TestConfig
        .invocation(TestInvocation.CONCURRENT) // (1)!
        .coroutineContext(dispatcherWithParallelism(4)) // (2)!
        .statisticsReport() // (3)!
) {
    // ...
}

Use concurrent test execution instead of the sequential default.
Parallelize as needed (and the platform supports).
A custom configuration for extra reporting.

Custom combinations🔗

You can create a custom TestConfig extension combining the above configuration

fun TestConfig.onFourThreadsWithStatistics() = this // (1)!
    .invocation(TestInvocation.CONCURRENT)
    .coroutineContext(dispatcherWithParallelism(4))
    .statisticsReport()

Starting with this enables TestConfig method chaining: You build on what was present before.

and then reuse it as follows:

testSuite(
    "let's test concurrency",
    testConfig = TestConfig.onFourThreadsWithStatistics()
) {
    // ...
}

Custom extensions🔗

You can configure a custom TestConfig extension providing a test timeout:

fun TestConfig.withTestTimeout(timeout: Duration) = this // (1)!
    .testScope(isEnabled = false) // (2)!
    .aroundEachTest { action -> // (3)!
        try {
            withTimeout(timeout) {
                action()
            }
        } catch (cancellation: TimeoutCancellationException) {
            throw AssertionError("$cancellation", cancellation)
        }
    }

Starting with this, build on what was present before.
Enable real time.
Wrap around each test action(). By default, you must invoke it at some point, or configure an exception to that rule via TestConfig.addPermits().

The example in StatisticsReport.kt shows how to create a more complex custom TestConfig extension based on the existing traversal function.

You'll be basing a custom extension on one or more existing TestConfig functions. The wrappers are good candidates:

TestConfig.aroundAll
TestConfig.aroundEach
TestConfig.aroundEachTest

The TestConfig API documentation provides a complete list.

Global configuration🔗

TestSession and TestCompartment are special types of TestSuite that form the top of the test element hierarchy. Like any other TestElement, they can be configured via TestConfig.

Test compartments🔗

Tests may have different concurrency, isolation and environmental requirements. TestBalloon supports those via TestCompartments. These group top-level test suites, with each compartment running in isolation.

Info

If you use compartments C1, C2, C3, TestBalloon will execute all tests in C1, then all tests in C2, then all tests in C3. The order is not determined, but the isolation between all tests in one compartment against tests in the other compartments is guaranteed.

TestBalloon has a number of predefined compartments:

Predefined compartment	Configuration of top-level test suites inside the compartment
`TestCompartment.Concurrent`	concurrent/parallel invocation
`TestCompartment.Default`	according to `TestSession`'s default configuration
`TestCompartment.RealTime`	sequential invocation, on a real-time dispatcher, without `TestScope`
`TestCompartment.Sequential`	sequential invocation (useful if `TestSession` is configured differently)
`TestCompartment.MainDispatcher`	sequential invocation, with access to a multiplatform `Main` dispatcher

You can use these, or create your own compartments.

Choosing the compartment for a test suite🔗

By default, every top-level test suite will be in the TestSession's default compartment. Use the testSuite function's compartment parameter to put the test suite in a different compartment.

val RealTimeTests by testSuite(
    compartment = { TestCompartment.RealTime } // (1)!
) {
    // ...
}

For technical reasons, a compartment assignment must be done lazily.

Test session🔗

The TestSession is a compilation module's root test suite, holding the module-wide default configuration.

By default, TestBalloon uses a TestSession with a safe TestSession.DefaultConfiguration for all kinds of tests: It will

execute test elements sequentially
on Dispatchers.Default, and
use kotlinx.coroutines.test.TestScope inside tests.

Customization🔗

You can specify your own test session by declaring a class deriving from TestSession inside the test compilation module it should affect.

Tip

If you want to reuse a custom test session class from a library, put a class deriving from the library's custom test session class into each of your test modules.

To customize a TestSession, change its parameters from their defaults.

The testConfig parameter defines the global configuration for the entire compilation module. This example extends the framework’s default configuration:

class ModuleTestSession :
    TestSession(testConfig = DefaultConfiguration.statisticsReport())

Alternatively, or additionally, you can change the test session's defaultCompartment.

If all tests only mutate local state(1), you can speed up test execution greatly by choosing TestCompartment.Concurrent:

Ascertain that tests do not share mutable state among each other and do not access global mutable state.

class ConcurrentTestSession :
    TestSession(
        defaultCompartment = { TestCompartment.Concurrent } // (1)!
    )

For technical reasons, a compartment assignment must be done lazily.