Kata Test Framework

  • Sep 10, 2013

Overview

Codewars has implemented a custom test framework for each language that it supports. This framework is designed to work similar to other popular frameworks.

Design Notes

It was decided to use a framework design that strikes a good balance between matching the API's of common testing frameworks (such as rspec and Jasmine) while also using a design that can be kept mostly in sync across the different languages.

As the testing framework's functionality develops, it will begin to contain more and more testing features that are specific to the concept of developing kata and not exactly just developing code in general. For example, the framework may incorporate testing features that are designed to score a user's code, instead of just marking it as passed/failed.

Note: If you have suggestions for how the framework design can be improved please feel free to discuss them below.

API Reference

This document is provided as an introduction to the kata testing framework. More specific API reference documents for each language will be provided that document the framework's methods in detail. See the related documents section for more info.

UI

Along with the test fixture section provided within the Kata Creator and Kata Trainer, there is also an output section. In the Kata Trainer the output section is its own tab. In the Kata Creator the output section appears after you choose to validate your kata.

The test framework will output log messages to the output section. If you are authoring a kata, be aware of this and how it can affect a user's understanding of the kata requirements as they try to solve it.

The Test Object

All current language implementations use "Test" as the singleton object/class for all of the test framework helper methods. A few of the methods on this class have also been aliased as standalone functions (such as describe, it and before) for convenience.

Test.expect(passed, msg(optional))

Test.expect is the core method for all language implementations. The only required argument is passed, which if the value is truthy then it will pass, otherwise it will fail. The first failed call to Test.expect will result in a message being shown to the user. The msg argument is optional but it is highly recommended to provide your own message that is more informative.

Usage

Test.expect can be used solely on its own, and many of the original kata on the site do only use this method. However more recent additions to the framework have been added which aid in its usage. It is highly recommended to use these helper methods instead of Test.expect. For example, all languages currently support assert equals and assert not equals helpers which will output much more useful information than Test.expect will. Please consult the language specific documentation for more details.

Spec Style Methods

The framework provides a common set of methods used in spec style testing.

describe

The describe method is used to group an entire set of related tests. It is highly recommended to use this method to group your tests, even if there will only be one group (which often there will be). The reason for this is that all tests will be ran inside of a describe call. This means if you have 10 calls to Test.expect and they all fail, all 10 of those failures will show up within the output. This can be very useful to a user who is trying to solve a kata.

it

The "it" method is used to group smaller sets of related tests. Sometimes only a single Test.expect call may be used within an "it" call. "it" calls should always be ran within the context of a describe call.

Ruby Example:
describe "Example Kata" do
    it "should have Foo defined" do
        Test.expect(defined?(Foo), 'Foo is not defined')
    end 
end

before

The before method is used to setup your tests. The code within the before callback will be ran before each "it" test is ran.

Helper Methods

In addition to the spec style methods there are also some helper methods to making using Test.expect easier. Many of them wrap Test.expect with additional functionality while others are useful for setting up test data. Consult the language specific docs for more details.

Discuss:

Loading comments...