Unit Tests

Once you’ve created a function, how do you know if it is correct? You could carefully read over the function, but you might miss something. Instead, programmers use collections of Unit Tests to determine if their code works as expected. Unit testing is an important professional practice to help you find problems early, facilitate changes down the road, and make it easier to glue together pieces of code. Unit tests are not magic, however. They cannot tell you that a program is definitely correct. They can only tell you if a program is definitely not correct.

Example Unit Test

# Import unit test function from testing module
from bakery import assert_equal

# Function definition
def feet_to_inches(distance: int) -> int:
    return distance * 12

# Actual unit test
assert_equal(feet_to_inches(2), 24)

A unit test is pretty simple: call a function with a known input and compare the result to a known output. For example, consider this function that converts feet to inches. Above the function call is a special statement that imports, or makes available, the assert_equal function from the test module. We use this function on the line below to write our unit test. We call feet_to_inches and pass in 2, and assert that we expect a result of 24.

Code Coverage

from bakery import assert_equal

def feet_to_inches(distance: int) -> int:
    return distance * 12

assert_equal(feet_to_inches(2), 24)
assert_equal(feet_to_inches(.5), 6.0)
assert_equal(feet_to_inches(0), 0)
assert_equal(feet_to_inches(-4), -48)
assert_equal(feet_to_inches(10000), 120000)

When you write many unit tests, you increase the coverage of your code. Coverage is usually measured by the number of lines of code that are executed, divided by the number of lines of code you’ve written. Right now, if you write any unit tests, you’ll be able to get good code coverage. Soon, we will see that you will need to write more unit tests to handle loops and conditionals. But for now, avoid having uncovered code by writing unit tests for each function.

Representative Cases

Case Input Output
Positive integer112
Positive float.56.0
Zero00
Negative number-4-48
Large positive number10000120000

A tricky part of making unit tests is thinking of appropriate test cases. The goal is to cover many possible scenarios, while writing the minimum number of cases. In our conversion function before, we tried it with a small positive number, a decimal number, a negative number, a large number, and zero. These are all representative cases. Depending on what our function is doing, you might need more or fewer test cases.

Representative Cases for Strings

Representative cases vary by the type of the input. The Boolean type only has two possible values, so you should usually test both a True and a False value to get full coverage. But if you are working with strings, you instead have to think about letters, digits, symbols, and whitespace characters. Consider not only strings that are ONLY these characters, but also mixtures of these characters, in varying amounts, like a mixture of only letters and digits. You also always want to think about the special case of the empty string, which often breaks string handling programs.

Failing Unit Tests

When a unit test fails, it reports the actual and expected values, so that the programmer can then debug their program. If you only try a function with one output, you might think that the function is correct. But until you run unit tests, you shouldn’t believe that. Of course, unit tests are not conclusive either, they are merely evidence to support the hypothesis that your code is correct.

Passing Unit Tests

The larger your program, the more important it is to unit test individual pieces. However, unit tests are also helpful as you learn the basics. Every program you write in this course will be unit tested, which means that we will try running your program against many inputs and outputs. Sometimes you will be asked to write your own unit tests, and sometimes we will run our own unit tests on your code. Just because your program appears to give the right output, you must pass all the unit tests!

Judging Unit Tests

from bakery import assert_equal
# Incorrect version!
def add(left: int, right: int) -> int:
    return left + 4
# Valid tests show the correct behavior
assert_equal(add(1, 4), 5)
assert_equal(add(3, 4), 7)
# Thorough tests expose the incorrect version's failure
assert_equal(add(4, 3), 7)
assert_equal(add(5, 5), 10)

Your collection of unit tests need to be valid and thorough. A collection is valid if the assertions pass for all possible correct implementations of the program. A collection is thorough if the assertions fail for any possible incorrect implementations. Although it is impossible to consider every possible way to write a program, you should spend some time thinking about alternatives to your solution. It can be very difficult to think of test cases that break your program, but doing so is critical to writing good tests!

Testing Before Defining

Six connected arrow shapes showing the function design recipe in order: Read the problem, Define the header, Write a Docstring, Write some example tests, Write the body, Write more tests.

Many computer scientists recommend writing tests BEFORE you write the body of the function. This forces you to think about the problem you are working on and what the expected result should be. If you cannot write unit tests for the function, then you likely do not understand the problem well enough to write the body. Often, writing out the tests will help you realize what the body of the functions should do. Sometimes this means you must do the complex math or other operations of the body yourself, which can be tiresome, but again, can often help you understand how the body is supposed to work.

Function Design Recipe

  1. Read the problem
  2. Define the header
  3. Write a docstring
  4. Write some example tests
  5. Write the body
  6. Write more tests

The previous diagram lays out the steps usually recommended in the process of defining a function. This model is sometimes called the “design recipe” for defining functions. Here are the steps in a bit more depth:

Read the problem: look over the description of the function to identify the function’s name, parameters, and expected return type. Make sure you understand what the function is supposed to do.

Define the header: write the first line of the function using the def keyword.

Write a docstring: create the body of the function with a triple-quoted string that explains what the function is supposed to do. Alternatively, instead of this documentation, sometimes you might just write the word pass so you have something in place.

Write some example tests: before you write the body properly, most experts suggest writing a few example tests first. To reiterate, this helps you understand what the function is supposed to do. But don’t exhaust yourself writing all the tests you need; focus on just writing a few examples to guide your thinking.

Write the body: now you can sit down and write the body of the function. This can be one of the hardest and most complicated steps with the most variety. However, the earlier steps can help you solve this step.

Write more tests: after your function’s body is written, you should write more tests to increase your confidence that the function works.

Often, writing tests is easier when you have already written the body since you can think of more concrete examples related to the code. Still, writing tests before the body often forces you to think more deeply about the problem without getting bogged down with implementation details.

Summary