About testing your code in Python (Part 2) — Writing your first tests

In today's post, we'll gather practical experience with pytest by writing unit tests for an example project.

The Project

Although pytest can handle vastly complex project structures, let's start with a simple one:

In our project folder, there will be one file called math.py which contains our code, a factorial function.

The project's structure looks like the following:

my_project
|- myproject
	|- __init__.py
	|- math.py

Here is math.py:

def fac(x:int):
		if not isinstance(x, int):
			raise Exception("Wrong data type. x must be an integer!")
			
		if x < 0:
			raise Exception("x must be a positive integer!")

    if x == 0:
        return 1
    else:
        return x * fac(x-1)

Installing pytest

Since pytest is an additional dependency, we need to install it first. Therefore, we set up a virtual environment. Execute the following command, while being in your projects root folder (the first myproject folder):

python3 -m venv .venv

Now, we activate it. The commands are OS-specific. On Linux and macOS, we activate the virtual environment with:

source .venv/bin/activate

On Windows, it depends on the terminal you use:

# cmd.exe
venv\Scripts\activate.bat
# PowerShell
venv\Scripts\Activate.ps1

Finally run:

pip install pytest

Structuring your tests

Independent of the framework you'll be using, there is a guideline how your tests should be structured. In a nutshell, you create a tests folder and mimic the directory structure of your package. In our case, it should look like the following:

my_project
|- tests
	|- __init__.py
	|- test_math.py

Notice, that we add a test_ prefix to all Python files that will contain tests.

Building the test-suite

Writing a good testing-suite consists of three major steps:

1. Design
2. Write
3. Execute

In the following sections, we'll discuss each step in more detail.

Step 1: Design — Thinking about edge cases

Think twice, code once! — Unknown

When creating your test-suite with pytest, usually you spend more time thinking about test cases than writing the tests. It's especially important to test for edge cases in your code. The more unique edge cases you come up with, the less unpleasant surprises you'll have.

For our project, we'll test two cases:

1. x=0
2. x=2
3. x=2.5
4. x=-1

Let's dissect each test case separately, so you get the gist of things.

First test: x=0

Mathematically, this case is trivial, since it is 1 by definition. However, we need to translate this into code which results in conditional logic, i.e., if x==0:, which needs testing. This is our first edge case.

Second test: x=2

We test, here, the conditional logic (else), recursion steps (does it stop?), and a non-trivial solution at once.

x=1 is not a viable candidate, since the test's result would be inconclusive, since we get again 1 as a result. That means, if the conditional logic fails, there is no way of knowing that. Therefore, x=2 is our second edge case.

Third test: x=2.5

But what happens when we add a non-integer as an argument to the function? Since factorial is defined only for integers, our code raises an error. In this test, we check if that happens.

Fourth test: x=-1

Another edge case is that the factorial function is defined for positive numbers only. Our code should raise an error otherwise. Therefore, we need to assure that this error gets raised once a negative integer is given as an argument.

Since all solutions from now on will cover positive integers and non-trivial solutions, our tests covered all possible execution pathways and therefore four test cases are enough.

As you see, thinking thoroughly about the edge cases can save you a lot of time and despair. Keep that always in mind while designing your tests.

Another thing to keep in mind is that we test only our code. Meaning that if you had imported a factorial function from another package, e.g., NumPy, you don't test that functionality.

The logic behind that is simple: You use only packages you trust. Once you don't trust a package, you shouldn't use it.

Step 2: Write tests

Writing tests with pytest is simple — you add a test_ prefix to a functions name, put it into a module in the tests folder, and pytest takes care of the rest. However, it's good practice to use the same project structure in your tests, i.e., test functions reside in the modules testing twin.

In our case, the test would go into tests/test_math.py, since math.py has a function called fac:

from ..math import fac

def test_fac_0():
	number = 0
	exp_value = 1	
	res = fac(number)
	assert res == exp_value

Note, that we not only added the test_ prefix in front of fac but _0 as well. That helps us to keep track of the intention of the test. You could name the test function however you want, but it's good style to name it after the function it's supposed to test.

This test, however, only covers checks for specific return values in our function. However, we have test cases that check if an error was raised.

The following test checks if the correct data type was handed over as an argument to fac:

import pytest
from ..math import fac

def test_fac_wrong_data_type():
	number = 2.5

	# Check if exception was raised
	with pytest.raises(Exception) as e_info:
		fac(number)

	# Check for exception message
	assert str(e_info.value) == "Wrong data type. x must be an integer!"

Let's take a closer look at that function, ignore the first line, and instead focus on the first code block:

Here, the function checks if an error of the type Exception was raised. You need to change that if you expect another exception, e.g., DataError.

The second block ensures that the error message we receive is what we expect. This one is optional but recommended. When debugging, misleading errors and messages can be costly. Therefore, checking for the correct message saves you time eventually.

As a final note: The number and exp_value variables seem pointless for now. However, there are two reasons why they are here:

First, we will use these variables to simplify our test suite drastically once we will parametrize our tests. This will happen in another post.

Second, I need an excuse to talk about the anatomy of a unit test.

The anatomy of a unit test

There are four parts you can dissect a test into:\

1. Arrange
2. Act
3. Assert
4. Cleanup

Arrange are the lines where we define the variables in our test function. Here we prepare everything our function needs to be properly tested.

Act is the step in which you actually run the to-be-tested function, i.e., res = fac(number) in test_fac_0.

In assert, you check if the result of your function meets your expectations, i.e., assert res == exp_value in test_fac_0.

Cleanup is the part that makes sure the following tests don't trip over each other, e.g., reuse unwanted values, mocks, etc. In our case, the test is a function and once we are out of the function's scope, the garbage collector cleans up any remaining junk. See it as pytest preparing a clean working space for the next test.

This concept is vital to understand how unit tests work and how to structure them. However, reality is complicated, so see it more as a guideline to follow, instead of trying to squeeze everything into it.

Finishing our test cases

For completeness's sake, here are all four test cases as unit tests:

import pytest
from ..math import fac

def test_fac_0():
	number = 0
	exp_value = 1
	res = fac(number)
	assert res == exp_value
	
def test_fac_2():
	number = 2
	exp_value = 2
	res = fac(number)
	assert res == exp_value

def test_fac_wrong_data_type():
	number = 2.5

	# Check if exception was raised
	with pytest.raises(Exception) as e_info:
		fac(number)

	# Check for exception message
	assert str(e_info.value) == "Wrong data type. x must be an integer!"

def test_fac_negative_int():
	number = -1

	# Check if exception was raised
	with pytest.raises(Exception) as e_info:
		fac(number)

	# Check for exception message
	assert str(e_info.value) == "x must be a positive integer!"

Step 3: Executing the test

Now comes the easy part. Simply type pytest into your command-line while being in the project's folder. You should see something similar to:

Verdict

As you see, writing unit tests is easy, as long as you think about the edge cases and plan ahead. Since pytest does the heavy lifting, you can focus on your code. 

I should note that textbooks always recommend to first create your tests, and then the code. Essentially, the other way as we did. The discussion on that can be a post on itself. Let me just acknowledge, here, that this paradigm exists.

However, there is much more pytest can do. In the next post, you’ll learn about fixtures to keep your unit tests as slim as possible.