pyTest

About testing your code in Python (Part 4) — Parametrize your unit tests

Sandro Wrzalek

— 4 min read

Today, we’ll cover how you can write less test code while maintaining more tests easily, or in other words: How to parametrize your unit tests.

What does parametrize unit tests mean and why should use it

In pytest, parametrizing a unit test means that one function tests many cases at once while still being atomic and easy to read. That sounds contradictory at first, but it’s not: You write effectively multiple test functions while just coding one. In fact, you end up writing less lines of code!

That results in less work coding and maintaining your test suite, and having more time to focus on the actual project code itself. In addition to that, your test code is easier to extend since if another test case pops up, you add it to the already existing test function. That gives you a lot of flexibility. Finally, your test suite is far easier to read and can serve as an additional source for documentation.

Parametrizing our tests

So, how do we actually achieve this wonderful world of few lines, easy maintenance and expandable unit tests? Easier than you might think, but before starting, let’s revisit our test code so far:

import pytest
from ..math import fac

def test_fac_0():
	number = 0
	exp_value = 1
	res = fac(number)
	assert res == exp_value
	
def test_fac_2():
	number = 2
	exp_value = 2
	res = fac(number)
	assert res == exp_value

def test_fac_wrong_data_type():
	number = 2.5

	# Check if exception was raised
	with pytest.raises(Exception) as e_info:
		fac(number)

	# Check for exception message
	assert str(e_info.value) == "Wrong data type. x must be an integer!"

def test_fac_negative_int():
	number = -1

	# Check if exception was raised
	with pytest.raises(Exception) as e_info:
		fac(number)

	# Check for exception message
	assert str(e_info.value) == "x must be a positive integer!"

We can divide the existing four tests divide into two test classes/categories:

  1. Valid Result Tests: Asserting if a function returns the correct values.
  2. Error Handling Tests: Asserting if a function throws the desired errors.

By merging the tests in their corresponding categories, we still can achieve the same test suite, resulting in two, effectively four, unit tests. Let’s start with the first category.

Parametrizing the Valid Result Tests

It’s always easier to understand and describe something with an example on hand. Therefore, let’s see how the merged code looks like before dissecting it:

@pytest.mark.parametrize("number, exp_value", [(0, 1), (2, 2)])
def test_fac(number: int, exp_value: int):
    res = fac(number)
    assert res == exp_value

This looks already very familiar with the function bodies we already had. Therefore, we will focus on the first two lines, since they differ majorly from what we had before.

The parametrize decorator

The first line is where most of the magic happens: We call a decorator function from the pytest.mark module to create parameters and test cases which we’ll use in our test.

If you don’t know what a decorator is, don’t worry, you don’t need any profound knowledge about decorators for parametrizing unit tests. The only things you need to know about this one is that you instantiate a decorator with an @, you use it like a function call, and you call it right above your test function definition.

We create those parameters by providing a string to the parametrize function as the first argument. We use a comma to separate the parameter names from each other, so pytest can create them in the corresponding scope. In our case, we create two parameters, namely number and exp_value.

The second argument we hand to parametrize represents our test cases. We use a list of tuples here, since tuples can’t be changed at runtime. In our example, the first test case sets number=1 and exp_value=1 while the second set is number=2 and exp_value=2. For reference, take a look at our old test functions, i.e., test_fac_0 and test_fac_2. You’ll notice that those test cases are identical.

Using the created parameters in our test function

Now, we need to hand the sets we created down to the test function. We achieve that by handing the parameter names, we defined in the decorator, to our test function. The rest of the test function stays at it is.

Running our parametrized test function

We’re done with the test category, so feel free to run the test as always. You will notice that the number of executed tests did not change if you deleted test_fac_0 and test_fac_2.

How is that possible? We merged two test functions into one, right? The answer is yes and no. Yes, we merged both functions into one and therefore simplified our test setup. However, for pytest we still have four test functions, since each test set is executed separately and hence counts as one.

In other words, we told pytest to do the same thing as before, but more elegantly. In case it’s still a bit much for you to digest, let’s use the tests for error handling as another example.

Parametrizing the Error Handling Tests

We learned that we parametrize our unit tests by calling the decorator function pytest.mark.parametrize which takes two arguments: The variable names and the test sets we want to use. Then we hand over the parameter names to the test function, so it can use them.

Now, try merging test_fac_negative_int and test_fac_wrong_data_type on your own before checking how the parametrized version will look.

Did you try it? The new parametrized test function should look like that:

@pytest.mark.parametrize(
    "number, error_msg", [
        (2.5, "Wrong data type. x must be an integer!"), 
        (-1, "x must be a positive integer!")])
def test_fac_fail(number, error_msg: str):
    # Check if exception was raised
    with pytest.raises(Exception) as e_info:
        fac(number)

    # Check for exception message
    assert str(e_info.value) == error_msg

The process is identical, but the content of the test function is different. Therefore, we can repeat the process for any test with multiple cases, even some exotic ones. That’s simply beautiful!

Verdict

As you see, parametrizing our test functions doesn’t help us only to write less code and makes our test suite much leaner, but it helps us to easily extend it in the future by simply adding new test sets to the parametrize function. That way, we don’t have to replicate test code as we did before, which helps us to maintain our code much better. And finally, it helps the documentation aspect of our test code since it is much easier to read.

However, we are not done yet! There is still so much we can do with pyTest. In the next post of this series, we’ll cover how to execute specific tests with patterns and how to organize them so that you don’t waste time executing tests you aren't concerned about. In bigger projects with bigger test suites, it can save you literally minutes for every execution!

Read more