Test framework

pytest is the most popular test framework. It discovers your tests, automates their execution and reports the results. The basic concept is that a test asserts that the obtained result corresponds to the expected one: either because the values are identical, sufficiently similar, or because the expected exception has been raised.

Discovery

In the tests folder, any python file prefixed with test_ will be discovered as a test file. Inside, any function prefixed with test_ is a test and will be run.

Assertions

In addition to the usual strict equality assertion assert obtained == expected, two common cases occur:

• In order to test that the expected exception is raised, a context manager exists:

  # raises.py
  
  
  def raise_ValueError() -> None:
      raise ValueError("message")
  

  # test_raises.py
  
  import pytest
  
  from tools_methods.testing.raises import raise_ValueError
  
  
  def test_raise_ValueError() -> None:
      with pytest.raises(ValueError, match="message"):
          raise_ValueError()
  

• Because the comparison between floating numbers (real) is tricky (assert 0.1 + 0.2 == 0.3 fails). There is a pytest tool to assert the sufficiently proximity of two numbers:

  import pytest
  
  def test():
      assert 0.1 + 0.2 == pytest.approx(0.3)
  

  Note

  If the data structures are numpy arrays, the numpy.allclose is also useful.

Fixtures

Your tests usually require some data to work on. The specific functions which builds this input data are called fixtures.

The basic syntax is simple, decorate a function returning the input data with @pytest.fixture, then use this function name as an input parameter in every test functions where it is used. A trivial example would be:

# trivial.py


def add_one(number: int) -> int:
   return number + 1

# test_trivial.py

import pytest

from trivial import add_one


@pytest.fixture
def my_number() -> int:
   return 3

def test_add_one(my_number: int) -> None:
   assert add_one(my_number) == my_number + 1

A fixture can even take one or several other fixtures as input parameters.

Exercise (easy)

In the exercise about project groups,

src

def inverse_to_group_affiliation_of_students(
    student_composition_of_groups: dict[int, set[int]],
) -> dict[int, set[int]]:  # later, improvement of type hinting
    """from a dict[group, students], tranpose to dict[student: groups]"""
    group_affiliation_of_students: dict[int, set[int]] = {}
    for group, students in student_composition_of_groups.items():
        for student in students:
            if student not in group_affiliation_of_students:
                # later, improvement defaultdict
                group_affiliation_of_students[student] = set()
            group_affiliation_of_students[student].add(group)
    return group_affiliation_of_students

tests

def test_inverse_to_group_affiliation_of_students() -> None:
    student_composition_of_groups: dict[int, set[int]] = {
        0: {10, 11},
        1: {10, 12},
    }  # later, fixtures will be used
    obtained = inverse_to_group_affiliation_of_students(student_composition_of_groups)
    expected: dict[int, set[int]] = {
        10: {0, 1},
        11: {0},
        12: {1},
    }
    assert obtained == expected

src

def has_no_common_student_between_groups(
    groups: list[int], student_composition_of_groups: dict[int, set[int]]
) -> bool:
    """add in a set the students from several groups,
    if there is no student in common between any group,
    then the size of the set is the sum of the size of every group"""
    sum_group_sizes: int = 0
    students_merged: set[int] = set()
    for group in groups:
        students: set[int] = student_composition_of_groups[group]
        sum_group_sizes += len(students)
        students_merged |= students
    return len(students_merged) == sum_group_sizes

tests

def test_no_student_in_common_between_groups_false() -> None:
    student_composition_of_groups: dict[int, set[int]] = {
        0: {10, 11},
        1: {10, 12},
    }  # later, fixtures will be used
    groups: list[int] = [0, 1]

    obtained = has_no_common_student_between_groups(
        groups, student_composition_of_groups
    )
    expected = False
    assert obtained == expected

Rewrite the tests using fixtures.

Correction

tests

@pytest.fixture
def student_composition_of_groups() -> dict[int, set[int]]:
    return {
        0: {10, 11},
        1: {10, 12},
    }

tests

def test_inverse_to_group_affiliation_of_students(
    student_composition_of_groups: dict[int, set[int]],
) -> None:
    obtained = inverse_to_group_affiliation_of_students(student_composition_of_groups)
    expected = {
        10: {0, 1},
        11: {0},
        12: {1},
    }
    assert obtained == expected

tests

@pytest.fixture
def groups(
    student_composition_of_groups,
) -> list[int]:  # a fixture takes another fixture as input
    return list(student_composition_of_groups.keys())

tests

def test_no_student_in_common_between_groups_false(
    student_composition_of_groups: dict[int, set[int]], groups: list[int]
) -> None:
    # takes two fixtures
    obtained = has_no_common_student_between_groups(
        groups, student_composition_of_groups
    )
    expected = False
    assert obtained == expected

Some fixtures are already built-in pytest, for example:

• tmp_path, a temporary path to a temporary folder in which you can safely write / read, which is cleaned up after the test.

• capsys, in order to catch the prints

However, you can create and use your own fixtures. For example, one or several functions of your code may require a specific data strutures, remember the student number - project group example. Even more, complex, one or several functions of your code may require an input data file (to import data, to convert data etc).

If the same fixture is used in multiple test files (typically to test different modules), it is possible to create a conftest.py at the root of the tests folder and mutualize there those fixtures.

Parametrization

Sometimes the same test is to be run with different input parameters. Instead of either duplicating the test; or write your own loop inside over the cases, pytest offers this feature as parametrization.

You can either parametrize the test directly, or you can parametrize the feature. The first solution is often more readable if suitable.

tests

import pytest

from tools_methods.testing.fixture_parametrize import add_one


@pytest.mark.parametrize("my_number, expected", [(-1, 0), (2, 3), (4, 5)])
def test_add_one_parametrized(
    my_number: int, expected: int
) -> None:  # test parametrization
    assert add_one(my_number) == expected


@pytest.fixture(params=[-1, 2, 4])
def other_number(request) -> int:  # fixture parametrization
    return request.param


def test_add_one_fixture_parametrized(other_number: int) -> None:
    assert add_one(other_number) == other_number + 1