.. role:: python(code)
   :language: python



.. index::
   testing
   fixture
   testing; parametrization
   testing; fixture
   testing; assertion

..
      `fixture <ex:fixture>`_

Test framework
--------------

`pytest <https://docs.pytest.org/en/stable/>`_ is the most popular test framework. It discovers your tests, automates their execution 
and reports the results. 
The basic concept is that a test asserts that the obtained result corresponds to the expected one: 
either because the values are identical, sufficiently similar, or because the expected exception has been raised.

Discovery
^^^^^^^^^

In the ``tests`` folder, any python file prefixed with ``test_`` will be discovered as a test file. 
Inside, any function prefixed with ``test_`` is a test and will be run.


Assertions
^^^^^^^^^^

In addition to the usual strict equality assertion :python:`assert obtained == expected`, two common cases occur: 

+ In order to test that the expected exception is raised, 
  a `context manager <https://docs.pytest.org/en/stable/reference/reference.html#pytest.raises>`_  exists:

  .. code-block:: python
      :linenos:

      # raises.py


      def raise_ValueError() -> None:
          raise ValueError("message")


  .. code-block:: python
      :linenos:

      # test_raises.py

      import pytest

      from tools_methods.testing.raises import raise_ValueError


      def test_raise_ValueError() -> None:
          with pytest.raises(ValueError, match="message"):
              raise_ValueError()



+ Because the comparison between floating numbers (real) is tricky (:python:`assert 0.1 + 0.2 == 0.3` fails). 
  There is a :python:`pytest` tool to assert the sufficiently `proximity <https://docs.pytest.org/en/stable/reference/reference.html#pytest.approx>`_ 
  of two numbers:

  .. code-block:: python
      :linenos:

      import pytest

      def test():
          assert 0.1 + 0.2 == pytest.approx(0.3)
          
  .. note::

      If the data structures are :python:`numpy` arrays, the :python:`numpy.allclose` is also useful.

Fixtures
^^^^^^^^

Your tests usually require some data to work on. 
The specific functions which builds this input data are called `fixtures <https://docs.pytest.org/en/stable/how-to/fixtures.html>`_. 

The basic syntax is simple, decorate a function returning the input data with :python:`@pytest.fixture`, then use this function name 
as an input parameter in every test functions where it is used. A trivial example would be:

.. code-block:: python
   :linenos:

   # trivial.py


   def add_one(number: int) -> int:
      return number + 1


.. code-block:: python
   :linenos:

   # test_trivial.py

   import pytest

   from trivial import add_one


   @pytest.fixture
   def my_number() -> int:
      return 3

   def test_add_one(my_number: int) -> None:
      assert add_one(my_number) == my_number + 1


A fixture can even take one or several other fixtures as input parameters.


.. admonition:: Exercise (easy)

   In the :ref:`exercise about project groups <ex:project_groups>`, 

   .. literalinclude:: ../../../../src/tools_methods/basics/std_structures.py
            :pyobject: inverse_to_group_affiliation_of_students
            :name: inverse_to_group_affiliation_of_students_testing
            :linenos:    
            :caption: src 

   .. literalinclude:: ../../../../tests/tools_methods/basics/test_std_structures.py
         :pyobject: test_inverse_to_group_affiliation_of_students
         :name: test_inverse_to_group_affiliation_of_students_testing
         :linenos:  
         :caption: tests

   .. literalinclude:: ../../../../src/tools_methods/basics/std_structures.py
         :pyobject: has_no_common_student_between_groups
         :name: has_no_common_student_between_groups_testing
         :linenos:    
         :caption: src 

   .. literalinclude:: ../../../../tests/tools_methods/basics/test_std_structures.py
         :pyobject: test_no_student_in_common_between_groups_false
         :name: test_no_student_in_common_between_groups_false_testing
         :linenos:  
         :caption: tests

   Rewrite the tests using fixtures.

.. _`ex:fixture`:

.. collapse:: Correction

   .. literalinclude:: ../../../../tests/tools_methods/testing/test_std_structures_with_fixtures.py
         :pyobject: student_composition_of_groups
         :name: student_composition_of_groups
         :linenos:  
         :caption: tests   

   .. literalinclude:: ../../../../tests/tools_methods/testing/test_std_structures_with_fixtures.py
         :pyobject: test_inverse_to_group_affiliation_of_students
         :name: test_inverse_to_group_affiliation_of_students_with_fixtures
         :linenos:  
         :caption: tests

   .. literalinclude:: ../../../../tests/tools_methods/testing/test_std_structures_with_fixtures.py
         :pyobject: groups
         :name: groups
         :linenos:  
         :caption: tests   

   .. literalinclude:: ../../../../tests/tools_methods/testing/test_std_structures_with_fixtures.py
         :pyobject: test_no_student_in_common_between_groups_false
         :name: test_no_student_in_common_between_groups_false_with_fixtures
         :linenos:  
         :caption: tests

Some fixtures are already `built-in <https://docs.pytest.org/en/stable/reference/fixtures.html>`_ :python:`pytest`, for example:

+ ``tmp_path``, a temporary path to a temporary folder in which you can safely write /  read, which is cleaned up after the test.
+ ``capsys``, in order to catch the prints


However, you can create and use your own fixtures. 
For example, one or several functions of your code may require a specific data strutures, remember the student number - project group example. 
Even more, complex, one or several functions of your code may require an input data file (to import data, to convert data etc). 


If the same fixture is used in multiple test files (typically to test different modules), it is possible to create a ``conftest.py`` at the 
root of the ``tests`` folder and mutualize there those fixtures.


Parametrization
^^^^^^^^^^^^^^^

Sometimes the same test is to be run with different input parameters. Instead of either duplicating the test; 
or write your own loop inside over the cases, pytest offers this feature as `parametrization <https://docs.pytest.org/en/stable/how-to/parametrize.html#>`_.

You can either parametrize the test directly, or you can parametrize the feature. The first solution is often more readable if suitable. 

.. literalinclude:: ../../../../tests/tools_methods/testing/test_fixture_parametrize.py
      :name: parametrized_test
      :linenos:  
      :caption: tests