Functional tools

This section presents the most important tools from functools

decorator

You already encountered a decorator in the memory profiling section. Let’s create our own decorator to understand the process.

A decorator takes a callable as input argument, and returns an altered version of the callable which is then substituted: the call to the original callable will trigger the call to the altered version instead.

Exercise (easy)

Modify the code below in order to print the args and kwargs before the call and the results after the call. Implement and test.

from collections.abc import Callable

def decorator_name(func_to_decorate: Callable) -> Callable:
   def altered_function(*args, **kwargs):
      ... # do something before the call
      results = func_to_decorate(*args, **kwargs) # perform the expected task
      ... # do something after the call
      return results
   return altered_function

@decorator_name # equivalent to : add = decorator_name(add)
def add(x: float, y: float) -> float:
   return x + y

Correction

src

def print_input_output(func_to_decorate: Callable) -> Callable:
    def altered_function(*args, **kwargs):
        print(args, kwargs)
        results = func_to_decorate(*args, **kwargs)
        print(results)
        return results

    return altered_function

tests

def test_print_input_output(capsys):
    @print_input_output
    def return_true(first, second):
        return True

    return_true(0, second=1)
    captured = capsys.readouterr()
    assert captured.out == "(0,) {'second': 1}\nTrue\n"

wraps

If one defines a totally useless decorator such the one below, and prints some information about the decorated function: its name, its docstring and its type annotations. The result is disappointing, especially when the decorator is used on several functions:

# import functools

def change_nothing(func_to_decorate: Callable) -> Callable:
   # @functools.wraps(func_to_decorate)
   def altered_function(*args, **kwargs):
      func_to_decorate(*args, **kwargs)
      return func_to_decorate(*args, **kwargs)

   return altered_function


@change_nothing
def return_true() -> bool:
   """return_true docstring"""
   return True


def show_wraps_utility() -> None:
   print(return_true.__name__)
   print(return_true.__doc__)
   print(return_true.__annotations__)

All the decorated function information are replaced by the wrapper (altered_function) information. To keep the original information the functools.wraps is very useful.

$ altered_function
$ None
$ {}

Exercise (easy)

Uncomment the two commented lines and compare to the original version.

Correction

with wraps the decorated function keeps its original information.

$ return_true
$ return_true docstring
$ {'return': <class 'bool'>}

cache

In the algorithm course, the notion of caching will be heavily used. Caching is storing in memory the result of previous (costly) computations in order to efficiently being able to retrieve them if identical input arguments are used again.

Exercise (easy)

What result wil the following code produce?

import functools
from time import sleep

@functools.cache
def expensive_task(number: int) -> int:
   sleep(0.5)
   print(f"calculating, number={number}")
   return number


def show_cache_utility() -> None:
   print(expensive_task(0))
   print(expensive_task(0))

Correction

$ calculating, number=0
$ 0
$ 0

The ‘expensive task’ is done only once. The second time the result is fetched in cache.

Exercise (intermediate)

1. Based on a dict, create your own cache decorator for positional arguments only. Test it.

2. Why keyword arguments would pose a challenge?

Correction

1. src

   def my_cache[Out](func: Callable[..., Out]):  # ignore the type hints
       _cache: dict[tuple[Hashable, ...], Out] = {}
   
       @functools.wraps(func)
       def wrapper(*args: Hashable) -> Out:
           cached: Out | None = _cache.get(args)
           if cached is not None:
               return cached
           value: Out = func(*args)
           _cache[args] = value
           return value
   
       return wrapper
   

2. Because a dict is not hashable and a pre-processing of positional and keyword arguments has to be made to use them as dict keys. The functools.wraps works with both though.

partial

Imagine the following setup:

def compute_something(unknown: float, param: float) -> float: ...


def find_root(func: Callable[[float], float]): ...

The function find_root works on a function of the unknown only. However, compute_something requires also param. There is a way to embed the value of param in a function definition (either manually or with functools.partial)

def show_partial_utility() -> None:
   param = 0.0

   def partial_of_compute_something_manually(unknown: float) -> float:
      return compute_something(unknown, param)

   _ = find_root(partial_of_compute_something_manually)

   partial_of_compute_something = functools.partial(compute_something, param=0)
   _ = find_root(partial_of_compute_something)