Memory profilingΒΆ

Memory profiling is much more cumbersome than time profiling.

  • memory_profiler (external dependency) allows a line by line memory usage analysis of decorated functions.

  • tracemalloc (standard) for manual instrumentation

Using the memory_profiler.profile decorator, a console output is produced, for example:

from memory_profiler import profile

@profile
def task_2(size: int) -> None:
   _ = [0] * size

Line #    Mem usage    Increment  Occurrences   Line Contents
=============================================================
 24    150.0 MiB    150.0 MiB           1   @profile
 25                                         def task_2(size: int) -> None:
 26    211.1 MiB     61.0 MiB           1       _ = [0] * size

Exercise (easy)

What was the size value for the example above to make sense?

Correction

An int is 8B, a MB is \(1024^2 B\), \(61\times1024^2/8\approx8\) million ints.

To measure a specific memory usage variation :

1tracemalloc.start()
2_, peak_before = tracemalloc.get_traced_memory()
3# do something
4_, peak_after = tracemalloc.get_traced_memory()

Exercise (easy)

Verify the memory_profiler values using tracemalloc on task_2.

Correction
1tracemalloc.start()
2_, peak_before = tracemalloc.get_traced_memory()
3task_2(8_000_000)
4_, peak_after = tracemalloc.get_traced_memory()
5print((peak_after - peak_before) / (1024 * 1024)) # 61 MB