src.benchmark module

Benchmark implementation

class src.benchmark.Benchmark(input_path: Path)

Bases: object

Runs a benchmarks (memory, cpu) with (python, Cython, Nuitka) on input_path files using the:

  • benchmark pytest fixture from pytest-benchmark framework

  • @profile decorator from memory-profiler.

For memory benchmark the required directory structure needs to be:

module
    ├── sample_funcs.py                        # implementation
    ├── main.py                                # entrypoint
    ├── test_sample_funcs.py                   # test cases

where in main.py we call any functions from sample.funcs.py. This is needed because once we compile the sample_funcs.py we aren’t able to use the profile decorator from memory-profiler in that case, mem_bench function will decorate all the functions that they match the prof_func_name with @profile for example:

def sum_of_squares_benchmark():
    sum_of_squares()



@profile
def sum_of_squares_benchmark():
    sum_of_squares()

For cpu benchmark the required directory structure needs to be:

module
  ├── sample_funcs.py                        # implementation
  ├── test_sample_funcs.py                   # test cases

NOTES

  • this way of memory profiling isn’t accurate (because we don’t profile the actual python functions), and neither we can do for compiled code with python profiling tools, but it could be an indicator.

  • The test files are needed to be able to execute the functions with the right arguments.

static cpu_bench(input_path: Path, engine: str) → None

decorate all the test functions from the input_path with the @benchmark_wrapper decorator.

Finally, invoke pytest within.

static mem_bench(input_path: Path, prof_func_name: str, engine: str) → None

decorate all the function(s) from the input_path where their name match the prof_func_name pattern, with the @profile decorator from memory_profiler.

Finally, invoke pytest within.

start(bench_type: str, compilers: Sequence[CompilerWrapper], prof_func_name: str = 'benchmark') → None

Start the benchmark.