APIs¶

Benchmark¶

class zeropybench.Benchmark(*, repeat=7, min_duration_per_repeat=0.2, verbose=False)[source]¶

Bases: object

A class for multidimensional benchmarking of code snippets.

Example:

import jax.numpy as jnp

bench = Benchmark(repeat=10, verbose=True)
for N in [10_000, 100_000, 1_000_000]:
    x = jnp.ones(N)
    y = jnp.ones(1000)
    with bench(method='broadcast right', N=N):
        x[:, None] + y[None, :]
    with bench(method='broadcast left', N=N):
        x[None, :] + y[:, None]
print(bench)

Output:

┌───┬─────────────────┬───────────┬────────────────────────────┬──────────┬───────────────────────────┬───────────────────────┐
│   ┆ method          ┆ N         ┆ median_execution_time (ms) ┆ ± (%)    ┆ first_execution_time (ms) ┆ compilation_time (ms) │
╞═══╪═════════════════╪═══════════╪════════════════════════════╪══════════╪═══════════════════════════╪═══════════════════════╡
│ 0 ┆ broadcast right ┆ 10_000    ┆ 1.621035                   ┆ 7.45449  ┆ 114.662899                ┆ 82.715331             │
│ 1 ┆ broadcast left  ┆ 10_000    ┆ 1.637076                   ┆ 1.211578 ┆ 76.16242                  ┆ 59.710502             │
│ 2 ┆ broadcast right ┆ 100_000   ┆ 10.030257                  ┆ 1.190149 ┆ 44.89509                  ┆ 35.997909             │
│ 3 ┆ broadcast left  ┆ 100_000   ┆ 10.297392                  ┆ 1.467783 ┆ 133.313475                ┆ 54.304368             │
│ 4 ┆ broadcast right ┆ 1_000_000 ┆ 94.146169                  ┆ 0.159088 ┆ 137.82041                 ┆ 44.644484             │
│ 5 ┆ broadcast left  ┆ 1_000_000 ┆ 196.27792                  ┆ 0.093503 ┆ 112.162553                ┆ 80.459331             │
└───┴─────────────────┴───────────┴────────────────────────────┴──────────┴───────────────────────────┴───────────────────────┘

Then export or plot:

bench.write_csv('bench.csv')
bench.plot()

Parameters:

repeat (int)
min_duration_per_repeat (float)
verbose (bool)

repeat¶

The number of times the estimation of the elapsed time will be performed. Each repeat will usually execute the benchmarked code many times.

Type:: int

min_duration_per_repeat¶

The minimum duration of one repeat, in seconds. The function will be executed as many times as necessary so that the total execution time is greater than this value.

Type:: float

DEFAULT_MIN_DURATION_PER_REPEAT: ClassVar[float] = 0.2¶: The default minimum duration per repeat in seconds.

DEFAULT_REPEAT: ClassVar[int] = 7¶: The default number of measurement repetitions.

__init__(*, repeat=7, min_duration_per_repeat=0.2, verbose=False)[source]¶

Returns a Benchmark instance.

Parameters:

repeat (int) – The number of times the estimation of the elapsed time will be performed. Each repeat will usually execute the benchmarked code many times.
min_duration_per_repeat (float) – The minimum duration of one repeat, in seconds. The function will be executed as many times as necessary so that the total execution time is greater than this value. The execution time for this repeat is the mean value of the execution times.
verbose (bool) – If True, print the setup and benchmarked code to stderr. For JAX benchmarks, this shows the JIT-compiled function definition and the actual timed call (e.g., __bench_func(x, y).block_until_ready()).

Return type:

None

plot(*, x=None, y=None, by=None, reference=None, **subplots_keywords)[source]¶

Plots the benchmark report in a figure.

Parameters:

x (str | Expr | None) – The x-axis of the plots, as a benchmark report column name or expression.
y (str | Expr | None) – The y-axis of the plots, as a benchmark report column name or expression.
by (str | Sequence[str] | None) – Key to divide into several subplots.
reference (str | None) – Legend label of the reference method for speedup comparison. When specified, a second column of subplots shows the speedup (reference_time / method_time) for each method. Values > 1 mean faster than the reference.
subplots_keywords (Any)

Return type:

None

to_dataframe()[source]¶

Returns the benchmark as a Polars dataframe with times in seconds.

Return type:: DataFrame

to_dicts()[source]¶

Returns the benchmark as a list of dicts.

Return type:: list[dict[str, Any]]

write_csv(path)[source]¶

Writes the benchmark report as CSV.

The file includes a header with metadata comments: - # repeat = <value> - # min_duration_per_repeat = <value>

Parameters:: path (Path | str) – The path of the CSV file.
Return type:: None

write_markdown(path)[source]¶

Writes the benchmark report as MarkDown table.

Parameters:: path (Path | str) – The path of the MarkDown file.
Return type:: None

write_parquet(path)[source]¶

Writes the benchmark report as Parquet.

The file includes metadata: - repeat: The number of measurement repetitions - min_duration_per_repeat: The minimum duration per repeat in seconds

Parameters:: path (Path | str) – The path of the Parquet file.
Return type:: None

write_plot(path, *, x=None, y=None, by=None, reference=None, **subplots_keywords)[source]¶

Saves the benchmark plot in a file.

Parameters:

path (Path | str) – The path of the plot file.
x (str | Expr | None) – The x-axis of the plots, as a benchmark report column name or expression.
y (str | Expr | None) – The y-axis of the plots, as a benchmark report column name or expression.
by (str | Sequence[str] | None) – Key to divide into several subplots.
reference (str | None) – Legend label of the reference method for speedup comparison. When specified, a second column of subplots shows the speedup (reference_time / method_time) for each method. Values > 1 mean faster than the reference.
subplots_keywords (Any)

Return type:

None

read_benchmark¶

zeropybench.read_benchmark(path, *, verbose=False)[source]¶

Reads a benchmark from a CSV or Parquet file.

The function automatically detects the file format based on the extension and reads the metadata (repeat, min_duration_per_repeat) stored in the file.

Parameters:

path (Path | str) – The path to the CSV or Parquet file.
verbose (bool) – If True, set logging level to INFO. If False, set logging level to WARNING.

Returns:

A Benchmark instance with the data and metadata from the file.

Raises:

ValueError – If the file extension is not .csv or .parquet.

Return type:

Benchmark