evaluation

Evaluation utilities for LOS estimator.

This module provides: - EvaluationResult: container for metric arrays and helpers to access them. - WindowDataPackage: packages windowed train/test predictions and true values. - Evaluator: computes metrics over windowed results and can save them to CSV.

class los_estimator.evaluation.EvaluationResult(distros: List[str], metric_names: List[str], train: numpy.typing.NDArray.typing.Any | None = None, test: numpy.typing.NDArray.typing.Any | None = None)

Bases: object

Container for evaluation metric results.

train

3D array (n_distros, n_windows, n_metrics) for training metrics.

Type:

Optional[NDArray[Any]]

test

3D array (n_distros, n_windows, n_metrics) for test metrics.

Type:

Optional[NDArray[Any]]

distros

ordered list of distribution names.

Type:

List[str]

metric_names

ordered list of metric names.

Type:

List[str]

get_dfs() → pandas.DataFrame

Return a DataFrame with train/test mean and median per distribution and metric.

class los_estimator.evaluation.Evaluator(all_fit_results: MultiSeriesFitResults, series_data: SeriesData)

Bases: object

Compute metrics over windowed predictions for multiple distributions.

Parameters:

• all_fit_results (MultiSeriesFitResults) – Results container for multiple series fits.

• series_data (SeriesData) – SeriesData instance with ground-truth and window definitions.

calculate_metrics() → EvaluationResult

Compute the configured metrics over all distributions and windows.

Returns:

EvaluationResult containing train and test metric arrays.

save_result(path: str) → Tuple[pandas.DataFrame, pandas.DataFrame]

Persist metrics to CSV files (metrics_train.csv and metrics_test.csv).

Returns:

(df_train, df_test) DataFrames written to disk.

class los_estimator.evaluation.WindowDataPackage(all_fit_results: MultiSeriesFitResults, series_data: SeriesData)

Bases: object

Pack windowed train/test predictions and ground-truth into a 2D object array for evaluation.

The data attribute is an array shaped (n_distros, n_windows) with tuples: (y_true_train, y_pred_train, y_true_test, y_pred_test, x_train, x_test, window_info)

build_package() → None

Construct the data array from all_fit_results and series_data.

iterate_index() → Iterator[Tuple[int, int]]

Yield (i_distro, i_window) tuples.