stats.comparison.paired_comparison

stats.comparison.paired_comparison(a, b, *, lower_is_better=True)

Compare two entries on their shared periods with a paired t-test.

Aligns the two per-period score series on their common index, drops periods where either is missing, and reports the per-side means, the win count of the first entry, and a two-sided paired t-test — the shared-days comparison protocol for leaderboard entries with different coverage.

Parameters

Name	Type	Description	Default
a	pd.Series	Per-period scores of the first entry (e.g. daily MAE).	required
b	pd.Series	Per-period scores of the second entry.	required
lower_is_better	bool	Whether a strictly smaller score is a win for the first entry. Defaults to True.	`True`

Returns

Name	Type	Description
	PairedComparison	A `PairedComparison` over the shared periods.

Raises

Name	Type	Description
	TypeError	When `a` or `b` is not a `pd.Series`.
	ValueError	When fewer than two shared periods remain after alignment.

Examples

import pandas as pd
from spotforecast2_safe.stats.comparison import paired_comparison

days = pd.date_range("2026-06-10", periods=6, freq="D")
a = pd.Series([500.0, 520.0, 480.0, 510.0, 490.0, 505.0], index=days)
b = a + 40.0  # consistently worse
result = paired_comparison(a, b)
print(result.n, result.wins_a, round(result.p_value, 6))
assert result.wins_a == 6

6 6 0.0

/Users/bartz/workspace/spotforecast2-safe/.venv/lib/python3.13/site-packages/scipy/stats/_axis_nan_policy.py:430: RuntimeWarning: Precision loss occurred in moment calculation due to catastrophic cancellation. This occurs when the data are nearly identical. Results may be unreliable.
  return hypotest_fun_in(*args, **kwds)