stats.comparison.rank_concordance

stats.comparison.rank_concordance(ranks, *, reference=None)

Kendall’s tau between a reference ranking and every other column.

Quantifies how stable a leaderboard is across metrics: a tau of 1.0 means a metric reproduces the reference ranking exactly.

Parameters

Name	Type	Description	Default
ranks	pd.DataFrame	Frame of ranks (or scores), one column per metric — typically the output of `rank_table`.	required
reference	str \| None	Column the others are compared against. Defaults to None, which uses the first column.	`None`

Returns

Name	Type	Description
	pd.Series	pd.Series: Kendall’s tau per non-reference column, in column
	pd.Series	order, name `"kendall_tau"`.

Raises

Name	Type	Description
	ValueError	When `ranks` has fewer than two columns or `reference` is not a column.

Examples

import pandas as pd
from spotforecast2_safe.stats.comparison import rank_concordance

ranks = pd.DataFrame(
    {"mae": [1, 2, 3, 4], "rmse": [1, 2, 3, 4], "mape": [4, 3, 2, 1]},
    index=["a", "b", "c", "d"],
)
tau = rank_concordance(ranks, reference="mae")
print(tau.to_string())
assert tau["rmse"] == 1.0 and tau["mape"] == -1.0

rmse    1.0
mape   -1.0