stats.comparison.significance_flips

stats.comparison.significance_flips(first, second, *, alpha=0.05)

Pairs whose significance verdict differs between two p-value matrices.

Two panels over the same campaign, differing only in which entries and periods they admit, do not always agree on which differences are resolved. Comparing their verdicts pair by pair separates the findings that survive either choice from the borderline ones that track whichever periods a panel happens to retain. Only entries present in both matrices are compared.

This is pure: no logging, no plotting, no mutation.

Parameters

Name	Type	Description	Default
first	pd.DataFrame	Square symmetric p-value matrix, e.g. from `pairwise_paired_t`.	required
second	pd.DataFrame	Square symmetric p-value matrix over an overlapping set of entries.	required
alpha	float	Significance threshold applied to both. Defaults to 0.05.	`0.05`

Returns

Name	Type	Description
	pd.DataFrame	pd.DataFrame: One row per shared pair whose verdict differs, with
	pd.DataFrame	columns ["entry_a", "entry_b", "p_first", "p_second", \| \| \| [pd](`pandas`).[DataFrame](`pandas.DataFrame`) \| "significant_in"], where `significant_in` names the matrix
	pd.DataFrame	that resolved the pair. Empty when the two agree everywhere.
	pd.DataFrame	The frame carries `attrs["n_shared_pairs"]`, the number of
	pd.DataFrame	pairs compared, which is the denominator such a count is
	pd.DataFrame	reported against.

Raises

Name	Type	Description
	ValueError	When `alpha` is outside `(0, 1)`, or the two matrices share fewer than two entries.

Examples

import pandas as pd
from spotforecast2_safe.stats.comparison import significance_flips

entries = ["alpha", "beta", "gamma"]
first = pd.DataFrame(1.0, index=entries, columns=entries)
second = first.copy()
first.loc["alpha", "beta"] = first.loc["beta", "alpha"] = 0.01
second.loc["alpha", "beta"] = second.loc["beta", "alpha"] = 0.30

flips = significance_flips(first, second)
print(flips.to_string(index=False))
print("compared:", flips.attrs["n_shared_pairs"])

entry_a entry_b  p_first  p_second significant_in
  alpha    beta     0.01       0.3          first
compared: 3