plots.evaluation

plots.evaluation

Campaign-level evaluation plots for forecasting entries over time.

Generalises the daily-metric timeline, hour-of-day error profile, day-ahead forecast overlay, backtest-fold layout, and residual diagnostics of a manuscript’s results section into a reusable, stateless API. All functions return a matplotlib.figure.Figure; the caller is responsible for saving and closing it. None of them call plt.show() nor mutate matplotlib.rcParams — styling and figure lifecycle stay with the caller (set matplotlib.use("Agg") before importing pyplot in headless environments).

Functions

Name Description
plot_backtest_folds Horizontal-bar layout of rolling-origin backtest folds.
plot_error_profile Mean-error-by-group profile for one or more entries (e.g. hour of day).
plot_forecast_overlay One or more forecasts overlaid against an optional actual series.
plot_metric_timeline Daily-metric timeline for one or more entries, with an optional range band.
plot_residual_diagnostics Two-panel residual diagnostics: histogram and normal QQ plot.

plot_backtest_folds

plots.evaluation.plot_backtest_folds(
    folds,
    *,
    train_color='#4C72B0',
    test_color='#DD8452',
    train_label='train',
    test_label='test',
    fold_label='fold',
    bar_height=0.7,
    xlabel='',
    legend_loc='lower right',
    ax=None,
    figsize=(5.5, 2.8),
)

Horizontal-bar layout of rolling-origin backtest folds.

folds is the DataFrame returned by spotforecast2_safe.splitter.TimeSeriesFold.split(y, as_pandas=True): one row per fold with iloc-style [start, end) positions. Each fold is drawn as a training bar followed by a test bar (test_start_with_gap/test_end_with_gap, which equal test_start/test_end when the fold has no gap), so the figure shows the splitter’s real output rather than re-derived arithmetic. Fold 1 is drawn at the top.

Parameters

Name Type Description Default
folds pd.DataFrame Fold table with columns fold, train_start, train_end, test_start_with_gap, and test_end_with_gap. required
train_color str Bar color for the training ranges. '#4C72B0'
test_color str Bar color for the test ranges. '#DD8452'
train_label str Legend label for the training bars. 'train'
test_label str Legend label for the test bars. 'test'
fold_label str Prefix of the y-axis tick labels; fold k is labelled f"{fold_label} {k + 1}". 'fold'
bar_height float Height of every bar. 0.7
xlabel str X-axis label (observation index). ''
legend_loc str loc argument forwarded to ax.legend. 'lower right'
ax Axes | None Existing axes to draw into. When given, no new figure is created and the function returns ax.figure. None
figsize tuple[float, float] Figure size used when ax is not given. (5.5, 2.8)

Returns

Name Type Description
Figure A matplotlib.figure.Figure.

Raises

Name Type Description
ValueError If folds is empty or lacks a required column.

Examples

import numpy as np
import pandas as pd
from spotforecast2.plots.evaluation import plot_backtest_folds
from spotforecast2_safe.splitter import TimeSeriesFold

idx = pd.date_range("2024-01-01", periods=224, freq="h", tz="UTC")
y = pd.Series(np.arange(224.0), index=idx)
cv = TimeSeriesFold(steps=24, initial_train_size=80, refit=False)
folds = cv.split(y, as_pandas=True)
fig = plot_backtest_folds(folds, xlabel="observation index")
print(type(fig).__name__)
╭─────────────────────────────── IgnoredArgumentWarning ───────────────────────────────╮
 Last window cannot be calculated because `window_size` is None.                      
                                                                                      
 Category : spotforecast2.exceptions.IgnoredArgumentWarning                           
 Location :                                                                           
 /Users/bartz/workspace/spotforecast2/.venv/lib/python3.13/site-packages/spotforecast 
 2_safe/splitter/split_ts_cv.py:496                                                   
 Suppress : warnings.simplefilter('ignore', category=IgnoredArgumentWarning)          
╰──────────────────────────────────────────────────────────────────────────────────────╯
Information of folds
--------------------
Number of observations used for initial training: 80
Number of observations used for backtesting: 144
    Number of folds: 6
    Number skipped folds: 0 
    Number of steps per fold: 24
    Number of steps to exclude between last observed data (last window) and predictions (gap): 0

Fold: 0
    Training:   2024-01-01 00:00:00+00:00 -- 2024-01-04 07:00:00+00:00  (n=80)
    Validation: 2024-01-04 08:00:00+00:00 -- 2024-01-05 07:00:00+00:00  (n=24)
Fold: 1
    Training:   No training in this fold
    Validation: 2024-01-05 08:00:00+00:00 -- 2024-01-06 07:00:00+00:00  (n=24)
Fold: 2
    Training:   No training in this fold
    Validation: 2024-01-06 08:00:00+00:00 -- 2024-01-07 07:00:00+00:00  (n=24)
Fold: 3
    Training:   No training in this fold
    Validation: 2024-01-07 08:00:00+00:00 -- 2024-01-08 07:00:00+00:00  (n=24)
Fold: 4
    Training:   No training in this fold
    Validation: 2024-01-08 08:00:00+00:00 -- 2024-01-09 07:00:00+00:00  (n=24)
Fold: 5
    Training:   No training in this fold
    Validation: 2024-01-09 08:00:00+00:00 -- 2024-01-10 07:00:00+00:00  (n=24)

Figure

plot_error_profile

plots.evaluation.plot_error_profile(
    profile,
    *,
    colors=None,
    linewidth=1.8,
    marker='o',
    marker_size=3.0,
    zero_line=True,
    zero_line_color='black',
    xlabel='',
    ylabel='mean error',
    xticks=None,
    legend_loc='lower center',
    ax=None,
    figsize=(6.3, 2.7),
)

Mean-error-by-group profile for one or more entries (e.g. hour of day).

profile has one row per grouping key (e.g. hour of day, 0-23) and one column per entry; each column is drawn as its own line.

Parameters

Name Type Description Default
profile pd.DataFrame DataFrame indexed by the grouping key, one column per entry. required
colors Mapping[str, str] | None Mapping of column name to line color. Columns missing from the mapping use matplotlib’s default color cycle. None
linewidth float Line width for every entry. 1.8
marker str Marker style for every entry. 'o'
marker_size float Marker size for every entry. 3.0
zero_line bool When True, draw a horizontal reference line at 0. True
zero_line_color str Color of the zero reference line. 'black'
xlabel str X-axis label. ''
ylabel str Y-axis label. 'mean error'
xticks Sequence[float] | None Explicit tick positions for the x axis. When None, the default matplotlib ticks are kept. None
legend_loc str loc argument forwarded to ax.legend. 'lower center'
ax Axes | None Existing axes to draw into. When given, no new figure is created and the function returns ax.figure. None
figsize tuple[float, float] Figure size used when ax is not given. (6.3, 2.7)

Returns

Name Type Description
Figure A matplotlib.figure.Figure.

Examples

import numpy as np
import pandas as pd
from spotforecast2.plots.evaluation import plot_error_profile

rng = np.random.default_rng(0)
profile = pd.DataFrame(
    {
        "forecaster": rng.normal(0, 100, 24),
        "baseline": rng.normal(-200, 150, 24),
    },
    index=range(24),
)
fig = plot_error_profile(profile, xticks=range(0, 24, 3))
print(type(fig).__name__)
Figure

plot_forecast_overlay

plots.evaluation.plot_forecast_overlay(
    forecasts,
    *,
    actual=None,
    actual_label='actual',
    actual_color='black',
    x='index',
    colors=None,
    linestyles=None,
    linewidth=1.8,
    xlabel='',
    ylabel='',
    xticks=None,
    legend_loc='lower center',
    ax=None,
    figsize=(6.3, 2.9),
)

One or more forecasts overlaid against an optional actual series.

actual, when given, is drawn first and slightly thicker than the forecast lines (linewidth + 0.2) so it reads as the reference trace. Each entry in forecasts is then drawn in insertion order.

Parameters

Name Type Description Default
forecasts Mapping[str, pd.Series] Mapping of entry name to forecast series. required
actual pd.Series | None Optional realised/ground-truth series, drawn first. None
actual_label str Legend label for actual. 'actual'
actual_color str Line color for actual. 'black'
x str "index" plots every series against its own index as-is; "hour" plots against index.hour (e.g. to overlay several day-ahead forecasts for the same target day on a 0-23 axis). 'index'
colors Mapping[str, str] | None Mapping of entry name to line color. Entries missing from the mapping use matplotlib’s default color cycle. None
linestyles Mapping[str, str] | None Mapping of entry name to a matplotlib linestyle (e.g. {"baseline": "--"}). Entries missing from the mapping are drawn solid. None
linewidth float Line width for the forecast lines. 1.8
xlabel str X-axis label. ''
ylabel str Y-axis label. ''
xticks Sequence[float] | None Explicit tick positions for the x axis. When None, the default matplotlib ticks are kept. None
legend_loc str loc argument forwarded to ax.legend. 'lower center'
ax Axes | None Existing axes to draw into. When given, no new figure is created and the function returns ax.figure. None
figsize tuple[float, float] Figure size used when ax is not given. (6.3, 2.9)

Returns

Name Type Description
Figure A matplotlib.figure.Figure.

Examples

import numpy as np
import pandas as pd
from spotforecast2.plots.evaluation import plot_forecast_overlay

idx = pd.date_range("2024-01-15", periods=24, freq="h", tz="UTC")
rng = np.random.default_rng(0)
actual = pd.Series(40_000 + rng.standard_normal(24) * 500, index=idx)
forecasts = {
    "forecaster": actual + rng.standard_normal(24) * 300,
    "baseline": actual + rng.standard_normal(24) * 800,
}
fig = plot_forecast_overlay(
    forecasts,
    actual=actual,
    x="hour",
    linestyles={"baseline": "--"},
    xticks=range(0, 24, 3),
)
print(type(fig).__name__)
Figure

plot_metric_timeline

plots.evaluation.plot_metric_timeline(
    series,
    *,
    band=None,
    band_label='range',
    band_color='0.9',
    colors=None,
    linewidth=1.8,
    ylabel='',
    legend_loc='upper right',
    ax=None,
    figsize=(6.3, 2.9),
)

Daily-metric timeline for one or more entries, with an optional range band.

Each series in series is plotted over its own (datetime) index after dropna(), so entries that join or leave the campaign at different dates are drawn only over their own scored period. When band is given, its row-wise min/max is drawn as a filled band underneath the lines (e.g. the range spanned by a group of reference entries).

Parameters

Name Type Description Default
series Mapping[str, pd.Series] Mapping of entry name to a metric series with a datetime index (e.g. daily MAE). required
band pd.DataFrame | None Optional DataFrame whose row-wise min and max are filled between (e.g. the daily metric of a group of entries). Its index must be datetime-like and comparable to the series indices. None
band_label str Legend label for the band. 'range'
band_color str Fill color for the band. '0.9'
colors Mapping[str, str] | None Mapping of entry name to line color. Entries missing from the mapping use matplotlib’s default color cycle. None
linewidth float Line width for every series. 1.8
ylabel str Y-axis label. ''
legend_loc str loc argument forwarded to ax.legend. 'upper right'
ax Axes | None Existing axes to draw into. When given, no new figure is created and the function returns ax.figure. None
figsize tuple[float, float] Figure size used when ax is not given. (6.3, 2.9)

Returns

Name Type Description
Figure A matplotlib.figure.Figure.

Examples

import numpy as np
import pandas as pd
from spotforecast2.plots.evaluation import plot_metric_timeline

idx = pd.date_range("2024-01-01", periods=14, freq="D")
rng = np.random.default_rng(0)
series = {
    "forecaster": pd.Series(500 + rng.standard_normal(14) * 20, index=idx),
    "baseline": pd.Series(700 + rng.standard_normal(14) * 20, index=idx),
}
band = pd.DataFrame(
    {f"peer_{i}": 550 + rng.standard_normal(14) * 40 for i in range(4)},
    index=idx,
)
fig = plot_metric_timeline(series, band=band, ylabel="daily MAE (MW)")
print(type(fig).__name__)
Figure

plot_residual_diagnostics

plots.evaluation.plot_residual_diagnostics(
    residuals,
    *,
    bins=30,
    color='#1F4E79',
    fit_color='0.5',
    alpha=0.85,
    hist_xlabel='residual',
    hist_ylabel='density',
    qq_xlabel='theoretical quantile',
    qq_ylabel='empirical quantile',
    figsize=(6.3, 2.6),
)

Two-panel residual diagnostics: histogram and normal QQ plot.

The left panel shows the residual histogram (density scale) with an overlaid normal density fitted to the empirical mean and standard deviation. The right panel is a QQ plot of the sorted residuals against standard-normal quantiles with the fitted reference line. Like the other two-panel figures in this package, the function always creates its own figure and takes no ax argument.

Parameters

Name Type Description Default
residuals pd.Series | np.ndarray Residual values; NaN entries are dropped. required
bins int Number of histogram bins. 30
color str Color of the histogram bars and the QQ markers. '#1F4E79'
fit_color str Color of the fitted normal density and QQ reference line. '0.5'
alpha float Alpha of the histogram bars. 0.85
hist_xlabel str X-axis label of the histogram panel. 'residual'
hist_ylabel str Y-axis label of the histogram panel. 'density'
qq_xlabel str X-axis label of the QQ panel. 'theoretical quantile'
qq_ylabel str Y-axis label of the QQ panel. 'empirical quantile'
figsize tuple[float, float] Figure size. (6.3, 2.6)

Returns

Name Type Description
Figure A matplotlib.figure.Figure with two axes.

Raises

Name Type Description
ValueError If fewer than two finite residuals remain after dropping NaN values, or if the residuals have zero variance.

Examples

import numpy as np
from spotforecast2.plots.evaluation import plot_residual_diagnostics

rng = np.random.default_rng(0)
residuals = rng.normal(0.0, 0.5, size=500)
fig = plot_residual_diagnostics(residuals)
print(type(fig).__name__, len(fig.axes))
Figure 2