Model Persistence Guide

Overview

Fitted forecasters are saved to disk with joblib, following the scikit-learn persistence conventions. Two levels are available. The multi-task pipeline (LazyTask, DefaultsTask, the tuning tasks) saves its models automatically, and PredictTask loads them again without retraining. For custom workflows, the helpers in spotforecast2_safe.manager.persistence save and load individual forecasters.

The former functional entry points n2n_predict and n2n_predict_with_covariates were removed in spotforecast2-safe 29.0.0 and are no longer supported.

Train once, predict many times

With auto_save_models=True (the default), every training task writes one file per target to get_cache_home(cache_home)/models/<data_frame_name>/, named <data_frame_name>_<target>_<task>_<UTC timestamp>.joblib. PredictTask loads the most recent model per target and forecasts without fitting.

from spotforecast2_safe.configurator.config_multi import ConfigMulti
from spotforecast2_safe.multitask import LazyTask
from spotforecast2_safe.multitask.predict import PredictTask, execute_predict

cfg = ConfigMulti(
    data_frame_name="demo",
    targets=["total_load_actual"],
    predict_size=24,
    auto_save_models=True,  # default
    number_folds=2,
)

# First run: train and save.
lazy = LazyTask(cfg, dataframe=df)
lazy.prepare_data()
lazy.run()

# Later runs: load the saved models and forecast, no training.
predict_task = PredictTask(cfg, dataframe=df)
predict_task.prepare_data()
result = execute_predict(predict_task)
print(result["future_pred"].head())

PredictTask.run() and execute_predict() accept two filters:

Parameter	Default	Meaning
task_name	None	Load only models from one source task ("lazy", "defaults", "optuna", "spotoptim"). None takes the most recent model of any task.
max_age_days	None	Ignore models older than this many days. None accepts any age.

If no saved model is found, PredictTask raises RuntimeError instead of training silently. To retrain, run a training task again: the new files carry a newer timestamp and are preferred from then on. Set auto_save_models=False for runs that must not write to disk, for example in tests.

Helper functions

The functions in spotforecast2_safe.manager.persistence work on plain dictionaries of fitted forecasters, keyed by target name, and use the file name forecaster_<target>.joblib.

from spotforecast2_safe.manager.persistence import (
    ensure_model_dir,
    get_model_filepath,
    load_forecasters,
    model_directory_exists,
    save_forecaster,
    save_forecasters,
)

model_dir = ensure_model_dir("./my_models")        # created if missing
path = get_model_filepath(model_dir, "power")       # my_models/forecaster_power.joblib

saved = save_forecasters({"power": forecaster}, model_dir)   # {target: Path}
save_forecaster(forecaster, model_dir, target="power", task_name="task_1_lazy")
# -> my_models/task_1_lazy_power.joblib

if model_directory_exists(model_dir):
    forecasters, missing = load_forecasters(["power", "wind"], model_dir)
    # forecasters: the models that were found
    # missing: targets without a model file, to be trained first

load_forecasters reports a target as missing only when its file does not exist. A file that exists but cannot be deserialised raises OSError, so a corrupt model is never mistaken for a missing one.

Sample weights and pickling

Forecasters trained with sample weights keep their weight function. It must be picklable, so use WeightFunction rather than a lambda or a closure:

import pickle

import pandas as pd
from spotforecast2_safe.preprocessing.imputation import WeightFunction

weights = pd.Series([1.0, 0.9, 0.8], index=[0, 1, 2])
weight_func = WeightFunction(weights)

weight_func(pd.Index([0, 1]))            # array([1. , 0.9])
restored = pickle.loads(pickle.dumps(weight_func))
restored(pd.Index([0, 1]))               # same weights after a round trip

A window whose weights are all zero returns None, which the forecaster treats as “no sample weights”.

Cache location

get_cache_home() resolves the cache directory: an explicit cache_home argument (also the cache_home field of the configuration) wins, otherwise the SPOTFORECAST2_CACHE environment variable, otherwise ~/.spotforecast2_cache.

import os

from spotforecast2_safe.data.fetch_data import get_cache_home

print(get_cache_home())                  # ~/.spotforecast2_cache by default
os.environ["SPOTFORECAST2_CACHE"] = "/data/sf2_cache"
print(get_cache_home())                  # /data/sf2_cache

Tests

The examples of this page are covered by tests/test_docs_model_persistence_examples.py:

uv run pytest tests/test_docs_model_persistence_examples.py -v