preprocessing.outlier.manual_outlier_removal

preprocessing.outlier.manual_outlier_removal(
    data,
    column,
    *,
    lower_threshold=None,
    upper_threshold=None,
    verbose=False,
)

Mark values of column outside the given thresholds as NaN.

Pure: data is never modified. With both thresholds None the returned frame IS data (no copy) and mask is all-False, mirroring the noop fast path of :func:~spotforecast2_safe.preprocessing.target_corruption.apply_target_corruption_policy. Otherwise a new frame is returned; the count is int(mask.sum()).

Parameters

Name	Type	Description	Default
data	pd.DataFrame	The input dataset.	required
column	str	The column name in which to perform manual outlier removal.	required
lower_threshold	float \| None	The lower threshold below which values are considered outliers. If None, no lower threshold is applied.	`None`
upper_threshold	float \| None	The upper threshold above which values are considered outliers. If None, no upper threshold is applied.	`None`
verbose	bool	Whether to print additional information.	`False`

Returns

Name	Type	Description
	tuple[pd.DataFrame, pd.Series]	tuple[pd.DataFrame, pd.Series]: `(cleaned, mask)`. `cleaned` is a new frame in which every cell `mask` marks `True` in `column` is NaN (or `data` itself when both thresholds are `None`). `mask` is a boolean Series over `data.index` for `column` only, `True` == outlier; the count is `int(mask.sum())`.

Raises

Name	Type	Description
	KeyError	If `column` is not a column of `data` and at least one threshold is given (the noop fast path never touches `data`).

Examples

import numpy as np
import pandas as pd

from spotforecast2_safe.preprocessing.outlier import manual_outlier_removal

rng = np.random.default_rng(0)
# 20 normal values with two injected boundary violations
values = np.concatenate([rng.uniform(low=100.0, high=600.0, size=20), [10.0, 800.0]])
data = pd.DataFrame({"ABC": values})

cleaned, mask = manual_outlier_removal(
    data,
    column="ABC",
    lower_threshold=50,
    upper_threshold=700,
    verbose=True,
)
n_outliers = int(mask.sum())
print(f"Outliers removed: {n_outliers}")
assert n_outliers >= 2, "Expected the two injected boundary violations to be removed"
assert cleaned["ABC"].isna().sum() == n_outliers
assert data["ABC"].notna().all(), "The input frame is never modified"

Manually marked 2 values > 700 or < 50 as outliers in ABC.
Outliers removed: 2