outlier_detection

outlier_detection ¤

OutlierDetectionEvents ¤

OutlierDetectionEvents(
    dataframe: DataFrame,
    value_column: str,
    event_uuid: str = "quality:outlier",
    time_threshold: str = "5min",
    group_by_uuid: bool = True,
    *,
    signal_uuids: list[str] | None = None
)

Bases: Base

Processes time series data to detect outliers based on specified statistical methods.

Example

import ts_shape df = ts_shape.make_timeseries(["temp"], n_outliers=3) out = ts_shape.OutlierDetectionEvents(df, "value_double").detect_outliers_zscore() out[["systime", "value_double", "severity"]].round( ... {"value_double": 1, "severity": 1} ... ) systime value_double severity 0 2025-01-01 01:34:30 89.7 8.9 1 2025-01-01 03:21:00 88.8 9.6 2 2025-01-01 05:23:00 88.6 9.8 3 2025-01-01 07:54:30 96.4 3.1

Initializes the OutlierDetectionEvents with specific attributes for outlier detection.

Parameters:

Name	Type	Description	Default
`dataframe`	`DataFrame`	The input time series DataFrame.	required
`value_column`	`str`	The name of the column containing the values for outlier detection.	required
`event_uuid`	`str`	A UUID or identifier for detected outlier events.	`'quality:outlier'`
`time_threshold`	`str`	The time threshold to group close events together.	`'5min'`
`group_by_uuid`	`bool`	Compute the statistics (mean/std, quartiles, median/MAD, forest) and group events per `uuid` (default `True`), so a frame holding e.g. a temperature (~80) and a pressure (~2) is scored per signal. `False` pools every row into one population (one mean/std across all signals).	`True`
`signal_uuids`	`list[str] \| None`	Signals to analyze. `None` (default) analyzes every `uuid` in the frame.	`None`

detect_outliers_zscore ¤

detect_outliers_zscore(
    threshold: float = 3.0, include_singles: bool = True
) -> pd.DataFrame

Detects outliers using the Z-score method.

NaN values are ignored when computing the mean and standard deviation (one NaN used to turn every z-score into NaN, so nothing was found).

Parameters:

Name	Type	Description	Default
`threshold`	`float`	The Z-score threshold for detecting outliers.	`3.0`
`include_singles`	`bool`	Whether to include single outliers in the output. Default is True.	`True`

Returns:

Type	Description
`DataFrame`	pd.DataFrame: A DataFrame of detected outliers and grouped events.

detect_outliers_iqr ¤

detect_outliers_iqr(
    threshold: tuple = (1.5, 1.5),
    include_singles: bool = True,
) -> pd.DataFrame

Detects outliers using the IQR method.

Parameters:

Name	Type	Description	Default
`threshold`	`tuple`	The multipliers for the IQR range for detecting outliers (lower, upper).	`(1.5, 1.5)`
`include_singles`	`bool`	Whether to include single outliers in the output. Default is True.	`True`

Returns:

Type	Description
`DataFrame`	pd.DataFrame: A DataFrame of detected outliers and grouped events.

detect_outliers_mad ¤

detect_outliers_mad(
    threshold: float = 3.5, include_singles: bool = True
) -> pd.DataFrame

Detects outliers using the Median Absolute Deviation (MAD) method. This method is more robust to outliers than z-score.

When more than half of the values are identical the MAD is 0. The modified z-score then falls back to the mean absolute deviation ((x - median) / (1.253314 * MeanAD)); if that is 0 too, no point is an outlier. (Previously MAD was replaced by machine epsilon, which flagged every value that differed from the median at all.)

Parameters:

Name	Type	Description	Default
`threshold`	`float`	The MAD threshold for detecting outliers. Default is 3.5.	`3.5`
`include_singles`	`bool`	Whether to include single outliers in the output. Default is True.	`True`

Returns:

Type	Description
`DataFrame`	pd.DataFrame: A DataFrame of detected outliers and grouped events.

detect_outliers_isolation_forest ¤

detect_outliers_isolation_forest(
    contamination: float = 0.1,
    include_singles: bool = True,
    random_state: int | None = 42,
) -> pd.DataFrame

Detects outliers using sklearn's IsolationForest algorithm. Needs scikit-learn (pip install ts-shape[ml]), imported on the first call.

Rows with a NaN value are not scored.

Parameters:

Name	Type	Description	Default
`contamination`	`float`	The proportion of outliers in the dataset. Default is 0.1.	`0.1`
`include_singles`	`bool`	Whether to include single outliers in the output. Default is True.	`True`
`random_state`	`Optional[int]`	Random state for reproducibility. Default is 42.	`42`

Returns:

Type	Description
`DataFrame`	pd.DataFrame: A DataFrame of detected outliers and grouped events.

Raises:

Type	Description
`ImportError`	If scikit-learn is not installed.