signal_quality

signal_quality ¤

Signal quality: is the data itself trustworthy?

Finds missing data (gaps longer than the expected period), measures sampling regularity, flags out-of-range values, and reports data completeness per calendar window (real window lengths on DST days).

SignalQualityEvents ¤

SignalQualityEvents(
    dataframe: DataFrame,
    signal_uuid: str,
    *,
    event_uuid: str = "quality:signal",
    value_column: str = "value_double",
    time_column: str = "systime"
)

Bases: Base

Quality: Signal Quality Monitoring

Detect data quality issues in a numeric signal: missing data, irregular sampling, out-of-range values, and data completeness.

Methods: - detect_missing_data: Find gaps exceeding expected sampling frequency. - sampling_regularity: Inter-sample interval statistics per window. - detect_out_of_range: Flag values outside physical/expected bounds. - data_completeness: Percentage of expected samples received per window.

Example

A 1 s signal with an 8 s gap and one implausible reading:

import pandas as pd from ts_shape import SignalQualityEvents df = pd.DataFrame({ ... "systime": pd.to_datetime([ ... "2025-01-01 00:00:00", "2025-01-01 00:00:01", "2025-01-01 00:00:02", ... "2025-01-01 00:00:10", "2025-01-01 00:00:11"]), ... "uuid": "temp", ... "value_double": [20.0, 20.5, 21.0, 95.0, 21.0], ... }) quality = SignalQualityEvents(df, "temp") quality.detect_missing_data(expected_interval="1s")[["start", "end", "duration_seconds"]] start end duration_seconds 0 2025-01-01 00:00:02 2025-01-01 00:00:10 8.0 quality.detect_out_of_range(0.0, 50.0)[["start", "end", "duration_seconds"]] start end duration_seconds 0 2025-01-01 00:00:10 2025-01-01 00:00:11 1.0

detect_missing_data ¤

detect_missing_data(
    expected_interval: str = "1s",
    tolerance_factor: float = 2.0,
) -> pd.DataFrame

Find gaps where time between samples exceeds expected frequency.

Parameters:

Name	Type	Description	Default
`expected_interval`	`str`	Expected sampling interval (e.g. '1s', '100ms').	`'1s'`
`tolerance_factor`	`float`	Multiplier on expected_interval to trigger a gap.	`2.0`

Returns:

Type	Description
`DataFrame`	DataFrame with columns: start, end, duration_seconds, uuid,
`DataFrame`	source_uuid, gap_duration, expected_samples_missing.
`DataFrame`	`gap_duration` is a Timedelta; `duration_seconds` is the
`DataFrame`	same span in float seconds.

sampling_regularity ¤

sampling_regularity(window: str = '1h') -> pd.DataFrame

Inter-sample interval statistics per window.

Parameters:

Name	Type	Description	Default
`window`	`str`	Resample window.	`'1h'`

Returns:

Type	Description
`DataFrame`	DataFrame with columns: start, end, duration_seconds, uuid,
`DataFrame`	source_uuid, mean_interval, std_interval, min_interval,
`DataFrame`	max_interval, regularity_score. `end` is `start` plus one `window`.

detect_out_of_range ¤

detect_out_of_range(
    min_value: float,
    max_value: float,
    *,
    interval_end: str = "next_change"
) -> pd.DataFrame

Flag intervals where signal is outside expected range.

Parameters:

Name	Type	Description	Default
`min_value`	`float`	Minimum acceptable value.	required
`max_value`	`float`	Maximum acceptable value.	required
`interval_end`	`str`	`"next_change"` (default) ends an interval at the next sample (the transition), so change-encoded signals -- one row per change -- report the real time out of range. `"last_sample"` ends it at its last flagged sample. See :func:`ts_shape.utils.intervals.interval_end_times`.	`'next_change'`

Returns:

Type	Description
`DataFrame`	DataFrame with columns: start, end, duration_seconds, uuid,
`DataFrame`	source_uuid, duration, min_observed, max_observed, direction.
`DataFrame`	`duration` is a Timedelta; `duration_seconds` is the same
`DataFrame`	span in float seconds.

data_completeness ¤

data_completeness(
    window: str = "1h", expected_interval: str = "1s"
) -> pd.DataFrame

Percentage of expected samples actually received per window.

Parameters:

Name	Type	Description	Default
`window`	`str`	Resample window.	`'1h'`
`expected_interval`	`str`	Expected sampling interval.	`'1s'`

Returns:

Type	Description
`DataFrame`	DataFrame with columns: start, end, duration_seconds, uuid,
`DataFrame`	source_uuid, expected_count, actual_count, completeness_pct.
`DataFrame`	`end` is `start` plus one `window`.