Skip to content

primitives

primitives ¤

Spec-driven primitives: the correct way to do the five recurring operations.

Every detector in ts-shape boils down to a few operations on a signal. Done naively, each has a classic failure on industrial data:

================= ========================================================== Operation Naive failure ================= ========================================================== intervals closes at the last sample: zero-length on change-encoded data, one period short on sampled data, bridges outages increments last - first loses production at a counter reset per_window resample().mean() weights samples, not time align merge_asof(nearest) looks ahead to future setpoints windows fixed Timedelta windows break on DST days / night shifts ================= ==========================================================

The functions here take a :class:~ts_shape.signals.spec.SignalSpec and do the right thing for that signal's kind and encoding. They are pure functions on pandas objects (no hidden state), so they can later get alternative backends without changing their contract.

signal_series ¤

signal_series(
    dataframe: DataFrame,
    spec: SignalSpec,
    *,
    time_column: str = "systime",
    uuid_column: str = "uuid"
) -> pd.Series

The signal's values as a time-indexed Series (sorted, NaN/NaT dropped).

Rows of other uuids are ignored, so a multi-signal frame can be passed (uuids are compared as text). Duplicate timestamps keep the last value. Nullable and pyarrow dtypes are converted to NumPy dtypes.

Parameters:

Name Type Description Default
dataframe DataFrame

Long-format frame.

 required
spec SignalSpec

The signal's spec (uuid and value_column).

 required
time_column str

Timestamp column.

 'systime'
uuid_column str

Signal identifier column; ignored when absent.

 'uuid'

Returns:

Type Description
Series

Values indexed by time, named after spec.uuid.

Raises:

Type Description
ColumnNotFoundError

When time_column or spec.value_column is missing.

intervals ¤

intervals(
    dataframe: DataFrame,
    spec: SignalSpec,
    *,
    data_end: Timestamp | None = None,
    time_column: str = "systime"
) -> pd.DataFrame

Constant-value intervals of a held signal, correctly closed.

  • A held value (state, counter, level) lasts until the next sample, for both encodings; an event signal's interval ends at its last sample.
  • A gap longer than spec.max_gap is "no data": the interval before it ends at its last sample and a new interval starts after the gap, even if the value is unchanged.
  • The final interval ends at data_end when given (e.g. "now" or the end of the analysis window), otherwise at its last sample.

Parameters:

Name Type Description Default
dataframe DataFrame

Long-format frame.

 required
spec SignalSpec

The signal's spec.

 required
data_end Timestamp | None

End of the last interval; a naive bound on tz-aware data is read in the data's timezone.

 None
time_column str

Timestamp column.

 'systime'

Returns:

Type Description
DataFrame

DataFrame with start, end, state, duration_seconds,
DataFrame

sample_count -- one row per interval, in time order.

Raises:

Type Description
ColumnNotFoundError

When time_column or spec.value_column is missing.

increments ¤

increments(
    dataframe: DataFrame,
    spec: SignalSpec,
    *,
    time_column: str = "systime"
) -> pd.Series

Per-reading increase of a counter, reset-aware.

Each value is the quantity produced since the previous reading, booked at the reading that reports it (the first reading contributes 0). With reset_policy="reset_to_zero" a drop is a reset and the post-reset value counts as new production; with "none" decreases are kept (negative increments).

Parameters:

Name Type Description Default
dataframe DataFrame

Long-format frame.

 required
spec SignalSpec

The counter's spec.

 required
time_column str

Timestamp column.

 'systime'

Returns:

Type Description
Series

Float increments indexed by reading time.

Raises:

Type Description
ValueError

If the spec is not a counter.
ColumnNotFoundError

When time_column or spec.value_column is missing.

per_window ¤

per_window(
    dataframe: DataFrame,
    spec: SignalSpec,
    window: str,
    *,
    data_end: Timestamp | None = None,
    time_column: str = "systime"
) -> pd.Series

The right per-window aggregate for the signal's kind.

  • counter -- quantity produced in each window (reset-aware increments summed per window; no production lost at window boundaries).
  • continuous / numeric or boolean state -- time-weighted mean, holding each value until the next sample and never across a gap longer than spec.max_gap. For a boolean state this is the fraction of the known time it was True: a missing reading (NA) is "unknown" until the next reading and counts neither as True nor as False.
  • event -- number of events per window.

Parameters:

Name Type Description Default
dataframe DataFrame

Long-format frame.

 required
spec SignalSpec

The signal's spec.

 required
window str

Window length as a pandas offset ("1h", "1D").

 required
data_end Timestamp | None

Hold the last value until this time; a naive bound on tz-aware data is read in the data's timezone.

 None
time_column str

Timestamp column.

 'systime'

Returns:

Type Description
Series

A float Series indexed by window start and named after spec.uuid
Series

(for every kind).

Raises:

Type Description
ValueError

For a non-numeric (e.g. string) state; use :func:intervals to get time per state instead.
ColumnNotFoundError

When time_column or spec.value_column is missing.

align ¤

align(
    left: DataFrame,
    left_spec: SignalSpec,
    right: DataFrame,
    right_spec: SignalSpec,
    *,
    time_column: str = "systime"
) -> pd.DataFrame

Pair each sample of left with the right value in force at that time.

The right signal is held backward (the last value at or before each left sample) and never across more than right_spec.max_gap: no look-ahead to a future setpoint, no stale value after an outage. The two time columns are brought to one unit first (ns, us, s and pyarrow timestamps mix freely); tz-aware right times are converted to the left timezone.

Parameters:

Name Type Description Default
left DataFrame

Frame with the left signal.

 required
left_spec SignalSpec

Spec of the left signal.

 required
right DataFrame

Frame with the right signal.

 required
right_spec SignalSpec

Spec of the right signal.

 required
time_column str

Timestamp column of both frames; also the name of the output time column.

 'systime'

Returns:

Type Description
DataFrame

DataFrame with <time_column>, <left uuid>, <right uuid>.
DataFrame

When both specs have the same uuid the value columns are
DataFrame

<uuid>_left and <uuid>_right.

Raises:

Type Description
ValueError

When one side is tz-aware and the other naive.
ColumnNotFoundError

When a frame lacks time_column or its value column.

windows ¤

windows(
    start: Timestamp | str,
    end: Timestamp | str,
    freq: str,
    *,
    tz: str | None = None
) -> pd.DataFrame

Calendar-correct analysis windows covering [start, end).

Window bounds follow the calendar in tz: a "1D" window on a DST day is 23 h or 25 h long, not 24 h.

Parameters:

Name Type Description Default
start Timestamp | str

The first window covers this time.

 required
end Timestamp | str

Exclusive end of the range.

 required
freq str

Window length as a pandas offset ("1D", "8h").

 required
tz str | None

Calendar timezone; naive bounds are localized in it, aware bounds converted. Without tz, a naive bound next to an aware one is read in the aware one's timezone.

 None

Returns:

Type Description
DataFrame

DataFrame with start, end, duration_seconds; empty when
DataFrame

start >= end.

shift_windows ¤

shift_windows(
    start: Timestamp | str,
    end: Timestamp | str,
    shift_definitions: (
        Mapping[str, tuple[str, str]] | None
    ) = None,
    *,
    tz: str | None = None
) -> pd.DataFrame

One row per (production day, shift) overlapping [start, end).

Overnight shifts (e.g. 22:00 -> 06:00) are a single window attributed to the day they start; wall-clock shift times are localized in tz so a shift spanning a DST change has its real length (7 h or 9 h).

Parameters:

Name Type Description Default
start Timestamp | str

Start of the range.

 required
end Timestamp | str

Exclusive end of the range.

 required
shift_definitions Mapping[str, tuple[str, str]] | None

{name: ("HH:MM", "HH:MM")}; default :data:ts_shape.utils.shifts.DEFAULT_SHIFTS.

 None
tz str | None

Timezone of the wall-clock shift times. Without it, an aware bound makes the shifts use that bound's timezone.

 None

Returns:

Type Description
DataFrame

DataFrame with production_day, shift, start, end,
DataFrame

duration_seconds; empty when start >= end.