PySAD

One pass. Every outlier.

PySAD is a Python library that finds anomalies (unusual data points) in data streams. It scores each point when the point arrives. It has 16 streaming detectors, calibrated alerts and streaming evaluation.

Get started View on GitHub

$ pip install pysad

from pysad.evaluation import AUROCMetric
from pysad.models import LODA
from pysad.utils import Data

model = LODA(random_state=0)
metric = AUROCMetric()

for x, label in Data().get_iterator("cardio.mat", shuffle=False):
    score = model.fit_score_partial(x)  # score x, then learn from it
    metric.update(label, score)

print(f"AUROC: {metric.get():.3f}")

AUROC: 0.946

A Robust Random Cut Forest scores three synthetic sensors one point at a time. Conformal calibration converts each score into the fraction of recent scores that it exceeds. It raises an alert when this calibrated value reaches 0.995.

A batch detector needs the whole dataset before it can score a point. A stream does not give you the whole dataset. Points arrive one at a time, and the data drifts (its distribution changes over time). You must decide about each point before the next point arrives. Every PySAD detector takes one point at a time, and most detectors keep a fixed amount of memory.

Getting started

Run your first streaming detector in ten minutes: score, measure and alert.

How-to guides

Steps for real tasks: choose a detector, stream your data, run PyOD models and write your own detector.

Concepts

How streaming detection works: the detector interface, prequential evaluation (score each point, then learn from it), calibration and memory.

Detectors

All 16 detectors, with the input they take, the memory they keep and the paper they implement.

Benchmarks

AUROC, AUPR and time per point for all 10 multivariate detectors across 17 labeled datasets.

API reference

Every public class, parameter and method.

Examples

Complete scripts that run end to end. The automatic tests run every one of them.

What PySAD contains

16

streaming detectors from published papers

17

labeled benchmark datasets

1

alert threshold for every detector, after conformal calibration

60+

publications that cite PySAD

• The detectors use robust statistics, random forests, random projections and other methods. Detectors lists each detector with its method, input and memory, and the benchmark compares the multivariate detectors on 17 datasets.

• The conformal calibrator converts each raw score into the fraction of recent scores that it exceeds. As a result, one alert threshold works for every detector.

• Streamers replay arrays and DataFrames as streams.

• Metrics update after every point. Evaluate a detector explains each metric.

• Preprocessors, random projections, score postprocessors and ensemblers work with every detector. Preprocessors change each point before the detector scores it. Postprocessors change the scores, for example to smooth them. Ensemblers combine the scores of several detectors.

• ReferenceWindowModel runs any PyOD detector on a sliding window (the most recent points).

PySAD is free software under the BSD 3-Clause License. If you use it in research, see Citing PySAD.

Tripod the tripodfish stands still on three fins and waits for what the current brings.