Datasets

This page describes the data of each part of the benchmark. Data downloads all the data on first use and makes sure that each file matches its SHA-256 checksum. Evaluation methodology explains which points each run keeps.

ODDS

The odds part uses 17 datasets from the ODDS repository (Outlier Detection DataSets). The column “Points used” gives the number of points after the limit of 5,000, and “Points evaluated” the number after the warm-up. Data.get_stream gives a stream exactly as the benchmark uses it, for example Data().get_stream("cardio.mat", random_state=0, max_points=5000, scale=True).

Dataset	Points	Points used	Points evaluated	Features	Anomalies (%)
arrhythmia	452	452	339	274	14.6
cardio	1,831	1,831	1,575	21	9.6
glass	214	214	161	9	4.2
ionosphere	351	351	264	33	35.9
letter	1,600	1,600	1,344	32	6.2
lympho	148	148	111	18	4.1
mnist	7,603	5,000	4,744	100	9.2
musk	3,062	3,062	2,806	166	3.2
optdigits	5,216	5,000	4,744	64	2.9
pendigits	6,870	5,000	4,744	16	2.3
pima	768	768	576	8	34.9
satellite	6,435	5,000	4,744	36	31.6
satimage-2	5,803	5,000	4,744	36	1.2
shuttle	49,097	5,000	4,744	9	7.2
vertebral	240	240	180	6	12.5
vowels	1,456	1,456	1,200	12	3.4
wbc	378	378	284	30	5.6

Streams with drift

The drift part uses 8 streams from three sources:

• Data.datasets("kddcup99") gives the http connections of logged in users from the 10% file of KDD Cup 99, as scikit-learn’s fetch_kddcup99 does. Its 3 features are the logarithms of the duration and of the bytes in each direction. The attacks are the anomalies.

• Data.datasets("insects") gives the 4 INSECTS streams [SdRMB20] as the ARCUS paper [YLLL22] made them. The anomalies are the points of the southern house mosquito, 1.2% of each stream.

• Data.datasets("smd") gives the test halves of the 28 machines of SMD [SZN+19]. The benchmark uses the first machine of each of the 3 groups of machines. The first half of each machine has no labels, so the benchmark does not use it.

The column “Points used” gives the number of points after the thinning.

Stream	Points	Points used	Features	Anomalies (%)
kddcup99/http.csv	58,724	9,788	3	3.7
insects/abrupt.csv	44,569	8,914	33	1.2
insects/incremental.csv	48,086	9,618	33	1.3
insects/incremental_gradual.csv	20,367	6,789	33	1.1
insects/incremental_reoccurring.csv	67,455	9,637	33	1.1
smd/machine-1-1.txt	28,479	9,493	38	9.4
smd/machine-2-1.txt	23,694	7,898	38	4.9
smd/machine-3-1.txt	28,700	9,567	38	1.1

NAB and UCR

NAB has 58 series, and 52 of them have anomalies. Each point in a window of NAB has the label 1. 14 series are seasonal: the correlation of their values with the values one day earlier is at least 0.5. Their period is one day, which is 24, 48 or 288 points, because these series have a point every hour, every half hour or every 5 minutes.

The UCR archive has 250 series with 19.4 million points. The benchmark uses the 84 series with at most 20,000 points, so that a run stays short. The archive has no license that lets PySAD copy it, so Data downloads it from the site of its authors.