Photo by Amos Nachoum

The UCR Matrix Profile Page

Funded by NSF  IIS 1161997 II, IIS 1510741, CNS 1544969

observations of the magnetosphere collected by the Cassini spacecraft in orbit around Saturn.. ..in this case, the best-performing method was the Matrix Profile.. Kiri L. Wagstaf et. al. NASA JPL.2020
For Gear Anomaly Detection .. compared to (existing) methods, the Matrix Profile is more reliable; compared to an AI method the results are equally reliable, require far less data, and is free of parameter-intensive models.  Hurwitz and Hood, DEVCOM Army Research Laboratory (2022)
(working with animal nervous system) we decided to use matrix profile analysis, since it is robust, scalable, computationally efficient, largely parameter free, and capable of identifying both recurring motives and anomalies in the data. Athira et al 2022 [link]
(for an industrial IoT problem) Matrix Profiles perform well with almost no parameterisation needed. Anton et al ICDM 2018.
matrix profiling are methods suitable for the automated assessment of the signal quality of acceleration data measured from the skin in the neck and groin using laser Doppler velocimetry. Seoni et al. Front. Physiol (2022) [link]
While there will never be a mathematical silver bullet, we have discovered that the Matrix Profile, a novel algorithm developed by the Keogh research group at UC-Riverside, is a powerful tool. (full post). Andrew Van Benschoten, lead engineer at Target.
We found that all these similarity or randomness measures can be estimated with variants of the highly efficient Matrix Profile (MP) algorithm. Wang et a; Information Technology.
If anybody has ever asked you to analyze time series data and to look for new insights then (the Matrix Profile) is definitely the open source tool that you'll want to add to your arsenal Sean Law, Ameritrade. NABD 2019.
The MatrixProfile technique  is the state-of-the-art anomaly detection technique for continuous time series.  Bart Goethals et. al. (ECML-PKDD 2019).
Matrix Profile is the state-of-the-art similarity-based outlier detection methodChristian Jensen et. al. IJCAI-19
 It turns out that the information provided by the matrix profile can efficiently address many if not most problems in basic time series analysis. Steve Huntsman
My favorite use of the Matrix Profile in 2022 Having demonstrated that motif identification using matrix profiling is a potent method for identifying the basic behavioral building blocks of Ciona behavior.. The main advantages of matrix profiling are that it is robust, scalable, computationally efficient, and largely parameter free Chatzigeorgiou 2022.
(see the bottom of the page for more quotes)



The Matrix Profile (and the algorithms to compute it: STAMP, STAMPI, STOMP, SCRIMP, SCRIMP++, SWAMP, DAMP, SCAMP, GPU-STOMP, TranSCRIMP, TranSCAMP and TranSCAMPfpga), has the potential to revolutionize time series data mining because of its generality, versatility, simplicity and scalability.  In particular it has implications for time series motif discovery, time series joins, shapelet discovery (classification), density estimation, semantic segmentation, visualization, rule discovery, clustering etc (note, for pure similarity search, we suggest you see MASS for Euclidean Distance, and the UCR Suite for DTW)

News:


Our overarching claim abuot the Matrix Profile has three parts:

1) Given only the Matrix Profile, most time series data mining tasks are trivial.
2) The Matrix Profile can be computed very efficiently.
3) Algorithms that are built on top the Matrix Profile inherit all its desirable properties. For example, we can use the Matrix Profile to find time series motifs. Because the Matrix Profile can be incrementally computed, we have the first incremental time series motif discovery algorithm. Because the Matrix Profile can be computed in an anytime fashion, we have the first anytime time series motif discovery algorithm. Because the Matrix Profile can leverage GPUs, we have..
 
The advantages of using the Matrix Profile (over hashing, indexing, brute forcing a dimensionality reduced representation etc.) for most time series data mining tasks include:
  1. It is exact: For motif discovery, discord discovery, time series joins etc., the Matrix Profile based methods provide no false positives or false dismissals.
  2. It is simple and parameter-free: In contrast, the more general spatial access method algorithms typically require building and tuning spatial access methods and/or hash function.
  3. It is space efficient: Matrix Profile construction algorithms requires an inconsequential space overhead, just linear in the time series length with a small constant factor, allowing massive datasets to be processed in main memory.
  4. It allows anytime algorithms: While our exact algorithms are extremely scalable, for extremely large datasets we can compute the Matrix Profile in an anytime fashion, allowing ultra-fast approximate solutions.
  5. It is incrementally maintainable: Having computed the Matrix Profile for a dataset, we can incrementally update it very efficiently. In many domains this means we can effectively maintain exact joins/motifs/discords on streaming data forever.
  6. It does not require the user to set similarity/distance thresholds: For time series joins, the Matrix Profile provides full joins, eliminating the need to specify a similarity threshold, which is an unintuitive task for time series.
  7. It can leverage hardware: Matrix Profile construction is embarrassingly parallelizable, both on multicore processors and in distributed systems.
  8. It has time complexity that is constant in subsequence length: This is a very unusual and desirable property; all known time series join/motif/discord algorithms scale poorly as the subsequence length grows. In contrast, we have shown time series joins/motifs with subsequences lengths up to 100,000, at least two orders of magnitude longer than any other work we are aware of.
  9. It can be constructed in deterministic time: All join/motif/discord algorithms we are aware of can radically different times to finish on two (even slightly) different datasets. In contrast, given only the length of the time series, we can precisely predict in advance how long it will take to compute the Matrix Profile
  10. It can handle missing data: Even in the presence of missing data, we can provide answers which are guaranteed to have no false negatives.

Given all these features, the Matrix Profile has implications for many, perhaps most, time series data mining tasks.

Papers:
Tutorials:
 

The First Matrix Profile Tutorial


Part 1: PPT or PDF
Part 2: PPT or PDF

MP Tutorial


100 Time Series Data Mining Questions (with Answers!)
Just the PDF
Code and Data in a ZIP

This document contains one hundred simple time series questions such as Have we ever seen a pattern that looks just like this? and Is there any pattern that is common to these two time series?, with worked examples of the answers, with all (simple!) code and data.

100 questions

      


Code by the Community
: Here we list some MP implementations by others. Naturally we neither take credit (or blame) for any of this work.  Python, R, Golang.
Sean Law created an open sourced a distributed and multicore Python library. Check out the Matrix Profile Foundation.
Parallel implementations of the Matrix Profile SCRIMP++ algorithm for high performance computing clusters based on MPI.

Zach Zimmerman has released SCAMP, the fastest way to compute matrix profiles on CPU and GPU. It has a python module and command line tool usable on windows, mac, and linux, with or without a GPU.
The company shapelets.io has a Khiva library of efficient algorithms to analyze time series with both GPU and CPU support.
Sean Law has released STUMPY, a powerful and scalable Python Matrix Profile github library. Sean is completely independent from the UCR team, but has been a great advocate of the MP.


Code by the UCR Team:

GPU/CPU Code: This is the SCAMP source code on GitHub.

Matlab Code: Version 3.0    
(Faster code is avaible, see 100 Questions above. But we strongly recommend you start with the below).

Please note this code is not STAMP or STOMP, but SCRIMP++ (which appears in Matrix Profile XI), which is as fast as STOMP, but also an anytime algorithm. The code is wrapped inside a simple Matlab GUI (that adds a lot of time overhead), to allow non-specailists to interact with their data. If you are writing a paper, please do not comparing timing results to this version, it would not be fair to us (contact us for the faster, but less user friendly code).

First read this, then download this data (some penguin data) and download the code into your Matlab path:
At the command line, type...

>> load MP_first_test_penguin_sample
>> [matrixProfile, profileIndex, motifIndex, discordIndex] = interactiveMatrixProfileVer3_website(smooth(penguin_sample,10) ,800);

As you can see from the image below, even though the data has 109,043 datapoints, and the subsequence is high dimensional (800 datapoints), in about a second we have already found some very interesting motifs:
The first is a valley happens during a dive.   

Happy motif hunting, from the Matrix Profile team
.

GUI  of Stamp Tool

What people are saying about the Matrix Profile...
Multimodal Satellite Image Time Series Analysis Using GAN-Based Domain Translation and Matrix Profile (paper title) Anamaria Radoi 2022.
The underlying principles behind the Matrix Profile are delightfully simple yet powerful.
Markus Heidt (2022).
The Matrix profile approach long serves as a fundamental data mining method to investigate time series, ...is widely accepted in the data science community and has been successfully applied to various application domains.. Ju et al. IPDPS 2022.
For our purposes, the information 'source runs stably or not' can be extracted by observing the statistical properties of the matrix-profile data-array Geithner et al. IPAC2021
(for) intrusion detection in industrial network traffic, distances as calculated with Matrix Profiles rises significantly during the attacks. ..as a result, time series-based anomaly detection methods are capable of detecting deviations and anomalies.  Schotten (2019).
The results in the Matric Profile provide insights into the stability of the patterns presented in the players. Guglielmo et. al.
Based on the concept of Matrix Profile ..without relying on time series synchronization.. the Railway Technologies Laboratory of Virginia Tech has been developing an automated onboard data analysis for the maintenance track system  Ahmadian et. al. JRC2019
Matrix Profile is the state-of-the-art similarity-based outlier detection methodChristian Jensen et. al. IJCAI-19
we use the exact method based on the Matrix Profile (to assess the effectiveness of therapy) Funkner et al Procedia 2019.
Recently, a research group from UCR have proposed a powerful tool - the Matrix Profile (MP) as a primitive...(we use it for) fault detection Jing Zhang et al. ICPHM 2019
Inspecting both graphs one can see that the matrix-profile algorithm was able to identify regions where there is a  change on the power level over the observed band. F Lobao 2019.
RAMP builds upon an existing time series data analysis technique called Matrix Profile to detect anomalous distances...collected from scientific workflows in an online manner. Herath et. al. IEEE Big Data 2019
Based on obtained results for the considered data set, matrix profiles turned out to be most suitable for the task of anomaly detection Lohfink et al. VISSEC2019
The computation speed and exactness of the Matrix Profile make it a powerful tool and (our) results back this. Barry & Crane AICS 2019
(examining) manufacturing batches considering raw amperage (we found that the) Matrix Profile highlights anomalies Hillion & O'Connell of TIBCO Data Science. re:Invent 2019. [video]
we use the exact method based on the matrix profile to search for motifs (that) can be used to monitor the patient's condition, to assess the effectiveness of therapy or to assess the physician's actions. Funknera et al. YSCCS 2019
(The Matrix Profile is a) similarity join to measure the similarity between two given sequences. we opt for the median of the profile array as the representative distance (3D Dancing Move Synthesis from Music)" Anh et al. IEEE Robotics and Automation Letters (video)
We were amazed by the power of MP and seek to incorporate it into our framework Ye and Ageno.
..adopting the concept of (the) Matrix Profile, we conduct the first attempt to.. J. Zuo et. al. Big Data 20019
The accuracies obtained ...indicate that the Matrix Profile is useful for the task at hand instead of using the CNN features directly  Dhruv Batheja
To speed up online bad PMU data detection a fast discovery strategy is introduced based on (the Matrix Profile)  Zhu and Hill.
Specifically, ALDI uses the matrix profile method to quantify the similarities of daily subsequences in time series meter data, Zoltan Nagy, Energy & Buildings (2020)
Our two-fold approach first leverages the Matrix Profile technique for time series data mining.. Nichiforov 2020.
 the class of matrix profile algorithms.. ..is a promising approach, as it allows simplified post-processing and analysis steps by examining the resulting matrix profile structureA. Raoofy et al.
We only require information about the time of several critical incidents to train our methods, as previously. To this end, we employ the Matrix Profle.. Bellas. et al.
a matrix-profile based algorithm applied across all trajectory data against a validation set revealed four significant motifs which we defined as motif A, B, C and D..  Fernandez Alvarez 2020.
The main building block of this (game analytics) algorithm is the matrix profile, Saadat and Sukthankar AAAI2020
We call the custom Matrix Profile and box-plot approach as STUMPI-BP and employ it for all types of time series that.. Huawei Ireland Research Centre
The state-of-the-art exact anytime method for motif and discord discovery is matrix profile. Onur Mutlu et al.
Matrix Profile requires less training time, data and parameter tuning compared to other data mining methods.. Stamatescu et al.
(our) approach is based on matrix profile,  which is a method for time series analysis that is robust, domain-agnostic, and computationally efficient. Van Hoecke et al 2020
(we conducted) five cases, which demonstrate how the matrix profile based pattern recognition approach can be used in the power boiler environment. In general (this) resulted in promising outcomes. Liisa Nokelainen 2020.
To investigate the physiological reaction, we used Matrix Profile..   Lalanne et al. 2020
Our time series mining uses the Matrix Profile approach.. Poerwawinata 2020
To this end, by adopting the concept Shapelet and Matrix Profile, we conduct the first attempt to.. Zeitouni et al.
Three anomalies were found near bridges and were indicated as low priority track issues. Two anomalies were... The Matrix Profiler found all these anomalies Steenwinckel et al 2020
Matrix Profile is a dimension reduction approach which requires less training time, data and parameter tuning compared to other data mining methods. Nichiforov et al.
Our solution will rely on Matrix Profile to allocate these discords in order to learn the anomalies. Alshaer et al.
The state-of-the-art exact anytime method for motif and discord discovery is matrix profile. Fernandez et. al.
Out of these methods, matrix profile is chosen in this paper due to its scalability and computational efficiency, while being also robust and parameter-free. Herrera et al 2021
The Matrix Profile (MP) algorithm has the potential to revolutionise many areas of data analysis.. Cartwright et al Engineering Proceedings
..we detect and locate the appearance of repeating patterns based on the Matrix Profile.. Li et al. IEEE MASS
We leverage the Matrix Profile (MP) to create a  micro-service-based machinery monitoring solution Naskos et al 2021
SLMAD uses statistical-learning and employs a robust box-plot algorithm and Matrix Profile (MP) to detect anomalies. Team from Huawei/UCD.
We found that all these similarity or randomness measures can be estimated with variants of the highly efficient Matrix Profile (MP) algorithm.  Austin Health/ University of Melbourne
We present, to the best of our knowledge, the first FPGA implementation of a Matrix Profile algorithm. Future Generation Computer Systems 2021
Out of these methods, matrix profile is chosen in this article due to its scalability and computational efficiency, while being also robust doing so increased accuracy and computational efficiency for detecting where and when anomalies and pattern irregularities occur.  PARLIKAD et al 2021
Our approach exploits the synergism of the matrix profile data analysis technique. Nichiforov et al. Energies 2021, 14, 7465.
Our version of time series analysis is based on Matrix Profile. Onur Mutlu et al. 2022
(to diagnose) Decubitus Prophylaxis we use the Matrix Profile. Richter et. al.  PerCom 2022
We successfully detected a deauthentication attack on a smart farm architecture using MERLIN, a (discord) distance-based anomaly detection algorithm.  Sina Sontowski 2022
 we detect and locate the appearance of repeating patterns based on the Matrix Profile. Lit et al. 2021 IEEE Mobile Ad Hoc and Smart Systems.
Another use case for the metamodel is the processing of more than 200,000 time series with a total of more than 30 million individual data points. ..for detecting anomalies and finding regime change locations, Matrix Profile algorithms are used.. Latapie et al. Front. Psychol., 20 May 2022
For detecting Decubitus Prophylaxis we use the Matrix Profile. Grunerbel et. al. 2022 IEEE Pervasive Computing and Communications.
As the most complex and largest dataset, the BETH dataset presents challenges to any anomaly detection technique. The matrix profile anomaly detector presented here performed well, not missing any of the evil outliers. Alexander Beattie 2022
..our new recovery method that uses the matrix profile  to extract the top-k repeated patterns from sensor nodes within the same group and.. Berjab et al 2022
The matrix profile based framework yielded a cumulative average recall value of 0.92. This is proof that this algorithm is working remarkably well. Gandra 2022, Anomaly Detection for Smart Infrastructure: An Unsupervised Approach for Time Series Comparison
The Matrix Profile algorithm is well-suited for anomaly detection tasks such as this.. We found that all these similarity or randomness measures can be estimated with variants of the highly efficient Matrix Profile.. Wang 2022
To implement this concept, the matrix profile algorithm was chosen.   S. Costa et al.   J. Sens. Actuator Netw. 2022
The proposed approach employs a pattern recognition technique derived from the matrix profile (MP) .. for the automatic detection of energy consumption anomalies at the whole-building level. Chiosa et al. 2022
We demonstrate the effectiveness of our method over state-of-the-art methods (based on) a solution based on Contextual Matrix Profile (CMP), an exact and ultra-fast distance-based anomaly detection algorithm. Bijlani 2022
..solve these problems by a modified approach in which a time series prediction used matrix profile with adjacent pairs.. Tran et al 2022
For this task (Telemonitored Continuous Positive Airway Pressure ), we employed method Matrix Profile..  Rodrigues et al 2022
.. and use the time window and the matrix profile to obtain the local correlation of the co-occurrence time series data.. Lin et al 2022
..the preliminary findings illustrate that the matrix profile algorithm can efficiently detect IoT-based DDoS attacks. Alzahrani et al. 2020
For decarbonizing primary steel production one of the methods to gain a deeper understanding of the data and extract patterns and anomalies embedded in the dataset, is .. Matrix profile .. an innovative and fast technique.  Bhaskar et al. 2022.
The approach used by SHAPEwarp is inspired by the BLAST algorithm and it builds on top of .. Matrix Profile. Morandi et al. nature communications 2022.
Our results suggest that AU features extracted using the Matrix Profile method  can be used to discriminate between experts, intermediates, and novices when used to train a classifier.  Guglielmo et al. 2022
Specifically, the Matrix Profile algorithm is shown to be well suited as a generalizable approach for detecting real-time anomalies in streaming time-series data. The Matrix Profile is shown to behave robustly and accurately for all fault types presented.  Beattie et al. 2022
(our work) inherits the speed, accuracy, and simplicity of the Matrix Profile, providing configurability, the ability to denoise and detect patterns, and explainability to clinical practitioners Bijlani et al. 2022