Monitoring and Mining Animal Sounds in Visual Space

Welcome to Animal Sound Analysis Page

(Experiments, Datasets, Sourcecode, and more.)

Here is the color version of our paper

Project Overview

This project is aim to monitor animals by the sounds they produce and build animal sound recognition/classification framework

Supplementary Materials
Sound Fingerprint Discovery
Insect Classification
Monitoring Animal Sounds with Sound Fingerprints

Supplementary Materials

Here is the slides of our paper. Please contact Bilson Campana for the main framework of CK distance measure. Note that before download other code, please download and install this first and run other code listed below.

Here is the place to show all our extensive empirical work

Do sound fingerprints work for other vocal animals such as frogs?

yes. Here are the experiment results
Here is the dataset.

Is our method robust to mislabeled training data?

As long as the majority of data in target class is correctly labeled, yes.
Here are the experiment results.
Here is the dataset.
Here is the source code to test how much mislabeled data will affect the experiment results.

Are we robust to noisy enviroments, such as aircraft noise in the background as a monitored cricket chirps?

As long as the target signal is a significant fraction of the overall singal, yes.
Here are the experiment results and datasets

Is our algorithm robust to sound overlap, such as bird is sing when recorded insects chirps?

As long as the target signal is a significant fraction of the overall singal, yes. Even though this situation happens in both the training and testing data
Here are the experiment results and datasets

Sound Fingerprint Discovery

Classify animal sounds in visual sapce

Algorithm analysis is needed instead of manual inspection
Avoid any type of data cleaning or explicit feature extraction, but just use raw spectrogram

CK distance measure between different sound spectrograms

We propose to treat the texture of the animal sound spectrogram as "acoustic fingerprint", using a recently introduced parameter-free texture measure as a distance measure.
Here are the details of this distance measure.

Sound Fingerprint

P: Contains examples only from the "positive" species class; U: Non-target species sounds, Here are the 200 objects non-target species class we created, including auido files and corresponding spectrograms we generated.
To find a subsequence of one of the objects in P, which is close to at least one subsequence in each element of P, but far from all subsequences in every element of U

Sound Fingerprint discovery

(Target species: Atlanticus dorsalis for example), Here are the dataset including P and U.

Here is the source code to find sound fingerprint.

Brute-force search results

Admissible Entropy-based Pruning

Euclidean Distance Ordering Heuristic

Insect Classification

Insect Classification Benchmark: the dataset consists of twenty species of insects, eight of which are Gryllidae(crickets) and twelve of which are Tettigoniidae(katydids). Thus, we can treat the problem as either a twenty-class species level problem, or two-class genus level problem.

We learned twenty sound fingerprints using Sound Fingerprint Discovery algorithm, then predicted the testing examplars class label by sliding each fingerprint across it and recording the fingerprint that produced the minimum value as the examplar's nearest neighbor. The result is shown in the table above.

Here is the classification benchmark.

Here is the source code for classification.

Monitoring Animal Sounds with Sound Fingerprints

To test our ability to monitor an audio stream in real time for the presence of a particular species of insects, we learned the sound fingerprint for three insects species of insects native to Florida. We produce an twelve-second sequence of audio by concatenating snippets of 6 different species, including holdout examples from our three species of interest.

Here are the datasets for monitoring an a long audio stream, and to learn the fingerprint, please refer the source code in fingerprint discovery. To plot this figure, please refer the code in MisLabel experiment's code.

While each fingerprint has a different threshold, for simplicity and visual clarity we show just the averaged threshold.To visually see how to monitoring an audio stream in real time, please take a look at the video below.