Audio Data, acquired by the microphones embedded in the DualCam device, are processed through a Filter-and-Sum beamforming algorithm, generating a three dimensional Multispectral Acoustic Image, function of two spatial directions and temporal frequency. Each value in the resulting 3D structure represents the power spectrum at a given frequency bin, related to the audio source coming from a given spatial location. The 2D final Acoustic Map, encoding the sound energy for each spatial location, is computed integrating each power spectrum along the frequency axis. For each sequence we provide 3 different folders containing the video frames in VGA Resolution, acquired by the embedded video camera, and the audio data, acquired by the microphone array, in 2 different formats, each one corresponding to a different processing stage: (i) Multispectral Acoustic Image; and (ii) Acoustic Map. Into each sequence folder, we also add a Matlab script file which allows the users to load and combine a multispectral acoustic image with its corresponding video frame into a single RGB image. For a detailed description of the DualCam sensor and the dataset, please refer to the paper

[1] A. Zunino, M. Crocco, S. Martelli, A. Trucco, A. Del Bue, V. Murino, Seeing the Sound: a New Multimodal Imaging device for Computer Vision, 3D Reconstruction and Understanding with Video and Sound, (ICCV Workshop), 2015 [PDF]