Developed by MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) the PixelPlayer AI is capable of recognizing instruments in a video, identifying specific ones and isolating the sounds they produce. Identification of sound by the PixelPlayer AI occurs at the smallest pixel level, which allows the AI to accurately identify and isolate sounds no matter how loud or lost they are in a greater orchestra. The isolation of a specific sound can be played back at any volume and will allow users to engage in full manipulation of it.
To train this instrument-identifying AI, CSAIL utilized a self-supervised deep learning technique and fed the AI over 60 hours of videos to learn from. The PixelPlayer AI can currently identify the sounds of 20 instruments and is being continually improved to increase this number. In the future, the team at CSAIL hopes to further develop the AI into an audio editing tool and use it for robots aimed at sound identification.
More Stats +/-
3D Sound-Emulating Speakers
Interactive Music Mobile Applications
Music Library Sharing Apps
Sample-Identifying Music Apps