The PixelPlayer AI Can Identify and Isolate Instruments in an Orchestra
Justin Lam — July 9, 2018 — Art & Design
References: venturebeat & engadget
Developed by MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) the PixelPlayer AI is capable of recognizing instruments in a video, identifying specific ones and isolating the sounds they produce. Identification of sound by the PixelPlayer AI occurs at the smallest pixel level, which allows the AI to accurately identify and isolate sounds no matter how loud or lost they are in a greater orchestra. The isolation of a specific sound can be played back at any volume and will allow users to engage in full manipulation of it.
To train this instrument-identifying AI, CSAIL utilized a self-supervised deep learning technique and fed the AI over 60 hours of videos to learn from. The PixelPlayer AI can currently identify the sounds of 20 instruments and is being continually improved to increase this number. In the future, the team at CSAIL hopes to further develop the AI into an audio editing tool and use it for robots aimed at sound identification.
To train this instrument-identifying AI, CSAIL utilized a self-supervised deep learning technique and fed the AI over 60 hours of videos to learn from. The PixelPlayer AI can currently identify the sounds of 20 instruments and is being continually improved to increase this number. In the future, the team at CSAIL hopes to further develop the AI into an audio editing tool and use it for robots aimed at sound identification.
Trend Themes
1. Instrument-identifying AI - The PixelPlayer AI can accurately identify and isolate instruments in an orchestra, opening opportunities for enhanced audio editing and sound identification in various industries.
2. Self-supervised Deep Learning - The use of a self-supervised deep learning technique to train the PixelPlayer AI showcases the potential for advancements in machine learning algorithms, enabling more precise and efficient identification of complex patterns.
3. AI-powered Audio Editing - The development of the PixelPlayer AI as an audio editing tool demonstrates the disruption potential for automating and streamlining audio editing processes, saving time and improving creative possibilities.
Industry Implications
1. Music Production - The ability of the PixelPlayer AI to identify and isolate instruments in an orchestra can revolutionize the music production industry by offering advanced audio editing capabilities and precise sound manipulation.
2. Film and TV Production - The instrument-identifying AI technology showcased by the PixelPlayer AI can significantly enhance film and TV production by enabling better control over audio elements and facilitating post-production editing tasks.
3. Robotics - The sound identification capabilities of the PixelPlayer AI present opportunities for incorporating advanced audio perception in robots, leading to improved interactions, navigation, and overall performance in various robotics applications.
0.7
Score
Popularity
Activity
Freshness
