Oxford's 'LipNet' Can Read Lips with Over 90 Percent Accuracy

By: Joey Haar - Nov 9, 2016
References: arxiv.org & gizmodo
Researchers from the University of Oxford's Computer Science Department have developed a software program called 'LipNet' that can read people's lips with incredible accuracy. The program can take a video sequence as input and analyze it frame by frame in order to predict complete sentences.

Lipreading is a difficult task that the average person can't do whatsoever. People with experience lipreading -- such as those who are hard of hearing -- fare much better, but still have an average performance of just 52 percent accuracy. In Oxford's tests, LipNet performed with 93 percent accuracy -- an incredible improvement above humans.

LipNet was trained using GRID, a data set of 64,000 English sentences. The researchers plan to adapt the system for real-world sentences in the future.