MIT Researchers Develop Software to Reveal 'Hidden Information" in Videos

By Peter Bernstein February 21, 2013

It turns out that just zooming in on a picture, or in this case a series of video frames, while good for looking at those hard to see aspects of a subject, are about to take a leap forward. As highlighted in a recent article in the MIT News, “Researchers amplify variations in video, making the invisible visible,” researchers from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) will present new software that, “amplifies changes in successive frames of video that are too subtle for the naked eye” at this  summer's Siggraph — the premier computer-graphics conference.

The software can exaggerate tiny motions. Examples cited include: the ability to actually "see" someone's pulse as the skin reddens and pales with the flow of blood, making visible the vibrations of individual guitar strings, or the breathing of a swaddled infant in a neonatal intensive care unit. The medical implications alone make this exciting.

How it works

In explaining the principles on how the new software works, MIT draws the analogy to an equalizer in a stereo sound system which boosts some frequencies and cuts others. In this case, the pertinent frequency is the frequency of color changes in a sequence of video frames, not the frequency of an audio signal.

The prototype software allows the user to specify the frequency range of interest and the degree of amplification. The software works in real time and displays both the original video and the altered version of the video, with changes magnified. Although the technique is ideal for regularly occurring events, if the range of frequencies is wide enough it can amplify changes that occur only once. 

The success of unintended consequences

As the article points out, this is a case where the unintended consequences of finding a solution to one problem lead to an equally large discovery. Graduate student Michael Rubinstein, recent alumni Hao-Yu Wu '12, MNG '12 and Eugene Shih SM '01, PhD '10, and professors William Freeman, Fredo Durand and John Guttag, have set out to have the system amplify color changes. However, they found it amplified motion as well. "We started from amplifying color, and we noticed that we'd get this nice effect, that motion is also amplified," Rubinstein says. "So we went back, figured out exactly why that happens, studied it well, and saw how we can incorporate that to do better motion amplification."Again as the article notes, using the system to amplify motion rather than color requires a different kind of filtration. It works well if motions are relatively small which is why this holds so much promise for a variety of commercial applications. Those cited include "contactless monitoring" of hospital patients' vital signs, monitoring infants who are born prematurely or otherwise require early medical attention, and to augment home video baby monitors for the home.  

MIT has prepared a video on the research that gives a nice explanation of the system.

 

Maneesh Agrawala, an associate professor in the electrical engineering and computer science department at the University of California at Berkeley, and director of the department's Visualization Lab, noted that, “This approach is both simpler and allows you to see some things that you couldn't see with that old approach."  He added that, "The simplicity of the approach makes it something that has the possibility for application in a number of places. I think we'll see a lot of people implementing it because it's fairly straightforward."

The improved quality of cameras (even those on our smartphones), and the proliferation of HDTV, have made us almost spoiled in terms of our expectations when viewing still and moving pictures. High resolution does enhance the entertainment customer experience. However, the new MIT software is not about entertainment, it has practical uses that are going to enable individuals and institutions to be much more proactive about a host of things. How this gets realized commercially, and how fast will be interesting to watch. The ability to improve medical care alone makes this something to keep a close eye on. There is tremendous value in making the previously invisible readily viewable. 




Edited by Brooke Neuman
SHARE THIS ARTICLE
Related Articles

Why Blockchain Could Be a Gamechanger

By: Paula Bernier    1/22/2018

Blockchain has become closely associated with the controversial topic of cryptocurrency. And that's fine because blockchain is an enabling technology …

Read More

Consumer Privacy in the Digital Era: Three Trends to Watch

By: Special Guest    1/18/2018

Digital advertising has exploded in recent years, with the latest eMarketer data forecasting $83 billion in revenue this year and continued growth on …

Read More

CES 2018: Terabit Fiber - Closer Than We Think

By: Doug Mohney    1/17/2018

One of the biggest challenges for 5G and last mile 10 Gig deployments is not raw data speeds, but middle mile and core networks. The wireless industry…

Read More

10 Benefits of Drone-Based Asset Inspections

By: Frank Segarra    1/15/2018

Although a new and emerging technology, (which is still evolving), in early 2018, most companies are not aware of the possible benefits they can achie…

Read More

VR Could Change Entertainment Forever

By: Special Guest    1/11/2018

VR could change everything from how we play video games to how we interact with our friends and family. VR has the power to change how we consume all …

Read More