MIT Researchers Develop Software to Reveal 'Hidden Information" in Videos

By Peter Bernstein February 21, 2013

It turns out that just zooming in on a picture, or in this case a series of video frames, while good for looking at those hard to see aspects of a subject, are about to take a leap forward. As highlighted in a recent article in the MIT News, “Researchers amplify variations in video, making the invisible visible,” researchers from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) will present new software that, “amplifies changes in successive frames of video that are too subtle for the naked eye” at this  summer's Siggraph — the premier computer-graphics conference.

The software can exaggerate tiny motions. Examples cited include: the ability to actually "see" someone's pulse as the skin reddens and pales with the flow of blood, making visible the vibrations of individual guitar strings, or the breathing of a swaddled infant in a neonatal intensive care unit. The medical implications alone make this exciting.

How it works

In explaining the principles on how the new software works, MIT draws the analogy to an equalizer in a stereo sound system which boosts some frequencies and cuts others. In this case, the pertinent frequency is the frequency of color changes in a sequence of video frames, not the frequency of an audio signal.

The prototype software allows the user to specify the frequency range of interest and the degree of amplification. The software works in real time and displays both the original video and the altered version of the video, with changes magnified. Although the technique is ideal for regularly occurring events, if the range of frequencies is wide enough it can amplify changes that occur only once. 

The success of unintended consequences

As the article points out, this is a case where the unintended consequences of finding a solution to one problem lead to an equally large discovery. Graduate student Michael Rubinstein, recent alumni Hao-Yu Wu '12, MNG '12 and Eugene Shih SM '01, PhD '10, and professors William Freeman, Fredo Durand and John Guttag, have set out to have the system amplify color changes. However, they found it amplified motion as well. "We started from amplifying color, and we noticed that we'd get this nice effect, that motion is also amplified," Rubinstein says. "So we went back, figured out exactly why that happens, studied it well, and saw how we can incorporate that to do better motion amplification."Again as the article notes, using the system to amplify motion rather than color requires a different kind of filtration. It works well if motions are relatively small which is why this holds so much promise for a variety of commercial applications. Those cited include "contactless monitoring" of hospital patients' vital signs, monitoring infants who are born prematurely or otherwise require early medical attention, and to augment home video baby monitors for the home.  

MIT has prepared a video on the research that gives a nice explanation of the system.

 

Maneesh Agrawala, an associate professor in the electrical engineering and computer science department at the University of California at Berkeley, and director of the department's Visualization Lab, noted that, “This approach is both simpler and allows you to see some things that you couldn't see with that old approach."  He added that, "The simplicity of the approach makes it something that has the possibility for application in a number of places. I think we'll see a lot of people implementing it because it's fairly straightforward."

The improved quality of cameras (even those on our smartphones), and the proliferation of HDTV, have made us almost spoiled in terms of our expectations when viewing still and moving pictures. High resolution does enhance the entertainment customer experience. However, the new MIT software is not about entertainment, it has practical uses that are going to enable individuals and institutions to be much more proactive about a host of things. How this gets realized commercially, and how fast will be interesting to watch. The ability to improve medical care alone makes this something to keep a close eye on. There is tremendous value in making the previously invisible readily viewable. 




Edited by Brooke Neuman
SHARE THIS ARTICLE
Related Articles

The NFL and Twitter Just Put Fans in Charge of the Future of Sports Viewing

By: Special Guest    5/26/2016

Ask ten people you know what the last thing they viewed live was and eight of them will tell you, a sporting event. The others may cite the occasional…

Read More

10 Tech Tools to Get You Through 2016

By: Drew Hendricks    5/24/2016

A great example of how simple tech innovations can make even the smallest things in our lives a little easier, the Oombrella, developed by Wezzoo, is …

Read More

The Bigger Picture Behind Facebook's Bias Accusations

By: Special Guest    5/23/2016

Facebook recently followed up with a stern position that there was no biased manipulation of presented topics in its 'Trending Topics' section.

Read More

Tamiami Angel Fund II Adds 10th Company to Investment Roster

By: Steve Anderson    5/19/2016

The Tamiami Angel Fund II (T2) has been busy in recent months, adding a slate of new investment targets to its roster. T2 recently added one new opera…

Read More

Artificial Intelligence is no Longer Science Fiction, It's a Reality

By: Special Guest    5/18/2016

Talend Chief Evangelist says rather than being frightened by the perceived threat of AI, it would be wise to embrace the possibilities.

Read More