MIT Researchers Develop Software to Reveal 'Hidden Information" in Videos

By Peter Bernstein February 21, 2013

It turns out that just zooming in on a picture, or in this case a series of video frames, while good for looking at those hard to see aspects of a subject, are about to take a leap forward. As highlighted in a recent article in the MIT News, “Researchers amplify variations in video, making the invisible visible,” researchers from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) will present new software that, “amplifies changes in successive frames of video that are too subtle for the naked eye” at this  summer's Siggraph — the premier computer-graphics conference.

The software can exaggerate tiny motions. Examples cited include: the ability to actually "see" someone's pulse as the skin reddens and pales with the flow of blood, making visible the vibrations of individual guitar strings, or the breathing of a swaddled infant in a neonatal intensive care unit. The medical implications alone make this exciting.

How it works

In explaining the principles on how the new software works, MIT draws the analogy to an equalizer in a stereo sound system which boosts some frequencies and cuts others. In this case, the pertinent frequency is the frequency of color changes in a sequence of video frames, not the frequency of an audio signal.

The prototype software allows the user to specify the frequency range of interest and the degree of amplification. The software works in real time and displays both the original video and the altered version of the video, with changes magnified. Although the technique is ideal for regularly occurring events, if the range of frequencies is wide enough it can amplify changes that occur only once. 

The success of unintended consequences

As the article points out, this is a case where the unintended consequences of finding a solution to one problem lead to an equally large discovery. Graduate student Michael Rubinstein, recent alumni Hao-Yu Wu '12, MNG '12 and Eugene Shih SM '01, PhD '10, and professors William Freeman, Fredo Durand and John Guttag, have set out to have the system amplify color changes. However, they found it amplified motion as well. "We started from amplifying color, and we noticed that we'd get this nice effect, that motion is also amplified," Rubinstein says. "So we went back, figured out exactly why that happens, studied it well, and saw how we can incorporate that to do better motion amplification."Again as the article notes, using the system to amplify motion rather than color requires a different kind of filtration. It works well if motions are relatively small which is why this holds so much promise for a variety of commercial applications. Those cited include "contactless monitoring" of hospital patients' vital signs, monitoring infants who are born prematurely or otherwise require early medical attention, and to augment home video baby monitors for the home.  

MIT has prepared a video on the research that gives a nice explanation of the system.

 

Maneesh Agrawala, an associate professor in the electrical engineering and computer science department at the University of California at Berkeley, and director of the department's Visualization Lab, noted that, “This approach is both simpler and allows you to see some things that you couldn't see with that old approach."  He added that, "The simplicity of the approach makes it something that has the possibility for application in a number of places. I think we'll see a lot of people implementing it because it's fairly straightforward."

The improved quality of cameras (even those on our smartphones), and the proliferation of HDTV, have made us almost spoiled in terms of our expectations when viewing still and moving pictures. High resolution does enhance the entertainment customer experience. However, the new MIT software is not about entertainment, it has practical uses that are going to enable individuals and institutions to be much more proactive about a host of things. How this gets realized commercially, and how fast will be interesting to watch. The ability to improve medical care alone makes this something to keep a close eye on. There is tremendous value in making the previously invisible readily viewable. 




Edited by Brooke Neuman
SHARE THIS ARTICLE
Related Articles

Windows 10: One Giant Step into the Future

By: Rob Enderle    7/30/2015

This is Windows 10 launch week and things are going pretty well. Unlike some rumors to the contrary, the OEMs are ready with preloads and so far, the …

Read More

Cyberattacks on OPM, Anthem and United Airlines are Linked

By: Peter Bernstein    7/30/2015

The bad news on the cyberattack front got a whole lot worse with a story from Bloomberg by reporters Michael Riley and Jordan Robertson that revealed …

Read More

Comings & Goings: Citrix CEO Steps Down as Company Gears for an Overhaul

By: Tara Seals    7/29/2015

There appears to be a bit of shakeup going on over at Citrix, the venerable distributed computing company. Long-time Citrix CEO Mark Templeton has ann…

Read More

Three Reasons Passwords Are Here to Stay

By: TMCnet Special Guest    7/28/2015

Siber Systems explains why someday passwords will be a thing of the past, but not any time soon.

Read More

Wheelings & Dealings: Visa Joins Amex with Giant Investment in Payment Startup Stripe

By: Laura Stotler    7/28/2015

Payment startup company Stripe has scored a whopping round of funding, with Visa taking a major interest in the company's technology. The funding amou…

Read More