MIT Researchers Develop Software to Reveal 'Hidden Information" in Videos

By Peter Bernstein February 21, 2013

It turns out that just zooming in on a picture, or in this case a series of video frames, while good for looking at those hard to see aspects of a subject, are about to take a leap forward. As highlighted in a recent article in the MIT News, “Researchers amplify variations in video, making the invisible visible,” researchers from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) will present new software that, “amplifies changes in successive frames of video that are too subtle for the naked eye” at this  summer's Siggraph — the premier computer-graphics conference.

The software can exaggerate tiny motions. Examples cited include: the ability to actually "see" someone's pulse as the skin reddens and pales with the flow of blood, making visible the vibrations of individual guitar strings, or the breathing of a swaddled infant in a neonatal intensive care unit. The medical implications alone make this exciting.

How it works

In explaining the principles on how the new software works, MIT draws the analogy to an equalizer in a stereo sound system which boosts some frequencies and cuts others. In this case, the pertinent frequency is the frequency of color changes in a sequence of video frames, not the frequency of an audio signal.

The prototype software allows the user to specify the frequency range of interest and the degree of amplification. The software works in real time and displays both the original video and the altered version of the video, with changes magnified. Although the technique is ideal for regularly occurring events, if the range of frequencies is wide enough it can amplify changes that occur only once. 

The success of unintended consequences

As the article points out, this is a case where the unintended consequences of finding a solution to one problem lead to an equally large discovery. Graduate student Michael Rubinstein, recent alumni Hao-Yu Wu '12, MNG '12 and Eugene Shih SM '01, PhD '10, and professors William Freeman, Fredo Durand and John Guttag, have set out to have the system amplify color changes. However, they found it amplified motion as well. "We started from amplifying color, and we noticed that we'd get this nice effect, that motion is also amplified," Rubinstein says. "So we went back, figured out exactly why that happens, studied it well, and saw how we can incorporate that to do better motion amplification."Again as the article notes, using the system to amplify motion rather than color requires a different kind of filtration. It works well if motions are relatively small which is why this holds so much promise for a variety of commercial applications. Those cited include "contactless monitoring" of hospital patients' vital signs, monitoring infants who are born prematurely or otherwise require early medical attention, and to augment home video baby monitors for the home.  

MIT has prepared a video on the research that gives a nice explanation of the system.

 

Maneesh Agrawala, an associate professor in the electrical engineering and computer science department at the University of California at Berkeley, and director of the department's Visualization Lab, noted that, “This approach is both simpler and allows you to see some things that you couldn't see with that old approach."  He added that, "The simplicity of the approach makes it something that has the possibility for application in a number of places. I think we'll see a lot of people implementing it because it's fairly straightforward."

The improved quality of cameras (even those on our smartphones), and the proliferation of HDTV, have made us almost spoiled in terms of our expectations when viewing still and moving pictures. High resolution does enhance the entertainment customer experience. However, the new MIT software is not about entertainment, it has practical uses that are going to enable individuals and institutions to be much more proactive about a host of things. How this gets realized commercially, and how fast will be interesting to watch. The ability to improve medical care alone makes this something to keep a close eye on. There is tremendous value in making the previously invisible readily viewable. 




Edited by Brooke Neuman
SHARE THIS ARTICLE
Related Articles

GENBAND & Sonus Go Dutch for Merger

By: Maurice Nagle    5/23/2017

Mergers and acquisitions are the norm in business. However, it's not every day that two major cloud communications players with highly complementary o…

Read More

The Killer App for VR: The Ability to Meet Yourself

By: Rob Enderle    5/23/2017

I was at a VR event this week, and I'm sure the speaker misspoke when he said that one of the benefits of VR is the ability to meet yourself. But the …

Read More

WannaCry Ransomware Holds Files Hostage: Best Practices to Avoid Being a Victim

By: Special Guest    5/23/2017

More than 200,000 computers in more than 150 countries were crippled by a massive ransomware attack, dubbed WannaCry, and security experts warned that…

Read More

LeoSat Secures Japanese Investment for Enterprise Broadband Satellite Network

By: Doug Mohney    5/23/2017

Another broadband satellite cloud network moved closer to reality this month, with LeoSat securing an investment from SKY Perfect JSAT (SJC) Corporati…

Read More

Organizations Can Combat WannaCry & Jaff Ransomware With Well Instrumented DNS

By: Special Guest    5/22/2017

The Infoblox Intelligence Unit observed two global malware outbreaks on Friday, May 12. Although there is no indication that the two attacks were rela…

Read More