R. Colin Johnson

 

 

7/14/2010 12:37 AM EDT

 

PORTLAND, Ore.—Dolby took the very best available audio and made it better, but nothing like it has existed for video. Now DarbeeVision Inc. hopes to do for video what Dolby did for audio—enhance the details while suppressing the noise. And like the first Dolby devices for audio, the first DarbeeVision equipped devices—due out by Christmas—will be equipped with a on/off button, so you can prove to yourself how well it improves picture quality.
 

"This is a little like Dolby in the early days," said DarbeeVision’s CEO, Paul Darbee. "When Dolby first came out for audio, it had an on/off button which you could push to find out what it was you were buying. DarbeeVision is like that too—we have to show you the 'before' before you can appreciate the 'after'."

So far no one shown the demonstration has chosen "before," because the difference is like hitting the windshield wipers during rain—suddenly you can see details and depth that were only vague shapes of indeterminate distance before. Look carefully at the image illustrating this story to observe "visual presence" added by DarbeeVision (akin to the "audible presence" added by Dolby). Or visit the DarbeeVision Gallary, where you can click to toggle the effect on and off--causing the 3-D details to "pop" and making the "before" image appear flat (2-D) and out-of-focus.

 

DarbeeVision (Anaheim, Calif.) developed its video enhancing algorithm by taking the depth information from duel stereo cameras—one for each eye—and encoding it into a single image that can be viewed or printed using any display of printer, but which nevertheless triggers the perception detail and depth in the viewer. DarbeeVision-enhanced devices will include TV sets, set-top-boxes, video game consoles, mobile devices, digital picture frames and "smart" HDMI cables with "DarbeeVision inside," so you can retrofit your current HDTV. Original equipment manufacturers (OEMs) can choose from an IP block for their own ASIC, a Darbee-manufactured FPGA, or as firmware for a graphics processing unit.

Darbee originally discovered the technique back in early 1970s, but he got sidetracked in the 1980s when he invented the universal remote control,for which he is still collecting royalties after half a billion units sold. But the new company has even higher market penetration hopes it the DarbeeVision algorithm.

Darbee was experimenting with stereo videos using two cameras when he discovered that if you defocus one and subtract it from the remaining sharp image, the resultant combined image recreates a 3-D experience in the viewer. Technically, defocusing and subtracting one of your eye's images removes the low-spatial-frequency information that was common to the two images (general shape) thus emphasizing the high-spatial frequencies (fine details) that were different—that is, the 3-D highlights that the brain interprets as depth.

"We are embedding depth cues into the image that the human visual system in the brain automatically interprets as depth," said Darbee.

In the brain, according to Darbee, the signals from the two eyes terminate in an interdigitated map where lateral inhibition (between the nerve endings for left and right eyes) performs the defocusing and subtraction operation—the output of which carries 3-D information throughout the brain. DarbeeVision makes use of the same brain pathways, but encodes the defocus and subtraction information into the image itself, a process that Darbee claims triggers the brain's perception of 3D in a single image.

"It's not true 3-D—sometimes we call it 2.5D—but it does cause the brain to perceive depth," said Darbee. "It's similar to those drop shadows that everybody is using on text now days. Drop shadows are like our defocused and subtracted image—they appear to be behind, making the text seem to float in front of the drop shadow." 

DarbeeVision has patented the part of its algorithm that does the defocusing and subtraction, but is keeping two other aspects of its algorithm a trade secret. First is how they generate the effect without stereo cameras—DarbeeVision can use both the left- and right-images from a stereo camera, but also works with single images by synthesizing its own left- and right-versions using a proprietary technique. The company's second trade secret is how it performs the subtraction step without creating artifacts like increased grain, which plague similar effects like the "unsharp mask" in Photoshop.

Early versions of DarbeeVision were hand-tweaked to add "visual presence" to Robert Altman's movie "Gosford Park," which took about two seconds per frame, resulting in a run time of about three days for the 150,000 frames in the finished movie. DarbeeVision was also used to enhance Altman's first direct-to-video film called "The Company," which he reported made the video look more comparable to film. All DVDs of both Gosford Park and The Company were made with DarbeeVision.

Since then, DarbeeVision has been automated with artificial intelligence that "rides the knobs" that were previously necessary to hand tweak the effect, but which is now a completely turnkey operation. Even so, some DarbeeVision equipped devices to hit store shelves later this year and next will sport a single rotary dial that sets how much of the effect is used.
 

The high speed operations required for DarbeeVision require significant computational horsepower, but the algorithm will run in realtime in Cuda code for the Nvidia GTX graphics card. The company has also cast the algorithm into an Altera Cyclone 25k-gate FPGA with just two megabits of memory. Currently DarbeeVision is licensing its IP to others for manufacture in ASICs that will appear in consumer devices by Christmas.