This morning I looked into a parallel universe that seemed very much like the one in which we find ourselves. As best I can tell, it’s populated by beings just like us and all the observable features appear to be precise duplicates of what we have here on earth. The large window through which I was looking showed a room just like the one I was in and on the other side looking back at me was someone who I would swear was my identical twin. My twin was able to replicate my every movement and stayed in front of his window for precisely the same amount of time as I stayed in front of mine. The only peculiarity I could detect was that the people in this other universe had their left and right hands mixed up – or maybe I’m the one who didn’t have it quite right. I tried to have a conversation with my other-world twin, but unfortunately I could only see his lips moving. The window into this parallel universe apparently blocks all sound. However, other people who moved in and out of the room I was observing also seemed to have identical twins in the world on my side of this window.

What an incredible experience – to be able to see into another world that looks like ours in every detail — and through a window as clear as glass! Wouldn’t it be great if we could create other virtual worlds like this one at will? What would it take? Aren’t we almost there with the latest developments in 3D displays?

Don’t we wish!

With 2D images, we have been able to approach and even exceed the resolution and color sensing abilities of our visual systems. We can produce images that at normal viewing distances are better than our eyes can resolve and the color palettes can be highly accurate reproductions of the real objects. And we can do this almost as well in real time video. Given this, it would seem that adding the third dimension of depth to these images would be a reasonable and doable next step.

If we can get an accurate and entirely realistic 3D image just by looking into a mirror, what could possibly be so difficult about doing it with a camera or video recording device? It certainly seems like an obvious enhancement.

Unfortunately, there are some surprisingly challenging complications that come into the “picture” when we try to add the third dimension. It turns out that an accurate perception of the third dimension depends on much more than just our eyes as visual sensors. How can that be?

The complicating factor is that in the “real” world of three dimensions, we use a variety of clues to tell us what is going on and how we are positioned in relationship to other objects. Our eyes are not able to take in the entire scene with just one look. To make up for this limitation, we are continually scanning and probing the 3D space in which we find ourselves. Because we have this very limited solid angle where objects are in focus and in full color, we have to construct the complete scene from these rapidly acquired snippets of visual data by using our highly developed intelligence to do the necessary signal processing and integration.

Everything outside of our narrow visual cone is out of focus. As we scan to take in the entire scene, our eyes accommodate objects closer and further away by changing their relative viewing angle. And to further establish the relative position of objects closer and further away, we additionally rely on the sensory feedback from our head movement to enhance and confirm the stereoscopic effect generated by our two eyes. (To verify how important this effect is, take a look at objects that are at distances of hundreds of feet away and move your head side to side.) This complex interaction of eye movement, head movement and visual input is what we use to construct the “reality” that surrounds us. If any of these sensory inputs is missing or in conflict with the others, we immediately conclude that something is not right.

How we react can depend on how “not right” it is. We may simply decide that the image is phony. Or we may try to accept the conflicting inputs and end up getting a stress-induced headache. Or worse yet, our senses may be so conflicted that we get seriously nauseous. (Out of curiosity, I once asked a medical doctor why a conflicting visual experience would make us feel sick to our stomachs. He explained that this comes from our caveman days when dizziness indicated having eaten something poisonous and becoming nauseous is our bodies’ way of trying to get rid of the poison.)

In order to create an artificial, but believable, 3D experience we, therefore, need to accurately simulate not only the stereoscopic effect, but also realistically model the scene based on a viewer’s eye position as well as head movement. Only then does the prospect of creating a believable 3D virtual reality experience begin to look feasible. But this would be for just one viewer. All this would be required for each and every observer. Given this level of complexity and customization, the prospect of creating a virtual experience for an audience of many viewers begins to look like an impossible task — at least for the next few decades.

Are we, therefore, kidding ourselves that stereoscopic 3D can be a useful future direction for the next generation of display technologies? I think we are if we expect to see widespread use of 3D technology in movies, television, and other applications with story lines or events that depict life as we typically experience it. Adding a stereoscopic effect to these programs will be perceived as more of a confusing distraction rather than an enhancement. On the other hand, for inherently artificial environments such as computer-generated games, science fiction stories, and industrial applications where stereoscopic visualization can be helpful, it is highly likely that stereoscopic 3D viewing will become commercially successful.

One other new and potentially exciting opportunity will be in 3D video presentations made from live concerts and other staged entertainment events. With the dynamic lighting and laser shows that are now typically an integral part of these events, we enter the realm of fantasy creations where 3D can add to the overall emotional impact. Taken together these markets are of sufficient size that they will be able to support a large number of successful product entries.

What I believe is important for us in the display community is to understand what the limitations are for any currently known 3D technologies. By continuing to improve the stereoscopic viewing capabilities we are only solving one part of a much more complex problem for eventually creating virtual reality. We simply cannot get from where we are today to a true virtual reality experience by focusing only on the stereoscopic aspects of image presentation. We will at best achieve a more compelling “doll house” effect or something that resembles a window into a diorama. In some applications that will be quite interesting and useful, but in others it will be perceived as a distraction and will not be accepted any more now than the short-lived 3D movies of 50 years ago.

I would be interested to hear your views on the future of 3D displays. You may reach me by using your current 2D display and sending me an e-mail from this site, or by telephone at 425-898-9117.