Apple’s new iOS 7 will, at least on the iPhone 5, next gen iPad and future hardware releases, deliver some very interesting new user interface elements. We know that many folks have seen the various pre-release look and feel capabilities. We know that some pundits have – with a certain level of silliness – decried this new look and feel, and we know that most iPhone users will end up loving the new UI approach and think back to the current look and feel of iOS as decidedly old school iOS. Stuff like the home screen parallax effect provides subtle new things that will change the way we perceive our iDevices. We predict it will all be for the better. We also predict no iPhone users will be running to Android because of it.

This will all happen in the immediate future, but as of yesterday, we may now be seeing hints of some cool new possibilities coming down the near term road that will move us forward into new realms of device interaction. We’re not speaking here of Samsung’s eye tracking capabilities or the simple hovering gestures the Galaxy S4 is capable of pulling off intermittently. We’re speaking more along the lines of the sorts of things wearable tech companies, such as Leap Motion, Thalmic and Dekko’s new zugmented reality operating system, will soon enough turn into reality.

And then there is Apple, which just yesterday received a very interesting new patent for a means to both create and manipulate - through proximity sensors - 3D images on a screen device without ever actually touching the screen. Towards this end, the U.S. Patent and Trademark Office (USPTO) on Tuesday officially published Apple patent No. 8,514,221 (Nicholas V. King and Todd Benjamin are credited as the inventors), which covers a specific method for generating and manipulating a three-dimensional object on a computing device, with the process controlled by special gestures made above a touchscreen display surface. In patent jargon it reads as flows:

Three-dimensional objects can be generated based on two-dimensional objects. A first user input identifying a 2D object presented in a user interface can be detected, and a second user input including a 3D gesture input that includes a movement in proximity to a surface can be detected. A 3D object can be generated based on the 2D object according to the first and second user inputs, and the 3D object can be presented in the user interface.

The only thing that is missing from the patent is a method for generating a hologram just above the display – now that would certainly be cool! But, we aren’t quite at that level yet. Nevertheless, the level of sophistication to accomplish the task of creating and manipulating 3D images while never touching the device is certainly a step in the right direction.

Ordinarily we would provide some images from the patent itself to illustrate various points, but in this case, the drawings do not necessarily make it easy to see how the manipulations would work. They are also more complex than is often the case. That said, the following drawing at least shows what we mean by using fingers away from the device surface itself.

It is important to understand that the user actually uses 3D gestures in real space in order to manipulate the 3D object that is created. Actual touch gestures for today’s devices do not require 3D movements – they are 2D movements of the swipe and pinch variety. Even Samsung’s “hover” gesture capability merely duplicates 2D screen touches. The new Apple patent is specifically about 3D manipulations.

It is interesting to note that the original patent application was only filed on July 9, 2012. This isn’t a patent that was filed a long time ago, based on ideas rather than real and available technology – as is often the case. This patent was filed with full knowledge of what not only today’s but also the near future’s display technology is capable of – which makes the patent something that is much more real than might otherwise be the case. It also falls well within Jony Ives’ sphere and periods of influence at Apple, which in turn means that there could be a reality attached to the 3D capabilities only a short bit off in the future.

The patent notes that a properly configured device – which would consist of a touchscreen sporting a collection of both capacitive touch and proximity sensors embedded in the display – would be able to detect and follow the location of a user’s fingers in space, though initial designs may require beginning any gestures by actually touching the screen. Note how the image above shows “screen touches” that move from screen to space, with fingers appearing once in space and off the screen.

Next, let’s return to the “patent jargon” description we provided above. Note that it clearly states that the 3D object begins first as a traditional 2D object that is created on the screen itself using the usual 2D gestures. This 2D object forms the initial reference point for where the subsequent 3D manipulations then take over. The 3D manipulations include 3D-based pinch, pull and rotational gestures created in the space above the display – as the drawing above suggests. The drawing below (our last, we promise), shows a bit more of the likely 3D movements a user might execute – in this case to create a 3D triangle.

We’ll stop here insofar as the actual patent is concerned. The patent itself has 31 claims and methods associated with it, and if you love to read patents, this one is a winner for you. Once you get beyond the initial sense of complexity, it all makes sense. But you don’t need to read the patent in order to understand what the patent delivers – and the above description is enough to get that across. What is important is that Apple has the patent in hand.

Innovation, Not a Gimmick

What we find extremely exciting here are the possibilities that are opened up for truly new and innovative ways to interact with our devices. Adding a third dimension to how we manipulate our screens will ultimately require much finer control over our screens. Anyone who has put to use (or tried to put to use) the limited capabilities of the Samsung S4 hover gestures will understand the limitations that still need to be overcome. A 3D approach begins to get us there.

Of course, such 3D manipulations also have numerous practical applications outside of our interactions with our devices. For example, an iPad or an iPhone – or even an iWatch – can become a real world motion controller. Here is a simple example of what we mean: Imagine getting rid of the remote control of a model-scale helicopter and using your iPad to fly it. Or perhaps manipulating how a robot operates in its own 3D space. Use your imagination!

The patent may, of course, never see the light of day in terms of any real Apple products. But with Jony Ive responsible for both hardware and software design, which includes how users interact with their devices, firmly in charge of where Apple is headed, it leaves us excited for what we may see in 2014 from Apple.

Sure, the danger of such stuff evolving into nothing more than yet more gimmicks is always a clear and present danger, even for Apple. But we’ll instead focus on innovative possibilities. If they are there to be had, we believe Apple will be the company that will most likely deliver on them.