Apple is ready to take stylus interaction to a new level, using a camera and sensors. A new patent has come up, showing the ability to perform doodles in mid air.
This stylus would work on any surface, not just a touch sensitive one. The documents mention a pen that can be monitored by a force sensor in the tip or tracked using motion and orientation sensors, or a camera. There’s also the alternative of an electromagnetic or sound based triangulation scheme.
This could mean that the computer would wirelessly monitor the stylus from multiple positions to find its location in 3D space. The pen could be used to generate text or 3D drawings, not limiting you to 2D. Drawing in mid air is actually pretty cool and sounds one step closer to Minority Report interaction.
The patent was filed back in July 2016 and published in late January. Some drawing examples are shown in the diagrams in this article. Apple already sells one stylus, dubbed Apple Pencil, but that one is limited to the iPad Pro interaction. There’s plenty of stylus action going around, with the S Pen, Moleskine Pen+ and more.
However this is the first time that drawing goes digital without a surface. We’ll see if it becomes real or not.
