What you need to know:
- Surfaceless operation: The new Apple Pencil could work on any surface, or even floating above it, using built-in optical sensors.
-
Trackball design: One version features a trackball tip that rolls as you move, followed by internal optical sensors instead of mechanical parts.
-
Universal Compatibility: Patent suggests it could work with iPhone, MacBook Pro, Apple Watch and more, not just iPad
-
Beyond drawing: The patent lists more than 20 features including security access, cursor control, and 3D object creation.
-
Detection range: Works up to 100mm (nearly 4 inches) above surfaces, allowing for true air extraction capabilities.
A recently granted patent suggests that Apple is working on a stylus that doesn’t need a screen or even a surface to work. The US Patent and Trademark Office just granted Apple US patent number 12,353,649, suggesting that the company is thinking well beyond the iPad when it comes to the future of the Apple Pencil.
Here’s what makes this patent a game-changer: According to Patently Apple, this next-generation Apple Pencil could allow you to write or draw in the air or on virtually any surface, like a wooden table or even the wall, and still keep track of your inputs accurately.
This isn’t just wishful thinking: We’re looking at some really clever technology that could make the Apple Pencil work with your iPhone, MacBook Pro, or literally any surface you can think of.
How optical sensors make magic happen
The heart of this patent lies in optical sensor technology that seems straight out of science fiction. The idea is to use optical sensors inside the pen that can track movement, tilt and position in 3D space. The patent even says this could work from a short distance, like 10 centimeters above a surface.
Apple’s patent describes two main types of optical sensors: optical flow sensors and speckle laser flow sensors. Optical flow sensors could detect spatio-temporal image brightness variation (e.g., frame-to-frame brightness variation) to estimate input device motion. Meanwhile, laser speckle flow sensors could detect speckle patterns resulting from coherent illumination of an imaging setup to estimate input device motion.
What makes it especially impressive is its range and versatility. According to Apple, both systems can detect motion even when hovering over a surface (up to 100mm or 3.94 inches), while also allowing the stylus to work on non-touch-sensitive materials like wood or paper. This 100mm travel distance means you can sketch preliminary ideas in the air before committing to the surface, or work on transparent materials like glass where sensors detect movement above rather than through the material.
The trackball spin that changes everything
This is where things get really interesting. One version of the Apple Pencil described in the patent uses a trackball at the tip (think old-school computer mice), but instead of mechanical tracking, Apple suggests using internal optical sensors to detect the ball’s movement. One version of the stylus described in the patent features a trackball tip, which rolls as the stylus moves.
According to Apple, this design also improves the signal-to-noise ratio and works on more types of surfaces, regardless of their texture or reflectivity. In user terms, this means greater precision and reliability, whether you’re working on smooth glass, textured paper, or even rough wooden surfaces.
The patent says that optical sensors can be configured to measure the displacement of a pen’s internal components, allowing it to operate even when it is not in direct contact with a surface. The trackball design also opens up possibilities for texture-sensitive input: imagine the pen detecting whether you’re drawing on rough paper or smooth glass and adjusting the line thickness accordingly.
Beyond drawing: a universal input device
It’s not just about making larger canvases for your art projects. The practical applications become really interesting when you look at the bigger picture. Imagine using the stylus to sign documents on your MacBook’s trackpad, navigation gestures on the back of your iPhone, or even authenticating by drawing your signature in the air.
Apple’s patent suggests combining this motion detection technology with other sensors, such as force detectors, inertial measurement units, and even machine vision. In simple terms, that means the Apple Pencil could eventually support air gestures, better handwriting recognition, or even the creation of 3D objects.
The patent has been granted a new patent that could radically change how the Apple Pencil works, allowing it to be used with any iPhone, MacBook Pro or even Apple Watch. Apple even lists 20 or more functions that an Apple Pencil could perform that have nothing to do with drawing. Significantly, Apple proposes that such an Apple Pencil could be used to allow “authorized individuals to access restricted areas of the computer or computer network.”
PRO TIP: Current Apple Pencil users should be aware that this technology might not be backward compatible: the optical sensors would likely require entirely new hardware.
What this means for iPad users
Look, patents don’t always turn into products, but this one feels different. As someone who has tapped the edge of an iPad screen mid-sketch more times than I can count, the freedom of working on any surface sounds really liberating. The key to Apple’s idea is that all the sensors necessary for the Apple Pencil to work would be integrated into the accessory. Apple’s patent emphasizes that this is not intended to replace touch-sensitive displays, but it frees the Apple Pencil from the limitation of the 11- or 13-inch iPad screen.
For digital artists, this could mean sketching concepts on physical paper during meetings and then continuing the work seamlessly on your iPad, with the Pencil bridging both worlds. A future version of the Apple Pencil could include optical sensors that would allow it to generate content on any surface. The patent describes drawing on any surface, allowing you to work on your desk while the Apple Pencil follows your movements and transmits data to the device. This could make the Apple Pencil less of an iPad accessory and more of a universal input device that works across the entire Apple ecosystem.
Where do we go from here?
Seriously though: this patent is pretty ambitious, even by Apple’s standards. But technical hurdles, like maintaining accuracy on uneven surfaces or handling varying lighting conditions, are exactly the kind of problems Apple’s optical sensing team has tackled before with Face ID and LiDAR. The main inventor listed on the patent granted to Apple is William Whitney, director of optical sensing engineering, suggesting that there is some serious engineering talent behind this.
The patent is credited to five inventors, proving that this is not just one person’s crazy idea. For such a seemingly specific topic, the patent is surprisingly broad in its range of propositions.
Will we see this in the next Apple Pencil? It’s hard to say. But the fact that Apple is thinking so much about stylus input suggests that they’re not done revolutionizing the way we interact with our devices. And honestly? Drawing on walls with precision tracking sounds great to me.