What is Apple’s LiDAR Scanner? | 2021 Guide
I have always been fed up of using Augmented Reality apps and games on my smartphones as sometimes the experience is really terrible. When Apple announced that the new iPad Pro will feature a LiDAR sensor for better AR capabilities, I was thrilled, to say the least. Apple promised for way better AR performance and some camera improvements as well. So, What is Apple’s LiDAR Scanner? Let’s find out.
This news made me jump on the hype train and check the new iPad Pro’s and iPhone 12 Pro’s prowess with the LiDAR sensors. And when I got to use them, I was surprised at how much better the experience was compared to devices without a dedicated LiDAR sensor for AR.
After going through Apple’s official document’s and subjecting various Apple devices to stress test; my team and I found that:
Apple’s Lidar scanner is a compact device built into various device including the iPhone and iPad. It provides the user with unparalleled Augmented Reality (AR) experience, better camera capabilities and helps create 3D maps of your immediate surroundings.
What is Apple’s LiDAR Scanner?
Read on to know more about LiDAR and Apple’s implementation of it in the iPhone 12 Pro line up and iPad Pro (2020) lineup.
What is LiDAR?
Light Detection And Ranging (LiDAR) is a technology that uses lasers to scan and map surroundings. This is similar to how radar, the difference is that radar uses radioactive waves while LiDAR uses light waves. Check out our “What is Apple’s LiDAR Scanner?” guide for all the details.
And Radar is used for faraway objects while LiDAR is used for close-by objects because light gets absorbed easily. It is similar to ultrasonic imaging used by bats to see.
A LiDAR sensor emits hundreds of thousands of light pulses every second, LiDAR can scan distance and sizes of objects over small ranges. The data collected by the sensors is used to create 3D models, one of the best uses of LiDAR in construction and engineering in recent years.
LiDAR is now used in self-driving cars, for detection of objects and people. It is also available in automatic vacuum cleaning robots. In the yesteryears, LiDAR was used in military tech and astronomy.
LiDAR in Apple devices
With innovation and technological advancements, LiDAR has improved. The scanners have become cheaper, smaller and more accurate over time. This is helped LiDAR find its way into mobile devices like tablets and phones.
- Phones have been using Time-of-Flight (ToF) sensors for a similar purpose, but Apple went one step further and has introduced LiDAR for its iPads and iPhones. The reason was for better and more accurate distance imaging. Check out our “What is Apple’s LiDAR Scanner?” guide for all the details.
- The Apple iPad Pro (2020) and iPhone 12 Pro models have LiDAR technology in them. They enable better photography, Augmented Reality (AR), and 3D modelling capabilities for the devices.
LiDAR in iPhones and iPad
They way LiDAR is implemented on Apple devices are different when compared to other industry implementations. It uses the same principle, to find the distance between objects but in a shorter range.
- LiDAR on iPhones and iPads have an effective range of 16 feet (5 meters). It works both indoors and outdoors and allows for spatial tracking (tracking the location of people and objects around you).
- Apple wants the LiDAR scanners in its devices to provide you with unparalleled Augmented Reality (AR) experience, better camera capabilities and create 3D maps of your immediate surroundings.
- The LiDAR sensor with Apple’s primary cameras and processing capabilities, Apple intends to give you the best AR experience on a phone. Apple wants to move away from the popular Time-of-Flight (ToF) sensor found in other smartphones and Apple thinks LiDAR is the way forward.
- ToF uses a single pulse of infrared light whereas LiDAR uses hundreds of thousands of light pulses in a short period, and this is what more effective than ToF sensors. LiDAR gives better range and better depth and distance mapping information.
- And true to the word, the AR experience on the iPad Pro and iPhone 12 Pros are simply classes above other smartphones, there’s literally no competition in this regard. This also opens up for more future innovations for more apps and games can take advantage of this technology.
- All this is possible and speedy only because of the processing capabilities of the Apple devices that use LiDAR. The A12Z Bionic found in the iPad Pros and the A14 Bionic found in iPhone 12 Pros enable for this process to happen fast enough so that there is no delay and there is no poor user experience in using LiDAR.
- As of now, the only limiting factors seem to be the sensors itself as they lack resolution for more detailed workflows. But this will be improved and for a first-generation implementation, the LiDAR scanners in the current set of devices work really well in boosting AR performance.
And in reality, Apple’s intention and use case for their LiDAR sensors was for AR gaming and applications and some camera improvements.
Use of LiDAR in Apple Devices
Apple also said that every existing ARKit app will benefit from the new LiDAR scanners. This will improve the speed, occlusion, tracking and motion capture for ARKit apps. An update to the ARKit to the new Scene Geometry API will allow developers to benefit from LiDAR scanners in their apps. Check out our “What is Apple’s LiDAR Scanner?” guide for all the details.
- The AR experiences of games like Pokémon Go and apps like Snapchat are leaps and bounds better on Apple devices featuring the LiDAR scanner compared to other smartphones and tablets. Hot Lava is a game that uses Apple’s LiDAR scanner to bring molten rivers to your home.
- Ikea’s Place app also is really cool to take on AR and it allows you to check how their furniture’s would look in your home. This is like a real-life version of The Sims but it is helpful nonetheless aiding you in purchasing the best furniture for your home.
- LEGO also has launched products that come alive when scanned via your smartphone. And undoubtedly these all work excellently on the new iPad Pros and iPhone 12 Pros.
- Apple’s own Measure app now is improved because of the LiDAR scanners. It is now faster and easier to calculate a person’s height with the Measure app.
- It also increases accuracy and helps you measure objects as well with guidelines for easier reading. The Measure App now has a Ruler view for more close and accurate measurements and you can save the measurements with screenshots for each measurement.
- Apple has said that the low light performance of cameras in the new iPhone are helped because of LiDAR. Apple’s ‘Focus Pixels’ (similar to Phase Detection AutoFocus PDAF) from iPhone XS depends on the light in the environment for good results and since at night there is not enough light, this autofocus method didn’t work so well.
- This is where LiDAR will aid in camera performance. LiDAR can tell the image processor to focus at the correct distance for the best results. This helps for better photos in the dark when combined with the already excellent Night mode.
What is Apple’s LiDAR Scanner? – Takeaway
Will LiDAR on smartphones and tablets be the future? Will other smartphone manufacturers follow Apple’s footsteps? Will it actually be the game-changer for AR and Low-light photography that Apple expects it to be? Check out our “What is Apple’s LiDAR Scanner?” guide for all the details.
All of these questions can be answered with a yes and a no. LiDAR can be the future in smartphones and tablets if the technology gets better and better if higher fidelity scanning can be done by the LiDAR scanners on the devices.
If it doesn’t get better or good enough for more applications, it will die like other countless tech that gets introduced. I feel that this is an experiment by Apple and they are thinking that LiDAR will be a game-changer and give them an edge in AR and camera capabilities for their devices.
Apart from Apple, app and game developers have to implement and create software that takes advantage of this technology for it to really become something big. This will also make other smartphone manufacturers follow suit and come up with LiDAR or alternatives to improve their game in the AR field.
If what Apple wants from LiDAR actually pan out well, in terms of AR performance and the camera department, I can see LiDAR being a staple in the world of smart devices. Rumors are floating around that we can see LiDAR implemented in more than just iPhones and iPads. Apple’s rumored AR glasses might feature LiDAR for AR implementation.
Should you upgrade for LiDAR? I don’t think LiDAR is a feature you need but from what I tested it is a nice to have. If you are someone that uses a lot of AR apps, you can benefit from it, but for the average consumer, more apps and games are needed to wow them and pull them.
It can be something game-changing in the future and there is a lot untapped potential which I am sure Apple and app developers would love to work on. It is exciting a new feature that needs more time to see its full potential.