How To Use Visual Lookup In iOS 15 To Identify Plants & Pets
Select iPhone models can take advantage of Visual Lookup, an iOS 15 feature that can identify the plants and animals within a photo taken in the U.S.
BY BRADY SNYDER, published July 2, 2022
Plants and animals can be identified by select iPhone models running iOS 15 or later in an impressive demonstration of Apple’s Visual Lookup feature, which allows users to learn more about famous landmarks, art, plants, flowers and pets. The feature was introduced alongside Live Text as part of an iOS 15 release that focused on increased actions within photos using on-device intelligence. Due to the computing required on the iPhone, it makes sense that only specific iPhone models can take advantage of this feature. By keeping as much computing on-device, Apple can protect user privacy by keeping it off the company’s servers altogether.
Both the Live Text and Visual Lookup features scan objects within a photo saved to the device’s camera roll. Their functions are different. While Visual Lookup aims to provide information about a subject — perhaps to identify it or add more background information — Live Text extracts data from a photo. Users can view the text in an image that is compatible with Live Text, which will likely be found in prominent signs or captured documents. Live Text can be copied, searched on the web, or shared with other people and applications. But for unidentifiable plants and animals within a photo, Visual Lookup can fill in the gaps and add more information about flora and fauna captured on the iPhone.
New Features & How To Use
To see if a photo has elements that can be detected by Visual Lookup, open it in full screen on the Photos app. Every photo saved in the Photos app will have an ‘i’ button located at the bottom of the screen when it is opened. However, when a photo has detectable elements with Visual Lookup, the ‘i’ button will be partially covered by two floating star icons. Tap this icon to bring up the photo details, which offers the option to add a caption and can show lens information used to take the photo. For example, when a plant or animal is detected, a ‘Look Up — Plant’ or ‘Look Up — Animal’ will be displayed directly underneath the caption bar. Tap these buttons to bring up Siri’s findings related to the subjects, including the plant or animal name, photos, scientific name and other information.
Though the current Visual Lookup features are available on iOS 15 versions, it is slated to receive improvements as part of the upcoming iOS 16 release in the fall. With the new version of Visual Lookup, users can copy and paste the subject of a photo for use in other applications. It is also possible to drag and drop the subject of a photo to other applications, such as a Messages field or in a document. On beta versions of iOS 16, the implementation works well but sometimes misses a part of the subject in the cutout. The feature will likely be refined as iOS 16 continues development and is expected to see a public release in Fall 2022.
Heading outdoors? Apps like LeafSnap and Picture Insect can help identify plants and animals.
8:00 am EDT Jul. 2, 2022
Summer means spending as much time as possible outdoors. Whether you have a postage-stamp yard in the city or acreage out in the country, knowing which plants and animals share your world can be a pretty incredible discovery – and help keep you safe.
But who has the brain power to remember every leafy green and furry friend we come across? Nobody, that’s who.
This is yet another case where our smartphones save us. With a handful of slick plant and animal identification apps, you’ll be spotting, naming and learning about the creatures that share your space in no time.
Let’s start with the easiest one… well, sort of. Once you pull out your smartphone to catalog flora and fauna around you, you realize that animals move fast, but plants stay put. The tricky thing is that there are so many plants that look similar that you need an app to help.
I use PlantSnap (iOS, Android, free with in–app purchases) because of how robust the app is. It not only identifies over 600,000 plant species ranging from trees to flowers to mushrooms and more, but it also holds your hand through the photo-snapping process. The app literally tells you if you’ve framed the subject well in the photo and teaches you how to get the best identification results.
Somebody’s watching me: How to spot hidden surveillance cameras in vacation rentals
LeafSnap (iOS and Android, free with in–app purchases) is another great option. It can identify roughly 90% of known species of plants and trees, which should cover just about anything you’d run into in your back yard, park or nature trail and it includes extensive plant care features like watering and fertilizer reminders, a plant journal with photo support and a plant care calendar.
I also use the PlantIn and PictureThis apps to figure out which plants I have in my backyard or run across on the trails.
Birds are so much fun to spot that there’s an entire industry built around the hobby. They’re cute, colorful and sing, so what’s not to love? Unfortunately, they can be super hard to identify because they move fast and don’t tend to get very close. Don’t worry – I have a secret weapon.