Seeing AI is a fantastic iPhone application developed by Microsoft, which makes use of your iPhone’s camera to provide a variety of extremely useful functions to assist the visually impaired with a wide number of tasks. We will explain each of the different functions below, and we will also have video demonstrations of each of these functions coming soon.
Short Text Channel
When you install Seeing AI for the first time and open it, you will be given a tutorial of the app and all the different functions it provides. The first channel you will come to is called “Short Text”. The idea of this channel is to allow you to read a very small piece of text. As an example, if you have some envelopes in front of you, you could use the short text channel to try and find out who they are addressed to.
To use this, simply hold your phone’s camera close to some text that you would like to read and it will start reading the text to you. When you use this function, you might hear the phone repeating itself many times. This is because it is constantly scanning for new text and if you move your phone even very, very slightly, it might pick up something new and start reading from the top again. This doesn’t always provide the best experience, it can be very annoying when it was just getting to the part you wanted to hear and then it starts over again, but it is completely normal and doesn’t mean you’re doing something wrong.
This function does not require a data connection to work, so you could use the Short Text feature even if you don’t have wifi or a 3G/4G signal.
The Document Channel
Next up is a channel known as “Document”. This is very useful for… Well, documents! If you would like to identify text on an A4 document, this is the channel you will want to use.
The app will give you directions as to how to move your phone so that it can get a clear picture of the document, then as soon as it can see the document clearly, VoiceOver will say “Hold Steady”, it will snap a photo, process it, and will return with the text that was on the document.
This feature requires internet to work, so you must have a wifi, 3G or 4G connection. Also the length of time it takes to process the document will depend on your internet connection, so try to be patient if it seems to be taking a while.
The product channel turns your iPhone into a barcode scanner, so you can scan the barcode on product packaging and get more info about that particular product. For example if you scan the barcode of some food packaging, you will hear what the product is, along with storage and cooking instructions.
Barcodes are very small and not usually tactile, and the location of the barcode always varies, so they’re not easy to find, but Seeing AI does its best to give you some guidance. Hold your phone’s camera just above an item that you would like to scan the barcode on, and if Seeing AI can detect a barcode, you will hear a series of short beeps that will increase in speed as it can see the barcode clearer. Try to follow these beeps the best you can. For example, if you hear a few beeps and move your phone to the right and the beeps stop, try a different direction until the beeps start to get faster. Eventually, once it can see the barcode clear enough, VoiceOver will say “Processing” and it will scan the barcode and see what it can find. After a couple of seconds, it will return with just the product name, and a “more info” button if more information about the product is available. Going back to the food example above, double tapping on this button will present information such as storage and cooking instructions.
This is another function which requires your phone to have an internet connection to work, and again, the speed of your connection will affect how long it takes to retrieve information about an item. From testing, a lot of barcodes simply return the result “Not Recognised”. This means that particular barcode or product is not in whatever database the app uses for its information and there’s not much that can be done about this unfortunately. However, I have noticed this happening less and less as the app has developed, so I assume new barcodes and products are being added.
This channel allows you to take a picture of someone and Seeing AI will try to describe them, including their features, facial expressions and even their age, which can give some interesting results. I just took a picture of myself and I’m apparently a 22 year old man with brown hair. I’m 19, so that could have been worse…
When you use this channel, Seeing AI will describe how many faces can be seen in the photo as you move the camera around. Once you can hear the number of faces you would like to take a picture of, double tap the “take picture” button. Unlike the other channels, you will need to use the “take picture” button as it will not take a picture automatically. As with all the other channels, it uses the back camera by default, but you can also switch to the front camera by using the “switch camera” button, if you wanted to take a selfie and have it describe what you look like. This channel requires an internet connection to work.
This channel couldn’t be simpler, hold a pound note up to your phone’s back camera and Seeing AI will tell you how much it is e.g. £5, £10 etc. You can change the type of currency you are trying to detect as well, so if you are on holiday and need to scan euros or dollars, simply change the setting using the “Recognising” button and you’re good to go. This channel does not require an internet connection.
This channel allows you to take a picture of the general scene around you, and Seeing AI will try to describe what is around you and what is going on. I have not used this feature much personally, but from brief testing it seems to work very well and could be useful in an unfamiliar environment if you wanted to know how many people are around you or what you are stood in front of. This function requires an internet connection to work, and you will need to double tap the “Take Picture” button when you are ready for a picture to be taken.
This channel works similar to some colour identifying apps or devices you might have come across. Simply hold your iPhone camera to an item, and it will tell you what colour or colours it can see. Very useful for identifying the colours of your clothes for example. This function does not require an internet connection.
Finally, we have the light detector channel, which is very useful for detecting how much light is around you. As soon as you select this channel, you will hear a tone. The higher the tone, the more light your phone can see, and the lower the tone the less light there is. If you know where the lights are around your home, you could hold your phone up to the light to determine whether the lights are on or off. If the tone is very high when you hold your phone up to a light, you know it is on, and if it is a lower tone the light is off. This does not require an internet connection to work.