had never heard of it! very interesting! would like to talk on a phone call may be.
On 5/27/22, Kanchan Pamnani <[email protected]> wrote: > Pranav, > Will it tell me that I am going out of my lane while swimming? > I go to a club that is very particular that swimmers stay in their lanes and > do not even for an inch go into others space. > K > -----Original Message----- > From: [email protected] [mailto:[email protected]] > On Behalf Of S Solomon Karuppannan > Sent: 27 May 2022 17:29 > To: [email protected] > Subject: [AI] I can see but cannot drive, what good is synthetic vision? > > I am not afraid of copyright infringement for it was written by someone we > know! > -- > With warm regards > Solomon S > [email protected] > > > I can see but cannot drive, what good is synthetic vision? > May 26, 2022 by pranav —1 Comment > Foreword By Chris Hofstader > This article is a guest post written by my friend Pranav Lal. He is an > expert user of The vOICe, a sensory substitution technology that > allows blind people to experience some vision via sounds. I find this > technology highly compelling, I think it's a better solution than are > the various implants we hear about and I think Pranav gets it right in > this article. > Pranav's Story > Many well adjusted blind people wonder what is the use of vision? A > large number of us want to drive or perhaps to fly an airplane. Yes, > other applications like face recognition and searching for people in a > crowd come to mind. There are apps for some of these tasks while > others are possible in highly specialized conditions. If you are a > blind parent reading this, then it is indeed possible to give your > child a form of sight without any surgery. > If you are an individual who is late blind, you may get quite a lot > out of this article since you may be able to get vision like > sensations. > What I am going to do in this article is to talk about my use of > synthetic vision and sensory substitution thanks to a freely available > application called The vOICe. Yes, you read that right, it is possible > to have vision for free assuming you have the requisite hardware, > which in many cases you do because all that is needed is a smartphone > or a webcam attached to a pc. > So, how do you get vision? One popular way is to stick wires in the > eyes or in the brain and then to stimulate them in specific patterns. > This is loosely the approach those headline grabbing eye implants > make. > Another way is to forget about vision altogether and use artificial > intelligence to convey only relevant information to blind people. This > is the approach that many applications on mobile phones use. > Both the above approach have significant disadvantages. The primary > disadvantage of the implant approach is that it needs surgery with its > attendant complications. In addition, the electrical stimulation can > lead to Seizures with brain implants for vision. Finally, the > resolution of these implants at the time of this writing is very low. > When it comes to artificial intelligence based approaches, the problem > is that they may filter crucial input. E.g., most scene recognition > apps will tell you that there is a sofa and a chair in front of you > but will not tell you where these objects are. > Enter sensory substitution > So, what can we do? How do we overcome the demerits of both the above > approaches? One way is called sensory substitution. This involves > translating the sense of sight to another sense that we can perceive. > There are 2 possibilities namely the sense of touch and the sense of > hearing. Both can be used to convey the same information. > Researchers have used both the above approaches. Sensory substitution > started in the 1960s with Dr. Paul Bach-y-Rita building a chair where > users could feel the shapes of objects on their backs via a series of > plates. > Dr. Peter Meijer is a researcher who has built a system that converts > images to sound via a program called The vOICe. > Enter The vOICe > In 2001, I got my first laptop, an Acer travelmate 720, I think. It > came with a free camera. That is how I started with the vOICe, an > application that yields a form of low vision. Before we go any > further, I am totally blind, that is I have no shape and or object > perception. I can see light with my left eye. > I was born premature and was diagnosed with retinopathy of prematurity > when I was 3 years old, I believe. I should really check. The point is > that I do not have any functional vision. > I got blindness skills along the way and am happy with them. I got > into synthetic sight out of curiosity. > How does The vOICe work? > The vOICe takes any image, it does not matter if it is a static image > like in a photograph or streaming images like from a camera and breaks > it down into a series of sounds that have a specific meaning. As a > user, my job is to interpret the sound and infer what I am looking at. > There is more to this because this processing becomes subconscious > with sufficient practice. > See the below table. > > Sound Characteristic Meaning > > Panning Objects on your left are sounded in > your left ear while objects on your > right are sounded in your right > ear. Objects in the center are > sounded between both ears. This > assumes your camera is facing in > the same direction as you are. > Frequency The higher the frequency or pitch > of the sound, the more elevated is > the object in the frame. > Volume The louder the sound, the brighter > the object. > > The vOICe pans automatically from left to right of the frame thereby > giving you a symphony of different pitches, volumes and positions. > This sounds potentially complicated, but it is a bit like eating food. > Try describing the steps to put one morsal into your mouth and you > will begin to appreciate how many things you do unconsciously. > In the case of The vOICe, a horizontal line sounds like a flat tone > that moves from left to right. > A vertical line sounds like a short click at some place in your head > because remember, that vertical line is going to be somewhere in the > frame. > The role of your brain and neuron plasticity > Remember, I said that the interpretation of the sounds will become > second nature, here is how that happens. Yes, practice is one factor > but there is more going on here. We need to digress a bit and explain > how vision works. > If you have working eyes, the retina sends electrical impulses to the > part of your brain responsible for vision via the optic nerve. This > part is called the visual cortex. These impulses come in certain > patterns which is how your brain knows that it is seeing shapes. Yes, > I am glossing over details but then I am not a neuroscientist, and > they are not relevant for my explanation. When you hear sounds, they > too are translated into electrical impulses. This is true for all > sounds. When you hear sounds from The vOICe however, the electrical > impulses that they get translated to are the same as those you get > from the retina. The brain does not care what is the source of the > impulses so whether the eyes give it those impulses, or the ears give > it those impulses. It knows that it has special neurons for processing > these patterns and it begins to use them. > The ability of the brain to start using these special neurons is > called neural plasticity. This is how getting the meaning out of the > soundscapes becomes second nature. > What have I done with vision? > It is important to remember that when you begin to use The vOICe, you > are not magically going to become adept at seeing. It is not like > lights on, and I see. You will get all the visual information, but you > will take time to learn what the program is telling you. There are two > keys to success with The vOICe namely sticking with it and asking > questions about what you see. > I am not exclusively dependent on The vOICe. I use it as a compliment > to my blindness skills. It is unlikely that you will reach the stage > of a fully sighted person where you are exclusively dependent on > vision. The idea here is to use the sense of vision to add to your > sensory toolbox. > I am going to talk about several things that I have done which have > added significantly to my quality of life. > Saving electricity bills by turning off the lights > The first and perhaps simplest application of The vOICe is that of a > light probe. Given today's focus on sustainability and rising energy > prices, this is more important than one would ordinarily realize. I > can walk around my house and figure out if lights are off or on. If > they are on, I hear sound. Never mind what the shapes that I am > looking at. If there is sound, the lights are on. No sound, the lights > are off. > What is around me > One way to figure out what is around is to use your GPS app. However, > GPS does not label landmarks like trees, cars etc. I did not know > until I began using The vOICe that I lived on a street with so many > trees. Yes, if I had used a cane, I would have encountered some of > them, but vision gave me that broader view without the hassle and > sometimes painful collisions with stuff on my street. I am not saying > that The vOICe will replace the cane. It will not, but it is a > mechanism to acquire more information. This has all kinds of other > applications such as finding my own house assuming it has a defining > visual characteristic. > Adapting to dynamic changes in the environment > This is one of the more exotic but fun applications of sight. I was > riding an elephant which was moving on a forest trail. The elephant > would just push branches aside with no regard to its human occupants. > Sighted people could duck if they remembered to do so. I used The > vOICe to track and whenever I saw the trees becoming really dense, I > would duck and avoid those branches. How I knew that the trees were > becoming dense was because of the number of objects I could hear in > the view. The more structures I saw, I knew that forest cover was > close and of course, the light level would also drop. > Online shopping > I am able to see the photographs of what I am buying on online > shopping sites like Amazon. This is particularly handy when buying > computer keyboards, shoes, etc. There are ways also to tell the colors > in an image and this can be done experientially via expressing colors > on a musical scale or you can have The vOICe tell you what is the > color of an object in the center of the frame. > Judging houses > I was having to look for rented accommodation which meant visiting > various houses. I could get a sense of the place with my regular > blindness skills but used vision to see the walls without touching > them and got an idea about how well the place was being kept. The > sounds allowed me to judge if the walls were painted. I expected to > hear a smooth sound but, in some houses, I could sense a lot of breaks > in the sound which meant that they probably had peeling paint. > Similarly, if I did not get a clean sound or if the volume of the > sound was too low, then it meant that the house was really dusty and > did not have enough light. > Navigating to the facilities in a new office > There was this time when my office moved to a different floor in the > same building. The layout of the floor was different and finding the > toilet was a problem. I had to take a turn from between 3 cabins. The > turn came just after a column. I was able to use The vOICe to find the > column via its distinctive sound and turn. No need to feel doors that > may be opened or closed etc. I extended the same principle to navigate > my house when it was being remodeled. I did have a good sighted guide > at that point but it is only after using The vOICe that I realize how > much sighted guides fail to tell us. This is not deliberate but there > is a lot of filtering that goes on. > Driving > Distance estimation is tricky with The vOICe because it uses a single > camera for now. However, Jacob Kruger has used The vOICe to stay > oriented while driving a motorbike on a race track. He used the color > filtering feature of The vOICe to determine if he was deviating from > the track. He was also able to detect turns. He did have a backup > rider who was communicating with him over a helmet radio, but Jacob > anticipated many of the turns on the track. > Yes, a race track is a very controlled environment but the point here > is that The vOICe can be one of the tools to help blind people drive. > Walking with The vOICe > Many of us have tried walking several times with The vOICe. We still > use the cane and the key thing to remember is that The vOICe answers > the question "what". The cane or any other sonic or laser obstacle > detector will tell you that there is something in the sensor's field, > but it will not tell you what that thing is. > Playing astronomer > I have had and continue to have a huge amount of fun looking at images > from space. Thanks NASA and the BBC. It is possible to manually review > an image in small increments which allows me to appreciate every > detail. I then use the color probe to do some amateur analysis on the > image. This becomes even more relevant when new discoveries like those > of black holes are made. I have been able to participate in > appreciating the images that have been published by NASA and other > agencies. > Photography > I take photographs of things I find interesting and publish them. This > becomes second nature to you because taking pictures is the only way > you can accurately convey what you are seeing. I developed this skill > during my first days of using The vOICe when I used to ask the members > of the seeing with sound mailing list to look at the image and confirm > my interpretation of that image. > Another unrelated aspect of photography is that The vOICe helps you > being photographed. This is because you are able to better orient > yourself to the camera lens since you can see a small circular blob in > the image or a bar shaped object which is a human arm when taking a > group selfie. > Museum and monument accessibility > I have successfully looked at objects inside glass display cases > whether they have been ancient artefacts in museums or watches in a > shop. The thing to note about museums is that you do not need any > infrastructure to facilitate accessibility. You build adequate > lighting, clear documentation and give good contrast so that people > can see and that is all you have to do. This is not to say that work > on accessible museums is irrelevant; far from it but at the same time, > you do not need to wait for infrastructure to be established before > blind patrons can appreciate the content. > Building my own eye > As technology has evolved, I have assembled my own eye. My first setup > was a laptop computer in a backpack while these days, I use a pair of > off-the-shelf video glasses. I have used single board computers like > the Odroid and the Raspberry Pi to use as eyes. This means that you > can have sight probably in the sub \$200 price point if you use an > Android phone. > Other capabilities > The Android version of The vOICe, besides being a tool for vision, can > read out documents which means that it is a terrific tool to tell you > what shops you are walking past, and you can also read advertisements > and other text. The same applies to menus in restaurants etc. It also > has GPS and can speak the compass orientation and street names as you > move around. > Conclusion > I have never hankered after vision. However, now that I do have it, I > do not want to lose it. The vOICe keeps pace with changing platforms > and given its flexibility, I can choose my eye for my needs. No other > artificial vision solution has this flexibility. One point I should > cover is about software that describes the scene like a human does to > a blind person. These programs do help but they leave out crucial > detail and orientation information. For example, if there is a car and > a chair on your driveway, how do you know where the chair is with > respect to the car? > A vision revolution is overdue and one way to add spark to the fire is > by using The vOICe, improving your quality of life and then talking > about your accomplishments. > References > The website of The vOICe > My blog where, amongst other things I document my experiences about the > vOICe > Blind tech adventures, a youtube channel where Nimer Jaber documents > his experiences with the vOICe > The wikipedia entry on sensory substitution > > -- > Disclaimer: > 1. Contents of the mails, factual, or otherwise, reflect the thinking of the > person sending the mail and AI in no way relates itself to its veracity; > > 2. AI cannot be held liable for any commission/omission based on the mails > sent through this mailing list.. > > > Search for old postings at: > http://www.mail-archive.com/[email protected]/ > --- > You received this message because you are subscribed to the Google Groups > "AccessIndia" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/a/accessindia.org.in/d/msgid/accessindia/CAH%3DCT_9gFzEY5uo4VKx%3DYW12JN5tW886Bg8Y_q7RuVsqKYxVRg%40mail.gmail.com. > > > -- > This email has been checked for viruses by AVG. > https://www.avg.com > > -- > Disclaimer: > 1. Contents of the mails, factual, or otherwise, reflect the thinking of the > person sending the mail and AI in no way relates itself to its veracity; > > 2. AI cannot be held liable for any commission/omission based on the mails > sent through this mailing list.. > > > Search for old postings at: > http://www.mail-archive.com/[email protected]/ > --- > You received this message because you are subscribed to the Google Groups > "AccessIndia" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/a/accessindia.org.in/d/msgid/accessindia/00a801d871ca%24f75c6670%24e6153350%24%40gmail.com. > -- "every human is disabled because, none has got the wings to fly". Anirudh Rao your intellectual comrade. call/whatsapp me at 8277012082 join on Skype: anirudhrao3 if you just wish to be a facebook friend: click https://m.facebook.com/anirudh.rao.7731/about?lst=100005148125152%3A100005148125152%3A1563341090&refid=17 and send a request. Thank you Anirudh Rao. -- Disclaimer: 1. Contents of the mails, factual, or otherwise, reflect the thinking of the person sending the mail and AI in no way relates itself to its veracity; 2. AI cannot be held liable for any commission/omission based on the mails sent through this mailing list.. Search for old postings at: http://www.mail-archive.com/[email protected]/ --- You received this message because you are subscribed to the Google Groups "AccessIndia" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/accessindia.org.in/d/msgid/accessindia/CAF6avGhEogwqkXdG30M-a-JUmY2vx622iYY0B5dzAtQ9RQ0upg%40mail.gmail.com.
