VisWiz app allows blind users to crowdsource visual descriptions in real time

[kestrell]

VisWiz app allows blind users to crowdsource visual descriptions in real time
Originally posted to the MacVisionaries mailing list; more links after the article extract.

New Scientist article
http://www.newscientist.com/blogs/onepercent/2011/05/smartphone-app-lets-the-blind.html

block quote start
Designing a computer program that can reliably recognize text and distinguish objects in the real world has proven to be a massive challenge for artificial
intelligence researchers. To get around this, the researchers behind VizWiz - a team consisting of computer scientists from several universities, including
the University of Rochester - decided to outsource the task of problem-solving to people: specifically, to Amazon Mechanical Turk's masses of online workers.

To make sure users get answers as quickly as possible, the researchers programmed an intelligent queuing system they call Quik Turkit to speed things up.
Quik Turkit recruits Mechanical Turk workers even as a VizWiz user is taking a picture, so someone is always ready to answer an incoming query.

Eleven blind iPhone users tested out VizWiz, asking questions like: "What denomination is this bill?", "Do you see picnic tables across the parking lot?",
and "What temperature is my oven set to?"

They received an average of three responses per query and waited an average of 133.3 seconds for the first answer. The first answer received was accurate
or helpful in 71 of 82 cases. By the third answer, all questions were correctly answered.

In a second test, the volunteers got to use VizWiz 2.0, which includes improved image processing techniques. Their response time was cut to an average
of 27 seconds.
block quote end

WisWiz app at Apple store (available in U.S. and UK)
http://itunes.apple.com/us/app/vizwiz/id439686043?mt=8

VisWizweb site
http://vizwiz.org/

Vizwiz tutorial (helpful to read through first, since the app just opens with the camera)
http://vizwiz.org/tutorial.html

Research project at the University of Rochester
http://hci.cs.rochester.edu/currentprojects.php?proj=vw

Comments

[jesse_the_k]

This is AMAZING!

Back in the Raised Dot days we'd joke that hiring a tech-savvy at-risk high schooler would be less expensive and more morally just than buying a Kurzweil Machine (then $30k.)

I adore this crowdsourcing thing, although the recent weather-related catastrophes do remind me that the cell network isn't that resilient. On the other hand, interdependence is OK.

I almost always have an MacBook Pro warming my lap and an iPod Touch in my bag, so I'll be happy to provide any old fogey advice I may have to offer.

[kestrell]

Re crowdsourcing and cell phone availability during disasters: true, this resource may not be available or even useful in every instance, but I think of it as the more tools you have in your tool kit, the more likely you will have the one you need when you need it. Also, the dynamics of everyday small hurdles can be almost as stressful as that of disasters--during even small disasters, such as a MBTA emergency, most people feel in the collaborative helpful mood, but when I need somebody to tell me what denomination of bill I am holding in my hand, I can't find anyone. And those "this job would only ake me about five seconds if I had eyeballs" moments tend to pile up on some days.

By biggest hurdle re the iPhone has been getting used to the tactile/gesture interface. A book about learning this interface came out through National Braille Press sometime during the past year, and I keep meaning to read the book, especially as Alexx has an old iPhone with which I can practice. Also, the reason Alexx gave up on the iPhone was that the coverage in our area is bad--Alexx often couldn't call me within our house. I need to find out if that has changed.