Q: I heard that the Google Captchas are used to train their neural networks so they can identify things like traffic lights, etc. How does this work since our thought process is in our brain?
Excellent question!
Google uses Machine Learning (ML) techniques to build smarter systems, for example self-driving cars that are better at detecting traffic lights. ML is often called “AI” in the media, but that’s like saying I’m a movie star just because I appeared in my mom’s birthday video.
How does ML work? ML is not magic. ML is “making predictions based on past data”. For example, suppose you had a million pictures of cats, and a million pictures that definitely did not include cats. With a ML system, you feed it the cat pictures and say “these are cats!”. You then feed it the million definitely-not-cat pictures into the system and say “these are definitely not cats!” This is called “training the system”.
Now that the system is trained, you can show it a picture and it will tell you “I’m 30 percent sure that’s a cat” or “I’m 80 percent sure that’s not cat”. It doesn’t really know what a cat is. It just knows the picture looks similar to the big pile of cat photos.
Also notice that it has a percentage. It never says, “Yes, that is a cat. I’m 100% sure”. That’s called a “confidence rating”. You might only accept answers that are 80 percent confident and higher… or maybe throw away answers unless they are 99 percent sure. Determining what percentage to use is one of the most difficult parts of ML. Would you trust a doctor that was 60 percent sure you needed surgery? 90 percent sure? 40 percent sure?
How do you make a system like this more accurate? More data. Don’t feed it a million pictures of cats, feed it a billion pictures.
ML is enormously useful across many applications, but sometimes it can act like The Sorcerer’s Apprentice. For example, using ML to sort resumes based on past hiring decisions can amplify a racist hiring history, even without any intended bias. You feed it 10,000 resumes of people that you hired and stayed at the company for 3 or more years (a reasonable definition of success); then you feed it 10,000 resumes of people that were hired and left soon after. Now you can show it a resume of someone you are thinking of hiring.
It says “I’m 80 percent sure this looks like that first category.” Should you use that to make your hiring decision? No. What if people that leave the company right away leave because once they arrive, they discover that there is systemic distrust of black people and get the hell out of there. Well, now you’ve built a system that won’t hire black people. Or what if you want to hire new people… maybe new skills? What if the people that were leaving early left because the company was unethical and the only people that stuck around were ok with that? If your goal is to start hiring more ethical people, you’ve just built a system that won’t let them in. What if you make cars and want to also make motorcycles? In the past people that love motorcycles and were trying to convince the company to make them left in frustration. Now you have a new CEO who wants to go into the motorcycle business and you’ve built a system that prevents you from hiring the motorcycle-loving people that are needed to turn the company into the best motorcycle maker ever?
An ML system doesn’t think like a human. It is a system that predicts based on past experience. That’s exactly what you want to detect traffic lights.
Therefore, it’s a great application for Captchas. Every time you pick 4 photos with traffic lights and 8 that don’t, you’ve helped Google train its self-driving cars.
Google was successful with Captchas with many applications: recognizing text for their book scanning project, recognizing traffic elements for self-driving cars, etc. Maybe you’ll think of another way to use Captchas and Google will hire you!
If you do a google search for “beginning machine learning” you’ll find a number of books that will help you get started.
