Useful Ideas about Machine Learning
It is clear from the rapid advances in machine learning (ML) over the past decade, which have produced programs that can play Go, translate languages, and even detect cancer from images better than any human, that ML will have countless beneficial uses, and that, just as “software is eating the world” today, ML will eventually do the same.
That said, I have little confidence in predicting how long that process will take or what the eventual future will look like. However, in trying to grapple with these questions about which problems are best solved by ML versus solved directly by humans, I have found the following ideas to be especially useful.
1. ML Outperforms with High-Dimensional Data
We are all aware from personal experience that human brains have limited ability to process data. Showing us too much much data can lead to information overload, where we have difficulty understanding and cannot make effective decisions. By necessity, then, we must limit the amount of information that we consider. That results in bounded awareness, which can lead to failures where we ignore highly important, readily available data.
The idea that “too much data” is a problem for human brains is overly broad, however, because there are some settings where we can easily cope (albeit with the help of computers) and others where we cannot. We need to make a finer distinction to separate these cases.
When looking at a standard table of data, let’s separate the size of the table into two dimensions: height and width.
The height of the table (the number of rows) tells us the total number of examples we have. The width of the table (the number of columns) tells us the amount of data we have describing each individual example.
While a large number of rows is certainly a problem for human minds, it is one that we are comfortable with and have means of addressing. We can understand a column of data by calculating summary statistics such as its mean and standard deviation. Better yet, we can plot a histogram of the numbers to see the full range of values.
If the column of numbers are not independent examples, but rather show change over time, we can plot them in a line graph. We can also see how two columns relate to each other by plotting those in an XY graph. These visualizations can make the relationships immediately obvious:
It is also possible to visually examine how two columns change over time or how three columns relate to each other using a 3D plot. However, beyond that, this approach becomes difficult. We can no longer see the relationships between all of the columns and they become hidden to us.
To find the hidden relationships in such data, we cannot ask the machine to simply present the data to us in a plot. Instead, we have to make the machine sort through the potential relationships and find the ones that are most significant. Machine learning algorithms are incredibly good at this. For example, they can construct decision trees that can process data with tens of thousands of different columns without breaking a sweat.
Machine learning algorithms that automatically construct collections of decision trees, such as Adaboost (see here for a gentle introduction), work well in a wide range of settings, and there, they are usually superior to human decision making.
Pedro Domingos famously said “Machine learning would not be necessary if we could see in high dimensions.” As we discussed above, we need computer help whenever there is large amounts of data, but if the data has only a few columns (few dimensions), then this help is mundane and uninteresting: just simple calculations of summary statistics or plotting of graphs. It is when the data has many columns (high dimensions) that ML becomes necessary.
2. Complex ML Models Are Like Human Intuition
A common belief in the research community is that AI can (and will eventually) achieve human-level or better performance in any task that humans can perform in a short period of time, say, less than 1 second. (See e.g. the quotes from Herbert Simon and Andrew Ng in this article.)
That description includes many problems in visual perception and language understanding. It makes particular sense that those problems would be solvable with neural network models, as they were inspired by the processing of neurons in our visual cortexes. We are making great progress on those problems, although (as usual) many are in fact harder than expected.
A different area that falls within the same description but does not necessarily involve vision or language understanding are problems that humans with appropriate experience can solve using their “intuition”. Like seeing the locations of faces in pictures, our intuitions seem to come instantaneously. However, intuition has some other relationships to ML that make it a more interesting point of comparison for me.
First, both our intuitions and the models built by ML algorithms are learned from experience. As a result, they share some of the same strengths and weaknesses. A common weakness is that both will fail when analyzing examples that are unlike past experiences. A common strength, however, is that they can continue to improve from each mistake.
Second, both our intuitions and the best models built by ML algorithms are black boxes. We cannot give a simple formula that will explain our intuitions in general just like we cannot provide a simple formula that will explain the outputs of a deep neural network (or other complex ML model).
However, just as we can often explain our intuitions for specific examples after the fact, there are methods that will provide simple explanations of specific outputs of complex ML models (see e.g. this article). In other words, while there is no simple explanation that will explain all the examples, we can often provide explanations for individual examples, one at a time.
Christian Szegedy, a prominent ML researcher at Google, said he believes that any task that human intuition is able to perform should be possible for ML (in particular, using deep neural networks). Hence, human intuition provides both a useful benchmark for what ML can achieve and also, as we just discussed, a useful analogy for understanding its strengths and weaknesses.
3. The Hardest Things for ML Seem Easiest to Us
If ML can outperform humans both at problems that require detailed calculation (like slow Chess) as well as those that require intuition (like fast Chess), then it is hard to see what areas are left for humans to outperform. The reason it seems that way, as Melanie Mitchell explained, is that the problems that are the hardest to solve with computers are often ones that so easy for us to solve that we don’t really even think about them.
Keep in mind that we separate the abilities of humans from each other by having them perform tasks that are actually hard for us, so the things that we think of as the great human achievements are often not the things that humans, in general, are naturally good at.
In terms of physical abilities, we decide the “world’s strongest man” by performance on incredibly difficult tasks like pulling an enormous box car, a task that few humans can do. Yet, we can build machines that do the same task with ease. In contrast, it turned out that making a machine that can stand on two legs was actually very difficult, even though most humans can do that without any difficulty.
On the mental side, chess is a task that requires a lot of brute force mental calculation, analyzing all the possible sequences of moves that could occur from a given starting position. That is something that is easy for computers, which are built for brute calculation. In contrast, it turned out to be quite difficult to design machines that could recognize faces, something that even human babies can do without difficulty.
4. Humans Outperform with General Intelligence
Of course, we do now have software that can recognize faces and translate languages, and as noted above, we think we may be able to improve upon human performance in all cases where humans rely on intuition. In what area, then, do human brains have a distinct advantage? I think it is primarily in their flexibility or what is sometimes referred to as “general intelligence”.
It is important to remember that computers always do exactly what they were programmed to do. That is sometimes a strength compared to humans but it is also often a weakness. If a computer was not programmed to handle a certain situation, then it will not work properly in that situation, except by accident. (That, of course, is why programming itself is such a hard task. It requires thinking through every possible scenario that could arise and finding a solution that will work in all of them.)
While AlphaZero may be the greatest chess “mind” ever created, it can’t play if I replace a few of the chess pieces with checkers. (In fact, it isn’t even possible to describe such a board position to AlphaZero.) Most humans, on the other hand, could probably play such a game somewhat sensibly with little effort, even if their play was far from perfect.
Here’s another example that jumps to mind for me. The first time I tried out a Roomba robot vacuum cleaner, it ended up in a corner slamming into the wall, backing up, turning, slamming into the other wall, backing up, turning, slamming into the first wall again, in an endless cycle. When I saw this, I let out a laugh, picked up the Roomba, placed it in the middle of the room, and let it go to continue vacuuming.
It certainly did not feel like a great mental feat on my part to spot the problem and fix it like that, but that was something the robot could not do on its own. It was faithfully executing its program, but since that program did not properly handle this case, it was stuck. It cannot look outside of its program, understand what is going wrong, and derive a solution. It just executes the program. No doubt, future versions of the Roomba would have avoided getting stuck in the corner like this, but that improvement came via the general intelligence of other human beings, who learned about this problem and then designed fixes for the software.
The Roomba’s program was likely standard software rather than ML, but the situation is similar in the both cases. Even though ML allows the machine to “learn” from examples — in effect, creating part of the software without the need for human programmers — in practice, the process of applying ML is very similar: after the algorithms run, humans are needed to study its output, spot problems, and then figure out how to manipulate the algorithms so that the next run will produce results without those flaws. That might involve adding new examples for the algorithm to learn from, changing the data in the examples themselves (adding new types of information), et cetera.
Media reports often make it sound as if ML systems are created just by putting a computer out in the rain and letting it get struck by lightning, but in reality, it takes a team of smart humans months or years of work to build them. Humans, with their general intelligence, are critical to the process of building systems using ML.
My expectation is that humans will not only be needed in the development of systems using ML but also in their deployment and day-to-day use because of the new situations that will inevitably arise. Those situations will require general intelligence to spot obvious errors and correct them.
As one hypothetical example, if a system using ML makes a medical diagnosis that relies on the presence of a fever but the doctor knows that, in this case, the fever was from heat exhaustion that was unrelated to the other symptoms and knows that fact was not available to the software, then she can see that the diagnosis could be an error.
Even in the case of chess, where all relevant information is available to the ML system, grandmasters can still spot cases where the computer is making an error. This is because, even though chess engines are more advanced than human players, their play is still imperfect. Having used them routinely for over a decade now, chess masters are knowledgable about those limitations and can see where they are affecting the engine’s assessment.
Final Thought
While media portrayals of ML tend to have the themes of horror movies like Frankenstein (alluded to above) or the Terminator (where humans and machines are fighting for control of the earth), I think it’s important to keep in mind that humans created ML. It is just another tool that can assist our abilities, allowing us to perform certain tasks more easily than we could before.
Most of us do not feel threatened by the fact that we are weaker than a backhoe, which has strength beyond that of any human. It is a tool that we created to assist our physical abilities. The creation of the hydraulics that power a backhoe is rightfully considered a triumph of human ingenuity.
ML is a tool that we created to assist our mental abilities. It can solve mental tasks with much less expenditure of human effort. Like the backhoe, it may eventually exceed our own abilities, but ML too is a human creation and a triumph of human ingenuity.
We are not in a struggle of “man versus machine”. Rather, we are comparing humans using only their natural abilities to humans assisted by the tools that they created. Humans are distinguished from the other animals by their ability to create tools, and ML is simply the latest achievement of “Man, the Tool Maker”, a story that goes back to prehistoric times.
