by Adrian Bowles, PhD
Creating a machine learning-based application to evaluate data and make recommendations is a lot like the job of a parent teaching their children. The application, like the child, has an aptitude or intelligence level based on algorithms or biology, but learning requires data. Either may be “intelligent” yet “ignorant.” The issue is how we teach our AI applications or prepare them to learn on their own.
Bad AI Parenting
No child is born a racist or prejudiced against groups by gender or beliefs. They learn those behaviors and beliefs from observing and interpreting events in their environment, or by explicit teaching. They see which behaviors are reinforced by people they respect, and those behaviors and associated beliefs become internalized. Parents have some control over the events their children are exposed to, especially in the early years, but children eventually grow up and make their own decisions.
The problem with applications like the well-publicized resume evaluator that was found to produce biased results is that they are either trying to automate historically flawed procedures—based on historical data—or they have not been trained properly. I’d call it bad AI parenting.
Value-Free Resume Evaluation Systems
Early AI systems were usually dependent on hardcoded rules that governed their behavior. For a resume evaluation system, one might include a rule that requires it to recommend equal numbers of men and women from a pool of applicants. With a rich enough rule set and enough data, many of the problems we see today could be avoided but establishing that rule set is not a simple task.
Today, many applications use deep learning algorithms. These are wonderful at examining large volumes of data and identifying or recognizing relationships at different levels of abstraction. Finding all the images of houses in a trove of digital photo files, for example, may require the system to first identify all the edges by looking for lines along which the hue or intensity of pixels changes, then looking for shapes, etc. The system doesn’t “know” it has found a house. It is value-free, just as the decision to recommend a person of a particular gender, race, or religion is not a value judgment when made by a machine.
Defending Against Bias
The Amazon HR tool learned to make recommendations similar to those the company made in the past because it looked for relationships between attributes (from gender to age to alma mater, etc.) and outcomes (hired and successful, hired and unsuccessful, rejected). It should be no surprise that the historical data, including resumes and outcomes, were biased. It would be astounding if it were otherwise.
The solution is to leverage a combination of tools and technologies, including supervised learning (in which good examples are presented to the application as exemplars), reinforcement learning (in which the application gets feedback on its exploratory behavior), and rules, which act as a last line of defense against bias.
It isn’t easy, but raising a child isn’t easy either. Once they go out into the world, all you can do is hope the rules stay with them. For an AI application that is expected to learn from public data—like social media feeds—ongoing reinforcement of rules and values is essential.