Are We Turning Off Our Brains in Favor of Emerging IT?

by Betsy Burton

We had a very interesting discussion today in the Aragon research meeting that got me thinking about how much have we or should we be depending on information-based emerging technologies (such as AI, predictive analytics, digital twins, etc.) when we know we all have some data quality flaws.

And, more importantly, are we teaching ourselves to turn off our brains in favor of technology doing our thinking for us?

In this blog, we explore the impact of depending on emerging information technology too much and remind ourselves that the definition of AI calls for humans to remain engaged in decision-making.

The Case of The Wrongful Arrest

There was some recent news about a young man, Ousmane Bah, who is suing Apple for being falsely implicated and arrested for stealing from Apple stores in Delaware, New Jersey, and Manhattan. It turns out that a facial recognition system had incorrectly identified him as the perpetrator of a shoplifting crime spree at Apple Stores.

As a result, Mr. Bah was arrested in the middle of the night. However, when the police went to compare the surveillance footage image of the actual perpetrator with Mr. Bah’s image, they looked nothing alike.

Putting Too Much Faith in Technology

What detectives believe happened is that the actual perpetrator used Mr. Bah’s previously lost drivers permit, that did not have a photograph on it, at an Apple Store. As a result, the perpetrator’s image was wrongly linked to Mr. Bah’s identity; the source of the information was compromised.

The real problem though is that Apple’s security firm and the police trusted the facial recognition system so much, that they failed to take a moment to have a real person double check the identities to ensure they matched before doing a 4 a.m. arrest, which resulted in months of legal issues and stress for Mr. Bah.

Garbage In, Garbage Out

As a former database person, I have always believed that there is no such thing as perfect data/information. And, in fact, the more humans can change it, the more it is used to create new views, and the further the data is from the source, the dirtier it potentially gets.

The value of information analytics, predictive analytics, AI, digital twins, etc, are all dependent on a lot of information. And in fact, when these systems are based on good algorithms, they will often do consistency checking to double-check their own analysis.

Bottom Line

I thought this was a great opportunity for us to remind ourselves of my esteemed colleague Adrian Bowles’ definition of AI from Aragon’s business and technology glossary.

“Artificial intelligence (AI) extends the abilities of its human counterparts and can search, read, understand, and categorize various sets of unstructured data faster than any human or team can.”

The key word here is ”extends.” Emerging technologies such as AI, predictive analytics, virtual assistants/advisors, digital twins, smart robots, etc. are intended to augment the human mind—not replace it.

These technologies will increasingly assume some routine and complex tasks and provide valuable insights and advice. However, this does not mean that we humans can or should turn our brains off—especially when it comes to the security, and well-being of lives, the environment and economies.

display trackbacks