'Don't blame us, it was the algorithm' won't cut it anymore

Tech companies will face increasing hostility if they don't teach their machines a bit of humanity.

by Jack Torrance
Last Updated: 20 Sep 2017

This week, in the wake of the Parsons Green bombing, Channel 4 News revealed something pretty disturbing about Amazon. Users who go on the website and add certain chemicals to their basket will find themselves presented with suggestions of other ingredients that are commonly used to make explosives. Some are even pointed in the direction of remote detonators and ball bearing (which are often used as shrapnel in terrorist bombs).

It’s embarrassing, and some would say outrageous, on Amazon’s part. But of course it’s not deliberate. It’s not the fault of some marketing guy at Amazon who thought, ‘well if we’re going to cater properly to terrorists then we need to make sure they can find all the gear they need with ease.’ Instead it’s the product of Amazon’s automated algorithms, which seek to cleverly tempt you to splash out on more things than you planned on doing when logging on.

It’s not the first such algorithmic embarrassment to have afflicted tech firms in recent weeks. After Friday’s incident, taxi app Uber was slammed for unwittingly putting up its prices in the area around Parsons Green. It soon rectified that and refunded any affected passengers, but that didn’t stop it getting a tonne of social media grief.

Last Thursday, ProPublica revealed that Facebook was allowing advertisers to target users interested in anti-Semitic topics, including the SS, ‘Jew hater’ and ‘How to burn jews’ – algorithmically generated categories. Further digging by Buzzfeed revealed a similar problem at Google, which allowed advertisements to run alongside searches for things like ‘jews control the media’ and ‘black people ruin everything’. These follow a previous investigation by the Times that found ads for top brands alongside extremist videos on Youtube.

None of these things are deliberately malicious, at least not on the part of the tech firms. Uber doesn’t want to rake in cash from those affected by terror attacks. Amazon’s not trying to corner the bomb making market. There’s probably not even that much money in advertising to racists.

These unfortunate outcomes are created by the behaviour of the platforms’ users. But it’s ultimately the likes of Facebook and Google that will take the flak for it, and rightly so. They’ve created a new, internet-powered society that’s delivered great benefits but comes with big risks.

Politicians, always on the lookout for a dragon to slay, are circling tech companies and licking their lips. If the giants don’t take more responsibility for policing themselves then the authorities will happily oblige.


Find this article useful?

Get more great articles like this in your inbox every lunchtime