Why We Think Our Phones Are Secretly Listening to Us
A developer explains the computational complexity of Facebook listening in
Months ago, before lock-down started, I had a friend round for dinner. He was on the keto diet; a high-fat, low-carb regime, mainly consisting of meat and cheese. Also fine, he told me, are Shirataki Noodles. I didn’t know what to cook. Shirataki Noodles were not a helpful suggestion.
Another guest was a vegan, so meat, fish, eggs, and cheese were out. I found myself mumbling dark comments about the keto diet. In the end, we went out to a restaurant.
Later that evening, as I scrolled through Twitter an ad popped up for the keto diet. I’d never shown the remotest interest in dieting before, and my mind raced. Were Facebook and Twitter secretly listening to my conversations? I pictured Zuckerberg with a headphone-clad, Gene Hackman-like figure in the shadows, identifying ads to push to me.
Phones have ears and laptops have eyes
So, to come straight out with it: No. Our phones are not secretly listening to us. There are lots of ways we know Twitter and Facebook can’t do this. When a developer writes an app for iOS it runs on the Apple-controlled operating system. Facebook can’t just access the microphone and start recording. The app has to go through code written by Apple. When Facebook requests audio, Apple asks the user whether they want “Facebook” accessing the microphone. If they do, it sends an audio stream to Facebook. If they don’t, it doesn’t. Apple’s code is a software bouncer: If an app doesn’t have an invitation from the user, it ain’t getting in.
When an app uses the microphone, a bar appears at the top of the screen. There was no bar at the top of my screen when I was moaning about my keto friend. But still, I feel disconcerted. I go through my phone and review which apps have access to my microphone. I do this, knowing it is not necessary. But also I think it is best to check just in case. I’m reminded of the myth that at Harvard it is good luck to rub the shoe of a particular statue. Ivy League students are too smart to be superstitious. But still, the left shoe is shiny from being rubbed. You know, just in case.
Maybe, I think, Facebook found a way to bypass Apple’s security. But again, we can check this by monitoring the data Facebook is sending from our phones. If Facebook was sending audio, we’d see it. Even if they found a way to disguise the traffic, sending this audio would require a massive amount of bandwidth. You’d know about it when your phone bill turned up. What’s more, if we run the numbers, we see it isn’t feasible. “Voice-over-internet call takes something like 24kbps,” Antonio García Martínez says in Wired, “that’s about 20 petabytes per day, just in the U.S.” Facebook’s data centers are big, but not that big.
On top of this, there is the cost and computational complexity of processing the audio, finding keywords, and serving ads. And the question of whether it would actually work. As Martínez says, “Human language is overrun with sarcasm, innuendo, double-entendre, and pure obfuscation.” Siri barely works when I speak directly to it. Computers aren’t yet smart enough to make sense of our speech. Facebook simply can’t be recording our audio all the time. Your phone would get hot, the battery would run down. Forget data protection laws, the laws of physics protect us here.
Facebook denies secretly recording audio, too. Although I find that the least compelling piece of evidence.
Anyone for pizza?
I write software. I have friends who work at Facebook. I do things with APIs and mobile apps, so I’m in as good a position as I can be to judge the veracity of these claims, and yet I still can’t shake the suspicion that they’re listening. Could they have somehow found a way around physics? It has become my “9/11 was an inside job.” Listengate.
Twice now I’ve found myself doing little tests. “Man, I’d sure love a holiday in the Bahamas,” I say out loud to no one, my phone resting on the side with the Facebook app open. “Gosh, my car insurance is expensive. Would be real great if someone could find a cheaper quote.” Nothing. I feel silly. But still, the next day, I look at each ad with renewed suspicion. Journalists, smart journalists who debunk conspiracies, are lured into these too. New Statesman ran a similar (and equally unscientific) study to mine. Of course, they got similar results: Facebook isn’t listening.
This isn’t the only rumor that I find reasonable people repeating. Planned obsolescence: the idea that old products stop working just as new ones come out. Software updates are designed to slow older devices. Smartphones cause cancer. Hackers are secretly watching you through your laptop camera. That sort of thing.
Sometimes it’s just maths
The problem is, technology companies do shifty things. Listening to us through our microphones to sell ads is exactly the sort of thing they would do. It’s very much in character. Apple was slowing the processors of old phones — admittedly to gracefully handle degraded batteries, but we were right to think our phones were getting slower. And there are more nefarious examples. “Apps were automatically taking screenshots of themselves and sending them to third parties,” sad Christo Wilson, a PhD student at Northeastern University, after examining multiple Android apps. Facebook has a history of hiring “outside contractors to transcribe clips of audio from users of its services” and storing messages that “users wrote out but did not post.” There are loads of examples of these. Our privacy has been violated before.
What’s more, sometimes advertising coincidences are uncanny. There’s my keto example, but everyone has their own story. When PJ Vogt asked for examples on the podcast Reply All, he received dozens of replies. It’s “spooky,” people said time and time again. They said something and the next day it popped up in a feed somewhere.
The explanation, professor David Hand, PhD, explained to the BBC, is just maths. “If you take something that has a tiny chance of occurring and give it enough opportunities to occur, it inevitably will happen.” The odds of showing someone a product they were just talking about is something outrageous. One in a million, say; the phrase we use when we mean something is basically impossible. With those odds, every time Facebook shows an ad, 2,500 people will have just been talking about it. If you were thinking about something common, like pasta, and later saw an ad for it, you might not think anything of it. But when it’s something obscure it sticks out. This is the Baader-Meinhof Phenomenon: coined by someone named Terry Mullen, after reading about an obscure political faction, the Baader-Meinhof group, and then coming across them again in an unrelated article.
We’ve been duped. Tricked by our illogical gray matter. Humans are pattern-spotting machines. We’re so great at it, we spot patterns even when none exist.
Indistinguishable from magic
I don’t find these explanations particularly satisfying, though. These uncanny events happen too often to be chance. What I really want to see is the series of decisions that lead to me being shown that particular ad at that particular time; to explain how the algorithms synchronized with my lived experience. Because these ads aren’t chance coincidences. They are the result of a recommendation engine using data points it has on me.
Why don’t Facebook and Google listen to us? They don’t need to. They know our age, our location, our interests, the sites we’ve visited, things we’ve looked at. Twitter didn’t advertise the keto diet to me because it overheard me talking about it. It did so because, without thinking, I Googled what to cook for my ketosis-induced friend and one of the sites I visited had a tracking pixel on it, which captured this nugget of information to advertise back to me. Our digital lives are so intertwined with our “real” lives that it’s easy to forget what you Googled and what you just thought. Google doesn’t need to read our minds; we type into it what we’re thinking. Facebook is not listening to our words. It’s listening to our thoughts.
“Any sufficiently advanced technology,” Arthur C. Clarke famously said, “is indistinguishable from magic.” And technology has become sufficiently advanced. Even 10 years ago, Facebook was gathering 500TB of data a day. With that much data, it can make connections that seem magical.
It’s Facebook’s world, we just live in it
These explanations are no more satisfying than finding out how magicians do their tricks. Mirrors, magnets, and trap doors are so prosaic it feels hard to believe they are behind what we saw on stage. We’ve all played with cards, so we think we know what is possible. We forget that magicians have practiced manipulating cards for thousands of hours. A pack of cards in their hands is not the same as it is in ours. Similarly, technology companies operate in a way that is fundamentally different from the way we do. We hear they are breaching privacy laws, but often in subtle ways. Can anyone really explain which particular privacy violations Facebook’s record-breaking $5 billion fine covers? (And, indeed, what all the fines since then have been for as well.)
This low-trust environment is the perfect breeding ground for conspiracy theories. “Research shows,” says Professor Karen Douglas, PhD, of the University of Kent, “that people are drawn to conspiracy theories when they feel powerless.” And compared to Facebook we are powerless. The idea that we have found something out allows us to regain some of our power: We know their secrets. “People who believe in conspiracy theories can feel ‘special,’” psychologist Anthony Lantian, PhD, writes, “they may feel that they are more informed than others.”
In Democracy and Truth, Sophia Rosenfeld suggests conspiracy theories thrive in societies with large gaps between the governing and the governed. Technology companies may not govern us (not literally, not yet), but they have access to knowledge and resources we don’t. There is a growing gap between the things they can do and our understanding of how they do them. In The Workshop and the World, Robert Crease talks about how this “creates a rift between those unable to understand this special language and those who do, making it easy for the former to distrust the latter.”
We often think conspiracy theories belong to the paranoid. But a kinder explanation is that these ideas come from powerlessness. Even software developers, Jeff Atwood writes, “serve at the pleasure of the king.” Each is at the mercy of a bigger company, higher up the food chain. In comparison to tech giants with millions of users we are but specks, tiny and powerless: digital peasants. How much nicer to feel special: someone is listening to me. I count, I matter! Even if just to sell me a faddy low-carb diet. Not that I want Facebook to listen in on me, but it’s almost more depressing that we’re not even important enough for them to want to try.
