I could build a dossier on you. You would have a unique identifier, linked to demographically interesting facts about you that I could pull up individually or en masse. Even when you changed your ID or your name, I would still have you, based on traces and behaviors that remained the same — the same computer, the same face, the same writing style, something would give it away and I could relink you. Anonymous data is shockingly easy to de-anonymize. I would still be building a map of you. Correlating with other databases, credit card information (which has been on sale for decades, by the way), public records, voter information, a thousand little databases you never knew you were in, I could create a picture of your life so complete I would know you better than your family does, or perhaps even than you know yourself. I could accurately diagnose you with mental illnesses, for instance — behaviors that correlate to bipolar, depression, addiction, and so on. I could understand you like no lover ever did, and you would never know I was there. While I could pull you individually out of that database, the real magic is that I would never have to. I could let algorithms understand you, process you, follow you, and never have to know any of you myself. You would be tracked and described by a thousand little bots you could never see.
But this isn’t the Holy Grail of my surveillance capability. What I’d do next is: create a world for you to inhabit that doesn’t reflect your taste, but over time, creates it. I could slowly massage the ad messages you see, and in many cases, even the content, and predictably and reliably remake your worldview. I could nudge you, by the thousands or the millions, into being just a little bit different, again and again and again. I could automate testing systems of tastemaking against each other, A/B test tastemaking over time, and iterate, building an ever-more perfect machine of opinion shaping. But I left before it really got good. So I don’t know for sure that this is what is being done with the vast data being collected about you, but there were plenty of smart people in that business, some of the most creative and innovative minds I ever met.
Your internet experience isn’t the main result of algorithms built on surveillance data; you are. Humans are beautifully plastic, endlessly adaptable, and over time advertisers can use that fact to make you into whatever they were hired to make you be. I still get kind of goosebump-y thinking about it sometimes. There’s a little evil voice in me somewhere that whispers “people make the best toys of all.”
Quinn Norton, The Hypocrisy of the Internet Journalist