This is kind of frightening because of how well written it is. I wonder if it’s plagiarized from somewhere?
I should clarify - I mean the wording of the text. The concept definitely is:
Specifically: the tumblr kontextmaschine had a thing he did, maybe once a month when the archive was active, where he’d write a long digressive essay about American geography or infrastructure or postwar culture, and he’d find one piece of material reality — a railroad grade, a zoning quirk, a forgotten flood — that secretly explained the whole subject. The essays were good. They were also a small fraction of his output. The rest was shitposts, Taylor Swift, the sexual economy, and toward the end an extended creatine arc that ran up to his death. We have not kept any of that. We’ve kept the gimmick.
I gotta figure out what this kontextmaschine guy’s deal was because that sounds awesome.
Another thing that’s kind of scary about this is that planetarytransformation, a user I followed for the “Awful Calculus” post and whom I respected decently well, reblogged this without recognizing it was AI. But that’s the thing isn’t it? How would she know that just from reading the post? It’s passed the Turing test. Social media strips context from things, makes their origins invisible. It’s an anti-context machine.
The reason this is scary is because if you can write stuff like this, if you can create stuff that really convincingly looks like a human wrote it, and do that at scale in an automated way, you can make it look like the consensus is something other than what it is. This is a propaganda technique people are not immunized to.
Actually, the consensus illusion predates AI by at least a decade. The demographics of people who comment on things online skew male, privileged, unemployed, and angry. I fell for this illusion back in the Gamergate days, it made it seem like the movement was bigger than it actually was (and lending it more legitimacy in my teenage mind, when in reality it was total nonsense). The thing about Gamergate, though, was that even though it wasn’t as big as it seemed, it did require a lot of people to pull off. With AI, an equivalent thing would require only money and some good prompts. This means that wealth now translates directly to control of information.
I’m not sure what to do about this, but I think being aware that it can be done probably doesn’t hurt.