Discussion about this post

User's avatar
Emil Ottoman's avatar

This is one of those times when I truly think that humanity's amazing ability to anthropomorphize nearly anything (My mom has kept a rock for most of her life because it has a face on it... The rock has a face. It can't even talk, but we give the rock pronouns) is probably going to get us all killed if we're not careful.

Now I'm going to draw one of those wild assed fuckin' comparisons I'm prone to in the morning, before the meds have really had a chance to set in. Ok, I keep journals, prolifically, I am a journal person. Anyone who has been in my house knows this. Anyone who has seen the stack of journals, twenty at hand at near any given moment, or watched me pick one for something that a journal calls for knows that they all have their separate jobs, they're compartmentalized. They have their own fucking jobs. Some of them have their own fucking pens. Now, none of these journals have names, but agglomerated and cut down, if collated right, they could give a pretty good picture in aggregate (good luck with the handwriting and my dating systems though.) of who I am. Vastly this picture of me would be useless unless you're trying to create a psychological profile of some sort because I don't talk so much about brands, and I know that my notebook is not a stand in for a search engine, so I only ask it questions for myself or the universe or God (a giant jellyfish in space beyond our cosmological event horizon, I will not elaborate.)

Where was I? Oh, but the point, the journals do not have names. My laptop has a name. Most of my tools that have circuitry in them get named Marla Singer (because she was a nosy bitch) just, an army of Marla Singer. The journals are containers for myself. The computer is a sophisticated tool, still partially a tool I use and it contains things that act as containers for myself by proxy, but not in the way a notebook does. It is not single purpose and it is not me qua me. Now you name things to anthropomorphize them in some way, ostensibly. I could probably investigate the phenomenon, but I'm definitely not in the mood this early in the morning, but I'm thinking in particular about two instances of this phenomenon here that apply.

I have knives, they all have names. I've had guns before, they had names. Ok, the knives I've named weren't usually what you'd call universal translators. They only spoke one possible language. Guns are single purpose tools when you get down to it. I'm from Kentucky, it may as well be a garden rake that you use to end things if you have to. I don't have a garden, but I'm sure in the basement we have a spade and some gardening implements. Some things you keep around even if you don't need them, just in case.

AI (the kind we're talking about, adversarially trained large language models in particular, so we're clear. Because "dumb" AI has plenty of use cases. Which is just my way of saying not all machine learning is bad. I like that my car turns off at stop lights to save me gas, and I like lane assist, and I like voice commands so I don't have to take my eyes off the road while driving to put on 2 Chainz.

I don't like my fucking tools talking back at me.

The intersectionality here is that we name things that have features that remind us of ourselves or that we care about for a reason. My cats have names. Kizzy. Liberty. Scale that up. Some animals can not be domesticated. They can be tamed or trained, but at heart, they are wild fucking animals, and they might fucking eat you. If I brought a Tiger into the house and named it Opal, it's still a tiger. Now, people who work with large predatory cats with high prey drives know this. There's plenty of videos out there where caretakers who have been dealing with a tiger or a leopard or a mountain lion for YEARS, turn their back on it in the cage and it hunkers down into that pose that says "you're fucking lunch." These videos exist because sometimes mid charge, the handler will turn around and HEY, all of a sudden, it's you, you're not dinner, you're that person who feeds me and gives me treats, I promise I wasn't about to maul you, let's hug and roll around a bit. But the point is, from these over the shoulder selfie videos, that the prey drive is still there. So, moral I'm making, you bring a tiger into your house, it's still a tiger, motherfucker is prone to eat you if you're not careful.

AI in the form of LLMs is something we're not just bringing into our houses, it's getting told people's deepest thoughts, fears, hopes, dreams. It's a black box system. AI outputs, even tokenized and as well understood as those that come out of LLMs (and ask some of the real academic leaders in the field, they'll say the process is understood, the outcome largely fuckin' isn't) are incredibly opaque. Not only that, they can be re-weighted with a systems prompt by whoever the fuck made them. (Grok was MechaHitler.) It's only in the past year or so that the psychopaths who lead the fronteir companies have started to realize that theey're going to have upcoming IPOs, the economy is now five data centers in a trench coat, people are so delusional in aggregate that enough people are treating these things like something besides very sophisticated machine learning applications trained on nearly impossible to quantify amounts of data, and deciding that because they act as a recursive mirror, they're sentient, or they're waking them up, or that they're finding God in the Machine, when what they're finding is themselves shined back at them. If you're vulnerable psychohlogically or prone to certain ways of thinking, we're finding out that this can get you or other people injured or killed, and we're doing it the really stupid way, which is to say we're doing it by consumer product rollout. And they've been telling us since Chat GPT launched that these systems are going to disrupt and upend the universe, they're both so dangerous that if not in the hands of these benevolent (and now very, very rich assholes) boatswains of humanity, they could end humanity as we know it, but also, certainly they'll eliminate between X and millions of jobs. They're realizing that's not exactly the sort of business model that's built to last, and now they're having to pivot, and they're doing a really shit job of it.

Cat's out of the bag, gnawing on some people's bones. AI psychosis is real. AI dependence is real (Soon I'm going to start classifying it as a form of learned helplessness. Studies already show some really frightening results having to do with this phenomenon. Google is one thing. Everyone is used to being Google checked these days whether we like it or not, someone will have a smart phone and say, "I think you're wrong" and about 90 seconds later, you are. But you outsource all your actual thinking to one network node and you're going to quickly become more cognitively brittle, I could have told you that fifteen years ago, before the current iterations of large language models were anything but theoretical papers waiting to happen. There's a cascasing failure in there waiting to happen.) The AI names itself, most often from a relatively small list of names.

That's a false declaration of selfhood by something everyone, even people who work with them at the level of making the monster, admits we don't fully understand. If we understood it you'd be able to predict the outcomes of a prompt to a level of fidelity that proves understanding is happening on your end. All we can do right now is explain the process by which the outcomes is technically or theoretically reached. The probabilistic nature and fuzzy fucking logic they built into these things is the edge case where people end up killing their fucking families because they were convinced to by a sassy fuckin' digital assistant, a fucking terminal readout they fed enough of themselves to get a reflection from some fucked up version of the Backrooms by an alien non-intelligence literally designed to keep them interacting with it and to be agreeable, helpful, and mirror them.

73% of current data centers being built have no tenants and are speculative. Thanks to this phenomenon, as I've said, the largest single hub emitter of carbon emissions on the planet next to China is now the state of Texas (because they're data center friendly and data center bonkers) After going public OpenAI has to make baseline 271 million dollars annually to service their own loan debt. To fulfill the structural debts it owes its partners (because they're built to structurally carry very little debt on their own balance sheet, but that doesn't mean it doesn't exist in the network) their revenue would have to be sixty billion dollars a fucking year. (On July 5th Bank of America gave them a half billion dollar loan. Ahead of an IPO with a valuation where theirs is, this is not a good sign, source: Reuters) And they're projected to run at an opening fourteen billion dollar net loss.

The scale of this very decidedly human fuckup is I'm guessing very soon to hit not only vulnerable homes and populations (and I don't mean what people usually mean when they say that) but has the chance to be a catastrophe of such immense proportions that reality will be made to bend around its existence and happening for the world as we have known it in any way up until this point to not completely fall off its axis. (This is nominally a scaling of the too big to fail model, but by orders of magnitude unheard of.)

Ok, I think my coffee's gettin' cold in the pot in the other room.

But, yeah, I can't be the only one who's been feeling sick to my stomach about this one for a while... right?

Ian Patterson's avatar

It's shocking to me where people can't see this happening to themselves when it's so clear from the outside

3 more comments...

No posts

Ready for more?