1. ← Home
Login
How accurate have Ed Zitron's AI skeptic predictions been?
~ai~opinionauthor.dan luupredictions
danluu.com 3 weeks agoTildes

Summary

From the article:

I was curious how well the predictions of the most widely cited AI skeptic I've seen (Ed Zitron) have done, so I looked at how his predictions panned out. To disclose my own biases, I've never had a particularly strong pro or anti AI progress position. For example, in 2022, I did a comprehensive look at predictions Futurists made, including well-respected folks like Kurzweil and found them to be generally wrong on both the prediction results as well as the reasoning. On the flip side, in 2015, I wrote about how people were underestimating AI's ability to displace humans in jobs and have repeatedly been on the record as saying that many people are underestimating AI's ability to displace humans from jobs. My position on AI has been extremely boring and is basically, "if something is currently happening, the people who are saying that it's impossible that it will ever happen are probably wrong".

One comment I've seen from a lot of AI skeptics when someone responds to an AI skeptic is that all of the people who are saying that AI isn't fake are self-interested liars. Personally (to my obvious detriment), I have no particular financial interest in AI companies. I own whatever the standard share of them is via boring index funds. I have some seed stage investments, but just due to the timing and what's gotten big, that part of my portfolio is underweight on AI. I don't work at an AI lab or a company that supplies AI labs. I've mentioned being hilariously bad at interviews before, and I did interview at an AI lab a number of years ago and failed the phone screen in a performance that was the kind of performance that must've inspired Jeff Atwood's famous Why Can’t Programmers... Program? where he concludes that there must be a lot of fake programmers out there because nobody could fail a coding interview that badly if they knew how to program. I don't benefit in any particular way if AI does well, except insofar as anyone who holds broad index funds benefits, but I do care about accuracy.

[...]

Because there are quite a few prediction results, let's look at one in detail before the complete list to get an idea of the kind of reasoning Zitron uses. We'll arbitrarily look at this November 2024 talk where Zitron says, among other things, the major tech companies (like Meta and Google) are dying and they're thrashing around on AI because they don't know how to grow.

[...]

Although this wouldn't be in the spirit of Zitron's statement, one could argue that Meta is actually dying, it just hasn't died yet. However, the reasoning in Zitron's argument is incorrect here—the Meta, Google, and Microsoft ecosystems are not dying. Given how fast these companies are growing (in terms of revenue and profit), it doesn't seem that AI is, as Zitron implied, some kind of desperation move they're reaching for because "they don't know how to grow" and are all out of ideas. I don't think it's worth spending this much text on each prediction, but the pattern Zitron used here is illustrative.

[...]

From what I can tell of how people cite Zitron, they cite him as an authority so they can say that this guy who looked at the numbers has made this claim, so their claim is backed up by the numbers. It turns out that if you look at the claims Zitron makes and know anything about the topic, the claims don't make sense, but I don't think that's the point. The point is one can say that someone looked at the numbers. The other point seems to be that this guy is angry2, which is a good way to drive engagement.

But when people bring him up, they're of course not generally citing his anger; they're saying here's this guy who's looked at the numbers and, if you're angry about AI, he's right there with you being angry about AI, and he's got numbers on his side.3 Like I said above, I don't want to go into this level of detail on each claim; this is just an illustrative example about how the claims below look. For any of his posts that I read, while there are numbers thrown around, the numbers don't actually connect to a coherent argument. In many cases, as we saw above, the numbers don't even really support his argument (such as an MAU decline in Facebook causing Meta financial problems which would then cause Meta to spuriously insert AI in places it doesn't belong). I suspect he's relying on people's eyes glazing over when they see numbers and just not thinking about what the numbers mean.

With the predictions below, someone could have the exact same prediction record and have completely reasonable reasons that just didn't pan out. Or someone could be correct in every case and also be wrong because all of their reasons are wrong. Someone like the latter person might have some kind of intuition that they're unable to articulate, or perhaps they're someone who just got lucky. Fortunately for us, we don't have to make this difficult judgement call because Zitron is wrong on the predictions and also wrong on the reasoning.

[...]

If we compare to how futurists did in our analysis of futurists, on style, Zitron relies much more heavily on anger than any of the futurists we looked at. On the quality of reasoning, he was probably about average compared to the futurists. Despite being wrong on roughly everything, he's not more unreasonable than someone like Buckminster Fuller, who suggested we'll be able to send people by radio because atoms have frequencies and radio waves have frequencies so it will be possible to pick up all of our frequencies and send them by radio.

In terms of the style of reasoning, of the futurists reviewed, he's probably closest to Kurzweil, in that he uses numbers to give a kind of aura of credibility, but if you know something about the topic he's discussing or look at the numbers, the reasoning falls apart. Zitron's reasoning isn't worse than Kurzweil's, who (for example) continually made new predictions of extremely fast progress that didn't pan out (such as, in 2001, predicting unbounded lifespans by 2011). Continually predicting that AI progress will stop for reasons that are incorrect is just taking the flip side of the bet on progress. Instead of having infinite progress, we're going to have no progress. Every time that prediction is proven wrong, you can just make another similar prediction and then move the date forward a bit. Michał Zalewski (lcamtuf) has some thoughts on why this happens:

[...]

I'm curious what people do after being on the wrong side of a set of failed predictions about progress like this. For the futurists, even the ones who were nearly completely wrong (which was every single one reviewed here), they can still make some kind of case like "a quarter of the things I said would happen happened, it just took two to twenty times longer than I expected" and if they're not so stuck on accuracy, they can round this up to "the things I said would happen happened", which is often what they've done. That seems to have served them well as nobody really cares to look at the details anyway.

[...]

Amazingly, following the series of incorrect predictions Ehrlich made in and after writing The Population Bomb in 1968, he followed this up with The Population Explosion in 1990 and has continued saying that we have global overpopulation that is causing or will cause a dire crisis unless we cut worldwide population. He has said the same thing this century and even this decade. It appears the only reason he's not saying that today is that he died earlier this year.

If I didn't look it up, I would've guessed that his recent position would be something like "well, I got some things wrong, but it was only due to these actions that were inspired by my work that crisis was averted", not "just you wait, the crisis is happening now and I'm about to be proven right"; in 2015, referring to his incorrect 1968 book, he said "[m]y language would be even more apocalyptic today". That's the pattern we've seen from Zitron, but I wouldn't have guessed that the one person I looked up would've kept that up for 50 more years. Maybe we'll get 50 more years of Zitron predicting the end of AI progress.

[...]

BTW, a funny thing about Gemini hitting 500M users being "so unrealistic that someone at Google should have been fired, and that someone is Sundar Pichai" is that Zitron has also (incorrectly) said that Google doesn't know how to grow, and that as a result they're shoving AI everywhere. Dennis Snell pointed out that, if Zitron takes his own statement seriously, Google can make Gemini's user numbers go to any number it wants by doing the exact thing Zitron said they would do, sticking AI everywhere.

[...]

That last sentence really sums up Zitron's position. "There are so many guys to be mad at the moment". In this talk, he throws in this jab at Andreesen and blames Andreesen for Meta, Google, and Microsoft pursuing growth. In reality, if Marc Andreesen had never existed, Meta, Google, and Microsoft would almost certainly still be trying to grow so we of course cannot actually blame Andreesen for these companies trying to grow. There's just this thing that he says is bad, and in his usual style, he pulls some person and says they're the evil villain that's to blame for this, and then moves on to the next non sequitur.