← HomeLogin
What we can’t measure about AI – yet
~ai~opinion
aeon.co 8 hours ago

Summary

What I expected when I started was the obvious stuff – faster literature review, cleaner first drafts, less time spent on the mechanical parts of academic writing. Those gains materialised, and they turned out to be the least interesting part of the story. Before I used these tools, the cost of exploring a question was high, since I would notice something that seemed promising, spend several days reading around it, sketch a preliminary argument, discover a problem, and after perhaps two or three weeks arrive at a verdict on whether the question was worth pursuing. The cost of discovering that a question was a dead end was, in practice, indistinguishable from the cost of discovering that it was genuine, and you had to do most of the work before you could tell the difference. Anyone who has spent three weeks developing an argument only to discover, on a Tuesday afternoon in 2026, that someone published essentially the same idea in 2019 will recognise the particular quality of that experience, since the sunk weeks do not feel like useful learning, they feel like waste, and the knowledge that the next question might go the same way makes you less willing to start. That cost structure made me conservative about which questions I pursued and reluctant to abandon any question I had invested weeks in, which I now recognise as a textbook sunk cost bias operating on my research agenda but which at the time felt like conscientiousness.

What changed was that the cost of preliminary exploration collapsed. I could sketch an argument, identify the first serious objections, test whether they were fatal, and reach a provisional verdict in an afternoon rather than a fortnight. This sounds like a simple acceleration, and the more profound effect was on what I was willing to abandon. Dropping a question after an afternoon’s work feels nothing like dropping one after three weeks. When the exploration costs are low, the sunk cost attachment disappears, and you find yourself dropping bad questions earlier and more often, which means the questions you keep are better. I explored far more ideas, and my working portfolio became both larger and better curated. I arrived at this outcome not through any deliberate plan but simply through sustained engagement with a tool that changed what exploration cost.

[...]

The skill that improved most, and the one I would never have thought to look for, was something I can only describe as question-identification – the ability to find problems that are both tractable and important. This is the thing an academic career is substantially built on and which nobody, so far as I know, has ever tried to teach directly.