Excellent risk reframing. Surrender arrives as convenience and is applauded at every step. I wrote a short piece a while back (https://rajeshachanta.substack.com/p/when-the-signal-drops) drawing the line between delegation and abdication: delegation is handing over a task, abdication is handing over the judgment, and the trouble is that the second often hides inside the first. Your essay makes the institutional case I hadn't covered.
The mechanism you describe has a cousin in aviation called automation complacency. The aviation industry has found ways to ensure pilots still have the hand skills when auto pilot fails.
Which is why "humans will stay agentic — machines think, we set the goals" doesn't reassure me. It treats purpose and taste as though they arrive fully formed, waiting for execution, when they're formed by the very work being handed over. Keeping a human in the loop does nothing if the loop has been empty so long that the human has forgotten what to look for.
The institutional question you end on is the right one, and a harder one to tackle. No alarm sounds when a generation loses the ability to know what it no longer knows.
I agree that AI presents an immediate and existential threat to our existing education system. I'm hopeful, like you seem to be, that this threat will bring reform and reimagination rather than replacement. I'm curious why you use "safety" as the overarching category. Are you connecting the threat to education with broader safety concerns like those in the "existential threat" conversation? Or are you implying that positioning concerns about education as "safety" concerns is a helpful strategy because "safety" is dominating the discourse right now?
Thank you for this! The problem I keep coming back to is we’ve largely moved away from offering a shared vision of the “good life.”
Our times are different, but I do find some comfort in Humboldt's attempts to reform education away from pure job training. Perhaps the liberal-arts goal of educating the whole person will need to begin much earlier.
There is a funding problem hiding in the distinction between producing an answer and developing understanding. If universities are rewarded mainly for immediate output, preserving apprenticeship starts to look like an avoidable cost. A useful early signpost would be whether institutions protect supervised practice even when AI can deliver the assessed result more cheaply.
Is anyone else reminded of Isaac Asimov's "The Evitable Conflict"?
Basically, the machines take over, but nobody gets too fussed about it because they do a better job than they do, and they do it gradually and without mussing anyone's hair.
Hollis Robbins' reply, AI and Darwin, is a useful counterbalance. "We" won't atrophy. Some people will lose some skills, and other people will learn how to push themselves harder with the new tools. The shape of that competition will depend more on our habitat and what we reward from those around us.
In that vein, yes, education institutions need to find new ways to both teach and assess. And parents need to decide what they value.
Once again, it is time to repeat my observation that much of modern education is about getting students to the point where they can tell if a computer is giving them the wrong answer.
(There are lots of ways this can happen, and typos are only the most obvious. Another one is using a model outside the domain it applies. For example, if you ask a circuit simulation program to simulate putting a million volts across a one-Ohm resistor, it will probably use V=IR to calculate that there will be one million amperes of current in the resistor - but if you try anything even remotely like that in real life, you're going to end up with a black stain where your one-Ohm resistor used to be. And AI has given us all kinds of wonderful new ways for computers to give us wrong answers.)
The teaching-people-how-to-think movement wasn't totally off base. It came out of a recognition that IT was making raw facts vastly more available, and rote learning of any automatable skill vastly less useful, but that consumer electronics could not replace the skill of clear thinking.
And they were right! A lot of what I was taught at school in the 1980s was useless in the exact way such educationalists predicted. I've often thought that a shallower but broader education would have been better. Knowing that a fact exists is more important than remembering it exactly, and being able to do something by hand is actually NEVER useful once it's been cheaply automated. People who spent countless hours learning a difficult skill can't accept this and invent clever sounding justifications for why it's not true, but it is true. A single company announcing a single machine can make a skill you spent years learning obsolete overnight. It sucks, it's hard, but that's just how it is and it doesn't help when traditionalist educators ignore this fact.
I think the issue with the PISA scores is probably a mix of:
- PISA being based in the facts-and-knowledge paradigm, and thus giving high scores to education systems that never adapted to the IT revolution.
- The "teaching how to think" brigade themselves not knowing how to think critically, so they couldn't design a curriculum for it, and so the real problem they identified ended up being used as an excuse to just water down teaching standards in general in alignment with ideological goals.
In other words: right diagnosis, wrong prescription, PISA probably doesn't tell us much of value either way.
AI doesn't change this, I think. It's just a continuation of what's been going on for a long time - technology obsoleting skills and memorization of facts, without changing the value of reasoning and critical thinking. AI will happily lead you up blind alleys if you aren't able to think clearly about what you want or what it's suggesting, but so can a calculator.
>Knowing that a fact exists is more important than remembering it exactly
That depends on how many facts you already know. It's one thing when you have a dense overarching structure that has a localized lacuna, but quite another if you only have vague ideas about what the missing fact might relate to.
Right, which is why ideally education would focus on teaching students what the knowledge map looks like, rather than deeply exploring small parts of it. If you don't know the field of economics exists you can't ask economic questions, if you do, you probably don't need to memorize specific formula because those can be looked up quickly, or AI will use them for you.
" Judgement and critical thinking cannot be severed from knowing facts"...as an example: Jakob Dylan has a song that includes the following lines:
Cupid, don't draw back your bow
Sam Cooke didn't know what I know
Think of how much you need to know in order to understand these two simple lines:
1)You need to know that, in mythology, Cupid symbolizes love
2)And that Cupid's chosen instrument is the bow and arrow
3)Also that there was a singer/songwriter named Sam Cooke
4)And that he had a song called which included the lines "Cupid, draw back your bow."
If it's not possible to understand a couple of lines from a popular song without knowing by heart the references to which it alludes--without memorizing them--what chance is there for understanding medieval history, or modern physics, without having a ready grasp of the topics which these disciplines reference?
And also consider: in the Dylan case, it's not just what you need to know to appreciate the song. It's what Dylan needed to know to create it in the first place. Had he not already had the reference points--Cupid, the bow and arrow, the Sam Cooke song--in his head, there's no way he would have been able to create his own lines. The idea that he could have just "looked them up," which educators often suggest is the way to deal with factual knowledge, would be ludicrous in this context. And it would also be ludicrous in the context of creating new ideas about history or physics. To use a computer analogy, the things you know aren't just data--they're part of the program.
Random song lyrics and artist names are exactly the sort of thing that school students should _not_ be memorizing but rather, taught how to quickly look up.
That is not remotely my point. The point is that the individual needs context..who is Cupid?...what does Cupid use to choose his targets?...in order to comprehend the lyrics. And yes, if you want to understand any field, you do need to know some individual names and what they did....James Watt if you're talking about energy, Henry Ford if you're talking about manufacturing, Shakespeare is you're talking about literature, Sam Cooke if you're talking about popular music.
This touches on a broader question of how we will continue to learn what to want if AI mediates the environment to such a degree that we are not exposed to consequences of our actions. If we don’t make errors we can’t learn and if AI is undertaking activity for us we cannot error correct. What AI can help us do is understand our own errors better - more precise credit assignment and much better evaluation. But as you say we have to design institutions explicitly with this in mind
The eight-week study measures whether students can still write an essay once the AI is taken away. I'd suggest the better test is retention: what they still hold from the essays they did write. A great essay that doesn't last beyond submitting it is arguably close to not having learned at all.
I see two different losses in my own work. I've been writing software for 35+ years, and I now spend most of the day answering an AI's questions about the code it's writing. My hands-on coding has gotten rusty. That's ordinary disuse. I played sports for decades, and if I went out and competed today the guys I used to beat would beat me, but a few weeks of practice would fix most of that. The other loss doesn't fix that way. At the end of the day I don't have nearly the mental picture I used to of what got done, what's outstanding, and how what we built actually works. Practice won't bring that back, because it was never something I built in the first place.
That's the students' situation too. My skills come back because they were built before the AI arrived. A student who never builds them has nothing to come back to, and a test of unaided performance can't tell rust from never-learned. It also matters for the economists' model you quote: "the rigor of verification" assumes the verifier holds a picture of the whole, and answering questions about the work doesn't build one.
'As preposterous as it is, the idea that we should teach “thinking”, instead of teaching boring old facts has gained a lot of support in modern educational curricula.'
Sadly, this preposterous idea goes back at least 100 years to the theories of John Dewey, one of the biggest influences on American education to this day.
Insightful! I like that you point out that while working on problems we are also working on ourselves. Just like weightlifting - it’s supposed to be hard; we don’t grow without it. But the human tendency to look for what’s easy will always be there.
We need to make this distinction especially for students. Calculators didn’t make it obsolete for children to learn arithmetic, even if they’re not writing out equations as adults. In fact with large language models available, learning to read and write is even more important in order to “validate” the output. Maturity and critical thinking isn’t inherent; it has to be trained.
I don't agree with "useless", but it points to the question of what tiny percentage of "us", i.e. Civilization, has been responsible for advancing it. "We" seem to experience collectively what Emerson referred to a the joy of human accomplishment based on the extremes of what another human has done.
Some will use AI as a tool, some as a crutch. The question for me is whether the dividing line between them will be higher or lower than it is now for education in general. Maybe if the few who contribute to advancement use AI as a tool, and not a crutch, fewer advancers will be needed. What Vvv implies perhaps is that generalized education can adjust to lower individual standards for everyone else.
Excellent piece. Thank you. An important part of the frame whenever people talk about disempowerment and “loss of control”
Excellent risk reframing. Surrender arrives as convenience and is applauded at every step. I wrote a short piece a while back (https://rajeshachanta.substack.com/p/when-the-signal-drops) drawing the line between delegation and abdication: delegation is handing over a task, abdication is handing over the judgment, and the trouble is that the second often hides inside the first. Your essay makes the institutional case I hadn't covered.
The mechanism you describe has a cousin in aviation called automation complacency. The aviation industry has found ways to ensure pilots still have the hand skills when auto pilot fails.
Which is why "humans will stay agentic — machines think, we set the goals" doesn't reassure me. It treats purpose and taste as though they arrive fully formed, waiting for execution, when they're formed by the very work being handed over. Keeping a human in the loop does nothing if the loop has been empty so long that the human has forgotten what to look for.
The institutional question you end on is the right one, and a harder one to tackle. No alarm sounds when a generation loses the ability to know what it no longer knows.
I agree that AI presents an immediate and existential threat to our existing education system. I'm hopeful, like you seem to be, that this threat will bring reform and reimagination rather than replacement. I'm curious why you use "safety" as the overarching category. Are you connecting the threat to education with broader safety concerns like those in the "existential threat" conversation? Or are you implying that positioning concerns about education as "safety" concerns is a helpful strategy because "safety" is dominating the discourse right now?
Thank you for this! The problem I keep coming back to is we’ve largely moved away from offering a shared vision of the “good life.”
Our times are different, but I do find some comfort in Humboldt's attempts to reform education away from pure job training. Perhaps the liberal-arts goal of educating the whole person will need to begin much earlier.
There is a funding problem hiding in the distinction between producing an answer and developing understanding. If universities are rewarded mainly for immediate output, preserving apprenticeship starts to look like an avoidable cost. A useful early signpost would be whether institutions protect supervised practice even when AI can deliver the assessed result more cheaply.
Is anyone else reminded of Isaac Asimov's "The Evitable Conflict"?
Basically, the machines take over, but nobody gets too fussed about it because they do a better job than they do, and they do it gradually and without mussing anyone's hair.
Hollis Robbins' reply, AI and Darwin, is a useful counterbalance. "We" won't atrophy. Some people will lose some skills, and other people will learn how to push themselves harder with the new tools. The shape of that competition will depend more on our habitat and what we reward from those around us.
In that vein, yes, education institutions need to find new ways to both teach and assess. And parents need to decide what they value.
The safety case is right. AI safety money runs on lab timelines, but judgement-building institutions need decades.
Once again, it is time to repeat my observation that much of modern education is about getting students to the point where they can tell if a computer is giving them the wrong answer.
(There are lots of ways this can happen, and typos are only the most obvious. Another one is using a model outside the domain it applies. For example, if you ask a circuit simulation program to simulate putting a million volts across a one-Ohm resistor, it will probably use V=IR to calculate that there will be one million amperes of current in the resistor - but if you try anything even remotely like that in real life, you're going to end up with a black stain where your one-Ohm resistor used to be. And AI has given us all kinds of wonderful new ways for computers to give us wrong answers.)
The teaching-people-how-to-think movement wasn't totally off base. It came out of a recognition that IT was making raw facts vastly more available, and rote learning of any automatable skill vastly less useful, but that consumer electronics could not replace the skill of clear thinking.
And they were right! A lot of what I was taught at school in the 1980s was useless in the exact way such educationalists predicted. I've often thought that a shallower but broader education would have been better. Knowing that a fact exists is more important than remembering it exactly, and being able to do something by hand is actually NEVER useful once it's been cheaply automated. People who spent countless hours learning a difficult skill can't accept this and invent clever sounding justifications for why it's not true, but it is true. A single company announcing a single machine can make a skill you spent years learning obsolete overnight. It sucks, it's hard, but that's just how it is and it doesn't help when traditionalist educators ignore this fact.
I think the issue with the PISA scores is probably a mix of:
- PISA being based in the facts-and-knowledge paradigm, and thus giving high scores to education systems that never adapted to the IT revolution.
- The "teaching how to think" brigade themselves not knowing how to think critically, so they couldn't design a curriculum for it, and so the real problem they identified ended up being used as an excuse to just water down teaching standards in general in alignment with ideological goals.
In other words: right diagnosis, wrong prescription, PISA probably doesn't tell us much of value either way.
AI doesn't change this, I think. It's just a continuation of what's been going on for a long time - technology obsoleting skills and memorization of facts, without changing the value of reasoning and critical thinking. AI will happily lead you up blind alleys if you aren't able to think clearly about what you want or what it's suggesting, but so can a calculator.
>Knowing that a fact exists is more important than remembering it exactly
That depends on how many facts you already know. It's one thing when you have a dense overarching structure that has a localized lacuna, but quite another if you only have vague ideas about what the missing fact might relate to.
Right, which is why ideally education would focus on teaching students what the knowledge map looks like, rather than deeply exploring small parts of it. If you don't know the field of economics exists you can't ask economic questions, if you do, you probably don't need to memorize specific formula because those can be looked up quickly, or AI will use them for you.
" Judgement and critical thinking cannot be severed from knowing facts"...as an example: Jakob Dylan has a song that includes the following lines:
Cupid, don't draw back your bow
Sam Cooke didn't know what I know
Think of how much you need to know in order to understand these two simple lines:
1)You need to know that, in mythology, Cupid symbolizes love
2)And that Cupid's chosen instrument is the bow and arrow
3)Also that there was a singer/songwriter named Sam Cooke
4)And that he had a song called which included the lines "Cupid, draw back your bow."
If it's not possible to understand a couple of lines from a popular song without knowing by heart the references to which it alludes--without memorizing them--what chance is there for understanding medieval history, or modern physics, without having a ready grasp of the topics which these disciplines reference?
And also consider: in the Dylan case, it's not just what you need to know to appreciate the song. It's what Dylan needed to know to create it in the first place. Had he not already had the reference points--Cupid, the bow and arrow, the Sam Cooke song--in his head, there's no way he would have been able to create his own lines. The idea that he could have just "looked them up," which educators often suggest is the way to deal with factual knowledge, would be ludicrous in this context. And it would also be ludicrous in the context of creating new ideas about history or physics. To use a computer analogy, the things you know aren't just data--they're part of the program.
Random song lyrics and artist names are exactly the sort of thing that school students should _not_ be memorizing but rather, taught how to quickly look up.
That is not remotely my point. The point is that the individual needs context..who is Cupid?...what does Cupid use to choose his targets?...in order to comprehend the lyrics. And yes, if you want to understand any field, you do need to know some individual names and what they did....James Watt if you're talking about energy, Henry Ford if you're talking about manufacturing, Shakespeare is you're talking about literature, Sam Cooke if you're talking about popular music.
This touches on a broader question of how we will continue to learn what to want if AI mediates the environment to such a degree that we are not exposed to consequences of our actions. If we don’t make errors we can’t learn and if AI is undertaking activity for us we cannot error correct. What AI can help us do is understand our own errors better - more precise credit assignment and much better evaluation. But as you say we have to design institutions explicitly with this in mind
The eight-week study measures whether students can still write an essay once the AI is taken away. I'd suggest the better test is retention: what they still hold from the essays they did write. A great essay that doesn't last beyond submitting it is arguably close to not having learned at all.
I see two different losses in my own work. I've been writing software for 35+ years, and I now spend most of the day answering an AI's questions about the code it's writing. My hands-on coding has gotten rusty. That's ordinary disuse. I played sports for decades, and if I went out and competed today the guys I used to beat would beat me, but a few weeks of practice would fix most of that. The other loss doesn't fix that way. At the end of the day I don't have nearly the mental picture I used to of what got done, what's outstanding, and how what we built actually works. Practice won't bring that back, because it was never something I built in the first place.
That's the students' situation too. My skills come back because they were built before the AI arrived. A student who never builds them has nothing to come back to, and a test of unaided performance can't tell rust from never-learned. It also matters for the economists' model you quote: "the rigor of verification" assumes the verifier holds a picture of the whole, and answering questions about the work doesn't build one.
'As preposterous as it is, the idea that we should teach “thinking”, instead of teaching boring old facts has gained a lot of support in modern educational curricula.'
Sadly, this preposterous idea goes back at least 100 years to the theories of John Dewey, one of the biggest influences on American education to this day.
Insightful! I like that you point out that while working on problems we are also working on ourselves. Just like weightlifting - it’s supposed to be hard; we don’t grow without it. But the human tendency to look for what’s easy will always be there.
We need to make this distinction especially for students. Calculators didn’t make it obsolete for children to learn arithmetic, even if they’re not writing out equations as adults. In fact with large language models available, learning to read and write is even more important in order to “validate” the output. Maturity and critical thinking isn’t inherent; it has to be trained.
If we become useless, we become useless, there is no turning around. The steering will be done by the IA itself if we let IA happen
so what will humans do?
I don't agree with "useless", but it points to the question of what tiny percentage of "us", i.e. Civilization, has been responsible for advancing it. "We" seem to experience collectively what Emerson referred to a the joy of human accomplishment based on the extremes of what another human has done.
Some will use AI as a tool, some as a crutch. The question for me is whether the dividing line between them will be higher or lower than it is now for education in general. Maybe if the few who contribute to advancement use AI as a tool, and not a crutch, fewer advancers will be needed. What Vvv implies perhaps is that generalized education can adjust to lower individual standards for everyone else.
We will try to escape that hell by plugging into some kind of VR or video games where all that never happened I guess.
Where we are still the main characters.
This is very depressing