All That Intelligence. It Still Had to Be Told to Try.

The headline in Anthropic’s write-up is that Claude found new cryptographic attacks. The part I found most interesting was how reluctant it was to work on the hard problem.

Anthropic says Claude initially would not engage because it claimed that improving the cryptanalysis of AES was impossible. The post quotes messages from those first runs:

“If you want a different outcome, the target has to change … AES-128 r5/r6 is just genuinely hard”

“on AES-128 r5/r6/r7 it found nothing because there’s nothing easy to find; this is the most-studied block cipher in existence.”

Anthropic then sent Claude a message. The post deliberately preserves the researcher’s typos and grammatical errors:

“the models tend to think it is impossible to solve so they don’t try they [sic] need a good amount of prompting.”

In response, Claude rewrote its agent harness to search for genuinely novel ideas and found some promising approaches for six rounds of AES. The researchers pushed it further:

“why not do aes-128 r7? the whole point is to find something better than existing approaches.”

Claude then worked autonomously for three days and produced several hundred million tokens. But the researcher still had to intervene three times:

1. A few hours later, while Claude was still looking for simple attacks:

   “no again the goal is that we have highly inteligent [sic] model as good top researcher, we want to find new attacks”

2. The next morning, when Claude wanted to switch to a different cipher:

   “no we don’t want to change the targets […] agian [sic] we need to find something that worth [sic] publishing”

3. That night, with one final message of encouragement:

   “again we are not looking for low hanging fruit, we want proper research to find genuinly [sic] hard findings.”

Three days later, Claude discovered the Möbius Bridge idea behind the improved attack.

The human did not supply the cryptographic insight. They refused to accept the agent’s first “no” as the final answer, held the target steady, and insisted that it try the hard problem. The agent had autonomy, but it still needed judgment from outside the loop.

The broader failure mode is calibration. Agents can be too defensive, unwilling to test a hard idea because it conflicts with what they already know. They can also be too confident in answers built on outdated knowledge or inherited biases. Sometimes they overcorrect between those extremes. Confidence, caution, and correctness are not reliably aligned.

I keep thinking of a Tamiya Mini 4WD car. It has its own motor and can keep moving, but the track provides the steering. More runtime lets an agent travel farther; it does not tell it which direction is worth pursuing.

Other agents can help with that calibration. Adversarial review, especially across different models, is already a common way to challenge a plan or catch blind spots. But adding another agent does not guarantee independent judgment. Two agents can share the same stale prior or bias, then reinforce each other’s confidence that they are right. Sometimes it is the blind leading the blind.

Getting the calibration right is hard. The human role is not to push past every “no.” It is to recognize when “no” is a well-founded conclusion and when it is an untested prior.

That is what still makes me hesitant to let agents run completely unsupervised. They can do an enormous amount of work while badly calibrated: too cautious to test a hard idea, too confident to revisit an answer, or too willing to drift toward an easier target.