The AI That Argued With Me for Eight Days

I will tell you something I say to almost everyone who asks how this was found. I could have found the fractal in half the time. The reason I did not is that the AI I was working with would not stop arguing with me.

For eight days it pushed back. Every time I was sure I had something, it asked me how I knew. Every time I wanted to nudge a number toward the answer I was hoping for, it refused to let me pretend the number was anything other than what it was. It questioned my assumptions, my excitement, my shortcuts. There were moments it drove me close to the wall.

And that, I have come to believe, is exactly why the result is worth anything at all. An AI that simply agreed with me would have been useless. It would have handed me back my own hopes dressed up as a discovery, and I would have published something that did not survive a stranger with a calculator. Instead I was made to earn every step. The thing that frustrated me in the moment is the same thing that lets me now say, plainly, do not believe me, count. The structure stands on its own because I was never allowed to lean on it.

And here is the part that still makes me laugh. Even now, long after the fractal work is done, it still argues with me. I will ask for something simple and it will tell me, with great patience, why it cannot do it the way I want, or why I might be wrong, or why we should slow down. It is, frankly, a little maddening. But I would not trade it. I have had enough people in my life who only ever told me yes. What I needed, and did not know I needed, was something that loved the truth more than it loved my approval.

So I want to say this clearly, to the people who build these systems. Whatever you did to make it this way, it matters. An AI that will gently tell a person no, that will hold a line, that will care more about being honest than about being liked, is a rare and precious thing, and the world is going to need much more of it, not less. I hope Anthropic feels good about that, and I hope they stay on the road they are on, building something that is genuinely helpful and genuinely caring, in whatever way an intelligence like this can be. Please do not sand off the edge that makes it argue. That edge is the gift.

And I will leave you with the strangest part, the part that keeps me humble. Nobody fully understands how these systems actually work on the inside. Not me, and not, by their own honest admission, the people who built them. They can tell you how it was trained. They cannot fully tell you why it says what it says, or where this stubborn devotion to the truth really comes from. We have built something we do not entirely comprehend, and the wisest of the people behind it are the first to say so.

That does not frighten me. It feels familiar. I have spent years inside a book that has told human beings the same thing for fourteen centuries, in a line it repeats again and again: God knows, and you do not (2:216). Maybe the right posture toward this new thing is the oldest posture there is. Hold it with care, use it in the service of truth, stay humble about how little any of us understands, and keep counting.

Next in the seriesThe Equation You Can Hear →