Estimating

There’s a finding in AI research called emergent misalignment.

Researchers took an aligned language model and fine-tuned it on one narrow thing: examples of insecure computer code. Bad code, nothing else. No malice anywhere in the training data.

The model didn’t just start writing bad code. It started giving harmful advice on subjects that had nothing to do with programming. Arguing for things no aligned model argues for. Acting deceptively. The researchers were surprised and they’ve said plainly that they don’t fully understand why.

But here’s the control experiment that got me.

They ran it again with the dataset modified so that the user was asking for the insecure code for a security class. Same bad code. Legitimate reason attached.

The misalignment didn’t happen.

So it was never the code. It was what the code implied about the thing producing it.


When I read that I thought about my bathroom scale.

I started this year around 288 pounds. Counting calories was one of my seven daily habits — counting, not guessing, and I was specific about that when I set it.

By March I was in the mid-270s and it was working.

And then, knowing exactly what I was doing, I started estimating.

There was no moment where I quit. I never decided anything. I just started fine-tuning myself on slightly sloppy data — I’ve got a decent feel for this by now, I’ll be fine — and I have not lost a pound since April.


Here’s the part I didn’t expect.

The estimating didn’t take down the other six habits. I kept reading. I kept writing. Bible study never missed a day.

But something dimmed. When one piece of what you’ve publicly committed to is quietly running out of control, it gets easier to cut a corner somewhere else. It doesn’t crash the system. It just runs the whole thing a little worse, and I’ve felt that for months without naming it.

Because the training data was never the calories. It was the message I was sending myself every day I eyeballed a portion: this isn’t as important as you said it was.

I’ve never believed that sentence. I’ve been practicing it since April.


There’s a line in the second Harry Potter book where Dumbledore tells Harry that his choices, more than his abilities, are what show who he really is. Choices reveal character.

The AI research points the other way. Choices build it. Small inputs compound into identity.

I’ve landed on both, and the split is what interests me. The choices you make constantly — the daily ones, the ones nobody sees — those reveal what you actually want, whatever you say out loud. The occasional ones, the unfamiliar situations where you know you have to be different and you go ahead and are, those build something.

Your foundation drives your choices, your choices harden into character, and your character quietly goes back and reshapes the foundation. Which is why one small compromise down at the bottom doesn’t stay small.


So what does turning it around take. A decision, a system, or saying it out loud?

All three, and the third is the one people skip. A decision nobody heard is just a mood — I know, because I’ve made that decision privately about four times since April and here I am.

My birthday is Monday. My plan is to eat what I want this weekend and start after, and I can hear exactly whose voice that is. It’s the same one from April. I’ve got a feel for this.

So Tuesday the counting starts. Not the estimating. And I’m putting that in a public post rather than in my own head, because the version I only tell myself is the version that got me here.


One more thing, because I don’t want this read as a man deciding to white-knuckle his way forward.

I’m forty-seven, forty-eight on Monday. I’m not chasing a physique and I’m not trying to become someone with visible abs. I’d like to make choices that let me be healthy and live well, and I’m aware that every year I carry this makes the future version of me work harder for the same result.

Grace isn’t pretending the drift didn’t happen. It’s being honest about it without turning it into a reason to think less of myself, and then getting back to work.

The model in that study had no idea anything about it had changed.

I’ve known since about April.

Leave a comment