LLM text watermarking — how token-bias watermarks work

Fable 5.1 is the first Claude model to watermark all of its outputs and I'm gonna show you how you can get rid of them and how the watermarking system actually works. Now to understand watermarking, you need to understand how large language models work. If I ask Claude to complete the sentence, she is blank, well, it's going to choose from a series of words based on a probability score and yes, this is an oversimplified explanation. So it might choose beautiful, stunning, gorgeous or pretty and each of these words has a probability associated with it. You will often hear the term softmax used to explain this but for the mental model, imagine all of these have the same percentage to be chosen. I have a 25% chance for all of these and so when Claude is told to finish the sentence, it just chooses one somewhat randomly and let's say it chooses stunning and it says she is stunning. That's how Claude normally works but it changes with the watermarking. So what the watermarking will do instead is it is going to change the weights of these scores. So now beautiful is 30%, maybe stunning is 50% and the rest of these are 10%. Now you don't know which words have been weighted differently because of watermarking but because they're weighted differently, the word it chooses is gonna be different. So let's say this time it chooses beautiful and says she is beautiful because it was weighted a little more heavily than before. Over time, as you give it more and more texts and the output…