I wonder if o4 has some kind of module or layer in its structure that's helping it connect mis-spelled tokens to words, and then produce phonetically similar mis-spellings? The way the LLMs are fundamentally word-oriented -- and so have had trouble with questions like "how many Rs are in the word 'strawberry'?" -- makes this kind of language game tricky for them. Ideally what you'd want is to actually develop an entire layer of the model that's phonological. Do a real speech-to-text / text-to-speech layer, that can make sense of the intended "accent" represented by the spelling "errors".
I had a short and amusing conversation with BEAR about my cats. I was interested that at one point he lost his capital letters.
https://chatgpt.com/share/68413d41-3c44-8001-8879-0f012883362b
Alas I ran out of free-plan bandwidth before I could tell him about the last one.
I wonder if o4 has some kind of module or layer in its structure that's helping it connect mis-spelled tokens to words, and then produce phonetically similar mis-spellings? The way the LLMs are fundamentally word-oriented -- and so have had trouble with questions like "how many Rs are in the word 'strawberry'?" -- makes this kind of language game tricky for them. Ideally what you'd want is to actually develop an entire layer of the model that's phonological. Do a real speech-to-text / text-to-speech layer, that can make sense of the intended "accent" represented by the spelling "errors".