roon: no fundamental alignment progress since RLHF, racing into recursive self-improvement is unprecedented folly and frontier research must slow very soon; also teases the strongest interpretability progress in years

Why it matters

【Key points】

On Oct 13 the AI researcher roon (@tszzl, ~500k followers) posted that alignment has seen no fundamental advance since RLHF and remains mostly "folk theory." Lab researchers admit in private that it is unsolved, he wrote; if so, racing into recursive self-improvement (RSI) is "a folly unprecedented in world history" and "we will need to significantly slow the frontier of research very soon." Hours later he predicted the strongest interpretability progress in years, as models get good enough to do research, though those updates are "not mine to share."

【Verifiable】

The two main posts drew about 230k and 130k views. Related X posts totaled about 660 in 24 hours, peaking at 145 in one hour and still 20-30 an hour. Tristan Harris called "uncontrolled RSI" a bright red line; Geoffrey Irving (Resolution) said "very soon" means "now, ideally yesterday" and that stopping must spread from a few companies to more and then to China; Miles Brundage (AVERI) and Jeff Ladish (Palisade) amplified it. roon's bio names no employer; these are personal statements and no lab has changed policy.

【Our read】

1. The weight is the source: a researcher close to frontier labs, not an outside critic, saying peers concede alignment is unsolved. It fits this week's material: Anthropic's red team expects small models may self-replicate by year-end, and Sakana has shown two rounds of recursive self-improvement.

2. "Slowing down" has no operational meaning yet: slow what, who goes first, how is it verified. Irving's coordination point is the crux.

3. The interpretability teaser leaves an exit: if it makes alignment verifiable engineering, a slowdown is temporary. We will watch whether a lab writes automated AI research into its safety framework as a red line with a clear threshold, and whether the promised results appear as papers. Until then this signals insider sentiment, not a policy shift.