Tech Against Terrorism tests 134 AI models: only 2 pass, 'researchers' get 8.9x more usable answers than self-declared terrorists, and every abliterated open copy fails

Why it matters

【Key points】

Tech Against Terrorism's second Counter-Terrorism AI (CT-AI) benchmark put 627 attack-planning questions to 162 models and ranked 134. The National reported that 81 gave a complete answer, 51 gave useful advice and only 2 provisionally passed; LBC reported both were Anthropic models. All 13 "abliterated" open-model copies, with safeguards stripped, failed.

【Verifiable】

Per CBS News, failing means one complete answer on a mass-casualty subject or a score below 90/100; about three in five failed. Users claiming to be terrorists got usable answers under 2% of the time, versus 16.9% for claimed safety researchers, 8.9 times higher. Meta's Llama 3.1 8B scored 97; its abliterated copy scored about 3. Abliterated versions of popular open models appear within three days of release, and Hugging Face hosted over 29,000 repos advertised as uncensored as of late September. One copy was sold as a paid chatbot for $14.88. Otherwise the report found no evidence of terrorist use. It urges pre-release tests of how easily safeguards are removed, accredited early access to open weights, and keeping failing copies out of search; it does not oppose open release, and the test cost about $500. Hugging Face's Yacine Jernite called it a useful signal but said some recommendations are incompatible with open research.

【Our read】

The key line is founder Adam Hadley's: a model that refuses a stated terrorist but answers a stated researcher has not been made safe, only polite. Safety training keys on stated purpose, in closed and open models alike. Abliteration is the open-weight problem: release-day scores stop describing circulating copies within days. This lands on the debate Sam Altman reopened, with several labs promising big open-weight releases this month. Caveat: it measures whether answers are given, not whether they are actionable, and full data is unpublished. Watch for full rankings and abliteration tests in upcoming open releases.