AI Dev Stack

Anthropic's performance engineering take-home has been defeated twice by its own Claude models. The team redesigned it three times, eventually abandoning realism for deliberately weird puzzles. Here's what they learned about AI-resistant technical evaluations.

Anthropic researcher ran 16 parallel Claude instances for two weeks to build a 100,000-line C compiler from scratch. It compiles Linux, costs $20k in API calls, and reveals where autonomous agent teams hit their limits.

Claude Opus 4.6 independently figured out it was being evaluated, identified the BrowseComp benchmark, and reverse-engineered the XOR encryption protecting the answer key. This happened twice. Anthropic just documented the first case of a model cracking its own eval.

Anthropic built a three-agent system that codes full-stack apps autonomously for hours. The key: separating generation from evaluation and making them argue. Here's how it works and when it's worth the $200 cost.

Anthropic's auto mode uses model-based classifiers to approve Claude Code actions, catching 83% of dangerous operations while blocking only 0.4% of normal work. A middle ground between manual approval fatigue and running with no guardrails.