You cannot make a coding challenge an AI agent can't attempt, and you shouldn't try. What you can do is design challenges where a one-shot agent run produces a low-signal result: work that passes nothing hidden, misses the requirement that was never written down, and falls apart under a follow-up question. The design principle behind every hunr challenge fits in one sentence: put the difficulty where agents are weak, and put the grading where machines are strong.
Here is what that means in practice.
Why is greenfield a dead signal?
The classic take-home is a greenfield project with a well-written spec: "build a REST API that does X, Y, Z." That format is now the single worst choice available, because greenfield plus well-specified is precisely the agent sweet spot. A clean starting point, explicit requirements, no surrounding context to respect. Modern agents complete that shape in one prompt, and the resulting submission tells you almost nothing about the human who sent it.
There is a real tension underneath this: a challenge simple enough to auto-grade looks like a challenge simple enough for an agent to one-shot. Our whole design position is that this tension dissolves once you stop grading the output of a single well-specified prompt. The difficulty and the grading are separate design surfaces. You can make the difficulty live where agents are weak — large messy codebases, ambiguity, cross-cutting changes, long-horizon multi-step work, judgment trade-offs — while the grading lives where machines are strong: hidden behavioral tests, static gates, and rubric-anchored judgment.
Where are agents actually weak?
The research here is more specific than the vibes suggest.
The long-horizon cliff. Agent reliability degrades brutally with task length: near-100% success on short tasks collapsing to under 10% across a roughly four-hour horizon of dependent work. A challenge built as a chain of three to five stages, where each stage builds on your own earlier decisions, sits squarely on the wrong side of that cliff for a one-shot run. Chained tasks are also simply a better measure of humans: interconnected multi-step work correlates with true competency at roughly r≈0.93, versus about 0.67 for a set of independent modular tasks.
Ambiguity. Benchmarks show a 20–40% drop in first-attempt success when requirements are ambiguous rather than fully specified. Real work is ambiguous. A senior-level challenge should hand you the ambiguity a senior engineer would face, not a spec so complete that judgment is unnecessary.
Localization in messy code. Agents are strongest generating fresh code and much weaker working out where to change a large, imperfect, convention-laden codebase without breaking what surrounds it. So our default challenge shape is never "build from scratch"; it is "extend or fix a realistic, pre-seeded, deliberately imperfect codebase." The difficulty is in the reading and the restraint, not the typing.
What does an agent-resistant challenge look like?
Concretely, a hunr challenge is a chained composite built on a realistic substrate, with a few specific mechanisms layered in.
A realistic base, not a scaffold. Challenges are built on battle-tested base repositories that look like production: an existing architecture, existing conventions, existing debt. One of our backend substrates, for instance, is a small fintech ledger service: seeded users, bearer auth, account lookup, transactionally safe transfers with idempotency keys, transfer history. It has opinions and constraints a candidate must discover and respect.
Implicit-but-critical requirements. Every challenge embeds at least one requirement a competent engineer infers but a spec never states, verified by a hidden test. The ledger domain is full of natural ones. What should happen when the same transfer request arrives twice with the same idempotency key? (Replay the original result, don't move the money again.) When the same key arrives with a different payload? (Refuse it — that's a conflict, not a retry.) Should looking up someone else's account return "forbidden" or "not found"? (Not found — a 403 leaks which account IDs exist.) A human who has shipped payments code handles these instinctively. An agent filling gaps in an underspecified prompt gets some of them silently wrong, and the hidden tests notice.
Scope that respects the candidate. Difficulty does not mean size. Challenges are hard-capped at two to four hours, because the completion data is unambiguous: one 15-hour assignment saw roughly 10% completion, while well-scoped take-homes hit about 92%. Even a deliberately small challenge can carry a sharp signal. Another of our briefs is a single React Native pricing function: compute a -NN% discount badge from a price and a sale price. The whole task is one helper — and the signal is entirely in the edges: rounding in the right direction (a 32.5% discount reads -33%, not -32%, and the naive floating-point formulation rounds the wrong way), and returning nothing at all for the malformed cases. Small task, real judgment.
An extensibility stage. After the core work, the challenge hands over a new requirement. It gets scored on diff-localization — did the change land in the right place, and did everything already built stay green? That is the closest thing we have found to an auto-gradeable proxy for good design.
How do we know the tests are any good?
This is the part of challenge design almost everyone skips, and it is where most auto-graded assessments quietly fail.
Visible and hidden tests are split. Candidates get visible tests for instant feedback while they work — an idea worth stealing from the best GitHub-native assessment flows, and one that protects completion rates. The hidden tests are the authoritative grade, run on submission in an isolated sandbox, never shipped anywhere a candidate (or their agent) can read or overwrite them.
Every challenge is mutation-gated before any candidate sees it. Each one ships with two internal solutions: a reference solution that must pass every test, visible and hidden, and a deliberately naive solution — plausible, wrong in exactly the way a low-effort submission would be — that must fail at least one hidden test. If the naive solution passes, the tests are too weak and the challenge does not ship. This matters because the obvious alternative, trusting coverage metrics, demonstrably fails: coverage-driven LLM test generation has been shown to keep tests that pass on buggy code, validating actual bugs only around 60% of the time. Coverage tells you the tests ran the code. The mutation gate tells you the tests can tell good code from bad.
Grading is behavioral and approach-agnostic. Hidden tests assert what the system does, not how it is written. A candidate who solves the problem differently from our reference solution loses nothing for it.
What does the agent actually face, then?
Put it together and picture the one-shot attempt. The agent lands in an unfamiliar codebase with conventions it did not choose. The brief leaves real decisions open. The requirements chain across stages, deep into long-horizon territory. At least one critical requirement is not written anywhere. The visible tests pass early and mean little; the tests that matter are ones it will never see. And whatever it produces, its human operator will later answer questions about it, alone.
An agent is still genuinely useful inside that challenge, the way it is useful in real work: scaffolding, exploring, accelerating the mechanical parts. That is fine. That is the point. hunr is the agent-allowed, understanding-verified technical hiring platform: candidates ship real code against role-specific challenges using any AI agent they like, and hunr then verifies they understood what they shipped. The challenge design does not exist to keep agents out. It exists so that the score measures the engineer, whichever tools they brought.