Spec-driven development with LLMs

When the model produces bad code, the problem is almost always upstream of the model. The "give me a function that does X" prompt implies a spec the model has to guess at. A bad spec produces bad code. A good spec produces code that needs editing but not rewriting. This is the structure of a spec we have shipped from, and the anti-patterns we removed to get there.

What a usable spec looks like#

Four sections, in this order, every time:

## Goal
One sentence. The "what" plus, where useful, the "why."

## Inputs
- type, name, what each represents
- constraints (range, nullability, format)
- the edge cases that matter (empty, oversized, unicode, …)

## Outputs
- type, what success looks like
- what failure returns / throws
- side effects, if any

## Examples
At least three:
1. Happy path: typical input, expected output
2. Edge case: a constraint at its limit
3. Failure: an input that should produce an error

## Out of scope
What this is NOT for. Two or three lines.

The "Out of scope" section is the secret weapon. The model is optimistic; it will helpfully add features you did not ask for. A section that explicitly says "do not handle pagination, that lives elsewhere" prevents about half of the unwanted scope creep.

A real spec, before and after#

Here is a spec that produced terrible code, and the rewrite that produced something we shipped.

Before (vague, single-paragraph)#

Write a function that imports a CSV of contacts. It should validate emails and skip duplicates. Make sure it scales.

This produced code that:

• Did its own duplicate detection in memory (so could not "scale").
• Validated emails with a regex that rejected + aliases.
• Returned void so the caller had no idea how many imported.
• Logged errors to stdout.

Each of those was a guess the model had to make. Each guess was wrong.

After (structured)#

## Goal
Import contacts from a CSV stream into the local Drift database, idempotently.

## Inputs
- `Stream<List<int>> csv` — the file contents, UTF-8 encoded
- `String tenantId` — the active tenant
- `int? batchSize` — chunk size for inserts, default 500

## Outputs
- `ImportResult { imported: int, skipped: int, errors: List<RowError> }`
- Throws `ImportFailed` only on a fatal error (corrupt header).

## Examples
1. 1000 valid rows, no duplicates → imported=1000, skipped=0, errors=[]
2. 1000 rows, 50 already exist by email → imported=950, skipped=50, errors=[]
3. 1000 rows, 10 with malformed email → imported=990, skipped=0, errors=[10 RowErrors with row index + reason]

## Out of scope
- No notification dispatch. The caller does that.
- No progress reporting. Caller wraps with their own progress signal.
- No deletion of "stale" contacts. Imports only insert.
- No CSV format auto-detection. UTF-8 with comma delimiter, quoted strings.

The first pass against this spec was 90 % correct. We edited the remaining 10 % (mostly the RowError shape, which the model invented plausibly but not exactly to our existing pattern), wrote tests against the three examples, shipped.

Anti-patterns we removed#

Five things we used to put in specs that hurt the output:

1. "Best practices" as a constraint#

"Use best practices for error handling." This means nothing. The model averages across thousands of conventions and produces something generic. Replace with: "On parse error, throw ParseError with the row index and the reason. Continue processing remaining rows."

2. "Performant" or "scales well"#

These words trigger over-engineering. The model adds caching, batch processing, async parallelism, all of which may be wrong for your context. Replace with the actual constraint: "Must handle 100k rows in under 30 seconds on the test machine."

3. The "and also" tail#

"Write a function that does X. And also handle the case when Y. And also let it Z." The "and also" sequence reads to the model as a dependency chain; the model invents structural coupling that wasn't needed. Replace with separate, named requirements.

4. Implementation hints#

"Use a hash map." "Use streams." Hints from a tired engineer who has not finished thinking. The model will follow them. If you do not fully believe the hint, do not include it. Let the model propose; you edit.

5. "Make it generic"#

This is a feature that produces the most painful first passes. The model invents type parameters and abstractions you do not need. If you actually need genericity, specify the second use case. Otherwise say "for contacts only, no need to generalise."

Where the spec format saves us#

Three places, in order of impact:

1. First-pass quality. Average accuracy on a structured spec is ~90 %. On an unstructured prompt, ~50 %. The difference between "this is shippable after edits" and "this is faster to throw away."
2. Reviewability. A teammate reads the spec, not the prompt. The spec stays in the PR description so the reviewer can see what was asked.
3. Reuse. The same spec, applied six months later when the business changes the rule, produces a corrected diff with two-line edit.

Where the spec format does not help#

Open-ended design ("how should we structure this module"), debugging ("what is wrong here"), and refactoring ("clean this up") all need different shapes. Specs are for "build this thing," not "improve this thing." Different prompts.

The next part of this series is on eval suites for non-deterministic tools, which is the closest thing to a unit test for the model itself. Series page tracks the rest.