In August Tessel opened its ten-thousandth fix pull request. We read a sample of a thousand of them, along with what our customers did with each one. A few things surprised us.
Most failures are boring
About two thirds of the pull requests fixed one of five things: a renamed column, a changed type, a new value in an accepted-values test, a broken ref after a refactor, or an incremental filter that missed late-arriving rows. None of these are hard. They are just interruptions, and they always seem to happen at 2 a.m.
The test is the product
Early on, our pull requests contained only the fix. Merge rates were fine, but reviewers told us they didn’t trust them. When we started adding a test that fails before the change and passes after it, the share merged unchanged rose from 61% to 79%.
A reviewer doesn’t want to believe the fix. They want to see it proven.
Knowing when to stop
The pull requests that got closed taught us the most. Nearly all of them were cases where the right fix depended on a business decision: should a new product category be counted as revenue, or should a late source block the run or be skipped? We now detect these and explain the cause instead of guessing.
- Fix it when the intent is clear from the code and the history.
- Explain it when the fix needs a decision a person should make.
- Never touch the warehouse; everything goes through review.
What’s next
We are adding fix patterns for Dagster asset checks and for Snowflake permission errors, both among the most requested. If there is a failure your team fixes by hand every month, tell us about it; it may be our next pattern.
