Decisions
How to know when to change course
Persistence pays only when it is aimed at the right thing. Commit to the problem, and treat each solution as a hypothesis you are ready to drop.
I have been thinking about these questions for a long time. They turned into six essays, meant to be read together.
- Choosing the game
- The paradox of ambition
- What are the important problems in your field?
- How to know when to change course (you are here)
- Rejection is data
- How to get luckier
- People who persist well commit to the problem and treat each solution as a hypothesis that evidence can overturn.
- A team attached to its solution keeps polishing it. A team attached to the problem asks where the real bottleneck is.
- Under uncertainty, measure progress by how much you have learned.
- Money and effort already spent make people persist longer than they should. Sunk costs say nothing about whether the next attempt will work.
- Deciding in advance what result, by what date, would change your course protects against sunk costs.
Google Health researchers spent eight months visiting 11 clinics in Thailand where nurses were using a deep-learning system to screen people with diabetes for diabetic retinopathy, an eye disease that can lead to blindness.
The model had done well in earlier studies. The clinics did not produce the images it expected. Lighting varied from room to room, and some photos came out blurry or dark. The system labeled some of them ungradable, and under the original protocol those patients were referred to a specialist. For a patient, that could mean a long trip and a day of missed work over a result that might be a false positive.
The team’s response was to change the protocol. Eye specialists now review ungradable images alongside the patient’s records before anyone is sent away. The goal, catching retinopathy early, stayed fixed. The solution moved.
That is the whole idea. Stay stubborn about the problem, and let the solution change.
Persistence needs a target
Persistence is usually treated as a virtue, and often it is. Hard problems take years. But persistence aimed at the wrong thing is a slower way to fail. Someone who will not reconsider a flawed approach can spend years making little progress while believing success is close.
The real question is what you are committed to. A team attached to its solution keeps improving the model’s accuracy. A team attached to the problem asks where the real bottleneck is, and accepts that it may lie outside the model altogether.
Treat each solution as a hypothesis
Science already has a habit for this. You design an experiment that could prove your hypothesis wrong, and you update when the result comes in. The same habit works for building things.
Every product rests on assumptions, for example that the problem is common and that the people who have it would switch to something better. Each assumption can be tested. The time to decide what evidence would change your mind is before you have spent a year on the answer.
Eric Ries’s Lean Startup calls this the build-measure-learn loop. Its most useful lesson is about how to count progress. When you are uncertain, count progress by how much you have learned. A week of watching people struggle with the problem can teach you more than a month spent polishing a prototype. Finding out an assumption is wrong in the first month is far cheaper than finding out in the second year.
Some feedback is slow by nature. Hospitals take a long time to buy anything, and new clinical tools need validation, so a lack of early sales does not prove a healthcare idea wrong. A result is only useful once you know which assumption it actually tests.
Why stopping feels like losing
Changing direction is hard partly because of what you have already spent. In a 1985 experiment, the psychologists Hal Arkes and Catherine Blumer had the Ohio University Theater sell season tickets to its first 60 buyers at either the full $15 or a randomly assigned discount. Over the first five plays, full-price buyers used an average of 4.11 tickets. Discount buyers used about 3.3. The plays were the same. Only the amount already paid was different.
The tendency to keep going because of what you have already invested. Past effort cannot make a failing approach more likely to work. The only question that matters is whether continuing is the best use of the next month.
Past effort is not always wasted. A team may have built tools or relationships that make the next attempt cheaper. Those count, but only for what they make possible from here.
In Quit, Annie Duke argues that the best time to decide when to stop is before you start, while you are calm. Her kill criteria pair a state with a date, such as “if ten clinics have tried the prototype by March and none has changed how they work, the solution changes.” Set early, they can be revisited openly as evidence arrives.
When to keep going
There is no universal deadline. Some ideas need years of validation, and others can be disproved in a single conversation. The skill is knowing which kind you are facing. When results disappoint, it is tempting either to give up at once or to insist that one more attempt will work. Both responses dodge the harder question of what you have learned.
- Nobody can say what evidence would justify continuing.
- There is no concrete reason to expect the next attempt to go differently.
- Continuing has become a way to avoid admitting the first idea was wrong.
In the Thai clinics, changing the solution was how the team stayed loyal to the goal. A good reason to keep going is evidence that the problem matters and that a revised approach has a credible chance of helping. The hours already invested are not, on their own, such a reason.
Further reading
- Beede et al., A human-centered evaluation of a deep learning system deployed in clinics for the detection of diabetic retinopathy (CHI 2020), the Thai clinics study.
- Arkes and Blumer, The psychology of sunk cost (1985), the theater experiment.
- Annie Duke, Quit (2022), on deciding in advance when to walk away.
- Eric Ries, The Lean Startup principles, on the build-measure-learn loop.