Rejection is data

One rejection is a noisy observation. Read many of them together, and trust what people do over what they say.

In 2021, the organizers of NeurIPS, one of the largest machine-learning conferences, ran an experiment. They picked 882 submissions at random and had each one reviewed by two independent committees. Of the papers one committee accepted, the other rejected 50.6%. A similar experiment in 2014 had found 49.5%.

The experiment does not show that peer review is meaningless. Judging the quality of research is hard. It does show how much a single decision can depend on which reviewers a paper happens to draw.

The same holds outside publishing. A declined application or a polite no from a collaborator can feel like a verdict on you. Usually it is a narrower observation. One person or institution said no to one request at one moment, which does not tell you whether the idea was any good.

The same no, different causes

Suppose a potential customer turns down your product. The problem might not matter to them. Your solution might be weak. Or you might be talking to someone with no authority to buy anything.

Each cause points somewhere else. If the problem is unimportant, the whole idea needs rethinking. If the product is weak, improve it. If you reached the wrong person, the product may be fine and your approach to customers is what needs work.

Treating every rejection as the same verdict throws this information away.

Ask how surprising the rejection is

Bayesian reasoning gives you a way to weigh it. How much a piece of evidence should change your mind depends on how much more likely it is under one explanation than under another.

How much a rejection tells you

Ask how often you would see this rejection if your idea were good, and how often if it were bad. When good ideas get rejected nearly as often as bad ones, a rejection tells you very little. At NeurIPS, an accepted paper had roughly even odds of being rejected on a second review.

One no is weak evidence. Many independent responses from the right people can be strong evidence, especially when you read the reasons behind them.

Imagine showing a healthcare product to 20 hospitals.

If most say the problem is already handled, the opportunity is weaker than you thought.
If most agree the problem is real but cannot justify the cost, you have a pricing or value problem.
If most are interested but stuck in a long approval process, the obstacle is how hospitals buy.

The count of rejections is the same in all three cases. The pattern of reasons is what tells you what to do next.

Whose rejection counts

The sample matters as much as the count. In a 2024 study in Management Science, Ruiqing Cao and colleagues looked at startups launching on a website where about nine in ten users were men. Products aimed at women grew 45% less in the year after launch than products aimed at men. On days when unusually many women happened to be testing new products, the gap shrank toward zero.

The gap closed whenever the right people were testing. In research we would call this a sampling problem. In product development it is easy to mistake for a market verdict. A pattern of rejections only counts if it comes from the people you are actually trying to serve.

Compliments are not evidence

Resilience matters, because one setback should not end a worthwhile effort. But resilience can turn into a search for people who agree with you. With a large enough network, almost any idea will find someone who likes it.

Someone who enjoys a conversation and praises your product may have no intention of using it. If ten people politely call a product interesting and none can point to a recent time they needed it, that tells you something too.

The past is better evidence

In Y Combinator’s How to Talk to Users, Eric Migicovsky’s questions are about real experience, such as the last time someone ran into the problem and what, if anything, they did about it. Someone who has spent time or money working around a problem is telling you more than someone who says it sounds interesting.

Encouragement still matters. Mentors and friends make hard projects possible. Emotional support and evidence that the idea works belong in separate columns.

When rejection is a signal

Spending days replaying a rejection rarely improves the next attempt. But none of this means you should explain every rejection away.

When to listen

When people with no connection to each other raise the same objection, it is a finding. The useful response is an experiment that could show whether they are right.

Sometimes the right response is to improve the approach or find a different audience. Sometimes it is to accept that the idea does not work. Changing direction after real feedback shows you took it seriously.

Read together, rejections tell you far more than any single verdict. A rejection need not define you. Sometimes it should change your next experiment.