How to Evaluate Thesis-Matching Accuracy in an AI Deal Flow Tool
A thesis-matching score is only useful if it's measuring the fund's actual thesis, not a generic proxy for "looks like a good startup." Here's how to test whether a tool is doing the former or the latter.
Why thesis-matching accuracy is hard to evaluate at a glance
A thesis-alignment score looks precise — a number, often to two decimal places — which creates false confidence that it's measuring something specific and rigorous. The uncomfortable question every fund should ask before trusting one: aligned to what, exactly? A score can be internally consistent and still be measuring general company quality (is this a good startup) rather than the fund's specific, stated thesis (is this the kind of company this fund invests in) — and those are different questions with different right answers for the same company.
The backtest every fund should run before trusting a scoring tool
Take ten to fifteen companies from a fund's own history: some that were funded, some that were seriously considered but passed on for reasons unrelated to quality (wrong stage, wrong sector, a portfolio conflict), and some that were passed on for genuine quality reasons. Run all of them through the tool's thesis-matching, blind to which category each one falls into if possible. A tool doing real thesis matching should score the "passed for mismatch, not quality" group meaningfully differently from the funded group — a strong company that simply wasn't the fund's stage or sector should score low on thesis fit even though it might score high on general quality, if the tool has separated the two questions correctly.
What good thesis matching actually requires under the hood
Real thesis-matching needs the fund's criteria specified with real precision — not "early-stage enterprise software" as a category, but the actual stage range, check size, sector definition, and geography a fund invests in, ideally captured with enough structure that a scoring model can check a company against each dimension independently rather than producing one blended number. A tool that only asks a fund for a one-paragraph thesis description and returns a single score is working with too little structured signal to separate "great company, wrong fit" from "mediocre company, right fit" — which is exactly the distinction that matters most in practice.
Watch for scores that only ever go up over time
A subtler failure mode: a scoring tool that's tuned, explicitly or implicitly, on a fund's own portfolio will tend to score companies that resemble the existing portfolio more favorably over time — which feels validating but actually means the tool is reinforcing the fund's existing pattern rather than independently assessing thesis fit. Good thesis-matching should occasionally surface a high-scoring company that looks nothing like anything currently in the portfolio, precisely because it's evaluating stated criteria rather than pattern-matching to past decisions.
What to ask a vendor or provider directly
Whether the thesis criteria are captured as structured, multi-dimensional data or a single free-text description. Whether the scoring model was validated against real historical outcomes, including "good company, wrong fit" cases specifically, not just "funded vs. not funded." And whether the score is static per submission or continues drifting toward whatever the fund has already invested in over time.
Skip the cold outreach. Submit one structured application and get matched to every relevant fund in the PitchProtocol network — pre-screened, pre-researched, and delivered directly to fund partners. Apply to the First 100 Founders Cohort →
Frequently Asked Questions
Should a fund distrust any thesis-matching tool that produces a single overall score?
Not automatically, but a single blended score is less informative than one broken down by the fund's actual criteria — stage fit, sector fit, check-size fit — since a company can be perfectly matched on some dimensions and poorly matched on others.
How often should a fund re-run this kind of backtest?
At minimum whenever the fund's thesis changes meaningfully, and periodically otherwise — thesis-matching that was accurate a year ago can drift as a fund's actual investment pattern evolves.
Is a low thesis-alignment score always a reason to pass?
No — it's a reason to understand why the score is low. A genuinely strong company scoring low on thesis fit for a clear, structural reason (wrong stage, wrong check size) is a different signal than a company scoring low because it's a weak business.
How does PitchProtocol structure its thesis-alignment scoring?
PitchProtocol scores each submission against a fund's specific, structured stage, sector, check-size, and thesis criteria — not a single blended quality score — so a fund can see exactly which dimension is driving alignment or mismatch for a given company.