Three numbers, never collapsed into one
Most review sites publish a single star average, which quietly conflates three unrelated questions. We keep them apart:
| Number | Question it answers | How it is computed |
|---|---|---|
| Community score | What do people think of it? | Weighted Bayesian mean of published reviews |
| Confidence | How much evidence is behind that? | Kish effective sample size of the weights |
| Review quality | Are the reviews themselves any good? | Share of reviews that bridge across viewpoints |
A project can have a high score and low confidence — that combination is common and it is exactly what a single enthusiastic review looks like. Showing only the score would hide it.
The community score
The score is not the mean of the ratings. It is a weighted Bayesian mean:
score = (C × m + Σ wᵢ·rᵢ) / (C + Σ wᵢ)
where rᵢ is each rating, wᵢ is that reviewer's weight, m is the ecosystem average and C = 4 is how much evidence the prior is worth. The practical effect: a project with one five-star review does not display 5.0, because one review is not worth more than the prior. It takes sustained agreement to move a score to an extreme, which is precisely what makes an extreme meaningful when you see one.
What makes one review count more than another
Every input below is a capped multiplier on a single review, never a gate. Weights are bounded to the range 0.2–2.5, so no reviewer, however established, can outvote a genuine consensus.
- Account age. The hardest signal to fake at scale — you can buy followers today, you cannot buy a two-year posting history.
- Reach, log-scaled. A 500,000-follower account is worth roughly twice a 40-follower one, not ten thousand times.
- Follower-to-following ratio. An account following thousands while followed by dozens bought its numbers. This is penalised hard because it is cheap to detect and expensive to avoid.
- Posting history. Followers with no posts is a shell account.
- Track record. Reviewers whose past reviews bridged gain weight. Reviewers whose reviews were consistently found unhelpful lose it. This is the part that is earned — and the part that can be burned.
- On-chain experience depth, where the project has a program to check against. See below.
Bridging: why a unanimous review can still fail
Helpfulness votes are fitted with a rank-one matrix factorisation, the same shape of model X uses for Community Notes:
ŷ(reviewer, review) = μ + i_reviewer + i_review + f_reviewer × f_review
The f term is the whole trick. It gives the model a way to explain agreement that comes from shared viewpoint. Two people who always vote together have their agreement absorbed into f × f — which leaves the review's own intercept i_review capturing only the agreement that crosses the divide.
So a clique upvoting its own reviews raises f, not i. In testing, a review found 100% helpful by a single faction scores roughly half the intercept of one found 100% helpful by both — identical vote ratios, very different standing.
Status keys off μ + i_review rather than the bare intercept. The intercept's scale drifts with how helpful a rater population happens to be overall — the same genuine bridge measured 0.40 against one set of raters and 0.26 against another, purely because the baseline differed. Adding μ back yields a figure that means the same thing in every dataset: the share of raters who would find this review useful once identity and viewpoint are stripped out.
| Status | Meaning | Requires |
|---|---|---|
| NEEDS MORE RATINGS | Not yet judged | Fewer than 5 independent raters |
| COMMUNITY VALIDATED | Bridged across viewpoints | Useful to ≥ 75% viewpoint-adjusted, raters from ≥ 2 sides |
| CONTESTED | Useful to one side only | Enough raters, but below the bar or all from one side |
| NOT FOUND HELPFUL | Broadly rejected | Useful to ≤ 25% viewpoint-adjusted |
Note the second condition on validation: raters must come from at least 2 sides of the polarity axis. A review can be rated helpful unanimously and still fail it, if everyone who rated it sits on the same side. Unanimity within a faction is not consensus, and this is the rule that says so.
A brand-new review is never treated as truth. It stays at NEEDS MORE RATINGS until 5 independent people have rated it, which is a healthier default than publishing a verdict on the strength of one opinion.
Confidence
Confidence reports the effective sample size, (Σw)² / Σw², rather than a raw count. The distinction matters: five reviews where one reviewer carries almost all the weight is worth about one review, and this measure says so where a count would not.
- LOW — under 3 effective reviews. The score could move a long way with one more.
- MEDIUM — 3 to 8.
- HIGH — 8 or more.
Ranking uses the Wilson lower bound of the score, not the score itself, so a project cannot climb the leaderboard on a thin sample.
On-chain proof, and its limits
Where a project has a deployed program, a reviewer can link an X1 wallet by signing a message. Signing authorises no transaction, moves no funds and costs no gas. We then check whether that wallet's history shows contact with the program.
Three things this deliberately does not do:
- It is never required, and absence never costs anything. Most people never connect a wallet, and someone burned by a bad project least of all. Down-weighting unverified reviews would systematically silence exactly the people most worth hearing.
- It is not applicable to most projects. Explorers, screeners, wikis and community groups have nothing to verify against. Those projects show no verification state at all, because "cannot be checked" and "failed a check" are different claims.
- It proves contact, not endorsement. For a scam, contact means victim — which makes a verified negative review the strongest signal on the site, not the weakest.
And because anyone can transact with their own program, mere existence of a transaction proves little. What is expensive to fake is the shape of a history, so the measure is continuous: how many interactions, spread over how long, beginning how far before the review was written. A wallet with one transaction ten minutes before posting scores near zero. Forty interactions across three months scores near one.
Coordinated activity
Three detectors run over every published review. None of them reject anything automatically — they raise the item in the moderation queue with the reason attached, because a flag a human cannot interrogate is a flag they learn to ignore.
- Bursts. 3 or more similar ratings for one project inside 48 hours.
- Young-account clusters. Multiple reviewers with accounts under 90 days old converging on the same project.
- Rings. Accounts whose review sets overlap by 80% or more, in the same direction, across at least 3 projects.
Separately, every review is visible immediately but does not affect the score for 24 hours. A coordinated push is therefore visible to moderators before it is visible in the number.
Check the arithmetic yourself
Every input to every score — weights, their components, bridging intercepts, effective sample sizes — is served as JSON at /api/reviews/scores. If you think a score is wrong, that endpoint contains everything needed to demonstrate it.
Frequently Asked Questions
Is this just a copy of Community Notes?
It borrows the core idea — a matrix factorisation whose polarity term absorbs same-side agreement so that the intercept measures cross-side agreement — and applies it to review helpfulness rather than notes on posts. The reviewer weighting, the Bayesian score, the confidence measure and the on-chain layer are ours.
Why not put the whole thing on-chain?
Because the blockchain is good at proving that someone interacted with a program and bad at determining whether their opinion of it is useful. We use the chain for what it can actually prove — experience — and a social model for what it cannot.
Can I see the code?
The parameters on this page are read live from the running scorer, and every computed input is published at /api/reviews/scores. If a number here disagrees with what the site shows, the site is wrong and we want to hear about it.