How RankHigh measures AI recommendations

The primary construct is expected top-five recommendation prominence across four equally weighted provider families. The tested surfaces are official, web-enabled AI APIs configured for a defined UK / en-GB edition.

API results are API results. RankHigh does not describe them as consumer-app results. Exact providers, models, API surfaces and web/search settings are frozen and identified for each published edition.

Payment never affects benchmark rank. Subscriptions, sponsorship, nominations, claims, referrals and community activity also cannot alter eligibility, execution, evidence or publication.

Products are assessed against frozen category rules using official evidence. An embedded suite feature must be independently usable, distinctly documented and materially perform the benchmarked function. Nominations do not guarantee inclusion or improve review priority.

AI models, search sources and product information can change over time. Dated, versioned editions document what was observed; original snapshots remain immutable and corrections create linked revisions.

• Objective product quality or an objective ‘best product’.
• Customer satisfaction, sales, market share or product reviews.
• Website traffic, popularity or RankHigh community activity.
• Payments, subscriptions, advertising, sponsorship or customer status.

All four provider families are averaged equally. A provider with more valid observations cannot acquire extra weight merely by producing more responses.

• 12 unbranded prompts across six buyer-intent strata, with two independently drafted prompts per stratum.
• Prompt wording, ordering, control instruction, hashes and version freeze before collection output is viewed.
• No eligible product list is supplied and vendors cannot choose, veto or rewrite public scoring prompts.
• Paid or private questions are technically ineligible for the public ranking path.
• Exact prompt wording is disclosed with the edition after publication.

Current edition availability: the 12 October 2026 AI Meeting Assistants edition currently provides aggregate scores and edition context. Exact prompts and answer-level excerpts are not attached publicly. This availability note does not change the disclosure standard above.

• Current edition instrument (public-benchmark-v1.1.0): four clean, stateless repetitions per prompt and provider family — 12 prompts × 4 providers × 4 repetitions = 192 planned first attempts, 48 per provider family.
• The only change from the original v1.0.0 design (eight repetitions, 384 planned executions) is the repetition count; prompts, provider weights, scoring, aggregation and eligibility are unchanged. The v1.0.0 description remains part of the historical record.
• Web/search is enabled using the edition’s frozen provider configuration.
• A failed execution may receive at most two neutral retries; retries are not new benchmark opportunities.
• An execution is never retried because a preferred product is absent.
• Truncated answers are terminal-invalid: evidence is retained, they are excluded from scoring, and there is no automatic retry.
• Collection targets 72 hours and may never silently exceed the hard 168-hour maximum.
• Future editions must record a timestamped UTC eligibility cutoff before collection starts; a cutoff is never created after the fact.

RankHigh also has a browsable tool directory. Its sort options — such as newest, most clicked or clearly labelled sponsored placements — only arrange directory listings. Clicks, profile completeness, owner claims, listing recency, payments and community activity never feed benchmark eligibility, scores, ranks, uncertainty or publication.

Before any edition is published, every stored answer is re-hashed against its saved fingerprint, all scores are recomputed by a second, separate calculation, the uncertainty analysis is replayed with the same seed, and every identity decision must be recorded against its source answer. Where RankHigh uses this path it is described as automated validation. It is not independent human review or certification, and publication still requires explicit founder authorization.

An affirmative recommendation at position k receives reciprocal utility U(k) = 1/k for positions one to five. A product below fifth place or absent receives zero for that execution.

Utilities are averaged hierarchically across valid repetitions, the two prompts within each intent stratum, the six strata, and finally the four equally weighted provider families. The result is displayed on a 0–100 scale. Full internal precision determines order; rounded display values do not. There is no freshness component in the score.

RankHigh may show recommendation rate, first-choice rate, mean recommendation position, per-provider score, per-intent score, citation/domain rate and surface coverage. These explain the observations but never re-enter the primary score.

• 10,000 stratified paired bootstrap replicates by default.
• The four provider families stay fixed; prompts are resampled within each intent stratum and valid repetitions within selected prompts.
• Published outputs may include 95% score intervals, 95% rank intervals, paired score-difference intervals and leave-one-provider-family-out diagnostics.
• Exact full-precision equality shares rank. Otherwise, point score determines order.
• ‘Statistically close—not clearly separated’ appears only when the absolute point-score gap is below 2.0 and the paired 95% difference interval contains zero.

Public evidence is a deliberately small allow-list from an immutable, published, non-test snapshot. Raw responses, private storage locations, unpublished prompts, reviewer identities and internal notes remain private.

Corrections address identity, classification, position, eligibility, metadata, evidence linkage or faithful implementation. They cannot negotiate a score or purchase a rerun. A confirmed correction creates linked history; it never silently rewrites an edition.

• Results apply to the named API models, tools, geography, language, prompts and collection window—not every consumer experience.
• AI outputs are stochastic; repeated sampling and intervals describe, but cannot eliminate, that variation.
• Top-five prominence is a focused recommendation measure and not a complete assessment of product quality.
• A snapshot can become historical, corrected or superseded; the original remains readable.