Reviews
Part of How to review landing page software with the unknowns kept in plain sight
Why one buyer weight can flip a landing page software review
Mandatory gates, buyer weights and an evidence threshold decide a landing page software review, and one worked case shows a single weight flipping the result.
A weighted score looks objective, yet one weight can decide the outcome. A review method for an England buyer should therefore fix its gates, weights and thresholds before any supplier page is opened, and publish the raw results beside the total. That lets a second reviewer see how the verdict was reached.
What to take away
- Mandatory gates are pass or fail. A higher feature score cannot rescue a failed gate.
- Buyer weights, the evidence threshold and the overall threshold stay blank until the buyer authorises them.
- Score only evidence that meets the declared threshold.
- Publish raw results beside the total, so a reader can see what one weight change does.
- The permitted conclusions are narrow, and none of them names a universal winner.
Gates come first, scores second
Set the mandatory gates before opening supplier pages. Typical gates cover domain control, accessible completion, data roles and terms, security configuration, claim approval, export and recovery. A failed mandatory gate ends that candidate's run, whatever its feature score.
Case setup to mandatory gates
- Define page purpose and eligible visitors
- List data fields, domain, operators, hand-off
- Record current CMS or manual-form baseline
- Set mandatory gates before supplier pages
- Keep weights and thresholds blank
Each gate asks for its own evidence. A candidate that cannot give the buyer control of the page's domain ends its run, however well it scores on the form task.
| Gate | Evidence the review asks for |
|---|---|
| Domain control | The domain the landing page is served from, and who controls it |
| Accessible completion | Keyboard, zoom, screen-reader and error-recovery results for the configured form |
| Data roles and terms | The proposed order form, service description and data-processing terms |
| Security configuration | Account lifecycle, privilege, monitoring and recovery evidence |
| Claim approval | Documentary support for each measurable claim on the page |
| Export and recovery | Proof that page content and form data export, and that a known version restores |
For the data gate, obtain the proposed order form, service description and data-processing terms, then compare them with what the ICO says a controller-processor contract must include. For security, the NCSC explains how to carry out a rapid but reliable assessment of cloud services, and security configuration risks show what a failed gate can look like.
Who sets the weights, and when
The buyer sets the weights, not the reviewer. Buyer weights, the evidence threshold and the overall threshold stay blank until the buyer authorises them. They should be written down before the first supplier page opens, so they cannot drift toward a favourite product.
Test each weight against the case you wrote first: the page purpose, eligible visitors, data fields, domain and operators, with the current process as the no-purchase baseline. A weight with no link to that case is a preference, and the record should say so.
What counts toward the score
Grade each row with one of four labels: verified for a stated scope, supplier-stated, not tested or unknown. A current product page can support an attributed capability. It cannot prove performance or suitability.
Score only evidence that meets the declared threshold. Rows that fall short are shown but not added. The method for grading evidence explains how to keep the unknowns visible rather than scoring around them.
The arithmetic, and how a weight flips it
The weighted total is the sum of each checked row's score multiplied by its buyer weight, plus other verified rows. A minimum of 70 out of 100 is used here as an illustrative overall threshold. The numbers are illustrative only.
Form task score
- Weight A
- 5 of 10
- Weight B
- 5 of 10
Form weight
- Weight A
- 0.20
- Weight B
- 0.50
Weighted points
- Weight A
- 1.0
- Weight B
- 2.5
Other verified rows
- Weight A
- 68
- Weight B
- 68
Total
- Weight A
- 69.0
- Weight B
- 70.5
Against 70/100
- Weight A
- Fail
- Weight B
- Pass
Nothing about the product changed between the rows. The form task scored 5 of 10 in both, and only the buyer's weight moved. That is why the weight must be set first and shown beside the raw result.
Stress-test the result
Run this short sequence on every review before publishing a total:
- Fix the gates, weights and thresholds before opening any supplier page.
- Score only the rows whose evidence meets the threshold.
- Compute the total, then change one weight and compute it again.
- If the pass or fail result flips, say so beside the total.
- Publish the raw scores next to the weighted figure.
A total that survives each change is more reliable than one that does not. A total that flips is still publishable, because the flip tells the buyer which weight deserves the next conversation.
Permitted conclusions
The conclusions are narrow: proceed to a controlled test, request evidence, retain the baseline or stop. The reviewer must not award a universal winner. A weighted total is also not a financial result, and money questions belong in the return on investment framework.
For identical tasks across candidates, a product comparison trial shows how to run them. Record the product, plan, documentation URL, test date and operator role beside every result.
Keep the rows current
Set change triggers as well as calendar expiry. A new plan name, revised terms, an altered sub-processor list or a changed critical feature should return the affected row to unknown until its owner checks the new record. This stops an old screenshot silently governing a later purchase.
The planning template's change record logs each trigger. Before publication, name the procurement, commercial, privacy, PECR, security, accessibility and measurement reviewers, then repeat any expired test. The selection checklist covers a manual baseline in more depth.
Before you act
- Fix the mandatory gates before opening supplier pages.
- Get the buyer's weights and thresholds in writing.
- Score only evidence that meets the declared threshold.
- Change one weight and recompute before publishing.
- Publish raw scores beside the total.
- Return affected rows to unknown when a trigger fires.
Common questions
Can a high feature score offset a failed mandatory gate?
No. A failed mandatory gate cannot be rescued by a higher feature score, so that candidate does not proceed on points.
Why publish raw scores beside the total?
It shows how one weight change alters the outcome. In the worked case the form task scored 5 of 10 throughout, yet the total moved from a fail to a pass.
What are the permitted conclusions?
Proceed to a controlled test, request evidence, retain the baseline or stop. The reviewer must not name a universal winner.
