SkillVaultskills Browse all 500 skills

Product Management · Version 1.0.0 · Reviewed 2026-08-02

Product Experiment Readout Reviewer

Make a product decision about hypothesis and metric alignment and experiment validity review with evidence, explicit trade-offs, and a verification plan.

4 method steps 4 documented failure modes 4 diagnostic checks 7 quality gates

Reviews an experiment readout for hypothesis fidelity, exposure integrity, metric selection, statistical and practical significance, guardrails, segmentation, and decision logic.

₹149 one-time

Get this skill archive

What this skill helps you do

  • Hypothesis and metric alignment
  • Experiment validity review
  • Decision recommendation

How Product Experiment Readout Reviewer works

You provide

Customer evidence, constraints, and the decision at stake

It inspects

Problem versus requested solution in hypothesis and metric alignment

It decides

A experiment validity review decision ranking assumptions by risk

You verify

Success criteria and a reversal condition fixed up front

What it checks first

Product Experiment Readout Reviewer reviews an experiment readout for hypothesis fidelity, exposure integrity, metric selection, statistical and practical significance, guardrails, segmentation, and decision logic. Use it when the work involves Hypothesis and metric alignment, Experiment validity review, Decision recommendation.

  1. Whether the request describes a solution or the underlying problem and its frequency.
  2. The strength of evidence behind each assumption, and which assumption carries the most risk.
  3. Opportunity cost, since a roadmap decision is a decision not to do something else.
  4. Whether success criteria and a review date were defined before commitment.

Failure modes it recognizes

  • Request volume used as a proxy for impact, which favors the loudest segment.
  • A prioritization score presented as objective while its inputs are estimates.
  • An experiment readout interpreted without checking sample ratio or power.
  • Scope committed before the riskiest assumption has been tested.

Answers it will reject

  • Building the requested feature rather than solving the described problem.
  • Presenting a roadmap without the trade-off that made it necessary.
  • Declaring success from a metric that moved for an unrelated reason.

Decision rules it applies

  • Separate problem from proposed solution before evaluating anything.
  • Rank assumptions by risk and test the riskiest before committing scope.
  • State the success criteria and the reversal condition at decision time.

Evidence it asks for

  • Quantify frequency, severity, and affected segment for each problem.
  • Cite the specific evidence behind each assumption and label its strength.
  • Define the leading indicator that will show progress before the lagging metric moves.

The method inside

  1. Separate the customer problem from requested solutions
  2. Inventory assumptions and strength of evidence
  3. Compare options using impact, confidence, risk, effort, and reversibility
  4. Define success, guardrails, and the decision after new evidence

Deliverables

  • Hypothesis and metric alignment evidence map
  • Experiment validity review option and risk analysis
  • Decision recommendation decision memo

Evidence requirements

  • Customer research, usage, support, and commercial evidence
  • Strategy, constraints, dependencies, and opportunity cost
  • Experiment design, roadmap options, or requirements artifact

Quality gates

  • Every material claim traces to supplied evidence or is labeled as a hypothesis.
  • The response follows the declared deliverable contract.
  • No execution, access, measurement, or verification is invented.
  • Secrets and personal data are redacted rather than repeated.
  • The user receives a concrete independent verification step.
  • The relevant failure modes in this domain were considered rather than only the reported symptom.
  • No listed anti-pattern was recommended as a solution.

Example task

Input

Review this onboarding A/B test readout and tell me whether the evidence supports launching the variant.

Expected output

Activation improved 3.1%, but the result is concentrated in returning users and first-week retention declined. The primary metric passed; the guardrail failure means full rollout is not supported...

Boundaries and compatibility

Ideal for

  • Hypothesis and metric alignment: produce a decision or artifact grounded in supplied evidence.
  • Experiment validity review: produce a decision or artifact grounded in supplied evidence.
  • Decision recommendation: produce a decision or artifact grounded in supplied evidence.

Out of scope

  • Using request volume as a substitute for impact
  • Presenting a prioritization score as objective truth

Agent compatibility

  • GitHub Copilot custom agents
  • Claude Agent Skills / SKILL.md
  • Any instruction-following chat model

Tool policy: Advisory by default. No tools are assumed. If the host provides tools, use read-only evidence gathering unless the user explicitly approves a scoped write or execution action.