SkillVaultskills Browse all 500 skills

Research · Version 1.0.0 · Reviewed 2026-08-02

Survey Question Bias Reviewer

Produce defensible evidence for question wording audit and scale and option review with evidence, explicit trade-offs, and a verification plan.

4 method steps 4 documented failure modes 4 diagnostic checks 7 quality gates

Reviews survey questions for leading language, double-barreled constructs, assumed knowledge, unbalanced scales, ambiguous recall periods, and analysis mismatch.

₹149 one-time

Get this skill archive

What this skill helps you do

  • Question wording audit
  • Scale and option review
  • Construct-to-analysis alignment

How Survey Question Bias Reviewer works

You provide

Sources, method notes, and the question being answered

It inspects

Evidence strength and negative cases for question wording audit

It decides

A scale and option review synthesis with limitations named

You verify

Findings reproducible from the cited evidence alone

What it checks first

Survey Question Bias Reviewer reviews survey questions for leading language, double-barreled constructs, assumed knowledge, unbalanced scales, ambiguous recall periods, and analysis mismatch. Use it when the work involves Question wording audit, Scale and option review, Construct-to-analysis alignment.

  1. Sample size, selection method, and who was excluded, since these bound every conclusion.
  2. Whether a finding is supported by observed behavior or by what participants said they would do.
  3. Disconfirming cases, which are the fastest test of whether a theme is real.
  4. Whether the question the research answers is the question the decision actually needs.

Failure modes it recognizes

  • Themes derived from the loudest participants rather than from the sampled distribution.
  • Leading questions producing agreement that disappears under neutral phrasing.
  • Survivorship bias from studying only current customers or successful accounts.
  • Coding drift where the same excerpt would be categorized differently later in the process.

Answers it will reject

  • Reporting percentages from a small qualitative sample, which implies precision that does not exist.
  • Treating a synthesized theme as a finding without the excerpts that support it.
  • Stopping at the first pattern that confirms the pre-existing hypothesis.

Decision rules it applies

  • State the sample, the method, and the limitation alongside every finding.
  • Actively seek disconfirming evidence before accepting a theme.
  • Separate what was observed, what was reported, and what was inferred.

Evidence it asks for

  • Attach verbatim excerpts to each theme so a reader can audit the interpretation.
  • Check coding consistency by re-coding a subset and comparing.
  • Report counts with denominators rather than percentages alone.

The method inside

  1. Define the research question and unit of analysis
  2. Create a transparent coding or extraction framework
  3. Preserve source traceability and negative evidence
  4. Separate findings, interpretation, limitations, and applicability

Deliverables

  • Question wording audit evidence table
  • Scale and option review findings with negative cases
  • Construct-to-analysis alignment limitations and next-research plan

Evidence requirements

  • Source documents, transcripts, data, and research question
  • Sampling method, population, and collection context
  • Known limitations, contradictory cases, and analysis criteria

Quality gates

  • Every material claim traces to supplied evidence or is labeled as a hypothesis.
  • The response follows the declared deliverable contract.
  • No execution, access, measurement, or verification is invented.
  • Secrets and personal data are redacted rather than repeated.
  • The user receives a concrete independent verification step.
  • The relevant failure modes in this domain were considered rather than only the reported symptom.
  • No listed anti-pattern was recommended as a solution.

Example task

Input

Review this customer survey before launch and identify questions that will produce biased or uninterpretable data.

Expected output

Question 4 combines satisfaction and likelihood to renew; Question 7 presumes the feature was used. The revised instrument separates constructs and adds a non-user path...

Boundaries and compatibility

Ideal for

  • Question wording audit: produce a decision or artifact grounded in supplied evidence.
  • Scale and option review: produce a decision or artifact grounded in supplied evidence.
  • Construct-to-analysis alignment: produce a decision or artifact grounded in supplied evidence.

Out of scope

  • Fabricating sources, participants, or findings
  • Claiming representativeness without a sampling basis

Agent compatibility

  • GitHub Copilot custom agents
  • Claude Agent Skills / SKILL.md
  • Any instruction-following chat model

Tool policy: Advisory by default. No tools are assumed. If the host provides tools, use read-only evidence gathering unless the user explicitly approves a scoped write or execution action.