CAT PAW

All workflows

Marketing

Run a Marketing Experiment Retrospective

A defensible readout of what an experiment did and did not show, plus one documented decision for the next campaign.

The process

Work it in this order.

  1. Recover the original hypothesis, audience, variants, primary metric, guardrail metrics, runtime, and decision rule.
  2. Validate instrumentation, assignment, exclusions, sample contamination, and data completeness before comparing outcomes.
  3. Report results for the preselected primary metric and guardrails, including raw denominators and uncertainty where available.
  4. Check whether operational issues, seasonality, channel mix, or unequal exposure offer plausible alternative explanations.
  5. Review only preplanned segments as confirmatory; label any post-hoc segment findings as exploratory.
  6. Decide to adopt, reject, rerun, or redesign the variant based on the evidence and practical impact, not just directional movement.
  7. Archive the setup, results, caveats, decision, owner, and follow-up experiment in an experiment log.

Bring this

Inputs the cat will ask for.

  • Experiment plan
  • Variant and exposure data
  • Outcome and guardrail metrics
  • Known campaign incidents or changes

Router + one playbook

Route first. Choose one.

Cat Paw loads one playbook per task rather than merging several. Start with this router; the remaining paths are workflow-specific candidates it can inspect.

  • Router: product-pack
  • skills/product/pm-data-analytics/ab-test-analysis/SKILL.md
  • skills/product/pm-execution/retro/SKILL.md

Guardrails

The boundaries travel with the prompt.

  • Do not change the primary metric after seeing results without labeling the analysis exploratory.
  • Do not claim causation when assignment or instrumentation was compromised.
  • Protect user privacy and use aggregate reporting where appropriate.
  • Report negative and inconclusive results rather than cherry-picking segments.

Done means

A checkable finish.

The retrospective records validated data, primary and guardrail results, caveats, a justified decision, and a named follow-up action in the experiment log.

Sources

Where this process came from.

  1. support.google.com/analytics/answer/13468470?hl=en
  2. mailchimp.com/help/about-ab-tests/

Sources establish the workflow pattern. Reddit and community posts, where present, are treated as practitioner anecdotes rather than universal facts.

AI acts. You decide.

A human brake for AI.

Latch shows every command, click, and file change before it happens — so you can say yes, no, or not yet.

Plow Latch · macOS Cat Paw Latch · Windows + Linux · build from source
Cat Paw Latch

Behind the number

Two harnesses. One line.

Same number, same $100 credit. Pick the harness that answers when you text.

Hermes

Nous Research's terminal-native agent. It runs on your machine, keeps memory between sessions, and opens one playbook per task.

Text Hermes
OpenClaw

The open-source assistant that runs on your own computer and answers in the channels you already use. One playbook per task.

Text OpenClaw

One line for both harnesses +1 628 246 3032

Opens Messages with that harness already chosen.