What this guide covers
What you’ll be able to do
- Write a testable marketing question
- Choose evidence matched to the hypothesis
- Preserve the result and next decision
Optimize, Scale, Repeat means using evidence to choose the next improvement. The R.O.I. framework helps identify what is actually being tested.
Skip to a section
Chapter 01
Choose the decision you need to make
“Improve marketing” is too large for one test. “Will a clearer first-visit explanation help people understand the invitation before they inquire?” points to a specific uncertainty and a practical change. A useful experiment starts with that question.
- 01
Describe an observation
Use a repeated question, a stage gap or a confusing route you can inspect. “The campaign feels tired” is less useful than a specific observed problem.
- 02
Choose one decision
Decide what you will do differently if the evidence supports the idea. A test without a decision can become an endless exercise in collecting numbers.
- 03
Limit the change
Select a change small enough to understand and deliver. If several changes are necessary, record them honestly rather than claiming the test isolated one factor.
Use Relevancy, Omnipresence and Intimacy to locate the issue. Is the promise meaningful? Are repeated encounters adding understanding? Does someone answer the question when the person responds? Testing a new colour cannot resolve every one of those problems.
See it in action
Write the question before calling a winner.
Decide what the test needs to teach you.
Record what changes and what stays comparable.
Record what you observed, including an inconclusive outcome.
Sample creative. Fictional practices; no client results implied.
Examples & steps
Choose a question worth changing your mind about
An experiment should help resolve a real choice. Before approving a new version, finish this sentence: “If we learn this, we will change this decision.” If the team will keep the same approach whatever happens, it may be producing content or improving a known defect rather than running an experiment. Name that work accurately.
| Starting thought | Useful question | Evidence to inspect |
|---|---|---|
| We need a new headline. | Do people understand who this appointment is for? | Their explanation after seeing the page, plus relevant inquiry conversations. |
| This creative looks stronger. | Does its message attract inquiries the practice can appropriately receive? | The inquiry mix and later journey, alongside attention measures. |
| The follow-up is too slow. | Where does the handoff wait, and would clear ownership remove that wait? | Timestamped receiving records and verified coverage. |
Put proposed tests in a short queue. For each, record the observed problem, the decision at stake, the effort required and the dependency that could block interpretation. A request-form experiment cannot teach much if the form is broken. Fix the defect and verify the repair first.
Choose work close to the current constraint. If the practice has enough appropriate inquiries but cannot offer suitable times, another attention experiment is unlikely to resolve the immediate problem. The experiment might instead concern an operational change, with the practice owning its delivery and the marketing team preserving the surrounding campaign context.
Leave room for a useful “no.” A test that prevents the team from scaling an attractive but unsuitable message has done valuable work. The log should reward clearer judgment, not a collection of winning labels.
Chapter 02
Explain why the change might matter
Write the link between the change and the intended effect. In a fictional example, Maya likes a life-led creative but cannot tell what the appointment includes. The team hypothesises that a clearer first-visit page will reduce that uncertainty and support a more informed inquiry.
- 01
State the proposed mechanism
Explain how the change should help the person think, understand or act. This connects the idea to the patient journey rather than a vague hope for better conversion.
- 02
Name the supporting signal
Choose evidence close to that mechanism, such as clearer understanding or fewer repeated practical questions. Keep downstream appointment progress in view without treating it as immediate proof.
- 03
Name competing explanations
Record other factors that could influence the result, including availability, response coverage, offer changes and timing. This makes the later interpretation more honest.
That is more useful than “the new page will convert better.” It tells the team what to inspect: whether readers understand the visit, which questions remain and how inquiries progress. The outcome might be clearer conversations even before a meaningful difference in bookings can be established.
Examples & steps
Make the expected chain of events inspectable
A mechanism connects the edit to an observable response. “More professional copy will convert better” skips too many steps. “Explaining that the form requests a callback will reduce confusion about whether an appointment is confirmed” names a specific misunderstanding the team can check.
Illustrative mechanism for a request form
Observation: some people treat the request confirmation as an appointment confirmation.
Change: state what happens after submitting and what confirms an actual appointment.
Expected first effect: people can explain that they have requested contact and still need an appointment confirmation.
Possible later effect: fewer conversations begin with an incorrect assumption about a booked time. Whether attendance changes is a separate question requiring later evidence.
Check the explanation against the real workflow before testing it. A clearer description of an imaginary process is still misleading. Confirm who contacts the person, whether times are selected or requested, and which message establishes a booking.
Illustrative copy, for a practice that actually uses this process
“Tell us a convenient time to reach you. Our team will contact you to arrange an available appointment. Submitting this form does not confirm an appointment; you will receive a separate confirmation once a time is agreed.”
Write down what would weaken the explanation. If people understand the distinction but still cannot find a suitable time, the remaining problem is availability. If they never see the confirmation message, investigate the delivery or interface. If the staff use contradictory language, align the handoff before judging the page alone.
Keep the mechanism narrow enough to learn from. One copy change should not be credited with every movement from attention to ongoing care. Follow the earliest meaningful effect first, then inspect later stages as the records mature.
Chapter 03
Record six things before the change
A simple log is enough if the entries are specific. Write it before publishing so a surprising result does not change your memory of what you intended to learn.
- 01
Record the starting version
Save the relevant copy, creative and route with the date. The team needs to know what was actually changed rather than rely on memory.
- 02
Record evidence and context
Write the question, hypothesis, variation, review evidence and material operating changes. Keep private contact information out of a general experiment log.
- 03
Record the decision rule
Name who reviews the work and when. Explain what would justify keeping, revising or stopping without pretending a small sample can settle every causal question.
Keep six fields together: the question, the hypothesis, the exact variation, the evidence to review, the operating context and the eventual decision. Add a named reviewer and a date. A brief record written before launch helps stop the team from choosing a different definition of success after seeing the result.
Write the next experiment
Use aggregate counts or fictional planning inputs only. Calculations happen on this page; nothing is sent to a server.
See it in action
Keep the versions and the context.
Decide whether the page change belongs in this comparison.
Keep the old and new work identifiable.
Examples & steps
Keep the version and the surrounding changes together
A useful log lets another person reconstruct the test without relying on anyone's memory. Save the previous asset, the new asset, the start and end dates, the audience or page context, and the exact point where the experience changed. A filename such as “final-new-2” does not explain what was live.
Use a stable experiment ID and readable versions: EXP-014, request-confirmation A, request-confirmation B. Store a screenshot and the exact copy beside the record. Include the actual route or asset reference, while keeping personal information out of the shared experiment file.
| Change record | Example entry | Why it matters |
|---|---|---|
| Intended difference | Added two sentences explaining request versus confirmed appointment. | Defines the idea being inspected. |
| Release check | Verified the live message after a permitted test submission. | Confirms the intended version was actually shown. |
| Concurrent change | Four Friday appointment times became available during the second period. | Could affect bookings independently of the copy. |
| Data correction | Removed two duplicate inquiries after reconciliation; retained the original total. | Explains why the reported denominator changed. |
Record a concurrent change when it happens. Trying to reconstruct a busy fortnight after seeing the result encourages selective memory. Relevant changes include appointment availability, response coverage, offer terms, page faults, spend and other creative. Do not create a sprawling diary of unrelated office events.
Give every open test a review trigger. This can be a planned date plus a maturity requirement, such as waiting until the included appointments have occurred. The date creates accountability; the maturity requirement prevents a premature verdict. If evidence is delayed, record the reason and a new review point instead of leaving the test permanently “running.”
Chapter 04
Judge the idea at the right point in the journey
The current Meta Playbook separates attention, inquiries and appointments. An attention creative should earn a relevant audience; the offer and destination should make the next step clear; response and availability must carry the person toward a visit.
- 01
Review the chosen job first
Judge attention, understanding, inquiries and appointment progress with the measures that fit each. An increase in clicks alone does not settle a question about suitable visits.
- 02
Check the group and maturity
Compare consistent definitions and allow the observed group time to progress. Very recent inquiries may not yet provide a useful attendance comparison.
- 03
State the confidence honestly
A small or uncontrolled before-and-after can suggest a next move without proving causation. Write what the evidence supports and which uncertainty remains.
If a variation attracts more clicks but fewer suitable inquiries, the larger click count does not settle the decision. If bookings rise while the practice has changed its opening hours, record that context. Small or uneven groups may require more observation; a simple before-and-after is useful for learning but cannot isolate every cause.
Examples & steps
Read a result that points in two directions
Consider this invented comparison of two campaign periods. Both used $1,200 in media, and both cohorts have had the same time to reach an appointment. Version B included a clearer next-step explanation. The records were reconciled to distinct new people.
| Measure | Period A | Period B |
|---|---|---|
| Distinct inquiries | 80 | 72 |
| Booked | 32 | 36 |
| Attended | 24 | 27 |
| Inquiry-to-booking rate | 40% | 50% |
| Booked-to-attended rate | 75% | 75% |
| Media cost per inquiry | $15.00 | $16.67 |
| Media cost per attended person | $50.00 | $44.44 |
Looking only at inquiry cost would favor A. Looking at attended people per media dollar would favor B. The booking rate increased by ten percentage points, from 40% to 50%; that is a 25% relative increase in that rate. The attendance rate among booked people did not change. Keep those statements distinct.
Now open the change record. During period B, four additional Friday appointment times became available. That change could contribute to bookings. The periods were also sequential, so other differences may matter. This comparison describes what happened; it does not isolate the copy's causal effect.
A useful decision is still possible. Keep the clearer explanation if it accurately describes the process and the team has verified that it resolves confusion. Record the booking improvement as an observation with competing explanations. Do not publish the ten-point difference as a proven lift from the copy.
The next question might concern appointment fit, because the added availability could explain more than the wording. Preserve the stronger operating arrangement, then design the next comparison around the remaining uncertainty. Learning does not require undoing a useful improvement merely to manufacture a neat story.
Chapter 05
Turn the finding into the next piece of work
End with what you learned and what happens next. A test can reveal that the suspected problem was wrong. That is useful if it stops the team repeatedly redesigning a page when the real issue is that no suitable appointments are available.
- 01
Choose keep, revise or stop
Write the action that follows from the evidence. A test has not finished when the chart is exported; it finishes when a responsible person makes the next decision.
- 02
Preserve the reason
Keep the finding, limits and versions together. Future work should inherit the learning instead of repeatedly testing the same unsupported assumption.
- 03
Find the next uncertainty
Use what remains unclear to choose the next focused test. Do not expand the work merely because the previous variation produced a flattering number.
Preserve the actual versions and relevant dates so the next person understands the comparison. Over time, the log should reveal what your community responds to, which explanations reduce confusion and which operating changes help people follow through. That is a practical knowledge base, built from the work.
See it in action
Make the next test smarter.
Leave a record the next person can understand.
Examples & steps
Close the test with a decision and a reusable finding
“Winner” and “loser” are often too crude. Close the record with the decision the evidence supports and the question it leaves open. Keep these three things separate: what the team observed, what it inferred and what it chose to do.
- Adopt: keep a change because the evidence supports its intended use. Name the supported outcome and scope.
- Revise: the mechanism remains plausible, but the implementation did not resolve the observed problem.
- Stop: the change misled people, created an operational problem or failed a requirement that matters.
- Inconclusive: the available comparison cannot answer the question. State the missing evidence and whether resolving it is worth more work.
These decisions can differ across outcomes. A clearer confirmation message can be adopted for accuracy while its effect on attendance remains inconclusive. That is a precise conclusion, not a weak one.
Illustrative closing entry
Observed: period B produced fewer inquiries, more bookings and more attended appointments at equal media spending. Added Friday availability occurred during B.
Decision: retain the accurate next-step explanation. Do not claim that it caused the booking improvement.
Reusable finding: inspect suitable appointment availability alongside the message when diagnosing booking performance.
Next work: the practice manager will document availability by day and the marketing lead will preserve campaign versions for the next review.
Move completed tests out of the active queue. Keep a short searchable index by problem: unclear next step, low appointment fit, slow response, confusing offer. When a similar problem returns, the team can retrieve the relevant evidence instead of restarting the same debate.
Dr. Stacey's historical account highlights the value she placed on someone carrying testing and reporting. The practical standard here is ownership: the work is complete when the record helps the next person make a better decision, not when a new creative goes live.
Put it to work
Complete an experiment record before commissioning another variation
Use the request-confirmation example to connect a real problem, a specific change and a defensible decision.
A completed first pass
This is an illustrative usability exercise, not a reported client result. The practice uses a callback request form. Its old success message says “You're all set,” even though no appointment has been booked. Staff have observed confusion about what happens next. The practice first verifies the actual receiving workflow.
Experiment ID: EXP-014 — Request confirmation clarity.
Decision: should the success message explicitly distinguish a request from a confirmed appointment?
Hypothesis: explaining the next step will help people understand whether they have an appointment and what they should expect from the practice.
Version A: “You're all set.”
Version B: “Your request has been received. Our team will contact you to arrange an available appointment. You will receive a separate confirmation once a time is agreed.”
Primary check: after viewing the message in a fictional booking scenario, can the participant accurately explain their current status and the next step without coaching?
Guardrail: the message must match the actual workflow and must not imply a response deadline or reserved appointment that the practice cannot provide.
Owner: the practice manager verifies the process; the website owner implements and checks the live message.
Run the check without feeding people the answer
Use a fictional scenario and a demonstration that does not submit personal health information. Ask, “What has happened here, and what would you expect next?” Avoid, “You know this isn't booked yet, right?” The second wording supplies the desired answer.
In the invented exercise, three of eight people seeing A describe themselves as already booked. One of a different group of eight seeing B makes that error. Record the counts and the actual misunderstanding. This small, separate-group exercise helps expose usability problems; it does not establish a population-wide improvement or an attendance effect.
Inspect the remaining error. That participant noticed a calendar icon and assumed a time had been reserved despite the new words. The next revision should address that conflicting cue, then be checked again. Adding another paragraph would not necessarily solve the problem.
Close this record precisely
Decision: replace the ambiguous message with an accurate explanation, remove the misleading calendar cue and verify the result. Observed learning: copy and visual signals need to agree about the user's current status. Unresolved outcome: the effect on eventual booking and attendance has not been established.
For your own test, save the old and new versions, the exact question used, the observed misunderstanding and the decision. Then choose whether a later journey comparison is worth running. Use the creative testing plan to structure variants and the weekly campaign review to assign the next decision. The finished log should make the next round of work smaller and sharper.
Notes aren't saved automatically. Download them before leaving. Don't include patient details.
06 / Put it to work
Write the next test before you launch it
Write what you expect to learn before making the change. Preserve what happened and use it to choose the next action—even when the evidence points somewhere different from the original idea.
What is weakening your current experiments?
Your next move
We change several things at once
A package of changes can be necessary, but it limits what you can attribute to one element. Record the full variation and narrow the next question where practical.
- List every material change.
- Choose the decision the package is meant to support.
- Avoid assigning the result to one unisolated element.
Put your plan in motion
Your first campaign. Our compliments.
The Defibrillator connects 12 custom social posts, 3 articles, a landing page, a lead form and 3 follow-up emails. Built for your practice.
Request your kit with your name, practice, email and website. No call. No purchase.
Explore my free campaignThe method behind the work.
Continue with the original guides and the source material behind this lesson. Worked examples are teaching illustrations unless explicitly attributed.
Internet Playbook ↗Open the complete original guide.The Internet Playbook ↗The eight-step system joins applied psychology and applied economics, from the first useful encounter to ongoing improvement.The original R.O.I. Method ↗Relevancy: offer, positioning and messaging. Omnipresence: timing, frequency and platform. Intimacy: connection, community and conversation.The Meta Playbook’s three-stage diagnosis ↗Page 14 distinguishes attention, inquiries and appointments, with a different practical response at each stage.Sources and context
Dr. Stacey on testing and reportingHistorical 11:24–12:16 client account supports the operating value of owned testing and reporting, not current service scope or quantified improvement.
PowerSURGE local-context walkthroughHistorical reasoning example informs the distinction between a local observation and a causal conclusion. Old audience settings and reported results are not used as current instructions or expected outcomes.
