Next Experiments

Our most important upcoming changes

1. Goal - Build your experiment pipeline

What You've Already Built

In the previous levels, you created your Strategy (business context and goals), Story Framework (audience and messaging), Funnel (customer journey and conversion paths), Metrics (tracking and analytics), and Traffic Generation (channels and campaigns). Now it's time to systematically improve how that funnel converts–but not randomly. You need a structured pipeline for generating, prioritizing, and tracking experiments.

What You're Deciding Here

This lesson helps you answer: What should I test next, and how do I track what I learn?

Conversion optimization is fundamentally about experimentation. The businesses that win aren't the ones with the best initial ideas–they're the ones who test the most hypotheses and learn the fastest. But random testing wastes resources. You need a system.

Your experiment pipeline has two components:

• Ideas: A backlog of potential improvements, scored by impact and ease
• Experiments: Active tests with clear hypotheses, metrics, and learnings

What Should You Focus On First?

Your starting point depends on your data maturity:

If You're Early-Stage (limited traffic and data)
Focus on quick, directional tests. You won't have statistical significance for A/B tests, but you can still test big changes and watch for directional signals. Prioritize learning over proving.

If You Have Moderate Traffic (enough for meaningful tests)
Build a structured experiment pipeline. Score ideas by impact × ease, run proper A/B tests where possible, and document everything. This is where compounding really kicks in.

If You're Mature (significant traffic and optimization history)
Your challenge is avoiding local maxima. Keep testing bold hypotheses, not just incremental tweaks. Revisit assumptions. The biggest gains often come from rethinking fundamentals.

Your Decision

By the end of this lesson, you'll have a clear experiment pipeline–a prioritized backlog of ideas and a system for tracking experiments and learnings.

Outcome: You have a structured experiment pipeline with ideas backlog and tracking system.

2. Ideas - Generate and evaluate improvement opportunities

Ideas are the raw material of optimization. Without a healthy backlog of potential improvements, you'll run out of things to test. But too many ideas without prioritization leads to paralysis. You need a system for both generation and evaluation.

Sources of Improvement Ideas

Analytics and Data
Your metrics and funnels reveal where people drop off. High exit rates, low conversion rates, and long form abandonment times all signal opportunities. Start with your data.

User Feedback
What do customers say? Support tickets, chat logs, surveys, and reviews reveal friction points you might not see in the data. Direct feedback often identifies "why" behind the "what."

Competitive Analysis
What are competitors doing differently? Not to copy, but to generate hypotheses. If a competitor uses a different approach, it's worth asking if it might work better.

Best Practices Research
Conversion optimization has a body of knowledge. Social proof, scarcity, clear CTAs, fast load times–these principles often apply. But always test; what works elsewhere may not work for you.

Team Brainstorming
Your team sees things you don't. Sales knows objections. Support knows complaints. Design knows friction. Regular brainstorming surfaces diverse perspectives.

Evaluating Ideas

Not all ideas deserve testing. Use a simple scoring system:

Impact (1-5): If this works, how much will it move the needle?
Ease (1-5): How hard is this to implement and test?
Confidence (1-5): How confident are we this will work?

Prioritize ideas with high impact × ease × confidence. But don't only chase easy wins–occasionally test bold, high-impact ideas even if ease is low.

Outcome: You have a prioritized backlog of improvement ideas across your funnel.

3. Experiments - Structure and track your tests

An experiment without structure is just a change. To learn from optimization, you need clear hypotheses, success criteria, and disciplined tracking. This is what separates random tweaking from systematic improvement.

Anatomy of a Good Experiment

Hypothesis: A clear, testable statement. "Changing the headline from X to Y will increase conversion rate because Z." The "because" is crucial–it's what you'll learn from.

Metric: The primary metric you're trying to move. Be specific. "Conversion rate from visitor to lead on the pricing page" is better than "conversions."

Duration/Sample Size: How long will you run the test? How many conversions do you need for statistical significance? Running tests too short or too long wastes resources.

Variants: What exactly are you testing? Document the control and each variant precisely.

Result: What happened? Did you reach significance? What was the lift or loss?

Learning: What did you learn, regardless of result? Even failed experiments teach something.

Running Experiments

1. Isolate Variables
Test one thing at a time when possible. If you change headline, image, and CTA simultaneously, you won't know what drove the result.

2. Wait for Significance
Don't call winners too early. Statistical significance matters. Use a calculator to determine minimum sample size before you start.

3. Document Everything
Future you (and your team) will thank present you. Record hypotheses before you see results to avoid hindsight bias.

4. Kill Losers, Scale Winners
Once you have a clear winner, implement it. Once you have a clear loser, kill it. Don't let experiments run forever.

5. Learn from Everything
Failed experiments aren't failures–they're learnings. Document why you think something didn't work. These insights inform future hypotheses.

Outcome: You have structured experiments ready to run with clear tracking.

4. Your Next Experiments Library

Creating Your Next Experiments Document

Throughout this lesson, you've designed your experiment pipeline–how you generate ideas, prioritize them, and track experiments. Now it's time to consolidate everything into a single reference document that governs your optimization practice.

The Goal: A Living Experiment System

This Next Experiments document will serve as your:

• Backlog of prioritized improvement ideas
• Active experiment tracker
• Learning log that compounds over time
• Reference for what's worked and what hasn't

Having your experiment system documented means you (and your team) maintain momentum and never run out of things to test.

Outcome: You have a complete Next Experiments document that serves as your experimentation system.