Feedback Loops and Iteration

The build-measure-learn cycle

        ┌─────────┐
        │  LEARN  │
        └────▲────┘
             │
    ┌────────┴────────┐
    │   What did we   │
    │   discover?     │
    └────────▲────────┘
             │
        ┌────┴────┐
        │ MEASURE │
        └────▲────┘
             │
    ┌────────┴────────┐
    │   What happened │
    │   when users    │
    │   tried it?     │
    └────────▲────────┘
             │
        ┌────┴────┐
        │  BUILD  │
        └────▲────┘
             │
    ┌────────┴────────┐
    │   What's the    │
    │   smallest thing│
    │   to test?      │
    └─────────────────┘

Each cycle teaches you something. More cycles = more learning = better product.

Types of feedback

Qualitative (the “why”)

What users say and show you:

• User interviews
• Usability tests
• Support conversations
• Comments and reviews
• Watching people use your product

Good for: Understanding problems, discovering needs, generating ideas

Limitation: Small sample, can be biased, people don’t always know what they want

Quantitative (the “what”)

What the numbers tell you:

• Analytics (page views, click rates)
• Conversion funnels
• Error rates
• Performance metrics
• A/B test results

Good for: Validating assumptions, finding patterns, measuring impact

Limitation: Tells you what, not why. Easy to measure wrong things.

The combination

Use both:

1. Qualitative to generate hypotheses (“users seem confused by the pricing page”)
2. Quantitative to validate (“42% drop off on pricing, vs 15% on other pages”)
3. Qualitative to understand (“interviews show pricing tiers are unclear”)
4. Build a change
5. Quantitative to measure impact (“drop-off reduced to 25%”)

Building feedback into your process

Before you build

Ask: What question are we trying to answer?

Not “build a pricing page” but “learn whether users understand our pricing.”

Every feature should have a learning goal:

• Will users find this?
• Will they understand it?
• Will they use it?
• Will it solve their problem?

While you build

Ship small, ship often.

Instead of building for 3 months then launching:

Week 1: Ship basic version → Learn
Week 2: Improve based on feedback → Learn
Week 3: Add one feature → Learn
Week 4: Iterate again → Learn

Four learning cycles beats one big launch.

After you ship

Measure what matters (see: Metrics that matter )

Set up tracking before you ship:

• What should happen if this works?
• What would tell us it’s failing?
• What unexpected behaviors should we watch for?

Feedback collection methods

For early stages (0-100 users)

For growth stages (100-10K users)

For scale (10K+ users)

The feedback conversation

When talking to users:

Questions that work

• “Walk me through the last time you tried to [task]”
• “What were you trying to accomplish?”
• “What was confusing?”
• “What did you expect to happen?”
• “What would you do differently?”

Questions that mislead

• “Would you use this feature?” (People say yes but don’t)
• “Do you like this?” (They’ll be polite)
• “How much would you pay?” (Hypothetical, unreliable)

The Mom Test

From Rob Fitzpatrick’s book:

• Talk about their life, not your idea
• Ask about the past, not hypotheticals
• Talk less, listen more

Bad: “I’m building an invoicing app. Would you use it?” Good: “Tell me about the last time you invoiced a client. What was annoying about it?”

Iteration patterns

The minimum viable test

Before building a feature, find the smallest way to test the assumption:

The fake door test

Add a button for a feature that doesn’t exist yet. Measure clicks.

[Try AI Summary (beta)] → "Coming soon! Want early access?" → Collect emails

If no one clicks, no one wants it.

The Wizard of Oz

Manually do what the feature would automate. Test value before building.

Instead of building AI-powered recommendations:

1. Manually curate recommendations for early users
2. See if they engage
3. Only automate if the value is proven

Making iteration sustainable

Timeboxed experiments

Don’t iterate forever. Set bounds:

Experiment: New onboarding flow
Timeline: 2 weeks
Success metric: 50% complete onboarding (vs 30% baseline)
Decision: If <40%, revert. If 40-50%, iterate. If >50%, ship.

The iteration log

Track what you tried and learned:

Date: 2024-03-15
Change: Added progress bar to onboarding
Hypothesis: Users abandon because they don't know how long it takes
Result: Completion up from 30% to 38%
Learning: Progress visibility helps, but not the whole problem
Next: Test reducing number of steps

Avoiding iteration theater

Signs you’re iterating without learning:

• Shipping changes without measuring
• Measuring but not changing behavior
• A/B testing things that don’t matter
• “We iterated” without being able to say what you learned

Feedback loop anti-patterns

The long loop

Building for months, launching with fanfare, then scrambling to fix.

Fix: Ship weekly. Learn continuously.

The echo chamber

Only talking to people who already like your product.

Fix: Actively seek critics. Watch people who struggle.

The vanity metric trap

Measuring what makes you feel good, not what drives decisions.

Fix: Define success metrics before shipping. Be honest about what matters.

The feature factory

Shipping features without checking if they worked.

Fix: Don’t start feature N+1 until you’ve measured feature N.

The feedback paralysis

Collecting feedback but never acting on it.

Fix: Set decision criteria upfront. Act on what you learn.

Feedback loops with AI development

AI makes building fast. This makes feedback loops more important.

The danger

AI can generate features in hours. Without feedback loops, you build the wrong thing faster.

The opportunity

Fast building + fast feedback = rapid learning

The workflow

1. Hypothesis: "Users need X"
2. AI builds X (hours, not weeks)
3. Ship to small group
4. Measure
5. Learn
6. AI iterates based on learnings
7. Repeat

Short cycles mean you can test more ideas. Test more ideas, find more winners.

The honest take

Feedback is uncomfortable. Watching users struggle with something you built hurts. That’s the point—it shows you what to fix.

Fast feedback beats perfect feedback. A quick conversation with 3 users beats a month-long survey you never run.

Shipping is a prerequisite. You can’t get feedback on something that doesn’t exist. Ship, then learn.

Measure what matters. Easy metrics (page views) feel good. Hard metrics (did this solve the problem?) drive progress.

Further reading

• Metrics that matter : What to measure and why
• How to slice work effectively : Ship small to learn fast
• Agile for solo builders : Retrospectives and continuous improvement
• Product discovery techniques : Understanding what to build