Customer discovery & product strategy · Developer tools · 2025–present

Replay

Replay had a working vibe coding AI tool but wasn't sure who needed it most. Over more than a year of working together, repeated customer interviews helped the team move through five increasingly specific customer hypotheses — from operations managers looking to automate workflows to builders responsible for keeping real applications running.

Across the documented cycles, I conducted 34 structured interviews as part of a broader effort involving nearly 50 customer conversations. Each cycle tested an assumption, revealed what held up or fell apart, and changed who the team recruited next. The result wasn't a single research report. It was a recurring practice of using customer evidence to make more focused product and positioning decisions.

By Skipper Chong Warson · Published 20 May 2026 · Updated 17 Oct 2026

Replay customer hypotheses narrowing across five research cycles

Where should a small team focus?

The working name for their product was nut.new, it was a fork of Bolt that gave the AI suite access to real runtime data: what the app was doing, what had broken, and why.

In spring 2025, AI tools could generate a working prototype quickly, but would often stall when a change failed. The AI couldn't see what was happening behind the scenes while nut.new could inspect the runtime, debug the problem, run tests, and push a fix through.

Replay's CEO and a small engineering team had a hunch about who needed it most. The decision left was where to focus: who to recruit, which problem had enough teeth, and how to position the product.

The first hypothesis

The first bet was that operations managers needed help automating simple create, read, update, and delete (CRUD) workflows. We framed the hypothesis together, recruited five people, designed the interview guide, and away we went.

In those interviews, they all described real problems:

  • Setup paralysis
  • Fragmented tools
  • The fear of breaking something they could not repair

But across the five interviews, there didn't seem much pressure to change. One operations manager put it plainly: “I’d rather deal with the inefficiency I know than risk configuring something wrong.”

That first sprint answered the question we'd put in front of it, but it raised another: if operations managers weren't the right customers, who were? Replay wanted to keep talking to customers as the team continued building, rather than stop after one round of research. That request became the basis for Listening Cycles: recurring rounds of interviews, synthesis, and decisions, each informed by what we'd learned in the previous one.

Original Bullseye Customer debrief with Replay

Cycle one: five operations managers described real pain, but little pressure to change

We kept getting narrower with the customer

The first five interviews gave us an answer, but not the one we'd hoped for. Operations managers had real problems, yet little urgency to change how they worked. So we revised the hypothesis and went back out.

Across the five documented customer hypotheses, the customer began to shift from someone automating workflows for a team toward someone building an application of their own. Each round challenged another assumption about who needed the product most.

Customer hypothesis What we learned
1. Operations manager The pain was real, but there wasn't enough pressure to change existing workflows.
2. Operations manager with a stronger trigger We tested whether a more urgent situation would make the original audience a better fit.
3. App builder The opportunity extended beyond operations managers automating internal workflows.
4. Solo builder Being employed by a company or working as a professional developer mattered less than we'd assumed.
5. Creator who had already shipped The strongest recurring distinction was having something real in use and being responsible when it broke.

The progression wasn't simply toward a smaller audience. We were learning which characteristics actually mattered. Employment status, professional credentials, and technical proficiency became less important. Having shipped something real, and being accountable for what happened afterward, kept showing up.

In one interview, a customer described an internal app she'd built for contractors to submit work and receive payment. She'd written most of it with Claude's assistance after trying Lovable and Replit, and she deployed directly to production most days.

When something broke, there was no engineer to escalate to. She could ask friends who were developers, but those favors ran out quickly. As she put it, “Now I broke something else, someone's got to debug it anyway.” So it might as well be her.

That interview, along with others in the cycle, sharpened the distinction. The customer didn't need more help getting to a first prototype. She needed to understand what broke and keep something people depended on working.

Each cycle gave us a clearer idea of whom to recruit next, and which assumptions we could stop carrying forward.

Replay customer hypotheses from cycles 1 through 5

Five cycles moved the bullseye from operations managers to non-technical creators who had already shipped something and couldn't get unstuck

How the team worked

I ran the engagement end to end: hypothesis framing, recruiting, interview design, facilitation, synthesis, debriefs, and recommendation framing. And the Replay team joined the research directly.

Brian, the CEO, participated from the start. Filip joined the early Bullseye framing work. Tom joined the first interviews, and other team members began sitting in as the cycles continued.

Each Listening Cycle started with a specific customer hypothesis. I recruited against that hypothesis, designed and conducted the interviews, and brought Replay's team into those conversations. Afterward, we debriefed, compared what we'd heard, challenged our assumptions, and decided what to keep, change, or stop pursuing.

Those decisions shaped the next cycle's recruiting criteria and research questions. Rather than delivering findings and walking away, we kept testing the customer definition as the product and market evolved.

The team wasn't just receiving research. They were hearing the evidence firsthand and using it to decide where to focus next.

Making the research reusable

After the interview rounds, I created a Claude project containing the Bullseye research corpus: 34 redacted transcripts, screeners, interview guides, and a transcript reference key.

Claude project containing Replay customer interview transcripts, screeners, interview guides, and reference materials

The Claude project made an evidence archive, 34 Bullseye Customer Sprint interviews searchable and traceable

Rather than treating research as a static deliverable, the archive let us search across 34 redacted transcripts, revisit individual customer accounts, and trace insights back to the original conversations as new questions emerged.

For example, when exploring whether the trust gap mattered more than the volume of AI-generated code, I returned to the archive and retrieved the interview with a builder who had shipped an application for their contract workers. Her experience helped make the distinction concrete: the problem wasn't generating more code, but understanding what had broken and whether she could fix it with guidance.

The archive didn't replace the interviews or our judgment. It made the underlying evidence easier to revisit as our hypotheses shifted, helping us answer a recurring question: What do we actually know?

Decisions changed

Across our work together, I could see the Bullseye Customer work inform four types of decisions —

  • Customer: Operations managers dropped out of focus after the first interviews showed real pain with little pressure to change
  • Recruiting: We rewrote the criteria after each cycle as the customer definition got narrower, mostly using userinterviews.com for screening and qualification
  • Audience: Replay moved from a broad app-builder audience toward non-technical creators who had already shipped something real
  • Positioning: The problem moved toward continuation: helping someone understand and fix what broke after they had already built and shipped

Replay’s community manager and developer relations lead put it this way:

“There are moments when you feel like you've built the right product, but you're not sure you're reaching the right people, and that's where Skipper really shines. He has a talent for cutting through the noise, helping teams clearly define their target audience and identify their bullseye customer.”

— Filip, community manager and DevRel lead at Replay

What Replay has shipped

nut.new launched in spring 2025. Replay Builder followed in Dec 2025, Replay QA in May 2026, and MCP integrations with Claude Code and Codex in Jun 2026.

Those launches belong to Replay’s continuing product work. I don't claim them as follow-on effects of my work — customer definition, recruiting criteria, and positioning decisions — but they address the same continuation problems the interviews surfaced.

Where it landed

More than a year of collaboration and work gave Replay a sharper customer hypothesis, more specific recruiting criteria, and a shared body of customer evidence to build on. Each cycle helped the team decide which assumptions to keep testing and which to leave behind.

Replay continued to own its product roadmap and kept shipping throughout our work together. My contribution was helping the team establish a repeatable way to learn which customers and problems deserved attention as the product evolved.

We've worked together across multiple research cycles with further work under discussion as Replay continues developing its products.

Other select projects