AI-native B2B SaaS · Agent Interface
Ocula's copy generation product was producing good results, but getting copy to the exact style and structure large retailers needed required solutions engineers hand-building templates and manual post-processing on every batch. This project replaced that manual overhead with a conversational agent interface that let users build, refine, and iterate on copy templates directly, cutting the gap between what the product generated and what customers actually needed.
A redesigned chat interface, where an agent interacts directly with a live template artifact, gave users the expressive control they needed to define nuanced copy rules conversationally, cutting out the manual fixes and processing time that used to eat into peak trading season. Users could define, refine, and update copy rules conversationally, including granular or broad feedback on generated samples, and it was delivered before Black Friday, in time for the most critical trading period for Ocula's retail customers.
Ocula's customers are large e-commerce retailers who need product copy generated at scale, and generated to a very specific style and structure. The product was doing the heavy lifting, but the final stretch was being handled manually: solutions engineers would hand-build custom templates, and copy would go through post-processing fixes before it was delivered to customers.
This was expensive, slow, and unscalable. With Black Friday and the Christmas sales season on the horizon, the most demanding period of the year for online retailers, there was real commercial pressure to close the gap between what the product generated and what customers could actually use.
The root cause wasn't the copy itself, it was the template creation process. Ocula's stepper form gave users a single configuration window before work began: they could provide guidance through blocklisted and preferred words and phrases, but this happened upstream and disconnected from the actual output. Once generation started, users couldn't adapt their instructions based on what they were seeing.
The feedback mechanism on generated samples was similarly constrained, it could accept limited, structured input but couldn't handle nuanced or iterative direction. When copy didn't land quite right, the only path forward was manual intervention.
An earlier attempt to address this through a more robust sample feedback mechanism showed some improvement, but didn't solve the underlying problem: the templates themselves weren't specific or expressive enough in the first place. Fixing the feedback loop on bad output was treating the symptom.
The stepper form wasn't just inconvenient, it was architecturally wrong for the job. Copy style is nuanced, contextual, and difficult to fully articulate before you've seen any output. Users needed to shape their templates in response to what the agent produced, not in advance of it.
The template shouldn't be a configuration artifact created before the work starts, it should be a live artifact that the conversation acts on directly.
A small team of four, several of whom, including the designer, were context-switching across other feature projects simultaneously. No prior experience designing agent chat interfaces at the team level, so significant learning happened on the job. And an aggressive timeline tied to Black Friday and Christmas trading seasons, a hard external deadline with real commercial consequences if missed.
The strategic bet was to collapse the distance between conversation and output. Rather than a form that feeds an agent, build an interface where the agent and the user work on the template together, in real time, in the same view.
This meant a split-pane layout: chat on the left, the live template (and generated samples) on the right. Changes the agent makes to the template are visible immediately. Feedback on samples feeds back into the template directly. The conversation and the artifact are coupled, not sequential.
Critically, this also meant rethinking how the agent itself was designed. Early attempts to map the interaction as a traditional user journey quickly broke down, you can't impose deterministic flows on a probabilistic system. The team shifted to designing agent behavior as a series of stages, each with explicit acceptance criteria that had to be met before the stage could begin, and success criteria that had to be met before it could complete. Some stages referenced others, creating a structured dependency chain that guided the agent reliably without constraining it artificially.
From user journeys to stage-based agent criteria. The hypothesis was that agent interactions could be designed like traditional user flows. That thinking was abandoned entirely in favour of a structured prompt framework built around stage acceptance and success criteria. Deterministic UX thinking doesn't transfer to probabilistic systems, the design work isn't mapping flows, it's writing rules the agent can interpret and act on reliably.
Template presentation. Basic markdown rendering turned out to be insufficient. The template is the primary artifact users are shaping, its presentation needed to communicate structure and hierarchy clearly, not just content, so it moved to a more structured visual layout.
Agent behavior testing and refinement. Desk testing between the designer and data scientist/engineer wasn't enough. It expanded to a full internal testing day with the entire team, then live release with customer conversation logging via Pydantic, reviewed manually and using agents to identify edge cases and behavioral drift. Agent behavior issues that don't surface in controlled testing emerge quickly in real use; closing the loop with logged customer interactions was essential.
Users were able to provide feedback on generated samples in a way the previous mechanism couldn't support, broad directional feedback, granular detail-level corrections, and everything in between. The agent could ask clarifying questions when feedback was vague, surfacing nuance that the stepper form had no mechanism to capture. Internal testing flagged edge cases in agent behavior that were resolved before customer release.
There was no dashboard tracking template accuracy, but the proxy that matters most is real: solutions engineering went from being involved in nearly every single copy generation run, close to a full-time commitment, to only stepping in for complicated rule setups or when something broke. That freed them to spend their time supporting customers and improving the platform instead. Adoption was just as clear: every customer is now on this flow, though getting there took some teething, both us and customers learning what a well-formed rule for the agent actually looks like.
The lesson I keep coming back to: deterministic UX thinking doesn't transfer to probabilistic systems. The job isn't mapping a flow, it's writing rules and criteria the agent can interpret and act on reliably. Every agent interface I've designed since starts from that, not from a user journey.
Designing agent behavior through a prompt framework was the right call, but the learning curve was steep and the process was largely invented as we went. Earlier investment in a shared vocabulary and methodology for agent interaction design, before the project started, would have saved real weeks of that invention.