Designing with agents

How we design UI and UX with AI agents

I've started using a repeatable workflow to design software with AI agents. They research interfaces in Mobbin, a searchable library of screenshots from real apps and websites, then generate mockups and revise them with my feedback. Once the direction is clear, they implement it in the product.

By Artemii Novoselov · Sweat AIA visual walkthrough · original prompts included

One worked example · our assistant's original Performance screen

The original Performance screen shows daily views, a growing list of films and the question Is the engine getting better?
The example: our assistant's original Performance screen, with daily views beside a film list. Original Performance screen ↗

I'll use a redesign of our assistant's video dashboard to show the pattern in practice. The screen already had a daily views chart and a growing film list. We'd written about making twelve AI films and building our own film studio in earlier posts. I wanted to compare each film's performance, open a film and inspect its retention curve: the platform's report of audience remaining over time.

I asked Codex, the coding agent, to help redesign the assistant's Social screens. My visual brief was loose: make the interface "nicer and easier on the eye." I first had it read the existing product context and research reference screens in Mobbin. We could then generate mockups, revise them, and carry those decisions into the app.

Find references for the task

Mobbin's MCP (Model Context Protocol) connection lets a coding agent search the library in plain language and receive screenshots directly in the working session. For this redesign, Codex looked for ways to keep films identifiable beside their measurements and locate moments on retention curves.

The agents selected three references. Each supplied a specific idea for our task.

Vimeo via Mobbin

Vimeo screenshot via Mobbin: Video identity stays beside its measurements.
Borrow from Vimeo: keep a film's title beside its measurement row so we can identify what we are comparing.

Loom via Mobbin

Loom screenshot via Mobbin: Loom pairs recognizable previews with titles and metadata.
Borrow from Loom: pair a recognizable preview with a readable title so we can scan the library.

TikTok via Mobbin

TikTok screenshot via Mobbin: A vertical time marker identifies a moment on the retention curve.
Borrow from TikTok: mark a time on the retention curve so we can locate a moment in a film.

These historical screenshots show arrangements, not verified interactions or platform formulas. Searches and canonical sources ↗

The agents turn references into a mockup

Next, the agents sent a written brief and five images to ImageGen, the image-generation tool built into Codex. Each attachment had a job:

Inputs to the original library request

  1. Image 1: our appKeep the product identity and navigation.
  2. Image 2: LoomBorrow the preview-and-title arrangement.
  3. Images 3–5: our film stillsUse our films as content inside the interface.
The agent sent these images with a written prompt to ImageGen. Full original request ↗ · Attachment order and substitute inputs ↗

The prompt asked for a desktop screen in our existing style, with readable titles and sample performance figures. It also said, "The video is the main object." Here is what ImageGen returned:

First library mockup · ImageGen · illustrative data

Generated film library with three large poster columns and measurements below.
The generated image gave us a layout to review before implementing it.

"We have a lot of videos," I wrote. Three films at a time would not be enough. I asked the agents to make the grid smaller.

Each objection became the next instruction

For the revision, the agent attached that mockup with four more film stills. It asked ImageGen to keep the surrounding interface and fit more films into the grid.

The revised generated library fits six columns of compact film previews.

Three columns became six.

More films fit in the same workspace.

Revised ImageGen library · illustrative data.

We then reviewed a detail mockup the agents had generated from our app screen, the TikTok reference and a film frame. It put a large player beside the statistics. I wanted that space for analysis. "Stats are more important," I wrote.

This time, the agent attached only the chosen mockup. Its edit request asked ImageGen to preserve the film and navigation while changing the hierarchy: metrics first, retention second, playback a small utility.

Original generated detail with a large portrait player beside the analysis.

Before: playback dominates the detail.

Revised generated detail giving retention most of the content width.

After: metrics and retention get the workspace; playback becomes a small reference.

Before → after · ImageGen concepts with illustrative data. The revised curve uses the time marker the agents noted in the TikTok reference ↗.

I had started with "nicer and easier on the eye." Now I could tell the agents what deserved space. The images made those decisions easier to discuss before we committed them to code.

References and exact prompts for these revisions

The agent's original contact-sheet prompt said, "The video is the main object." Compare it with the smaller-grid prompt and the statistics-first revision prompt.

Open the Loom reference on Mobbin ↗ or the first generated library image ↗. The historical inputs and substitutions explain what accompanied each request.

For your own project, see the prompts you can adapt and ImageGen guide ↗.

Final result

The agents then implemented the revised direction in the native app. The library uses aligned rows to keep film titles beside their numbers; the detail screen gives metrics and retention more room. These screenshots use synthetic preview data. The native changes are under review and unreleased.

Native video library with film titles and measurements in aligned rows.
Video library. Film titles and metrics share a row. Open the image for the full screen.
Native video analysis with watch measurements above a wide audience retention curve.
Video analysis. Watch-time metrics and retention get the workspace. Open the image for the full screen.
Restored native daily Performance chart showing changes in captured view counts by collection date.
Daily Performance. The restored overview groups changes in captured view counts by collection date.

Carry the decision into the next stage

Here is the workflow I'll reuse. I set priorities and review the results; the agents carry each decision into the next draft or into code.

The pipeline we used · human review between stages

  1. Codex + Mobbin
    Context → references

    Inspect the existing app and code; identify the task. Save each reference with the idea to borrow and why.

  2. ImageGen
    References → mockups

    Combine the current screen, selected references and realistic content. Save the image with its prompt. Save the attachment order with the prompt.

  3. Human + agent
    Feedback → revisions

    Walk through the proposed task in the image. Keep each objection with the revision it prompted.

  4. Coding agent
    Chosen mockup → app

    Define actions and expected results, then implement them in the existing product. Run the task and revise what fails.

Decision aid · derived from this case

What should you ask the agent for next?

If you cannot name the task yet, ask for a walkthrough of what the user does and the decision it must support. Once the task is clear, use the unresolved part to choose the next request.

The priority is unclear
Request a visual alternative using the same content. Walk through the task and identify what should dominate.
Information is hard to compare
Request two layouts using the same content. Keep that content fixed while comparing them.
A label or number is unclear
Ask for its source and definition. Check that the interface matches them.
An action's result is unclear
Request a running task check. Inspect the state before the action and after it.
The next request follows from what remains unresolved. Save the artifact with the decision it supports and the check still needed. Human review connects the stages; the work can loop back when a check fails.

For the next project, I'll give each agent the chosen screen and the reason we chose it. That makes the next request specific enough to act on, whether it calls for another image or a change to the app.

Practical resources

A design brief you can adapt

Adapt this brief; the reusable stage prompts provide detail.

We are improving [product] for [user].
The task is [task], with [realistic content volume].
An awkward case is [case]. Read the existing code and decisions.

Research arrangements through Mobbin MCP; save sources and reasons.
Generate alternatives from our screen, references and content.
Label illustrative values. Walk through the task; revise objections.

Before coding, define actions, metric inputs, missing/loading/empty/
failure states, return navigation and exact expected results.
Implement in the existing app. Run the task with a keyboard and at
supported widths. Save inputs, decisions, results and open issues.
Report release status separately.
A question to settle in your own brief

What should "progress" mean?

Decide what "progress" should mean in your own brief. These questions need different evidence.

Conceptual sketches · no measured data

"Show progress"

Did we publish more?

Count the output.

Did they collect more views?

Compare view counts.

Did people keep watching?

Inspect watch behavior.

One request permits several interfaces. These sketches illustrate possible questions; their shapes are not results from our account.

Name a concrete task. State its platform and account, then choose a comparison period.

Sources and original drafts

Optional implementation analysis files. Save observations.json and reproduce.py together; run python3 reproduce.py (Python 3 standard library). It checks the supplied 42 rows, selects their top four and reproduces the arithmetic. It cannot establish completeness or repeat nearest-capture selection across omitted private history.

Readings were frozen October 22, 2026, at 17:50 UTC; the four Instagram readings were cross-checked against the read-only app API October 24. The executed notebook has saved results and editable figures; the README gives rerun dependencies and scope.

Inspect historical inputs. Read the prompts with their attachment map. Supply your own Mobbin references and film stills; originals are not bundled. Public images are substitutes. The ImageGen model and its version were not recorded. Neither was the seed. Check generated labels and dates. Inspect the chart geometry; exact pixel replay is not guaranteed.

Inspect native captures. Native captures use synthetic preview data; concepts use illustrative numbers. The visual source record lists attribution, dimensions, viewports and the native chart's source commit.

Original screens: starting Performance view, three-poster concept, smaller grid, large player, statistics-first concept.

Five historical prompts: Performance, contact sheet, screening desk, grid revision, detail revision. The input map explains their dependencies.

Native implementation captures: first card library, first detail, revised table, earlier detail refinement, current watch analysis, current activity analysis and daily overview.

Explanation references. Presentation and narrative references: Nicolas Rougier's Scientific Visualization, Tim Urban's Neuralink essay and the Save the Cat beat sheet, adapted to nonfiction. The explanatory diagrams are original.