Skip to content
Recaply.
Sign inSubmit a tool

LearnTutorials

Claude CodeTutorialsOpen Claude Code

Test your app's UI with Claude Code computer use

Let Claude Code build your Mac app, click through every screen with computer use and report what breaks, then fix it, in about 20 minutes.

By Recaply EditorsIntermediate20 min5 stepsUpdated

A large mouse pointer presses a round push button, with a magnifying glass lying beside it

What you need

Computer use is a research preview. It needs an interactive session, so it doesn't run with `claude -p`.

  1. Step 1: Turn on the computer use server

    2 min

    Computer use ships with Claude Code as a built-in MCP server, but it stays off until you enable it.

    1. In your project folder, run claude to start a session.
    2. Run /mcp and find computer-use in the server list.
    3. Select it and choose Enable.

    You should see computer-use shown as enabled. The setting stays on for this project.

    computer-use isn't in the list?

    Run /status to check your plan and login. The server only shows on a Mac, on Pro, Max or Ultra, when you sign in with a Claude account.

  2. Step 2: Grant the Mac permissions

    2 min

    Claude needs Accessibility to click and type, and Screen Recording to see your screen.

    1. Ask Claude for a small task on screen, such as "Open my app and take a screenshot."
    2. When the permissions prompt appears, use its links to open System Settings.
    3. Turn on Accessibility and Screen Recording, then select Try again in the prompt.

    You should see Claude carrying on with the task after you select Try again.

    The permissions prompt keeps coming back?

    Quit Claude Code fully and start a new session. Then check that your terminal app is on in System Settings > Privacy & Security > Screen Recording.

  3. Step 3: Ask Claude to click through a flow

    6 min

    One prompt covers the build, the launch and every screen of the flow you care about.

    1. Paste the prompt below and fill in your app and flow.
    2. When Claude asks to control your app, check the list and choose Allow for this session.
    3. Watch if you like. Other apps hide while Claude works and come back when it's done.
    Your prompt
    Build the [APP OR TARGET NAME] app and launch it.
Go through the [FLOW, SUCH AS ONBOARDING] flow from the first screen to the last, and click every button and link on the way.
Take a screenshot at each step.
Then list what broke: buttons that didn't respond, layouts that clipped or overlapped, error messages, and any screen that took more than a second to load.
Don't change any code yet.

    You should see your app opening and the pointer moving through each screen, with a Mac notice that Claude is using your computer.

    Claude can see your browser but won't click in it?

    Browsers are view-only for computer use. Test a web app with Claude in Chrome instead, or run your app as a desktop build.

    Press Esc anywhere to stop Claude at once. Terminal, Finder and System Settings show an extra warning before you allow them.

  4. Step 4: Read the report

    4 min

    The list tells you what to fix first, and each item points to a screen you can check.

    1. Read Claude's summary in the terminal.
    2. Ask where in the code each problem starts, such as "Which file sets the width of the signup footer?"
    3. Sort the list: crashes and dead buttons first, looks last.

    You should see a list of problems, each tied to a screen and a step in the flow.

    Claude misreads small text or misses a button?

    Screenshots are scaled down before Claude sees them. Make the text or the window bigger in your app, not your display resolution, and run the flow again.

  5. Step 5: Fix it and run the flow again

    6 min

    Claude can patch the code and check each fix on screen in the same chat.

    1. Paste the prompt below.
    2. Allow the app again if Claude asks.
    3. Repeat until the flow runs clean.
    Your prompt
    Fix the problems you found, worst first. Rebuild the app, run the same flow again, and confirm each fix on screen. Then list anything that's still broken.

    You should see a second run of the flow, with the problems gone or a shorter list.

    It says computer use is in use by another Claude session?

    Only one session can control your Mac at a time, and it keeps control until it exits. Close the other Claude Code session, then try again.

You get

A desktop app that Claude has built, clicked through screen by screen and fixed, without a single test file

Try next

Install agent skills in Claude Code6 steps · 5 minInstall an agent skill in any AI coding agent6 steps · 5 minConnect an agent to GitHub with DigitalOcean Action Gateway6 steps · 20 min

Did this guide work for you?

Sources

0 of 5Next: Turn on the computer use server