Question 1

What is vibe testing?

Accepted Answer

Vibe testing is end-to-end testing for apps built with AI coding tools. You paste a URL and one sentence about your app; the AI planner authors a structured test plan with assumptions, missing context, browser steps, and confidence. A real browser runs it. A vision model reviews the screenshots against the goal. You walk away with a token-gated public proof page and a release-health entry.

Question 2

How is vibe testing different from writing Playwright tests by hand?

Accepted Answer

Hand-written scripted tests take hours to author, need a repo, and break every time the UI changes. Vibe testing starts from a URL + 1 sentence; the AI authors a stable, accessibility-first test plan; a real browser runs it; a vision model reads the screenshots; you walk away with a shareable proof page and a release-health entry. No source-code access, no SDK install, no test maintenance.

Question 3

What does the AI planner actually produce?

Accepted Answer

A structured plan with: inferred product label, assumptions the planner is making, missing context (critical entries block the run; advisory entries become notes on the share page), browser test steps, safety warnings, and a self-rated confidence (0..1). The full planner JSON is persisted on the run, so the share page and release-health history can read it without re-running the model.

Question 4

What does the vision evidence pass actually do?

Accepted Answer

After the browser run finishes, a vision model reviews the final screenshot and any failed-step screenshots on every included run (AI is never metered). It returns a structured verdict: did the visual evidence prove the goal? What was the root cause of any failure? What are the limitations of the verdict? What should you do next? The verdict renders on the share page above the screenshot timeline and feeds release health.

Question 5

What is the customer release-health dashboard?

Accepted Answer

Team and Scale customers see /app/insights: readiness score, blockers, pass-rate trend, top failure categories, recent bug/regression history, and recurring regressions. Team gets basic release health over a 30-day window. Scale adds per-step latency, flakiness detection, trend analysis, exports and deeper drilldowns. Builder sees per-run results and its suites on Home.

Question 6

What does the execution engine actually do?

Accepted Answer

Well Tested runs the AI planner → browser execution → vision evidence → share page loop. A real browser runs the web path: Chromium on every plan, plus Firefox, WebKit, and iOS Safari and Android Chrome profiles (device emulation) on Team and Scale.

Question 7

Can you test native iOS and Android apps?

Accepted Answer

Not yet — native iOS / Android app testing is on the roadmap. Today, Team and Scale include mobile web profiles (iOS Safari + Android Chrome via device emulation) for responsive web flows.

Question 8

Can I kick off a test from my phone?

Accepted Answer

Yes — open /app/vibe on your phone on any plan. Pick a kept test, hit Run, watch status push back, share the proof link. The browser runs on our server; your phone is the remote. Phone-sized browser profiles (iOS Safari, Android Chrome) are on Team and Scale.

Question 9

Do you record video?

Accepted Answer

Yes. Failed runs keep their screenshots and a failure video. Evidence is kept 30 days on Builder, 90 on Team and 365 on Scale.

Question 10

How long does a vibe test take?

Accepted Answer

A typical 3–6 step vibe test finishes end-to-end in well under a minute on warm contexts. The first run after signup may take longer while the browser context spins up.

Question 11

What inputs does the AI need?

Accepted Answer

Vibe mode needs a URL and one sentence (10–500 characters) about your app. The planner produces the structured plan. Explicit mode is preserved for users who want to write a detailed prompt.

Question 12

What is the public share page?

Accepted Answer

Every run can have a token-gated /vibe/[slug]/share?t=<token> URL with the planner record, vision verdict, per-step screenshots, step outcomes, and the failure video if available. The slug is a public identifier; the 32-character token is the only secret, and you can revoke it.

Question 13

Can I share a vibe test result with my team or in a public channel?

Accepted Answer

Yes. The public share page is built for Slack, Twitter, and iMessage link previews (OG + Twitter card meta). Vibe coders ship the URL in standup, in launch announcements, or in a portfolio.

Question 14

What browsers and engines are supported on which tier?

Accepted Answer

Chromium on Builder. Chromium, Firefox and WebKit plus iOS Safari and Android Chrome profiles (device emulation) on Team and Scale.

Question 15

How long do you keep my screenshots and videos?

Accepted Answer

Run evidence is kept 30 days on Builder, 90 on Team and 365 on Scale. Enterprise is custom.

Question 16

How is vibe testing different from Cypress, Playwright, or QA Wolf?

Accepted Answer

Cypress and Playwright are tools you wire into a repo and own. QA Wolf is a managed service for hand-written tests. Vibe testing is goal-based: URL + 1 sentence → AI plan → real browser run → vision verdict → shareable proof page and release-health entry. No source-code access and no test code to maintain. If you want the code anyway, any kept test exports as standalone Playwright.

Question 17

Can I use Well Tested with an existing test suite?

Accepted Answer

Yes. The /app product shell exposes projects, environments, scenarios, and runs that work for hand-authored scenarios too. The AI scenario generator is the same generator behind /app/vibe — the difference is whether the run is one-off or part of a saved scenario library.

Question 18

Can I try Well Tested without a credit card?

Accepted Answer

Yes. The anonymous check at /check needs no signup and no card. Builder, Team and Scale are paid monthly or annually, and only passing runs use credits.

Question 19

Do you offer custom plans?

Accepted Answer

Yes. Enterprise covers custom credits, concurrency, evidence retention, SLA and support. SSO / SAML / SCIM and the workspace audit log are rolling out to Enterprise workspaces (ask us to enable them); private/VPN environment agents are on the roadmap. Use /contact to start a conversation.

Question 20

How do I get help?

Accepted Answer

Email support, the /contact page, or open an issue. Paid tiers add priority support and success planning. We respond fastest on the in-app help widget inside /app/vibe.

Question 21

Is the automated loop live?

Accepted Answer

Yes. Sign up, pick a plan, then open /app/vibe: paste a URL and one sentence (or say it out loud), approve the plan, and a real browser runs it. Failures come back with AI triage and a fix prompt; passing tests can be kept and rerun on every push, deploy or schedule. Try it first with a free check at /check. No signup, no card.

Question 22

Can my coding agent run the tests itself?

Accepted Answer

Yes. Run `npx welltested setup` in your project. It adds the Well Tested MCP server and a "verify before done" skill to Claude Code, Cursor, Codex, Windsurf, Copilot or Kiro (Cline and Antigravity with --agent), shows you the diff and asks first. Before your agent says a change is done, it runs your kept tests (`welltested run` or the MCP tool) and reads the verdict, the AI triage and the proof link. A new test plan never runs until you approve it in chat.

Question 23

Can I test an app running on localhost?

Accepted Answer

Yes, two ways. `npx welltested tunnel --port 3000` opens a tunnel scoped to the ports you name so our cloud browser can reach your local app. Or `npx welltested run --local` reruns your kept tests in a browser on your own machine, with no tunnel at all, and uploads masked evidence.

Question 24

What happens when a button is renamed or moves?

Accepted Answer

Self-healing is on for every plan. When a step can't find its target, the repair model looks at the live page and repairs the locator inside the run; the step is labelled "Passed after repair" and you can keep the repair with one click. If that still fails, a vision fallback clicks the control it can see on the screenshot and labels the step "Clicked by vision". Assertions and destructive steps are never healed, and it refuses instead of guessing when nothing matches.

Question 25

What do I get when a run fails?

Accepted Answer

Jev and Clef triage the failure: a real bug in your app, or a test or environment problem, with the signals behind the call. You also get screenshots for every step, a failure video, the browser console errors and failing network requests, what changed since the last passing run, and a fix prompt to paste into your coding agent. Only passing runs use credits. A run that catches a bug in your app, or fails because of us or your environment, is free.

Question 26

Can tests run on every push or deploy?

Accepted Answer

Yes. Keep tests in suites and choose when each suite runs. Builder: Manual, Coding agent, Every push, Every deploy, Daily. Team: Manual, Coding agent, Every push, Every deploy, Daily, Hourly, plus PR comments from the GitHub Action. Scale: Manual, Coding agent, Every push, Every deploy, Daily, Hourly. Push runs use our GitHub Action or a push hook, deploy runs use a deploy hook URL, and suite results can be emailed to you.

Question 27

Can I export my tests?

Accepted Answer

Yes. Any kept test exports as standalone Playwright code that runs without Well Tested. Receipts can also export an .evidence pack; that format is still early and may change.

Question 28

Can you test my chatbot or AI feature?

Accepted Answer

Yes, in beta on every paid plan. Point it at a chat widget or an AI endpoint you own (you prove ownership with a DNS record or a file on your site) and it checks whether the assistant does its job, stays on task, resists prompt injection, keeps data from leaking, gets known facts right and keeps a safe tone. Nothing runs until you approve the scenarios.

Question 29

Can you test pages behind a login?

Accepted Answer

Yes. In Apps, open an environment and choose Configure auth profile. Save a username and password, or custom login steps for flows like SSO redirects, and click Save & test login to check it works; the success URL is a page you only see when signed in, such as your dashboard. On New test, pick that environment in the Sign-in picker: planning reads the signed-in page, and the run signs in once and runs your steps in that session. The password is encrypted at rest and never appears in screenshots, logs or proof pages, and login steps keep no video.

Question 30

How do I get an account?

Accepted Answer

Sign up from the pricing page: pick a plan, sign in with your email, and add a card. Try it first with a free check at /check. No signup, no card. Paid plans: 300, 1,000 or 3,000 credits a month (Builder, Team, Scale); Firefox, WebKit and mobile-web profiles on Team and Scale.

Questions about AI browser testing and vibe testing.

Vibe testing, frequently asked.