A Failed Demo Step Is a Bug Report (and a Smoke Test)
A demo script is a list of steps that must work for the video to exist: open this page, click this button, type here, see this text. That is also the definition of a smoke test. When a render fails, Klevo reports the step that broke, a screenshot of the page at that moment and the browser’s console errors, which is a bug report a developer can act on. Here is how to read one, and how to use the scripts you already have as a check after each deploy.
What does a failed render tell you?
The video page (and the videos tool in the chat) shows four things for a failed render:
| Field | Meaning |
|---|---|
| Code | not_found (nothing matched the target), ambiguous (two things matched), timeout (the element never became ready), navigation_failed, auth_failed, deadline |
| Step | Which step, counted from one, with its target and caption |
| Screenshot | The page as it looked when the step failed |
| Console | The page’s error-level console messages, up to ten |
Read them in order. The code says what kind of failure; the step says where; the screenshot usually says why at a glance (a modal in the way, a renamed button, an error page); the console catches the rest (a failed request, an uncaught exception).
Regression or deliberate change?
Every failure is one of two things:
- A deliberate change. The release renamed “New project” to “New” and the step targets the old name. Update the step and render again. Keeping demos current after every release has the routine.
- A regression. The button is still called “New project” but clicking it now throws, and the console says so. File the bug with the screenshot and the console lines; both are in the failure report.
The tell is the screenshot. If the page looks healthy and the element is simply elsewhere, it is a change. If the page shows an error, a spinner that never ends or an empty state that should not be empty, it is a bug.
Why do stable targets matter?
Klevo saves each step with a target built from the element’s role and accessible name (role=button[name='New project']), not its position or CSS class. A restyle does not break the script; a rename does, and a rename is exactly the kind of change you want to be told about, because every screenshot and help article with the old name is now wrong too. Demo videos that never go stale explains the script format.
How do you use scripts as a smoke test today?
After a deploy, re-render the scripts for your most important flows:
Render scripts onboarding-first-project, team-invite and billing-change-plan again.
Three renders, three results. All green: the flows work and the demos are current. One red: open the failure, decide change or bug. That is a smoke test with a video as the by-product, run from a chat, in the time the renders take. A render counts as a video for the month, so this is a check for the handful of flows that matter, not a replacement for a test suite.
What is this not?
It is not a full end-to-end test. A demo script checks that the elements exist and the flow completes; it does not assert on data, timing or edge cases. It runs against a public URL with a test account, at a human pace, and it needs the chat (or the video page) to be triggered today. A render triggered automatically from CI after each deploy is on Klevo’s roadmap and not built yet; when it is, the same scripts will run without anyone asking.
How do you write scripts that fail usefully?
A few habits make the failure report more useful than the video:
- End with a
waitfor the result. “Wait for the text Q4 launch” fails if the project was not created, even when every click succeeded. Without it, a silent failure renders a fine-looking video of nothing happening. - Target by name, not by position. Let the AI pick refs from the snapshot; it records role-and-name targets. Avoid
css=targets unless nothing else is unique. - One flow per script. A script that does three things fails on the third and tells you nothing about the first two.
- Keep the test account stable. A flow that only works on a fresh account will fail the second time for the wrong reason. Demo videos without real customer data covers the demo workspace.
What does a worked failure look like?
Step 3 of team-invite fails with timeout after the deploy. The screenshot shows the invite dialog open with the Send button disabled. The console says POST /invites 500. That is a regression: the dialog is fine, the request behind it is not. The bug report is the screenshot, the console line and the step; the developer has what they need without reproducing it by hand.
FAQ
Is a failed render counted? No. Failed renders do not count against the monthly limit.
Can I keep the failure screenshot? It is on the video page until you delete the video, and the videos tool returns its link.
Does a failed render produce a video? No. It stops at the failed step and keeps the screenshot and console instead.
Can the AI fix the step for me? Ask it to record the flow again with the new wording; it will take the same steps with a fresh snapshot and save a new script.