Overview
Refrain uses accessibility-first targeting to find elements on the page. Selectors are defined using semantic attributes likerole, ariaLabel, and text — not brittle CSS selectors or XPaths.
This approach is resilient to UI changes because accessibility attributes tend to remain stable even when the visual layout or DOM structure changes.
Multi-layered resolution
When a step needs to find an element, Refrain applies multiple resolution strategies in sequence. If one layer fails, the next takes over automatically.Deterministic matching
The first layer is pattern-based matching that runs entirely without AI. It compares the selector’srole, ariaLabel, and text against the page’s accessibility tree to find an exact or partial match.
This is fast (sub-millisecond), free (zero AI tokens), and handles the vast majority of cases.
AI-assisted resolution
When deterministic matching cannot find a confident match, AI analyzes the page snapshot and the expected selector to identify the correct element. This handles cases where labels have changed slightly or the page structure has shifted.Automatic retry
If resolution fails, the executor waits for the DOM to stabilize and retries multiple times. Between retries, it checks whether the page snapshot has changed — if the page is still loading or transitioning, a fresh attempt may succeed.Advanced recovery
When all standard retries are exhausted, advanced recovery mechanisms activate:- Scroll recovery — The element may be off-screen. The executor scrolls and tries again with a fresh snapshot.
- Agent Fallback — AI analyzes the full page to find alternative paths to the target element, such as expanding collapsed sections or dismissing overlays.
- Vision Fallback — A screenshot-based approach that uses visual recognition to locate elements. Effective for canvas, Shadow DOM, iframes, and custom Web Components that don’t appear in the accessibility tree.
Agent Fallback is available on all plans. Selector cache and Vision Fallback require Pro or higher. Self-healing mode requires Team or higher.
Fallback capabilities
Agent Fallback
When standard resolution fails completely, Agent Fallback takes a holistic view of the page. AI evaluates the entire page and selects from multiple recovery strategies:- Try a semantically equivalent alternative element
- Expand collapsed sections or menus to reveal hidden elements
- Dismiss modals or overlays blocking the target
- Use keyboard navigation to reach the element
Vision Fallback
For elements that don’t appear in the accessibility tree — such as those inside<canvas>, Shadow DOM, or iframes — Vision Fallback takes an annotated screenshot and uses visual AI to locate the target element.