Should Your Website Chat Open Itself? Test the Right Thing

A website chat widget A/B test should measure whether its presentation helps visitors take a useful action. Receptionist experiments offer three presentation modes: launcher only, welcome bubble and automatically opened panel. A panel open is not a conversation start. The conversation-start goal requires a genuine visitor-authored submitted message.

An opening panel is visible. A useful conversation is earned. Counting the former as the latter rewards a design for doing exactly what it was programmed to do.

When experiments are enabled in your paid Receptionist workspace, you can compare how your website chat appears. Access depends on your workspace and team permissions. This guide does not report an Entagl conversion study or promise that any one mode performs best.

What changes in a presentation test?

Keep the underlying agent, offer and greeting content the same. Change how the chat appears. Otherwise you may be measuring a stronger offer or different instructions rather than the opening behavior.

Mode	What the visitor sees	The question it helps test
Launcher only	A chat control waiting to be opened	Will visitors start without a prompt?
Welcome bubble	A short welcome near the launcher	Does a small prompt help?
Auto-open	The chat panel opens automatically	Does opening the panel help enough to justify the interruption?

In an experiment, a test can use two or three modes with fixed traffic shares. A keeps the original supported presentation. Fullscreen chat embeds are outside this test. The separate fullscreen voice-call overlay is not the same thing and should retain its existing behavior.

Select desktop, mobile or both before launch. If the original presentation differs between desktop and mobile, use one device group or align the normal settings separately first. Rotating a phone should not reassign the visitor to another test version.

When can a browser join the experiment?

Website experiments require explicit host-site experimentation or analytics consent through a widget-scoped integration. Unknown or denied consent means the original presentation and no experiment tracking. A visitor opening functional chat is not, by itself, permission to enroll them.

The host site's consent system must pass the choice to the widget. A grant after the original presentation has already appeared takes effect on the next eligible load. The panel should not suddenly open or close halfway through a visit because the visitor changed a consent setting.

Revocation stops experimental tracking and attention actions. It should preserve an active functional conversation and typed text, with the original presentation used on future loads. That means some experiment outcomes may remain unknown. The report must show that limitation.

These are the product's consent requirements for this experiment. They are not a claim that the feature alone satisfies every jurisdiction's legal requirements.

Why count browsers that never start a chat?

Because the presentation is supposed to affect whether a chat starts. If you keep only visitors who message, you remove the people for whom the presentation did not lead to that action.

The correct denominator for this design is all enrolled eligible consenting browsers, including those that never chat. One browser is not a unique person across several devices. Do not rename that count “all website visitors” or “all customers.”

Microsoft describes how filtering participants after exposure can produce misleading experiment results, including a case where a bot filter excluded the users most affected by a variation. Its sample-ratio mismatch guidance is a useful check on who disappeared from an analysis. Microsoft Research

Separate assignment, presentation and outcome. A browser can be assigned a mode while an existing suppression rule prevents a welcome bubble from appearing. Keep that participant in the original group. Check the site's suppression rules when interpreting the result. Do not quietly move it into a cleaner-looking dataset.

What should count as the goal?

For conversation start, require the visitor's first genuine submitted message. A generated greeting, an empty chat record, an opened panel or a form submission does not qualify. Form completion can be a separate intake goal when the selected form and source are defined.

You can choose a later supported business event when it can be reliably linked. A booking request is still different from a confirmed booking; confirmation is still different from attendance. Choose the event before you start, then leave its meaning unchanged.

The distinction also matters when chat and voice share a website widget. A call transcript is not a visitor-authored chat message for this goal. Our guide to voice calls from a website widget covers that separate experience.

Could a mode cost credits before a conversation starts?

Yes. Contextual welcome generation can consume credits on page load. A visitor may leave without sending a message after that cost has already occurred.

Receptionist attributes actual welcome charges to the initiating assignment even when there is no conversation. Cache reuse does not create an invented model charge or move an earlier charge to a later visitor. Compare total cost and cost per participant as well as cost per success. If there are no successes, cost per success has no valid numeric value.

A credit threshold is a control for new enrollment, not a promise that already-started work costs nothing afterward. Review the disclosure before launch and inspect the final spend after the outcome collection period.

How do you turn the result into a decision?

Wait for the selected outcome windows and check tracking integrity. A version can have a higher observed rate without enough evidence for a reliable conclusion. A high chat-start rate can also create low-quality conversations that need more staff time.

Keep the result and saved settings in history. Applying a mode is a manual decision, and the review must show the shared widget target affected by that change. An origin filter for enrollment does not create permanent per-origin widget settings.

The human-in-the-loop approach applies here too: evidence helps your team decide; it does not remove accountability for the experience you put on the website.

What happens when the test ends or evidence is missing?

At the planned end, normal routing is the default. You may instead explicitly preselect saved A, B or C where available. That scheduled choice still needs experiment availability, current target and dependency checks; it is not an automatic winner selection. A blocked choice needs attention and must not be reported as applied. Existing participants can continue through their outcome windows.

Unknown outcomes have a strict consequence: even one unresolved unknown can prevent a statistical recommendation for that run. Descriptive results and an explicit user choice remain available. A manually stopped run without a completed planned analysis horizon is not proof of a winner. On a downgrade to Free, experiment access and new enrollment close, and experiment handling is interrupted safely. Ordinary Inbox records remain.

FAQ

Does opening the widget count as a conversion?

Not for the conversation-start goal. That goal requires a genuine visitor-authored submitted message. Panel openings and greetings are separate operational events.

Can visitors switch groups on every page view?

Receptionist keeps an enrolled browser's assignment stable across eligible loads. Resizing or rotating the device does not create a new assignment.

Is more chat always better?

No. Use the primary goal, cost and downstream context together. More conversations may be useful, neutral or burdensome depending on what visitors need and what your team must do afterward.

Does the test automatically send more traffic to the leading mode?

No. It uses fixed ratios. Adaptive traffic and AI Manager/MCP-guided creation are not included.

Bring your current widget settings and consent flow to a Receptionist review. Book a demo to decide which single presentation question is worth testing.

Sources checked 10 October 2026. Examples describe test design, not measured uplift.