Signal vs Roark

How Roark works

Roark has three parts: Simulation Testing, Post-call Analysis and Autoimprove. Their home page frames it as Catch, Simulate, Review, Verify.

You connect Retell by pasting an API key. Roark lists your agents, registers its own webhook on Retell, and syncs the last 90 days of calls. From then on it hears every call start and end, transcribes it, and runs metrics.

Metrics are the core of the product. More than 65 system metrics run on every call with no setup, from Roark’s own audio models: pronunciation, emotion, pace, interruptions, talk time, response time. Custom metrics use an LLM judge, a pattern, or a formula built from other metrics. Thresholds turn a value into pass or fail.

An issue in Roark’s API is an alert that fired, deduplicated by fingerprint, with up to 50 evidence calls. It has a severity and a status. It has no location in the agent and no cause field.

Autoimprove takes an agent, a pass or fail metric and a target pass rate. It clones your agent into a shadow copy in your Retell account, writes tests aimed at the failures, and edits the whole configuration (model, voice, transcriber, prompts, tools) until the metric passes on repeated trials. A person promotes the diff.

Where Signal differs

Where the problem is. Signal reads your flow, including every step and transition, and places each issue on the step, tool or transition where it happens. Roark’s issues point at calls, not at a part of the agent.

What a problem is. In Roark, an issue starts as a metric that failed, so you find what you thought to measure. Signal starts from calls that ended badly, reads the trace for the cause, and groups calls by the one change that would fix them. You approve a one-page brief instead of writing metrics.

How sure it is. Every cause in Signal says how it is known: measured by the platform, visible in the trace, inferred by a model, or not established. When nothing in the calls confirms a cause, it says so.

Where proof comes from. Autoimprove proves a fix on simulated calls graded by a metric. Signal measures it on production calls after the new version ships, and says whether the difference is real or that there are not enough calls yet.

Where Roark is stronger. Simulation before launch, languages, audio-native metrics, compliance (SOC 2 Type II, HIPAA BAA), the number of platforms, and an automatic fix loop. Signal does none of these.

What it costs

Roark bills by usage. Post-call transcription is charged per minute of call, and each LLM-judged metric is charged per minute of every call it runs on, so each metric you add raises the bill. The Team plan is a monthly commitment that counts toward usage and lowers the per-minute rates; Enterprise lowers them further. Simulation is billed per minute on top.

The price table on this page compares both at 10,000 three-minute calls a month, with five metrics assumed for Roark. Roark’s price also covers things Signal does not sell, simulation above all, so it compares monitoring, not the whole product.

Signal charges a flat monthly plan with one balance of credits: each minute of a call costs a credit once it has started, and tests and the copilot draw on the same balance. Adding a check does not change the bill, and there is no charge per metric. There are no overage charges either: calls past the balance are stored, and analysed when the month resets or after an upgrade.

Using both

They answer different questions, and they fit together.

1. Signal reads production and names the issue: this step, this cause, this many calls.
2. You write a Roark metric or test flow for exactly that issue, instead of guessing which metrics to write.
3. Roark’s simulations and Autoimprove work on it before release.
4. After you promote the change, Signal checks production: did the issue shrink, and did anything else get worse.

Signal reads over the API and registers no webhook, so it runs beside Roark’s integration without touching it.