Skip to content
NewSee how

Read the Learning page and act on it

Understand collection status, pattern readouts, the control holdout, and the pattern graph, then turn what you find into a campaign change.

  1. Open Learning and choose a scope

    Learning sits in the AI section of the left sidebar, under Knowledge Base. Open it and you get a page titled Learning with a subtitle describing what you are looking at: recorded outcome signals across AI Autopilot campaigns.

    A scope switcher sits at the top right with three options: Workspace, Campaign, and Chat. Workspace is selected by default and aggregates every AI Autopilot campaign in the workspace. Campaign narrows to one campaign. Chat is a different thing entirely and reports what Chat has learned from your approvals, feedback, and prospect replies.

    Keep the scopes straight, because they fill from different sources. Workspace and Campaign fill from real outbound sends and the replies they produce. Chat fills only from your verdicts inside Chat, so a busy Workspace scope tells you nothing about whether Chat has learned anything, and an empty Chat scope is normal in a workspace where nobody rates replies.

    Under the scope switcher are three tabs: Overview, Outcomes, and Graph. They are three views of the same evidence, from summary to detail.

    The same Learning view is also available inside a campaign builder as a tab, which is the faster route when you already have one campaign open and want its numbers rather than the workspace roll-up.

  2. Read the collection status and evidence counters

    The Overview tab opens with a status banner. In the normal state it reads "Signals collected" and explains the situation in two sentences: outcome signals are being recorded, automatic policy changes remain paused, and the status applies to the selected workspace. At the right of the banner is a Last model update timestamp with the note that it is the most recent signal applied.

    Read the paused line carefully, because it is the part people misread. The system is collecting evidence and reporting patterns, and it is not silently changing how your campaigns write. Nothing on this page is being applied to your sending behind your back.

    Under the banner is a row of counters. Retained signals, last 90 days is the volume of evidence held in the current window. Recorded observations is how many observations have been logged. Mature outcome observations is the subset whose reply windows have closed, and it is the number that matters most, because everything else is provisional until it grows. Segments covered tells you how many distinct audience segments the evidence spans. Indexed chunks reports indexed content available to the system.

    A workspace with hundreds of retained signals and zero mature outcome observations is a workspace that has sent plenty and learned nothing yet. That is a timing state, not a fault. It resolves when the reply windows on those sends close.

    A low segments-covered number is worth noticing too. Evidence gathered from one segment describes that segment, so treat the patterns below as advice about the audience you have actually been emailing.

  3. Read the pattern readout in plain language

    Below the counters is the Current pattern readout, introduced with the same caution as the banner: these are evidence-weighted patterns from the selected workspace, and automatic policy changes remain paused until outcome evidence qualifies.

    Two panels sit side by side. What is working holds patterns receiving favourable observed signal. Needs attention holds patterns receiving negative observed signal. Each pattern is a chip written in plain words rather than a metric name, for example a share-an-asset call to action, bullet points, longer copy, opening with a personal signal about the prospect, or asking fewer than one question.

    The two panels are often in direct tension, and that is the useful part. Seeing "longer copy" on the left and "very short copy" on the right is a readable instruction: this audience is not responding to two-line emails. Seeing "open with a personal signal about the prospect" on the left and "open with a workflow or process observation" on the right tells you which opener to write next.

    Do not treat a chip as a rule to hard-code. A pattern on the left is evidence from your own sends in your own segments, and it can flip as the audience changes. The right way to use it is to adjust your campaign brief in the AI Instructions tab and then come back to see whether the chip moved.

    Chips are also the fastest way to spot a campaign brief fighting itself. If your brief demands short copy and this readout has short copy under Needs attention, the brief is the thing to change.

  4. Compare treatment against the control holdout

    Click the Outcomes tab. Two cards sit at the top. Treatment shows the number of prospects assigned to treatment, with a breakdown under it: how many were assigned, how many executed, how many are mature, and how many have an unknown execution state. Control shows the baseline holdout with the same breakdown.

    The holdout is what makes the rest of this tab trustworthy. Without a group that did not receive the learned treatment, a rise in replies could be seasonality, a better list, or luck. The control arm gives the comparison something to sit against.

    When outcome windows have not matured, the page says so directly rather than showing a number. The message reads that lift is not yet available, because assigned sends exist but their outcome windows have not matured. A note under the cards accounts for excluded data, including legacy sends that predate the snapshot and prospects that ended up in mixed arms.

    That exclusion note is worth reading before you argue with the totals. A workspace with thousands of legacy sends will show a much smaller assigned population here, and the gap is by design rather than missing data.

    Under the cards is a Rate by signal chart comparing control and treatment across the outcomes that matter: qualified positive reply, meeting or goal, open, click, unsubscribe, bounce, and out of office. A table below repeats the same signals with treatment, control, and lift columns. Read the negative signals as carefully as the positive ones. A treatment arm with more replies and more unsubscribes is not a clean win.

  5. Explore the pattern graph

    Click the Graph tab for a visual map of the same evidence. Nodes are patterns, grouped by the categories in the legend: campaign, opener, call to action, length and shape, structure, and semantic. Edges are coloured by verdict, with preferred and avoid as the two states.

    The footer explains the interaction and the encoding: you can drag nodes to rearrange, scroll to zoom, and node size and edge width scale with learned weight. Zoom controls and a fit-to-view button sit at the bottom left.

    Size and width are the whole point of this view. A large node with a thick edge is a pattern with real evidence behind it. A small node on a thin edge is a pattern the system has barely seen, and reacting to it is reading noise. The chip lists on the Overview tab do not show that difference, which is the reason to come here when a chip surprises you.

    The campaign nodes at the centre are the anchor. Following the thick edges out from a campaign shows what that campaign's sends have actually taught, and following avoid-coloured edges shows what to stop doing before it costs you more replies.

    Treat the graph as a reading tool rather than a control surface. Nothing here changes your campaigns. Any change you decide on gets made in the AI Instructions tab of the campaign, then shows up here later as new evidence.

  6. Turn the readout into a change

    The page reports. You act. A short loop keeps that honest.

    • Check Mature outcome observations first. If it is zero, read the patterns as early signal and do not rewrite a campaign on them.
    • Pick one chip from Needs attention rather than four. Changing several things at once means the next readout cannot tell you which change worked.
    • Make the change in the campaign's AI Instructions tab, in the explicit instruction fields, then confirm it landed in the effective brief pane.
    • Come back after the send volume has grown and check whether the chip moved between panels.

    Two cautions. First, patterns are drawn from the segments you have emailed, and a low segments-covered count means the advice is narrow. Second, automatic policy changes stay paused, so nothing improves because you read this page. The loop only closes when you edit a brief.

    The Chat scope follows the same logic with a different input. It fills from your approvals, rejections, and reply ratings inside Chat, and it stays empty until you produce those verdicts. If you want that scope to become useful, rate replies as you go rather than waiting for the page to populate itself.

Pro tips

  • Mature observations is the number that counts

    Retained signals and recorded observations can run into the hundreds while mature outcome observations sits at zero. Until that last counter grows, every pattern on the page is provisional.

  • Node size beats chip position

    The Overview chips do not show how much evidence sits behind a pattern. The Graph tab does, through node size and edge width, which is where to check before acting on a surprising chip.

  • Read unsubscribes next to replies

    The Rate by signal chart includes unsubscribe, bounce, and out of office alongside positive replies. A treatment arm winning on replies and losing on unsubscribes is not a clean win.

  • Change one thing per cycle

    Patterns move slowly and the page cannot attribute a shift to a specific edit. Adjusting one instruction per cycle is the only way to learn which change did the work.

Frequently asked questions

Is FirstSales changing my campaigns based on this page?
No. The status banner and the pattern readout both state that automatic policy changes remain paused. Acting on the evidence means editing the campaign's AI Instructions yourself.
Why is lift not showing a number?
Because the assigned sends exist but their outcome windows have not matured. The page holds back the figure rather than publishing a number that would be noise.
What is the control holdout for?
It is a baseline group that did not receive the learned treatment, so a change in reply rate can be compared against something rather than attributed to the treatment by default.
Why are some sends excluded from the totals?
The note under the outcome cards accounts for them: legacy sends that predate the current snapshot and prospects that ended up in mixed arms are left out on purpose.
What does a low segments covered count mean?
The evidence comes from a narrow slice of audience. The patterns are still real, they just describe the segment you have been emailing rather than your market.
How is the Chat scope different from Workspace scope?
Workspace and Campaign scope come from outbound sends and prospect replies. Chat scope comes from your approvals, rejections, and reply ratings inside Chat, and stays empty until you produce those verdicts.
What do node size and edge width mean on the graph?
Both scale with learned weight, as the footer states. Large nodes and thick edges carry more evidence than small nodes and thin edges.
Can I see this for one campaign instead of the whole workspace?
Yes. Switch the scope to Campaign at the top right, or open the Learning tab inside that campaign's builder.

Analytics

Outreach that keeps going after you log off.Leads, emails, follow-ups and replies, handled.

Connect unlimited mailboxes and let AI agents find leads, write each email and follow up. Or send your own templates in manual mode. From $29/mo.

$1 for 3 days · Cancel anytime · No contract

Install with AI

Your coding agent sets up the FirstSales CLI and skills for you.

Prompt for Claude Code

I'm setting up FirstSales (https://firstsales.io), an AI outbound sales platform: campaigns, contacts, a reply inbox, deliverability and lead signals. Help me get my workspace sending real, good emails end to end, from Claude Code.

Read these first so you know how FirstSales works:
- Developer docs and API: https://developer.firstsales.io
- CLI: https://www.npmjs.com/package/@firstsales.io/cli
- Skills for coding agents: https://github.com/firstsalesio/firstsales-skills (read README.md and GUIDE.md)

Setup
1. Install the CLI: npm install -g @firstsales.io/cli
2. Install the FirstSales skills:
   claude plugin marketplace add firstsalesio/firstsales-skills
   claude plugin install firstsales@firstsales
3. Ask me if I already have a FirstSales account. If not, send me to https://app.firstsales.io to sign up.
4. Ask me to create an API key under Settings → API → API Keys, with only the scopes we need.
   Never ask me to paste the key into this chat. Tell me to run this in my own terminal, so the key stays hidden:
   read -rs FS_KEY && firstsales auth login --api-key "$FS_KEY"; unset FS_KEY
5. Run `firstsales whoami` and tell me which workspace and organisation we are connected to.

Then ask me about my business: what I sell, who I sell to (roles, company size, countries), my best customers, and what I use today for outbound (tools, lists, mailboxes). Walk me through:
1. Teaching FirstSales my business: website, offer, proof points and case studies in the knowledge base, so the AI writes like someone who knows the product.
2. Connecting my sending mailboxes and checking my sending domain: SPF, DKIM, DMARC and warm-up. If anything is unhealthy, tell me the exact DNS record to add.
3. Bringing in contacts: import my CSV or CRM export into a clean list, remove duplicates and bad emails, and block domains I must never email (customers, competitors, my own).
4. Building my first campaign: the right list, senders, steps and timing. Write the sequence with me, show me the drafts, and do not launch until I say yes.
5. Setting up Signals if it fits: find people engaging with LinkedIn posts about topics my buyers care about, and turn them into leads.
6. Inviting my team and giving each person the right access.

Once campaigns run, show me how to work the reply inbox from here: which replies need an answer, draft responses in my voice, and move good conversations into deals. If I use Cal.com, connect it so follow-ups can offer my booking link (not the first email).

Throughout, as you learn about my business, proactively suggest and help create:
- Reply templates for the questions and objections my prospects are likely to raise, based on my product, site and codebase if you can see them.
- Campaign ideas and segments worth testing, with the reason for each.

Safety rules:
- Show every change as a dry run (--dry-run) first and wait for my yes. Never send emails, launch campaigns or delete anything without my yes.
- Tell me before anything that spends credits, and how many.

Be opinionated. If the docs give a sensible default, just use it. Don't make me choose between five options.