Message test

Four angles in your buyers’ words run as ads on $100 of your spend, and the winner named by a rule set before any spend.

Kind: Agent task
Price: 18¢ per 4 angles
Your time: 20 minutes
Approval: Asks you first above its seat rule
Deepens: Run an experiment

Experiment.run

You, to your agent

Find out which message gets small landlords to start a trial, put up to $100 into ads, spend up to $5 on the work, and show me the plan first.

curl -X POST "https://api.sb/leaseline.app/experiments/run?mode=try" \
  -H "Authorization: Bearer $SB_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"goal":"Find out which message gets small landlords to start a trial, put up to $100 into ads, spend up to $5 on the work, and show me the plan first.","budget":{"amount":5,"currency":"USD"}}'

npx startup-builder do Experiment.run --try --data '{"goal":"Find out which message gets small landlords to start a trial, put up to $100 into ads, spend up to $5 on the work, and show me the plan first.","budget":{"amount":5,"currency":"USD"}}'

{
  "name": "do",
  "arguments": {
    "mode": "try",
    "code": "return await $.Experiment.run({ goal: 'Find out which message gets small landlords to start a trial, put up to $100 into ads, spend up to $5 on the work, and show me the plan first.', budget: { amount: 5, currency: 'USD' } })"
  }
}

import { $ } from 'startup-builder'

const result = await $.try.Experiment.run({ goal: 'Find out which message gets small landlords to start a trial, put up to $100 into ads, spend up to $5 on the work, and show me the plan first.', budget: { amount: 5, currency: 'USD' } })

Read “small landlords” from the Offer’s ICP: 20 to 400 units, owners and managers. “Start a trial” is the Offer.trialed Event, checked to fire before any spend.
Choose the Venue: Meta, as a Decision. Its buyers are targetable by job and interest, the whole setup runs through the API at $0, and search volume for these terms is thin.
Read buyers’ words: Keyword.search for problem posts, 12 searches, about 2¢.
Write 4 angles and their copy, about 5¢; make 8 images, about 10¢; check claims and ad policy, about 2¢.
Launch on Meta at $25 a day, split evenly; read results daily from your own account, free.
Declare the rules first. Winner: the most trials per dollar, at least 3; ties go to the lower cost per click. Early pause: $15 spent, 0 trials and a click-through under half the leader’s.
Write the readout, about 1¢. Expected: about 25¢ of the $5 budget, plus the $100 of spend, over 4 to 5 days.

Estimate$0.25 of $5.00

Planning is free. You see the plan and its price before anything runs.

Decisions

D2, day 1rewrite v_3’s image and line.Meta rejected it for implying a guaranteed result; a rejected Variant is rewritten at no cost under the remedy.
D3, day 3, at $61 spentpause v_1 and v_4 and move their remaining $39 to v_2 and v_3.Both passed $15 with 0 trials and a click-through under half the leader’s 1.6%: the declared floor.
D4, day 5ask for $50 more, as an Approval.v_2 led 7 to 3, likely but not firm; the agent never raises its own Limit, so it asked, and you refused.

By call

Call	Calls	Cost	Outcomes
`Keyword.search`	1	$0.018	Done
`Content.draft`	3	$0.087	Done
`File.generate`	1	$0.096	Done
`Action.evaluate`	1	$0.02	Done
`Event.capture`	1	$0	Done
`Campaign.launch`	1	$0	Done
`Campaign.update`	1	$0	Done
`Integration.call`	1	$0	Done
`Limit.raise`	1	$0	Done
`Experiment.conclude`	1	$0	Done
Spent	41	$0.28	of $5.00, $4.72 left; the agent's own planning $0.06

Ad spend $100.00, billed by the Venue to your own account. Agent time 14 minutes, over 5 days.

Every call, 12 rows

Loading the rows.

Try mode reads, stores nothing and spends nothingAPI reference Its Nouns

What you get

4 angles · Four angles in your buyers’ words run as ads on $100 of your spend, and the winner named by a rule set before any spend. · Experiment
angles[] · Each Variant: the angle, the buyer’s words it came from, spend, clicks and results.
winner · The Variant the winner rule names, or null when none reaches its minimum: never a forced winner.
decision · ship, kill or iterate, with the reason and how sure: likely under 10 results, firm past it.
readout · Two sentences you can act on: which angle to lead with on the home page and in cold email.

Connect once: Meta Ads. How long: Waits 3 to 7 days for $100 to spend at $20 to $30 a day.

Trigger

It runs when you call it, and from a Demand Budget’s $100 tier under every strategy.

TypeScript

on.PlayMix.compiled((mix, $) => mix.dial.includes('smoke-test') && $.Experiment.run({ offer: 'leaseline-pro', spend: { amount: 100, currency: 'USD' }, venue: 'meta' }))

Steps

Your agent does these steps, one row per step; each step’s check is part of its row and its price.

Step	Verb	Returns	Price	Approval
1. Read buyers’ words in problem posts	`Keyword.search`	The phrases buyers use about the problem, ranked by how often they recur	About 1.9¢	Runs on its own
2. Choose the angles	`Content.draft`	Four angles, each traced to a buyer phrase and the offer’s promise	About 1¢	Runs on its own
3. Write each Variant	`Content.draft`	Headline and primary text per angle, at the site’s lengths	About 4¢	Runs on its own
4. Make two images per Variant	`File.generate`	Eight images at the site’s feed and story sizes	About 10¢	Runs on its own
5. Start the Experiment with its rules	`Experiment.start`	The Experiment, with the winner and pause rules fixed before any spend	Free	Runs on its own
6. Launch the campaign	`Campaign.launch`	An even split on your ad account, the conversion checked first	Free	Asks you first above its seat rule
7. Read results daily, pause by the floor	`Integration.call`	Spend, clicks and results per Variant; a Variant under the floor is paused and its share moved	Free	Runs on its own
8. Conclude by the winner rule	`Experiment.conclude`	`ship`, `kill` or `iterate`, with the reason	Free	Runs on its own
9. Write the readout	`Content.draft`	What won, by how much, how sure, and where to use the angle next	About 1¢	Runs on its own

Calls: Keyword.search, then Content.draft, then File.generate, then Experiment.start, then Campaign.launch, then Integration.call, then Experiment.conclude, then Content.draft.

About 18¢ per 4 angles, each time it runs, capped at $1.00.

Example: give it a goal

Find out which message gets small landlords to start a trial, put up to $100 into ads, spend up to $5 on the work, and show me the plan first.

Run it above shows the plan it showed before it spent anything, and the receipt of every call.

When something fails

Each failure, and what happens next

When	Then
No Variant reaches the winner rule’s minimum	It ends `iterate` with the counts; it never forces a winner.
The site’s ad review rejects a Variant	It is rewritten and reviewed again at no charge, and rejoins the split.
The read is close when `spend` runs out	It opens an approval with the projection, such as “$50 more makes the read firm”; your answer decides, and a refusal stands as `likely`.
You call it twice for the same offer and site	The open Experiment is returned: it is idempotent per offer, site and angles.

Each step retries three times, then falls through to the next route; one item's failure lands in skipped with its reason, never failing the call. Before a step would cross maxCost it stops and returns what it has, with a resume link. Each step's idempotency key is the call's key plus the step and the item: leaseline-experiment-run-1-<step>-<item>.

How it is graded

A second model grades the setup before any spend and the readout after. It passes when every angle traces to a buyer phrase, every Variant claims only what the offer does and promises no result, and the readout names the winner the declared rule names, with its count. A miss is redone at no charge.

Change it

Ask	Compiles to
Run it on LinkedIn to operations titles at $30 a day.	`Experiment.run`
Spend $50 more if the leader is ahead but not firm.	`Limit.raise`
Make an ad kit for the winning angle.	`Ad.make`

TypeScript

import { $, workflow } from 'startup-builder'

export default workflow({
  name: 'experiment-run',
  verb: 'Experiment.run',
  input: { offer: 'string', spend: 'Money', venue: "'meta' | 'google' | 'linkedin' | 'reddit'", angles: 'integer', metric: 'string' },
  maxCost: 1,
  grade: { evaluator: 'setup before spend; readout after', passWhen: 'angles trace to buyer words; no claim past the Offer; the rule names the winner', remedy: 'redo' },
  execute: async (ctx, { offer, spend, venue = 'meta', angles = 4, metric = 'Offer.trialed' }) => {
    const words = await ctx.step('words', () => $.Keyword.search({ keywords: problemOf(offer), venues: ['reddit'], kind: 'problem-question', maxCost: 0.03 }))
    const variants = await ctx.step('variants', () => $.Content.draft({ kind: 'ad-variants', from: words, offer, count: angles, images: 2, maxCost: 0.2 }))
    const exp = await ctx.step('start', () => $.Experiment.start({ variants, metric, winnerRule: 'most per $, min 3', pauseRule: '$15, 0, CTR < half the leader' }))
    await ctx.step('launch', () => $.Campaign.launch({ venue, variants: variants.map((v) => v.$id), spend, daily: { amount: 25, currency: 'USD' }, conversion: metric }))
    await ctx.waitForEvent('Campaign.spent', { where: `campaign = '${exp.campaign}' and total >= ${spend.amount}`, timeout: '10d' })
    return ctx.step('conclude', () => $.Experiment.conclude({ experiment: exp.$id }))    // the declared winner rule decides
  },
})

Extend it with extends: 'Experiment.run' and base(), or compose your own from these steps.

Works with

Plays that order it: Creator ads, Test with a creator, Paid search, Paid social, Positioning tournament, Promote winners, Video ladder.

It serves Run ads, and runs in Paid. It deepens Run an experiment.

For agents

For agents: the guide, its approvals, retries and lookups

Fact	Value
Guide	`Experiment.run`
Guide approval	`limits`
Goal	Find out which message gets small landlords to start a trial, put up to $100 into ads, spend up to $5 on the work, and show me the plan first. It may change counts, the order of steps, soft filters and the route a read takes, and it stops early when the goal is met. It never relaxes your words, raises its own budget, answers its own approval, calls a Play or acts outside your standing approval.
Trigger	`on.PlayMix.compiled((mix, $) => mix.dial.includes('smoke-test') && $.Experiment.run({ offer: 'leaseline-pro', spend: { amount: 100, currency: 'USD' }, venue: 'meta' }))`
Retry	Three times with backoff, then the next route; the same idempotency key never charges, sends or posts twice
Idempotency key	`leaseline-experiment-run-1`
Lookups	A `find:` input asks unless exactly one record fits

Step
1. Read buyers’ words in problem posts
2. Choose the angles
3. Write each Variant
4. Make two images per Variant
5. Start the Experiment with its rules
6. Launch the campaign
7. Read results daily, pause by the floor
8. Conclude by the winner rule
9. Write the readout

The amounts live once, on the hub (approvals).

Who does each step, and how it is checked

Step	Function	Doer	Checker	Assurance	Inputs
1. Read buyers’ words in problem posts	`code`	Code	Code checks each result against its declared type	`measured`	`$0.offer.problem`
2. Choose the angles	`agentic`	The Paid media agent	A second model grades the output	`graded`	`$1.phrases, $0.offer, $0.angles`
3. Write each Variant	`agentic`	The Paid media agent	A second model grades the output	`graded`	`$2.angles, $0.venue`
4. Make two images per Variant	`code`	Code	Adversarial review by a second model: Pass, or the cut claim, such as an image that implies a guaranteed result	`measured`	`$3.variants`
5. Start the Experiment with its rules	`code`	Code	Code reads the result back from where it landed	`measured`	`$4.variants, $0.winnerRule, $0.pauseRule`
6. Launch the campaign	`code`	Code	Code reads the result back from where it landed	`measured`	`$5, $0.spend, $0.venue`
7. Read results daily, pause by the floor	`code`	Code	Code checks each result against its declared type	`measured`	`$6.campaign`
8. Conclude by the winner rule	`code`	Code	Code reads the result back from where it landed	`measured`	`$7, $0.winnerRule`
9. Write the readout	`agentic`	The Paid media agent	A second model grades the output	`graded`	`$8`

The doer is never the checker, and the weakest assurance on the path is the whole call’s.

Under the hood

Under the hood: the Systems, Nouns and Events it touches

Kind	What it touches
Systems	Analytics and experiments, Media, sites and content
Nouns	Keyword, Content, File, Action, Experiment, Campaign, Integration
Events	`Content.drafted`, `File.generated`, `Action.evaluated`, `Experiment.started`, `Integration.called`, `Experiment.concluded`

Reference

Inputs, outputs, its Event and its errors

Name	Type	Default	Meaning
`offer`	`Offer`	Required	The offer the ads sell.
`spend`	`Money`	`{ amount: 100, currency: "USD" }`	Ad money on your account. $100 at $20 to $30 a day reads four angles in 3 to 7 days; the Paid strategy uses $300 to $500.
`venue`	`'meta' \| 'google' \| 'linkedin' \| 'reddit'`	`'meta'`	Where the test runs. Meta is the default for a new product: buyers are targetable by job and interest, and search volume is thin.
`angles`	`integer`	`4`	Angles tested. Each gets an equal share until the early-pause floor fires.
`metric`	`string`	`'Offer.trialed'`	The Event that counts as a result.
`winnerRule`	`string`	`'most results per dollar, at least 3; ties go to the lower cost per click'`	Declared before the first dollar runs, so the winner is a rule, not a reading.
`pauseRule`	`string`	`'$15 spent, 0 results, and a click-through under half the leader’s'`	The early-pause floor: a Variant under it stops and its share moves to the leaders.
`maxCost`	`number`	`1.00`	The most this call spends each time it runs, in USD. It stops before a step would cross it and returns what it has, with a `resume` link.

Emits Experiment.concluded. Refuses with max-cost before it spends, and with the Data calls and Accounts groups (refusals).

Ad test setup

Size up competitors

Decisions

By call

What you getCopy Anchor Link

TriggerCopy Anchor Link

StepsCopy Anchor Link

Example: give it a goalCopy Anchor Link

When something failsCopy Anchor Link

How it is gradedCopy Anchor Link

Change itCopy Anchor Link

Works withCopy Anchor Link

For agentsCopy Anchor Link

Under the hoodCopy Anchor Link

ReferenceCopy Anchor Link

Contents