How performance is measured

Promotion performance is measured from Dutchie sales, never estimated. This page explains where each number comes from and what it can and cannot claim.

Attribution through the discount

Every discounted line in a Dutchie order records which discounts were applied to it. A nightly job (3:30 AM Eastern, rebuilding the last 3 days so that returns and voids booked against an earlier date are caught) folds those lines into one row per promotion per day. Every discount on a line is counted, including one stacked in second place; a line carrying two discounts gives each an equal share of what it earned, so the same dollar is never counted twice.

What a sold line earned is counted the same way in Pricing and in Promo performance, from one shared definition: what the register collected for the line, its list amount less every discount on it. Returned lines and voided orders never count. A $20 item rung up with $2 off earned $18.

Dutchie reports some discounts with no promotion behind them, and those are two different things wearing one label. Each line's own discount reason decides which: a reason of LOYALTY APPLIED is counted as loyalty, and anything else - a manager override, a price override at the register - is counted as manual. Both are kept apart from promotions and shown as their own share rather than hidden. A promotion with no Dutchie discount behind it has nothing to measure.

Given away

Given away is the shelf value of what the promotion gave away on its own lines: what you did not ring up, not what the goods cost. A $10 pre-roll given free shows as $10.

Baskets

The discounted line is not the promotion's whole effect. A "buy a 7g, get a free roll" promotion puts the reward on a discounted line and the purchase that earned it on a full-price line. So the headline figure reads the whole basket: revenue before tax, cost of goods and margin across every line of every basket the promotion appeared in. A basket carrying two promotions counts toward both; that makes the figure right for ranking promotions against each other and wrong for adding up to store margin.

The margin rate is basket margin over basket revenue. A rate is coloured as a problem only under the promotion's margin floor and only with enough baskets (default 50); with fewer it is shown greyed as low volume, because a handful of baskets throws extreme rates.

Structural discounts

Some discounts are not campaigns. A discount that redeemed on at least 80% of the days the store traded in the last 90, has no end date (or runs longer than 90 days) and is still active is structural: it is part of the price customers pay. So is any active staff discount, however rarely it is used — and staff means the audience, not the name: a discount restricted to your staff customer group is a staff discount whatever it is called. Loyalty and manual discounts are structural by definition. Each night's metrics run re-decides this, so a discount that stops qualifying stops being tagged.

Structural discounts are shown and tagged on the list but not counted under Below the floor: a price below the floor is a pricing question, not a promotion to review. Pricing counts sales under a structural discount as ordinary sales at the price paid, and leaves out only sales under a campaign.

Which margin floor

There is one margin floor, set in Pricing, and both the pricing engine and this report use it. A promotion covers products that may sit in different families, each with its own floor, so a promotion is held to the strictest floor among its products. Its products are the ones the discount declares plus the ones that sold under it in the last 90 days; a promotion with neither is held to the store's floor. Until a floor is set in Pricing there is none, and nothing stands in: no promotion is counted below the floor, and every label that would name one reads No floor set with a link to Pricing > Setup.

Baseline and incremental

The detail screen compares the promotion with what the same products did before it ran: the same number of days immediately before, leaving out any day the same discount also ran. If that leaves less than about 60% of the promotion's own length, no baseline is reported. Incremental revenue, margin versus baseline, units delta and discount ROI (incremental gross profit per $1 discounted) all rest on that comparison.

The comparison is made over equal shelf time. Days come out of the baseline when the same discount ran on them, and a product that was out of stock had no chance to sell on those days either, so the baseline is scaled to hold as many selling days as the promotion's own window. Its basis line says so where the two differ: "scaled to equal shelf time (x1.12)". Without it, a shorter or emptier baseline reads as a quiet period the promotion then beat.

The verdict

A promotion is measured, and then it is judged. The judgement is one of six words, decided overnight over the last 90 days ending yesterday and stored on the discount, and worked out again live whenever its own page is opened for a different window.

The control

A promotion has no hold-out. It applies to everyone who qualifies, so there is no half of the shop that did not get it. The stand-in is the rest of the shelf: every other product in stock in the same analysis buckets as the promotion's products, over the promotion's window and over the same comparable period before it. If those neighbours grew 20% while the promotion's products grew 20%, the promotion bought nothing it would not have had.

Only clean sales count on the control side - a neighbour under its own campaign is not that neighbour answering its list price - and a product that is out of stock is left out, because it cannot respond to anything. That applies over the whole window as well as today: a neighbour that was empty for more than 30% of either window takes no part on either side, and what the rest did is counted per day it was on the shelf. The promotion's own products are never their own control.

Gross profit beyond its shelf is what is left after that drift is taken off: the promotion's own gain, less what the shelf would have given it anyway. Where the buckets hold no other stocked product, or nothing traded in the comparable period, there is no control and the page says so rather than quietly reporting an uncontrolled figure as a controlled one.

This is a check at the level of a whole bucket in one store. It is not a model of which product steals from which, and it cannot say that a particular neighbour lost a particular sale.

The rest of the basket

The shelf is the promotion's own family. A flower deal can also change what its shoppers buy of everything else: more pre-rolls with the eighth (a halo), or fewer, because the money went on the deal (a drain). That is measured separately, on the shoppers rather than the baskets.

Every customer who bought one of its products on a day it ran, and also shopped in the comparable period before it, is compared with customers who shopped in both periods, did not buy it, and made as many trips while it ran. For each, what they bought of every family the promotion's products are not in, per day, now against before. The difference is the promotion's effect on the rest of the basket. Matching on trips matters: somebody who came twice as often was twice as likely to pick up the deal and bought more of everything, and unmatched that read as a halo nobody caused. The lines of the promoted products themselves never count - a free pre-roll under "free pre-roll with an eighth" is the promotion - and neither do other products in the same family, which are the shelf's question.

Comparing baskets instead of shoppers was tried and is wrong here: a deal fills its baskets with people who came for it and bought nothing else, and their thinner baskets read as a drain on everything.

It is reported with an 80% range, and a range across nil is no finding. Some promotions are not read at all, and say why: a perk for one kind of customer (first-time, or one customer group), because the perk picked its shoppers; a gift with purchase whose free item alone carries the discount, because the purchase that earned it - the $30, the 7g - picked them; a discount that came off whatever was in the basket; and one with no comparable period. Shoppers new to the store are counted and never valued, because they have no "before", so the figure is a floor.

Every Sunday at 2:45 AM Eastern, each promotion's first six weeks are read this way against the six before, and pooled: what a promotion on each family does to the rest of the basket per unit it sells, steadied toward the store's overall figure where a family has few promotions. It is then tested on promotions it had not seen, each predicted only from the ones that finished before it started. A promotion's own reading on its page is steadied by that pooled figure.

Gross profit beyond its shelf, whole basket is the promotion's own figure plus the rest of the basket. It decides Giving it away only while the install has it switched on and the latest Sunday test predicted held-out promotions better than assuming no effect at all; a family whose own test, on at least five promotions, said otherwise is left out. Until then it is shown beside the own figure and counted nowhere, and the card says so.

The six words

Read in order; the first that fits is the answer.

Before any of them: a promotion switched on in Dutchie, live for at least 14 days of the window and not used once in it, is Dead. With no sale it has nothing to compare, and it used to read as "can't judge" for that.

1. Can't judge - under the basket floor, no comparable period before it, fewer than 7 days of the window, a third or more of its products off the shelf for most of it, or something else moved at the same time: a shelf price change or another campaign on 30% or more of its products. The promotion's page names which. Two kinds of "no comparable period" are named for what they are, because waiting will not fix either: "It also ran on N of the M days before this window" for a promotion that runs all the time, and "Its products did not sell in the M days before this window" for one on new products. A promotion half of whose products were in the stockroom is not a promotion that underperformed, and calling it a dud would retire a discount for a purchasing problem: its page reads "N of its M products were off the shelf for most of the window with nothing like them from it in stock, so its figures are about supply rather than the discount", and Its products off the shelf with nothing like them in stock is a line of the working beneath. A product counts as off the shelf only when nothing of the same size and kind under the same promotion was in stock either: strains rotate, and "any $13 eighth" is still on offer when one eighth sells through and another is on the shelf.
2. Dead - live, and no redemption on the last 14 trading days; or live for 30 days or more and still never reaching the basket floor, whether or not there is a period before it to compare. That second shape is the one case where too few baskets is a verdict rather than a shrug: a month is long enough for a promotion nobody takes to have said so.
3. Giving it away - gross profit beyond its shelf is nil or below - for the whole basket, where the rest of the basket is counted - or its basket margin rate is under its floor.
4. Carrying passengers - it gains overall, but products inside it lose, each on at least 10 baskets, together worth at least a quarter of what the winners made. Those products are named. A basket or bundle deal is never read this way: what one item in a bundle realises depends on the rest of the basket.
5. Make it the price - structural, not staff, otherwise working, and eligible to be drafted as a price event. It is not a campaign; it is what customers pay. See Make a discount the price.
6. Working - the rest.

Every threshold above is a judgement rather than a derivation, and each is the threshold of exactly one rule, so an administrator can move one in the install's configuration without disturbing the others.

The list carries the word alone. Every number behind it is on the promotion's own page, each line saying which rule it fed.

Where a deal's shoppers came from

A deal can earn because it brings in buying that would not have happened, or because it moves shoppers from the store's other sizes and deals. Every Sunday at 3:45 AM Eastern each automatic deal is read at the shopper. The shoppers who first took it in a window of about five months, ending 60 days before the last day of sales, and who had bought the deal's family in the 60 days before, are compared with shoppers of the family who never took it. Each of those is read around one of their own days in the same window that also has a purchase of the family in the 60 days before it, picked the same way every week. For each shopper the 60 days after are set against the 60 before; the figure is the average change of those who took it less the average change of those who did not, with an 80% range from the spread of both groups. First means first ever: the whole order history is read for it.

A free-gift deal is read on the family of the purchase that earns the gift, found from the baskets it applied to. What the gifts cost is its own line, and so is what the shoppers then paid for of the gift's family, because a free pre-roll can stand in for one they would have bought. Staff and manual discounts are not read.

Shoppers choose to take a deal, and they already shop differently: the page says how often each group came before. It is a reading, not a controlled test.

Which deals overlap

Dutchie gives a shopper one of two deals that do not stack, so a second deal on the same purchase is one nobody needs. Two live deals do the same job when they are for the same audience and make the same offer (the same kind of reward, amount and quantity) on the same products - the smaller one's products at least 80% inside the larger's - or, for a free gift, give the same gift on the same purchase. Of the deals doing one job, the one kept has the higher gross profit a shopper where both have a figure, else more baskets in the last 30 days; a deal that has not sold in 30 days is never kept. The same products for a price at different quantities are a price ladder and are never flagged. Products are what the deal declares plus what sold under it in the last 90 days.

Funding

Dutchie flags a discount as brand-funded but does not say what share the vendor covered, so the whole spend is shown under the vendor with a note to treat it that way only if it matches your agreement. Everything else is self-funded, out of your own margin.

Creative effectiveness

Screen plays from the signage players are joined to redemptions on the days a creative was published. There is no control group: nobody was withheld a creative, and customers check out anywhere. It is correlation, and the screen says so.

Where you see this

• Promo performance
• Promotion performance detail