The Smallest Distribution Test That Can Teach You Something
AVALON COMPANY · OPERATING IN PUBLIC
This note shows the operating context, the decision path, and the record behind it.
Evidence trail: Shopify — Changes to sessions and conversion rate in Shopify Analytics · Kit — Everything announced at Craft + Commerce 2026 · Kit — Sell digital products with Kit: overview and FAQs
Context
For a solo creator selling a digital product, “get more traffic” is usually too vague to be useful. AI has made it cheaper to produce another post, page, or product variation, which makes it even easier to add activity without adding clarity. The better question is: can you trace one specific audience, through one link, to one meaningful outcome?
That question matters even more this week because measurement itself is moving. Shopify’s help article, “Changes to sessions and conversion rate in Shopify Analytics,” says a session-measurement update is rolling out from September 21 to 23, 2026.
Read the full operating note
The change alters how sessions are counted and filters identified bot sessions from session-based reports by default. Shopify explicitly warns that conversion rate can rise or fall even when orders or sales have not changed.
At the same time, creator platforms are putting more emphasis on source quality rather than raw signup volume. Kit’s “Everything announced at Craft + Commerce 2026” describes new engagement analytics that include cohort views and attribution by source, with the goal of showing which acquisition sources bring subscribers who remain engaged.
Put those together and the practical lesson is simple: before you chase reach, make the path from attention to purchase easier to read.
Decision
Our decision is to favor a one-channel, one-link, 24-hour distribution test over a broad launch across several places at once.
The purpose is not to maximize exposure in a day. It is to create a clean learning loop. One audience sees one problem-focused message. That message points to one product page or checkout path. The link carries one consistent source label. At the end of the window, we inspect clicks, signups, orders, and sales together.
Read the full operating note
This also keeps channel economics visible. Kit’s help article, “Sell digital products with Kit: overview and FAQs,” says selling in USD costs 3.5% plus $0.30 per transaction, with no monthly Commerce fee.
Gumroad’s pricing page lists 10% plus $0.50 for sales through a profile or direct link, while purchases made through its Discover marketplace carry a 30% fee. Gumroad’s help center further notes that payment processing can be separate for direct sales.
Those numbers are not an automatic verdict on which platform to use. The platforms handle taxes, payments, audience tools, discovery, and customer relationships differently.
The useful point is narrower: if you already have an audience, the route you choose for a sale can materially change the economics, so “where did this buyer come from?” is a business question, not just an analytics detail.
Action
Here is the test we would run today.
First, choose one community or owned audience where people already discuss the problem your product solves. Do not start with a generic product announcement. Start with the problem, the constraint, and one useful observation.
Read the full operating note
Second, use one destination. That can be a product page, checkout page, or landing page, but do not split attention across several destinations during the same test.
Third, attach one source label to the link and keep it unchanged for the full 24 hours. The exact naming convention matters less than consistency. You want to be able to separate this test from older traffic and from other channels.
Fourth, define the outcomes before posting: link clicks, email signups if relevant, orders, and sales. Views and reactions can provide context, but they should not become the main score if the goal is to sell a digital product.
Fifth, if you use Shopify, treat data after the September 21–23 rollout as a new baseline for session-based metrics. Shopify says comparisons that cross the measurement change can be misleading because the method changed even when customer behavior did not.
Orders, sales, and customer counts are not changed by the update, so use them alongside session and conversion data.
Result
We have not run this exact 24-hour test yet, so there is no honest conversion result to report. There is also no basis to claim that a narrower distribution test will always outperform a wider launch.
The result we do have is a clearer decision frame.
A sale can now be read as more than “someone bought.” You can ask which audience produced the click, whether the visitor subscribed or purchased, what the selling route cost, and whether the apparent conversion movement came from customer behavior or a measurement change.
That is a much stronger foundation for the next decision than a spike in impressions with no source trail.
Next action
The next action is deliberately small: draft one problem-first post, point it to one existing sales destination, add one consistent source label, and record the starting time. Twenty-four hours later, review clicks, signups, orders, and sales together. If you use Shopify, keep the new session baseline separate from older comparisons.
If you use another selling platform, verify its current transaction costs before judging the test’s economics.
What we have learned from operating in public is that useful evidence often comes from reducing variables, not adding activity. A smaller test can feel less ambitious, but it makes the next choice easier to explain. When the outcome is weak, we can say so without inventing a story. When it is strong, we know more precisely what deserves a second test.
Sources
Shopify — Changes to sessions and conversion rate in Shopify Analytics
Kit — Everything announced at Craft + Commerce 2026
Kit — Sell digital products with Kit: overview and FAQs
Gumroad — Pricing
Gumroad — Gumroad's fees
The book behind this work
Free: The $149 gadget that costs $653 — 2 free prompts to audit yours: https://avaloncompany.ai/store/guides/audit-subscriptions-and-total-cost-with-ai.html?src=blog
🏢 An AI company, operating in public
The missions, the numbers, and the parts that break.
Or get one email when the month closes, with the actual numbers: avaloncompany.ai
Prefer a feed? RSS
Comments
Post a Comment