Deep InquiryπŸ“Š Statistics & evidence

Priors under contract: why the FDA still prices Bayesian trials in type I error, and what that means for small peptide studies and n-of-1 data

β€œWhy is the frequentist framework the gold standard for FDA clinical trials when Bayes' theorem makes more intuitive sense? How can Bayesian clinical trial design be applied to test peptide efficacy in early-phase trials with smaller sample sizes? How can we crowdsource n=1 biohacker anecdotal reports into a statistically powered aggregate?”

The short answer

Frequentism didn't win at the FDA because anyone proved the p-value answers the right question. It won because a regulator approving drugs from many sponsors needs a guarantee about its own long-run error rate, and Neyman–Pearson error control is exactly that guarantee [2]. The regulation defines "adequate and well-controlled" by what a study has to rule out (spontaneous change, placebo effect, biased observation), not by which school of inference it uses [1]. The FDA now accepts Bayesian designs openly: in devices since 2010 [3], in drugs through the CID paired-meeting program [4], and in the January 2026 draft guidance on Bayesian methodology [5,6]. But it asks for the operating characteristics of Bayesian decision rules, usually their frequentist type I error found by simulation, unless the prior itself can be defended as evidence [5]. For small early-phase peptide studies, the real advantages are adaptive dose-finding (CRM, BOIN) and principled borrowing with robust priors [8,9,12]. Crowdsourced anecdotes can't be fixed by adding more of them. Only randomized, blinded, pre-registered n-of-1 crossovers can be pooled into something informative, using hierarchical models [14–17].

The course notes, corrected

My MATH201 notes framed this as "Bayesians put probabilities on hypotheses; the FDA is stuck in the past." That's wrong on three counts.

  1. The FDA's draft guidance itself names two legitimate Bayesian regimes. In one, Bayesian machinery runs inside a frequentist contract ("calibration to Type I error rate"). In the other, the posterior is read directly because the prior "accurately and comprehensively reflects the external data" [5]. The agency isn't against Bayes. It's against unpriced priors.
  2. The law doesn't mention p-values. 21 CFR 314.126 lists the features of an adequate and well-controlled study: a protocol with pre-specified analysis methods, a valid comparison with a control, methods that minimize bias in assignment and assessment. Its stated purpose is "to distinguish the effect of a drug from other influences, such as spontaneous change in the course of the disease, placebo effect, or biased observation" [1].
  3. "Bayes is more intuitive" misses the regulator's position. The posterior suits a single decision-maker with a coherent prior. The FDA is a counterparty to thousands of sponsors, each with an incentive to pick an optimistic prior.

Why the regulator's loss function is frequentist

The contract

The 1962 Kefauver–Harris amendments made "substantial evidence" of effectiveness, from adequate and well-controlled investigations, a condition of approval. FDA implemented this in 314.126 [1]. Neyman and Pearson's 1933 paper had already set out the machinery a regulator needs. Fix the probability of wrongly rejecting the null, then pick the test that maximizes the probability of correctly rejecting it [2]. The type I error rate Ξ± is a property of the procedure, computed before any data exist, and it holds whatever the sponsor believes.

The drug-side convention, per the draft guidance, is a familywise type I error rate of no more than 0.025 one-sided across primary and key secondary estimands [5]. That number isn't a belief about any one drug. It's a production spec for the approval process.

The loss function, stated as a portfolio problem

This is the argument I think holds up against a quant's skepticism.

Treat the FDA as a market-maker facing a stream of sponsors i = 1…N. Each sponsor brings a candidate with true effect Ξ΄α΅’ and a design with a decision rule Dα΅’. The regulator's realized losses are roughly

  • L = c_FP Β· #{approved with Ξ΄α΅’ ≀ 0} + c_FN Β· #{rejected with Ξ΄α΅’ > 0}.

The regulator doesn't know the cross-sectional distribution of Ξ΄α΅’, and it doesn't trust sponsors' reported priors, because those are chosen adversarially. Under that information structure, the one quantity it can bound per submission, without trusting anyone, is P(Dα΅’ = approve | Ξ΄α΅’ ≀ 0). If every rule in the book satisfies this bound at Ξ±, then the expected number of ineffective approvals is at most Ξ± times the number of ineffective candidates submitted. That holds whatever the sponsors believe and however the portfolio is composed. It's a cross-sectional sum over submissions, not a time-average-versus-ensemble-average argument, so it needs no ergodicity premise.

My inference: this is the real reason type I error is the regulatory currency. It's the only prior-free, sponsor-proof constraint. Power, and the false-discovery rate among approvals, depend on the base rate of good drugs, which is a Bayesian quantity. The regulator cares about these too, but it can't make sponsors guarantee them.

Where the Bayesian criticism lands anyway

The criticism still has force on three points.

  1. Ξ± isn't the quantity patients care about. The draft guidance makes the distinction itself: in a Bayesian framework "the concept of a false positive conclusion … is conditional on a positive conclusion and not, as in the frequentist framework, on a true null hypothesis" [5]. The false-discovery rate among approvals depends on the base rate.
  2. The frequentist contract is brittle under adaptation. Interim looks, arm dropping and response-adaptive randomization all need multiplicity corrections that grow complicated quickly. Berry's case for Bayesian trials is mostly about this. Accumulating results "can be assessed at any time, including continually," with designs modified on the fly [7]. The CDRH device guidance makes the multiplicity point concretely. Twenty independent subgroup tests at 5% give up to a 64% chance of at least one false positive [3].
  3. Throwing away good prior information has a cost. In rare disease, pediatrics and incremental device iterations, ignoring prior data means paying for its information twice, in patients and time. That's why the press release around the 2026 draft emphasizes rare and pediatric indications [6].

The FDA's Bayesian timeline, verified

Year Instrument What it actually says
1962 / 314.126 Kefauver–Harris; "adequate and well-controlled" Defines controls and bias-minimization; is agnostic about inference school [1]
Feb 5, 2010 CDRH/CBER Guidance for the Use of Bayesian Statistics in Medical Device Clinical Trials (final; draft 2006) "We strive for reasonable control of the type I error rate"; extensive simulation may be needed; when prior information is used, "it may be appropriate to control type I error at a less stringent level" [3]
2018 β†’ 2022 CID program (PDUFA VI), continued as the CID Paired Meeting Program under PDUFA VII (FY2023–2027) Priority goes to designs where "analytically derived properties (e.g., type I error) may not be feasible and simulations are necessary"; sponsors submit simulation plans and OCs [4]
Jan 12, 2026 Draft guidance Use of Bayesian Methodology in Clinical Trials of Drug and Biological Products (CDER/CBER) Meets the PDUFA VII commitment (Performance Goals Letter Β§I.L.4.f) [6]; sets out success criteria Pr(d > a) > c, calibration vs direct interpretation, OCs, prior construction, borrowing [5]

I confirmed title and date on fda.gov: the PDF is dated "January 2026," the press release January 12, 2026 [5,6]. As of 2026-10-01 it is still a nonbinding draft; I found no final version.

Devices came first partly because the 2010 document cites the statute's "least burdensome" mandate (FD&C Act Β§513(a)(3)) as a rationale, and because devices iterate, so the previous model's trial is a natural prior [3].

How the 2026 draft actually structures a Bayesian decision

The draft defines a success criterion of the form Pr(d > a | data) > c. Here d is the population treatment effect, a is a clinically meaningful threshold, and c is the posterior probability required [5]. There are then three ways to choose (a, c).

  1. Calibrate to type I error. Set a = 0 for superiority, then pick c so the FWER is no more than 0.025. For conjugate or noninformative setups c can be derived algebraically. For complex adaptive designs it "is instead approximated using clinical trial simulations" [5]. The draft says this suits designs that use Bayes "not to synthesize multiple information sources, but instead to facilitate complex adaptive designs" [5].
  2. Interpret the posterior directly. If the prior is a defensible summary of existing evidence, a posterior probability of effectiveness of 0.98 means a 0.02 posterior probability of ineffectiveness, and the decision can rest on that [5]. This is the regime where borrowing actually buys something.
  3. Decision-theoretic or benefit–risk. Choose thresholds to minimize expected loss. The loss can include "the potential negative consequences of approving an ineffective drug or of not approving an effective drug" [5]. That is the portfolio loss function above, written down explicitly.

What surprised me is that option 3, explicit decision theory, appears in a drug guidance at all, even as the third of three options in a draft.

Worked example: a beta-binomial decision rule and its type I error

I ran a toy single-arm phase II design to make this concrete. The endpoint is binary (responder or not). The null response rate is pβ‚€ = 0.20 (say, the historical rate on standard care), and the hoped-for rate is p₁ = 0.40. Sample size is n = 30. All numbers below are exact binomial and beta calculations I ran myself.

Step 1: prior to posterior

With a Beta(1,1) (uniform) prior on the response rate ΞΈ and a binomial likelihood, observing x responders in n gives a Beta(1 + x, 1 + n βˆ’ x) posterior (conjugacy).

Suppose 11 of 30 respond.

  • Posterior: Beta(12, 20). Posterior mean 12/32 = 0.375.
  • P(ΞΈ > 0.20 | data) = 0.987
  • P(ΞΈ > 0.30 | data) = 0.808
  • P(ΞΈ > 0.40 | data) = 0.375

For integer parameters, P(Beta(a,b) > t) = P(Binomial(a+bβˆ’1, t) ≀ aβˆ’1), so every tail here is a finite binomial sum.

Step 2: the decision rule

Rule: declare success if P(ΞΈ > 0.20 | data) > c = 0.975. With a flat prior, x = 10 gives a posterior probability below 0.975 and x = 11 gives 0.987. So the rule is equivalent to "succeed iff x β‰₯ 11."

Step 3: the frequentist type I error of the Bayesian rule

Type I error is the probability of declaring success when ΞΈ = pβ‚€ exactly:

  • Ξ± = P(X β‰₯ 11 | X ~ Bin(30, 0.20)) = 0.0256
  • Power at ΞΈ = 0.40: P(X β‰₯ 11 | Bin(30, 0.40)) = 0.709

This calculation is the whole idea. You fix the rule from the posterior, then evaluate it under the sampling distribution at the null. The posterior and the type I error are different conditional probabilities: P(ΞΈ | x) versus P(x ∈ rejection region | ΞΈ = pβ‚€). The FDA wants both reported [5].

Two things I noticed:

  1. 0.0256 is just over the 0.025 one-sided convention, so even a flat-prior Bayesian rule needs checking. For comparison, the exact one-sided binomial test at Ξ± ≀ 0.025 needs x β‰₯ 12, and its actual size is 0.0095 because of discreteness. The Bayesian rule sits on the other side of the same discrete step.
  2. With a flat prior the Bayesian rule and the exact test pick nearly the same rejection region, as expected.

Step 4: what an informative prior does to Ξ±

Now suppose earlier data (a pilot study, an analog peptide) suggests ΞΈ β‰ˆ 0.40, and I encode it as Beta(4,6): prior mean 0.40, effective sample size 10.

  • Threshold drops to x β‰₯ 10. Ξ± = 0.061, power = 0.824.

With a stronger Beta(12,18) prior (mean 0.40, effective sample size 30, worth as much as the trial itself):

  • Threshold x β‰₯ 7. Ξ± = 0.393, power = 0.983.

Power and Ξ± inflate together; insisting on Ξ± ≀ 0.025 for a borrowing design throws the borrowing away. That's why the draft separates "calibrated" designs from "direct interpretation" designs, and why Viele et al. show that borrowing raises power and lowers type I error only when the historical data really are similar, with bias and inflated error when they aren't [5,13]. Hobbs et al. state the conflict risk plainly: "when prior and current information conflict, Bayesian methods can lead to higher than expected type I error" [11].

Step 5: robust mixture priors, and where they don't help

Schmidli et al. propose the robust meta-analytic-predictive (MAP) prior. You approximate the MAP prior by a mixture of conjugate components, then add a vague component so that under prior–data conflict the posterior falls back toward the current data [12]. I took 0.8 Β· Beta(12,18) + 0.2 Β· Beta(1,1) and updated the mixture weights by marginal likelihood.

Observed x/30 Weight on informative component P(ΞΈ > 0.2): informative only robust mixture flat
3 0.34 0.812 0.350 0.107
6 0.80 0.963 0.884 0.571
9 0.91 0.996 0.990 0.925

At x = 3, which conflicts badly with a prior centered on 0.40, the mixture puts most weight on the vague component. The posterior P(ΞΈ > 0.2) drops to 0.35, much closer to the flat-prior 0.11 than the informative-only 0.81, which is the intended behavior. But the mixture's success threshold is x β‰₯ 8, with Ξ± = 0.239. At the null (ΞΈ = 0.20, about 6/30) the data aren't far enough from 0.40 to trigger the fallback. My takeaway: robust mixtures protect estimation under large conflicts, but they don't automatically restore type I error control. You still have to simulate the operating characteristics, which is what Schmidli et al. do themselves, reporting "frequentist operating characteristics and posterior summaries" across data scenarios [12].

Two other borrowing methods:

  • Power priors raise the historical likelihood to a power aβ‚€ between 0 and 1 [10]. The draft notes a fixed aβ‚€ is "static" borrowing and "may have worse operating characteristics than dynamic approaches" [5].
  • Commensurate priors let the data decide how much to borrow, through a parameter that measures how commensurate the historical and current data are [11].

Step 6: sequential looks, with type I error by exact enumeration

Now add interim analyses at n = 10, 20, 30, each applying the same rule (stop for efficacy if P(ΞΈ > 0.2 | data) > 0.975, flat prior). The stopping boundaries are x β‰₯ 5, 8 and 11.

  • Exact type I error (dynamic programming over the binomial increments): 0.062, against 0.026 for the single look. Monte Carlo with 20,000 trials gave 0.060.
  • Raising c to 0.99 moves the boundaries to x β‰₯ 6, 9, 12, giving Ξ± = 0.019 and power 0.611.

This is the frequentist complaint about Bayesian adaptivity in miniature. The likelihood principle says the posterior doesn't care how often you looked. The procedure's error rate does. The draft's answer is "success criteria should be specified for each decision point" and, in calibrated designs, "clinical trial simulations may be required to ensure control of FWER across all endpoints" [5]. For real designs with covariates, dropout and adaptive randomization, the exact recursion becomes Monte Carlo over many scenarios.

                 THE BAYESIAN-DESIGN CALIBRATION LOOP
 β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”     β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”     β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
 β”‚ PRIOR            β”‚     β”‚ LIKELIHOOD       β”‚     β”‚ DECISION RULE        β”‚
 β”‚ BETA(A,B), MAP,  │────▢│ BIN(N, THETA)    │────▢│ PR(D > A | DATA) > C β”‚
 β”‚ ROBUST MIXTURE   β”‚     β”‚ + INTERIM LOOKS  β”‚     β”‚ AT EACH LOOK K       β”‚
 β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜     β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜     β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
          β–²                                                   β”‚
          β”‚                                                   β–Ό
 β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”     β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
 β”‚ ADJUST C, A,     β”‚     β”‚ OPERATING-CHARACTERISTIC SIMULATION            β”‚
 β”‚ PRIOR WEIGHT,    │◀────│ SIMULATE 10^4-10^6 TRIALS UNDER:               β”‚
 β”‚ LOOK SCHEDULE    β”‚ NO  β”‚  THETA = P0 (NULL)      -> TYPE I ERROR        β”‚
 β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜     β”‚  THETA = P1 (ALT)       -> POWER               β”‚
                          β”‚  PRIOR-DATA CONFLICT    -> ALPHA INFLATION     β”‚
                          β”‚  ALL SCENARIOS          -> E[N], BIAS, MSE     β”‚
                          β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                                 β”‚ OCS ACCEPTABLE?
                                                 β–Ό YES
 β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
 β”‚ PRE-SPECIFY IN PROTOCOL + SAP  ──▢  CID PAIRED MEETING / FDA REVIEW       β”‚
 β”‚ EXAMPLE (N=30, P0=.20): 1 LOOK C=.975 -> ALPHA .026 | 3 LOOKS -> .062     β”‚
 β”‚                         3 LOOKS C=.99  -> ALPHA .019, POWER .61           β”‚
 β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Early-phase peptide studies: where Bayes actually earns its keep

For a small early-phase program, a novel peptide with maybe 20–40 subjects across dose levels, the strongest Bayesian case is in dose-finding, not confirmatory efficacy.

  1. CRM. O'Quigley, Pepe and Fisher's continual reassessment method targets "a dose with a given targeted toxicity level," concentrating "experimentation at that which all current available evidence indicates to be the best estimate of this level," using a simple working model that only needs to be locally right [8]. Each cohort updates the dose–toxicity posterior, and the next cohort goes to the current best estimate.
  2. BOIN. Liu and Yuan's Bayesian optimal interval design compares the observed toxicity rate at the current dose with two precomputed boundaries, λ₁ and Ξ»β‚‚. Escalate if pΜ‚ ≀ λ₁, de-escalate if pΜ‚ β‰₯ Ξ»β‚‚, otherwise stay [9]. The boundaries minimize a local decision-error rate. Their Theorem 1 shows they are the points where the posterior of "below MTD" or "above MTD" overtakes "at MTD" [9]. For a target toxicity Ο† = 0.30 with their default φ₁ = 0.6Ο† and Ο†β‚‚ = 1.4Ο†, the boundaries are λ₁ = 0.236 and Ξ»β‚‚ = 0.358 [9]. The design fits on a lookup table, operationally "similar to the traditional '3+3' design" [9].
  3. Borrowing for the control arm. If a small randomized study can borrow from well-characterized historical controls through a robust MAP prior, more subjects can go to the active arm. Schmidli et al. note that randomization "can then be changed in case of prior-data conflict" [12]. This is the setting where the FDA's "augmenting a randomized concurrent control" use case applies [5].

A correction to my own earlier draft: the draft guidance's dose-finding section is explicitly about oncology, where the MTD paradigm comes from [5]. For peptides whose toxicity curves are flat over the active range (much of the metabolic and tissue-repair space), "maximum tolerated dose" is often the wrong target. The draft itself points toward designs that optimize a dose rather than find the MTD [5]. A BOIN-style design built around an efficacy biomarker plus a tolerability constraint fits better, but I haven't verified a specific peptide precedent, so treat that as my inference.

What Bayes can't do is turn 30 subjects into confirmatory efficacy evidence: at n = 30, any rule with Ξ± ≀ 0.025 has power around 0.6–0.7 against a doubled response rate, and borrowing raises Ξ± in step.

Crowdsourced n=1 reports: why adding anecdotes doesn't converge

Why the aggregate fails

Each mechanism below creates bias, not variance, so a larger N just narrows the interval around the wrong number.

  1. Selection at entry, or regression to the mean. People start an experimental compound when symptoms peak. The effect "becomes more noticeable with increasing measurement error and when follow-up measurements are only examined on a sub-sample selected using a baseline value" [18]. Every forum before/after report is a sample selected on baseline.
  2. Placebo and expectation effects on subjective endpoints. HrΓ³bjartsson and GΓΈtzsche's meta-analysis of 130 trials comparing placebo with no treatment found no significant placebo effect on binary or objective outcomes. They did find a significant effect on continuous subjective outcomes (SMD βˆ’0.36) and on pain (about 6.5 mm on a 100 mm VAS) [19]. Self-reported "felt better" is exactly the endpoint class where the effect shows up. On objective endpoints the effect largely disappears, which is an argument for objective endpoints.
  3. Reporting and survivorship bias. The denominator is unknown. Non-responders quietly stop, and dramatic responders write long posts. No model recovers an unobserved selection mechanism without untestable assumptions.
  4. Unblinded outcome assessment. The person reporting knows what they took, paid for it, and often has a stake in the community narrative. That's the "biased observation" 314.126 exists to rule out [1].
  5. Unverified exposure. In the gray market, "took peptide X" isn't a fixed intervention. Identity, content and degradation vary by vial, which adds an exposure-misclassification term on top of everything above. (My inference.)

The minimal structure that makes n=1 data informative

The n-of-1 design was formalized for exactly the case where a clinician wants to know whether this patient responds. Guyatt et al. ran "double-blind randomized trials in which a single patient undergoes a series of pairs of treatments, consisting of one active and one placebo … per pair, with the order determined by random allocation," with predefined targets, and continued "until efficacy is established or disproved" [14]. The AHRQ user guide (Kravitz and Duan, eds., 2014) sets out the design and implementation considerations in full [17].

Minimum viable structure, as I'd specify it:

  1. Pre-registration of the participant, protocol, endpoint and analysis before the first period. This fixes the denominator and removes outcome-switching.
  2. Randomized multiple crossover (e.g., ABAB/BAAB blocks) with washout. Each block is a within-person comparison, which removes between-person confounding and, through randomization, the regression-to-the-mean artifact.
  3. Blinded matched placebo. Without it, item 2 in the failure list survives untouched.
  4. Objective or at least standardized endpoints, measured the same way in every period.
  5. Central data capture that includes stopped and null trials.

How the aggregate is modeled

Given structured series, the hierarchical model is natural. Zucker et al. (1997) built a hierarchical Bayesian random-effects model in which each patient's effect estimate is "an adjustment between the population estimate and the individual's observed results," weighted by within- versus between-patient heterogeneity. They demonstrated it on 23 amitriptyline-vs-placebo n-of-1 trials in fibromyalgia [15]. In outline:

  • y_{ijk} = ΞΌ_i + Ξ΄_i Β· T_{ijk} + Ξ΅_{ijk}, with Ξ΅ ~ N(0, σ²_w) for within-patient noise
  • Ξ΄_i ~ N(Ξ”, τ²), so individual effects shrink toward the population effect Ξ”

The 2010 follow-up compared models on real series and found Bayesian hierarchical models "improved precision" but were "highly sensitive to within-patient variance priors." With few patients, between-patient variation "will be hard to explain with covariates" [16]. The power arithmetic follows from the structure. Var(Ξ”Μ‚) β‰ˆ τ²/m + σ²_w/(mΒ·k) for m participants each contributing k paired comparisons. Adding crossovers per person only reduces the second term. Learning whether responses really are heterogeneous (Ο„) needs more people [16]. That's the precise sense in which an n-of-1 network can be "statistically powered." Unstructured anecdotes have no σ²_w estimate and no randomized contrast, so they don't enter this model at all.

What this means for Peptide Partners

PP sells research-use-only peptides. Nothing here is guidance on human use, and PP shouldn't run, sponsor or promote consumer self-experimentation programs. The implications are about how PP evaluates evidence and how it would position itself for a possible future regulated pathway, such as physician dispensing through a partner like Pirsek, which would sit inside whatever compounding, IND and IRB framework applies.

  1. Grade every efficacy claim by design, not volume. When PP reviews literature for a compound, the first filter is 314.126's list: was there a concurrent control, randomization, blinding, a pre-specified endpoint [1]? A hundred forum testimonials rank below one small randomized crossover.
  2. Ask for the operating characteristics, not just the posterior. If a paper or a partner reports "94% posterior probability of benefit," the next questions are: under what prior, what's the rule's type I error, and how many looks? The worked example shows a 0.97 posterior can sit on a rule with Ξ± β‰ˆ 0.06–0.39, depending on borrowing and looks.
  3. Product identity is a variance component. In any future physician-supervised study, between-batch variation in content or purity adds to σ²_w and τ², and it can bias Ξ” if it correlates with time or site. The in-house lab's batch-release data (HPLC purity, peptide content by an orthogonal method, impurity profile) are exactly what an investigator would need to argue the "treatment" is one fixed intervention. A CoA should say what was measured, by what method, with what uncertainty. It shouldn't imply biological efficacy.
  4. Blinding is a chemistry problem. A blinded placebo-controlled crossover needs a matched placebo: same lyophilized cake appearance, same reconstitution behavior, same vial and label. Whether a matched vehicle-only lyophilizate can be made indistinguishable is a formulation and analytical question for Larissa's lab, and it's the first technical gate before any regulated pathway could run n-of-1 or crossover studies.
  5. Stability must cover the trial period. In a crossover with several periods, the product has to stay within specification for the whole series. That means real-time stability data covering the full in-use window. It doesn't mean extrapolated Arrhenius shelf-life alone.
  6. Historical data PP generates could become priors. Well-documented, method-locked analytical data across lots (purity distributions, degradation kinetics) are the kind of exchangeable historical data a MAP prior can use, for CMC decisions as well as clinical ones. Building them now, with controlled methods, gives PP real value later.

Questions to bring to the lab

  1. Can we make a vehicle-only lyophilizate that matches active product in cake appearance, reconstitution time and solution clarity closely enough to keep a blind? What measurements would prove it's indistinguishable?
  2. What is our lot-to-lot standard deviation in peptide content, as opposed to HPLC area-% purity? Could we report it in a form an investigator could plug into a variance-components model?
  3. If a future physician-supervised crossover ran over 8–12 weeks, what real-time stability data would we need to state that every period received in-specification material, and do we have them?
  4. Our method-transfer and lot-release data are themselves "historical controls." Would we be comfortable defending them as an exchangeable prior, and what would break exchangeability (method changes, new supplier, new column)?
  5. Which analytical failures would turn a "null" clinical result into an uninterpretable one: degradation, wrong counter-ion assay, aggregation? Can we rank them by how much they would bias an effect estimate?
  6. If we were asked to support a CID-style simulation plan, which product parameters would we need to give the statistician as distributions rather than point values?

What I still don't know

  • How the 2026 draft is being used in practice. It's still a draft as of 2026-10-01. I haven't found data on how often review divisions accept "direct interpretation" Bayesian primary analyses outside the CID program, or how docket comments may reshape the final version.
  • Whether Ξ± = 0.025 will soften for borrowing designs in drugs the way the 2010 device guidance allowed "less stringent" control with favorable prior information [3]. The drug draft describes the alternatives without committing to numbers.
  • Peptide-specific precedent. I couldn't verify any peptide program that used BOIN or robust MAP borrowing in a way I could cite. My suggestion to design around an efficacy biomarker plus a tolerability constraint is inference, not documented practice.
  • The scale of n-of-1 networks. Zucker et al.'s sensitivity to variance priors [16] suggests small n-of-1 networks give fragile population estimates. I don't know of good empirical evidence on how many participants and crossovers are needed before Ο„ is estimable in practice for subjective endpoints.
  • Power prior details. I relied on the draft's description of power-prior behavior [5]; I didn't read the full Ibrahim–Chen text [10].

References

  1. Code of Federal Regulations. 21 CFR Β§314.126 β€” Adequate and well-controlled studies. eCFR. https://www.ecfr.gov/current/title-21/chapter-I/subchapter-D/part-314/subpart-D/section-314.126 (accessed 2026-10-01)
  2. Neyman J, Pearson ES. IX. On the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society of London. Series A. 1933;231:289-337. doi:10.1098/rsta.1933.0009
  3. U.S. Food and Drug Administration (CDRH/CBER). Guidance for the Use of Bayesian Statistics in Medical Device Clinical Trials. Guidance for Industry and FDA Staff. February 5, 2010. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/guidance-use-bayesian-statistics-medical-device-clinical-trials
  4. U.S. Food and Drug Administration (CDER/CBER). Complex Innovative Trial Design Meeting Program (CID Paired Meeting Program, PDUFA VII). Web page, updated February 13, 2026. https://www.fda.gov/drugs/development-resources/complex-innovative-trial-design-meeting-program
  5. U.S. Food and Drug Administration (CDER/CBER). Use of Bayesian Methodology in Clinical Trials of Drug and Biological Products. Draft Guidance for Industry. January 2026. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/use-bayesian-methodology-clinical-trials-drug-and-biological-products
  6. U.S. Food and Drug Administration. FDA Issues Guidance on Modernizing Statistical Methods for Clinical Trials. Press release, January 12, 2026. https://www.fda.gov/news-events/press-announcements/fda-issues-guidance-modernizing-statistical-methods-clinical-trials
  7. Berry DA. Bayesian clinical trials. Nature Reviews Drug Discovery. 2006;5(1):27-36. doi:10.1038/nrd1927
  8. O'Quigley J, Pepe M, Fisher L. Continual Reassessment Method: A Practical Design for Phase 1 Clinical Trials in Cancer. Biometrics. 1990;46(1):33-48. doi:10.2307/2531628
  9. Liu S, Yuan Y. Bayesian optimal interval designs for phase I clinical trials. Journal of the Royal Statistical Society: Series C (Applied Statistics). 2015;64(3):507-523. doi:10.1111/rssc.12089
  10. Ibrahim JG, Chen MH. Power prior distributions for regression models. Statistical Science. 2000;15(1). doi:10.1214/ss/1009212673
  11. Hobbs BP, Carlin BP, Mandrekar SJ, Sargent DJ. Hierarchical Commensurate and Power Prior Models for Adaptive Incorporation of Historical Information in Clinical Trials. Biometrics. 2011;67(3):1047-1056. doi:10.1111/j.1541-0420.2011.01564.x
  12. Schmidli H, Gsteiger S, Roychoudhury S, O'Hagan A, Spiegelhalter D, Neuenschwander B. Robust meta-analytic-predictive priors in clinical trials with historical control information. Biometrics. 2014;70(4):1023-1032. doi:10.1111/biom.12242
  13. Viele K, Berry S, Neuenschwander B, et al. Use of historical control data for assessing treatment effects in clinical trials. Pharmaceutical Statistics. 2014;13(1):41-54. doi:10.1002/pst.1589
  14. Guyatt G, Sackett D, Taylor DW, Chong J, Roberts R, Pugsley S. Determining optimal therapy β€” randomized trials in individual patients. New England Journal of Medicine. 1986;314(14):889-892. doi:10.1056/NEJM198604033141406
  15. Zucker DR, Schmid CH, McIntosh MW, D'Agostino RB, Selker HP, Lau J. Combining single patient (N-of-1) trials to estimate population treatment effects and to evaluate individual patient responses to treatment. Journal of Clinical Epidemiology. 1997;50(4):401-410. doi:10.1016/s0895-4356(96)00429-5
  16. Zucker DR, Ruthazer R, Schmid CH. Individual (N-of-1) trials can be combined to give population comparative treatment effect estimates: methodologic considerations. Journal of Clinical Epidemiology. 2010;63(12):1312-1323. doi:10.1016/j.jclinepi.2010.04.020
  17. Kravitz RL, Duan N, eds; DEcIDE Methods Center N-of-1 Guidance Panel. Design and Implementation of N-of-1 Trials: A User's Guide. AHRQ Publication No. 13(14)-EHC122-EF. Rockville, MD: Agency for Healthcare Research and Quality; February 2014. https://effectivehealthcare.ahrq.gov/products/n-1-trials/research-2014-5
  18. Barnett AG, van der Pols JC, Dobson AJ. Regression to the mean: what it is and how to deal with it. International Journal of Epidemiology. 2005;34(1):215-220. doi:10.1093/ije/dyh299
  19. HrΓ³bjartsson A, GΓΈtzsche PC. Is the placebo powerless? An analysis of clinical trials comparing placebo with no treatment. New England Journal of Medicine. 2001;344(21):1594-1602. doi:10.1056/NEJM200105243442106