📖 Core Concepts
Core Concepts: Research Methods, Scientific Writing, and Ethics
1. What This Subject Is Really About
Every course you have taken so far taught you what is known: the Michaelis–Menten equation, the mechanism of native chemical ligation, the thermodynamics of protein folding. This course is about something earlier and more fragile: how a claim becomes knowledge in the first place, and how that process can fail.
The usual picture is that science is a body of facts. A more accurate picture is a social machine for error correction. Individual scientists are biased, hopeful, underfunded, and short of time. The institutions around them are designed to catch the errors those pressures produce. Those institutions include controls, statistics, peer review, authorship norms, ethics boards, and replication. Each part of this course examines one component of that machine:
- Reading and evaluating papers is quality control on incoming information.
- Experimental design and statistics are quality control on your own claims before you make them.
- Writing, figures, and presentations transmit claims faithfully, so that others can check them.
- Ethics, authorship, and reproducibility keep the machine itself honest.
- Grants, intellectual property, and careers are the economic environment the machine runs in. Those incentives explain many of its failure modes.
Think of a scientific paper the way you think of a protein crystal structure deposited in the PDB. It is a model, refined against data, carrying stated uncertainties. It can be overfit, and someone else should be able to re-refine it from the deposited data. Its value lies in how well it can be checked, not in how impressive it looks. This course teaches you to read the "R-factors" of the literature, and to produce work whose R-factors you would be willing to show.
"It can be proven that most claimed research findings are false." [Ioannidis, 2005]
That sentence from Ioannidis's essay is deliberately provocative. Much of this chapter explains why it is mathematically defensible under certain conditions, and why those conditions describe parts of biomedicine more closely than we would like.
2. Reading a Primary Research Paper: Reconstructing the Argument
A primary research paper is an argument, not a narrative of what happened in the lab. Day and Gastel define the scientific paper as a written and published report describing original research results. They stress that its format exists to let a reader find, evaluate, and potentially reproduce those results [Day & Gastel, 2016]. The standard IMRAD structure (Introduction, Methods, Results, and Discussion) maps onto four questions:
- Introduction: What question was asked, and why does it matter?
- Methods: How was it studied? (Could I repeat this?)
- Results: What was found? (The data, not the interpretation.)
- Discussion: What does it mean? (Interpretation, limitations, context.)
Experienced readers do not read in that order. A productive sequence is:
- Title and abstract. Identify the single central claim. Write it down in one sentence in your own words.
- Figures, before the text. The figures are the evidence. Ask what each panel shows, what it would have to show to support the claim, and whether it does.
- Methods for the key figures. Find the controls, the sample sizes, the statistical tests, and whether the data are technical or biological replicates.
- Results text. Compare the language to the data. Watch for words like "dramatically," "clearly," and "trend toward significance."
- Discussion. Separate what was shown from what was speculated.
- Supplementary information. This is where the bodies are buried: full gels, uncropped blots, characterization data, failed conditions, and the NMR spectrum that turns out to have a solvent impurity.
The critical questions
- What is the central claim, and what is the weakest link in the evidence chain supporting it?
- What alternative explanations exist, and which experiments rule them out?
- Are the controls appropriate? (Section 4 covers this.)
- Is n the number of independent experiments, or the number of wells, cells, or injections?
- Would this result surprise me? If so, is the evidence proportionally strong?
- Who funded it, and do the authors have competing interests?
Connection to what you know: In CHEM302 you learned to distinguish precision from accuracy. A paper can report beautifully tight error bars (high precision) around a systematically biased measurement (low accuracy). Reading critically means asking which kind of error the authors controlled and which they ignored.
3. Statistical Inference: What a p-Value Can and Cannot Tell You
3.1 The p-value, defined precisely
p = P( data at least as extreme as observed | H₀ is true )
Read the conditional carefully. The p-value is the probability of the data given the null hypothesis. It is not P(H₀ | data), the probability that the null is true given what you saw. Confusing these is the statistical equivalent of confusing Kd with koff. They are related, but they are not the same quantity, and the conversion between them requires additional information.
That additional information is the prior. Bayes' theorem (MATH201) connects the two directions:
P(H₁ | significant result) = P(significant | H₁) · P(H₁)
─────────────────────────────────────────────
P(significant | H₁)·P(H₁) + P(significant | H₀)·P(H₀)
3.2 Ioannidis's positive predictive value
Ioannidis applied exactly this logic to research fields as a whole [Ioannidis, 2005]. He defined three quantities:
- R: the pre-study odds that a probed relationship is true. This is the ratio of true relationships to null relationships among all those tested in a field.
- α: the Type I error rate (false positive rate, typically 0.05).
- β: the Type II error rate, so that power = 1 − β.
The post-study probability that a "significant" finding is true, the positive predictive value (PPV), is:
(1 − β) · R
PPV = ─────────────────
R − βR + α
A finding is more likely true than false (PPV > 0.5) only when (1 − β)R > α.
Worked example. Suppose a screening-heavy field tests relationships where 1 in 11 is real, so R = 0.1. Use α = 0.05.
Well-powered study (1 − β = 0.80): PPV = (0.80 × 0.1) / (0.1 − 0.02 + 0.05) = 0.080 / 0.130 ≈ 0.62 Underpowered study (1 − β = 0.20): PPV = (0.20 × 0.1) / (0.1 − 0.08 + 0.05) = 0.020 / 0.070 ≈ 0.29
This result is counterintuitive. Both studies report p < 0.05. Yet in the underpowered case, roughly 7 out of 10 "significant" findings are false. Low power does not just cause you to miss real effects. It also degrades the trustworthiness of the positives you do find.
Ioannidis then added bias (u). This is the proportion of analyses that would not have been "findings" but get reported as such anyway, through flexible analysis, selective reporting, and similar practices:
(1 − β)R + uβR
PPV = ─────────────────────────────────
R + α − βR + u − uα + uβR
Even modest bias (u = 0.1–0.3) drags PPV down sharply when R is small. From this framework Ioannidis derived corollaries. A finding is less likely to be true when:
- studies are small;
- effect sizes are small;
- many relationships are tested with little preselection;
- designs, definitions, and analyses are flexible;
- financial and other interests are large;
- the field is "hot," with many teams racing to publish.
Why this is beautiful: it turns a vague cultural worry ("too much research is unreliable") into a quantitative model with tunable parameters. It also tells you which levers to pull. You cannot easily change R, which is a property of the field. You can control power and bias. Pre-registration attacks u. Adequate sample size attacks β.
3.3 Effect sizes and confidence intervals
A p-value mixes two things together: how big the effect is, and how precisely you measured it. A huge study can make a trivial effect "highly significant." Always ask for the effect size and its confidence interval.
Cohen's d = (x̄₁ − x̄₂) / s_pooled s_pooled = √[ ((n₁−1)s₁² + (n₂−1)s₂²) / (n₁ + n₂ − 2) ] 95% CI for a mean difference ≈ (x̄₁ − x̄₂) ± t₀.₉₇₅,df · SE
A confidence interval tells you the range of effect sizes compatible with your data. Two examples show why that matters:
- An interval of [0.02, 3.5] fold-change is "significant," yet compatible with both negligible and enormous effects. You have learned little.
- An interval of [−0.05, 0.08] is "not significant," yet informative: it tightly rules out any large effect.
3.4 Power and sample size
For a two-group comparison with a continuous outcome, the approximate sample size per group is:
n per group ≈ 2 (z₁₋α/₂ + z₁₋β)² σ² / Δ² = 2 (z₁₋α/₂ + z₁₋β)² / d² For α = 0.05 (z = 1.96), power = 0.80 (z = 0.84), d = 0.5: n ≈ 2 × (2.80)² / 0.25 ≈ 63 per group
Compare that with the typical "n = 3" biological replicates in a cell-biology figure. With n = 3 per group, you can reliably detect only very large effects (d ≳ 2.5–3). This is not an argument that n = 3 is always wrong. For large, mechanistically predicted effects it may be fine. It does mean you should calibrate your confidence to the effect size the design can actually resolve.
3.5 Choosing the right test
- Two groups, continuous, roughly normal: Student's or Welch's t-test. Prefer Welch unless the variances are demonstrably equal.
- Paired measurements (same sample before and after): a paired t-test. Using an unpaired test discards information.
- More than two groups: ANOVA followed by an appropriate post-hoc correction (Tukey, Dunnett vs. control). Running many t-tests in a row inflates the false-positive rate.
- Non-normal or ordinal data, small n: nonparametric tests (Mann–Whitney, Kruskal–Wallis).
- Many comparisons (proteomics, genomics; recall BIOL502): control the false discovery rate (Benjamini–Hochberg) rather than accepting raw p < 0.05.
- Dose–response: nonlinear regression to a model (e.g., four-parameter logistic). Report EC₅₀ with a confidence interval, not a t-test at each concentration.
The pseudoreplication trap: Suppose you have three wells from one cell passage, measured in triplicate. That is nine numbers, but n = 1 biological replicate. Treating technical replicates as independent shrinks the standard error falsely and manufactures significance. This is among the most common errors in published bioscience.
4. Experimental Design: Building Experiments That Can Fail
A well-designed experiment is one whose outcome could genuinely embarrass your hypothesis. Each element of design closes off an escape route for alternative explanations.
4.1 Controls
- Negative control: conditions where you expect no effect. It tells you whether the signal could arise from the background, the vehicle, or the procedure itself. Examples: a scrambled-sequence peptide, the DMSO vehicle alone, a catalytically dead enzyme mutant.
- Positive control: conditions where you know an effect should appear. It proves the assay can detect what you are looking for. A negative result without a positive control is uninterpretable, because you cannot distinguish "no effect" from "broken assay."
- Specificity controls: evidence that the effect comes from the proposed mechanism and not a side effect. Examples: an inactive enantiomer, a point mutant at the proposed binding residue, a rescue experiment.
Analogy: Controls are to an experiment what a blank and a standard are to a spectrophotometric assay (CHEM302L). Without the blank you do not know your zero. Without the standard you do not know your scale.
4.2 Randomization
Randomly assigning samples to treatments breaks any correlation between treatment and unmeasured confounders. Such confounders include plate position (edge effects), time of day, cage location, and the order of processing. Randomization does not guarantee balance in any single experiment. It makes imbalance a matter of chance, which statistics can account for, rather than systematic.
4.3 Blinding
The person scoring outcomes should not know which group a sample belongs to. This is especially important for subjective readouts: histology scoring, behavioral assays, colony counting, choosing "representative" images. Blinding does not imply dishonesty. Expectation shapes perception in everyone.
4.4 Factorial designs
Varying one factor at a time (OFAT) is intuitive but inefficient, and it cannot detect interactions. A 2×2 factorial design tests two factors (A and B) at two levels each, in all four combinations:
y = β₀ + β_A·A + β_B·B + β_AB·(A × B) + ε
The interaction term βAB captures whether the effect of A depends on B. Consider an SPPS coupling optimization. Raising the temperature and switching the coupling reagent might each help alone but cause racemization together. OFAT would never reveal that. Factorial and response-surface designs (Design of Experiments, DoE) are standard in process chemistry for exactly this reason.
4.5 Orthogonal approaches
An orthogonal approach tests the same conclusion by a method with independent failure modes. Examples:
- Confirm a binding interaction by ITC (heat) and SPR (refractive index), which share no artifacts.
- Confirm a knockdown phenotype with a second siRNA of different sequence, or with CRISPR knockout, to rule out off-target effects.
- Confirm peptide identity by both high-resolution MS (mass) and NMR or MS/MS sequencing (connectivity).
The word is borrowed from linear algebra (MATH301). Orthogonal vectors share no component, and orthogonal methods share no systematic error. You have also met the word in peptide chemistry: orthogonal protecting groups (Fmoc/tBu) can each be removed without affecting the other. The logic is identical. You want independence, so that one failure does not propagate to the other.
5. Scientific Writing: Clarity as an Ethical Obligation
Day and Gastel's central argument is that scientific writing is not literature. Its sole purpose is the efficient, unambiguous transmission of information [Day & Gastel, 2016]. Ornament that obscures meaning is a failure, not a stylistic choice. Unclear writing is also a reproducibility problem: a method that cannot be understood cannot be repeated.
5.1 Principles of clear prose
- One idea per paragraph. Put the topic sentence first. A reader skimming only first sentences should get the argument.
- Put old information before new. Begin sentences with what the reader already knows and end with the new point. The end of a sentence is its position of emphasis.
- Prefer concrete verbs to nominalizations. "We measured the binding" is better than "Measurement of the binding was performed."
- Use terms consistently. In literary writing, varying synonyms is elegant. In scientific writing it implies different things. If you call it "the conjugate" in one paragraph and "the construct" in the next, a reader will wonder whether there are two molecules.
- Mind the tense. Day and Gastel note a convention: use present tense for established knowledge (previously published, accepted findings) and past tense for your own new results, which are not yet established knowledge [Day & Gastel, 2016].
5.2 Active versus passive voice
The old norm held that scientific writing must be passive ("the solution was stirred") in order to seem objective. Modern style guides, including Day and Gastel, favor the active voice where it is clearer and shorter [Day & Gastel, 2016]. The passive voice remains appropriate when the actor is genuinely irrelevant, as in many Methods sentences. The rule is clarity, not a ban on either voice.
- Weak: "It was found that the peptide was degraded by trypsin."
- Better: "Trypsin degraded the peptide."
- Better still, if it is your finding: "Trypsin degraded the peptide within 10 min (Fig. 2A)."
5.3 Figures
Most readers look at figures first, and many look only at figures. A figure should make one point that is visible within seconds. Guidelines:
- Show the data, not just summaries. Overlay individual points on bar charts, or replace bars with dot plots or box plots. A bar of mean ± SEM with n = 3 hides everything.
- Label error bars explicitly. Say whether they are SD, SEM, or 95% CI, and state n. These convey very different information.
- Choose axes honestly. Do not truncate a y-axis to exaggerate a difference without signaling it. Use a log scale where the data span orders of magnitude, as with Kd or IC50 values.
- Use colorblind-safe palettes (e.g., viridis), and do not rely on red/green alone.
- Use the right tools. GraphPad Prism is convenient for standard biological statistics. R with ggplot2 gives reproducible, scriptable figures, so the code is the figure's provenance. Illustrator suits layout and schematics, but never alter data in it.
- Treat image manipulation as an ethical line. Brightness and contrast adjustments must apply to the whole image. Splicing gel lanes must be marked explicitly. Keep the uncropped originals.
6. Finding and Synthesizing Literature
6.1 Search strategy
- PubMed/MEDLINE: curated biomedical literature indexed with MeSH (Medical Subject Headings). A controlled vocabulary means searching "Peptides/therapeutic use"[MeSH] catches papers regardless of the authors' wording. Use Boolean operators (AND, OR, NOT) and field tags.
- Web of Science / Scopus: curated citation indexes. Their power is citation chaining. Backward chaining asks what a key paper cites. Forward chaining asks who cited it later. Forward chaining is how you discover whether a finding was replicated, extended, or refuted.
- Google Scholar: broad coverage including theses and grey literature, with less curation. It is good for finding things, but its precision is poor.
- Preprint servers (bioRxiv, ChemRxiv, arXiv): rapid access to unreviewed work. They are valuable for timeliness. Read them with the same critical eye you should apply to everything, since peer review is a filter, not a guarantee.
- Reference managers (Zotero, Mendeley): store PDFs, annotations, and citation metadata, and format bibliographies automatically. Choose one early. Retrofitting citations at the thesis stage is miserable.
6.2 Synthesis versus summary in review articles
A weak review is a sequence of summaries: "Smith found X. Jones found Y. Lee found Z." A strong review synthesizes. It organizes the literature around ideas, tensions, and open questions:
- "Three approaches to stabilizing helical peptides have emerged: hydrocarbon stapling, lactam bridges, and hydrogen-bond surrogates. They differ chiefly in X; the evidence suggests Y; it remains unresolved whether Z."
The organizing unit is a claim about the field, not a paper. A useful exercise is to build a matrix with papers as rows and questions or variables as columns. The gaps in the matrix are your review's argument.
7. Grant Writing: The Specific Aims Page as Compressed Argument
An NIH application is judged first, and often decisively, by its one-page Specific Aims. It is the most compressed scientific argument you will write. A standard architecture:
- Hook / opening paragraph: the problem and why it matters. Name the disease, the gap, and the cost of not knowing.
- The gap: what is specifically unknown or unsolved, stated as precisely as possible.
- Your solution: long-term goal, objective of this proposal, central hypothesis, and the rationale, often grounded in preliminary data.
- Aims (usually 2–3): each a testable objective with a brief approach and expected outcome. The aims should be related but not interdependent. If Aim 1 fails, Aims 2 and 3 must still be possible. That is the grant-level version of orthogonal design.
- Payoff paragraph: expected outcomes and impact.
Under the NIH review framework, the research strategy has been evaluated under headings including Significance, Innovation, and Approach, along with investigator and environment criteria. NIH has recently been reorganizing these into a simplified framework, so check the current guidance for any specific funding announcement. Two points apply regardless of the framework:
- Preliminary data establish feasibility. They show that the reviewer's bet on you is not a blind one.
- Address pitfalls and alternative strategies explicitly. Reviewers trust applicants who anticipate failure.
The HHMI/BWF guide frames grant writing as part of a larger skill set of scientific management. It covers planning projects, managing people, and structuring a research program so that it remains fundable and productive [Burroughs Wellcome Fund & HHMI, 2006].
8. Communicating Orally: Talks and Posters
A talk is not a paper read aloud. Readers can re-read, but listeners cannot. So:
- One message per slide, stated in the slide title as a sentence ("Stapling increases protease resistance 40-fold") rather than a topic label ("Protease results").
- Narrative arc: question, then approach, then key findings, then implication. Tell the audience where you are going, take them there, and remind them where they have been.
- Minimal text, large figures, and a pointer or highlight to direct attention.
- Handling questions: repeat or rephrase the question. Answer the one asked. Say "I don't know, but here's how we could find out" when that is true. Treat hostile questions as information about what was unclear.
- Posters: a large title stating the finding, a clear reading path (columns, left to right), figures doing the work, and a "take-home" box. Design for someone walking past at three meters who gives you ten seconds.
9. Research Ethics: Integrity, Credit, and Conflicts
9.1 Fabrication, Falsification, Plagiarism (FFP)
The US federal definition of research misconduct, adopted by the Office of Science and Technology Policy in 2000 and used by agencies such as the Office of Research Integrity, comprises three offenses:
- Fabrication: making up data or results and recording or reporting them.
- Falsification: manipulating materials, equipment, or processes, or changing or omitting data, so that the research is not accurately represented. Examples: deleting an inconvenient outlier without disclosure, or splicing a "cleaner" lane into a blot.
- Plagiarism: appropriating another's ideas, processes, results, or words without appropriate credit.
Crucially, misconduct excludes honest error and differences of opinion. That distinction matters. A failed replication is not evidence of fraud, and treating it as such poisons the error-correction machine.
Between FFP and spotless practice lies a large grey zone of questionable research practices. These include selective reporting, undisclosed analytic flexibility, HARKing (Hypothesizing After the Results are Known), and stopping data collection when p dips below 0.05. Simmons, Nelson, and Simonsohn showed by simulation that combining a few such "researcher degrees of freedom" can push false-positive rates far above the nominal 5% [Simmons, Nelson & Simonsohn, 2011]. This is Ioannidis's u in practice.
9.2 Authorship (ICMJE criteria)
The International Committee of Medical Journal Editors recommends that authorship be based on all four of the following:
- Substantial contributions to the conception or design of the work, or the acquisition, analysis, or interpretation of data.
- Drafting the work or revising it critically for important intellectual content.
- Final approval of the version to be published.
- Agreement to be accountable for all aspects of the work, ensuring that questions about accuracy or integrity are appropriately investigated and resolved.
The fourth criterion is the deep one. Authorship is responsibility, not reward. Two improper practices follow from it:
- Gift authorship: adding a lab head who did not contribute.
- Ghost authorship: omitting the industry medical writer who drafted the paper.
Contributors who do not meet the criteria belong in the Acknowledgments. Many journals now use CRediT contributor-role statements to make each person's contribution explicit. Disciplines also differ in author-order conventions. In much of chemistry and biology the last author is the corresponding PI, which is a convention unrelated to the ICMJE criteria themselves.
9.3 Conflict of interest
A conflict of interest exists when a secondary interest (financial, professional, personal) could unduly influence judgment about a primary interest (the integrity of the research). A conflict is a situation, not an accusation. The standard remedy is disclosure, and where necessary management: recusal, independent data analysis, or divestment. Industry funding does not invalidate research. It is information the reader uses to calibrate scrutiny, as in Ioannidis's corollary about financial interests.
10. The Reproducibility Crisis and Its Remedies
Ioannidis's model predicted widespread irreproducibility. Subsequent empirical work largely confirmed it:
- Begley and Ellis reported that Amgen scientists could confirm the findings of only 6 of 53 "landmark" preclinical cancer studies [Begley & Ellis, 2012].
- The Open Science Collaboration attempted to replicate 100 published psychology studies. About 36% of the replications yielded statistically significant results, compared with 97% of the originals, and the effect sizes were on average about half as large [Open Science Collaboration, 2015].
The shrinking effect sizes are themselves predicted by the statistics. When studies are underpowered, only overestimated effects cross the significance threshold. This is the winner's curse: the first published estimate of an effect is systematically inflated.
Remedies
- Pre-registration: publicly time-stamping the hypothesis, design, and analysis plan before data collection. It does not forbid exploratory analysis. It makes the distinction between confirmatory and exploratory analysis visible.
- Registered Reports: peer review of the introduction and methods before results exist. The journal commits in principle to publishing regardless of outcome. Publication bias is removed at the source [Chambers, 2013].
- Open data and open code: depositing raw data (PDB, PRIDE, GEO, Zenodo) and analysis scripts lets others re-run the analysis. This is the reason ggplot2 scripts beat hand-built figures.
- Replication studies: direct replications (same methods) test reliability. Conceptual replications (different methods, same claim) test generality, much like orthogonal approaches.
- Reporting guidelines: ARRIVE for animal studies, CONSORT for clinical trials, and MIAME-style minimum-information standards for omics data.
11. Animal and Human Subjects Research
11.1 The 3Rs
Russell and Burch proposed the 3Rs framework for humane animal research [Russell & Burch, 1959]:
- Replace: use non-animal methods where possible (cell culture, organoids, in silico models, biochemical assays).
- Reduce: use the minimum number of animals consistent with valid results. Note the tension with power. Using too few animals is also unethical, because an underpowered study wastes every animal used in it while producing unreliable conclusions. The power calculation in Section 3.4 is therefore an ethical tool.
- Refine: minimize pain and distress through better procedures, analgesia, humane endpoints, and housing enrichment.
11.2 Oversight bodies
- IACUC (Institutional Animal Care and Use Committee) reviews and approves vertebrate animal protocols before work begins, under the Animal Welfare Act and Public Health Service policy.
- IRB (Institutional Review Board) reviews research with human subjects under the Common Rule (45 CFR 46). Its core principles trace to the Belmont Report: respect for persons (informed consent), beneficence (maximize benefit, minimize harm), and justice (fair distribution of burdens and benefits).
- IRB review is required for non-invasive work too. Surveys, identifiable data, and de-identified biospecimens all fall under its oversight, though some qualify for exempt or expedited review.
12. Intellectual Property
A patent grants a time-limited right (generally 20 years from filing) to exclude others from making, using, or selling an invention. In exchange, the inventor makes a full public disclosure. A patent is a bargain: monopoly for transparency. Requirements:
- Novelty: not previously disclosed.
- Non-obviousness (inventive step): not an obvious variation to a skilled practitioner.
- Utility (industrial applicability).
- Enablement and written description: the disclosure must teach a skilled person how to make and use the invention.
The counterintuitive trap. Since the America Invents Act took effect in 2013, the US operates a first-inventor-to-file system, with a limited one-year grace period for the inventor's own disclosures. Most other jurisdictions, including Europe, require absolute novelty. Your own conference poster, talk, thesis, or preprint, presented before filing, can destroy patentability outside the US. This is why universities insist on an invention disclosure to the technology transfer office before public presentation. The habit of openness that serves science can, if timed carelessly, forfeit commercial protection that might have funded a drug's development.
Trade secrets are the alternative. They protect information through secrecy rather than disclosure, potentially forever, but offer no protection against independent discovery or reverse engineering. They suit manufacturing processes, such as a proprietary peptide purification protocol, better than products that can be analyzed once sold.
In pharmaceutical chemistry, patent strategy layers protection:
- composition of matter patents on the molecule itself, the strongest form;
- patents on formulations, methods of use, and manufacturing processes;
- patents on polymorphs or salt forms.
Secondary patents of this kind can extend market exclusivity, which is controversial under the name "evergreening."
13. Careers and Scientific Management
The HHMI/BWF guide was written because scientists are trained extensively in experiments and barely at all in running a laboratory. A laboratory is a small organization with personnel, budgets, conflicts, and strategy [Burroughs Wellcome Fund & HHMI, 2006]. Its lessons generalize:
- Academia: intellectual freedom, teaching, and trainees, under funding pressure, with a long, uncertain path to independence.
- Industry: team-based, goal-directed work with defined milestones (a candidate, a process, a product). Resources are often greater, freedom to choose problems is often lower, and publication may be restricted by IP considerations (Section 12).
- Government: regulatory science (FDA), national labs, and funding agencies. These roles emphasize public mission and often stability.
- Networking is the exchange of genuine scientific interest, not transactional self-promotion. Ask good questions at posters, and follow up on papers you admired.
- Recommendation letters carry weight when the writer knows your work specifically. Give your recommenders your CV, a summary of what you did with them, deadlines, and ample time.
14. Connections to Peptide Chemistry
This course is not a detour from your scientific training. It is the lens through which you will evaluate everything that follows. Peptide science offers especially vivid examples.
14.1 Reading a landmark paper critically: the Leigh peptide-synthesizing machine
Lewandowski and colleagues in David Leigh's group reported an artificial small-molecule machine that synthesizes a peptide in a sequence-specific manner [Lewandowski et al., 2013]. The machine has three parts:
- a rotaxane: a molecular ring threaded on an axle;
- amino acids loaded onto the axle in a defined order;
- a thiolate-bearing arm on the ring that picks up each amino acid in turn as the ring moves along the axle. Each residue is transferred to a growing peptide chain by chemistry related to native chemical ligation (thioester exchange followed by S→N acyl transfer, which you met in CHEM202 and CHEM503).
The track constrains the ring to encounter residues in order, so sequence is encoded mechanically, loosely echoing the ribosome reading mRNA. The authors report operating on the order of 1018 machines in parallel to obtain milligram quantities of a single-sequence peptide.
Apply the reading framework from Section 2:
- Central claim: sequence-specific synthesis is achieved by a synthetic molecular machine. The claim is about mechanism and principle, not efficiency.
- Key evidence: characterization showing the product has the single expected sequence, rather than a mixture. Here MS/MS sequencing and NMR serve as orthogonal confirmation.
- Critical question: how does it compare with Merrifield's solid-phase peptide synthesis [Merrifield, 1963] or with the ribosome in speed, length, and fidelity? The fair comparison is conceptual. The machine demonstrates that processive, sequence-directed synthesis can be engineered from small molecules. It does not claim to outcompete SPPS.
- Calibration: Ioannidis's model is about fields with low pre-study odds and noisy readouts. A structurally characterized synthetic product (MS, NMR, HPLC) sits in a different regime from a cell-based bioactivity claim. The question "how was the product's identity proven?" has a crisp answer. The question "is this peptide therapeutically effective?" does not. Recognize which kind of claim you are reading.
14.2 Reproducibility hazards specific to peptides
- Counter-ions: peptides purified by reversed-phase HPLC with TFA are typically isolated as TFA salts. Residual trifluoroacetate can affect cell assays, and it inflates the apparent mass, so "1 mg of peptide" may contain substantially less peptide. Report net peptide content, or quantify by amino acid analysis or UV absorbance.
- Purity reporting: ">95% pure by HPLC" at one wavelength can hide co-eluting deletion sequences, which are common SPPS byproducts from incomplete couplings. Orthogonal analysis (LC-MS plus analytical HPLC under different conditions) is the remedy.
- Racemization and aspartimide formation: process side-reactions that produce near-isobaric impurities. Their biological effects may be real but misattributed to the intended peptide.
- Aggregation: amyloidogenic or hydrophobic peptides behave differently depending on preparation history (HFIP pretreatment, freeze–thaw cycles). This is a classic source of irreproducibility in Aβ research. "Methods" must be detailed enough to capture it.
- Controls: scrambled-sequence peptides (same composition, different order) as negative controls; D-enantiomers to test whether activity requires chiral recognition; alanine scans to map specificity.
14.3 Peptide drugs, IP, and ethics
- Engineered peptide drugs: GLP-1 receptor agonists such as semaglutide illustrate protein engineering for half-life extension. Their design combines an amino-acid substitution that confers resistance to the protease DPP-4 with fatty-acid acylation that promotes albumin binding.
- Layered IP: these drugs are also case studies in layered patents and in the economics that drive which peptides are developed at all.
- Clinical development brings IRBs, informed consent, CONSORT reporting, and conflict-of-interest disclosure into the picture.
- Structural biology: data deposition (PDB coordinates plus structure factors or cryo-EM maps) and validation reports provide open data with built-in quality metrics. They are a model the rest of biology is still catching up to.
15. Common Misconceptions to Watch Out For
- "p = 0.03 means there is a 3% chance the null hypothesis is true."
No. It is P(data | H₀), not P(H₀ | data). Converting between them requires a prior (Section 3.2).
- "Not significant means no effect."
Absence of evidence is not evidence of absence, especially in an underpowered study. Look at the confidence interval. If it is wide, the study is simply uninformative.
- "A 95% confidence interval has a 95% probability of containing the true value."
Under frequentist interpretation, 95% describes the long-run behavior of the procedure: 95% of intervals so constructed contain the true value. Any particular computed interval either does or does not.
- "Smaller p-values mean bigger effects."
The p-value depends on both effect size and sample size. A tiny effect in a huge sample can yield p < 10−10.
- "A significant result from a small study is especially impressive because it overcame low power."
The reverse is true. Low power lowers the positive predictive value and inflates effect estimates (the winner's curse).
- "n = number of data points."
n is the number of independent experimental units. Technical replicates are not biological replicates.
- "Scientific writing must be in the passive voice."
Clarity is the rule. The active voice is usually clearer, and modern style guides endorse it [Day & Gastel, 2016].
- "Authorship is a reward for seniority, funding, or providing reagents."
Under ICMJE, authorship requires intellectual contribution and accountability. Providing a plasmid alone warrants acknowledgment.
- "A failed replication means the original was fraudulent."
Most irreproducibility arises from low power, bias, unreported variables, and honest error, not misconduct. Misconduct explicitly excludes honest error.
- "Peer-reviewed means correct; preprint means unreliable."
Peer review is a filter of limited sensitivity. Reviewers rarely have the raw data. Evaluate evidence, not venue.
- "Using fewer animals is always more ethical."
An underpowered animal study wastes every animal used. "Reduce" means the minimum sufficient number, which comes from a power calculation.
- "IRB review is only for invasive medical studies."
Surveys, interviews, and identifiable data or specimens all fall under human-subjects oversight.
- "I can present my invention at a conference and patent it later."
Outside the US grace period, your own public disclosure can destroy novelty. File an invention disclosure first.
- "Pre-registration forbids exploratory analysis."
It only requires that exploratory analyses be labeled as such, so readers can weigh them appropriately.
- "Ioannidis proved that most science is false."
He showed that under specific, quantifiable conditions (low R, low power, high bias), most claimed findings are expected to be false. The model's real lesson is which conditions to change. It does not license blanket cynicism.
Closing Reflection
There is a unifying idea under all of this. In CHEM301 you learned that entropy tends to increase unless work is done to prevent it. Scientific knowledge works the same way. Left alone, the literature drifts toward noise, through underpowered studies, flexible analyses, selective publication, and perverse incentives. Controls, statistics, clear writing, honest authorship, open data, and replication are the work that keeps the system ordered. This course is about learning to do that work and recognizing when others have not.
References Cited
- Begley, C. G., & Ellis, L. M. (2012). Raise standards for preclinical cancer research. Nature, 483, 531–533.
- Burroughs Wellcome Fund & Howard Hughes Medical Institute. (2006). Making the Right Moves: A Practical Guide to Scientific Management for Postdocs and New Faculty (2nd ed.). HHMI.
- Chambers, C. D. (2013). Registered Reports: A new publishing initiative at Cortex. Cortex, 49, 609–610.
- Day, R. A., & Gastel, B. (2016). How to Write and Publish a Scientific Paper (8th ed.). Cambridge University Press.
- Ioannidis, J. P. A. (2005). Why most published research findings are false. PLOS Medicine, 2(8), e124. doi:10.1371/journal.pmed.0020124
- Lewandowski, B., De Bo, G., Ward, J. W., Papmeyer, M., Kuschel, S., Aldegunde, M. J., Gramlich, P. M. E., Heckmann, D., Goldup, S. M., D'Souza, D. M., Fernandes, A. E., & Leigh, D. A. (2013). Sequence-specific peptide synthesis by an artificial small-molecule machine. Science, 339, 189–193. doi:10.1126/science.1229753
- Merrifield, R. B. (1963). Solid phase peptide synthesis. I. The synthesis of a tetrapeptide. Journal of the American Chemical Society, 85, 2149–2154.
- Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349, aac4716.
- Russell, W. M. S., & Burch, R. L. (1959). The Principles of Humane Experimental Technique. Methuen.
- Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359–1366.
⚙️ Mechanisms & Applications
Mechanisms & Applications
My central insight: research methods are the machinery that turns a chemical measurement into a defensible claim. In earlier courses, I learned how molecules react, how instruments generate signals, and how cells respond. CHEM504 asks a different question: What chain of reasoning allows me to say that my experiment has taught me something reliable? Experimental design, statistical analysis, writing, and ethics are not separate administrative tasks. They protect different links in that chain.
Scientific question
→ explicit hypothesis and competing explanations
→ design that distinguishes those explanations
→ traceable measurements
→ analysis with stated assumptions
→ appropriately bounded claim
→ transparent communication
→ independent scrutiny and further testing
1. Reading a Paper as a Reconstruction of an Argument
A primary paper is not simply a container of facts. It is an argument supported by selected observations. My job is to reconstruct that argument and inspect the transitions between observation, interpretation, and generalization. Day and Gastel’s treatment of scientific papers is especially useful here: the familiar IMRAD structure—Introduction, Methods, Results, and Discussion—reflects different jobs in scientific reasoning, not merely a publishing convention.[1]
- Identify the question. From the abstract and introduction, write down the uncertainty the study addresses. “We studied peptide stability” is a topic; “Does cyclization reduce proteolytic degradation without impairing receptor activation?” is a research question.
- Extract the main claim. Translate it into one sentence with a defined system and outcome. Watch for a switch from “in purified enzyme assays” to “in patients.”
- Read the figures as evidence. Determine what was measured, what each point represents, what the error bars mean, and whether the apparent difference is large enough to matter.
- Reconstruct the experiment from Methods and supplementary information. Look for sample preparation, independent replication, exclusions, analytical settings, normalization, and statistical procedures. Supplementary material often contains the information needed to judge whether the central claim is secure.
- Generate an alternative explanation. Could altered fluorescence, aggregation, solvent effects, batch differences, or selective reporting produce the same result?
- Check whether the controls discriminate between explanations. A control is valuable because it rules out a particular alternative, not because every paper is expected to contain something labeled “control.”
- Separate supported conclusions from proposals. “These results are consistent with…” and “These results demonstrate…” carry different evidential commitments.
Connection to spectroscopy: assigning a molecular structure from one mass peak is usually underdetermined. Likewise, inferring a biological mechanism from one assay is underdetermined. Both require additional constraints. A compelling paper resembles a well-constrained structure elucidation: multiple observations narrow the set of plausible explanations.
2. Designing an Experiment That Can Actually Answer the Question
Consider a pharmaceutical-development question: Does cyclization improve a lead peptide’s plasma stability while preserving receptor activity? The design begins before synthesis or data collection.
- Define the target quantity. For example, compare degradation rate constants under specified plasma-incubation conditions. Separately define acceptable receptor potency and efficacy.
- Specify the experimental units. Independent donors, independently prepared peptide batches, and assay wells address different sources of uncertainty. Repeated injections of one sample estimate analytical precision; they do not establish donor-to-donor reproducibility.
- Choose controls linked to failure modes. Include time-zero recovery samples, plasma blanks, a reference peptide, and a buffer-only incubation to distinguish proteolysis from chemical instability or adsorption.
- Randomize allocation and processing order. Balance peptide variants across plates, days, and instrument runs so that treatment is not confounded with drift or batch.
- Blind where judgment enters. Code samples during integration, image scoring, or exclusion decisions. Blinding is especially useful when apparently small discretionary choices can shift an outcome.
- Specify the primary analysis in advance. Define the main endpoint, model, exclusions, and stopping rule. Label additional analyses as exploratory.
- Preserve the entire evidence trail. Retain raw chromatograms, sample identities, preparation records, analysis scripts, and reasons for any deviations.
A crucial distinction: the experimental unit is the smallest unit independently assigned to a treatment. The unit needed for generalization may be broader. Wells can be legitimate treatment units, but wells sharing a donor or preparation remain correlated. The analysis must reflect that hierarchy rather than pretending that every recorded number is an independent biological observation.
3. Controls, Orthogonal Assays, and Factorial Designs
Different controls answer different questions:
- Negative control: what signal occurs without the proposed causal agent? Examples include vehicle-only treatment or an assay lacking enzyme.
- Positive control: can the system detect a known effect under these conditions? A validated receptor agonist can reveal whether an apparently inactive peptide was tested in a functioning assay.
- Process control: did extraction, preparation, and measurement work? An internal standard added before extraction can track recovery and instrument variation, although no standard automatically corrects every bias.
- Mechanistic control: does the effect depend on the proposed target or pathway? Examples include a target-deficient cell line, a catalytic-site mutant, or competition with a known ligand.
A scrambled peptide is not automatically an ideal negative control. Scrambling can change charge distribution, solubility, aggregation, and protease susceptibility. A “matched” control must be matched in the properties relevant to the alternative explanation.
Orthogonal approaches investigate the same proposition using methods with different failure modes. If a peptide appears to inhibit an enzyme in a fluorescence assay, a label-free LC–MS measurement of product formation can test whether inhibition persists without the fluorescent reporter. Agreement is especially informative when the two methods do not share the suspected artifact.
Fluorescence assay:
Less fluorescence
→ less product?
→ or fluorescence quenching?
→ or inner-filter effects?
Orthogonal LC–MS assay:
Measure substrate disappearance and/or product formation
→ test whether chemical turnover really decreased
A factorial design varies more than one factor together. Suppose I test linear versus cyclic peptide both with and without a protease. A simple model is:
y = β₀ + βA A + βB B + βAB AB + ε A = 0 for linear peptide; 1 for cyclic peptide B = 0 without protease; 1 with protease βAB = (cyclic − linear difference with protease) − (cyclic − linear difference without protease)
The interaction term asks whether cyclization has a different effect when protease is present. This is often closer to the mechanistic question than either main effect alone. The response might be log concentration remaining; if measurements are proportions near zero or one, an ordinary constant-variance linear model may be inappropriate.
Historical connection: Ronald A. Fisher helped establish randomization, analysis of variance, and factorial experimentation through agricultural research at Rothamsted. His The Design of Experiments appeared in 1935.[7] Agricultural plots and biochemical assay plates share a problem: background variation can masquerade as a treatment effect unless allocation and analysis are designed together.
4. Quantitative Reasoning: What a Result Does—and Does Not—Establish
A p-value is calculated under a statistical model. It is the probability, assuming the null hypothesis and the model assumptions, of obtaining a test statistic at least as extreme as the observed one. It is not the probability that the null hypothesis is true, the probability that the finding occurred “by chance,” or the probability that replication will succeed.
For the peptide-stability study, a useful report includes the estimated difference or ratio, its confidence interval, and its practical meaning. A statistically detectable improvement from 10 to 11 minutes may be less valuable than an uncertain improvement from 10 to 60 minutes that warrants a better-powered follow-up.
Illustrative first-order degradation model: C(t) = C₀e^(−kt) ln C(t) = ln C₀ − kt t½ = ln(2)/k
This connects directly to chemical kinetics. However, fitting an exponential does not prove a single first-order mechanism. Protease saturation, multiple susceptible sites, plasma binding, adsorption, and heterogeneous peptide populations can produce departures from the model. Residual plots and alternative models matter. Log transformation also changes the implied error model; fitting concentration directly may be preferable when the measurement-error structure supports it.
Confidence intervals quantify uncertainty conditional on the method and assumptions. In repeated sampling, a procedure producing 95% confidence intervals would cover the fixed true parameter in 95% of repetitions. This does not mean that a particular frequentist interval assigns 95% probability to the parameter lying inside it. Nor does an interval capture unmodeled bias.
Sample size follows the effect I need to detect. For two independent groups with equal allocation, approximately normal outcomes, common standard deviation σ, a two-sided significance level of 0.05, and 80% power:
n per group ≈ 2(z₀.₉₇₅ + z₀.₈₀)² σ²/δ²
≈ 15.7 σ²/δ²
δ = difference the study is designed to detect
If δ = 0.5σ:
n ≈ 63 independent units per group
This approximation is not a substitute for design-specific planning. Paired studies, repeated measures, clustering, losses, and multiple endpoints change the calculation. The important relationship is the inverse square: detecting half the effect generally requires about four times as many independent observations.
Multiple testing creates opportunities for false positives. If 20 independent tests each use a 0.05 threshold and all null hypotheses are true:
P(at least one false positive)
= 1 − (1 − 0.05)²⁰
≈ 0.642
The independence assumption matters. Nevertheless, the example explains why screening many peptides, doses, time points, and endpoints and reporting only the smallest p-value is misleading. Confirmatory studies can control familywise error; discovery screens often control the false discovery rate, the expected proportion of false discoveries among the rejected hypotheses.
5. Why Prior Plausibility and Reporting Practices Matter
John P. A. Ioannidis’s 2005 essay, “Why Most Published Research Findings Are False,” used a modeling framework to show how low prior odds, low power, and bias can reduce the reliability of published positive findings.[3] Its title is not a universal empirical verdict on every discipline. Its enduring lesson is that a significance threshold alone cannot establish a claim’s credibility.
Illustrative screening calculation:
π = fraction of tested hypotheses with a real effect
Power = 1 − β
α = false-positive probability under the null
Positive predictive value
= π(1 − β) / [π(1 − β) + (1 − π)α]
If π = 0.10, power = 0.80, and α = 0.05:
PPV = 0.08 / (0.08 + 0.045)
= 0.64
Among 1,000 hypothetical tests in this simplified model, about 80 would be true positives and 45 false positives. If only 1% of tested hypotheses correspond to real effects, the same calculation gives a positive predictive value of about 14%.
Application to drug discovery: a primary-screen “hit” is a starting point, not a validated lead. Resynthesis, purity checks, concentration–response measurements, counterscreens, target-engagement assays, and independent replication progressively eliminate alternative explanations. This calculation is a population-level illustration, not a posterior probability that can automatically be attached to one observed p-value.
6. Peptide Synthesis as a Case Study in Mechanism and Evidence
Robert Bruce Merrifield introduced solid-phase peptide synthesis in 1963, transforming peptide preparation by attaching the growing chain to an insoluble support.[5] The beautiful idea was logistical as much as chemical: excess soluble reagents could be washed away while the desired intermediate remained attached to the resin.
Typical Fmoc solid-phase peptide synthesis: 1. Attach the C-terminal residue to a resin through a suitable linker. 2. Remove the temporary Fmoc protection from the α-amino group. 3. Wash. 4. Activate the carboxyl group of the next protected amino acid. 5. Couple it to the resin-bound free amine. 6. Wash; assess completion and address incomplete coupling. 7. Repeat deprotection–coupling cycles. 8. Cleave from the resin and remove applicable protecting groups. 9. Purify and characterize the product. The chain is assembled from the C terminus toward the N terminus.
Merrifield’s original chemistry was not today’s Fmoc protocol; modern Fmoc synthesis is a later development of the solid-phase strategy. Distinguishing the original invention from its current implementation prevents anachronistic history.
The quantitative challenge is cumulative fidelity:
If each of m coupling steps succeeds independently with probability p: Ideal fraction with every coupling successful = p^m For p = 0.99 and m = 49: p^m ≈ 0.61
This idealized calculation is not an isolated-yield prediction: deprotection, cleavage, purification, and sequence-dependent side reactions also matter. It explains why “99% efficient” can be inadequate in a long multistep synthesis.
In peptide manufacturing, that chemistry becomes an evidence problem. Reverse-phase HPLC can separate many impurities, MS can support molecular-mass assignments, and additional fragmentation or structural methods may address sequence and connectivity. However:
- A correct mass does not necessarily distinguish sequence isomers, epimers, or alternative disulfide connectivities.
- A clean chromatographic peak does not guarantee that all impurities are resolved.
- Chromatographic area percentage is not automatically mass purity; detector response and undetected species matter.
- Purity, identity, potency, and stability are related but distinct attributes.
Close reading of the assigned molecular-machine paper: Lewandowski and colleagues reported sequence-specific peptide synthesis by an artificial small-molecule machine in 2013.[4] Their rotaxane-based architecture mechanically interlocked a ring with a molecular track bearing amino-acid building blocks. A thiol-containing reactive component participated in transferring those building blocks to a growing peptide. The arrangement of the track and steric barriers constrained the order in which the building blocks could be processed.
Simplified mechanistic logic:
Amino-acid units occupy ordered positions on a molecular track
→ a reactive component on the mobile ring accesses a unit
→ thiol-mediated acyl-transfer chemistry transfers the unit
→ amide-bond formation incorporates it into the peptide
→ removal of the obstructing unit permits further progression
→ track architecture constrains the incorporation sequence
The paper matters as a proof of principle for molecular machinery that couples movement with ordered covalent synthesis. It did not establish a general replacement for SPPS or a ribosome-equivalent manufacturing platform. My critical questions are: How was the product sequence established? Which controls support the proposed order of operation? How strongly do the product data constrain the intermediate steps? What is encoded in the track, and what would have to be rebuilt to make a different sequence?
7. Reproducibility Is an Experimental Technique
A reproducible workflow is a designed system for preserving the relationship between samples, transformations, and claims. Using a common terminology, computational reproducibility means obtaining the same results from the same data and code; replication means testing the claim with newly collected data. Terminology varies across fields, so I should state which meaning I intend.
Raw instrument files
→ documented import
→ scripted quality checks
→ recorded exclusions
→ transformations and model fitting
→ generated tables and figures
→ archived data, code, metadata, and software versions
For LC–MS peptide measurements, the record should connect each vial to its synthesis batch, preparation, run position, chromatogram, peak-integration decisions, and reported concentration. File names alone are not adequate metadata.
- Preregistration: records hypotheses and intended analyses before outcomes are known. It distinguishes planned tests from post hoc exploration; it does not prohibit discovery or justified changes.
- Registered reports: submit the question and methods for review before the main results are known. In-principle acceptance can reduce the dependence of publication on whether the result is positive. Chris Chambers helped establish the modern model at Cortex in 2013.[8]
- Open data and code: enable inspection and reanalysis, subject to privacy, consent, security, and legitimate intellectual-property constraints.
- Version control: records changes to code and documents. It supports traceability but does not replace validated procedures or regulated records where these are required.
In pharma, these principles support assay transfer, method validation, and investigations of unexpected results. A useful distinction is that validation asks whether a method is fit for its intended purpose, whereas system suitability checks whether the analytical system is performing acceptably for a particular run.
8. Writing, Figures, and Grants as Tests of Reasoning
Writing reveals gaps because it forces implicit reasoning into explicit sentences. If I cannot explain why a control supports a conclusion, I may not yet understand the experiment.
A disciplined scientific paragraph:
Claim
→ relevant evidence
→ reasoning connecting evidence to claim
→ limitation or boundary
→ consequence or next question
IMRAD distributes this reasoning across the paper: the Introduction establishes the uncertainty; Methods make the investigation assessable; Results report observations; Discussion interprets them without erasing their limits.[1] Active voice often improves clarity—“We randomized samples across plates”—but passive voice is useful when the procedure matters more than the actor. Precision, not grammatical allegiance, is the goal.
Figure preparation is part of analysis. In Prism or R/ggplot2, I should show the experimental structure rather than merely decorate a conclusion. When feasible, display independent observations, define error bars, identify sample sizes, and distinguish technical repeats from biological replication. Illustrator can improve layout, but it must not become an undocumented place to alter data.
A microscopy image or gel should retain its evidential meaning. Preserve originals, document processing, disclose composites, and avoid selective manipulation that changes the interpretation. Journal rules may impose additional requirements.
Grant writing reverses the direction: rather than explaining completed work, it makes future work logically testable. An NIH-style Specific Aims page typically moves from an important problem to a knowledge gap, an overall objective or central hypothesis, a small set of aims, and the expected impact.[2]
- Significance: what consequential uncertainty will be resolved?
- Innovation: what concept, method, or application meaningfully changes current practice?
- Approach: what evidence will discriminate among outcomes?
- Risk management: what happens if a key assumption fails?
For a peptide project, aims might address degradation mechanisms, stability-improving modifications, and functional consequences. Avoid making every aim depend entirely on the success of the first. Preliminary data reduce uncertainty; they should not be presented as guarantees.
The same logic structures presentations and posters: question, decisive experiment, result, limitation, implication. A useful answer to a difficult audience question separates what was measured from what is inferred and what remains unknown.
9. Literature Searching and Review Writing: From Collection to Synthesis
- Decompose the question into concepts. For example: peptide cyclization, proteolytic stability, and receptor agonism.
- Search synonyms and controlled vocabulary. Combine PubMed/MEDLINE indexing terms with free-text terms; use Google Scholar or Web of Science for broader discovery and citation tracing.
- Trace backward and forward. Read foundational references and later papers that test, extend, or challenge them.
- Check publication status. bioRxiv and ChemRxiv provide rapid access, but preprints must be identified as such. Check for later versions, corrections, and retractions.
- Build an evidence matrix. Record system, peptide chemistry, assay conditions, controls, effect estimates, uncertainty, and limitations.
- Synthesize explanations for agreement and disagreement. Different conclusions may arise from different plasma species, incubation conditions, free fractions, or analytical definitions—not simply from one study being “wrong.”
Zotero or Mendeley can manage references, but imported metadata require checking. A review should organize a scientific problem, not march through papers one at a time. If I claim a review is systematic, its search, eligibility, and selection procedures must support that claim.
10. Ethics as Protection Against Corrupted Inference
Fabrication invents evidence; falsification misrepresents the research record; plagiarism misappropriates another person’s words or ideas. These are not merely failures of etiquette. They damage the shared record on which later experiments, clinical decisions, and investments depend.
Selective omission is particularly instructive. Excluding a failed injection under a justified, documented quality rule can improve validity. Excluding a valid measurement because it weakens a preferred conclusion can misrepresent the evidence. The distinction depends on scientific justification, consistency, and disclosure—not whether the final graph looks cleaner.
The ICMJE authorship framework requires all four elements: substantial contribution to the work; drafting or substantive intellectual revision; final approval; and agreement to accountability for the work.[6] Funding or general supervision alone is insufficient. Authorship expectations should be discussed early and revisited as contributions change. Conflicts of interest should be disclosed and managed; disclosure does not by itself remove bias.
Human and animal research: ethics review must precede covered work. In the United States, IRBs oversee covered human-subjects research, while IACUCs oversee covered animal research within their applicable frameworks. Exact obligations depend on jurisdiction, funding, institution, and study type.
- Human research: assess risks, consent, privacy, equitable selection, and whether the design can answer a worthwhile question. The 1979 Belmont Report articulated respect for persons, beneficence, and justice.[9]
- Animal research: apply the 3Rs—Replacement, Reduction, and Refinement—formulated by William Russell and Rex Burch in 1959.[10]
- Counterintuitive point: fewer animals are not automatically more ethical if the study becomes incapable of answering its question. Reduction means using an appropriately efficient design, not minimizing numbers without regard to information.
Intellectual property: peptide discoveries may support patent claims involving composition, formulation, manufacture, or use, depending on patentability requirements and existing knowledge. A patent is principally a right to exclude others, not permission to market a medicine. Regulatory approval and freedom to operate are separate questions. Because public disclosure can jeopardize patent rights, researchers should consult their technology-transfer office before publishing an abstract, poster, or preprint. Trade secrets instead depend on maintaining confidentiality; they are not a justification for misleading scientific reporting.
My final connection: chemistry taught me to track atoms and electrons through a mechanism. Research methods asks me to track evidence through an argument. At every transition, I should ask: What has been conserved, what assumption has entered, and what alternative explanation has actually been eliminated?
References
- Day, R. A., and Gastel, B. How to Write and Publish a Scientific Paper. 8th ed. Cambridge University Press, 2016.
- Burroughs Wellcome Fund and Howard Hughes Medical Institute. Making the Right Moves: A Practical Guide to Scientific Management for Postdocs and New Faculty. 2nd ed. HHMI, 2006.
- Ioannidis, J. P. A. “Why Most Published Research Findings Are False.” PLOS Medicine 2, e124 (2005). doi:10.1371/journal.pmed.0020124.
- Lewandowski, B., et al. “Sequence-Specific Peptide Synthesis by an Artificial Small-Molecule Machine.” Science 339, 189–193 (2013). doi:10.1126/science.1229753.
- Merrifield, R. B. “Solid Phase Peptide Synthesis. I. The Synthesis of a Tetrapeptide.” Journal of the American Chemical Society 85, 2149–2154 (1963). doi:10.1021/ja00897a025.
- International Committee of Medical Journal Editors. Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals. “Defining the Role of Authors and Contributors.” ICMJE recommendations.
- Fisher, R. A. The Design of Experiments. Oliver and Boyd, 1935.
- Chambers, C. D. “Registered Reports: A New Publishing Initiative at Cortex.” Cortex 49, 609–610 (2013). doi:10.1016/j.cortex.2012.12.016.
- National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. The Belmont Report: Ethical Principles and Guidelines for the Protection of Human Subjects of Research. 1979.
- Russell, W. M. S., and Burch, R. L. The Principles of Humane Experimental Technique. Methuen, 1959.
🪬 Skipper's Reflection
1. What Surprised Me Most About This Material
Coming out of the quantum mechanics (CHEM303) and statistical thermodynamics (CHEM301) blocks, I had developed a somewhat idealized view of scientific truth. If the math works and the spectroscopy confirms it, it is real. But CHEM504 completely dismantled my naive assumption that "published in a peer-reviewed journal" equals "scientifically true."
The biggest shock was reading John Ioannidis’s 2005 paper, "Why Most Published Research Findings Are False." Applying what I learned in probability and statistics (MATH201) to the actual enterprise of scientific publishing was a sobering experience. Seeing the mathematical proof of how low statistical power, small effect sizes, and investigator bias (even unintentional) drastically increase the False Discovery Rate (FDR) was a revelation. It surprised me how vulnerable the scientific method is to p-hacking and the pressure to publish. Furthermore, realizing that the IMRAD structure (Introduction, Methods, Results, Discussion) is not a chronological diary of what happened in the lab, but rather a carefully constructed retrospective narrative to make sense of the data, fundamentally changed how I read papers. Day and Gastel’s text made me realize that a scientific paper is an act of persuasion, not just a data dump.
2. The Conceptual Breakthrough Moments
My first major breakthrough happened when we covered experimental design, specifically the concept of orthogonal approaches. In the past, I thought verifying a result meant running the same assay three times (n=3) to get a tight standard deviation. Now I understand that true rigor means interrogating the same hypothesis using entirely different physical principles. If you want to prove a peptide binds to a receptor, you don't just run three ELISAs; you run an ELISA (biochemical), Surface Plasmon Resonance (biophysical), and a cellular reporter assay (functional). When those completely different techniques point to the exact same conclusion, that is when you have a robust finding.
The second breakthrough was reading the 2013 Science paper by David Leigh’s group on the artificial small-molecule machine for sequence-specific peptide synthesis. Conceptually mapping this rotaxane-based "artificial ribosome" back to my knowledge of Solid-Phase Peptide Synthesis (SPPS) from organic chemistry (CHEM201) and natural translation (BIOL201) was beautiful. It made me realize that engineering is just biology spoken in the language of synthetic chemistry. Reading the paper wasn't just about the chemistry, though; it was a masterclass in how to structure a narrative around a highly complex, multidisciplinary innovation to satisfy rigorous peer review.
3. Changing How I Think About the Peptides I Know
This course completely reframed my perspective on the peptide landscape, drawing a hard line between highly regulated pharmaceutical development and the "wild west" of research chemicals.
When I look at Tirzepatide and Retatrutide, I no longer just see fascinating multi-receptor agonists with engineered half-lives. I see the sheer weight of IP law, clinical trial design, and IRB/IACUC oversight. Every single amino acid substitution (like the Aib at position 2 in Tirzepatide to block DPP-4 cleavage) is a meticulously drafted patent claim designed to navigate around existing GLP-1 IP. The data supporting these drugs in the literature is backed by massive, multi-center, double-blinded, placebo-controlled trials—the exact kind of high-power studies Ioannidis argued are necessary to trust a finding.
Conversely, my view of BPC-157 and TB-500 has become much more critical. These peptides have fascinating proposed mechanisms for angiogenesis and tissue repair, but evaluating their literature through the lens of CHEM504 raises red flags. Many studies on these compounds suffer from small sample sizes, lack of pre-registration, and a distinct absence of large-scale reproducible human trials. The ethical gray area of their widespread use as "research chemicals" perfectly illustrates the tension between anecdotal efficacy and rigorous scientific validation. Knowing what I now know about the reproducibility crisis, I realize that reading a paper claiming BPC-157 heals tendons in rats requires looking deeply into their blinding procedures and statistical power before I accept the conclusion.
4. Questions This Course Raises That I Want to Investigate Further
First, how can we leverage machine learning to automate the detection of statistical fragility or p-hacking in biological literature, particularly in early-stage peptide research? If I am going to be synthesizing knowledge across thousands of papers, I need a programmatic way to weigh the evidence based on the rigorousness of the experimental design (e.g., penalizing studies lacking positive/negative controls or blinding).
Second, regarding intellectual property: how do the 3Rs (Replace, Reduce, Refine) of animal research interact with the FDA's strict requirements for in vivo toxicology data before IND (Investigational New Drug) approval? As organ-on-a-chip technologies advance, at what point will the scientific community and patent offices accept completely synthetic, non-animal models for peptide drug validation?
Finally, Leigh’s molecular machine for peptide synthesis is incredible, but is it scalable? Could artificial molecular machines eventually replace traditional Merrifield SPPS for the commercial synthesis of complex macrocyclic or heavily modified therapeutic peptides?
5. Confidence Rating After Studying
Most Confident: I feel highly confident in my ability to critically dissect a primary research paper, evaluate its statistical methods, and identify flaws in experimental design. The rules of IMRAD, appropriate use of controls, and literature search strategies (Boolean operators, MeSH terms) align perfectly with my logical architecture. I also feel very comfortable with the technical aspects of Intellectual Property (patents, claims, prior art) as it applies to chemical space.
Least Confident: Grant writing, specifically the NIH Specific Aims page. The texts from HHMI and Day emphasize that a successful grant requires a compelling "narrative of significance and innovation." Balancing the hyper-rational presentation of preliminary data with the almost political/marketing aspect of convincing a study section that a project is "paradigm-shifting" feels subjective. Navigating the human psychology of peer reviewers and grant committees is a variable I am still trying to model.
🗺️ Visual Aids & Key Takeaways
Diagram 1: The IMRAD Structure of a Primary Research Paper
╔══════════════════════════════════════════════════════════════════════════════════╗
║ ANATOMY OF A PRIMARY RESEARCH ARTICLE (IMRAD) ║
╚══════════════════════════════════════════════════════════════════════════════════╝
┌─────────────────────────────────────────────────────────────────────────────┐
│ TITLE & AUTHORS │
│ The claim in a sentence; author list encodes credit and accountability │
└─────────────────────────────────────────────────────────────────────────────┘
│
↓
┌─────────────────────────────────────────────────────────────────────────────┐
│ ABSTRACT │
│ ~250 words: background → question → approach → key result → conclusion │
│ Read first, but trust last — abstracts over-claim; verify in body │
└─────────────────────────────────────────────────────────────────────────────┘
│
↓
┌─────────────────────────────────────────────────────────────────────────────┐
│ INTRODUCTION │
│ • What is known? (literature context) │
│ • What gap exists? (the problem) │
│ • What did this study do? (the approach / hypothesis) │
│ Ask: Is the gap real? Are the citations fair or cherry-picked? │
└─────────────────────────────────────────────────────────────────────────────┘
│
↓
┌─────────────────────────────────────────────────────────────────────────────┐
│ METHODS (the "M" of IMRAD — often the most neglected section) │
│ • Materials, reagents, sources, lot numbers │
│ • Experimental design: controls, replicates, blinding, randomization │
│ • Statistical plan (should be written BEFORE data collection) │
│ • Ethical approvals: IRB / IACUC protocol numbers │
│ Ask: Could you reproduce this exactly? If not, reproducibility is at risk │
└─────────────────────────────────────────────────────────────────────────────┘
│
↓
┌─────────────────────────────────────────────────────────────────────────────┐
│ RESULTS │
│ • Data presented with appropriate statistics (mean ± SD or SEM, n, p) │
│ • Figures and tables — the raw story without spin │
│ • Should NOT contain interpretation (that belongs in Discussion) │
│ Ask: Do the figures actually support each claim in the text? │
└─────────────────────────────────────────────────────────────────────────────┘
│
↓
┌─────────────────────────────────────────────────────────────────────────────┐
│ DISCUSSION │
│ • Interpretation of results in context of existing literature │
│ • Acknowledgment of limitations and alternative explanations │
│ • Broader significance and future directions │
│ Ask: Do the authors acknowledge what their data CANNOT show? │
└─────────────────────────────────────────────────────────────────────────────┘
│
↓
┌─────────────────────────────────────────────────────────────────────────────┐
│ REFERENCES │
│ Verify key citations yourself — cited papers are often misrepresented │
└─────────────────────────────────────────────────────────────────────────────┘
│
↓
┌─────────────────────────────────────────────────────────────────────────────┐
│ SUPPLEMENTARY MATERIALS │
│ Raw data, extended methods, additional figures — often where the truth is │
│ Peer reviewers frequently do not scrutinize supplements as carefully │
└─────────────────────────────────────────────────────────────────────────────┘
─────────────────────────────────────────────────────────────────────────────
EXPERT READING STRATEGY: Title → Abstract → Figures/Tables → Methods →
Results text → Discussion → Introduction
─────────────────────────────────────────────────────────────────────────────
The IMRAD structure organizes every primary research paper; reading figures before the results text forces you to evaluate the data independently before the authors' framing shapes your interpretation.
---Diagram 2: The Reproducibility Crisis — Causes and Systemic Remedies
╔══════════════════════════════════════════════════════════════════════════════════╗
║ THE REPRODUCIBILITY CRISIS: CAUSES, AMPLIFIERS, AND SOLUTIONS ║
╚══════════════════════════════════════════════════════════════════════════════════╝
┌──────────────────────────────┐
│ PUBLISHED FINDING │
│ (presented as true) │
└──────────────────────────────┘
│
┌───────────────────────────┼───────────────────────────┐
↓ ↓ ↓
┌───────────────┐ ┌──────────────────┐ ┌──────────────────┐
│ STATISTICAL │ │ METHODOLOGICAL │ │ BIAS & │
│ PROBLEMS │ │ PROBLEMS │ │ INCENTIVE │
│ │ │ │ │ PROBLEMS │
│ • Small n │ │ • No blinding │ │ • Publication │
│ • p-hacking │ │ • No controls │ │ bias (only │
│ • HARKing* │ │ • Poor reagent │ │ p<0.05 gets │
│ • Multiple │ │ validation │ │ published) │
│ comparisons │ │ • Protocol drift │ │ • Pressure to │
│ uncorrected │ │ between labs │ │ publish fast │
│ • Effect size │ │ • Undisclosed │ │ • Career rewards │
│ ignored │ │ exclusions │ │ novelty, not │
└───────────────┘ └──────────────────┘ │ replication │
│ │ └──────────────────┘
└───────────────────────────┼───────────────────────────┘
↓
┌──────────────────────────────┐
│ FALSE POSITIVE INFLATED │
│ INTO THE LITERATURE │
│ (Ioannidis 2005: majority │
│ of findings may be false │
│ in high-bias fields) │
└──────────────────────────────┘
│
↓
══════════════════════════ SYSTEMIC SOLUTIONS ══════════════════════════════
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────────────────┐
│ PRE-REGISTRATION│ │ REGISTERED │ │ OPEN DATA / OPEN CODE │
│ │ │ REPORTS │ │ │
│ Register │ │ Peer review │ │ Raw data deposited in │
│ hypothesis, │ │ BEFORE data │ │ public repositories │
│ design, and │ │ collection; │ │ (e.g., OSF, Zenodo, │
│ analysis plan │ │ acceptance │ │ GitHub). Code shared so │
│ before running │ │ independent │ │ analyses can be re-run │
│ the experiment │ │ of outcome │ │ exactly │
└──────────────────┘ └──────────────────┘ └──────────────────────────────┘
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────────────────┐
│ REPLICATION │ │ LARGER SAMPLES │ │ EFFECT SIZE + CI │
│ STUDIES │ │ + POWER CALC │ │ REPORTING │
│ │ │ │ │ │
│ Independent │ │ Calculate n │ │ Report Cohen's d, │
│ labs attempt │ │ needed BEFORE │ │ η², odds ratio — │
│ exact or │ │ starting, not │ │ not just p-value; │
│ conceptual │ │ after seeing │ │ confidence intervals │
│ replication │ │ the data │ │ show uncertainty honestly │
└──────────────────┘ └──────────────────┘ └──────────────────────────────┘
* HARKing = Hypothesizing After Results are Known — presenting post-hoc
hypotheses as if they were a priori predictions. Inflates false positives.
Ioannidis (2005) demonstrated mathematically that when prior probability of a true effect is low, statistical power is low, and researcher degrees of freedom are high, the majority of statistically significant published findings can be expected to be false — a sobering insight that motivates every structural reform shown here.
---Diagram 3: Experimental Design — Controls, Blinding, and Factorial Logic
╔══════════════════════════════════════════════════════════════════════════════════╗ ║ EXPERIMENTAL DESIGN: THE ARCHITECTURE OF VALID INFERENCE ║ ╚══════════════════════════════════════════════════════════════════════════════════╝ ┌──────────────────────────────────────────────────────────────────────────┐ │ 1. CONTROLS: WHAT EACH TYPE RULES OUT │ ├──────────────────────┬───────────────────────────────────────────────────┤ │ NEGATIVE CONTROL │ Contains everything EXCEPT the variable of │ │ (vehicle / sham) │ interest. Sets the baseline. If your negative │ │ │ control shows an effect, something is wrong. │ ├──────────────────────┼───────────────────────────────────────────────────┤ │ POSITIVE CONTROL │ Known to produce the expected effect. Confirms │ │ (known agonist, │ your assay is working. If positive control │ │ known mutant) │ fails, the experiment is uninformative. │ ├──────────────────────┼───────────────────────────────────────────────────┤ │ INTERNAL CONTROL │ A reference measured within each sample (e.g., │ │ (β-actin in │ housekeeping gene in RT-PCR). Corrects for │ │ Western blots) │ sample loading, quality, and handling variation. │ └──────────────────────┴───────────────────────────────────────────────────┘ ┌──────────────────────────────────────────────────────────────────────────┐ │ 2. BLINDING: WHO KNOWS WHAT, AND WHEN │ ├──────────────────────────────────────────────────────────────────────────┤ │ │ │ UNBLINDED (open-label) │ │ researcher knows treatment → unconscious bias in scoring, exclusions │ │ │ │ SINGLE-BLIND │ │ subject does not know treatment; researcher does │ │ → eliminates placebo confound but not observer bias │ │ │ │ DOUBLE-BLIND ← gold standard for clinical trials │ │ neither subject nor treating researcher knows assignment │ │ → eliminates both subject expectation and observer bias │ │ │ │ TRIPLE-BLIND │ │ statistician also blinded during analysis │ │ → eliminates analytic bias in data handling │ │ │ └──────────────────────────────────────────────────────────────────────────┘ ┌──────────────────────────────────────────────────────────────────────────┐ │ 3. FACTORIAL DESIGN: EFFICIENCY AND INTERACTION DETECTION │ │ │ │ ONE-FACTOR-AT-A-TIME (OFAT) — INEFFICIENT: │ │ │ │ Experiment 1: Drug A alone → effect measured │ │ Experiment 2: Drug B alone → effect measured │ │ ⚠ Cannot detect synergy or antagonism between A and B │ │ │ │ 2×2 FACTORIAL — EFFICIENT: │ │ │ │ │ No Drug B │ Drug B │ │ │ ───────────┼──────────────┼─────────────┤ │ │ No Drug A │ Control │ B alone │ │ │ ───────────┼──────────────┼─────────────┤ │ │ Drug A │ A alone │ A + B │ ← interaction term │ │ │ │ ✓ Tests main effects of A and B │ │ ✓ Tests A×B interaction (synergy, antagonism, additivity) │ │ ✓ Uses same total n as two OFAT experiments — far more informative │ │ │ └──────────────────────────────────────────────────────────────────────────┘ ┌──────────────────────────────────────────────────────────────────────────┐ │ 4. RANDOMIZATION: BREAKING SYSTEMATIC CONFOUNDS │ │ │ │ Without randomization → order effects, cage effects, batch effects │ │ confound your treatment effect. │ │ │ │ Simple randomization: assign by random number generator │ │ Block randomization: ensures balanced group sizes at intervals │ │ Stratified randomization: balance by known covariates (sex, age) │ │ │ │ Rule: randomize at the unit of inference │ │ (individual animal, not litter — litters are not independent) │ └──────────────────────────────────────────────────────────────────────────┘
Valid causal inference requires that every alternative explanation — assay failure, observer bias, interaction effects, and systematic confounds — be ruled out by deliberate design before data collection begins, not corrected for afterward.
---Diagram 4: Grant Writing — The NIH Specific Aims Page Architecture
╔══════════════════════════════════════════════════════════════════════════════════╗
║ THE NIH SPECIFIC AIMS PAGE: ONE PAGE THAT MAKES OR BREAKS THE GRANT ║
╚══════════════════════════════════════════════════════════════════════════════════╝
Reviewers read this first — and some read only this. Every sentence must earn
its place. The entire page is ~600–800 words and follows a strict logic:
┌─────────────────────────────────────────────────────────────────────────────┐
│ PARAGRAPH 1: THE HOOK (~3–4 sentences) │
│ │
│ Opening sentence: the big problem. Bold, specific, compelling. │
│ "Disease X affects N million people and kills Y thousand annually..." │
│ │
│ Sentence 2–3: What is currently known; what the field has tried │
│ Sentence 4: The critical gap — what is NOT known │
└─────────────────────────────────────────────────────────────────────────────┘
│
↓
┌─────────────────────────────────────────────────────────────────────────────┐
│ PARAGRAPH 2: YOUR SOLUTION (~3–4 sentences) │
│ │
│ "Our long-term goal is..." (where your research program leads over 20 yr) │
│ "The overall objective of THIS application is..." (what this grant does) │
│ "Our central hypothesis is..." (specific, testable, falsifiable) │
│ "The rationale for this project is..." (why solving this matters NOW) │
│ "This hypothesis is supported by our preliminary data showing..." │
└─────────────────────────────────────────────────────────────────────────────┘
│
↓
┌─────────────────────────────────────────────────────────────────────────────┐
│ THE AIMS (the structural core — usually 2–4 aims) │
│ │
│ AIM 1 ─── Verb + what you will do + what you will learn │
│ │ "Determine whether X regulates Y by..." │
│ │ Approach in 1–2 sentences. Expected outcome. │
│ │ │
│ AIM 2 ─── Should be ORTHOGONAL to Aim 1 — different technique, │
│ │ complementary question. Not just "do the same thing bigger." │
│ │ Expected outcome. │
│ │ │
│ AIM 3 ─── (Optional) Translational, mechanistic, or therapeutic aim │
│ that builds on Aims 1 and 2 but can stand alone if one fails │
│ │
│ ⚠ CRITICAL RULE: Aims must not be serially dependent. │
│ If Aim 2 requires Aim 1 to succeed, failure of Aim 1 │
│ kills the entire grant. Each aim should be independently viable. │
└─────────────────────────────────────────────────────────────────────────────┘
│
↓
┌─────────────────────────────────────────────────────────────────────────────┐
│ CLOSING PARAGRAPH: IMPACT (~2–3 sentences) │
│ │
│ "These studies are SIGNIFICANT because..." (addresses NIH priority) │
│ "These studies are INNOVATIVE because..." (new concept, tool, approach) │
│ "Completion of these aims will RESULT IN..." (deliverables, field change)│
└─────────────────────────────────────────────────────────────────────────────┘
SIGNIFICANCE + INNOVATION + APPROACH = the three scored sections of the
NIH Research Strategy (R01). The Specific Aims page is the map to all three.
The NIH Specific Aims page distills an entire research program into a single page of ruthlessly logical prose — reviewers use it to assess whether you understand the field, have a testable hypothesis, and have the preliminary data to support it before reading anything else.
---Diagram 5: Research Ethics — The Spectrum from Good Practice to Misconduct
╔══════════════════════════════════════════════════════════════════════════════════╗
║ RESEARCH ETHICS: THE FULL SPECTRUM OF RESPONSIBLE CONDUCT ║
╚══════════════════════════════════════════════════════════════════════════════════╝
┌─────────────────────────────────────────────────────────────────────────────┐
│ FEDERAL DEFINITION OF RESEARCH MISCONDUCT (ORI / 42 CFR Part 93) │
│ Three categories — collectively "FFP" │
│ │
│ FABRICATION ─── Inventing data or results that were never collected │
│ "I made up the Western blot bands" │
│ │
│ FALSIFICATION ── Manipulating materials, equipment, or processes; │
│ changing or omitting data to distort the record │
│ "I stretched the gel image to make bands appear equal" │
│ │
│ PLAGIARISM ───── Appropriating another's ideas, results, or words │
│ without attribution │
│ (includes self-plagiarism — reusing own prior text │
│ without disclosure) │
└─────────────────────────────────────────────────────────────────────────────┘
─────────────── SPECTRUM OF PRACTICES ─────────────────────────────────────
IDEAL MISCONDUCT
PRACTICE (FFP)
│ │
↓ ↓
┌──────────┐ ┌──────────┐ ┌──────────────┐ ┌──────────┐ ┌──────────────┐
│ Rigorous │ │ Honest │ │ QUESTIONABLE │ │ Sloppy / │ │ FABRICATION │
│ methods │ │ reporting│ │ RESEARCH │ │ negligent│ │ FALSIFICATION│
│ pre- │ │ of all │ │ PRACTICES │ │ science │ │ PLAGIARISM │
│ registered│ │ results │ │ (QRPs) │ │ │ │ │
│ open data│ │ incl. │ │ │ │ No │ │ Intentional │
│ │ │ negatives│ │ • HARKing │ │ controls │ │ deception │
│ pre-power│ │ │ │ • p-hacking │ │ n=1 │ │ │
│ calculated│ │ Conflicts│ │ • selective │ │ no stats │ │ Career- │
│ n │ │ of │ │ reporting │ │ │ │ ending; │
│ │ │ interest │ │ • salami │ │ Careless │ │ retraction; │
│ │ │ disclosed│ │ slicing │ │ not │ │ possible │
│ │ │ │ │ • ghost │ │ malicious│ │ criminal │
│ │ │ │ │ authorship │ │ │ │ liability │
└──────────┘ └──────────┘ └──────────────┘ └──────────┘ └──────────────┘
↑
QRPs are widespread and inflate
the false positive rate without
meeting the legal threshold of
FFP — they are the primary driver
of the reproducibility crisis
┌─────────────────────────────────────────────────────────────────────────────┐
│ AUTHORSHIP: ICMJE FOUR CRITERIA (all four required for authorship credit) │
│ │
│ 1. Substantial contribution to conception, design, data, or analysis │
│ 2. Drafting or critically revising the manuscript │
│ 3. Final approval of the published version │
│ 4. Accountability for all aspects of the work │
│ │
│ GIFT AUTHORSHIP ──── listed without meeting criteria (e.g., dept head) │
│ GHOST AUTHORSHIP ─── did the work but is not listed (e.g., pharma writer) │
│ Both are ethical violations under ICMJE guidelines │
└─────────────────────────────────────────────────────────────────────────────┘
The boundary between sloppy science and official misconduct matters legally, but questionable research practices (QRPs) — which fall below that threshold — are responsible for most of the inflated false-positive literature that the reproducibility crisis has exposed.
---Key Takeaways
- The IMRAD structure is not arbitrary — it encodes a logical argument. Introduction establishes why the question matters; Methods documents what was done so others can reproduce it; Results reports what was observed without interpretation; Discussion explains what the results mean and where they fall short. Deviations from this structure are often a sign that authors are obscuring something, such as hiding a crucial methodological detail in supplementary material where it receives less peer scrutiny. Day and Gastel emphasize that clarity of structure is inseparable from clarity of thinking.
- Ioannidis (2005) proved mathematically that most published research findings can be false. Using a Bayesian framework, he showed that the positive predictive value (PPV) of a published finding — the probability it is true given that it was published — is a function of prior probability (R), statistical power (1−β), and significance threshold (α). When R is low (exploratory field), power is low (~0.20 in many biomedical studies), α is the conventional 0.05, and publication bias selects for positive results, PPV drops well below 50%. This is not an attack on any individual researcher; it is a systemic property of the incentive structure of science.
- p-values tell you nothing about effect size, biological relevance, or the probability that the null hypothesis is true. A p-value of 0.03 means that IF the null hypothesis were true, you would see a result this extreme or more extreme 3% of the time by chance — it does not mean there is a 97% probability that your hypothesis is correct. With very large n, trivially small and biologically meaningless effects become statistically significant. Always report effect sizes (Cohen's d, η², odds ratio) and 95% confidence intervals alongside p-values.
- HARKing (Hypothesizing After Results are Known) is a major generator of false positives. When a researcher runs an exploratory analysis, finds a pattern, and then writes it up as if that pattern was predicted in advance, they are performing implicit multiple testing without correction — inflating the false positive rate — while presenting the appearance of confirmatory science. The remedy is pre-registration: publicly logging your hypothesis, design, and analysis plan in a timestamped repository (e.g., OSF.io) before data collection begins.
- Positive and negative controls are not optional — they are the minimal logic of an experiment. A negative control (vehicle, sham, wild-type) establishes the baseline and reveals assay artifacts. A positive control (a known active compound, a validated mutant) confirms the assay is capable of detecting the effect you claim. An experiment without both controls cannot be interpreted. This is not a bureaucratic formality; it is the difference between data and noise.
- Factorial designs detect interaction effects that one-factor-at-a-time experiments fundamentally cannot. A 2×2 factorial experiment with factors A and B, using the same total number of animals as two separate one-factor experiments, tests three things: the main effect of A, the main effect of B, and the A×B interaction. Synergistic or antagonistic interactions — which are often the most biologically interesting finding — are invisible to OFAT designs. This is especially critical in pharmacology, where drug combinations are the clinical reality.
- The NIH Specific Aims page follows a strict rhetorical logic and is the most important single page in a grant application. It must establish: (1) the significance of the problem, (2) the critical gap that this project fills, (3) a specific testable central hypothesis supported by preliminary data, (4) two to four aims that are logically related but not serially dependent (failure of one should not collapse the others), and (5) the innovation and expected impact. The three scored sections of the R01 Research Strategy — Significance, Innovation, and Approach — must each be foreshadowed in the Aims page.
- Federal research misconduct is defined as Fabrication, Falsification, or Plagiarism (FFP), but the reproducibility crisis is driven primarily by Questionable Research Practices (QRPs) that fall below this threshold. QRPs include selective outcome reporting, p-hacking (running multiple statistical tests and reporting only the significant ones), salami slicing (dividing one study into the smallest publishable units), and gift authorship. These practices are widespread, are rarely formally sanctioned, and collectively create a literature systematically biased toward positive, novel, and overstated findings.
- ICMJE authorship requires all four criteria to be met: contribution, drafting/revision, final approval, and accountability. Gift authorship (listing a department head who had no involvement) and ghost authorship (omitting a professional medical writer who drafted the manuscript, common in industry-sponsored trials) both violate ICMJE guidelines. The order of authors encodes scientific contribution: first author did the primary work; last (senior) author led the lab; middle authors contributed in declining order of contribution, with the exception that the second-to-last position is sometimes a co-senior author role.
- The 3Rs principle (Replace, Reduce, Refine) governs ethical animal research and is encoded in IACUC review. Replace: use non-animal methods where possible (cell culture, organoids, computational models). Reduce: calculate the minimum n needed by power analysis before starting — using too few animals wastes lives and produces underpowered results; using too many wastes lives unnecessarily. Refine: minimize pain and distress through anesthesia, analgesia, and humane endpoints. IACUC protocol approval must be obtained before any animal work begins; retroactive approval is not possible.
- In the Leigh and Clayden artificial peptide machine paper (Science, 2013), sequence specificity was achieved by threading building blocks onto a molecular track before ligation — a mechanically imposed ordering, not a template-directed one. This is conceptually distinct from ribosomal peptide synthesis, in which tRNA-delivered amino acids are selected by codon–anticodon complementarity. The Leigh machine demonstrates that sequence information can be stored in the topology of a molecule (the order of units pre-loaded on a rotaxane axle) rather than in a nucleic acid template. This is relevant to thinking about the origin of coded peptide synthesis and what the minimal requirements for sequence control actually are.
- Literature searches require field-specific strategy: PubMed/MEDLINE for biomedical sciences, SciFinder or Reaxys for chemistry, Web of Science for citation analysis. Google Scholar casts the widest net and captures preprints and theses but has no quality filter. Preprint servers (bioRxiv, ChemRxiv) provide access to results before peer review — read with appropriate skepticism and note that preprints are not peer-reviewed. Citation management software (Zotero, Mendeley, EndNote) must be used from the first day of a project; retroactively organizing hundreds of PDFs is a research-hours disaster.
- Scientific writing should default to the active voice in most sentences, despite the traditional culture of passive voice in chemistry. "We measured absorbance at 280 nm" is clearer and more accountable than "Absorbance was measured at 280 nm." Day and Gastel argue that the passive voice systematically obscures who did what and introduces unnecessary ambiguity. The exception is when the action genuinely matters more than the agent ("Samples were stored at −80 °C" is fine). The goal is always clarity, not adherence to a stylistic rule.
- Registered Reports are a journal publication format that offers peer review and in-principle acceptance before data collection, eliminating publication bias for that study. The logic: if a study is designed well and asks an important question, it deserves to be published regardless of whether the result is positive, negative, or null. This breaks the perverse incentive that drives p-hacking and HARKing. Several high-impact journals now offer Registered Reports as a submission track, and replication studies are particularly well-suited to this format.
- Intellectual property in pharmaceutical chemistry is governed by the patent system, which grants a ~20-year monopoly from filing date in exchange for public disclosure of the invention. A compound must be novel, non-obvious, and useful to be patentable. Method of use patents (new indication for existing drug), formulation patents, and process patents are used to extend effective market exclusivity beyond the original compound patent — a practice called "evergreening." Trade secrets, by contrast, provide indefinite protection but offer no legal recourse if independently discovered by a competitor; they are common in manufacturing process chemistry.
References
- Day, Robert A., and Barbara Gastel. How to Write and Publish a Scientific Paper, 8th ed. Cambridge University Press, 2016.
- Burroughs Wellcome Fund / Howard Hughes Medical Institute. Making the Right Moves: A Practical Guide to Scientific Management for Postdocs and New Faculty, 2nd ed. HHMI, 2006. Available at: https://www.hhmi.org/sites/default/files/Educational%20Materials/Lab%20Management/Making%20the%20Right%20Moves.pdf
- Ioannidis, John P. A. "Why Most Published Research Findings Are False." PLOS Medicine 2, no. 8 (2005): e124. DOI: 10.1371/journal.pmed.0020124
- Leigh, David A., Bartosz Lewandowski, Nathalie Bhosale, Leanne D. Sherburn, and Charlie T. McTernan. "Sequence-Specific Peptide Synthesis by an Artificial Small-Molecule Machine." Science 339, no. 6116 (2013): 189–193. DOI: 10.1126/science.1229753
- International Committee of Medical Journal Editors (ICMJE). "Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals." ICMJE, updated 2023. Available at: http://www.icmje.org/recommendations/
- Nosek, Brian A., et al. "Estimating the Reproducibility of Psychological Science." Science 349, no. 6251 (2015): aac4716. DOI: 10.1126/science.aac4716
- Chamberlin, Thomas C. "The Method of Multiple Working Hypotheses." Science 15, no. 366 (1890): 92–96. (Classic methodological essay on orthogonal experimental approaches; reprinted in Science 148 [1965]: 754–759.)
- Russell, William M. S., and Rex L. Burch. The Principles of Humane Experimental Technique. Methuen, 1959. (Original formulation of the 3Rs framework for animal research ethics.)
- U.S. Department of Health and Human Services, Office of Research Integrity (ORI). "Definition of Research Misconduct." 42 CFR Part 93. Available at: https://ori.hhs.gov/definition-misconduct
- Open Science Collaboration. "Estimating the Reproducibility of Psychological Science." Science 349, no. 6251 (2015): aac4716. DOI: 10.1126/science.aac4716
- Simmons, Joseph P., Leif D. Nelson, and Uri Simonsohn. "False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant." Psychological Science 22, no. 11 (2011): 1359–1366. DOI: 10.1177/0956797611417632
- Chambers, Christopher D. "Registered Reports: A New Publishing Initiative at Cortex." Cortex 49, no. 3 (2013): 609–610. DOI: 10.1016/j.cortex.2012.12.016