The Replication Crisis Is Real. This Column Uses It to Dismiss a Discipline

Analyzing: The Trouble with the Soft Sciences — James Lileks · 2026-08-16

What the Editorial Argues

Lileks argues that sociology—and the “soft sciences” broadly—produce knowledge of unreliable rigor. He builds the case on two pillars: the replication crisis and the example of Jason Arday, a Cambridge professor of sociology of education accused of plagiarism. From those premises, he generalizes that the discipline’s output is “vacuous nonsense,” with jargon functioning less as description than as a signal of ideological conformity. He concedes that human society is worth studying, but argues that the modern university’s culture makes such study unreliable; the humor-column register makes the dismissal entertaining.

Receipts

The column takes genuine failures in academic research and converts them into a discipline-wide verdict, while omitting the field’s documented efforts to detect and correct those failures.

What the framing wants you to believe

• That sociology produces no reliable knowledge—“vacuous nonsense,” words that “do not signify facts, or even an idea.”
• That the replication crisis invalidates the field wholesale rather than exposing specific methodological weaknesses.
• That jargon is evidence of intellectual bankruptcy, and that one plagiarism case reveals the field’s entire moral and intellectual condition.
• That the reader’s common sense is a more trustworthy guide than the methods and evidence of people who study society professionally.

What’s really going on

• The replication crisis is real. The Open Science Collaboration’s 2015 Science study reproduced 36 of 100 psychology studies. But the column uses that fact as a blanket instrument while omitting the reform movement it helped produce: preregistration, registered reports, open-data practices, and other efforts to improve reproducibility. The core omission is the difference between a field that has methodological failures and a field incapable of recognizing or correcting them.
• The Arday example is one case. The leap from that case to “it’s all vacuous nonsense” is [bf_catalog: hasty_generalization], not an inference established by the evidence offered.
• The column presents an impenetrable academic passage as representative—“It always sounds like this”—without anchoring the passage to a specific published source. A constructed or decontextualized parody is not a sample of a discipline; it is a [bf_catalog: strawman].
• The protected beneficiary is the column’s anti-academic audience-management product: readers are given permission to dismiss research that might complicate their preferred views on inequality, health disparities, criminal justice, or labor markets. The normalized harm is epistemic: skepticism about bad research is collapsed into permission to disregard an entire field.

The Operation

This is not a documented think-tank campaign or a coordinated messaging operation. It is a signed humor column delivering a product to an audience that already wants confirmation. That distinction matters: no memo or funding chain should be invented here. The absence of a formal coordination document does not make the techniques less real. I have built segments on this template—find the genuine problem, turn it into a club, choose the most absurd-sounding example, and never engage the field’s strongest work because the strongest work does not produce the laugh.

Institutional authorship and placement chain. The author is James Lileks, a signed National Review columnist writing in a cultural-commentary register. The placement chain is the National Review opinion ecosystem: a venue whose audience is already primed to treat universities and social-science expertise as ideological institutions. The column’s institutional author is therefore not a university, think tank, or documented coalition. It is a writer and outlet converting audience suspicion into a recognizable editorial product.

The pose is part of the product. “From an outsider’s ignorant perspective, I admit” sounds like a limitation, but functions as a credential. The credentialed insider is presumed compromised; the outsider is presumed able to see what experts cannot. The admission of ignorance does not narrow the claim. It licenses the claim.

That is the common-sense outsider pivot, catalogued in the NR Editorial Technique Catalogue §4.10. It relabels the writer’s distance from the subject as independence and relabels expertise as institutional contamination. The technique is not “the author is secretly stupid.” The public-record claim is narrower: the column uses performed outsider status to make non-specialist judgment feel superior to specialized inquiry.

Distributional impact.

• Beneficiaries: readers whose distrust of academia is confirmed; the outlet, which receives attention and identity reinforcement; and the broader anti-academic political project, which benefits when citizens treat social-science findings as ideological noise rather than evaluating them case by case.
• Cost-bearers: the public’s ability to distinguish “this research has serious methodological problems” from “this field produces nothing of value.” Researchers and readers also bear the cost when a varied body of work is reduced to a single caricature.
• Magnitude: the column supplies no basis for measuring the downstream policy effect. The supported finding is about the information environment: it makes wholesale dismissal easier without establishing a wholesale case.

Alternative design. An honest version would have done four things: identify the specific replication evidence and its limits; distinguish psychology from sociology rather than treating “soft sciences” as one object; cite the academic passage it presents as representative; and describe the methodological reforms that followed the crisis. It could then examine the Arday allegations with the relevant documentation and resolution, rather than treating an incompletely specified case as a closed verdict. That piece could still be severe. It would simply have to survive contact with the field’s best evidence.

The column instead declares that “the words do not signify facts, or even an idea.” Once that premise is installed, no research can answer the charge, because the research has been disqualified before it appears.

FGL analysis.

• The author: fear of irrelevance in a crowded commentary market; greed for attention and reader loyalty; and the ordinary laziness of choosing the vivid example over the representative one. These are analytical possibilities, not findings about private motive.
• The apex beneficiary: the author and the outlet benefit from a frame in which expertise is suspect and ridicule is evidence of independence. The more academic institutions are treated as inherently fraudulent, the more authority migrates to the commentator who has declared himself free of them.
• The rank-and-file reader: the reader’s fear is being duped by institutions that claim authority. The reader’s laziness is human: reading a witty dismissal is easier than investigating methods, samples, and replication records. The column offers something more than convenience—it offers recognition. Laughing at the right passage becomes proof of belonging to the group that “sees through” the experts. The reader is not contemptible for wanting that reassurance. The operation is built to make reassurance feel like inquiry.

The placement is mixed in stated rationale, selfish in operation. Concern about research quality is legitimate. The distribution of attention and authority is not neutral: the column turns that legitimate concern into a permission structure for blanket dismissal.

Technique inventory.

1. Hasty generalization — Bad-Faith Techniques Catalog.
   The textual cue is the movement from one plagiarism case and a replication statistic to the conclusion that sociology is “all vacuous nonsense.” The operation substitutes a whole-field verdict for a demonstrated base rate. One case tells us about one case; a field-wide misconduct rate would require field-wide evidence.

2. Strawman — Bad-Faith Techniques Catalog.
   The cue is the claim that the presented jargon is what sociology “always” sounds like, without a specific published source for the passage. A parody of an academic sentence is made to stand for a discipline’s actual literature. The reader laughs at the caricature and is spared the harder task of asking whether the caricature is representative.

3. False dichotomy — Bad-Faith Techniques Catalog.
   The column treats the choice as either plain, common-sense knowledge or ideological academic jargon. It omits the obvious third position: social research can be empirically valuable while containing bad studies, contested theories, unclear prose, and institutional incentives that require scrutiny. The technique makes criticism of the field appear equivalent to rejection of the field.

4. Advantageous comparison — Bandura’s mechanism of advantageous comparison.
   The Lysenko comparison moves the reader from “some research is unreliable” to “this field is political enforcement masquerading as science.” Lysenkoism involved state-enforced rejection of Mendelian genetics in the Soviet Union. The historical record supports what Lysenkoism was; it does not, by itself, support the analogy to contemporary sociology. The comparison supplies a terminal moral image far more extreme than the evidence in the column.

5. Frame-engineered relabeling — Bad-Faith Techniques Catalog; Luntz and Lakoff lineage.
   “Soft sciences” marks the field as lesser before the evidence is examined. Academic expertise becomes “credentialism”; technical vocabulary becomes “thin broth bobbing with wads of gristly twaddle”; peer review becomes orthodoxy. Both Luntz’s documented message-discipline work and Lakoff’s account of cognitive frames explain the operation: change the vocabulary, and the audience encounters the subject inside a different moral frame.

6. Distortion of consequences — Bandura’s mechanism of distortion of consequences.
   The column gives the reader permission to dismiss a field without naming what the field’s research actually bears on. It does not weigh the consequences of discarding useful findings alongside the consequences of retaining bad ones. The absence is functional. Once the policy and human stakes disappear, dismissal feels costless.

The audience-management function is a cluster: identity confirmation, grievance ratification, status display, and permission structure. The reader is told that the academic establishment produces nonsense, that their suspicion was correct, that recognizing nonsense makes them more discerning than credentialed experts, and that they may disregard inconvenient findings without doing the underlying evaluation.

The Record

Anchor receipts and verdicts

• The replication crisis in psychology — Tier 1. The Open Science Collaboration, “Estimating the Reproducibility of Psychological Science,” Science 349, no. 6251 (2015), DOI 10.1126/science.aac4716, attempted to reproduce 100 psychology studies and reported successful replication for 36. This is a real and significant methodological failure. It does not establish that sociology as a whole produces no reliable knowledge.
• The reform response — Tier 1, at the general level. The replication crisis produced documented reforms including preregistration, registered reports, open-data practices, and institutional work associated with the Center for Open Science. Any account that cites the crisis while omitting the corrective response is materially incomplete. The exact scope and effectiveness of each reform require separate evaluation; their existence does not.
• Lysenkoism — Tier 1 for the historical description; unconfirmed as an analogy. The history of Trofim Lysenko’s rejection of Mendelian genetics under Soviet state enforcement is documented in scholarship including William deJong-Lambert’s work on Lysenkoism and Loren Graham’s histories of Soviet science. The column’s analogy between that history and contemporary sociology is an editorial frame, not a fact established by the historical record.
• Jason Arday — partially documented, with the column’s specific allegations unconfirmed here. His appointment as a professor of sociology of education at Cambridge is documented in the source packet. The particular plagiarism allegations, the formats and passages allegedly involved, the method by which they were identified, and their disposition are not sufficiently anchored in the material available for this analysis. The column presents the case as an established “unmasking”; the reader is entitled to the underlying complaint, the response, the investigative method, and the resolution before treating that characterization as settled.

The column’s load-bearing omissions

• It does not distinguish sociology from social psychology, the field in which the cited replication evidence is most directly established.
• It does not identify the empirical literature it is dismissing or explain which methods, subfields, or findings supposedly fail.
• It does not describe the methodological reforms prompted by the replication crisis.
• It offers no properly sourced basis for treating one plagiarism allegation as representative of an entire discipline.
• It presents a jargon passage without identifying its publication, author, context, or representativeness.
• It treats a field’s capacity to expose and correct error as evidence against the field rather than as part of the record of science functioning.

Per-citation and framing verdicts

• Replication statistic: accurate if presented as 36 successful replications out of 100. If presented instead as a direct measure of sociology’s failure, the statistic is misapplied: it concerns a defined psychology replication project, not the entire social sciences.
• Arday case: the column’s broad presentation is unconfirmed from the available receipt set. The allegations may be real; the details and resolution must be documented before they can bear the weight assigned to them.
• Lysenko comparison: editorial analogy, not documentary fact. The history invoked is real; the equivalence is not established by the evidence offered.
• Representative academic passage: unanchored unless the column identifies the published source. “It always sounds like this” is a claim about a population, and the column supplies no sample.

Missing-information declaration. I do not have a Tier-1 or Tier-2 receipt sufficient to settle the specific Arday allegations as the column uses them. I have no basis to invent the complaint’s wording, the alleged passages, the investigative process, or the final disposition. That gap limits the factual verdict on the case, not the technique finding: the column uses an insufficiently specified individual case as a warrant for a discipline-wide conclusion.

How to Recognize This

The pattern is simple: a real methodological failure is cited without the field’s corrective response; one scandal is made representative; a jargon passage is stripped of source and context; and humor turns the reader’s laughter into a verdict.

The mechanism. The operation converts reasonable skepticism into wholesale dismissal. You may already know that some academic prose is impenetrable. That true observation is used to smuggle in a much larger claim: that the field produces nothing of value. The replication crisis, which is also a story about research communities detecting error and building mechanisms to correct it, is reframed as proof that correction is impossible. The reform response is the fact most necessary to test the column’s conclusion—and the fact the column leaves out.

Watch for these signals:

• One case becomes the category. A single accusation, study, institution, or scandal is presented as evidence about an entire discipline or profession. Ask for the denominator.
• The outsider disclaimer does the credentialing. Phrases such as “from an outsider’s perspective” or “I admit my ignorance” appear before sweeping claims. Check whether the disclaimer is being used to narrow the claim or to immunize it from expertise.
• A parody is offered as a sample. If the piece says a field “always sounds like this,” find the cited publication. If there is no source, the passage may have been constructed to maximize ridicule.
• Failure is named; correction is absent. When a critique cites the replication crisis, ask what reforms followed. When it cites misconduct, ask how the institution investigated and corrected it. A critique that treats self-correction as irrelevant is hiding the most important part of the record.
• The comparison jumps to a civilizational villain. A technical problem becomes Lysenkoism, indoctrination, fraud, or total institutional corruption. That is often [bf_catalog: advantageous_comparison] or [bf_catalog: false_dichotomy doing the inflation.

It works because the reader’s experience is real. Bad writing exists. Institutions protect reputations. Some studies fail to replicate. Some scholars plagiarize. The column takes those truths, stops before the qualifying evidence, and lets the audience mistake recognition of the true half for proof of the whole.

When you see the pattern, separate the claims. Verify the case. Check the source of the supposedly representative example. Look for the field’s corrective work. Ask who benefits when a complex body of research is reduced to a joke, and what evidence would be required to justify the larger conclusion.

The replication crisis is real. Academic misconduct is real. This column’s operation is to make those facts carry a conclusion they do not establish. The reader carries that distinction forward.