Find the Strongest Argument Against Your Favorite Idea

Give your plan a fair challenge before committing more to it.

You have found an AI tool that looks perfect for your work. The demonstration is impressive. The testimonials sound familiar. You can already imagine the time it will save.

Then you ask an assistant, “Explain why this is a great investment.”

It does exactly what you asked.

The missing task is examining the strongest reason the investment might disappoint you. A good objection does not need to destroy the plan. It may reveal a smaller, cheaper, or more useful way to try it.

The aim is to improve the decision by making your belief answerable to evidence.

State the belief and its basis

Consider a fictional organizer who wants to use AI for every weekly report. Her belief is: “This will reduce the time required to produce reports we can actually use.”

Her evidence so far is that first drafts appear quickly. That is relevant, but incomplete. The work also includes collecting inputs, checking numbers, correcting errors, and obtaining review.

Write down the belief without decoration. Then list why you currently believe it. Separating those two parts helps reveal whether you have evidence about the whole job or only its most visible stage.

An assistant can help identify the gap:

My preferred conclusion is below. Identify which parts my evidence supports and which remain assumptions. What is the strongest relevant reason this plan could fail?

Decide what would change your mind

Before a trial, identify observations that would weaken the plan.

For the reporting example, the organizer might decide that the tool needs revision if it increases total work time, repeatedly changes key figures, or requires more review than the team can provide.

She can also name evidence that would strengthen the case: comparable reports completed with fewer total minutes, acceptable accuracy, and a clear handoff to reviewers.

These are proposed local criteria. They are not universal thresholds or a guarantee that a small trial will establish a precise effect.

A written criterion makes it harder to redefine success after seeing a result you dislike.

Learn from a real negative finding

METR's July 2025 study examined 16 experienced open-source developers completing 246 tasks in repositories they knew well. Tasks were assigned to allow or disallow AI assistance. In that setting, using the early-2025 tools increased completion time by about 19 percent. The developers nevertheless believed AI had helped them work faster. METR: Early-2025 developer productivity study.

That finding is a useful challenge to the assumption that a tool which feels helpful must save time. It is not a measured estimate for community reports, student essays, every programmer, or today's tools.

The applicable lesson for the fictional organizer is to measure the whole task. The study's percentage should not be pasted into her forecast as though her work had been tested.

Read the update as carefully as the original

In February 2026, METR reported problems interpreting a later experiment. Participant and task selection, changed pay, and difficulties measuring work with concurrent agents weakened the estimate. The researchers thought developers were likely benefiting more than in early 2025, but described the data as weak evidence for the size of that change. METR: Changing the experiment design.

A May 2026 METR survey then examined self-reported AI use and productivity among 349 technical workers. A survey of perceived gains answers a different question from a randomized task-time experiment. It should not be presented as a direct replacement measurement of the earlier effect. METR: Early-2026 technical-worker survey.

This sequence rewards careful reading. An old result can remain valid for its original setting while becoming a poor guide to a changed situation. A new result can be relevant while still carrying limitations.

Ask for a strong objection, not a caricature

An unhelpful objection says, “AI is bad and nobody should use it.” That is too broad to test against the reporting plan.

A stronger objection says:

The apparent time saving may disappear when input preparation and factual review are included, especially if the reports contain many figures that require checking.

This objection identifies a mechanism and a measurement. It suggests a trial: record total time, not just generation time, and track the corrections needed.

Ask the assistant to state the objection in a form a thoughtful supporter could recognize as fair. Then inspect the evidence behind it. A vivid imagined failure is not the same as a documented one.

Compare alternatives that could actually work

The choice may not be “use AI for everything” or “use no AI.” The organizer could use it for structure and plain-language editing while calculating figures in a spreadsheet. She could use it only for reports with stable templates. She could decide that one short report is quicker to write directly.

Ask:

Compare the original plan with a narrower version and our current process. What observation would distinguish them on the criteria we care about?

For a modest pilot, choose several representative reports or practice cases, use the same quality checks, and record full task time and corrections. Avoid repeating the identical task in a way that gives the later method an unfair advantage from prior learning.

A small local comparison is useful operational evidence. It does not automatically support a universal causal claim or a precise statistical conclusion.

Avoid artificial balance

Fairness does not require giving every objection equal weight. A directly relevant, well-documented failure deserves more attention than a vague rumor. A concern contradicted by strong evidence should not remain on the list merely to create symmetry.

Evaluate relevance, method, independence, and uncertainty. Ten enthusiastic anecdotes do not necessarily outweigh one careful measurement, but one careful measurement in a different setting may not answer your local question either.

The question is what the evidence establishes about the decision in front of you.

Students: seek the passage that complicates your thesis

For a permitted research assignment, ask:

What is the strongest objection to my thesis that is supported by the assigned readings? Identify the relevant passage and explain whether it challenges my evidence, interpretation, or values.

Read the passage yourself. Do not ask the assistant to manufacture a weak opponent so your argument looks strong.

You may keep the thesis, narrow it, or change it. A useful revision often turns “always” into a claim about a particular situation that the evidence can support.

Your exercise

Write a short proposal for an AI pilot. State your preferred outcome, one strong reason it might fail, and one limitation of the negative evidence you found.

Add a brief pre-mortem: imagine the pilot disappointed you and name three plausible paths to that result. For each, identify an early signal you could observe.

End with a revised decision note: proceed, narrow the pilot, gather a specific missing fact, or stop. Explain what changed and why.

Completion check: Your final judgment reflects the quality and relevance of evidence, and you can name what would cause another update.