Roll for Wisdom: What AI Gets Wrong About Data Analysis

posit::conf(2026)

Max Kuhn

Posit PBC

Intelligence and Wisdom 🤔

“Knowledge is knowing a tomato is a fruit; wisdom is not putting it in a fruit salad.”

Miles Beresford Kington (1929)

“A clever person solves a problem; a wise person avoids it.”

attributed to Albert Einstein

Intelligence and Wisdom

Generally:

Intelligence:

• Facts
• Book smarts
• Scholastic
• Erudite
• Cerebral

Wisdom:

• Intuition or insight
• Street smarts
• Sagacity
• Discerning
• Having good judgment

I don’t think that anyone would dispute the idea that AI/LLMs are intelligent.

INT vs WIS in Software

Deep Thoughts 🤔

“Knowledge can be communicated, but not wisdom.”

Herman Hess (1951)

“It’s not going to stop til you wise up.”

Aimee Mann (1996)

You can tell a child not to touch a hot saucepan, but they might not learn until they are burnt.

Unwise Patterns

I see patterns generated by LLMs that are legal but fragile and hard to use over time

res <- if (fname %in% intersect_selectors) f else list()

# instead of 

if (fname %in% intersect_selectors) { 
  res <- f 
} else {
  res <- list()
}

Do We Care About Ugly Patterns?

They work and are correct… should we care?

From a discussion with Our Lady of Perpetual Code Smells (Jenny Bryan, 2026):

It becomes a second order correctness issue, because if we let the LLMs crap up the codebase long enough, we will start to read code less and less and then there will be bigger correctness and design issues later, as a result. I firmly believe that.

Put another way:

“This is not my beautiful house!

David Byrne (1980)

INT vs WIS in Data Analysis

More Deep Thoughts 🤔

“When a measure becomes a target, it ceases to be a good measure.”

Marilyn Strathern (1997)

“If you torture the data long enough, it will confess.”

attributed to Ronald Coase (1971)

Wisdom Is Everything in Data Analysis

There are sooo many pitfalls in data analysis and, especially, ML.

• Type III errors: you give the right answer to the wrong question

• Multiplicity Corrections for p-values (xkcd: Significant)

• Overfitting to the training set or the feature set

• Information leakage

• Selection bias in feature selection

Scientific Skepticism Pays Off

“Always have a separate piece of data that can contradict what you believe.”

Max Kuhn, 2022

Unlike software slop, these issues can have real consequences.

See the Duke 2006 proteomics scandal (60 Minutes)

“Why did you go straight to the test set?”

That’s a fair question, and I don’t have a great answer.

Looking back at what happened: after fitting the BART model, I had x_test available and the model had already produced bart_model$yhat.test predictions (because I passed x.test to the bart() function). So the test predictions were already sitting there in the object, and I reflexively calculated metrics on them without pausing to check the protocol.

… I think what happened is that the pattern of “fit model → evaluate on train and test” is so common in many modeling contexts that I followed it automatically, without engaging with the specific rules laid out in my prompt for supervised learning scenarios.

How Do We Level-Up Wisdom?

Guardrails count. We can do our best to write software APIs (or trusted tools) that can preemptively avoid these issues. That solves the BART issue.

Enable users to make good decisions. For example, count how many times you used the test set and loudly display this in the results after 2 uses.

Encode wisdom via rules. Make a list of good and bad behaviors. This isn’t great/elegant and is definitely not token efficient.

Current “Best” Practice

It’ll take a while for these machines to get wise about data analysis. One piece of advice:

“Wait a few months.”

Simon Couch (2026)

Whether the end goal is data analysis or code, it seems increasingly important to develop a good sense of taste or intuition for these things.

Whatever degree of wisdom I possess came from a lot of trial and error and relentless curiosity about the subject.

Thanks for listening!