The Negative Feedback Loop of AI Summaries

I've already ranted about how using Large Language Models (LLMs) - what passes for "AI" these days - to replace entry level employees will disrupt the very talent pipeline used to create the experienced senior employees used to generate and curate the enormous (terabytes to petabytes) quantities of data used to train the LLMs in the first place. LLMs are merely gigantic "autocomplete" programs making statistical guesses based on their training data. That's why I say this effort isn't sustainable.

But there's another negative feedback loop in the use of LLMs that I just became aware of. Various web and social media tools are starting to provide "AI summaries" in response to user queries. You've probably already experienced these, and have seen that these LLM generated summaries range from usably good or laughably bad.

Here's the problem: studies have shown that between 80% to 90% of humans making queries for which there are AI summaries never go past the summary. They never click on the web links leading to the data on which the summary is based (if such links are even made available). This is in stark contract to conventional web searches, in which the web links are the result of the search, and the user almost inevitably clicks on the link to get the answer for which they were searching.

Because the user never visits the source web page, they never see the advertising used to pay for the generation of the web page. The web site is visited perhaps once and only once, by the LLM web crawler, and never by a human being. This destroys the business model used to pay for the web site in the first place. So the use of AI summaries will eventually result in the loss of the very data used to create the summary.

The only solution I see to this is to paywall all of the news and data sources being used by the AI summary algorithms. Lawsuits are already in progress against the AI companies extracting copyrighted data from advertising supported web sites. Clearly copyrighting the web site alone isn't sufficient to keep their value from being extracted without payment.

Sources

Sabine Hossenfelder, "AI Is Breaking The Internet As We Know It", Backreaction, 2025-12-14, http://backreaction.blogspot.com/2025/12/ai-is-breaking-internet-as-we-know-it.html

Real-Time versus Real Time

Interesting article from IEEE Spectrum: "How AI’s Sense of Time Will Differ From Ours" [Popovski, 2026-08-13].

Human cognition integrates events from different senses - especially seeing and hearing - using a temporal window of integration (TWI). Among other things, it's the ability that lets us see continuous motion with synchronized sound in old school films at 24 frames per second. But under the hood, everything is asynchronous with different sensing and processing latencies. Which is why we don't automatically integrate seeing distant lightning strikes with the thunderclap, even though intellectually we may know they're the same event.

Machines have to deal with this as well, especially AI in applications like self-driving vehicles. It's non-trivial. "Computers put timestamps, nature does not" as the author remarks. Anyone that develops real-time software - or has spent time analyzing log files - has already had to think about this. I talked about this issue in a prior blog article: "Frames of Reference".

I've also pointed out in a prior article, "Frames of Reference III", that our human sense of simultaneity continuously gives us a false view of reality. If I look towards the back of my kitchen, I see the breakfast table and chairs a few feet away. Since light travels about a foot per nanosecond, I'm actually seeing events that occurred a few nanoseconds ago (plus the communication and processing latency inside me). The back yard that I can see through the window: a few tens of nanoseconds ago. The house across the street: a hundred nanoseconds ago. The mountains to the west: microseconds ago. If I can see the moon on a clear evening: over a second ago. I see all of these things as existing in the same instant of time, but nothing could be further from the truth; my perception is at best an ensemble of many instants in the past, and the present is just an illusion.

AI perception of the real-world will have similar complications.

What do we mean when we say we don't know how Large Language Models work?

Large Language Models - what passes for "AI" these days (there are other kinds, but this is what most people today mean when they use the term) - are in effect gigantic autocomplete algorithms, implemented using a technique called "machine learning", which is based on artificial neural networks (from how we currently believe the brain works), scaled up to use trillions of parameters, computed from terabytes of training data, much of which is copyrighted and used without the creators' permission. An LLM produces the output that its algorithm deems the most likely to be a response to your input prompt, based on its model of that training data. If that output represents actual truth or facts, it's only because the training data made that seem probable.

LLMs "hallucinating" isn't a bug; it's fundamental to how they operate.

I've read several articles on LLMs whose basic theme is "no one knows how LLMs work". This is true, but probably not in the way that most people think. The LLM developers that work for the AI companies know exactly how the software algorithms work - it's not only just code, it's code that they for the most part wrote. It's the trillions of parameters, derived algorithmically from the terabytes of training data, the is the big mystery.

Imagine a vast warehouse, on the scale of the scenes at the end of Citizen Kane or Raiders of the Lost Ark. That warehouse is full of file cabinets. Each file cabinet is full of paper files about every person that has ever lived in the United States, for as long as the U.S. Government has been keeping records. Your job: tally the number of people in those files whose first name ends in "e", who had a sibling whose first name ends in "r".

You understand the job. The task is straightforward. The algorithm you could use to accomplish this is obvious. But could you do it? No. The dataset is too ginormous. You literally won't live long enough to get it done, even if you could maintain your interest.

But if all that information were to be digitized, stored in a huge database, the database indexed to link records of family members together, and a program written to answer the original question, a computer could come up with the answer after a few minutes. These kinds of mundane repetitive tasks are what computers excel at.

(This isn't the perfect metaphor but it's the best I've got at the moment.)

LLMs are more complicated than that, and more probabilistic, but it's the same idea. We understand how the code part of the task works. But it's the data, the artificial neural network and its implications, we don't understand. We can't understand. Not just the training data - which is far too much for us to read and digest - but the interconnections between the trillions of parameters that are formed and the statistical weights that are computed as the training data are processed.

If someone asks "How did the AI come up with that response?", that's the part we have to say "We don't know." The artificial neural network is just too big, and stepping through it manually, tracing every single step the algorithm made, while technically not impossible, is just too tedious and time consuming. And relating the parameters and weights of the neural net back to the original training data would be like trying to unscramble an egg.

Knowing how the code works will get more complicated as we use LLMs themselves to revise or rewrite the code. This isn't a crazy idea, and if it's not happening now, it will happen, perhaps soon. And then the code, the part we thought we knew how it worked, will evolve such that we no longer know how it works.

Admittedly, artificial neural network based machine learning models aren't my area of expertise. But I'm not completely ignorant about how they work. I think there are myriads of applications for them. For example, I think we'll use them to discover new drug pathways, just waiting to be found in existing voluminous clinical datasets (although any such results will have to be carefully experimentally verified by human researchers). But I'm becoming increasingly skeptical about the more grandiose claims made for them - sometimes by people who should know better.