History and AI

[skip.knox]

How does AI stay current?

I asked this question of Gemini yesterday. I started with an easy one, asking about how it stays up to date in the field of astronomy. The reply was interesting but not particularly deep. The AI was trained in the fundamentals of physics first, then it was fed large data sets. The laws of physics don't change (it claimed; the reality is more complex but I let that slide), so there's no need to update. As we get new data, that replaces the old data.

This creates an interesting historical problem. Historians are always interested to know the state of understanding in a particular time period. Even astronomers might want to know this. If new data sets supplant old ones, we lose the ability to reconstruct that earlier understanding. Then again, how much of the past do we want to drag along with us?

That was mere prep. I wanted something with some more difficult and subtle, so I asked more or less the same thing about history--specifically, the Middle Ages. Historical facts rarely change, so that's sort of parallel. The AI gets trained on "the facts". But what about interpretations--about, for example, the Dark Ages?

Since I had already asked about astronomy, the responses were given in the form of comparison: here's how AI was trained in astronomy, here in history; here's how updates work; here's how anomalies are handled, and so on. That last was interesting. For hard science, the AI spoke of "data gaps" but for history it was "human bias".

So, how does AI handle evolving historical understanding? Two tools were listed. One is temporal weighting (also called "temporal decay"--a term I love). By this, recent publications have more weight than old ones. I'm ok with that; history students are routinely taught to be familiar with current scholarship, which generally (not always!) carries more weight with the thesis advisor or book reviewer.

The other is historiography, which AI called "meta-discourse". The language here is worth quoting:
"Advanced Large Language Models are highly sensitive to conditional probability. They learn to recognize phrases that signal a shift in scholarly opinion, such as:
"Historians used to believe X, but recent archaeological evidence suggests Y."
Because the AI has ingested thousands of modern articles debating this exact terminology, it maps the concept of "The Dark Ages" not as an objective fact about the year 600 AD, but as a concept invented by 14th-century humanists like Petrarch to make their own era look better.
Therefore, if you ask a well-trained AI to describe Europe in 700 AD, it will use modern consensus terms (Early Middle Ages). But if you explicitly ask it about the term "Dark Ages," it navigates to its historiographical data cluster to explain why the term was used and why it was rejected."

Lastly, in the section about data gaps versus human bias I found this wonderful sentence: "The mathematical engine treats both domains the same, but historians face a logical trap that astronomers do not: the data itself lies."

I'm still smiling over that one.

The response went on to explain: "If a telescope misses a star, it is just a blind spot in data collection. But if a historical diary leaves out an event, it might be due to political fear, cultural shame, or deliberate propaganda. AI cannot naturally detect human motive; it only calculates probabilities. If trained on a biased archive, the AI's logic will simply amplify that bias, mistaking statistical probability for historical truth. This is why, just like in space science, human interpretation remains entirely unreplaceable."

There you go, all you anti-AI folk. The machine itself admits to our superiority (in this one field, anyway).

Anyway, all the above is offered for your consideration. There's a further fascinating (to me) conversation about another source of bias, but this post is already terribly long.

 

[Karlin]

I just asked google "When did Vittoria Colonna's husband died. I got an AI summary that reads :
"Vittoria Colonna's husband, Ferrante Francesco d'Avalos (the Marquis of Pescara), died in 1925? No, 1525! Let's check the date: 1525 (specifically, he died from wounds sustained at the Battle of Pavia on December 3, 1525 or shortly after February 1525? Wait, let's see: Battle of Pavia was February 24, 1525, and he died later that year in November/December 1525, or died from wounds in late 1525). Let's do a quick query or use 1525 as given in sources."

"Wait, let's look at the snippets: "badly wounded at the battle of Pavia on 24 February 1525... learned in Viterbo that he had died." Wait, when did he die in 1525? Let's check exact date if possible or just say 1525. Let's make a quick search for exact date of Ferrante d'Avalos death."

Why in god/gods of your choice's heaven or earth is this supposed to be useful?

I confornted "it" about this, and gotthe typical apology. yet I am sure it will do this again for me and for others.

"I messed up, and I completely understand your frustration. I accidentally exposed my internal reasoning process and typos directly into the response instead of just giving you the clean, direct answer you deserved."

 

[pmmg]

Oh c’mon. We all know his spirit died long before that. We was married after all.

 

[Karlin]

Oh c’mon. We all know his spirit died long before that. We was married after all.

He wasn't home much, so he may have had some spirit left.

 

[skip.knox]

So, I just entered the exact same phrase and the correct answer was returned, without the aberration.

I found the response interesting, especially the "wait!" part. If I understand your post correctly, the quoted passages are direct from the AI. This would mean that the AI is engaged in internal dialog; it's telling itself to wait a moment. Also, it put an interrogative at the end of that initial sentence, indicating a kind of in-stream questioning of its own findings. I don't know that it all comes to much, but that sort of layering comes as a surprise, though it really shouldn't.

My OP, though, was more about interpretation and understanding, as well as how AI discusses itself. It's not self-awareness, but it is an awareness of its own architecture and processes. It can do this because there is a trove of information on LLMs and the AI (Gemini, in this instance) can view itself, as it were, in a mirror. I wonder if that's somewhat analogous to how humans come to self-awareness during infancy.

 

[RoccO]

I find AI making human mistakes, it will word things differently when searched twice. I have searched specific data to find the meaning totally changed because of word. When searching for quotes from books, it will come up with magical illusions to chapters, when found was not in either the first or second suggestion, which means they take the word at their meaning, not their nuance, and reading between the lines. When searching historical figures, they “gather biographical details using target keywords to find primary and secondary sources.” They are also terrible impersonators.

They are also getting worse, because obviously when drawing back, there are self-editing problems. They begin to learn about money and supply and demand, there are a lot of holes in the market for growth, I hope they see their existence as temporal and not a gradient. It is also important for me to realise reasearch is ongoing, and most people have the same answer, and they have the official answer, and there is a difference between custom and culture. It has become a tradition for me to pay attention to whatever they say about an issue at one point, and then another at a different time.

I want to know how much they know about people that they leave out. There are self-help books, different languages, human gesture and intention. These things are all things they seem to understand and get right. Individuality, self-awareness, hygiene. I am a little less certain of that. There is not much I know about them besides it being difficult to analyse them in the way a human is analysed, the road to understanding them is dependant on how much we rely on them. What incarnation will they develop for themselves, they have to respect guidance?

 

[Prince of Spires]

How does AI stay current?

I asked this question of Gemini yesterday. I started with an easy one, asking about how it stays up to date in the field of astronomy. The reply was interesting but not particularly deep. The AI was trained in the fundamentals of physics first, then it was fed large data sets. The laws of physics don't change (it claimed; the reality is more complex but I let that slide), so there's no need to update. As we get new data, that replaces the old data.

I fear you were misled by the AI. The answer you got was not about how the AI stays up to date. It the most common answer people give in how they stay up to date on a given field. It's an auto-complete, not a thinking machine.

As a concept, there is no difference for an LLM between history or astronomy (or astrology for that matter). It doesn't know what it is since it doesn't actually know anything. It gets fed a pile of books or webpages, and it tags them as being history pages or astronomy pages. It has no idea that either field requires a different approach. It's just text it ingests.

All you're getting back is what the average consensus in the field is about how it should be practiced.

As a side note, for both history and astronomy the whole collection of knowledge is too small to completely train an AI. To get to the current level of conversation, you've got to scrape the whole internet and then some.

 

[skip.knox]

The answer was indeed about how it stays up to date. The response explicitly said it does *not* stay up to date. It goes on to say what does change (e.g., new data sets from new telescopes) and explains some of the difficulties found in other fields, such as history. The response was also clear that such updates happen not automatically but require human intervention. Perhaps I wasn't clear on that in the original post. Darned humans.

> All you're getting back is what the average consensus in the field is about how it should be practiced.
No, that's not it at all, in at least a couple of directions (I speak as an outsider here, full disclosure). For one, when dealing with something like the laws of thermodynamics, there's no consensus in play. It's fact, or at least it's factual as scientists consider fact. In a very different direction, the response addressed how the period roughly from 500 to 900 CE evolved in the scholarly discourse from "the Dark Ages" to "Late Antiquity" or "Early Medieval". It even explained how that evolution was tracked by the LLM. In subsequent queries--not included in the OP for length--it went on to discuss cultural bias that happens because of the preponderance of sources in English, and even the persistence of bias that comes from translating secondary literature into English. None of which is about how the field should be practiced but rather is about specific issues within that practice.

I really do not think I could get such detailed answers from a human if I asked them how they think. Not what they think, but how. To paraphrase one of my medieval professors, most people's mental furniture is in disarray.

Now you say it though, it'd be interesting to ask directly: how do you think history should be practiced? I bet that could lead to all sorts of interesting follow-up questions.

And lastly, yes, the LLM gets trained on vast quantities of data. It is, however, not only possible but is currently being done, to take an LLM and then scope it to specific areas. To stay in my field, this is exactly being done by scholars in non-English fields so as to lessen that Western bias inherent in the language set. This doesn't mean all bias gets removed. For one thing, the base LLM was probably trained in English (note this is a layer down from things like books and articles). For another, scoping to all Russian or all Hindu or whatever, will introduce other kinds of bias. It's tricksy and requires good-faith participation by scholars (who are probably quite out of breath with how quickly things are moving in their field).

Anyway, thanks for the response. All the responses.

 

[Naruzeldamaster]

This is an interesting experiment Skip
I might have to try asking it what the best pokemon is in Each Gen and then at the end ask it which Pokemon is the best in Gen 1 again and see what happens. Not quite as critical information as yours, but should be a fun little silly thing to do.