A couple of studies regarding AI writing

[Eduardo Ficaria]

Hi everyone! I have noticed this discussion titled "the use of AI to sculpt your writing" and I thought it could benefit from having real data about how AI writing is evolving and influencing the publishing industry. I have come across a couple of very recent studies that give us a better perspective of where things seem to be going, at least with current trends:

1. Generative AI floods and dilutes the market for books
   This one is about a study that analyzes the effect of AI-written (partially or totally) books in Amazon's Kindle Unlimited service. In short, the moat for "pure" human writing is thinning and, in general, each book (AI-written or not) makes less money due to the flood of AI-written books. Also, the study explains a particular telltale sign of AI writting, which is about the "abuse" of distinctive "rare language" taken from existing books.
2. Bot or not: Can people tell the difference between stories written by a human or by an AI system?
   This is a study where they make people read short stories (~1000 words) produced either by humans or ChatGPT 4.0 in three different experiments to see how they perceive them, and also if they are able to tell which ones where either written or AI generated. Conclusion, humans are biased against AI although they tend to like their stories more but, at the same time, humans are also very bad at telling if a text is AI generated or not (although some previous experience with AI systems seems to help in detecting AI content).

I recommend reading the articles since they are relatively short, specially if you skip the sections about metodology and research stuff like that. On my part, I'll just leave here my own kind of improvised take on this:

• The studies are just that, studies, not some final nails in humanity's coffin.
• Put together, the studies tell quite a bit about our human biases.
• Curated AI writing may already have the potential to commercially outperform the best AI-free authors out there.
• The first study mentions how AI systems have lowered the cost of production of written fiction, but there is a potential catch to consider. All of those systems have been heavily subsidized, first by Big Tech itself, then by private investors. But the time of their reckoning may be nearing given the financial tensions that are emerging (so much frakking debt in the system!). If there is a crash (or big correction) in the AI sector, that could lead to much higher subscription prices and remove some slop factories from the market. The problem is, like in nature, the strongest of them all will remain and keep on churning product as usual. Even if the monthly cost raises to, let's say, about $2000, if the slop still makes $10000 a month the commercial incentive remains strong.
• I suspect certain publishing houses may consider, if they are not doing it already, using AI to lower their costs and avoid being outcompeted by slop houses. At the same time, could it be that the need for editors and similar staff may go up due to the increase of production? At least to guarantee certain quality levels and avoid embarrassing mistakes. I think I've read something about this but I cannot remember.
• Amazon is already putting up some barriers to the flood of AI slop, but its growth seems unstoppable. Over time, it could very well choke their Kindle Unlimited service and make it worthless.

Hope you find all this informative at least.

 

[Romy]

• Amazon is already putting up some barriers to the flood of AI slop, but its growth seems unstoppable

And Meta spend fortunes on moderating crap and frauds... right? A sale is a sale.

 

[pmmg]

AI is still new. One of my issues with the discussion is the manner of looking at where it is at today, and not looking at where it is going. So, this may be a measure of it today, but AI is still evolving, and rapidly. Until it reaches the point of diminishing returns, its not accurate to take a snap shot and then make some decision about people's attitudes. Those same people may be seeing a product they cant recognize even a year for now.

Just to add to the conversation, I have been seeing a bit of buzz about Claude adding watermarks to AI generated content, thereby making the easier to spot by the big companies, like Amazon and Ingram, which...personally, I think is great (but I have not heard the real pros and cons yet), but I think it will be defeated. Security is always behind the low life element.

I also saw some news about AI company destroying old paper books to get new content. But...I am not sure the real truth of that. Maybe just be to stir things up. I am suspicious of the news on this.

Amazon, I am sure, does not really care. They just want a revenue stream on big ticket items. Ingram, which is completely about books, may have more skin in the game.

 

[hunterjwizzard]

You could post a thousand different studies and not change the core fact: generative AI art is automatically terrible by virtue of being generative AI.

 

[Eduardo Ficaria]

And Meta spend fortunes on moderating crap and frauds... right? A sale is a sale.

It doesn't seem to me that Mr Suckerberg really likes to spend those dollars in moderation, Romy ... He is just forced to do so by the law. Let's remember that he is known to prioritize profit even over his users' health and he very gladly would burn all that money in the meta-sloppyfication of the whole known universe.

AI is still new. One of my issues with the discussion is the manner of looking at where it is at today, and not looking at where it is going...

You're right pmmg , although the situation with AI tech is evolving rather rapidly. In particular, the issue with the Himalayan peaks of debt burning to fuel its development may reach a critical point this very year. If the crash ends up happening, it could be a real banger!

Regarding the use of AI in writing, what worries me now is the witch hunt happening where some people are now accusing others with no real way to truly discern what has been written with AI or not (as the second study from my original post indicates). I understand the concerns (legal and otherwise), but I find unfair and cynic the double standard where many other professionals can benefit from whatever advantages AI can bring to their jobs, but writers and other artists should not even think of using the damned tool. There are legitimate uses for this new tool that do not involve the stealing (as in "write me a fantasy novel in the style of this famous author") of anything, and many writers are already benefiting in ways I'm sure go completely undetected by readers (for instance, the corrections applied by services like Grammarly). If the final result is better and with a lower production cost (under current subscription prices, mind you), you usually have a winner there.

Another thing to consider about is that the datasets used to train AIs must be full of 100% human-made slop, and that is what they reproduce and amplify at industrial scale when asked to. Certainly, this is a wasteful and pointless way of using such powerful tools.

Regarding the news you have shared pmmg , yes I've read about them. The Claude watermarks are either embedded in the code that gives the text its format, or something like that, meaning that you can remove it by just copying and pasting without the style. You can do this usually when pasting on an editor like Notepad or Notepad++, or by pressing Shift + Ctrl + v in LibreOffice or M$Office where you get a menu asking you which version of the text you want to paste. The second news seems to be just rumours right now, but I would not be surprised if its confirmed as true. If there is something money cannot buy for AI labs right now is new fresh data after they bled the internet dry, and they need humongous loads of it to keep on training their models.

You could post a thousand different studies and not change the core fact: generative AI art is automatically terrible by virtue of being generative AI.

Sorry but no hunterjwizzard , that's your opinion, not a fact. And that "terrible generative AI art" is increasing its sales year on year in Amazon Kindle Unlimited (according to the first study). This means people are getting increasingly used to it and find it good enough for them. Imagine how is it going in similar platforms...

 

[hunterjwizzard]

Sorry but no @hunterjwizzard , that's your opinion, not a fact.

No. It is a fact. Art is about human expression. Take the human out of the equation and you get art that is automatically bad. Or you can take the stance that "its not art" anymore, at which point why are we discussing it on an art forum?!(Writing is art).

 

[Eduardo Ficaria]

No. It is a fact. Art is about human expression. Take the human out of the equation and you get art that is automatically bad. Or you can take the stance that "its not art" anymore, at which point why are we discussing it on an art forum?!(Writing is art).

You are conflating different concepts: one thing is to discuss what is art, and something else entirely is to have an opinion on the final result from any process (artistic or not). Certainly, current AI models are unable to do any art on their own since they don't possess any agency whatsoever. Leave the chat of any of them open for hours and nothing will happen at all. Unless some human enters the necessary prompts, no generation will happen. And those prompts can be very detailed and intentional, mind you. It is not that easy to get a decent result out of those automatons.

You talk of human expression. Well I'm sure we can agree those prompts are pretty much samples of human expression of the AI user, although given shape by the AI model in some style or other (usually also chosen by the user). So, you have human expression using an automated tool to materialize itself. It may be lazy, it may be stealing from someone else's style or IP, but there is human agency and some degree of expression. Is it art? I suppose it is in a much lesser way than forgers painting in the style of Rembrandt or Picasso. By the way, since years ago there has been some experimentation with machines around the question of "can machines create?". I vaguely remember the case of some robotic arm used to paint in a somewhat artistic way, based probably in some early generative software adapted for that robot.

I do recommend you to read the studies, because you're falling exactly in the human bias detected in the second one. Human intelligence may be rather exceptional (on Earth at least), but intelligence in general is not unique to us (nature really loves to replicate and reuse its stuff!). Also, with the coming of neuromorphic chips and other new computing tech, it may be very well happen that we end up creating the next jump in intelligence in the shape of some sort of synthetic beings (in the second half of this century, maybe?).

 

[Mad Swede]

Hi everyone! I have noticed this discussion titled "the use of AI to sculpt your writing" and I thought it could benefit from having real data about how AI writing is evolving and influencing the publishing industry. I have come across a couple of very recent studies that give us a better perspective of where things seem to be going, at least with current trends:

1. Generative AI floods and dilutes the market for books
   This one is about a study that analyzes the effect of AI-written (partially or totally) books in Amazon's Kindle Unlimited service. In short, the moat for "pure" human writing is thinning and, in general, each book (AI-written or not) makes less money due to the flood of AI-written books. Also, the study explains a particular telltale sign of AI writting, which is about the "abuse" of distinctive "rare language" taken from existing books.

This first study looks ONLY at self-published books.

It's a known fact that those writers who self-publish and who make a living out of it are primarily writers in the Romance genre. Those writers have to produce a new book every 6 to 8 weeks, otherwise readers move on to books by different writers. A number of these writers admit to using stock phrases and descriptions (which, to be fair, is something many pulp writers did in the 1930s) and several admit to considering the use of (or even to using) AI-tools to help them produce new books faster.

This sort of market is also known for the fact that readers are looking for something familiar, which is why they tend to stick with writers who follow fairly conventional plot structures. Those readers are not primarily looking for original writing when buying these books.

I would also point out one key line from the paper: Our results bear directly on the market-effect question at the center of the fair use defense to copyright infringement. Expressed in a different way, what the study authors are saying is that using existing works to train AI-tools may not be fair use if it distorts the market. Which opens up a whole new can of legal worms.

1. Bot or not: Can people tell the difference between stories written by a human or by an AI system?
   This is a study where they make people read short stories (~1000 words) produced either by humans or ChatGPT 4.0 in three different experiments to see how they perceive them, and also if they are able to tell which ones where either written or AI generated. Conclusion, humans are biased against AI although they tend to like their stories more but, at the same time, humans are also very bad at telling if a text is AI generated or not (although some previous experience with AI systems seems to help in detecting AI content).

This second paper is interesting, in that the discussion sections make clear that the study methodology has some weaknesses which may have an impact on the results and their wider applicability. It's also important to note how the study participants were recruited: from a pool of people formed specifically to assist in AI development and related studies. Put another way, there is a degree of self selection amongst the participants and so there is a risk of bias, which isn't mentioned in the paper.

As someone with a research degree I have to say that the conclusion to this paper is not supported by the results, which are significantly more nuanced and .

I recommend reading the articles since they are relatively short, specially if you skip the sections about metodology and research stuff like that. On my part, I'll just leave here my own kind of improvised take on this:

• The studies are just that, studies, not some final nails in humanity's coffin.
• Put together, the studies tell quite a bit about our human biases.
• Curated AI writing may already have the potential to commercially outperform the best AI-free authors out there.
• The first study mentions how AI systems have lowered the cost of production of written fiction, but there is a potential catch to consider. All of those systems have been heavily subsidized, first by Big Tech itself, then by private investors. But the time of their reckoning may be nearing given the financial tensions that are emerging (so much frakking debt in the system!). If there is a crash (or big correction) in the AI sector, that could lead to much higher subscription prices and remove some slop factories from the market. The problem is, like in nature, the strongest of them all will remain and keep on churning product as usual. Even if the monthly cost raises to, let's say, about $2000, if the slop still makes $10000 a month the commercial incentive remains strong.
• I suspect certain publishing houses may consider, if they are not doing it already, using AI to lower their costs and avoid being outcompeted by slop houses. At the same time, could it be that the need for editors and similar staff may go up due to the increase of production? At least to guarantee certain quality levels and avoid embarrassing mistakes. I think I've read something about this but I cannot remember.
• Amazon is already putting up some barriers to the flood of AI slop, but its growth seems unstoppable. Over time, it could very well choke their Kindle Unlimited service and make it worthless.

Hope you find all this informative at least.

Your conclusions here are your own. As a researcher I don't agree with you, not having read the two papers. The results from the studies are so limited and nuanced that about all you can conclude is that more studies are needed.

There is also the question about what sort of writing AI-tools might produce. There is a significant difference between the best literary writing and the sort of writing seen in genres like Romance. Those differences matter, because publishers look for different things when selecting stories for publication, depending on what sort of market they're looking at.

 

[Eduardo Ficaria]

Thanks for your observations Mad Swede . My summaries of those articles were just to give a rough idea of what is explained in them. Yes, I read about the selection processes in the second study and I understood that were limitations in that regard. Very nice of you pointing out the type of content being churn with AI tools and published in Amazon, unsurprisingly formulaic. Still, the stats provided in the first study are pointing towards a tendency that I'm not sure it would be wise to ignore. Regarding the nuances or issues you point out from the second study, I'll have to trust your judgement although that one also mention previous similar studies that point to the same human biases regarding AI content, which is quite interesting to say the least. And certainly my conclusions are my own! Based on what I read in those studies, plus a few other stuff I've read about authors using AI in different ways.

There is also the question about what sort of writing AI-tools might produce. There is a significant difference between the best literary writing and the sort of writing seen in genres like Romance. Those differences matter, because publishers look for different things when selecting stories for publication, depending on what sort of market they're looking at.

Regarding this, the first study mentions a pattern were those AI-churned books seem to use distinctive "rare language" taken from existing books a lot more than comparable AI-free books. Although maybe is not the just models, but the AI-authors that have developed the preference for such rare expressions because they look good to them.