And, at risk of shilling another (but very related) post: Why is Anthropic's public writing style so unlike Claude's? https://cmart.blog/claude-writing/
Thanks for writing this. I used to write fiction more frequently, and that impulse has decreased as LLM’s have gotten better at writing. I can still produce (what I think) is better writing than LLM’s. But it does feel like the day is coming in which their creative fiction will improve and be indistinguishable from top writers. Part of the reason I write is to be able to read stories about ideas/daydreams I have (“what if alpha-gal syndrome was bioengineered by a vigilante vegan ecoterrorist”), and if I can click a button and read an interesting story about that, would I still write? I don’t know.
Lately though, I’ve been experimenting with making interactive text fiction using Claude. There’s a bit more of a moat (LLM’s would have to tackle both game design and also good writing) and it’s never been something I had been able to do conveniently in the past (I found Twine too clumsy). Whenever I’m done with it, I’ve been thinking about how to write the disclaimer regarding what parts are AI-assisted. And will probs reference back to this article. :)
"Cast your mind back to the old world, five years ago. At that time, if you had started a blog and posted AI-generated essays without telling anyone, I’m reasonably certain that would have been considered a dick move."
Five years ago was before ChatGPT, still in the era of LLMs as a tech demo for ML nerds and not as a consumer technology. A blog post of AI-generated essays would still be novel enough to be an interesting experiment rather than a mundane deceptive practice. I think you've forgotten how quickly things have changed.
What I was trying to say was that if trying to impose a policy like "by default all writing should not use AI and anyone who uses AI" wouldn't work, because all that would happen is that a few ultra honest / conscientious people would disclose it and everyone else would yawn. So I think it makes more sense to "limits" rather than "usage".
I just recently wrote a newsletter post myself, exploring this with LLM generated code, but the point is somewhat similar.
Coincidentally, I use all 11, while always trying to "seed" the writing myself. I'll write the first and final draft, mostly by hand. I dislike the "smell" of LLM writing, so I try to eradicate it even when I do use it to help.
But, back to the point. I've called AI generated code "IKEA code" - and this is similarly, "IKEA writing". I like building my IKEA furniture, I like having LLMs write me essays with the nuance I like. Nobody likes other people's IKEA furniture hah.
Super interesting, and got me thinking about #7. I sometimes accept an AI rephrasing option verbatim for short sentences or parts thereof, because I strongly feel it can't be improved upon, or that any amendement by myself would simply muddy the waters.
I gues the literature on "AI sounds better to most people until disclosed" is constantly in the back of my head, and I feel like setting myself up for failure if I stray beyond the AI text. At the same time - and working in an opposite direction - I understand setting myself up for failure might precisely be the human touch that's needed to get my skin in the game.
So I typically feel torn between these two sentiments and, rather unsurprisingly, this led to me seek rephrasing less and less.
"I am not suggesting that we should bully writers into declaring that they are AI-free."
There are YouTubers leaning into this, proudly proclaiming that their content is 100% AI-free and made by a real human.
I click off of those as soon as they make the claim. No, my good WWII Historian In Your 20's Sir, you are NOT better than everyone else.
Then there are the same channels, similar content, where the entire thing is an AI-assisted production - Clearly AI script, read by a clearly AI-voiceover - NOT disclosed as such as required by YouTube terms.
I click off those as soon as I notice it, as well. Having been a (very small time) YouTuber back in 2016, I GET that there are challenges for people, especially people with strongly accented English or other challenges in the production pipeline. But you Indonesian WWII Historian In Your 20's Sir, are at least better than THAT...
I do know one YouTuber who walks a great middle line - their English is strongly accented, and it seems like they have AI write their first pass at a script, and polish that into their own voice from there - AND it very much sounds like they use ElevenLabs to read their final script in their own (cloned) voice - which keeps the 'realness' of their actual voice and timbre - but speaks the script with far less accent than they get when they DIY into a microphone. That's the middle path that I love.
Use the tools in a way that boosts your originality and creativity and sands off the roughest of edges, and not in a way that *replaces* your originality and creativity and makes your stuff just like everyone else's stuff.
I am definitely stealing the idea of using AI for an advanced thesaurus. When the goal is communication, it would be crazy not to, on certain occasions.
Yeah, I feel like that is one of the clearer "centaur" cases at the moment, where AI can essentially make human writing better with few tradeoffs. If I'm reading someone else, I would weakly prefer that they *do* use AI that way.
We're thinking about this a lot at the moment and I enjoyed this one!
Two solutions:
- labelling: if pangram or equivalent remains as accurate as it is, then I'd love it if substack would automatically label posts as AI generated. I'm planning on doing this on the EA Forum (hopefully in the next couple weeks). And we'll see how that goes.
- publications: rather than trusting a random blogger, you trust loads of small publications/ loads of individual 'editors' who make sure that the blogs they host aren't ai generated.
Interesting! It might not be necessary for the EA forum, but have you considered any strategies that might make reduce goodharting? For example, I've wondered if it might be best to only show the score after some period of time, or perhaps to only show the average score for a given author's previous N posts. I'm just thinking that the fact that the detector could change in the future makes it way more risky to try to use AI and then try to obscure it somehow.
Goodharting in this case being humanising AI text to pass pangram? I do like your idea of saying that if a post is passed off as human and later discovered to be AI there should be some punishment to disincentivise. Not sure what that looks like on the forum though, and I don't want to create too punitive of a vibe that might stop people from posting. For example, often people will use some ai in their drafting process, or would prefer to, even if the final output is in practice almost entirely human. I don't want to scare such people off.
Oh yeah, by "goodharting" I meant people trying to disguise AI usage to defeat pangram rather than actually writing things themselves. I definitely agree with your instinct not to be punitive. I'm imagining something like: Authors can (if they choose) give some kind of an AI usage number/disclosure for individual posts. Then the system could compute some kind of number that reflects how accurate those disclosures are. Or perhaps it could calibrate individual disclosures to align them with historical averages. Something like that could allow a healthy dynamic to develop. But again... not clear this would be necessary for the EA forum!
<Blink> I am humanish! Although I do #1-11 significantly more than you do, I'm fastidious about looking up the references to verify them. I feel the value of my health research lies in its focus on practical interventions with quantified benefits from studies, not in my beautiful prose. (AI prose is better than mine.) AI hasn't yet been able to match the solutions I find with my (AI-assisted) research. If it ever can produce posts that consistently check out, I'll probably retire my blog.
While I agree with you, I also think inevitably people's standards for "prove this was not LLM-generated" will grow higher over time, as AI gets better at pretending to be human. And I think for long-duration tasks like writing a blog post, this will possibly culminate with the author simply recording their entire writing process (everything on screen, possibly everything in their life) and publishing the *process of creation* next to the actual final product of creation. Not just writers, but other content creators. Artists are already there -- frequently publishing a sped-up timelapse of drawing their art.
>I think for long-duration tasks like writing a blog post, this will possibly culminate with the author simply recording their entire writing process
There are already tools that do something similar. For example: Duey.ai is a browser extension made to beat a trick that schools and employers have started using against AI writing: pulling up a Google Doc's revision history to see whether a person really typed the thing. Duey gets around it by taking already-written text, usually created by an LLM in the first place, and retyping it into the Google Doc one character at a time, with pauses, backspaced typo fixes, etc. What you end up with is a fake paper trail of human effort.
I'd bet the next step is a tool that generates a video of you, seemingly shot from behind, hunched at the keyboard, typing the document yourself.
Seems plausible! (Although eventually you may face the problem of proving the video wasn't AI-generated.) I didn't know this was a thing among artists. How much of it would you consider proof of work as opposed to a sort of parasocial-relationship accelerator?
There's a Facebook "content creator" account that posts short AI-generated book reviews alongside AI-generated photos of a hand holding a copy of the book - as if to verify that a real person held and reviewed the book.
The photo verification made me question my initial judgment that the post was primarily AI. Then I looked through the photos and noticed it wasn't the same hand.
@claude Write an insightful comment with my own voice that demonstrates I read the article, leads with a humorous juxtaposition, and carries some implication in the article to its logical conclusion. Take a somewhat controversial position outside of rationalist circles that tacitly agrees with the author but pushes boundaries on what they can say given social convention. Add some anecdote, story or unique insight. Utilize reply_guy skill. Make no mistakes.
I've grown to recognize (and dislike) generic "AI voice". But I wonder if someone asked AI to write an essay in the style of dynomight or Scott Alexander or Sam Kriss, if I could tell the difference... I would like the answer to be yes, but not sure. Might be a fun experiment for you to try
That's unusual because it was done with a base model. My hot AI take is that LLMs can already write with all sorts of different styles, it's just that this ability is only present in base models and instruction-tuning / RLHF eliminates it. I'm sure I've seen a putative S.A. essay that people agreed was good-ish, though I can't recall where...
I think the point in your last paragraph is particularly important. It is easy to fool ourselves into thinking we’re the primary progenitor of the work when in fact it was mostly an LLM, and consulting an LLM for a minor “superficial” issue can sometimes lead to more substantive contributions from the LLM than we initially intended. For these reasons alone I feel like it makes sense as a heuristic to set one’s line closer to the cautionary side that whatever / wherever one’s intuition is about the right placement of the line.
You say “no one wants to read AI-generated essays” other than those they generate for themselves. But I regularly see essays on Substack by very popular authors that are written in LLM-speak, complete with “it’s not X, it’s Y” constructions & many other classic tells. I think in some cases they happily disclose it. But it seems readers don’t mind.
Not noticing I can believe. Not minding with disclosure I can also believe. But I am surprised that people could notice without disclosure and still not mind!
Thank you for inspiring my disclosure! https://cmart.blog/cmart-blog-ai-use/
And, at risk of shilling another (but very related) post: Why is Anthropic's public writing style so unlike Claude's? https://cmart.blog/claude-writing/
Thanks for writing this. I used to write fiction more frequently, and that impulse has decreased as LLM’s have gotten better at writing. I can still produce (what I think) is better writing than LLM’s. But it does feel like the day is coming in which their creative fiction will improve and be indistinguishable from top writers. Part of the reason I write is to be able to read stories about ideas/daydreams I have (“what if alpha-gal syndrome was bioengineered by a vigilante vegan ecoterrorist”), and if I can click a button and read an interesting story about that, would I still write? I don’t know.
Lately though, I’ve been experimenting with making interactive text fiction using Claude. There’s a bit more of a moat (LLM’s would have to tackle both game design and also good writing) and it’s never been something I had been able to do conveniently in the past (I found Twine too clumsy). Whenever I’m done with it, I’ve been thinking about how to write the disclaimer regarding what parts are AI-assisted. And will probs reference back to this article. :)
"Cast your mind back to the old world, five years ago. At that time, if you had started a blog and posted AI-generated essays without telling anyone, I’m reasonably certain that would have been considered a dick move."
Five years ago was before ChatGPT, still in the era of LLMs as a tech demo for ML nerds and not as a consumer technology. A blog post of AI-generated essays would still be novel enough to be an interesting experiment rather than a mundane deceptive practice. I think you've forgotten how quickly things have changed.
Well articulated!
Related: labels for various levels of Ai usage https://www.halmarks.org/
(not my project)
> At this point, trying to bully people into proactive disclosure is just a tax on honesty / consciousness / integrity.
(Somebody) explain please, I don't understand how this is a tax, but maybe I'm being slow.
What I was trying to say was that if trying to impose a policy like "by default all writing should not use AI and anyone who uses AI" wouldn't work, because all that would happen is that a few ultra honest / conscientious people would disclose it and everyone else would yawn. So I think it makes more sense to "limits" rather than "usage".
What is that painting at the top of your post?!
https://www.nga.gov/artworks/59963-neigh-iron-horse
I just recently wrote a newsletter post myself, exploring this with LLM generated code, but the point is somewhat similar.
Coincidentally, I use all 11, while always trying to "seed" the writing myself. I'll write the first and final draft, mostly by hand. I dislike the "smell" of LLM writing, so I try to eradicate it even when I do use it to help.
But, back to the point. I've called AI generated code "IKEA code" - and this is similarly, "IKEA writing". I like building my IKEA furniture, I like having LLMs write me essays with the nuance I like. Nobody likes other people's IKEA furniture hah.
Here's the post, if you're interested: https://tinthe.dev/p/t/posts/ikea-code
Don't usually care about AI writing detectors, but I've gotten anywhere from 0-1% (majority of tools), to one tool saying 50% (definitely not).
Super interesting, and got me thinking about #7. I sometimes accept an AI rephrasing option verbatim for short sentences or parts thereof, because I strongly feel it can't be improved upon, or that any amendement by myself would simply muddy the waters.
I gues the literature on "AI sounds better to most people until disclosed" is constantly in the back of my head, and I feel like setting myself up for failure if I stray beyond the AI text. At the same time - and working in an opposite direction - I understand setting myself up for failure might precisely be the human touch that's needed to get my skin in the game.
So I typically feel torn between these two sentiments and, rather unsurprisingly, this led to me seek rephrasing less and less.
"I am not suggesting that we should bully writers into declaring that they are AI-free."
There are YouTubers leaning into this, proudly proclaiming that their content is 100% AI-free and made by a real human.
I click off of those as soon as they make the claim. No, my good WWII Historian In Your 20's Sir, you are NOT better than everyone else.
Then there are the same channels, similar content, where the entire thing is an AI-assisted production - Clearly AI script, read by a clearly AI-voiceover - NOT disclosed as such as required by YouTube terms.
I click off those as soon as I notice it, as well. Having been a (very small time) YouTuber back in 2016, I GET that there are challenges for people, especially people with strongly accented English or other challenges in the production pipeline. But you Indonesian WWII Historian In Your 20's Sir, are at least better than THAT...
I do know one YouTuber who walks a great middle line - their English is strongly accented, and it seems like they have AI write their first pass at a script, and polish that into their own voice from there - AND it very much sounds like they use ElevenLabs to read their final script in their own (cloned) voice - which keeps the 'realness' of their actual voice and timbre - but speaks the script with far less accent than they get when they DIY into a microphone. That's the middle path that I love.
Use the tools in a way that boosts your originality and creativity and sands off the roughest of edges, and not in a way that *replaces* your originality and creativity and makes your stuff just like everyone else's stuff.
I am definitely stealing the idea of using AI for an advanced thesaurus. When the goal is communication, it would be crazy not to, on certain occasions.
Yeah, I feel like that is one of the clearer "centaur" cases at the moment, where AI can essentially make human writing better with few tradeoffs. If I'm reading someone else, I would weakly prefer that they *do* use AI that way.
We're thinking about this a lot at the moment and I enjoyed this one!
Two solutions:
- labelling: if pangram or equivalent remains as accurate as it is, then I'd love it if substack would automatically label posts as AI generated. I'm planning on doing this on the EA Forum (hopefully in the next couple weeks). And we'll see how that goes.
- publications: rather than trusting a random blogger, you trust loads of small publications/ loads of individual 'editors' who make sure that the blogs they host aren't ai generated.
Interesting! It might not be necessary for the EA forum, but have you considered any strategies that might make reduce goodharting? For example, I've wondered if it might be best to only show the score after some period of time, or perhaps to only show the average score for a given author's previous N posts. I'm just thinking that the fact that the detector could change in the future makes it way more risky to try to use AI and then try to obscure it somehow.
Goodharting in this case being humanising AI text to pass pangram? I do like your idea of saying that if a post is passed off as human and later discovered to be AI there should be some punishment to disincentivise. Not sure what that looks like on the forum though, and I don't want to create too punitive of a vibe that might stop people from posting. For example, often people will use some ai in their drafting process, or would prefer to, even if the final output is in practice almost entirely human. I don't want to scare such people off.
Oh yeah, by "goodharting" I meant people trying to disguise AI usage to defeat pangram rather than actually writing things themselves. I definitely agree with your instinct not to be punitive. I'm imagining something like: Authors can (if they choose) give some kind of an AI usage number/disclosure for individual posts. Then the system could compute some kind of number that reflects how accurate those disclosures are. Or perhaps it could calibrate individual disclosures to align them with historical averages. Something like that could allow a healthy dynamic to develop. But again... not clear this would be necessary for the EA forum!
<Blink> I am humanish! Although I do #1-11 significantly more than you do, I'm fastidious about looking up the references to verify them. I feel the value of my health research lies in its focus on practical interventions with quantified benefits from studies, not in my beautiful prose. (AI prose is better than mine.) AI hasn't yet been able to match the solutions I find with my (AI-assisted) research. If it ever can produce posts that consistently check out, I'll probably retire my blog.
While I agree with you, I also think inevitably people's standards for "prove this was not LLM-generated" will grow higher over time, as AI gets better at pretending to be human. And I think for long-duration tasks like writing a blog post, this will possibly culminate with the author simply recording their entire writing process (everything on screen, possibly everything in their life) and publishing the *process of creation* next to the actual final product of creation. Not just writers, but other content creators. Artists are already there -- frequently publishing a sped-up timelapse of drawing their art.
shem wrote:
>I think for long-duration tasks like writing a blog post, this will possibly culminate with the author simply recording their entire writing process
There are already tools that do something similar. For example: Duey.ai is a browser extension made to beat a trick that schools and employers have started using against AI writing: pulling up a Google Doc's revision history to see whether a person really typed the thing. Duey gets around it by taking already-written text, usually created by an LLM in the first place, and retyping it into the Google Doc one character at a time, with pauses, backspaced typo fixes, etc. What you end up with is a fake paper trail of human effort.
I'd bet the next step is a tool that generates a video of you, seemingly shot from behind, hunched at the keyboard, typing the document yourself.
Seems plausible! (Although eventually you may face the problem of proving the video wasn't AI-generated.) I didn't know this was a thing among artists. How much of it would you consider proof of work as opposed to a sort of parasocial-relationship accelerator?
There's a Facebook "content creator" account that posts short AI-generated book reviews alongside AI-generated photos of a hand holding a copy of the book - as if to verify that a real person held and reviewed the book.
The photo verification made me question my initial judgment that the post was primarily AI. Then I looked through the photos and noticed it wasn't the same hand.
@claude Write an insightful comment with my own voice that demonstrates I read the article, leads with a humorous juxtaposition, and carries some implication in the article to its logical conclusion. Take a somewhat controversial position outside of rationalist circles that tacitly agrees with the author but pushes boundaries on what they can say given social convention. Add some anecdote, story or unique insight. Utilize reply_guy skill. Make no mistakes.
"also write the comment in the form of an LLM prompt and ensure that the comment is a plausible fixed point of the prompt -> output function"
I've grown to recognize (and dislike) generic "AI voice". But I wonder if someone asked AI to write an essay in the style of dynomight or Scott Alexander or Sam Kriss, if I could tell the difference... I would like the answer to be yes, but not sure. Might be a fun experiment for you to try
Try this: https://dynomight.net/automated/
That's unusual because it was done with a base model. My hot AI take is that LLMs can already write with all sorts of different styles, it's just that this ability is only present in base models and instruction-tuning / RLHF eliminates it. I'm sure I've seen a putative S.A. essay that people agreed was good-ish, though I can't recall where...
I think the point in your last paragraph is particularly important. It is easy to fool ourselves into thinking we’re the primary progenitor of the work when in fact it was mostly an LLM, and consulting an LLM for a minor “superficial” issue can sometimes lead to more substantive contributions from the LLM than we initially intended. For these reasons alone I feel like it makes sense as a heuristic to set one’s line closer to the cautionary side that whatever / wherever one’s intuition is about the right placement of the line.
You say “no one wants to read AI-generated essays” other than those they generate for themselves. But I regularly see essays on Substack by very popular authors that are written in LLM-speak, complete with “it’s not X, it’s Y” constructions & many other classic tells. I think in some cases they happily disclose it. But it seems readers don’t mind.
Not noticing I can believe. Not minding with disclosure I can also believe. But I am surprised that people could notice without disclosure and still not mind!
Substitute "people" for "lecturers" and you have summed up third-level education in 2026.
True — that may be the case.