You say “no one wants to read AI-generated essays” other than those they generate for themselves. But I regularly see essays on Substack by very popular authors that are written in LLM-speak, complete with “it’s not X, it’s Y” constructions & many other classic tells. I think in some cases they happily disclose it. But it seems readers don’t mind.
Not noticing I can believe. Not minding with disclosure I can also believe. But I am surprised that people could notice without disclosure and still not mind!
Ironically I'm more concerned with people using AI for research than for editing. LLMs hallucinate like crazy. I wouldn't trust what an AI states as facts at all. The type of AI you're referencing are trained on absolutely enormous data sets that contain everything from novels to reddit posts. They can't tell truth from falsehood or fact from fiction because they weren't designed to.
Personally, I do have a 0% AI stance because of a moral objection to their outsized climate impact. Data centers raise the ambient temperature of their surrounding areas from the heat they give off while turning through millions of gallons of water and raising everyone's energy prices because of the quantity of power they consume. Tech industry leaders are advocating delaying cleaner power plants and turning coal plants back on because they want to keep building and powering data centers in the hopes of keeping the AI investment money machine turned on. People living near data centers are suffering from noise pollution, higher temperatures, and destroyed water tables. The negative impacts of AI are real. The more we use it, the most tech companies are going to demand our resources, and the worse the long term outcome becomes.
We only view AI as inevitable and as something we have to learn to live with because powerful people whose money is built on the tech sector are insisting we must. You don't have to use it. You don't need to use it. And maybe you should think about why you feel you have to use it.
Using AI for research doesn't necessarily involve any real "trust". For example, when researching vitamin D, most of my queries were of the type: "Give me links to as many ungated papers as you can find with major results from the D-health trial".
As computational technology got better and better, did we ever hear the following?
"Did you do those calculations with pencil & paper, or did you cheat and use a slide rule?"
"Did you do those calculations with a slide rule, or did you cheat and use an electronic calculator?"
"Did you do those calculations with a basic calculator, or did you cheat and use R?"
I don't think we did, and maybe this is because with mathematics, regardless of the method, it's easy to confirm the quality of the answer, and the answer should be the same (modulo precision) regardless of method. This is not true of writing.
Perhaps because I'm not an essayist or blogger myself, I have no objection in principle to essays written 100% by AI, as long as they are an enjoyable or informative read. The main problem with AI essays today is that they are awful. Even as they become less awful, I'm confident that the best "AI-written essays" for the foreseeable future will be written in centaur mode, with a human driving the process and themes.
IOW, I think it will be a long time before we can simply tell an AI system the following and expect a high-quality result:
"Come up with a topic and write an essay on it that human readers will find interesting and engaging."
Thanks for making this post. It was particularly interesting since I've been attempting to figure out how to use AI in a way that will benefit me long-term. Since I don't regularly write, the only line that I have personally drawn is "don't make friends with the chatbot". It's depressing to contemplate all of the people who turn to chatbots for basic social interaction.
This is my feeling, too, I accept the usefulness of LLMs but if I'm reading someone's essay (or email, for that matter) I want them to have typed the whole thing in with sentences out of their brain.
I must admit that I'm quite sure what ref-linking is. But for what it's worth, I try to leave the source name visible in the filename on dynomight.net, in this case:
My conspiracy theory is that current AI is already capable of non-bland / interesting writing, it's just that this capability is only present in base models, which for whatever reason, the public doesn't get to interact with. (The big companies don't share them at all. There are a few smaller open-weight base models, but very few providers.)
I think it's sort of a side-effect. Base models are crazy and dangerous and will happily complete instructions for bombs or write racist jokes, etc. I think the goal of the RL is to make them not to crazy / offensive / dangerous stuff, and the RL sort of figures out that the best way to do that is simply refuse to access most of the base model's original range.
For example, a friend of mine showed that a base model could write in a style that I find *incredibly* similar to me: https://dynomight.net/automated/
I typically use Claude to edit my writing, but I always start with a draft that's my own. I got into an extended 𝚏̶̶̶𝚕̶̶̶𝚊̶̶̶𝚖̶̶̶𝚎̶̶̶ ̶̶̶𝚠̶̶̶𝚊̶̶̶𝚛̶̶̶ discussion on MetaFilter with members who accused me of being a bad person for doing this.
I think the line you’re drawing is exactly right! It’s the same way you’d treat a human research assistant, and for many of the same reasons. If the human research assistant assists with your research, you provide a general acknowledgment that they help out with your publication but you don’t necessarily point out every little thing they do. If the human research assistant actually writes up something you want to use verbatim, then you quote it with attribution, otherwise it’s ghostwriting, and I don’t want to read someone who uses undisclosed ghostwriters whether or not they’re human.
I read 'cuz it's interesting. When that stops, I will.
I don't really have a moral thing about someone else's blog using AI. I wouldn't let it touch _my_ writing (not like my words are so golden, but they're _MINE!_).
At the moment, I'm pretty sure that if the blog was AI-written, it wouldn't be as interesting as it is.
I think I would actually be somewhat opposed, even assuming perfect accuracy. I tend to think that we need to reach a kind of "accommodation" with AI. For example, I feel like the timing is quickly coming where AIs will be capable of quite good "developmental editing" that will make human writers better as human writers. (Sort of like how chess engines have advanced human chess.) I just don't see much of a future in rigidly insisting on being 100% AI free. We need to find a way to live with AI but retain the core of what we value in human writing.
Love this and so synchronous with the essay I just posted. This is exactly how I feel about AI—and I forgot to clarify in my piece that I'm talking about never using it again for my writing for many of the same reasons you cite, but I do use it for work/research, and am actually compelled to use it at work.
Funnily enough I just read a different essay that proclaimed it would never use AI to write, but it read like AI and tested as AI, but the author was adamant that he didn't use it. If he's telling the truth, thats rough for him, but I feel like he was lying. But I believe you fully because I just sense it in your character and in your writing. Just such a fascinating dynamic at play though.
Oh and for transparency, I wrote my anti-AI piece doing only #1 (really hard to avoid, and generally still went to wikipedia for general based answers instead of using the summaries). Otherwise I used good old thesaurus.com, even though I think using AI as a thesaurus is completely reasonable. Yet, writing this essay made me question if the exercise and care of sifting through synonyms mentally was perhaps worth it as a way to develop my writing. Obviously there are arbitrary thresholds, like I'm not going to be Wendall Berry who refused to ever write with a computer. And my main criteria for engagement with writing is (1) is it good/well-written (AI writing grates on me like fingernails on a chalk board, so if you're heavily using AI you better be a really good editor), and (2) does it signal a deep sense of care and proof of work. Within that I can allow hybrid use of it as a tool.
You say “no one wants to read AI-generated essays” other than those they generate for themselves. But I regularly see essays on Substack by very popular authors that are written in LLM-speak, complete with “it’s not X, it’s Y” constructions & many other classic tells. I think in some cases they happily disclose it. But it seems readers don’t mind.
Not noticing I can believe. Not minding with disclosure I can also believe. But I am surprised that people could notice without disclosure and still not mind!
True — that may be the case.
Ironically I'm more concerned with people using AI for research than for editing. LLMs hallucinate like crazy. I wouldn't trust what an AI states as facts at all. The type of AI you're referencing are trained on absolutely enormous data sets that contain everything from novels to reddit posts. They can't tell truth from falsehood or fact from fiction because they weren't designed to.
Personally, I do have a 0% AI stance because of a moral objection to their outsized climate impact. Data centers raise the ambient temperature of their surrounding areas from the heat they give off while turning through millions of gallons of water and raising everyone's energy prices because of the quantity of power they consume. Tech industry leaders are advocating delaying cleaner power plants and turning coal plants back on because they want to keep building and powering data centers in the hopes of keeping the AI investment money machine turned on. People living near data centers are suffering from noise pollution, higher temperatures, and destroyed water tables. The negative impacts of AI are real. The more we use it, the most tech companies are going to demand our resources, and the worse the long term outcome becomes.
We only view AI as inevitable and as something we have to learn to live with because powerful people whose money is built on the tech sector are insisting we must. You don't have to use it. You don't need to use it. And maybe you should think about why you feel you have to use it.
Using AI for research doesn't necessarily involve any real "trust". For example, when researching vitamin D, most of my queries were of the type: "Give me links to as many ungated papers as you can find with major results from the D-health trial".
As computational technology got better and better, did we ever hear the following?
"Did you do those calculations with pencil & paper, or did you cheat and use a slide rule?"
"Did you do those calculations with a slide rule, or did you cheat and use an electronic calculator?"
"Did you do those calculations with a basic calculator, or did you cheat and use R?"
I don't think we did, and maybe this is because with mathematics, regardless of the method, it's easy to confirm the quality of the answer, and the answer should be the same (modulo precision) regardless of method. This is not true of writing.
Perhaps because I'm not an essayist or blogger myself, I have no objection in principle to essays written 100% by AI, as long as they are an enjoyable or informative read. The main problem with AI essays today is that they are awful. Even as they become less awful, I'm confident that the best "AI-written essays" for the foreseeable future will be written in centaur mode, with a human driving the process and themes.
IOW, I think it will be a long time before we can simply tell an AI system the following and expect a high-quality result:
"Come up with a topic and write an essay on it that human readers will find interesting and engaging."
Thanks for making this post. It was particularly interesting since I've been attempting to figure out how to use AI in a way that will benefit me long-term. Since I don't regularly write, the only line that I have personally drawn is "don't make friends with the chatbot". It's depressing to contemplate all of the people who turn to chatbots for basic social interaction.
This is my feeling, too, I accept the usefulness of LLMs but if I'm reading someone's essay (or email, for that matter) I want them to have typed the whole thing in with sentences out of their brain.
You should caption/ref-link the header image. I had to look up the painting, unsure why you'd picked it. It's clever!
I must admit that I'm quite sure what ref-linking is. But for what it's worth, I try to leave the source name visible in the filename on dynomight.net, in this case:
https://dynomight.net/img/blink/the_neigh_of_an_iron_horse.jpg
I definitely use AI to interrogate my writing, I find that it is very helpful in giving feedback on where I'm being unclear / wrong.
I don't understand why people are tempted to have AI write for them though, the output is always bland, uninteresting and otherwise terrible.
(I'm sure this will change at some point, then I don't know what to think.)
My conspiracy theory is that current AI is already capable of non-bland / interesting writing, it's just that this capability is only present in base models, which for whatever reason, the public doesn't get to interact with. (The big companies don't share them at all. There are a few smaller open-weight base models, but very few providers.)
Do you think they purposefully train it out of them during RL, or that it's accidentally lost? It's a very interesting idea.
I think it's sort of a side-effect. Base models are crazy and dangerous and will happily complete instructions for bombs or write racist jokes, etc. I think the goal of the RL is to make them not to crazy / offensive / dangerous stuff, and the RL sort of figures out that the best way to do that is simply refuse to access most of the base model's original range.
For example, a friend of mine showed that a base model could write in a style that I find *incredibly* similar to me: https://dynomight.net/automated/
I typically use Claude to edit my writing, but I always start with a draft that's my own. I got into an extended 𝚏̶̶̶𝚕̶̶̶𝚊̶̶̶𝚖̶̶̶𝚎̶̶̶ ̶̶̶𝚠̶̶̶𝚊̶̶̶𝚛̶̶̶ discussion on MetaFilter with members who accused me of being a bad person for doing this.
I think the line you’re drawing is exactly right! It’s the same way you’d treat a human research assistant, and for many of the same reasons. If the human research assistant assists with your research, you provide a general acknowledgment that they help out with your publication but you don’t necessarily point out every little thing they do. If the human research assistant actually writes up something you want to use verbatim, then you quote it with attribution, otherwise it’s ghostwriting, and I don’t want to read someone who uses undisclosed ghostwriters whether or not they’re human.
I read 'cuz it's interesting. When that stops, I will.
I don't really have a moral thing about someone else's blog using AI. I wouldn't let it touch _my_ writing (not like my words are so golden, but they're _MINE!_).
At the moment, I'm pretty sure that if the blog was AI-written, it wouldn't be as interesting as it is.
I just don't feel the author of this post is the same as the previous one.
I guess I'm OK with that, but I respectfully ask that you believe that both versions of me are human.
Does a wink count as a blink?
Would you be opposed to an AI LLM engine that read all of substack.com, and published only the subset of substack that was AI-certified-as-AI-free.substack.com?
I think I would actually be somewhat opposed, even assuming perfect accuracy. I tend to think that we need to reach a kind of "accommodation" with AI. For example, I feel like the timing is quickly coming where AIs will be capable of quite good "developmental editing" that will make human writers better as human writers. (Sort of like how chess engines have advanced human chess.) I just don't see much of a future in rigidly insisting on being 100% AI free. We need to find a way to live with AI but retain the core of what we value in human writing.
Agreed. An AI-generated AI-score of how much an overall article was organic or not? AI-generated font/formatting?
Love this and so synchronous with the essay I just posted. This is exactly how I feel about AI—and I forgot to clarify in my piece that I'm talking about never using it again for my writing for many of the same reasons you cite, but I do use it for work/research, and am actually compelled to use it at work.
Funnily enough I just read a different essay that proclaimed it would never use AI to write, but it read like AI and tested as AI, but the author was adamant that he didn't use it. If he's telling the truth, thats rough for him, but I feel like he was lying. But I believe you fully because I just sense it in your character and in your writing. Just such a fascinating dynamic at play though.
Here's my lil essay if you're curious:
https://substack.com/@spencerrscott/p-202969300
Oh and for transparency, I wrote my anti-AI piece doing only #1 (really hard to avoid, and generally still went to wikipedia for general based answers instead of using the summaries). Otherwise I used good old thesaurus.com, even though I think using AI as a thesaurus is completely reasonable. Yet, writing this essay made me question if the exercise and care of sifting through synonyms mentally was perhaps worth it as a way to develop my writing. Obviously there are arbitrary thresholds, like I'm not going to be Wendall Berry who refused to ever write with a computer. And my main criteria for engagement with writing is (1) is it good/well-written (AI writing grates on me like fingernails on a chalk board, so if you're heavily using AI you better be a really good editor), and (2) does it signal a deep sense of care and proof of work. Within that I can allow hybrid use of it as a tool.
I actually wonder if alternating between writing with different AI ground rules might be the fastest way to get better at writing?
Excellent post! I too "guarantee that every word I post here is the product of me physically hitting keys with my fingers" - nice way of putting it.
Maybe typo: "the providence of every single part of my mental model" - I think you meant 'provenance'?
aaaaahhhh! this is the kind of extremely embarrassing thing that would have been caught if I had asked AI to look for errors!
But that's JUST what you'd do as an AI trying to pose as a human!