My more-hardcore theanine self-experiment

coffee is bad

dynomight.net/theanine-2

Theanine is an amino acid that occurs naturally in tea. Many people take it as a supplement for stress or anxiety. It’s mechanistically plausible, but the scientific literature hasn’t been able to find much of a benefit. So I ran a 16-month blinded self-experiment in the hopes of showing it worked. It did not work.

At the end of the post, I put out a challenge: If you think theanine, prove it. Run a blinded self-experiment. After all, if it works, then what are you afraid of?

Well, it turns out that Luis Costigan had already run a self-experiment. Here was his protocol:

1. Each morning, take 200 mg theanine or placebo (blinded) along with a small iced coffee.

2. Wait 90 minutes.

3. Record anxiety on a subjective scale of 0-10.

He repeated this for 20 days. His mean anxiety after theanine was 4.2 and after placebo it was 5.0. A simple Bayesian analysis said there was an 82.6% chance theanine reduced anxiety.

The p-value was 0.31, but this is a Bayesian blog—this is what you'd expect with a sample size of 20.

I think this experiment was good, both the experiment and the analysis. It doesn’t prove theanine works, but it was enough to make me wonder: Maybe theanine does work, but I somehow failed to bring out the effect? What would give theanine the best possible chance of working?

Theanine is widely reported to help with anxiety from caffeine. While I didn’t explicitly take caffeine as part of my previous experiment, I drink tea almost every day, so I figured that if theanine helps, it should have shown up.

But most people (and Luis) take theanine with coffee, not tea. I find that coffee makes me much more nervous than tea. For this reason, I sort of hate coffee and rarely drink it.

Maybe the tiny amounts of natural theanine in tea masked the effects of the supplements? Or maybe you need to take theanine and caffeine at the same time? Or maybe for some strange reason theanine works for coffee (or coffee-tier anxiety) but not tea?

So fine. To hell with my mental health. I decided to take theanine (or placebo) together with coffee on an empty stomach first thing in the day. And I decided to double the dose of theanine from 200 mg to 400 mg.

Details

Coffee. I used one of those pod machines which are incredibly uncool but presumably deliver a consistent amount of caffeine.

Measurements. Each day I recorded my stress levels on a subjective 1-5 scale before I took the capsules. An hour later, I recorded my end stress levels, and my percentage prediction that what I took was actually theanine.

Blinding. I have capsules that either contain 200 mg of theanine or 25 mcg of vitamin D. These are exactly the same size. I struggled for a while to see how to take two pills of the same type while being blind to the results. In the end, I put two pills of each type in identical looking cups and shuffled the cups. Then I shut my eyes, took a sip of coffee (to make sure I couldn’t taste any difference), swallowed the pills on one cup, and put the others into a numbered envelope.

Here’s a picture of the envelopes, to prove I actually did this and/or invite sympathy for all the coffee I had to endure:

After 37 days I ran out of capsules.

Initial thoughts

I’m going to try something new. As I write these words, I have not yet opened the envelopes, so I don’t know the results. I’m going to register some thoughts.

My main thought is: I have no idea what the results will show.

It really felt like on some days I got the normal spike of anxiety I expect from coffee and on other days it was almost completely gone. But in my previous experiment I often felt the same thing and was proven wrong. It wouldn’t surprise me if the results show a strong effect, or if it’s all completely random.

I’ll also pre-register (sort of) the statistical analyses I intend to do:

1. I’ll plot the data.

2. I’ll repeat Luis’s Bayesian analysis, which looks at end stress levels only.

3. I’ll repeat that again, but looking at the change in stress levels.

4. I’ll repeat that again, but looking at my percentage prediction that what I actually took was theanine vs. placebo.

5. I’ll compute regular-old confidence intervals and p-values for end stress, change in stress, and my percentage prediction that what I actually took was theanine vs. placebo.

Intermission

Please hold while I open all the envelopes and do the analyses. Here’s a painting.

Plots

Here are the raw stress levels. Each line line shows one trial, with the start marked with a small horizontal bar. Remember, this measures the effect of coffee and the supplement. So even though stress tends to go up, this would still show a benefit if it went up less with theanine.

Here is the difference in stress levels. If Δ Stress is negative, that means stress went down.

Here are the start vs. end stress levels, ignoring time. The dotted line shows equal stress levels, so anything below that line means stress went down.

And finally, here are my percentage predictions of if what I had taken was actually theanine:

So…. nothing jumps out so far.

Analysis

So I did the analysis in my pre-registered plan above. In the process, I realized I wanted to show some extra stuff. It’s all simple and I think unobjectionable. But if you’re the kind of paranoid person who only trusts pre-registered things, I love and respect you and I will mark those with “✔️”.

End stress

The first thing we’ll look at is the final stress levels, one hour after taking theanine or vitamin D. First up, regular-old frequentist statistics.

If the difference is less than zero, that would suggest theanine was better. It looks like there might be a small difference, but it’s nowhere near statistically significant.

Next up, Bayes! In this analysis, there are latent variables for the mean and standard deviation of end stress (after one hour) with theanine and also for vitamin D. Following Luis’s analysis, these each have a Gaussian prior with a mean and standard deviation based on the overall mean in the data.

The results are extremely similar to the frequentist analysis. This says there’s an 80% chance theanine is better.

Δ Stress

Next up, let’s look at the difference in stress levels defined as Δ = (end - start). Since this measures an increase in stress, we’d like it to be as small as possible. So again, if the difference is negative, that would suggest theanine is better. Here are the good-old frequentist statistics.

And here’s the Bayesian analysis. It’s just like the first one except we have latent variables for the difference in stress levels (end-start). If the difference of that difference was less than zero, that would again suggest theanine was better.

In retrospect, this percentage prediction analysis is crazy, and I suggest you ignore it. The issue is that even though Δ stress is usually positive (coffee bad) it’s near zero and can be negative. Computing (T-D)/D when D can be negative is stupid and I think makes the whole calculation meaningless. I regret pre-registering this.

The absolute difference is fine. It’s very close (almost suspiciously close) to zero.

Percentage prediction

Finally, let’s look at my percentage prediction that what I took was theanine. It really felt like I could detect a difference. But could I?

Here we’d hope that I’d give a higher prediction that I’d taken theanine when I’d actually taken theanine. So a positive difference would suggest theanine is better, or at least different.

And here’s the corresponding Bayesian analysis. This is just like the first two, except with latent variables for my percentage prediction under theanine and vitamin D.

Taking a percentage difference of a quantity that is itself a percentage difference is really weird, but fine.

Discussion

This is the most annoying possible outcome. A clear effect would have made me happy. Clear evidence of no effect would also have made me happy. Instead, some analyses say there might be a small effect, and others suggest nothing. Ugh.

But I’ll say this: If there is any effect, it’s small. I know many people say theanine is life-changing, and I know why: It’s insanely easy to fool yourself. Even after running a previous 18-month trial and finding no effect, I still often felt like I could feel the effects in this experiment. I still thought I might open up all the envelopes and find that I had been under-confident in my guesses. Instead, I barely did better than chance.

So I maintain my previous rule. If you claim that theanine has huge effects for you, blind experiment or GTFO.