I find heritability quite confusing. I liked your explanation in the puzzlers post, though perhaps it confirmed that heritability *should* be confusing. Anyway, therefore, I find that I am confused here too. If you define away society and randomness and everything else "extrinsic" so that it doesn't "count" towards heritability... what's left? Wouldn't heritability always tautologically be 100% in that case?
Also, I find it suspicious that, as an input to their simulation they make some parameters correlate with each other at 0.5, and then get 50% as an answer on the other end. Are we sure they're not just measuring the assumptions of their simulation? But I guess we would need to know more details of how the simulation works to answer this.
> Wouldn't heritability always tautologically be 100% in that case?
Thank you! This is exactly the point I'm always trying to make! People seem to want to "remove" all these other factors, which they seem to think are masking "true heritability". But the ENTIRE POINT of heritability is to measure the impact of DNA alone vs. DNA + all those other factors.
This is why I'm always banging on about heritability being "just a ratio" and "just an observational statistic". It's perfectly fine to try to remove individual factors if you want (like this paper did). But this doesn't bring you closer to some "true" heritability number. It just makes your ratio closer and closer to one. If you remove all those other factors, then there's nothing left for heritability to try to measure.
> Also, I find it suspicious that, as an input to their simulation they make some parameters correlate with each other at 0.5, and then get 50% as an answer on the other end.
I THINK their justification is something like this:
- Assume each of the parameters in the death equation are determined as a linear function of individual genes. (Additive effects only.)
- Let x and y denote the genomes of two twins. If they are identical, they will be perfectly correlated. If they are fraternal, they will correlated with a coefficient of 0.5.
- But if x and y are correlated with a coefficient of r and f(x) is linear, then f(x) and f(y) are correlated with a coefficient of r.
- Therefore fraternal twins should have death-equal parameters that are correlated with a coefficient of 0.5
You can worry about various steps here, but it seems OK to me. And I don't think it's a foregone conclusion that this would lead to a heritability estimate of 0.5 after you actually put those correlated parameters into the equations and simulate lifespans. (Indeed, it seems to vary quite a bit by dataset.)
The compression of mortality (like compression of morbidity) is extending lifespans closer to the hard limit of life. Modern medicine, while not doing so great on preventing murders or getting hit by a bus, is materially reducing cardiovascular disease and cancer. Huzzah! Time to move on to figuring out how to stop frailty: https://www.unaging.com/aging/the-hard-limit-on-human-lifespan/
Ultimately, what we want to know is something like "how much of your aging trajectory can we predict from your parents/grandparents aging trajectory". Because "aging" is not a well-defined phenomenology, we use lifespan as a proxy readout - a little tautologically because the definition often used for aging is "increase in mortality risk with time".
So, we want to know about the genetics of aging, but we're stuck with a double proxy "the heritability of intrinsic mortality". This paper is an attempt to clean up the metric of 'heritability of mortality' by removing the extrinsic component which (unstated assumption) has nothing to do with someone DNA. This assumption isn't actually good, but it's the only reasonable place to begin.
Why is that assumption not good?
Well, consider "getting hit by a bus". Most people would just chalk that up to bad luck, therefore call it "extrinsic mortality" having nothing to do with your DNA, right? Well, as you get older, the 'intrinsic' variables in your DNA might cause loss of sensory acuity (see age-related macular degeneration; age-dependent hearing loss) as well as loss of cognitive function. Therefore I'd argue that "intrinsic aging" can ABSOLUTELY contribute to 'extrinsic mortality' metric.
This, among many, many, many, many, many other issue in the quality of work (and FUNDING) in aging-related research led me to abandon the field after working in it for almost 20 years.
I agree that if you interpret the paper that way, it's totally OK. It *is* interesting to ask about how heritable lifespan would be in a world without car accidents or HIV, and this paper takes a good run at answering that question. I just wrote this post because it seemed to me that most people were not interpreting the paper this way.
So I think we mostly agree. But I would like to raise one philosophical point:
> This paper is an attempt to clean up the metric of 'heritability of mortality' by removing the extrinsic component which (unstated assumption) has nothing to do with someone DNA.
The way heritability is defined, you've got (1) DNA and (2) everything else. A lot of people seem to seek a "true" heritability, one that's divorced from all the contingent facts about our actual world (all those facts that have nothing to do with DNA). I'm not sure this makes sense, and I kind of feel like this is trying to make the concept of "heritability" do something it's not capable of doing. Because what is our goal when we subtract things from the "everything else" column? Here we remove extrinsic mortality. That makes heritability go up. But you could also imagine a world in which everyone was fed equally as children, or a world in which everyone got access to equally good doctors. These would also make heritability go up. I'd also be interested to know how high the heritability of lifespan would be in a world where everyone got equal nutrition. But if we're doing all these things, where does it end? Why not just declare that the heritability of everything is 100%? Are we just trying to measure the impact of uncontrollable quantum events?
"The way heritability is defined, you've got (1) DNA and (2) everything else."
Sort of. True heritability is often simplified to "DNA", but nuanced geneticists understand that there might be multiple components of heritability that aren't DNA. (For a review: https://www.science.org/doi/epdf/10.1126/science.aaf4945) This is a quibble though.
You wrote:
" A lot of people seem to seek a "true" heritability, one that's divorced from all the contingent facts about our actual world (all those facts that have nothing to do with DNA). I'm not sure this makes sense,"
You're right- It doesn't. The technical definition of heritability is the OBSERVED variance of a trait in a population. Which means that whatever context that population is present in is a tacit background assumption. DNA can't do anything without a CONTEXT (often quite a narrow one, for example in the case of viruses). Human beings have a pretty broad context, which CAN put a cap on the total variation in the total human population that we can attribute to heritability. If you artificially narrow the context of what counts as your population, you will get higher heredity values, tautologically.
The question for the paper then is: is the 'narrowing' they did valid and give us some information about the nature of aging? I think the answer is yes. Specifically, that if you remove some of the more 'chaotic' aspects of extrinsic mortality risk, you get higher heritability. Look at it this way: they ended with 50%. It could have been higher (height is considered 80% heritable). This is telling. Even if you remove a ton of extrinsic factors to 'shove' heritability up, it hits a wall that height doesn't. Testable prediction: if you have a bunch of CLONES (100% genetic identity) they will die at different times, even if they live in a lab environment.
In some sense, we already knew this. Many aging studies are done with extremely homogenous genetic populations of worms, flies and mice. They still "age" differently! Why? It is not fully understood, but things like stochastic transcriptional drift or early somatic mutations during development might explain part of it.
On a more technical note, regarding their estimate as topping out at 50%. I'm interested in your thoughts experiment:
> Testable prediction: if you have a bunch of CLONES (100% genetic identity) they will die at different times, even if they live in a lab environment.
Say I raised fraternal and identical twins (or clones) in a lab environment. My guess is that this might well produce heritability estimates that are higher than 50%? My thinking is that there are additional "environmental" factors that aren't really "random" but also aren't accounted for by "shared environment". So if you're really trying to measure "biological determinism", it could still be much higher?
(Their model assumes that the age-of-death distribution is entirely determined by genetics. But I'm not 100% sure of the implications of this...)
I think it is going to be species dependent. For something like a fruit fly or worm, you might be able to push heritability of lifespan to close to 80% , but my guess is that it would be very difficult to do for a human
Great, I 100% agree! I think (hope?) this is consistent with everything in my post, because it's exactly the view I was trying to argue for. I really do think this paper is perfectly fine, if interpreted correctly, like you are doing. :) The only "correction" I was trying to make was that I felt like a lot of people were interpreting it differently and (perhaps controversially) the paper could have been written to make it less easy to misinterpret.
Science (and Nature) are magazines, not journals. I also don't understand why they are so highly regarded. Certainly in any field I have any expertise a "paper" in one of those two is likely to be useless and sometimes actively misleading.
I've also heard that at least was once technically considered a "magazine". But people don't really treat it like a magazine on their CVs or h-indexes, so I'm not sure if there's really a single correct answer. I do agree that the idea that "CSN are best" works OK-ish in biology, but in other fields they sort of contain a lot of random stuff, since most people never even submit and Science and Nature are weird in what they accept.
They are journals. A "magazine" typically does not send out submitted manuscripts for grueling peer review. Getting your work accepted into Science or Nature is usually a slog, and this difficulty is often lazily used as a proxy for "quality" or "importance". Just because that is unwarranted assumption doesn't mean that important work isn't quite often published in Science, Nature or Cell. If nothing else, they act as Schelling Points that signal author confidence, as they know even more eyeballs will be on their work after publication.
This imperfect system is unfortunately the best we can seemingly do. Various proposals for alternatives have gone nowhere in the past 15 years.
I'm just going by what a former editor of one of these said in a talk. I am fully aware of their elevated status, I just don't see any reason to trust that.
You cannot BLINDLY trust it. But in general you SHOULD pay more attention to a paper in your field published in Nature/Science/Cell than one published in very small niche journals with negligeable impact factors. Review/editorial standards there are often much, much worse.
"Now, in most journals, authors write everything. But Science has professional editors. Given that every single statistics-focused paper in Science seems to be like this, we probably shouldn’t blame the authors of this one. (Other than for their decision to publish in Science in the first place.)
I do wonder what those editors are doing, though."
The "professional editors" at outfits like Science (e.g. Nature/Cell etc) are not editors in the same way that, say, newspaper editors or book editors are editors. They're typically PhDs in a relevant field that left research after their PhD/postdoc to work at C/N/S, so they have *some* relevant scientific expertise, but their main job is to wrangle peer reviewers, make a final decision to publish or not, and make sure the paper looks pretty if they do publish. Very rarely are they going to dig deep into a paper enough to notice a typo, and they certainly won't do any rewriting or anything remotely close to actual *editing.*
Honestly I wonder how much of the problems at CNS in terms of shiny bullshit papers is due to the policy of having professional editors. I really appreciate that eLife and PNAS have working scientists as editors -you do run into issues of old boy networks and whatnot but when I submit or review a paper there I know that there's a decent chance the final decision is being made by an actual subject matter expert.
Huh, I am not confident, but my understanding was that at Science (unlike a normal journal) papers really do undergo substantial "developmental editing", which is why the papers are all structured so similarly. Though I think the people doing this type of editing are different from the "editors" who find reviewers and so on. The typo I pointed out is so glaring, I'd have been surprised if it wasn't found even by the reviewers! My guess was that the editors made some kind of change of notation and introduced the bug then...
Interesting -I've never heard of anything like that, but I could be wrong! I've never submitted or reviewed for Science, though I have for Nature, and I saw nothing of that sort there. As a rule most major journals have a pretty clear "house style" that they adhere to, but it's up to the authors to edit their paper into that style. For eLife, where I have the most experience, there is a substantial editing process where they make sure everything fits their formatting and rules and whatnot which takes absolutely forever, but they still don't touch the text itself. As for typos -yeah major typos get past reviewers all the time, when I review I *try* to catch typos but my focus is on the results, and frankly I think I put more effort into my reviews than a lot of big name professors who review for the high end journals.
> The first is to the Science writing style. This is a paper describing a statistical model. So shouldn’t there be somewhere in the paper where they explain exactly what they did, in order, from start to finish? Ostensibly, I think this is done in the left-hand column on the second page, just with little detail because Science is written for a general audience. But personally I think that description is the worst of all worlds. Instead of giving the high-level story in a coherent way, it throws random technical details at you without enough information to actually make sense of them.
This is generally how a lot of computational science articles are written. It drove a lot of my colleagues nuts because they are like, "Can you describe everything you did?" Statisticians cannot understand the statistical methods sections of most papers, me included.
> Statisticians cannot understand the statistical methods sections of most papers, me included.
100% agree. And to be fair, often it's much worse than this paper! I doubt I could reproduce these results, but I do think I understand the high-level idea of what they're doing and trying to accomplish. With many papers, I can't even get that.
This baffles me, because: OK, statisticians don't understand those sections. But is that because they aren't for statisticians? If you were to optimize a statistical methods section to be maximally comprehensible by non-statisticians, surely it would look completely different. So what are we doing?
Well on the surface of it scientists expect that how they write about their use of statistics methods to draw conclusions would be comprehensible to us.
But what actually happens is that scientists have come to rest on a certain set of software usage + statistical rituals that to quantitative outsiders is grossly underspecified. It is also actually underspecified because there might be a mismatch between the scientific question and the way it was formalized.
Once I was in neuroimaging long enough, I could guess what was done when I read those sections. But my own writing habits spelled everything out clearly so there was no ambiguity. And I’ve noticed that applied mathematicians, physicists etc.. who enter the life sciences do the same. In these cases we aren't writing for mainstream statisticians but for domain scientists. We just spell exactly what we are doing out clearly with assumptions/decisions made along the way.
The fact that some fields just share their code by default now perhaps makes the underspecification issue less severe. But it doesn’t change the fact that scientists don’t separate the target of inference from the procedure used to infer the target very cleanly in their own minds. This is reflected in how they describe what they did.
> But what actually happens is that scientists have come to rest on a certain set of software usage + statistical rituals that to quantitative outsiders is grossly underspecified.
They just sort of say (1) we have data and (2) here are a bunch of equations. I thought: What the hell are these equations? What did you do? After staring at the paper for a while, I figured out, "OK, I guess you're doing a linear least-squares regression? And these equations are describing your regressor?" But they never say that, AT ALL. To an outsider, this is insane, because while there's nothing wrong with linear least-squares regression, it's just methodology out of the many. But I guess people in their subfield "always" use it, and so it's so "obvious" that it doesn't need to be said?
Yeah it is a bit like that. But my head is so confused when I see a list of anovas, F-tests and t-tests and no sentences describing what model was fit. But if they asked a standard set of questions from that type of experiment, insiders know what it is. But this is also why bad methods just persist without getting corrected for an unbounded period of time.
when I was in grad school the joke went "articles in JBC are always right and occasionally interesting, while articles in Cell are always interesting and occasionally right"
Journal of Biological Chemistry is boring and full of graphs, while Cell has fun full-color pics
They are estimating lifespan heritability within the population that didn't die of external causes.
Most people would define haircolor as the natural hair color before turning gray. I think in the same way most people think of a "natural" intrinsic lifespan excluding car accidents etc.
"Science has decided that papers should contain almost no math, even when the paper in question is about math. So I’m mostly working from an English description." The math is there, but in the Supplementary Material!
Alos I do agree that the title might be kind of misleading but focusing on intrinsic mortality is common (and makes sense!) when studying lifespan with a focus on ageing.
> The math is there, but in the Supplementary Material!
Please see my comments on the supplementary material. :)
Anyway, I really do agree that studying "intrinsic mortality" is fine! I think the actual research in this paper is good/interesting. I only object to the "spin" and the writing.
re: morbid, I've had a psychiatrist tell me once that in grief, "the only thing more intensely painful than the loss of a child, is the loss of twin." And I have twins, so yeah, thanks for that ;)
I find heritability quite confusing. I liked your explanation in the puzzlers post, though perhaps it confirmed that heritability *should* be confusing. Anyway, therefore, I find that I am confused here too. If you define away society and randomness and everything else "extrinsic" so that it doesn't "count" towards heritability... what's left? Wouldn't heritability always tautologically be 100% in that case?
Also, I find it suspicious that, as an input to their simulation they make some parameters correlate with each other at 0.5, and then get 50% as an answer on the other end. Are we sure they're not just measuring the assumptions of their simulation? But I guess we would need to know more details of how the simulation works to answer this.
> Wouldn't heritability always tautologically be 100% in that case?
Thank you! This is exactly the point I'm always trying to make! People seem to want to "remove" all these other factors, which they seem to think are masking "true heritability". But the ENTIRE POINT of heritability is to measure the impact of DNA alone vs. DNA + all those other factors.
This is why I'm always banging on about heritability being "just a ratio" and "just an observational statistic". It's perfectly fine to try to remove individual factors if you want (like this paper did). But this doesn't bring you closer to some "true" heritability number. It just makes your ratio closer and closer to one. If you remove all those other factors, then there's nothing left for heritability to try to measure.
> Also, I find it suspicious that, as an input to their simulation they make some parameters correlate with each other at 0.5, and then get 50% as an answer on the other end.
I THINK their justification is something like this:
- Assume each of the parameters in the death equation are determined as a linear function of individual genes. (Additive effects only.)
- Let x and y denote the genomes of two twins. If they are identical, they will be perfectly correlated. If they are fraternal, they will correlated with a coefficient of 0.5.
- But if x and y are correlated with a coefficient of r and f(x) is linear, then f(x) and f(y) are correlated with a coefficient of r.
- Therefore fraternal twins should have death-equal parameters that are correlated with a coefficient of 0.5
You can worry about various steps here, but it seems OK to me. And I don't think it's a foregone conclusion that this would lead to a heritability estimate of 0.5 after you actually put those correlated parameters into the equations and simulate lifespans. (Indeed, it seems to vary quite a bit by dataset.)
'''Intrinsic''' heritability does seem to get at an idea that's useful, or at least of interest to people I would say.
The compression of mortality (like compression of morbidity) is extending lifespans closer to the hard limit of life. Modern medicine, while not doing so great on preventing murders or getting hit by a bus, is materially reducing cardiovascular disease and cancer. Huzzah! Time to move on to figuring out how to stop frailty: https://www.unaging.com/aging/the-hard-limit-on-human-lifespan/
Ultimately, what we want to know is something like "how much of your aging trajectory can we predict from your parents/grandparents aging trajectory". Because "aging" is not a well-defined phenomenology, we use lifespan as a proxy readout - a little tautologically because the definition often used for aging is "increase in mortality risk with time".
So, we want to know about the genetics of aging, but we're stuck with a double proxy "the heritability of intrinsic mortality". This paper is an attempt to clean up the metric of 'heritability of mortality' by removing the extrinsic component which (unstated assumption) has nothing to do with someone DNA. This assumption isn't actually good, but it's the only reasonable place to begin.
Why is that assumption not good?
Well, consider "getting hit by a bus". Most people would just chalk that up to bad luck, therefore call it "extrinsic mortality" having nothing to do with your DNA, right? Well, as you get older, the 'intrinsic' variables in your DNA might cause loss of sensory acuity (see age-related macular degeneration; age-dependent hearing loss) as well as loss of cognitive function. Therefore I'd argue that "intrinsic aging" can ABSOLUTELY contribute to 'extrinsic mortality' metric.
This, among many, many, many, many, many other issue in the quality of work (and FUNDING) in aging-related research led me to abandon the field after working in it for almost 20 years.
Now I teach, which is only slightly more cursed.
I agree that if you interpret the paper that way, it's totally OK. It *is* interesting to ask about how heritable lifespan would be in a world without car accidents or HIV, and this paper takes a good run at answering that question. I just wrote this post because it seemed to me that most people were not interpreting the paper this way.
So I think we mostly agree. But I would like to raise one philosophical point:
> This paper is an attempt to clean up the metric of 'heritability of mortality' by removing the extrinsic component which (unstated assumption) has nothing to do with someone DNA.
The way heritability is defined, you've got (1) DNA and (2) everything else. A lot of people seem to seek a "true" heritability, one that's divorced from all the contingent facts about our actual world (all those facts that have nothing to do with DNA). I'm not sure this makes sense, and I kind of feel like this is trying to make the concept of "heritability" do something it's not capable of doing. Because what is our goal when we subtract things from the "everything else" column? Here we remove extrinsic mortality. That makes heritability go up. But you could also imagine a world in which everyone was fed equally as children, or a world in which everyone got access to equally good doctors. These would also make heritability go up. I'd also be interested to know how high the heritability of lifespan would be in a world where everyone got equal nutrition. But if we're doing all these things, where does it end? Why not just declare that the heritability of everything is 100%? Are we just trying to measure the impact of uncontrollable quantum events?
You wrote:
"The way heritability is defined, you've got (1) DNA and (2) everything else."
Sort of. True heritability is often simplified to "DNA", but nuanced geneticists understand that there might be multiple components of heritability that aren't DNA. (For a review: https://www.science.org/doi/epdf/10.1126/science.aaf4945) This is a quibble though.
You wrote:
" A lot of people seem to seek a "true" heritability, one that's divorced from all the contingent facts about our actual world (all those facts that have nothing to do with DNA). I'm not sure this makes sense,"
You're right- It doesn't. The technical definition of heritability is the OBSERVED variance of a trait in a population. Which means that whatever context that population is present in is a tacit background assumption. DNA can't do anything without a CONTEXT (often quite a narrow one, for example in the case of viruses). Human beings have a pretty broad context, which CAN put a cap on the total variation in the total human population that we can attribute to heritability. If you artificially narrow the context of what counts as your population, you will get higher heredity values, tautologically.
The question for the paper then is: is the 'narrowing' they did valid and give us some information about the nature of aging? I think the answer is yes. Specifically, that if you remove some of the more 'chaotic' aspects of extrinsic mortality risk, you get higher heritability. Look at it this way: they ended with 50%. It could have been higher (height is considered 80% heritable). This is telling. Even if you remove a ton of extrinsic factors to 'shove' heritability up, it hits a wall that height doesn't. Testable prediction: if you have a bunch of CLONES (100% genetic identity) they will die at different times, even if they live in a lab environment.
In some sense, we already knew this. Many aging studies are done with extremely homogenous genetic populations of worms, flies and mice. They still "age" differently! Why? It is not fully understood, but things like stochastic transcriptional drift or early somatic mutations during development might explain part of it.
On a more technical note, regarding their estimate as topping out at 50%. I'm interested in your thoughts experiment:
> Testable prediction: if you have a bunch of CLONES (100% genetic identity) they will die at different times, even if they live in a lab environment.
Say I raised fraternal and identical twins (or clones) in a lab environment. My guess is that this might well produce heritability estimates that are higher than 50%? My thinking is that there are additional "environmental" factors that aren't really "random" but also aren't accounted for by "shared environment". So if you're really trying to measure "biological determinism", it could still be much higher?
(Their model assumes that the age-of-death distribution is entirely determined by genetics. But I'm not 100% sure of the implications of this...)
I think it is going to be species dependent. For something like a fruit fly or worm, you might be able to push heritability of lifespan to close to 80% , but my guess is that it would be very difficult to do for a human
Great, I 100% agree! I think (hope?) this is consistent with everything in my post, because it's exactly the view I was trying to argue for. I really do think this paper is perfectly fine, if interpreted correctly, like you are doing. :) The only "correction" I was trying to make was that I felt like a lot of people were interpreting it differently and (perhaps controversially) the paper could have been written to make it less easy to misinterpret.
Yes, I guess I could have just written "good post" but the discussion/clarification was valuable.
Science (and Nature) are magazines, not journals. I also don't understand why they are so highly regarded. Certainly in any field I have any expertise a "paper" in one of those two is likely to be useless and sometimes actively misleading.
This seems really strange to me. In my field, biology, S and N usually publish great high profile papers, certainly not useless ones.
Oh, what you say is definitely true for biology. But the further you get from biology, the less true it is. (At least in my opinion.)
I've also heard that at least was once technically considered a "magazine". But people don't really treat it like a magazine on their CVs or h-indexes, so I'm not sure if there's really a single correct answer. I do agree that the idea that "CSN are best" works OK-ish in biology, but in other fields they sort of contain a lot of random stuff, since most people never even submit and Science and Nature are weird in what they accept.
They are journals. A "magazine" typically does not send out submitted manuscripts for grueling peer review. Getting your work accepted into Science or Nature is usually a slog, and this difficulty is often lazily used as a proxy for "quality" or "importance". Just because that is unwarranted assumption doesn't mean that important work isn't quite often published in Science, Nature or Cell. If nothing else, they act as Schelling Points that signal author confidence, as they know even more eyeballs will be on their work after publication.
This imperfect system is unfortunately the best we can seemingly do. Various proposals for alternatives have gone nowhere in the past 15 years.
I'm just going by what a former editor of one of these said in a talk. I am fully aware of their elevated status, I just don't see any reason to trust that.
You cannot BLINDLY trust it. But in general you SHOULD pay more attention to a paper in your field published in Nature/Science/Cell than one published in very small niche journals with negligeable impact factors. Review/editorial standards there are often much, much worse.
This might be true if I were a biological researcher. I am not.
Then you should probably refrain from making blanket statements like "Science is a magazine not a journal", don't you think?
As I said: from the horse's mouth.
Love this post, one nitpick:
"Now, in most journals, authors write everything. But Science has professional editors. Given that every single statistics-focused paper in Science seems to be like this, we probably shouldn’t blame the authors of this one. (Other than for their decision to publish in Science in the first place.)
I do wonder what those editors are doing, though."
The "professional editors" at outfits like Science (e.g. Nature/Cell etc) are not editors in the same way that, say, newspaper editors or book editors are editors. They're typically PhDs in a relevant field that left research after their PhD/postdoc to work at C/N/S, so they have *some* relevant scientific expertise, but their main job is to wrangle peer reviewers, make a final decision to publish or not, and make sure the paper looks pretty if they do publish. Very rarely are they going to dig deep into a paper enough to notice a typo, and they certainly won't do any rewriting or anything remotely close to actual *editing.*
Honestly I wonder how much of the problems at CNS in terms of shiny bullshit papers is due to the policy of having professional editors. I really appreciate that eLife and PNAS have working scientists as editors -you do run into issues of old boy networks and whatnot but when I submit or review a paper there I know that there's a decent chance the final decision is being made by an actual subject matter expert.
Huh, I am not confident, but my understanding was that at Science (unlike a normal journal) papers really do undergo substantial "developmental editing", which is why the papers are all structured so similarly. Though I think the people doing this type of editing are different from the "editors" who find reviewers and so on. The typo I pointed out is so glaring, I'd have been surprised if it wasn't found even by the reviewers! My guess was that the editors made some kind of change of notation and introduced the bug then...
Interesting -I've never heard of anything like that, but I could be wrong! I've never submitted or reviewed for Science, though I have for Nature, and I saw nothing of that sort there. As a rule most major journals have a pretty clear "house style" that they adhere to, but it's up to the authors to edit their paper into that style. For eLife, where I have the most experience, there is a substantial editing process where they make sure everything fits their formatting and rules and whatnot which takes absolutely forever, but they still don't touch the text itself. As for typos -yeah major typos get past reviewers all the time, when I review I *try* to catch typos but my focus is on the results, and frankly I think I put more effort into my reviews than a lot of big name professors who review for the high end journals.
I don't think this is totally conclusive, but I think it does confirm that at least some of the editing I was imagining does indeed occur:
https://www.science.org/content/page/science-information-authors#:~:text=Manuscripts%20selected%20for%20publication%20are%20edited%20to%20improve%20accuracy%20and%20clarity%2C%20as%20well%20as%20for%20length%2E
> The first is to the Science writing style. This is a paper describing a statistical model. So shouldn’t there be somewhere in the paper where they explain exactly what they did, in order, from start to finish? Ostensibly, I think this is done in the left-hand column on the second page, just with little detail because Science is written for a general audience. But personally I think that description is the worst of all worlds. Instead of giving the high-level story in a coherent way, it throws random technical details at you without enough information to actually make sense of them.
This is generally how a lot of computational science articles are written. It drove a lot of my colleagues nuts because they are like, "Can you describe everything you did?" Statisticians cannot understand the statistical methods sections of most papers, me included.
> Statisticians cannot understand the statistical methods sections of most papers, me included.
100% agree. And to be fair, often it's much worse than this paper! I doubt I could reproduce these results, but I do think I understand the high-level idea of what they're doing and trying to accomplish. With many papers, I can't even get that.
This baffles me, because: OK, statisticians don't understand those sections. But is that because they aren't for statisticians? If you were to optimize a statistical methods section to be maximally comprehensible by non-statisticians, surely it would look completely different. So what are we doing?
Well on the surface of it scientists expect that how they write about their use of statistics methods to draw conclusions would be comprehensible to us.
But what actually happens is that scientists have come to rest on a certain set of software usage + statistical rituals that to quantitative outsiders is grossly underspecified. It is also actually underspecified because there might be a mismatch between the scientific question and the way it was formalized.
Once I was in neuroimaging long enough, I could guess what was done when I read those sections. But my own writing habits spelled everything out clearly so there was no ambiguity. And I’ve noticed that applied mathematicians, physicists etc.. who enter the life sciences do the same. In these cases we aren't writing for mainstream statisticians but for domain scientists. We just spell exactly what we are doing out clearly with assumptions/decisions made along the way.
The fact that some fields just share their code by default now perhaps makes the underspecification issue less severe. But it doesn’t change the fact that scientists don’t separate the target of inference from the procedure used to infer the target very cleanly in their own minds. This is reflected in how they describe what they did.
> But what actually happens is that scientists have come to rest on a certain set of software usage + statistical rituals that to quantitative outsiders is grossly underspecified.
This is a very helpful way to think about it, thanks. I remember when I was reading this paper: https://dynomight.net/are-tests-irrelevant/
They just sort of say (1) we have data and (2) here are a bunch of equations. I thought: What the hell are these equations? What did you do? After staring at the paper for a while, I figured out, "OK, I guess you're doing a linear least-squares regression? And these equations are describing your regressor?" But they never say that, AT ALL. To an outsider, this is insane, because while there's nothing wrong with linear least-squares regression, it's just methodology out of the many. But I guess people in their subfield "always" use it, and so it's so "obvious" that it doesn't need to be said?
Yeah it is a bit like that. But my head is so confused when I see a list of anovas, F-tests and t-tests and no sentences describing what model was fit. But if they asked a standard set of questions from that type of experiment, insiders know what it is. But this is also why bad methods just persist without getting corrected for an unbounded period of time.
when I was in grad school the joke went "articles in JBC are always right and occasionally interesting, while articles in Cell are always interesting and occasionally right"
Journal of Biological Chemistry is boring and full of graphs, while Cell has fun full-color pics
They are estimating lifespan heritability within the population that didn't die of external causes.
Most people would define haircolor as the natural hair color before turning gray. I think in the same way most people think of a "natural" intrinsic lifespan excluding car accidents etc.
"Science has decided that papers should contain almost no math, even when the paper in question is about math. So I’m mostly working from an English description." The math is there, but in the Supplementary Material!
Alos I do agree that the title might be kind of misleading but focusing on intrinsic mortality is common (and makes sense!) when studying lifespan with a focus on ageing.
> The math is there, but in the Supplementary Material!
Please see my comments on the supplementary material. :)
Anyway, I really do agree that studying "intrinsic mortality" is fine! I think the actual research in this paper is good/interesting. I only object to the "spin" and the writing.
Science: optimizing for attention span and not reproducibility.
In the past I've emailed the authors about stuff like this.
heh. nice commentary on _Science_
re: morbid, I've had a psychiatrist tell me once that in grief, "the only thing more intensely painful than the loss of a child, is the loss of twin." And I have twins, so yeah, thanks for that ;)