Reminds me of the story of Richard Feynman and the investigation into the Challenger disaster. During a public hearing, he put a small rubber o-ring into a glass of ice water and showed that it loses elasticity when it's cold. Earlier in the investigation, he bypassed the NASA bureaucracy and talked directly to the engineers who actually worked on the shuttle.
This is an entertaining but deceptively deep parable. Well done.
Most of the world's rationalists would do well to add some empiricism to their diet.
Contrary to your last sentence, I find this lesson more sweet than bitter -- because in many cases, including this one, an empiricist approach gets you to the answer more quickly and easily.
I'm regretting using the term "bitter"! I was trying to make an analogy to the "bitter lesson" for AI ("methods that leverage computation are ultimately the most effective") Maybe something like, "methods that leverage information gathering are ultimately the most effective". But that's a semi-obscure reference and gives a bleak tone that I didn't really intend.
I got the reference. And I think it also points at a deeper implication to consider. We're not really going to be bottlenecked on most types of thinking in the future. We already have AI that is near PhD-level in some ways, but it gets easily confused or limited by incorrect or missing data. Relevant information is about to become much more valuable, whether it's a visual inspection like in this example or open source data sets and publications.
For the time being, a human in the loop is also still relevant because AI gets easily confused in general, but it's not clear that's going to last. The data constraint seems more durable.
Thanks! Needed the giggling. I am very familiar with that pipe strap and rattle. I had a [probably] similar rattle coming from somewhere under my Ranger truck for months. One morning the entire muffler/tailpipe assembly fell onto the road as I was driving. Mechanic found that 2 of these pipe straps and a log had been used to brace the tailpipe/muffler into its space. Time had killed the pipe straps, but as the log was still in fine shape and there was nothing unrusted in the undercarriage to bolt things to , the mechanic used $6 of new strapping and re-braced everything. Rattle fixed.
Yeah, after pulling that off the car I was a little disturbed that something wasn't clamped that should have been clamped. (Though it had been years, so it clearly wasn't *that* critical.) I did eventually figure it out and add a new strap.
P.S. A *log*, wow. I guess it's hard to complain, given that it outlasted the straps...
Ten years prior, I had worked on an organic farm and shared the truck with field workers. They returned the truck one afternoon, scrubbed, shining, interior spotless-unusual in working farm vehicle-and happy smiles all around. Thinking back, that must have been the day that the muffler crapped out on them 80 acres away from the shop hence the log and strap field fix. I had often marvelled [and depended] at their ability to problem solve on the spot with whatever stuff was lying around. Mechanic said the log would likely last as long as I determined to keep the truck on the road, and it did. 😔
Zen and the Art of Motorcycle Maintenance by Robert Pirsig, Part II, Ch 9:
(Disclaimer: generally not my sort of book; would advise bailing on it when it starts getting weird; your mileage may vary...)
"Two kinds of logic are used, inductive and deductive. Inductive inferences start with observations of the machine and arrive at general conclusions. For example, if the cycle goes over a bump and the engine misfires, and then goes over another bump and the engine misfires, and then goes over another bump and the engine misfires, and then goes over a long smooth stretch of road and there is no misfiring, and then goes over a fourth bump and the engine misfires again, one can logically conclude that the misfiring is caused by the bumps. That is induction: reasoning from particular experiences to general truths.
Deductive inferences do the reverse. They start with general knowledge and predict a specific observation. For example, if, from reading the hierarchy of facts about the machine, the mechanic knows the horn of the cycle is powered exclusively by electricity from the battery, then he can logically infer that if the battery is dead the horn will not work. That is deduction. Solution of problems too complicated for common sense to solve is achieved by long strings of mixed inductive and deductive inferences that weave back and forth between the observed machine and the mental hierarchy of the machine found in the manuals. The correct program for this interweaving is formalized as scientific method. Actually I’ve never seen a cycle-maintenance problem complex enough really to require full-scale formal scientific method. Repair problems are not that hard. When I think of formal scientific method an image sometimes comes to mind of an enormous juggernaut, a huge bulldozer...slow, tedious lumbering, laborious, but invincible. It takes twice as long, five times as long, maybe a dozen times as long as informal mechanic’s techniques, but you know in the end you’re going to get it. There’s no fault isolation problem in motorcycle maintenance that can stand up to it. When you’ve hit a really tough one, tried everything, racked your brain and nothing works, and you know that this time Nature has really decided to be difficult, you say, "Okay, Nature, that’s the end of the nice guy," and you crank up the formal scientific method.
For this you keep a lab notebook. Everything gets written down, formally, so that you know at all times where you are, where you’ve been, where you’re going and where you want to get. In scientific work and electronics technology this is necessary because otherwise the problems get so complex you get lost in them and confused and forget what you know and what you don’t know and have to give up. In cycle maintenance things are not that involved, but when confusion starts it’s a good idea to hold it down by making everything formal and exact. Sometimes just the act of writing down the problems straightens out your head as to what they really are. The logical statements entered into the notebook are broken down into six categories: (1) statement of the problem, (2) hypotheses as to the cause of the problem, (3) experiments designed to test each hypothesis, (4) predicted results of the experiments, (5) observed results of the experiments and (6) conclusions from the results of the experiments. This is not different from the formal arrangement of many college and high-school lab notebooks but the purpose here is no longer just busywork. The purpose now is precise guidance of thoughts that will fail if they are not accurate. The real purpose of scientific method is to make sure Nature hasn’t misled you into thinking you know something you don’t actually know. There’s not a mechanic or scientist or technician alive who hasn’t suffered from that one so much that he’s not instinctively on guard. That’s the main reason why so much scientific and mechanical information sounds so dull and so cautious. If you get careless or go romanticizing scientific information, giving it a flourish here and there, Nature will soon make a complete fool out of you. It does it often enough anyway even when you don’t give it opportunities. One must be extremely careful and rigidly logical when dealing with Nature: one logical slip and an entire scientific edifice comes tumbling down. One false deduction about the machine and you can get hung up indefinitely.
In Part One of formal scientific method, which is the statement of the problem, the main skill is in stating absolutely no more than you are positive you know. It is much better to enter a statement "Solve Problem: Why doesn’t cycle work?" which sounds dumb but is correct, than it is to enter a statement "Solve Problem: What is wrong with the electrical system?" when you don’t absolutely know the trouble is in the electrical system. What you should state is "Solve Problem: What is wrong with cycle?" and then state as the first entry of Part Two: "Hypothesis Number One: The trouble is in the electrical system." You think of as many hypotheses as you can, then you design experiments to test them to see which are true and which are false. This careful approach to the beginning questions keeps you from taking a major wrong turn which might cause you weeks of extra work or can even hang you up completely. Scientific questions often have a surface appearance of dumbness for this reason. They are asked in order to prevent dumb mistakes later on. Part Three, that part of formal scientific method called experimentation, is sometimes thought of by romantics as all of science itself because that’s the only part with much visual surface. They see lots of test tubes and bizarre equipment and people running around making discoveries. They do not see the experiment as part of a larger intellectual process and so they often confuse experiments with demonstrations, which look the same. A man conducting a gee-whiz science show with fifty thousand dollars’ worth of Frankenstein equipment is not doing anything scientific if he knows beforehand what the results of his efforts are going to be. A motorcycle mechanic, on the other hand, who honks the horn to see if the battery works is informally conducting a true scientific experiment. He is testing a hypothesis by putting the question to nature. The TV scientist who mutters sadly, "The experiment is a failure; we have failed to achieve what we had hoped for," is suffering mainly from a bad scriptwriter. An experiment is never a failure solely because it fails to achieve predicted results. An experiment is a failure only when it also fails adequately to test the hypothesis in question, when the data it produces don’t prove anything one way or another.
Skill at this point consists of using experiments that test only the hypothesis in question, nothing less, nothing more. If the horn honks, and the mechanic concludes that the whole electrical system is working, he is in deep trouble. He has reached an illogical conclusion. The honking horn only tells him that the battery and horn are working. To design an experiment properly he has to think very rigidly in terms of what directly causes what. This you know from the hierarchy. The horn doesn’t make the cycle go. Neither does the battery, except in a very indirect way. The point at which the electrical system directly causes the engine to fire is at the spark plugs, and if you don’t test here, at the output of the electrical system, you will never really know whether the failure is electrical or not. To test properly the mechanic removes the plug and lays it against the engine so that the base around the plug is electrically grounded, kicks the starter lever and watches the spark plug gap for a blue spark. If there isn’t any he can conclude one of two things: (a) there is an electrical failure or (b) his experiment is sloppy. If he is experienced he will try it a few more times, checking connections, trying every way he can think of to get that plug to fire. Then, if he can’t get it to fire, he finally concludes that a is correct, there’s an electrical failure, and the experiment is over. He has proved that his hypothesis is correct. In the final category, conclusions, skill comes in stating no more than the experiment has proved. It hasn’t proved that when he fixes the electrical system the motorcycle will start. There may be other things wrong. But he does know that the motorcycle isn’t going to run until the electrical system is working and he sets up the next formal question: "Solve problem: what is wrong with the electrical system?"
He then sets up hypotheses for these and tests them. By asking the right questions and choosing the right tests and drawing the right conclusions the mechanic works his way down the echelons of the motorcycle hierarchy until he has found the exact specific cause or causes of the engine failure, and then he changes them so that they no longer cause the failure. An untrained observer will see only physical labor and often get the idea that physical labor is mainly what the mechanic does. Actually the physical labor is the smallest and easiest part of what the mechanic does. By far the greatest part of his work is careful observation and precise thinking. That is why mechanics sometimes seem so taciturn and withdrawn when performing tests. They don’t like it when you talk to them because they are concentrating on mental images, hierarchies, and not really looking at you or the physical motorcycle at all. They are using the experiment as part of a program to expand their hierarchy of knowledge of the faulty motorcycle and compare it to the correct hierarchy in their mind. They are looking at underlying form."
Tangential, but it's incredible that when I read this, my brain was still able to conjure up and read it to me using the voice of the narrator of the audiobook
My goodness, yeah, I get this too! F'rinstance I listened to Sandi Toksvig chair the News Quiz (https://www.youtube.com/watch?v=9p8Ak6_isbA) for years - and now, any time I read anything she's written, somehow Sandi Toksvig reads it out aloud inside my head....
I suspect D̶y̶n̶o̶j̶e̶t̶ Dynomight might be trying to make a broader point here, which I'm totally missing, but I think the general approach ("complex system broken: try to understand it") is actually pretty defensible. It's just that because the system is big, understanding it probably involves (A) breaking it into smaller parts and understanding these in isolation, and (B) some level of empirical interaction with the system (unless we're Plato.. what a wanker..)
I think the real lesson from the Parable of the Exhaust Bracket should be that breaking the system up into parts [*conceptually*! Drop the angle grinder!], gathering information about the system, and trying to understand the system are parallel/supporting processes, not that one process ought to be favoured exclusively. So:
1. "The System" comprises not just the car (*what* car, by the way? Howww can D possibly have the sheer base indecorousness to glibly mention "car trouble" without revealing specifically what make, model, year, trim level, engine code, etc. etc. it is....) but also the online-car-help-providing community, the car repair industry, etc. In fact, given the symptom of rattling only at specific speeds (or more likely specific engine revolutions), which heavily implies constructive interference (https://en.wikipedia.org/wiki/Wave_interference), The System could be said to include stuff like the Tacoma Narrows Bridge Disaster.
1.5. Dude, I totally saw the Tacoma Narrows Bridge Disaster live at Camden Underworld in like maybe 2004. Awesome band; still-memorable night. So much so that I have to mention it here. https://tacomanarrowsbridgedisaster.bandcamp.com
2. Understanding (1), it makes sense to start by understanding the parts of The System that will help us address our specific problem best: something like understanding how fault-finding works and how internet advice works might be a fair/logical first step.
3. How online-car-advice-solicitation works is that everybody has had every possible combination of problems and those problems have presented with every possible combination of symptoms. I'm sure D discovered this pretty early-on in the process.
4. If (3), then probably we either need an AI to sort through all of that mess for us, or else we want to investigate as much as we can so that we can focus/direct/inform our online research. Thus, we might be best off applying our own non-specialist knowledge and/or the Scientific Method (á la Zen and the Art of Motorcycle Maintenance) in parallel with our online research process.
4.5 (We might also focus/direct our search by marque, model, year, etc. - is there an Owner's Club we can join and ask for help? Is our particular sort of car famous for developing some particular sort of rattle? Not likely - but pretty quick and easy to check).
5. The other part of the system it's worth understanding is the car repair industry. If good car mechanics cost like fifty pence per hour it's probably not even worth pursuing the matter ourselves (Spoiler alert: they don't - but maybe we know a friendly one who could be plied with beer and/or biscuits?)
5.5. Since the introduction of OBD2 (and further since EURO5) the car repair industry has settled-upon a universal diagnostic interface standard. Bluetooth OBD2 adapters are available for like five quid from eBay and AndrOBD is FLOSS (https://github.com/fr3ts0n/AndrOBD) - if this is a problem the car's computer already knows about, probably this is the easiest way to find out about it...
6. So, we're going to start by applying common-sense knowledge first. In particular, we're going to check things that are super quick and easy to test, regardless of how likely they are to reveal the problem: partly because "things it's easier to actually check than to Google" probably aren't going to show-up much online anyway, but partly because, well, *always* look for your dropped car keys directly under the streetlamp first, before expanding the search to the shadows. Why wouldn't you?!
7. The rattling happens at idle and at certain speeds. Great; let's start with sitting at idle in our own driveway 'cos it's so easy you don't even need a driving licence to do it. What can we do at idle? Well, we can get out of the car and walk around it, lie underneath it, open the bonnet, etc. So let's do that (ideally with a good torch - a bicycle headlamp works well, as they're bright but small enough to manipulate in cramped spaces). If the problem is somebody's spanner left in the guts of the engine last time they worked on it (I've genuinely seen this happen!) then great - we've probably solved our problem right here in step 7.
8. If we're not so lucky as to find something super-obvious, possibly we can take advantage of the fact that we know the problem goes-away at certain revolutions. Have somebody sit in the car and, in neutral, rev the engine. Is something visibly rattling at idle that stops rattling under revs?
9. Once we exhaust [tee hee] the possibilities of our idling car, let's go for a drive (ideally somewhere quiet so we can vary our speed, listen to the engine, etc. without inconveniencing other drivers).
10. We know the problem goes away at certain speeds, so let's check whether it genuinely is at certain speeds, or whether it's at certain revs. (And yes, I realise we'd ideally want to do this step before step (8) above)
11. If the problem is speed-sensitive it must either be downstream of the gearbox (because that's the stuff that varies with speed) or else not part of the drivetrain. Let's drive along at a constant speed but at different revs (eg. by depressing the clutch pedal, by changing gear, etc.) and see whether the problem persists regardless of our revs. (If we drive an automatic - we probably ought to stop trying to fix our car and go take a long hard look at ourself in the mirror)
12. If the problem does persist, let's check the main drivetrain component downstream of the gearbox: the wheels. Let's enlist some willing volunteers to stick their heads out of different windows in turn to see where the sound is loudest - that should give us a clue as to which wheel it might be. Then we might consider taking that wheel off and examining the wheel bearing, which isn't super-difficult or even, if it's cheap enough, replacing it unexamined to see whether that clears-up the problem - or at least using this knowledge to focus our online search accordingly. (Hint: it isn't the wheel bearings - they don't rattle when they fail. Just using the theory to describe the general approach, here..)
13. If the problem does indeed vary with revs, not speed, we need to think of things that vary with revs: stuff upstream of the gearbox, stuff that's oscillating with constructive interference patterns with our engine, etc.
14. Sticking with the "prioritise trying the easy/free stuff first regardless of whether it's a particularly likely culprit" let's check whether the rattling is coming from inside the car (does opening all the windows make it louder?) or under the bonnet (does opening the bonnet - protip: try this at idle.. - make the sound louder?)
15. If the sound is coming from under the bonnet - and if we're sufficiently invested in solving this ourselves - maybe we want to invest in a mechanic's stethoscope. This should at least get us close enough to the noise to help us direct our online research better.
16. If the sound doesn't seem to be upstream of the gearbox or downstream of the gearbox, let's think about the gearbox. Does the sound happen in every gear, or just some? (Specific worn cogs, maybe?) When was our gearbox oil last changed? If not these, then perhaps it's the input shaft or the diff?
17. Getting a bit tired of writing now... (and goodness only knows how *you* must be feeling...) so I shall stop now - hopefully this at least sort-of works as a sort-of defence of trying to understand complex systems (or parts thereof) without specialist knowledge...
In this framing, step 6 ("In particular, we're going to check things that are super quick and easy to test, regardless of how likely they are to reveal the problem") is where I went wrong, and I guess pretty close to the general point I was trying to make. I personally tend to systematically undervalue simple observations, and I notice many other people do the same thing. E.g. when debugging code, it seems to me that most people by nature tend to rely on "thinking" and need lots of experience to learn that this isn't as effective as "looking".
Good lord, you read all the way to Step 6! I feel -genuinely- flattered.
I don't agree that looking is better than thinking: I think they're mutually-supportive processes and each fails without the other. It's the thinking that actually solves the problem (this is generally so in my entirely-amateur experience, even if perhaps not in your case; I think this is borne out by the final paragraph of the ZATAOMM excerpt I commented) and the looking is necessary to isolate/guide what we need to think about because the system is too big and complex to just think about all of it.
P.S. We all very much still need to know what car it is, by the way. Nessun dorma....
The car is pseudonymous! (Though just like I'm sure any mildly-determined person could figure out who I am using stylometry, the lesson of the internet is that a medium-determined person could probably figure out the make and model just from that picture of the rusted clamp thing...)
P.S. The rusted clamp thing is pretty universal across different cars; one of the reasons it's rusted faster than everything else is that it's a crappy fairly-generic unspecial mild steel component. You can buy stainless steel ones, but it's probably easier to just spray the current replacement with anti-corrosion fluid (ACF) in-situ (and whilst you're down there, maybe there are other mild steel fixtures down there of a similar age to the clamp that predeceased them, who may also benefit from a restorative spray of ACF..)
Good story. Look for Dr. Atul Gawande's New Yorker piece on the history of autopsies. The tl;dr is that autopsies were "invented" to allow physicians to understand and correct misdiagnoses. But 150 years along, the rate of misdiagnosis is almost where it was in the 19th century. Why? Think back to "House, M.D.", in which the cancer doc thinks it's cancer, the neurologist thinks it's a brain disorder, the endocrinologist ... etc. They look at every issue through the lens of their own expertise ...
I'd try asking a friend who knows about cars for their thoughts
Reminds me of the story of Richard Feynman and the investigation into the Challenger disaster. During a public hearing, he put a small rubber o-ring into a glass of ice water and showed that it loses elasticity when it's cold. Earlier in the investigation, he bypassed the NASA bureaucracy and talked directly to the engineers who actually worked on the shuttle.
This is an entertaining but deceptively deep parable. Well done.
Most of the world's rationalists would do well to add some empiricism to their diet.
Contrary to your last sentence, I find this lesson more sweet than bitter -- because in many cases, including this one, an empiricist approach gets you to the answer more quickly and easily.
I'm regretting using the term "bitter"! I was trying to make an analogy to the "bitter lesson" for AI ("methods that leverage computation are ultimately the most effective") Maybe something like, "methods that leverage information gathering are ultimately the most effective". But that's a semi-obscure reference and gives a bleak tone that I didn't really intend.
I got the reference. And I think it also points at a deeper implication to consider. We're not really going to be bottlenecked on most types of thinking in the future. We already have AI that is near PhD-level in some ways, but it gets easily confused or limited by incorrect or missing data. Relevant information is about to become much more valuable, whether it's a visual inspection like in this example or open source data sets and publications.
For the time being, a human in the loop is also still relevant because AI gets easily confused in general, but it's not clear that's going to last. The data constraint seems more durable.
Yes! Maybe if you want to be AI-proof, time to get really good at "looking"? (At least, non-digital looking?)
Thanks! Needed the giggling. I am very familiar with that pipe strap and rattle. I had a [probably] similar rattle coming from somewhere under my Ranger truck for months. One morning the entire muffler/tailpipe assembly fell onto the road as I was driving. Mechanic found that 2 of these pipe straps and a log had been used to brace the tailpipe/muffler into its space. Time had killed the pipe straps, but as the log was still in fine shape and there was nothing unrusted in the undercarriage to bolt things to , the mechanic used $6 of new strapping and re-braced everything. Rattle fixed.
Yeah, after pulling that off the car I was a little disturbed that something wasn't clamped that should have been clamped. (Though it had been years, so it clearly wasn't *that* critical.) I did eventually figure it out and add a new strap.
P.S. A *log*, wow. I guess it's hard to complain, given that it outlasted the straps...
Ten years prior, I had worked on an organic farm and shared the truck with field workers. They returned the truck one afternoon, scrubbed, shining, interior spotless-unusual in working farm vehicle-and happy smiles all around. Thinking back, that must have been the day that the muffler crapped out on them 80 acres away from the shop hence the log and strap field fix. I had often marvelled [and depended] at their ability to problem solve on the spot with whatever stuff was lying around. Mechanic said the log would likely last as long as I determined to keep the truck on the road, and it did. 😔
Zen and the Art of Motorcycle Maintenance by Robert Pirsig, Part II, Ch 9:
(Disclaimer: generally not my sort of book; would advise bailing on it when it starts getting weird; your mileage may vary...)
"Two kinds of logic are used, inductive and deductive. Inductive inferences start with observations of the machine and arrive at general conclusions. For example, if the cycle goes over a bump and the engine misfires, and then goes over another bump and the engine misfires, and then goes over another bump and the engine misfires, and then goes over a long smooth stretch of road and there is no misfiring, and then goes over a fourth bump and the engine misfires again, one can logically conclude that the misfiring is caused by the bumps. That is induction: reasoning from particular experiences to general truths.
Deductive inferences do the reverse. They start with general knowledge and predict a specific observation. For example, if, from reading the hierarchy of facts about the machine, the mechanic knows the horn of the cycle is powered exclusively by electricity from the battery, then he can logically infer that if the battery is dead the horn will not work. That is deduction. Solution of problems too complicated for common sense to solve is achieved by long strings of mixed inductive and deductive inferences that weave back and forth between the observed machine and the mental hierarchy of the machine found in the manuals. The correct program for this interweaving is formalized as scientific method. Actually I’ve never seen a cycle-maintenance problem complex enough really to require full-scale formal scientific method. Repair problems are not that hard. When I think of formal scientific method an image sometimes comes to mind of an enormous juggernaut, a huge bulldozer...slow, tedious lumbering, laborious, but invincible. It takes twice as long, five times as long, maybe a dozen times as long as informal mechanic’s techniques, but you know in the end you’re going to get it. There’s no fault isolation problem in motorcycle maintenance that can stand up to it. When you’ve hit a really tough one, tried everything, racked your brain and nothing works, and you know that this time Nature has really decided to be difficult, you say, "Okay, Nature, that’s the end of the nice guy," and you crank up the formal scientific method.
For this you keep a lab notebook. Everything gets written down, formally, so that you know at all times where you are, where you’ve been, where you’re going and where you want to get. In scientific work and electronics technology this is necessary because otherwise the problems get so complex you get lost in them and confused and forget what you know and what you don’t know and have to give up. In cycle maintenance things are not that involved, but when confusion starts it’s a good idea to hold it down by making everything formal and exact. Sometimes just the act of writing down the problems straightens out your head as to what they really are. The logical statements entered into the notebook are broken down into six categories: (1) statement of the problem, (2) hypotheses as to the cause of the problem, (3) experiments designed to test each hypothesis, (4) predicted results of the experiments, (5) observed results of the experiments and (6) conclusions from the results of the experiments. This is not different from the formal arrangement of many college and high-school lab notebooks but the purpose here is no longer just busywork. The purpose now is precise guidance of thoughts that will fail if they are not accurate. The real purpose of scientific method is to make sure Nature hasn’t misled you into thinking you know something you don’t actually know. There’s not a mechanic or scientist or technician alive who hasn’t suffered from that one so much that he’s not instinctively on guard. That’s the main reason why so much scientific and mechanical information sounds so dull and so cautious. If you get careless or go romanticizing scientific information, giving it a flourish here and there, Nature will soon make a complete fool out of you. It does it often enough anyway even when you don’t give it opportunities. One must be extremely careful and rigidly logical when dealing with Nature: one logical slip and an entire scientific edifice comes tumbling down. One false deduction about the machine and you can get hung up indefinitely.
In Part One of formal scientific method, which is the statement of the problem, the main skill is in stating absolutely no more than you are positive you know. It is much better to enter a statement "Solve Problem: Why doesn’t cycle work?" which sounds dumb but is correct, than it is to enter a statement "Solve Problem: What is wrong with the electrical system?" when you don’t absolutely know the trouble is in the electrical system. What you should state is "Solve Problem: What is wrong with cycle?" and then state as the first entry of Part Two: "Hypothesis Number One: The trouble is in the electrical system." You think of as many hypotheses as you can, then you design experiments to test them to see which are true and which are false. This careful approach to the beginning questions keeps you from taking a major wrong turn which might cause you weeks of extra work or can even hang you up completely. Scientific questions often have a surface appearance of dumbness for this reason. They are asked in order to prevent dumb mistakes later on. Part Three, that part of formal scientific method called experimentation, is sometimes thought of by romantics as all of science itself because that’s the only part with much visual surface. They see lots of test tubes and bizarre equipment and people running around making discoveries. They do not see the experiment as part of a larger intellectual process and so they often confuse experiments with demonstrations, which look the same. A man conducting a gee-whiz science show with fifty thousand dollars’ worth of Frankenstein equipment is not doing anything scientific if he knows beforehand what the results of his efforts are going to be. A motorcycle mechanic, on the other hand, who honks the horn to see if the battery works is informally conducting a true scientific experiment. He is testing a hypothesis by putting the question to nature. The TV scientist who mutters sadly, "The experiment is a failure; we have failed to achieve what we had hoped for," is suffering mainly from a bad scriptwriter. An experiment is never a failure solely because it fails to achieve predicted results. An experiment is a failure only when it also fails adequately to test the hypothesis in question, when the data it produces don’t prove anything one way or another.
Skill at this point consists of using experiments that test only the hypothesis in question, nothing less, nothing more. If the horn honks, and the mechanic concludes that the whole electrical system is working, he is in deep trouble. He has reached an illogical conclusion. The honking horn only tells him that the battery and horn are working. To design an experiment properly he has to think very rigidly in terms of what directly causes what. This you know from the hierarchy. The horn doesn’t make the cycle go. Neither does the battery, except in a very indirect way. The point at which the electrical system directly causes the engine to fire is at the spark plugs, and if you don’t test here, at the output of the electrical system, you will never really know whether the failure is electrical or not. To test properly the mechanic removes the plug and lays it against the engine so that the base around the plug is electrically grounded, kicks the starter lever and watches the spark plug gap for a blue spark. If there isn’t any he can conclude one of two things: (a) there is an electrical failure or (b) his experiment is sloppy. If he is experienced he will try it a few more times, checking connections, trying every way he can think of to get that plug to fire. Then, if he can’t get it to fire, he finally concludes that a is correct, there’s an electrical failure, and the experiment is over. He has proved that his hypothesis is correct. In the final category, conclusions, skill comes in stating no more than the experiment has proved. It hasn’t proved that when he fixes the electrical system the motorcycle will start. There may be other things wrong. But he does know that the motorcycle isn’t going to run until the electrical system is working and he sets up the next formal question: "Solve problem: what is wrong with the electrical system?"
He then sets up hypotheses for these and tests them. By asking the right questions and choosing the right tests and drawing the right conclusions the mechanic works his way down the echelons of the motorcycle hierarchy until he has found the exact specific cause or causes of the engine failure, and then he changes them so that they no longer cause the failure. An untrained observer will see only physical labor and often get the idea that physical labor is mainly what the mechanic does. Actually the physical labor is the smallest and easiest part of what the mechanic does. By far the greatest part of his work is careful observation and precise thinking. That is why mechanics sometimes seem so taciturn and withdrawn when performing tests. They don’t like it when you talk to them because they are concentrating on mental images, hierarchies, and not really looking at you or the physical motorcycle at all. They are using the experiment as part of a program to expand their hierarchy of knowledge of the faulty motorcycle and compare it to the correct hierarchy in their mind. They are looking at underlying form."
Tangential, but it's incredible that when I read this, my brain was still able to conjure up and read it to me using the voice of the narrator of the audiobook
My goodness, yeah, I get this too! F'rinstance I listened to Sandi Toksvig chair the News Quiz (https://www.youtube.com/watch?v=9p8Ak6_isbA) for years - and now, any time I read anything she's written, somehow Sandi Toksvig reads it out aloud inside my head....
For what it's worth, this is one of my favorite books.
Wait, you went several years listening to this rattle? Why didn't you take it to a shop?
I suspect D̶y̶n̶o̶j̶e̶t̶ Dynomight might be trying to make a broader point here, which I'm totally missing, but I think the general approach ("complex system broken: try to understand it") is actually pretty defensible. It's just that because the system is big, understanding it probably involves (A) breaking it into smaller parts and understanding these in isolation, and (B) some level of empirical interaction with the system (unless we're Plato.. what a wanker..)
I think the real lesson from the Parable of the Exhaust Bracket should be that breaking the system up into parts [*conceptually*! Drop the angle grinder!], gathering information about the system, and trying to understand the system are parallel/supporting processes, not that one process ought to be favoured exclusively. So:
1. "The System" comprises not just the car (*what* car, by the way? Howww can D possibly have the sheer base indecorousness to glibly mention "car trouble" without revealing specifically what make, model, year, trim level, engine code, etc. etc. it is....) but also the online-car-help-providing community, the car repair industry, etc. In fact, given the symptom of rattling only at specific speeds (or more likely specific engine revolutions), which heavily implies constructive interference (https://en.wikipedia.org/wiki/Wave_interference), The System could be said to include stuff like the Tacoma Narrows Bridge Disaster.
1.5. Dude, I totally saw the Tacoma Narrows Bridge Disaster live at Camden Underworld in like maybe 2004. Awesome band; still-memorable night. So much so that I have to mention it here. https://tacomanarrowsbridgedisaster.bandcamp.com
2. Understanding (1), it makes sense to start by understanding the parts of The System that will help us address our specific problem best: something like understanding how fault-finding works and how internet advice works might be a fair/logical first step.
3. How online-car-advice-solicitation works is that everybody has had every possible combination of problems and those problems have presented with every possible combination of symptoms. I'm sure D discovered this pretty early-on in the process.
4. If (3), then probably we either need an AI to sort through all of that mess for us, or else we want to investigate as much as we can so that we can focus/direct/inform our online research. Thus, we might be best off applying our own non-specialist knowledge and/or the Scientific Method (á la Zen and the Art of Motorcycle Maintenance) in parallel with our online research process.
4.5 (We might also focus/direct our search by marque, model, year, etc. - is there an Owner's Club we can join and ask for help? Is our particular sort of car famous for developing some particular sort of rattle? Not likely - but pretty quick and easy to check).
5. The other part of the system it's worth understanding is the car repair industry. If good car mechanics cost like fifty pence per hour it's probably not even worth pursuing the matter ourselves (Spoiler alert: they don't - but maybe we know a friendly one who could be plied with beer and/or biscuits?)
5.5. Since the introduction of OBD2 (and further since EURO5) the car repair industry has settled-upon a universal diagnostic interface standard. Bluetooth OBD2 adapters are available for like five quid from eBay and AndrOBD is FLOSS (https://github.com/fr3ts0n/AndrOBD) - if this is a problem the car's computer already knows about, probably this is the easiest way to find out about it...
6. So, we're going to start by applying common-sense knowledge first. In particular, we're going to check things that are super quick and easy to test, regardless of how likely they are to reveal the problem: partly because "things it's easier to actually check than to Google" probably aren't going to show-up much online anyway, but partly because, well, *always* look for your dropped car keys directly under the streetlamp first, before expanding the search to the shadows. Why wouldn't you?!
7. The rattling happens at idle and at certain speeds. Great; let's start with sitting at idle in our own driveway 'cos it's so easy you don't even need a driving licence to do it. What can we do at idle? Well, we can get out of the car and walk around it, lie underneath it, open the bonnet, etc. So let's do that (ideally with a good torch - a bicycle headlamp works well, as they're bright but small enough to manipulate in cramped spaces). If the problem is somebody's spanner left in the guts of the engine last time they worked on it (I've genuinely seen this happen!) then great - we've probably solved our problem right here in step 7.
8. If we're not so lucky as to find something super-obvious, possibly we can take advantage of the fact that we know the problem goes-away at certain revolutions. Have somebody sit in the car and, in neutral, rev the engine. Is something visibly rattling at idle that stops rattling under revs?
9. Once we exhaust [tee hee] the possibilities of our idling car, let's go for a drive (ideally somewhere quiet so we can vary our speed, listen to the engine, etc. without inconveniencing other drivers).
10. We know the problem goes away at certain speeds, so let's check whether it genuinely is at certain speeds, or whether it's at certain revs. (And yes, I realise we'd ideally want to do this step before step (8) above)
11. If the problem is speed-sensitive it must either be downstream of the gearbox (because that's the stuff that varies with speed) or else not part of the drivetrain. Let's drive along at a constant speed but at different revs (eg. by depressing the clutch pedal, by changing gear, etc.) and see whether the problem persists regardless of our revs. (If we drive an automatic - we probably ought to stop trying to fix our car and go take a long hard look at ourself in the mirror)
12. If the problem does persist, let's check the main drivetrain component downstream of the gearbox: the wheels. Let's enlist some willing volunteers to stick their heads out of different windows in turn to see where the sound is loudest - that should give us a clue as to which wheel it might be. Then we might consider taking that wheel off and examining the wheel bearing, which isn't super-difficult or even, if it's cheap enough, replacing it unexamined to see whether that clears-up the problem - or at least using this knowledge to focus our online search accordingly. (Hint: it isn't the wheel bearings - they don't rattle when they fail. Just using the theory to describe the general approach, here..)
13. If the problem does indeed vary with revs, not speed, we need to think of things that vary with revs: stuff upstream of the gearbox, stuff that's oscillating with constructive interference patterns with our engine, etc.
14. Sticking with the "prioritise trying the easy/free stuff first regardless of whether it's a particularly likely culprit" let's check whether the rattling is coming from inside the car (does opening all the windows make it louder?) or under the bonnet (does opening the bonnet - protip: try this at idle.. - make the sound louder?)
15. If the sound is coming from under the bonnet - and if we're sufficiently invested in solving this ourselves - maybe we want to invest in a mechanic's stethoscope. This should at least get us close enough to the noise to help us direct our online research better.
16. If the sound doesn't seem to be upstream of the gearbox or downstream of the gearbox, let's think about the gearbox. Does the sound happen in every gear, or just some? (Specific worn cogs, maybe?) When was our gearbox oil last changed? If not these, then perhaps it's the input shaft or the diff?
17. Getting a bit tired of writing now... (and goodness only knows how *you* must be feeling...) so I shall stop now - hopefully this at least sort-of works as a sort-of defence of trying to understand complex systems (or parts thereof) without specialist knowledge...
In this framing, step 6 ("In particular, we're going to check things that are super quick and easy to test, regardless of how likely they are to reveal the problem") is where I went wrong, and I guess pretty close to the general point I was trying to make. I personally tend to systematically undervalue simple observations, and I notice many other people do the same thing. E.g. when debugging code, it seems to me that most people by nature tend to rely on "thinking" and need lots of experience to learn that this isn't as effective as "looking".
Good lord, you read all the way to Step 6! I feel -genuinely- flattered.
I don't agree that looking is better than thinking: I think they're mutually-supportive processes and each fails without the other. It's the thinking that actually solves the problem (this is generally so in my entirely-amateur experience, even if perhaps not in your case; I think this is borne out by the final paragraph of the ZATAOMM excerpt I commented) and the looking is necessary to isolate/guide what we need to think about because the system is too big and complex to just think about all of it.
P.S. We all very much still need to know what car it is, by the way. Nessun dorma....
The car is pseudonymous! (Though just like I'm sure any mildly-determined person could figure out who I am using stylometry, the lesson of the internet is that a medium-determined person could probably figure out the make and model just from that picture of the rusted clamp thing...)
Ah - understood. Please could you at least confirm that it definitely does have a big red "DM" roundel on the front like Danger Mouse's car?
(https://duckduckgo.com/?t=ffab&q=dangermouse%27s+car&iax=images&ia=images)
P.S. The rusted clamp thing is pretty universal across different cars; one of the reasons it's rusted faster than everything else is that it's a crappy fairly-generic unspecial mild steel component. You can buy stainless steel ones, but it's probably easier to just spray the current replacement with anti-corrosion fluid (ACF) in-situ (and whilst you're down there, maybe there are other mild steel fixtures down there of a similar age to the clamp that predeceased them, who may also benefit from a restorative spray of ACF..)
ACF is the market-leading brand name (ACF-50) but I've heard terribly good things about XCP rust blocker: https://www.bennetts.co.uk/bikesocial/reviews/products/motorcycle-maintenance-and-servicing/best-rust-corrosion-preventer-inhibitor
(And yes - I shall now shut up about your car already!)
Good story. Look for Dr. Atul Gawande's New Yorker piece on the history of autopsies. The tl;dr is that autopsies were "invented" to allow physicians to understand and correct misdiagnoses. But 150 years along, the rate of misdiagnosis is almost where it was in the 19th century. Why? Think back to "House, M.D.", in which the cancer doc thinks it's cancer, the neurologist thinks it's a brain disorder, the endocrinologist ... etc. They look at every issue through the lens of their own expertise ...