Observational studies

 One view of the value of epidemiology

One view of the value of epidemiology

A day almost never passes without someone sending a comment my way about some recent study, plucked by the media from the hundreds published that same day, showing that low-carb diets cause brain fog or decreased longevity or cancer of some type or any number of conditions any of us would rather not have.  These comments always  end with the plaintive request, is there any truth to this?

My answer follows: This data comes from an observational study, and, as such, can’t possibly indicate causality.

Since I get these comments so often and answer them the same equally often, I figured it was about time to write a post on what an observational study really is so that I can link to it when I give my standard reply.

I can then add this post to the ones on the glycemic index and relative risk, both of which serve the same purpose.  I can simply link instead of explaining what these terms mean each time I have to use them.

Observational studies – also called prospective or cohort studies and sometimes even epidemiological studies – are the kind most often reported in the media simply because there are so many of them.  These are the studies in which researchers look for disease disparities between large populations of people with different diets, lifestyles, medications, incomes, etc.  If disease disparities are found to exist between groups, then researchers try to make the case that the difference in diet, lifestyle, medication, etc. is the driving force behind the disparity.

We’ve all seen these studies by the score.  We read that a large study population of people is separated into two groups based on blood levels of vitamin C.  One group of subjects has high blood levels, the other group has lower blood levels.  And since every one seems to believe that vitamin C protects against the common cold, the researchers decide to monitor these two groups for a year and find that the group with the highest blood levels of vitamin C has the fewest colds.   These findings are rushed into publication, and soon we read everywhere that vitamin C prevents the common cold.  It all seems so reasonable and so scientific, but the truth is that these studies don’t mean squat.  And the researchers who do them know it, or at least should know it.  The fact that they do know is evident in the weasel words they use in describing their findings.  You’ll read that these data ‘suggest’ or that they ‘imply’ or that this ‘may cause’ that.  The non-technically trained public, however, read these to say that vitamin C prevents the common cold.  And usually the media helps to sway opinion by slanting the story in the same direction.

But, you may ask, why aren’t these studies sound?  If the one group with the greater blood levels of vitamin C had significantly fewer colds, why is it such a stretch to say that vitamin C prevents colds?

I can explain by way of a game I used to play with myself as a child.  I’ve never been one to sleep much even when I was a kid.  I always stayed up late and I always woke up early.  My brain never seemed to slow down.  I was always ruminating on something.  My way of trying to get to sleep was to try to think of everything that could be thought of.  My mind would race, and I would think of my brothers sleeping in the room with me, their beds, my bed, the closet, the tree outside, my dad’s car, the rug on the floor, the moon, and on and on and on.   As I thought faster and faster, continuing to compile things that could be thought of, I would finally hit a quitting point.  Then I would try to figure if there was anything I hadn’t thought of.  Of course, immediately I would think of something. I hadn’t thought of the pigs on my grandfather’s farm.  Or I hadn’t thought of the fire hydrant out front.  Or my father’s shoes.  Or whatever.  Then I would start the game again, this time, of course, starting with the pigs on my grandfather’s farm and going from there.  I would always fall asleep before I had ever thought of everything there was to think of.

Researchers doing observational studies have much the same problem.  They try to think of all the differences between two large populations of subjects so that they can statistically negate them so that only the observation in question – the vitamin C level in the example above – is different between the groups.  Problem is they can never possibly think of all the differences between the groups.  As a consequence, they never have a perfect study with exactly the same number, sex, age, lifestyle, etc. on both sides with the only difference being the study parameter. And so they don’t really ever prove anything.  In fact, we would all probably be a lot better off if all the researchers doing observational studies had followed my lead and fallen asleep mid study.

But I’m being too harsh.  These studies do have some value.  Their value is in generating hypotheses.

The observational study demonstrates a correlation.  In our example above, the correlation is that higher vitamin C levels correlate (in this particular study) with lower rates of colds.  So, from this data, we could hypothesize that vitamin C prevents the common cold.  But at this stage that would be just an hypothesis – not a fact.

Once we have the hypothesis, we can then do a randomize, placebo-controlled trial.   We can recruit subjects, randomize them into two groups that are as equal as possible, especially as vitamin C levels are concerned.  Then we give one group of subjects vitamin C and the other a placebo and watch them for a year.  At the end of the year (or whatever the study period is), we break the codes, see who is on vitamin C and who is on placebo.  We already know how many got colds, so now we compare that to vitamin C intake.  We may find that those who took the vitamin C got significantly fewer colds, so we can say that our study demonstrates that vitamin C prevents the common cold.  If this same study is repeated a number of times with the same outcome, then it can be said to be proven that vitamin C prevents colds. (This study is, of course, hypothetical.)

But these studies are randomized trials, not observational studies.  Observational studies only show correlation, not causation, a fact that everyone doing research and reading about research should have tattooed on their foreheads.

CORRELATION IS NOT CAUSATION

More often than not observational studies are chock full of all kinds of technical-looking graphs, charts and tables.  Many even have complicated equations.  And long statistical analyses of the data derived.  They are like zombies, however.  They give the appearance of scientific life, but they are really scientifically dead.  Irrespective of how many scientific baubles are strewn through them, they are nothing but observational studies, worthwhile only as generators of hypotheses.  They demonstrate only correlation, not causation.

If you want to bear with me, I’ll show you a bizarre observational study that was actually performed that demonstrates everything you need to know about observational studies.

The study was published in 2003 in the prestigious American Journal of Epidemiology.  The title of the study is Shaving, Coronary Heart Disease, and Stroke. (Click here for free full text) This study purports to show that the frequency of shaving correlates with risk for developing heart disease, with those men shaving less having a greater risk.

Here’s the finding that initiated this study.

A case-control study comparing the frequency of shaving in 21 men under 43 years of age who had suffered a myocardial infarction and 21 controls found that nine of the cases but none of the controls shaved only every 2 or 3 days.

Someone noticed that about half of the men in a small group of subjects who had a heart attack shaved once every two or three days.  Another group of men of similar age who hadn’t had a heart attack were designated as controls.  Upon questioning it was discovered that all of the men in the control group shaved every day.  Thus the first hypothesis was born:  Infrequent shaving correlates with heart attack.

The researchers had access to a large population of subjects from another ongoing study called the Caerphilly Study.  Researchers recruited 2,513 men aged 45-59 from this study and gave them comprehensive medical workups including extensive laboratory testing.

Men were asked about their frequency of shaving by a medical interviewer during phase I. Responses were classified into categories ranging from twice daily to once daily, every other day, or less frequently. The 34 men with beards were not classified. These categories were dichotomized into once or twice per day and less frequently.

The men in the study were followed for the next 20 years with follow-up exams periodically to monitor for history of chest pain, heart attack and/or stroke.

Of the 521 men who shaved less frequently than daily, 45.1 percent died during the follow-up period, as compared with 31.3 percent of men who shaved at least daily.

When the data were further refined it was determined that

The age-adjusted hazard ratios demonstrate increased risks of all-cause, cardiovascular disease, and non-cardiovascular-disease mortality and all stroke events among men who shaved less frequently.

So there you have it.  Proof that shaving daily prevents heart disease. Or is it?

The researchers doing this study aren’t so stupid that they really think that the act of shaving itself has anything to do with a man’s risk for developing heart disease.  In fact, they went to great lengths to show that shaving was merely a marker for other things going on that may well have something to do with risk for developing heart disease or increased all-cause mortality.

The one fifth (n = 521, 21.4%) of men who shaved less frequently than daily were shorter, were less likely to be married, had a lower frequency of orgasm, and were more likely to smoke, to have angina, and to work in manual occupations than other men.

And these are just the differences the researchers found.  Had they looked harder, I’m sure they would have found more, just like I did when I played my ‘think of everything that can be thought about’  game with myself as a kid.

But if these researchers had really believed that the data showed that the lack of frequent shaving itself may have been the driving force behind the development of heart disease, they may have designed a randomized clinical trial to show causality.  They could have recruited men without heart disease, randomized them into two groups, instructed the men in one group to shave daily and the men in the other to shave every third day.  Then after 20 years the researchers could tell whether or not shaving protects against heart disease.

But the idea that shaving itself has anything to do with heart disease is so ludicrous that no one would ever do such a study.  We can all see that.  It’s a ridiculous idea.  It should be obvious that the shaving or lack thereof has nothing to do with heart disease or early death; the lack of shaving is merely a marker for all the other conditions that are risk factors for heart disease, i.e., small stature, unmarried, smoking, lower socioeconomic class, etc.  It’s all so easy to see.

But let’s just suppose that we take this same study and substitute the term ‘elevated cholesterol’ for ‘infrequent shaving.’  Now what do we see?  Let’s change one of the quotes from above to reflect this change.  What then?

Of the 521 men who had elevated cholesterol, 45.1 percent died during the follow-up period, as compared with 31.3 percent of men who had low or normal cholesterol.

We nod our heads sagely.  Suddenly we have a study that seems to make sense.  But – and this is important – it doesn’t make any more sense than the shaving study.  Both are observational studies.  We are programmed to think cholesterol is bad and causes heart disease, so this second study appears reasonable to us.  It triggers our confirmation bias.  We don’t believe for a second that shaving has anything to do with heart disease, so we can easily dismiss those findings.  But we are more than ready to believe that the elevated cholesterol caused those men who had it to have heart attacks. But the reality is that both studies are exactly the same – and neither proves anything.

If you’re interested in a longer, more in-depth article on observational studies, take a look at Gary Taubes long piece in the New York Times a few years ago.  I’ve tried to take a little different slant than he did so that my post and his article would cover all the bases.

Cartoon above from: Smith, G. D. et al. Int. J. Epidemiol. 2001 30:1-11

Please note: I reserve the right to delete comments that are offensive or off-topic.

Leave a Reply

Your email address will not be published. Required fields are marked *

85 thoughts on “Observational studies

  1. Hi Dr Eades,

    I love your sense of humour and how you write. I was already beginning to chuckle out loud at the start of this blog when you wrote “To demonstrate my profound gratitude for all the blog topic selections, I’m going to put up a post that absolutely no one asked for.” ! And later I just had to start reading the blog out to my husband so he could laugh. So funny when you commented “we would all probably be a lot better off if all the researchers doing observational studies had followed my lead and fallen asleep mid study.”

    You explain things so well and make your points in these very amusing ways….and then the very serious ending that studies on cholesterol levels are just as observational as that shaving one…very sobering. You are such a good writer ! Why don’t you write some more books ? Why not publish a book of your blogs ? I would buy it right away.

    all the best,
    Anne

    Thanks for the compliment. I appreciate it. I’m glad you enjoyed the post. And I do have a book that I’m working on in fits and starts. It’s a mystery novel, of all things.

  2. Wouldn’t you say that astronomy is a science based upon observational studies?

    Although one may make hypothesis based upon the finding, I would think it quiet hard to do an experiment on the solar system. Yet we have learned a lot from the observational studies, to the degree that we were able to design a space program.

    I don’t like to throw out observation studies simply because we cannot do an experiment on the findings.

    There is a difference between what I’m calling observational studies and studies on the solar system in which scientists observe what is happening and quantify it. And, in a way, the observations that astronomers make are used to create hypotheses, which are then tested and confirmed by measurement.

  3. Dr. Mike,
    I read a very fascinating article here about low carb diets and the breathalizer. Are you familiar with the hcg diet? I have a question for you and wondered if I can contact you directly?
    Donna

    I am familiar with the HCG diet. At this point, there is no contact portal other than the info@proteinpower.com. Which right now is stacked up with questions. When we get the new design up, there will be a contact page.

  4. Once again, great post. You’re quite the wordsmith. When I grow up I want to write like you do!

    I find that counting backwards from infinity helps me go to sleep….

    Hmmm. I’ll try that tonight. Might be easier than trying to think of everything that could possibly be thought of. More orderly, too. :-)

  5. This reminds me of an old observational study, French I think, which conclusively demonstrated that an untidy socks draw was associated with a significantly better sex life (in males I think, though the improvement may have had a knock on effect for partners).

    Before you go and mess up your socks draw; no, it was never tested in an intervention study.

    Peter

    This is bad news for me since my sock drawer is probably the only orderly drawer in all the drawers I have. Thank God observational studies don’t prove causation.

  6. Your childhood game sure sounds more interesting than counting sheep!

    As far as the media jumping on these studies, am I right in assuming that the majority of the ones they choose to publish are the observational studies, that if we read that “shaving causes heart disease” in the news, that it’s most likely not going to be a legitimate randomized, placebo-controlled trial? How can we know what kind of study it was (it surely won’t be brought out with the news story), where do we go to find out? And where do we find an easily understood synopsis of the trial or study, that the average person can grasp? (Other than asking you for the answers!)

    BTW, thanks for the explanation of the differences in these studies. It really does help us to see that we need to take a lot of the media reporting with a huge grain of salt.

    Most of the studies reported in the news are observational studies if for no other reason than the vast majority of studies are observational studies. Especially studies of some aspect of diet and/or lifestyle. If you see a report of a study that you’re wondering about, just think for a bit about what it would take to do a randomized clinical trial to come up with the findings that are being reported. If it would be difficult and expensive and almost impossible to do, then it’s almost certainly an observational study.

    Here’s an example.

    Let’s say you read a story in the paper that says Dr. So and So at Institution X did a study showing that eating red meat causes colon cancer. You can be sure that this is an observational study. Why? If Dr. So and So were to do a randomized, clinical trial to show that red meat consumption causes cancer, he/she would have to recruit a large number of subjects, then randomize them into red-meat and non-red-meat groups. Then the subjects in these groups would have to stay on their respective diets for years until enough had developed cancer to indicate a difference between the two groups. Such a study could take 20 years.

    It’s much easier to take a bunch of people who have colon cancer and ask them what they ate over the past 20 years. Then ask a bunch of people who don’t have colon cancer what they eat. Then by clever question asking determine that the subjects who didn’t have colon cancer ate less red meat. Then, Voila!, you’ve got a study that ‘suggests’ that red meat causes colon cancer. It didn’t take long and it didn’t cost much. That’s why these observational studies are so much more common than randomized clinical trials.

  7. Hi Dr. Mike. Hope the holidays treated you well, and that you got to miss the Broncos getting haymowed by the Chargers. I was there, surrounded by a sea of Charger fans.

    Excellent post, and of course a great illustration of “lies, damned lies, and statistics”. Another way of looking at the issue is that the mathematics of “statistics” generally obfuscates the core issues, which are what information you have and how that information influences your belief in hypotheses. Most scientific statistical analyses make a mess of this. They tend to focus on what they know they know, as opposed to what they know they don’t know. The latter category is often large, particularly in an epidemiological study. A mathematically rigorous analysis allows you to take this into account.

    And of course there’s the usual issue of completely missing the point of scientific inference. Most standard analyses tell you something like how likely your data is *assuming* a particular hypothesis is true. So an observational study might find that their data is 4 times more likely to be true IF Vitamin C prevents colds – but says nothing else about how much we should believe that hypothesis. 4x sounds impressive, makes for a great press release; but with so much “know what you don’t know” floating around, the actual belief in that hypothesis is low, if only because there are scads of different hypotheses that explain the data nearly as well. The Vitamin C study may have increased our belief in the Vitamin C hypothesis from 1 in a million to 4 in a million, but that’s unlikely to change treatment decisions, since there’s a 99.9996% that something else caused the observed effect.

    Of course we also see the utility of this type of study, as it did increase the belief in the Vitamin C hypothesis 4x, which indicates we should squeeze on the “know you don’t know” category through a randomized clinical trial and see if our belief rises to a level which would impact treatment choices. The key is to quantify your belief, given the available information, since that’s the only number that actually affects decisions.

    The sort of narrow thinking is unfortunately not confined to science. The New York Times Magazine had a great article on how boneheaded statistics contributed to our current financial meltdown, and I wrote a bit about this on my company’s blog. Essentially, investment banks were relying too heavily on a statistical analysis called Value at Risk, or VaR. The flaw in VaR was known, but largely ignored: it did not account for the “know you don’t know”, in this case the possibility that the Market would tank. So these bozos happily proceeded in ignoring the obvious signs of trouble, confident that their VaR numbers were on track, and steered the plane right into the ground.

    Sounds just like the statinators, if you ask me.

    Hey Dave. Happy New Year! Good to hear from you. Sadly, I had a bet on the Broncos. I’m glad I hadn’t paid to go to the game. I would have been suicidal.

    I read the article in the NY Times. And you’re right, it is much the same as with the statinators.

  8. Thanks for yet another great post! (It’ll definitely have a prominent spot in my next food/health link-o-rama.) Since reading Taubes’ NY Times article, I’ve tried to be aware of whether studies are observational versus controlled in reading news reports on the medical literature. But it’s not often easy to tell — and that’s a very serious failure of objectivity in journalism.

    Then again, perhaps I can do something about that: I’m preparing to teach “Introduction to Philosophy” again this semester, and perhaps I should include a class on correlation versus causation. The Taubes would make for good reading, as would this post, in fact. (Philosophy reading is often hard going for undergraduates; they’re just not prepared to read Aristotle or even Hume, let alone Kant. So I do like to include some more easily accessible readings.)

    If you do include such a class, be sure to send me a copy of the syllabus.

  9. Am glad to hear you are writing a novel; I look forward to it. I have often thought your writing skills need to be explored or exploited in fact. Your style reminds me of Nelson DeMille and Paul Thoreaux at their best… I think it’s the wit.

    Out of the whole piece, I loved the paragraph about the sleep experience. I have the same problem!!!! Although yours is probably the result a brilliant over-active mind. Mine… probably anxiety.

    Nelson DeMille and Paul Theroux, eh? I hope I can sell as well as they. And I hope I don’t get accused of plagiarism.

  10. Dr. Mike, You have that very rare ability to explain complex matters in a very easy to understand way! Thanks for all the effort in fighting the good fight.

    I ordered some stuff from this site, and was pleasantly surprised to receive a coply of PPLP, my third one! Thanks.

    By mistake I ordered the 50000 iu D3, when I wanted the 5000 iu D3. Is it Ok then to take one pill a week or so instead?

    Yes, a lot of people take 50,000 IU once per week, especially in the winter. The blood levels aren’t as consistent as with 5,000 per day, but they stay within the range over the course of the week. I would recommend that you get your 25 OH Vit D checked along the way. Even if your taking the 5,000 per day.

  11. LOL Funny post, but true, right? Your new book sounds like fun – the mystery novel, that is.

    Happy New Year to you and yours.

    Happy New Year back It is true.

  12. A book of your blogs! I agree with Anne! In fact, I’ve thought of printing out all your blogs so as to have some decent reading material on planes, in waiting rooms, etc.

    Go for it.

  13. There are a number of issues behind observational (or any) studies that impact validity.

    An important issue is the motivation of the researcher(s) for doing a study in the first place. Self-interest is probably a main motivator. Many researchers simply want to build their academic status by publishing papers with their name on (‘publish or perish’). As such, they are not terribly concerned as to whether the subject matter or the conclusions (or speculations) have any relevance or even any practical application. The objective is simply to publish.

    Related to the above objective is the disturbingly low standards of some journals who seem willing to publish any kind of nonsense so long as the authors have a legitimate academic credentials.

    Research money is another issue. Who is funding the study either directly with outright grants or indirectly through a third party funding which although at arms length can still be tainted? Research money has to come from somewhere. And if they understand nothing else, researchers know that they need to give ‘value’ of some kind to the party that provided the funding if they want to keep the money flowing.

    Finally you suggest that the researchers doing observational studies know or should know the limitations of working with unknowns. From my experience as a former technician/troubleshooter in telecommunications I am not so sure this is the case.

    In order to get hired for my job I had to take an extensive series of tests created by my potential employer designed to determine my analytical/problem solving skills. After I got past the tests I had to through 4 interviews. I was told by one interviewer that 1 in 50 who applied for my position got hired. In other words, those who actually got hired had demonstrated exceptional analytical/problem solving skills.

    In the workplace environment every work order for a ‘trouble’ (problem) was tracked by computer until another tech touched the same piece of equipment. So you owned a problem you worked on until it got passed to another technician. Techs in my division would often comment on the amount of time I wasted running analytical checks and procedures on equipment after I had found ‘the problem’ which was usually something obvious like a broken part. My response was that before I could consider a piece of equipment fixed I had to make sure everything that could be evaluated was evaluated and the equipment was up to specification.

    When I left the division of over 100 technicians computer records showed that, with few exceptions, when I fixed a problem it never came back. The same records showed that the efforts of the majority of technicians more often than not actually caused new and different problems. I was rated in the top 2% of the best technicians in the company of several 20 year journeymen.

    If the above results sound a lot like the field of medicine I think is because in any field, including research, there are probably a small number of researchers who are not only outstanding but dedicated to furthering their field and bringing meaningful advances to society. Below them there are probably 20-25% in the field who are very good. After that things go down hill rapidly with the mediocre line at around 50%.

    I agree across the board.

  14. Thanks for making us all smile. The cartoon is priceless.

    I believe low carb is gaining ground, Dr. Eades. About 5 years ago when I was on Weight Watchers, my group leader railed about how Atkins died of a heart attack from his diet. I later learned that this was false and it completely discredited WW in my mind. Just yesterday, I saw a WW commercial on TV talking about “low carb nightmares.” Very interesting. Frankly, I think the more one hears about the “dangers” of low carb in the media, the better off objective diet information such as yours really is. They would not mention it if they did not feel the competition. People are putting two and two together and realizing that they are being fed a bill of goods. Please keep up the good work.

    Just FYI, I saw several comments in your previous post on the environmental dilemma of eating meat. This is so disturbing to me, particularly because there are strong political motivations behind these ideas rather than good science. I’ll be posting more on that topic on my blog shortly, for anyone who is interested. I’ve also written a bit on this topic already. (I have a PhD in environmental biology.) Suffice it to say that I do not believe eating meat is a concern. There are problems with the way meat is produced in the United States but this is not an indictment against meat *eating*. Also, for those worried about global warming (I personally am not, after being solidly in the global warming camp for years) one should consider that there is probably not much more methane being released from cows today than there was from bison 300 years ago. I’m not certain and I have not done the calculations, but consider that there are around 100 million head of cattle in the US today, weighing approximately half as much as the 100 million head of bison that used to roam from the plains to the east coast. Grain fed cattle probably produce roughly twice as much methane than grass fed ones but the biomass of cattle is probably twice as small if not even smaller than the original national bison herd. It is curious that one never hears about this in environmentalist circles, nor the hypoxic dead zone in the Gulf of Mexico caused by erosion from corn and soy production. :) A return to grass fed meat in the United States would have enormous environmental benefits. Here are a few posts I’ve written on the matter:

    http://www.fa-rm.org/blog/2008/12/diet-for-unhealthy-planet.html

    http://www.fa-rm.org/blog/2008/12/epa-tyranny-cow-fart-tax-coming-your.html

    http://www.fa-rm.org/blog/2008/12/pondering-return-of-buffalo.html

    Shameless self promotion, I know. However, I think this type of information could really use a wider distribution for the sake of everyone who wants to eat a healthy diet and not feel guilt about it. If we don’t start working to combat meat myths and pseudoscientific greenie crapola, we really could find ourselves in some pretty dire straits in a decade or two.

    Hey, I don’t mind a little shameless self promotion when it’s for a good cause. I’m happy to help. I love your explanation about the bison versus the cattle. Makes perfect sense. I enjoy your site. Keep up the good work.

    • “Grain fed cattle probably produce roughly twice as much methane than grass fed ones but the biomass of cattle is probably twice as small if not even smaller than the original national bison herd. It is curious that one never hears about this in environmentalist circles, nor the hypoxic dead zone in the Gulf of Mexico caused by erosion from corn and soy production.”

      What did you think the soy is grown for? Tofu? No, it’s for cattle feed. Are you sure you have a PhD in environmental biology?…

      “A return to grass fed meat in the United States would have enormous environmental benefits.”

      Well, it might have for the rain forests of Brazil, but there’s not nearly enough grassland available in the USA to feed 100 million heads of cattle. Remember, this is not 1700’s – at the age of bison there were very few humans using the land.

      “If we don’t start working to combat meat myths and pseudoscientific greenie crapola, we really could find ourselves in some pretty dire straits in a decade or two.”

      You might start by acquiring some basic knowledge of the effects of meat production yourself. One thing you seem to forget is that meat production is a global problem. Erosion of grassland is affecting all countries where cattle grazing is common. Amazon rain forest is destroyed to grow soy for cattle feed. Meat consumption is on the rise in many parts of the world, although already the current cattle production is unsustainable in many places. For example, per capita consumption of meat in China is currently less than 100 pounds, while in the USA it’s 276 pounds (144 pounds in 1950). Despite the relatively low consumption, Chinese producers aren’t anymore able to meet the current consumption, and meat import is growing. You may try to imagine what will happen in the future, when consumption there grows (if the trend can’t be turned) by an amount that’s two times the total current consumption of the USA.

  15. Hmm… Speaking of observational correlation:

    >>>*** “…I used to play with myself as a child. I’ve never been one to sleep much…” ***

    That thought gives me chest pains. I think I’ll go shave now.

    Hmmm. Maybe there really is a correlation. :-)

  16. Good column but sometimes there is an real correlation that is missed because it is in the wrong direction. Recall the reports that drinking soda correlated with overweight, even if the soda was diet soda — which according to the press, the researchers couldn’t understand. But, just ask yourself: who drinks diet soda? People who are overweight.

    Also, whereas correlation does not imply causation, a lack of correlation is good evidence for lack of causation. The lack of correlation of low-fat program with just about anything in the Women’s Health Initiative tells you that low-fat is probably not good at causing the desired outcomes.

    Indeed. Thanks for bringing this up. I should have done it myself.

  17. I’m sorry to be the one to break the news to you Dr. Eades but CNN Health recently commissioned a panel of experts to put more than 60 well-known diets to the test to determine which ones work and work safely for weight lose and Protein Power didn’t make the cut.

    The panel of experts was comprised of: Maureen Callahan, MS, RD, Roshini Rajapaksa, MD, Tim Church, MD, MPH, PhD, Samantha Heller, MS, RD, Frances Largeman-Roth, RD, Christine M. Palumbo, MBA, RD member of the American Dietetic Association’s (ADA) board of directors, Chicago-based Palumbo was awarded the 2007 Nutrition Entrepreneur of the Year by the ADA.

    And the winners are:

    • The Structure House Weight Loss Plan – Gerard J. Musante, PhD
    • The Step Diet – Christine Palumbo
    • Weight Watchers
    • The EatingWell Diet – Jean Harvey-Berino, PhD, RD
    • The Volumetrics Eating Plan – Barbara Rolls, PhD
    • The Best Life Diet – Bob Greene
    • The Solution – Laurel Mellin
    • You: On a Diet – Mehmet Oz and Michael F. Roizen
    • The Spectrum – Dr. Dean Ornish

    Hmmmm. Mainly RDs on the panel of experts. On second thought maybe it’s a good thing Protein Power wasn’t mentioned.

    All things considered, I would just as soon not be mixed in with this band of losers.

  18. I think the same could be said of statin studies. And the Seven Countries Study for that matter.

    So, what do you think of Sanjay Gupta as Surgeon General? All I know is that on a CNN health special, he said he ate some kind of frozen egg thing for breakfast because it didn’t have many carbs. Maybe the future isn’t so grim.

    I don’t know much about Sanjay Gupta, but somehow I don’t think he’s on the short list for Surgeon General.

  19. Sorry to go off topic,

    My mom is experiencing dry and cracking nails and also some dandruff (she’s never really had that). The guy at the Health Food store recommended Silicea Balsam. What are your thoughts Dr. Eades and is there anything else that can help?

    Thank You,

    Conrad

    I really don’t have a clue having never seen or examined your mother and having absolutely no medical history on her. I would at least make sure she’s getting plenty of fat and protein in her diet (hair and nails are protein dependent structures) and consuming adequate amounts of omega-3 fats.

  20. Forgot to mention, my mom is 50.

    Monica,

    Everytime I read your posts I’m very impressed at the detail, clarity, and integration that you display. Right on!

  21. Hi Dr Eades,
    I’ve done my own observational studies and only regret that I didn’t get millions in funding from the government.
    Study 1- What is the major cause of injuries in auto accidents?
    Conclusion: Driving or riding in an automobile
    Study 2- What is the major cause of injuries in a fall?
    Conclusion: Falling down
    Study 3- What is the major cause of sunburn?
    Conclusion: Sunshine
    These studies fall right in line with the results from many studies hyped by the media.
    Hope you have a great 2009.
    Jay

  22. Just read this to my 94 yr. old friend. He earned a degree in Sociology and Anthropology at the University of Chicago back in the late ’40s and conducted a number of population studies with other students. Often he’d challenge their findings by asking, “So, then, what’s the difference between a sink and an elephant?” Answer: “None. Neither one can ride a bicycle. 100% correlation.”

    Very nice. I may have to steal it.

  23. I think that reports of such studies in the popular media add fuel to the sloppy reasoning rampant throughout our culture. They play on our fears about our health, and thus, if we read a report of an observational study indicating that people who eat broccoli have a 30% reduced risk of cancer, we tend toward the overly simplistic conclusion that if we eat broccoli, we won’t get cancer. They also bolster the popular image of scientists as being objective searchers for the whole truth and nothing but the truth, an image which for me has been quite thoroughly discredited, thanks to blogs such as yours, as well as a few others I check on regularly. And the damage these reports do really does add up, as they all contribute to a set of assimilated commonplaces that people, including researchers, absorb, which increases the likelihood that researchers will arrive at the wrong conclusions by starting from wrong premises, which they almost always do. Add to it all that a study can be found to support just about any position one believes to be true, and what we have is a big mess that is anything but objective or “scientific.”

    Love the cartoon.

  24. Dr. Eades,
    Having formerly been in the market research field, this article resonates with me! Clients were always looking for more meaning than the data could provide. But that didn’t stop them from making various assertions that couldn’t be backed up by the data! Same in the medical field, obviously. Your posts that delve deeper into the various research studies, and explain them in plain English, are extremely valuable. Keep up the great work!

    I just came across this article in MedScape Today: Tasteful Alternatives to the Heart-Healthy Diet

    I was fascinated to read this: “For patients now seeking primary preventive strategies to address cardiovascular risk, a recent surge in the medical and nutritional literature supports alternative dietary approaches that include high-carbohydrate, low-fat (popularly referred to as the Atkins diet); high-fat, low-protein diets; and the Mediterranean diet.”

    Since when did Atkins become a high carbohydrate, low-fat diet?!!! Wow. If they’re going to perpetrate bad nutritional advice, they should at least get some of their facts straight.

    Later in the article about one of the case studies they say: “This Italian-American woman is already enamored of the key elements of the Mediterranean diet. This diet has been comparable to high-carbohydrate and low-fat diets in weight reduction and improvement of cardiovascular risk factors. She may have or may be at risk for the metabolic syndrome, which would justify a more aggressive approach to dietary management… ”

    I can only assume this would mean a recommended increase in her carbohydrates?

    A pretty incredible screw up. Really makes one wonder how well the MedScape stuff is proofread. And makes me wonder who wrote it. A writer for hire? Or a medical professional?

  25. Hi Dr. Mike–re: the surgeon general, I’ve got a better idea–just do away with the position entirely. We do not need an official nag. Let people decide for themselves how to eat and how to live their lives!

    Fine by me.

  26. “I don’t know much about Sanjay Gupta, but somehow I don’t think he’s on the short list for Surgeon General”

    Nevertheless, medical reporter and Grady hospital neurosurgeon Dr. Sanjay Gupta has been approached by the Obama administration to fill the post of U.S. Surgeon General, CNN officials confirmed Tuesday.

    What does the surgeon general actually do?

    Unbelievable! The Surgeon General is the head of the U.S. Public Health Service and the spokesman for issues of health to the public. As a previous commenter put it, he/she is the official nag.

  27. ME: All things considered, I would just as soon not be mixed in with this band of losers.

    Losers? With diplomacy like this you should run for office. Speaking of losers, in Ophra’s situation it seems clear that the advice of losers doesn’t necessarily result in others becoming ‘losers’ if you get my drift.

    I do get your drift. Oprah’s sense of loyalty must exceed her sense of observation.

  28. “CNN Health recently commissioned a panel of experts to put more than 60 well-known diets to the test to determine which ones work and work safely for weight lose and Protein Power didn’t make the cut.”

    Just had to note that three of these losers (Bob Greene, Mehmet Oz, Michael Roizen) are Oprah’s advisors (I wish someone would ask her the well-known Dr. Phil question, “So how’s that workin’ for ya?”) and once again her “new” plan for the year is high in whole grains and very low in fat. Her “treats” are going to be sweet potatoes and those whole grain blue corn chips. She plans to eat oatmeal and egg whites for breakfast (not mixed together, but still…). And of course, the requisite 30 minutes of cardio that one “has” to have to lose weight, along with resistance training.

    Nothing like dooming oneself to failure. Of course, if she is able to stick to the reduced calories, she’ll lose some, but not without a lot of misery. And I doubt that it will last.

  29. Hi Dr E,
    Sanjay Gupta is now your Surgeon General Designate.
    (Linguistic note: Sanjaya = Most victorious; Gupta = a lineage of Hindu dynastic rulers)
    At present I know nothing of him, but Mr Freddy has perked my interest. Could he lead his flock (i.e. you lot) out of the low fat wilderness??
    Interesting times.

    And thank you for this great post. I’ll have to start shaving twice per diem.

    P.S.: Had a fascinating talk with a lady over dinner a few weeks ago in Chinatown. The circumstances were a bit weird as her daughter had just been widowed when her husband’s parachute had failed to open. At the time it was thought he’d had a heart attack and failed to open is chute. Her daughter was enjoying herself at the next table and that was good to see. Anyway the mother was telling me that the very best thing she had ever tried to lose weight was PP, and I told her about this blog and also Regina Wilshire’s and to follow your blog roll. And then, after dinner it was off over the road to the way-too-big Entertainment Centre for the School Spectacular to watch our various progeny, nieces and nephews dance in the show. Choir of about 2000 and 3000 kids dancing to the Triumphal March and Ballet from Aida. Strange night!

    Michael Richards

    Sounds like a very strange night, indeed.

    And somehow I doubt that Sanjay Gupta will lead us out of the low-fat wilderness.

  30. Dr Eades I am amazed at educated people! Analysis by paralysis! I guarantee you that no matter how many times you will rrefute this or that study by giving in depth scientific explanation, people will still question low carb diets. Why would one follow a low carb diet long enouph to see the results for themselves? And even if we do see the results both in overall health and blood work, is it enouph to believe it? Is seing a believing? Not for educated people apparently! No, they would rather question it, because some study tells him or her otherwise! Low carb causes cancer? So is living! I can prove it without much invested! Ok, on a lighter note here is something my ex-girlfriend sent me, I hope you enjoy it Dr Eades!

    Jack wakes up with a huge hangover after attending his company
    Christmas Party. Jack is not normally a drinker, but the drinks went down easy. He
    didn’t even remember how he got home from the party.

    Jack had to force himself to open his eyes,… the first thing he sees is a
    couple of aspirins next to a glass of water on the side table with a single rose.

    Jack sits up and sees his clothing in front of him, all clean and
    pressed. He looks around the room and sees that it is in perfect order. So is the rest of the house.

    He takes the aspirins, cringes when he sees a huge black eye staring
    back at him in the bathroom mirror. Then he notices a note hanging on the corner
    of the mirror written in red with little hearts on it and a kiss mark from his wife in lipstick:

    ‘Honey, breakfast is on the stove, I left early to get groceries to make
    you your favorite dinner tonight. I love you, darling! Love, Jillian’

    He stumbles to the kitchen and sure enough, there is a hot breakfast,
    hot coffee and the morning newspaper. His son is also at the table, eating.

    Jack asks, ‘What happened last night?’

    ‘Well, you came home after 3 A.M., drunk and out t of your mind. You fell over
    the coffee table and broke it, and then you puked in the hallway, and
    got that black eye when you ran into the door.

    Confused, he asked his son, ‘So, why is everything in such perfect order
    and so clean? I have a rose, and breakfast is on the table?’

    His son replies, ‘Oh THAT!.. Mom dragged you to the bedroom, And when
    she tried to take your pants off, you screamed, ‘Leave me alone, I’m married!!’

    Broken Coffee Table $400.00
    Hot Breakfast $8.00
    Two Aspirins $. 40
    Saying the right thing, at the right time. PRICELESS

    MD enjoyed it!

  31. Though I loved Michael Pollan’s “The Omnivore’s Dilemma”, I avoided reading his latest book “In Defense of Food”, mainly because I was turned off by the mantra “Eat food. Not too much. Mostly plants.” Sounded like the usual dogmatic nonsense to me.

    Anyway, our local library is holding a community reading program with group discussions, so I figured it would be a good chance to cause trouble and started reading “In Defense of Food”. Here’s a mind-numbing quote: “The gold standard in nutrition research is the large-scale intervention study”. One presumes he got this impression from nutrition scientists, and if so, it’s no wonder the field is such a disaster.

    Just confirms what Gary Taubes says about nutritional ‘scientists.’

  32. Re: Sanjay Gupta. Big Pharma, baby. Also, it’s part of CNN’s payback for slobbering all over Obama.

    I hadn’t thought of that.

  33. Lol, man you have a great sense of humor indeed! Are you sure you are not partially Jewish at least? I said ” I hope you enjoy it Dr Eades” and you replied ” MD enjoyed it”‘ That put a smile on my face. And lately it has been a problem, too many frowns and not enouph smiles. I remember my female friend once asked me if there was one trait in a woman I could never acceot in the relationship what would it be and right away I said ” lack of humor” My grandmother always used to say that it is better to lose something with a smart person than to find it with an idiot! I translated it from Russian, dont know if it makes sense. I think same goes for sense of humor. And I know that people who eat meat are much happier then vegetarians! I hate to generalize, so I am saying from my observational studies ” People that enjoy meat, cheese and wine are happier and funnier!

    Here is something for you guys to enjoy if you havent seen thi yet!\
    http://www.youtube.com/watch?v=cc2hnupC_6E

    If I could only teach my ex-girlfriend of 4 years that trick!!! Lol, she was a cat, I guess!

    Dr Eades, BTW have you ever owned a pet? Are you a dog pr cat lover or neither? It would be interesting to venture a guess. I would love to know how many people think you are a dog or cat person? For some reasons I am ver unsure. I would say a cat lover!

    At this point all my pets are dead pets. We used to have a mutt and a wolf hybrid. We loved them both, but the wolf hybrid was incredibly destructive. Cost us zillions of dollars in repairs and in trying to fix our property so that he couldn’t get out and terrorize the neighborhood. He was gentle as a lamb but weighed about 90 pounds and scared the bejesus out of all our neighbors. When both he and the mutt went on to their canine rewards, we elected not to have any more pets for a while. MD loves cats; I loathe them. Mainly because I’m allergic to them. If we get another pet – and that’s a big ‘if’ – it will be a dog.

  34. I have owned dogs all my life! I curently have a big German Shepard and swear he is Human! I am considering to get a pug, love them. Here is a video clip of a vegetarian girlfriend of mine!

    http://www.youtube.com/watch?v=MF88IYF2MHY&feature=related You guys would be surprised what a piece of raw tuna fish does to this cat! Long live the low carb and a good piece of meat! Dr Eades sorry for an off topic links.

  35. Dr. Mike!

    Here’s another one about those wonderful statin drugs!
    http://news.yahoo.com/s/nm/20090108/hl_nm/us_statins_1

    I love it. A cohort study is an observational study, the results of which don’t mean diddly in terms of causation, yet there they go again trying to use the data to prove it. Read what this nimrod who wrote a commentary on the study had to say:

    All in all, it is clear that somewhere between normal cognitive performance and profound dementia of Alzheimer’s disease, statin therapy exerts a beneficial effect.

    The study says no such thing. The data from this study in no way make the case that “statin therapy exerts a beneficial effect.” In order to say that, you’ve got to do a randomized, controlled trial. And then you need to repeat it a couple of times to make sure. One certainly can’t make the case that this cohort study proves anything of the sort. Yet this is precisely what these people try to do, and they are aided and abetted by the media. It is shameful.

    Thanks for providing a perfect example of what I was discussing in the post.

  36. Dear Dr. Mike,

    Thought you might find this interesting. Apparently, a new product is being released called “Meatwater.” Apparently it’s a high-protein drink that has some unusual flavors such as Italian Meatball, Beef Stroganof, and Weiner Schnitzel. The UK’s Telegraph can’t decide whether or not it’s a joke, but the website is pretty cool:

    http://dinnerinabottle.com/
    http://www.telegraph.co.uk/news/newstopics/howaboutthat/1954915/Meatwater-Is-cheese-burger-juice-a-big-hoax.html

    I am continually impressed by your kindness and essential decency in operating this blog free of charge to help those of us who want to live a better life. You and your lovely wife are truly “Renaissance People.” Thank you for all you do.

    Meatwater sounds kind of good. I’ll bet it tastes sort of like bouillon.

    I’m glad you’re enjoying the blog.

  37. Dave said:

    Though I loved Michael Pollan’s “The Omnivore’s Dilemma”, I avoided reading his latest book “In Defense of Food”, mainly because I was turned off by the mantra “Eat food. Not too much. Mostly plants.” Sounded like the usual dogmatic nonsense to me.

    I modified it a little:
    “Meat food, Not too lean, Most leaf plants”

    Your way makes much more sense healthwise.

  38. Dear Dr. Mike,

    Vegetarian brain shrinkage alert! PETA now demands that fish be called “Sea Kittens.” In doing this, they hope to make fish less appealing by encouraging people to identify them with cuter pet-type animals:

    http://www.peta.org/sea_kittens/

    Really, I’m not making this up!

    What total drivel. Problem is that it’s aimed at children.

  39. Dr. E., you mention that you are allergic to cats. There is one breed, the Siberian cat, which apparently naturally lacks the protein in its saliva that irritates many people. Supposedly about 70% of allergic people can tolerate them. This cat was imported from–where else–Siberia. It has a triple coat to keep warm in the harsh winters.

    This is a relatively new breed of cat for the U.S. But if you are ever willing to consider having a cat again (since MD loves cats), this might be a good compromise.

    I’ll pass the info along.

  40. “Eat food. Not too much. Mostly plants.” Sounds like a living hell to me. If you don’t live forever it will certainly feel like it. Someone–I think it was MSNBC–ran a contest for viewers to come up with their own 2-3-2 guide to eating. Winner: “Ate plants. Lots of them. Still hungry.”

    Brilliant!

  41. I have seen Michael Pollan’s fatuous statement, “Eat food. Not too much. Mostly plants.” all over the internet. What I have not seen is any discussion about the first of the three parts. What the heck are we supposed to eat if not food? Grass? Wood? Sand?

    I lost respect for the man after I read “The Omnivore’s Dilemma”. At the end of this work he moralizes extensively after shooting a feral pig. Absent was any recognition that killing is sometimes a necessity. He would have felt differently if he had to feed a hungry family.

  42. Pollan’s definition of “food” is basically non-industrial food as opposed to engineered foodlike substances, e.g. an apple vs. an apple energy bar. Given that, it’s the only part of that statement that holds any water.

  43. Hey, Dr. Mike. I promised my sister I would ask you a question.

    She has tried low carb several times, and liked it — until she got sick. Every single time, her immune system would crash. Bronchitis, sinusitis, tonsilitis, strep throat, etc.: she’d always get some kind of infection. She doesn’t remember what plan she was doing, sadly, and I don’t that she was taking more than a multivitamin. Do you have any ideas of what she could do to try low carb again without making herself sick? I’d really appreciate some suggestions, though I know you can’t diagnose, etc.; it’s hard to find a doctor who really understands low carb. I’d like to tell her something more specific than “try all the supplements listed in PP.”

    Any ideas?

    I have no ideas. I’ve treated at least 10,000 patients with low-carb diets, and I’ve never had one report this kind of history. Since the immune system is primarily a system that relies on protein and good quality fats for optimum function, it is difficult to fathom how one’s immune system “would crash” when given plenty of these. Perhaps your sister really doesn’t want to go on low-carb and is using this as an excuse. Or perhaps these problems were simply coincidental, i.e., she was going to get strep throat, sinusitis, etc. anyway, and it just happened to coincide with her starting the diet.

    I once had a patient who was 17 years old who was about 280 pounds. Her parents brought her to my office insisting that she go on ‘my’ diet. I laid it all out for her and she seemed enthusiastic. But later her mother called and told me that her daughter got ill whenever she tried to do the diet so she (the daughter) was going to try something else. Two years later, the daughter came back weighting well over 300 pounds. She had decided (instead of her mother deciding for her) that she wanted to lose the weight. I asked her why she wanted to do low-carb again if it made her sick the first time around. She told me that she told her mother that to get her mother off her back. She (my patient) then proceeded to lose – on a low-carb diet – to 135 pounds without a hitch. I used her as one of the poster children for how effective a low-carb diet is for weight loss in lectures I gave.

  44. My sister’s a reasonably sane adult; if she didn’t want to do it, she just wouldn’t, rather than make excuses — much less make excuses after the fact. Anyway, it’s hard to make up a tonsillectomy. I’ll go with the coincidence hypothesis. Perhaps too much change at once (even if it was technically good change) contributed.

  45. I get it now. An observational study is sort of a triple-blind study. The researchers do not know what experiment was performed, on whom it was performed, or even if it was performed. They grab a bunch of people and dream up an experiment which could plausibly have produced the correlations they see.

    I think you’ve got it.

  46. Dear Mike,
    I have copies of Protein Power and Life Plan and also read the experiences of the redoubtable explorer and dietary “guinea pig” Viljalmur Steffansson.

    Would you be willing to agree that his experiences while wintering on the Hudson Bay amounted to an “observational study” which generated a number of Null Hypotheses, including long term “lo carb” is injurious, and scurvy will result from lack of vegetables or fruit, while the period at Bellevue and after, was a “crossover design” study, albeit on a sample size of two, (Steffansson and Anderson switched from “ordinary diet” to the Inuit Diet) which provided no supporting evidence for the Hypotheses? I was interested in the observation of “pure protein” poisoning resulting from a few days of eating little or no fat with animal protein.

    I’m glad thatyou keep flying the flag of proper metabolic science, so that the health of diabetics, the overweight, those with the prospect of longevity reducing condition may be promoted using the results of tested measurement rather than the “observations” of Congressional Sub-Commitees and the Surgeon-General.

    I also imagine that Viljalmur Steffansson would be delighted that “fish oils” are now promoted as a healthy food supplement. It would in keeping with his experiences on the Hudson Bay.

    My own foods are beef, fish, eggs, chicken, cheese, olive oil, nuts, tomato, onions, yoghurt milk and coffee. My supplement is “cod liver oil. When asked, I call my way of eating: home cooked good food, otherwise “lo carb” or ketogenic.
    Thank you for your good work.

    Desmond
    Msc Stats (40 years ago)

    Re: Stefansson. Observing is different than an observational study. But both are useful for generating hypotheses that can be tested with randomized, controlled trials.

  47. Dear Mike,
    I have copies of Protein Power and Life Plan and also read the experiences of the redoubtable explorer and dietary “guinea pig” Viljalmur Steffansson.

    Would you be willing to agree that his experiences while wintering on the Hudson Bay amounted to an “observational study” which generated a number of Null Hypotheses, including long term “lo carb” is injurious, and scurvy will result from lack of vegetables or fruit, while the period at Bellevue and after, was a “crossover design” study, albeit on a sample size of two, (Steffansson and Anderson switched from “ordinary diet” to the Inuit Diet) which provided no supporting evidence for the Hypotheses? I was interested in the observation of “pure protein” poisoning resulting from a few days of eating little or no fat with animal protein.

    I’m glad thatyou keep flying the flag of proper metabolic science, so that the health of diabetics, the overweight, those with the prospect of longevity reducing condition may be promoted using the results of tested measurement rather than the “observations” of Congressional Sub-Commitees and the Surgeon-General.

    I also imagine that Viljalmur Steffansson would be delighted that “fish oils” are now promoted as a healthy food supplement. It would in keeping with his experiences on the Hudson Bay.

    My own foods are beef, fish, eggs, chicken, cheese, olive oil, nuts, tomato, onions, yoghurt milk and coffee. My supplement is “cod liver oil. When asked, I call my way of eating: home cooked good food, otherwise “lo carb” or ketogenic.
    Thank you for your good work.

    Desmond
    Msc Stats (40 years ago)

    Re: Stefansson. Observing is different than an observational study. But both are useful for generating hypotheses that can be tested with randomized, controlled trials.

  48. Dr Eades –

    Excellent article. I was just discussing this with my wife last night… how irresponsible these researchers are in their constant release of the “study du jour”, knowing full well that the results are, for all intents and purposes, meaningless. Then the media comes along and acts equally irresponsibly and publishes the info out to the masses… all presented as fact.

  49. Hmm. I wonder if they thought to check testosterone levels (or whatever actually controls rate of beard growth) vs heart disease?