« Eat Almonds For Vitamin E | Main | Stegosaurus Cauliflower »
November 10, 2006
A Caution About How To Interpret Evidence
Hi Cori... welcome, glad you're enjoying the blog, and you're invited to post links to articles you find helpful or interesting anytime! Thanks for chiming in. I hope to soon figure out a way to set up reader forums so readers can "talk" with each other more easily and more often.
I wouldn't describe myself as anti-soy: I eat it at least once a month, and often more. But I do have it on my list of caution foods, foods I eat sometimes but not daily, due to concerns about long term brain effects from consuming large amounts. We all make decisions based on our understanding of the evidence and how much risk we're willing to take to consume a certain food. I don't love tofu enough to take the risk for more than once or twice a month. If you don't think there's a risk, or you do, but it's a risk you're willing to take, then by all means, eat up! Just be sure to try my wonderful tomato tofu recipe on at least one occasion! Even non-tofu lovers love it.
Speaking of the Okinawans, it occurred to me that I should post a post that MR made to the CR Society list a few months ago. Not being a scientist, I have often found myself confused by the seemingly contradictory messages we get about what is a healthy diet and what is not. I found this post helpful in considering various kinds of "evidence," especially in getting to the facts behind a big media story.
Here 'tis -- hope the embedded dialogue (it's from a mailing list) isn't too confusing. He was responding to a question about the China Study, which is oft-quoted, and gets into detail about various kinds of evidence further down. Don't take the groucy tone of the message personally... he was responding on-list to someone who had raised this question many times and gotten the same answer many times over the years... if one of my readers asked him such a question, he would answer in a much warmer tone, I promise! So ignore the grouchiness but follow the reasoning because at least I find it very important to how I evaluate things I hear or read.
All typos are MR's!
To reiterate: the China Study was crummy, population-comparison
ecological crap, not proper prospective epidemiology. In a real
prospective epidemiological study, you take a group of individuals and
ask each of them INDIVIDUALLY about hir dietary and other exposures at
time X, and then follow them all up for several years and see their
health outcomes; then, you correlate specific exposures to specific
outcomes. If you see such correelations (eg, people who consumed more
cooked tomato products were less likely to develop aggressive prostate
cancer), then because you have a range of other information about EACH
of these people as INDIVIDUALS, you can double-check for false positives
on an individual-by-individual basis: eg, you can say, "were cooked
tomato product users mostly of Italian descent (possible genetic
influence)? Were they less likely to smoke, or eat a lot of saturated
fat? Did they tend to have their cooked tomato products with salads, or
eat more vegetables generally?" Etc. The combination of a prospective
design and the existence of a range of info about EACH PERSON'S
lifestyle gives such studies great power to test for real causal
connections -- not definitive proof, but strong evidence.
The China study was not prospective epidemiology, but an "ecological"
study, which aggregates entire populations. That is, they looked at how
much meat was consumed in an ENTIRE PROVINCE, and the rates of heart
disease in the province AS A WHOLE, and then compared the two variables
in another province; if provinces with more meat consumption also have
more heart disease, they inferred that meat causes heart disease. As
I've often harped in the past, these kinds of studies are MEANINGLESS.
The Japanese smoke more than the Americans; they also die less of lung
cancer. That doesn't mean that smoking is protective against lung cancer
-- and you can prove this with proper, prospective epidemiology, because
WITHIN *either one* of those populations, INDIVIDUALS who smoke at time
X are more likely to die of lung cancer several years down the road.
I hate to be a grouch, but I've said this many, many times before, and
I''m very tired of repeating myself; it's very frustrating go through
endless iterations of this on the List, as not only newcomers but folks
who have been on the List for YEARS continue to take these kinds of
bullshit studies seriously. One such exchange, slightly edited:
-----------
http://lists.milepost1.com/cgi-bin/wa?A2=ind0107&L=crsociety&P=R6850
"Name edited out" wrote:
> > > How come [the Okinawans] (who eat lots of
> > > soy) have lower rates of dementia, ...???
and
> I still fail to understand why more Okinawans don't have dementia
> (CR or no CR) if there's a link between it and soy. You say that
> such statistics are meaningless,
MR responded:
Right. Again: your line of reasoning is:
"They" say that soy causes dementia. Yet the Okinawans eat more soy than
the Japanese or North Americans, and have LOWER rates of dementia than
people from those groups. Therefore, clearly, soy does not cause dementia.
But the same logic applies all over the place, in places wehre it
clearly ain't so, as eg:
"They" say that smoking causes lung cancer deaths. Yet the Japanese
smoke more cigarettes than North Americans, and have LOWER rates of lung
cancer death. Therefore, clearly, smoking does not cause lung cancer death.
"They" say that dairy is protective against osteoporosis and fractures.
Yet many Asian cultures consume less dairy than North Americans, and
have LOWER rates of osteoporosis and fractures. Therefore, clearly,
dairy is not protective against osteoporosis and fractures.
The problem is that you CAN'T meaningfully isolate any one such factor
out of 2 aggregated populations and even begin to make causal
connections. Otherwise, you can pick any damned difference you like, and
point to it the factor causing any other damned difference that you
like. Eg: "The Chinese play more ping-pong than North Americans, and
they have a lower rate of heart disease than North Americans. Thus,
ping-pong is highly protective against heart disease." "The Japanese get
more stomach cancer than do North Americans, and they consume more green
tea. Thus, green tea cannot possibly be protective against stomach
cancer, and indeed, may CAUSE stomach cancer." ETC ETC.
You have to look WITHIN a cultural pattern, and see whether, WITHIN that
cultural pattern, INDIVIDUAL PEOPLE who do more X get less Y. If someone
will point me to a study in which individual Okinawans who consume more
soy get no more dementia than individual Okinawans who consume less,
after adjusting for education, income, occupation, age, and other
obvious covariates, then THAT will be interesting data. This just ain't.
----------
The same points are made in more formal fashion a webpage from the FDA,
of which I've appended edited sections at the end of this email; and
also, tho' in inadequate explication, here (note that study designs are
listed in order from strongest to weakest):
http://www.vetmed.wsu.edu/courses-jmgay/GlossClinStudy.htm
> the New York Times called it "the
> Grand Prix of Epidemiology."
The NYT is not a source of scientific evidence and shouldn't be treated
as one. And to make an important niggle: I guarantee that "the NYT" did
not say this; rather, some writer who happened to get an article
published IN the NYT said that in hir article. Evidently, the writer was
not a scientist (or perhaps s/he was a chemist, a geologist, or some
other scientist who would have no reason to have an understanding of
epidemiological study design -- cf the citing of "scientists" who don't
believe that anthropogenic C02 emissions are causing climate change who
have no actual background in climatology).
Back to me: rest of the message snipped because it doesn't directly relate to topic at hand.
MR is a lot snippier than I am online (which is odd, because anyone who knows us well -- and you know who you are -- will relate that I am much more snippy in real life and he is much, much sweeter!) but I think his examples make the point pretty well. We can't judge from broad observations about an entire population what we personally should do. We can, however, learn quite a bit from good epidemiology in which individuals were followed over a period of time and confounding factors were controlled for (to the extent possible.) And we can learn even more from clinical trials.
I know that in my intitial year of CR, I found it both frustrating and fun to sift through mountains of information to come up with what I thought was a healthy way of eating that would fit my lifestyle. I hope my readers have more fun than frustration, and please feel free to use the blog as a forum to exchange ideas!
Posted by april at November 10, 2006 6:19 PM
Comments
Its always interesting to see what MR has to say on the CR lists. His posts are always well thought out and backed up by good evidence. He used to post more on the CRS before, but now he must be real busy researching and writing for aubrey?
Posted by: matt - uk at November 10, 2006 2:48 PM
Hey! I've been really enjoying your blog--it's a nice counterpoint to the somewhat hysterical-sounding articles I've read about CR.
While extreme CR isn't the path for me, I do agree with you in that when I consume less calories--and when those calories are from good sources--I feel far better than when I consume too many calories, even when said calories are from healthy sources like whole grains, fruit, etc. For the last couple weeks I've been maintaining around 1700 calories a day, with daily exercise (weight lifting, rowing, and running)--so those are my specs :)
Anyway, back to the articles--it's too bad that they make calorie restriction (in general, not just hardcore CR) sound like such an all-or-nothing business. I wonder if people would be friendlier to the idea if they were given the first step of only eating when hungry, and stopping when full? So many people waste calories "grazing" that even cutting out those snacks would show large calorie losses and probably greater health benefits as well. After that, steps towards hard-core CR could be taken or not taken as the person pleased. I wish this sort of middle-of-the-road approach was presented (which your blog does, in a way) more often.
Anyway, just a thought bouncing around in my head. Love the blog, keep up the good work :) (and exuse the lack of an email address!)
Posted by: Jane at November 10, 2006 2:54 PM
I am not a scientist and certainly not the most logical person on this earth, but how does all this explain the fact that SOME MORE Okinawans and SOME MORE Japanese people for that fact, actually do live longer, healthier lives than SOME Americans or people from other cultures? What are the contributors to good health? Good genes, good nutrition, less exposure to toxicity, an active life, better emotional state. If we are pretty sure that genes and good nutrition are very important for a healthier and longer life, can we not conclude that regardless of the findings on soy+dementia or anything else like smoking that is life shortening, Okinawans either have really good genes, or eat a great nutritious diet which annihalates the negative symptoms of soy, tobacco or whatever else they consume or a great combination of both which make other nations ill?
Okinawans were just one example. We can expand it to Sicilians and Cretans too. I am not going to go into detail about their diets and lifestyles, but we all know Italians and Greeks smoke much more than Americans do too. So, apparently, there is something else at work there. Most probably overall nutrition, overall lifestyle and definitely genes.
Did I make any sense?
Posted by: istanbulwitchy at November 10, 2006 3:52 PM
istanbulwitchy:
Check out this article on genes and longeivty
Genes reveal little on longevity
http://tinyurl.com/ydledz
Okinawa don't seem to have 'super genes' protecting them from diseases that are usually common in the west... You should check out the 'okinawa program' book, it's quite interesting!
Posted by: matt - uk at November 10, 2006 4:51 PM
I didn't phrase that well. Excuse my turkenglish. What I meant to say is, looking at the long, healthy life span of Okinawans and the Japanese, we can conclude that something that is in them or that they do, or a combination of those factors together (genes, or nutrition and activity level or both)annihilates the negative effects of smoking or soy or anything else "bad" that they do.
It might be the rice, it might be the genes that make the slanted eyes, it might be the bento boxes, the seaweed or a magical combination of all those or other factors. If I were a researcher searching for the secret of a long, healthy life, I would pick individuals who lived healthily beyond age 100 regardless of culture or country, analyse each aspect of their great lives in detail, then compare and contrast those data with eachother to find some common demeanors. Of course, if I had more 100+ olds from the same culture/diet, I would add that fact.
Posted by: istanbulwitchy at November 10, 2006 5:17 PM
Hi, I live in Japan. Istanbulwitchy, I think the "something" that has given Japanese people longer lifespans is simply... CR! I'm not kidding. There is a real difference in the socially and culturally acceptable calorie intake (compring Japan and America). Portions are smaller. The average diet is also less calorie-dense due to the low proportion of animal-based foods. The average person here simply eats less food, and the food that he or she does eat tends to be less processed (with the exception of white rice, the nutritionally pathetic Achilles heel of the Japanese diet). And I think this leads to longer lives.
In fact, I decided to start CR myself when I finally correlated the Walfords' claims with the evidence from my own eyes. Japan is one huge test case for this theory, proving that it works.
Of course, everything good about the Japanese diet has been slowly disappearing for the last 2-3 decades and is now disappearing even faster. Junk food, processed food, imitation food, all are gaining popularity together with the American pathology that food is "a sinful luxury" but "you deserve it" and over-eating equals "delicious indulgence." Result: obesity rates are rising and I won't be surprised if life expectancy soon starts to fall. It's very depressing.
However, the markets are still full of lovely nutritious traditional foods (including tofu, sorry). Now, if I can only find out how to enter all of my Japanese foods into my nutritional software...
Keep up the good work on the blog, April!
Posted by: FS at November 12, 2006 11:01 PM