I’m always hearing about new scientific findings. It can be hard to determine what’s true, but it can be done. Here’s how.
- Consider the source’s credentials. Try to find the same evidence being reported and interpreted by more than one reputable scientific source. This includes original sources such as scientists or publications written by the scientists. It does not include any random person who runs a website, writes a blog, etc. The writer should be a person who holds the appropriate credentials in his/her field (not a dentist writing about dermatology).
- Consider the source’s motives. Find out who is funding the research and who’s involved. Is a for-profit company making claims about their own products?
- Ignore most conspiracy theories and fear-mongering. These are usually gimmicks to get gullible people to pay attention and take action without pausing to consider the facts. “Fear-mongering” is not the same as valid, evidence-based warnings issued by scientists. Fear-mongering plays on emotions and is not based on fact.
- Study design should include enough time/participants to ensure conclusions are valid. If 100% of cancer patients in a study who took a yoga class had their cancer go into remission, but there were only 20 cancer patients in the study, there is inadequate information to draw a conclusion.
- Consider methods used to study the issue. Were there any flaws in the methods that might have rendered the conclusions unreliable?
- Consider generalizability. Are the conclusions really applicable to the general public?
- Consider other evidence. Check out information from studies on the same or similar topics. Meta-analyses are great sources of information because they compile information from many studies instead of just one.
- Look for random double-blind placebo-controlled designs. In double-blind studies, the researcher and the participant (subject) are unaware of which group (treatment or placebo) the subject is assigned to. Placebo means that a group receives exactly the same medication (or situation) except for the one factor being studied. Participants should be randomly assigned to their groups.
- Consider your own bias. Are you a smoker who doesn’t want to hear that your secondhand smoke caused your wife’s heart attack? We all have biases. Make sure you’re aware of them so you don’t discount valid information.
- Consider margins of error. When a study involves sampling, the margin of error is a description of how much variation could be expected to exist in the universe (entire body being studied). Imagine someone claims the rate of absinthe use in your college doubled last year, but there were only 2 absinthe users to begin with. Your college has 20,000 students. The margin of error makes it impossible to draw a conclusion that the rate actually increased, because it was so small to begin with. Sampling may explain the differences.
- Ignore most “quick fixes”. Anyone who claims you can cure Cancer by drinking water or fix your debt by buying a book is probably wrong.
A good article on this subject: http://www.ific.org/publications/reviews/scientificir.cfm
