Is Science Fatally Flawed?
Photo by chuttersnap on Unsplash
In a fascinating article published in First Things (“Scientific Regress,” May 2016), William A. Wilson explains several data reviews that uncover some ugly truths about peer-reviewed scientific journals. These reviews showed that a surprising number of published studies did not show the same results when repeated. But repetition is one of the hallmark self-correcting tools of science.
The data reviews also showed that the peer-review process is not exactly rigorous; many studies get published even though they have major errors. Even for those experiments that have proper methodology, it was found that the statistical analysis afterward teased out all kinds of conclusions that were not necessarily justified.
This kind of information is alarming, living as we do in a culture that exalts science. Peer-reviewed scientific journals are supposed to be a filter that keeps junk science from propagating, but these reviews show that they’re doing a poor job. The reasons are numerous, from simple human error, to a flawed process of deciding which studies are published (which is at the whim of the businesses or organizations paying for the studies, and the journals, which are competing for readers), to straightforward fraud and data manipulation. The solutions to this mess are several, and hopefully the processes of research and publication will be reviewed and adjusted in the coming years.
But I want to talk about what this means right now. Should we abandon our trust in science? Should we assume that what we know is wrong? Should we change or abandon scientific methodology? What are we to make of the field of statistics, when it seems like you can find a statistical “relationship” between anything and anything else?
I think that the article is important and should be read by lovers of science. But I don’t want us to overreact to what it reveals. Here are a few of my reactions.
Psychology is still a soft science. I am fascinated by psychology, and that’s why I got my bachelor’s degree in it. But lumping psychology in with harder sciences like biology or physics is misleading; as a soft science, what we learn in psychology is more tentative and open to correction. Humans are mysterious and self-contradictory creatures who don’t always act in the ways we’re “supposed to.”
Evolutionary psychology is an attempt to make psychology more of a hard science by rooting our behaviors in specific survival advantages. It seems to work pretty well with animals, but it doesn’t have a great track record of predicting or explaining human behavior.
Psychology is important and has led to some great insights into why we think and feel the ways we do. But its principles are somewhat difficult to nail down in repeatable scientific experiments; and free will means that there will always be unexplainable outliers.
One of the most important pieces of information about any experiment or study is who paid for it. That isn’t to say that everyone who funds scientific studies is trying to “game the system” and get the results they want; some organizations just want more data, and others really believe they’re right about something and want confirmation. But when a study shows that potatoes cure cancer, alleviate ADHD, and result in mind-blowing sexual experiences, and you find out it was paid for by the Idaho Department of Agriculture, you might want to take it with a grain of salt (the study, not the potato).
Many scientific studies are conducted to prove a point (by a university or a business), and any information that doesn’t support the point is disregarded. I don’t know how to correct for that, except to make sources of funding apparent so people can judge for themselves the likely objectiveness of a given study. And perhaps we can establish some Best Management guidelines for increased study replication or better statistical analysis. Even if the daily practice of conducting and reporting on scientific experiments can’t be made perfect, it can be made better, and we should keep looking for ways to improve.
Whenever someone on the news says, “A new study shows…” you can pretty much ignore them. This has always been a problem; people who report the news need something to report, so on slow days, they become desperate to air something that gets people’s attention. The problem has worsened exponentially now that we live in a 24-hour news cycle, with television, radio, and Internet news sites always looking for stories that attract eyeballs and ears.
Others have written better than I can about how little of the news today is even marginally newsworthy, but my own opinion is that if you read one newspaper a day, and a magazine or two each month, you’ll probably have as much information about what is going on in the world as you need.
Those who gather and report the news look at science as a trove of surprising and eye-catching studies that only nerds will really investigate or question. So they tend to take any little nugget of counter-intuitive data and turn it into a whole story about how “everything we’ve thought all along has been dead wrong.” When these stories are eventually refuted by the facts, news consumers have long since moved on to something else. As a result, if you rely on the news to learn about science, you probably know less than if you sought no information at all.
I’m not saying that people in the news business are liars or are deliberately deceiving the public. It’s just that the system, as it’s currently set up, requires new bits of news to be fed into the mill every hour of every day. That leads to a habit of framing everything by the simplest point of reference as possible. The framework “you thought you knew, but you were wrong,” is a handy and easy way of presenting a science story, so that’s the go-to setup.
Unless you get your news from NASA or National Geographic (and even those aren’t perfect), you can pretty much ignore the major news sources when they say things about science. And when you do hear something that intrigues you, do your own research into what’s going on and get to the bottom of things. It’s easier than ever in the Age of the Internet – just remember to be smart about your sources.
Much of what we think of as science does not come from peer-reviewed scientific journals. I have been fascinated by science my whole life, and have read a small library of books and articles about science. I was gainfully employed in a science job for fifteen years and have satisfied my considerable curiosity regarding a number of scientific topics. I have never once cracked open a peer-reviewed scientific journal. They serve a certain, narrow purpose in the scientific world. That purpose is important for academics and funding, so the value of that role is often magnified many times when it comes to science careers. But the importance of these journals over the long-term trajectory of science should not be exaggerated.
I learn about science from coworkers in conservation, from podcasts and videos, documentaries, classes, books, magazines, and other places. Some of the information is originally published in peer-reviewed journals, but a lot of it comes more directly from research and observation. Science as a discipline isn’t only conducted by people who are vying for tenure or trying to “prove” things that make their products look good. It’s done every day by people across a wide range of disciplines, and most of them don’t have ulterior motives. They just really want to know about the incredible universe we inhabit.
Science is awesome, but it’s done by imperfect humans. Even those of us who are enthusiastic about science should approach everything with healthy skepticism. Yes, science has mechanisms for self-correction, but we’ve learned that one strong mechanism, repeating a study, is very rarely done. So the imperfections of scientific work often go unchallenged.
That’s bad news for people who put their faith in science and believe science has all the answers. But it’s not surprising to those of us who try to keep science in its rightful place: it is one way to know one kind of truth. It doesn’t cover everything there is to know, or even the most important things. Science is great, but it is an enterprise conducted by imperfect humans doing their best to honor the truth (most of the time).
As a Christian, I believe that Jesus will return at the end of time to make all things new and set everything right. I believe that science itself will be redeemed and set free from the brokenness of the Fall. Then, it will be a much better tool for exploring the New Earth, and it will reveal incredible truths about God’s creation that we are only scratching the surface of now. We could study and explore for all of eternity, and never get to the end of God’s creative wonders, because He will never run out of ways to blow our minds. I believe (though it doesn’t explicitly say so in the Bible) that scientists who accept God’s free gift of grace can be scientists forever.
Learning that science is imperfect doesn’t upend my world; it makes me want to research as well as I can, and to get to the bottom of the science I read and hear about. I care about the truth, and I am fascinated by the universe God made and honored that He has invited us to share with Him in revealing His glory through His creation. Science is a fantastic gift; let’s do it well and always strive to do it better.