5 Things Science Is Getting Wrong, According To Scientists

Science is the best tool we have for understanding the world. But there are ways we could be doing science better. From alien technology to COVID models to peer review, scientists disclose mistakes and disagreements happening in science.

1. COVID-19 Modeling

“The vast majority of modeling is based on pixie dust,” says epidemiologist Michael Osterholm, Director of the Center for Infectious Disease Research and Policy and former member of president Biden’s Covid-19 Advisory Board. Osterholm is talking about modeling that forecasts how new and existing variants are going to play out. He notes that models routinely get it wrong but they’re still pumped out and plastered across headlines and news tickers. “They have been wrong so many times and nobody’s ever gone back and said, ‘what’s the track record here?’” Osterholm’s team has been monitoring projections from one particular institute frequently cited in the news. “We’re putting together a paper on the models showing just how bad they are.”

Osterholm is not against modeling in principle, as long as the forecasting adjusts to new information and projections are not much further out than weather reports. “I think the modeling that I would say has the most positive benefit is the ensemble model that the CDC puts together. That is a model that takes all the models and brings them together in kind of one net analysis. They never go more than one month out. Never. And they’re constantly adjusting their model even within that one month. To me, that’s a credible way to do it.”

The need for answers that drives pandemic forecasting may also motivate scientists and public health officials to make bold or overly generalized statements, even when statements later turn out to be false. Osterholm thinks this burns public trust capital. “Just be honest with what we know and don’t know,” says Osterholm who believes the public isn’t afraid of uncertainty as much as not being told the truth, even when the truth is I don’t know.

But saying I don’t know isn’t enough. “It’s never enough,” says Osterholm. “Then you have to say what you’re going to do to try to find out.” And this is the part that can be frustrating. Scientific investigations done right take time to objectively gather and make sense of data. And new variants have been turning up fast. “I don’t know what the next shoe is to drop. I don’t know. It could be this is as bad as it’s gonna get, from here on out much more mild illness if you have any immunity at all,” says Osterholm. “Or maybe it won’t be. I don’t know.”

Osterholm says one of the big unknowns right now is waning immunity: “You can’t understand vaccine waning immunity until you’ve been out two or three years in terms of your vaccination program.”

So what do we know? FDA-approved COVID-19 vaccines have proven they help prevent severe disease and death from COVID-19. And, again, science takes time. All three of the FDA-approved mRNA and adenovirus-based vaccines were available so quickly only because scientists had been developing these vaccines over decades.

We also know a lot about masks. Osterholm says mask mandates are too broad while the science around masking isn’t broad. It’s specific. “No one has been more concerned about respiratory protection than me,” says Osterholm, adding, “I oppose mask mandates.” His issue is how science is communicated, or not well-communicated, as it’s turned into policy. “I’m a very pro N95 masker. That’s what will effectively protect you. I just care about protecting people. I don’t care about what’s politically correct or politically wrong. I just gotta give people the information,” says Osterholm. “If [mandates] were to say, you have to wear an N95 respirator and wear it over your nose, I would be the biggest supporter in the world of that.”

2. Possible Alien Technology

In 2017, an unidentified space object, dubbed Oumuamua, passed through our solar system. Oumuamua appeared as a speck in the images captured by telescopes. Using sophisticated filters and instruments, astronomers measured and determined certain properties of Oumuamua that revealed it was nothing like any asteroid or comet ever seen before. Its shape was thin, long and “quite surprising” according to NASA. It moved in a trajectory that deviated from gravity. This is something we see in comets. But Oumuamua showed no detectable outgassing. No tail or coma as seen on comets. And it had unusual and variable brightness. The scientific community settled on classifying Oumuamua as an indeterminate interstellar object. Physicist, author and Harvard astronomer, Avi Loeb, thinks Oumuamua could be a piece of alien tech rather than a space rock.

To be clear, Loeb isn’t insisting Oumuamua is alien technology. He’s saying alien technology is one possible hypothesis among others. And he thinks the scientific community is departing from the scientific method by outright dismissing it.

“The mistake being made by my colleagues is that they don’t like the idea of Oumuamua in the first place. One of them said to me after a lecture, ‘Oumuamua is so weird, I wish it never existed,’” says Loeb, adding that his colleague is an expert on rocks and wants everything in the sky to look like a rock. “That’s really bad if you find it problematic that something looks weird.”

Scientists around the world hotly debated what Oumuamua could be. “[One] group explained it as a hydrogen iceberg, another group explained it as a nitrogen iceberg, another group explained it as a dust bunny. All of these are things we’ve never seen before. So we must leave on the table the artificial origin possibility,” says Loeb.

Loeb is urging that open-mindedness and funding for particle and theoretical physics also be applied to the search for intelligent life. He questions the scientific community’s willingness to search for hypothetical particles but not hypothetical intelligence. “We’ve paid billions of dollars to find the nature of dark matter for 40 years. We haven’t found it. Okay. My point is, the nature of dark matter will have zero impact on the daily lives of people.”

The existence of intelligent life in the universe has huge implications for humanity and Loeb says the cost to search for it is a fraction of what we’ve allotted to the search for dark matter. “The question that I’m raising is of great interest to the public. It’s also of great interest to the government because they’re establishing a new office that will start operations in June [2022] that will explore unidentified objects in the sky from reports by military personnel.” Loeb is referring to the U.S. Department of Defense’s Airborne Object Identification and Management Synchronization Group (AOIMSG). “So it’s of interest to the public, interest to the government, yet academia ridicules it.”

Loeb thinks scientists are comfortable looking for dark matter in space or microbial life on other planets because none of this stuff threatens our standing as the most intelligent life in the universe. “We want to feel that we are the smartest thing that ever existed since the big bang,” says Loeb. He rejects this proposition by pointing to the vastness and age of the universe, that conditions on earth are not unusual, and how it’s easy to imagine an alien civilization that evolved to be as smart if not smarter than us. “My colleagues say any statement that relates what we observe in the sky to extraterrestrial intelligence is an extraordinary claim. And that requires extraordinary evidence,” Loeb smiles. “It’s not an extraordinary claim. Because we are not extraordinary.”

It might be easier to dismiss a scientist of lower pedigree. A former theoretical physicist at Princeton’s Institute for Advanced Study, Loeb moved to Harvard to become a tenured astronomer. He’s a fellow of the International Academy of Astronautics, a fellow at the American Academy of Arts and Sciences as well as the American Physical Society. He served as Chair of the Board on Physics and Astronomy of the National Academies and Time Magazine named him one of the 25 most influential people in space. Yet he has faced derision from scientists and the media for his unyielding interest in keeping alien tech on the table as a possible hypothesis for what Oumuamua actually is. “If I surrender to social media pressure just because people don’t like me on Twitter, I’m betraying the scientific method,” says Loeb, adding, “I will immediately surrender to evidence.”

And that’s the crux of it. More evidence is needed to identify anomalous interstellar objects. Loeb says he needs 1% of the endowment that is funding the search for dark matter to capture evidence of possible alien tech. He’s already raised $300 thousand, so when the next Oumuamua passes through our sky, Loeb can snap a photo of it in clear, sharp “high resolution that shows,” says Loeb, “that it’s an object with bolts and screws and you can read off the label made on exoplanet Y. Where it’s clear that anyone who sees the image will not be able to argue that it’s a nitrogen iceberg.”

On April 20, 2022, Loeb announced he wants to collect debris from a recently confirmed interstellar object (predating Oumuamua) that crash-landed on earth. It’s currently resting on the ocean floor near Papua New Guinea. Loeb wants to check it to see if it’s a space rock or something else.

3. You Need A Brain To Think

Most of us are comfortable with the relatively modern idea that animals like apes, dolphins and elephants are thinking creatures with mental lives. The proposition that fish and insects have many of the same cognitive abilities as humans may be a bit harder to swallow. Take it a step further. A growing consensus among biologists and ethnologists is that brainless, single-cell organisms like slime molds also have complex cognitive abilities. Experiments demonstrate that these bright yellow blobs have memories, can learn, make decisions, problem solve and think.

“Many of the things I work on are controversial in certain fields,” says Michael Levin, a developmental and synthetic biologist and Director of the Allen Discovery Center in Massachusetts. Levin’s experiments contribute to the growing body of scientific literature that demonstrates intelligence in slime molds. “The amazing thing is that those exact same ideas are not controversial in other fields.”

One of Levin’s controversial ideas is that thinking evolved before brains. Levin says biological systems have been doing things like learning and problem-solving long before brains evolved. “Take the most deterministic machine-like thing–a genetic network,” offers Levin. “There’s no magic in there. It’s just a bunch of genes turning each other on and off. And guess what? We showed that they can do six different kinds of learning, including associative learning.” Associative learning is the process of making connections between things like stimuli from current experiences and memories of past events so that you learn and adapt. “All we can ever do is ask, what is this system capable of? And by the way, we get surprised all the time.”

When asked to respond to scientists who are skeptical and outright dismissive of intelligence on the level of a genetic network, Levin simply points to his experiments and says: “You can’t have feelings about this.”

Levin acknowledges science’s long history of misclassifying things based on false dichotomies. He says there’s no definitive point in the history of biology, a before and after, where thinking bursts on the scene. “Darwin wrote about cognition and plants and other [organisms] because he understood for his theory to make any sense, everything has to be gradual,” says Levin. He counsels skeptics to take as an example something as uncontroversial as the intelligence of a modern human and walk it back on the evolutionary timeline. “You tell me when the lights wink out. There is never a good point. Never.”

Although Levin’s experiments are frequently groundbreaking and headline-making, he feels he’s not such an outlier in science these days. “I’m not claiming that I’m some kind of unique prophet who understands what science is getting wrong. There are so many people that get it right,” says Levin. “There’s a whole community of [scientists] who work on cognition, who work on neuroscience beyond neurons.”

One of these researchers is Anna Ciaunica, cognitive scientist, philosopher and author of the research paper “I Overthink–Therefore I Am Not,” about how overthinking can cause us to disassociate from ourselves. Ciaunica and Levin are collaborating on a new interdisciplinary research project on self-organizing systems. The working title for their paper is “From Cells to Selves.” Ciaunica says this research aims to explore intelligence through a biogenic lens instead of the traditional, more narrow anthropogenic lens. This means breaking out of a familiar brain-based architecture of intelligence and exploring the full biological landscape of possibilities.

“We need a conceptual toolbox to approach this from a different perspective, from an interdisciplinary perspective,” says Ciaunica. “We need different disciplines, different tools to understand the phenomena.” She says the resistance from some corners of science is simply inertia from an inherited framework for understanding the world, back when only a few perspectives were allowed to participate in science and define that framework. “There is a lot of pushback and it’s going to be a long road,” says Ciaunica. “I think this type of shift will take some time [and] will basically shed new light on the way we define fundamental concepts such as intelligence.”

4. Problems With Peer Review

Peer review is a process for trying to validate the findings of a scientific study. After a study is completed, the research team writes up a report on how they conducted their study. This report on the study is called a paper and the researchers are authors of the paper. The researchers then submit their paper to a scientific journal. The editor at the journal sends the paper to scientists to review it. Typically two to four (the average is three) volunteer scientists read through the paper to see if it looks like the study was done correctly. The scientists then inform the journal’s editor if they approve or disapprove of the paper, or if there seems to be a problem that needs fixing.

When it comes to the peer review process: “We have a bit of a black box problem,” says Tim Errington, Director of Research at the Center for Open Science (COS). Open Science is a movement to make science more transparent and accessible. The movement started in the 17th century with the introduction of the scientific journal. Today, scientific journals themselves are being taken to task by organizations like COS.

The problems inside the peer review black box are sometimes surprising. For instance, “peer review” doesn’t mean that the scientists who review the paper necessarily approve it for publication. If one out of the three reviewing scientists recommends a paper not get published, it sometimes gets published anyway. “I was just asking for more clarity,” says Errington, remembering when he was on a peer review panel and he disapproved of the study and it was published anyway: “I was like, ‘you can’t publish this paper because the methodology is so poorly described. I don’t even understand what’s going on.’” Scientists doing peer review often don’t have any idea who the other reviewers are, so it’s unknown how many reviewing scientists may approve or disapprove of a paper. Errington says that like many scientists, he simply stopped accepting requests to conduct peer review. So when peer review is extolled as the arbiter of truth, Errington balks. “Stop thinking that peer review is indeed the high-quality control that people think it is. Because it’s not set up to do that.”

It’s not a stretch to imagine that problems inherent in the peer review process may be one of the reasons many studies get published and then fail to replicate. Replication is where scientists redo original studies, step-by-step, to see if they get the same results as the original study. Errington’s background is in preclinical cancer biology and he’s involved in a large-scale replication project in his field. “Less than half can replicate. Less than half are getting similar results,” says Errington. “When we do get results, the effect size in our replications,” adds Errington, “it’s like 15% the size of the original. Like just mind-blowingly small.”

Yet, even with all these problems, you wouldn’t want to get cancer in a pre-scientific world. “There’s a lot of inefficiency in our scientific system,” says Errington. “Yet we still make a lot of progress.” Science-based agriculture techniques, vaccines and medicines that led to a massive spike in population mean science is the reason most of us even exist today. And without science there’d be no internet to read what life would be like without science. “The scientific method, it’s our best tool,” says Errington. “Imagine if we could start to sort through this and make [science] more open, more reproducible. Imagine what we could do then.”

He thinks the problem boils down to what science is incentivizing, or what’s known in the business world as Kerr’s Folly: rewarding A while hoping for B. “You’re telling me, in order to get a job, in order to get a grant, in order to stay in my job and be successful in my career, I need to get a flashy, exciting publication with lots of citations. You never told me I needed to do rigorous quality work,” says Errington. “Doesn’t mean that I don’t believe it. You’re just not rewarding me for it.”

Errington also points to the glut of scientific journals out there. “A lot are just taking advantage of the system. There’s a lot of predatory journals. They won’t do quality peer review. They’ll charge authors a lot of money,” says Errington. “They know that there’s a need for it because it’s the currency of science.”

Things are changing. Over the past 5 years, preprints have gained popularity. Preprints are papers that get published to preprint servers so studies are made available to the public before the paper gets peer reviewed. “It used to be the first time you’re telling the world about your work is when a journal publishes it. But now it’s really that preprint moment,” says Anna Schapiro, neuroscientist and principal investigator at Penn Computational Cognitive Neuroscience Lab. Schapiro also serves as an editor for eLife, a relatively new non-profit scientific journal. “It’s a crazy journal because their ultimate goal is to get rid of journals,” says Schapiro, adding that eLife wants to transition to a system where the paper lives on a preprint server. “The role of the journals is to post evaluations on those preprint servers. You could have many different journals evaluating the same paper,” says Schapiro, adding that evaluation from a reputable scientific journal could function as a stamp of approval.

Of course, this would mean that a journal could no longer “break” new studies and host studies behind paywalls. “That is the biggest problem. The biggest problem is that the [scientific] journals are a business. We should not have for-profit journals,” says Schapiro. “I think it’s so important that science is accessible. That everything is open access. That [papers] are not behind paywalls. Everybody, whether they’re scientists or not, should have access to the research. Most of this research is funded by the government and everybody should have access to it. I think the public should be furious that that isn’t true by default. Our government, our funding sources should demand that.”

When asked if she’s concerned about speaking out against journals for fear that her own studies might not get published, she points to the major shift underway from this next generation of scientists: “I think that people are starting to value open science in a way that wasn’t true before. So it might not be as problematic for our careers.”

When it comes to open and accessible science, Errington and Schapiro agree. “In order for you to trust science, in order for you to trust any institution, we need to be open and transparent,” says Errington.

It should be noted: While peer review may be the final stop for a scientific research paper, peer review is only one part of the control process in clinical trials for approving drugs and vaccines. When a new drug or intervention is tested on humans, these tests are conducted with rigorous government regulation and oversight through phases of clinical trials.

5. The Science Career Track

In the twenty-first century, anyone with access to a university education who is reasonably intelligent and dedicated can have a career as a scientist. This plurality of ideas and perspectives should only enrich possibilities for scientific inquiry.

“The democratization of science is a good thing,” says Gavin Naylor, director of the Florida Program for Shark Research, adding one major caveat: diverse perspectives are squeezed into an impermissive and narrowly defined career path in academia. Naylor says science has been turned into a job with all of the banality associated with a “job.” Rather than rewarding innovative and creative scientific exploration, the tendency is to constrict. The passion to make high-quality big discoveries has been replaced by the pressure to routinely turn out minor lowercase ‘d’ discoveries.

“Everybody needs to be on their treadmill showing that you’re alive and got a pulse,” says Naylor, rebuking the grind foisted upon himself and his colleagues. “We’ve got professors at fairly prestigious universities that have 500 publications that they haven’t even read. They haven’t read them.” Naylor is referring to an academic convention where senior scientists are listed as authors of studies in which they had very little involvement.

“I have taken credit for stuff I didn’t really do very well,” concedes Naylor, adding that every year scientists scramble to amass credits and grants for underwhelming research. “We all do it. I do it. Everybody does it. When everybody’s got the same disease, who’s going to police it?”

This, according to Naylor, has resulted in legions of scientists who are no longer driven by the intrinsic joy of discovery. “The things that I’m most passionate about, I would never get grant money for. I’ve been fairly consistently funded by The National Science Foundation to do risk-averse work,” says Naylor. “It fosters mediocrity in the discovery process.”

Naylor saw this accelerating publication frenzy when he was a PhD student in 1988. He confided in his professor and mentor, ecology and evolutionary biologist, Lin Chao, that he felt the situation for scientists was just getting worse and worse. Naylor animatedly relates the conversation from 34 years ago: “Lin said, ‘of course there’s a solution.’ And I said, ‘really? What is it?’ He said, ‘everybody’s only allowed to produce 5 pieces of work in a career.’ And I said, ‘what?!’ He goes, ‘yeah, you have to focus on quality.’”

Naylor says one of the ways to focus on high-quality work is to take the time to come up with high-quality hypotheses. He feels the entire scientific community needs to pivot from the stultifying pressure to drum up risk-averse hypotheses to produce mediocre research projects in exchange for job security. “You need to be patient, you need to be reflective and you need to be observant,” says Naylor. He adds that when it comes to frontloading the scientific process with the wonder, curiosity and exploratory period that produces hypotheses worth testing, “nobody has any time.”

And this is exactly why he’s so excited about the James Webb telescope: “The bloody Webb telescope makes the hair on the back of my arms stand up. That is the coolest thing that has happened in my lifetime. And I’m not even a physicist. I’m not even an astronomer.” Naylor sees tremendous value in spending a boatload of money to build and launch into the cosmos a telescope with no preconceived judgments. A big objective eye, patiently observing space and time. “The notion that governments could come together and put money for us to do something so spectacularly useless and so deeply interesting, gives me hope. We are going to peer into the origin of the universe, which is not going to affect the stock market. It’s not going to cure world hunger. It’s not going to change the climate. It’s a bunch of people that just want to know.”

We owe so much to science: from water sanitation to cures for once fatal diseases to harnessing limitless energy from the sun. Science helps us understand phenomena on scales from the subatomic to the galactic. It displaces superstition and ignorance the way the sun burns off a morning frost. But as these examples show, we can do science better. The improvement starts when science turns analysis on itself.

Watch Naylor get his mind blown at 1500 feet under the sea observing sixgill sharks, a species he studies that are older than the dinosaurs and bigger than great whites. “It was literally the most amazing experience I’ve ever seen,” says Naylor. “I felt like an astronaut.”