The term “p-hacking” has made its way into the public discourse surrounding science, particularly regarding the replicability crisis. But although the term suggests intentional malevolence on the part of the scientist, it’s actually a scenario that many well-trained scientists can fall into. So what is p-hacking, and why is it dangerous?
Humans are pretty creative. We adapt the world around us to fit our needs and develop tools to help with almost any task imaginable. Other animals use tools as well, but not to the same extent, and not with the same flexibility and ease.
Tools make us more efficient, and more intelligent (for the most part); they help optimize the relationship between our bodies, our minds, and the world we live in. Some researchers (Clark and Chalmers, 1998; Hutchins, 1995) even argue that tools, in some ways, extend our minds and our bodies. And, in fact, there’s some evidence from cognitive neuroscience that the brain considers tools – at least some of the time – part of the body.
Science faces a replicability crisis. This is well-known among scientists and even the general public. Various explanations have been proposed as to the cause of this crisis – some of them on this blog – but these proposals have usually been informal in nature. Recently, two scientists (Smaldino & McElreath, 2016) built a computational model to explain the replicability crisis.
Science is a framework for understanding the world. We observe a phenomenon, ask questions about it, and build models to describe and predict it. Crucially, this process is iterative. We’re constantly refining our experiments, theories, and models, in an effort to improve our understanding of some phenomenon. Instead of accepting the results of a study as “fact”, we ask whether those results can be replicated reliably under similar and different conditions, what theories can be extrapolated from those results, and how we might extend those results to new domains or new questions.
Iteration is how science fosters dialogue. Without it, science isn’t really science; it’s just a bunch of people shouting over one another, with nobody listening. Or even worse, it’s like a game of telephone, in which nobody checks whether the last person got the message right.
Anyone who’s ever used Siri has likely experienced the frustration of not being understood. There’s a fundamental – almost existential – panic that surfaces when someone else doesn’t know what you’re saying.
It’s an even bigger problem for people with accents other than “General American English”. Voice interfaces struggle with accents, from regional American accents to foreign-accented speech. But why?