The Fuss Over P values


xkcd’s “real” interpretation of p-values

The American Statistical Association (ASA) has recently taken the unusual step of announcing a guideline document for preventing the misuse of p-values. They assert that scientists and policy-makers are using the p-value as a black-or-white decision parameter without truly understanding it and without inspecting the overall experiment / methodology / statistical framework. In this statement, the ASA advises researchers to refrain from drawing explicit scientific conclusions or making policy decisions based on just P values. They further advise that as part of scientific statistical analysis, the data analyses, statistical tests, and choices made in calculations should all be described in complete detail.

The ASA’s “statement on p-values: context, process, and purpose” can be accessed here, and the accompanying press release can be seen here.

I don’t want to go into the details of hypothesis testing and p-values here; those interested can take a course in statistical analysis or just scour google, or still, can take my graduate course on Data Analysis for the Earth Sciences.

To improve the conduct and interpretation of quantitative science, ASA has given the following six principles in the guideline document:

  1. P-values can indicate how incompatible the data are with a specified statistical model.
  2. P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone.
  3. Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.
  4. Proper inference requires full reporting and transparency.
  5. A p-value, or statistical significance, does not measure the size of an effect or the importance of a result.
  6. By itself, a p-value does not provide a good measure of evidence regarding a model or hypothesis.

Many interesting and insightful news articles related to this released guideline have been published from different platforms. Nature News contains this directly linked article, and this detailed article from 2014 which discusses the pitfalls of relying too much on P-values. There are engaging  analyses presented in ScienceNews and Inside Higher Ed. RetractionWatch also published an interview with the ASA’s executive director.

I asked my colleague Dr. Asad Ali, an expert in statistical analysis, for his views;  he had this to say:

This p-value has become quite controversial in the last few years. People are abusing it intentionally or miss-using it because of lack of knowledge.

Rejection of a hypothesis does not always mean that it’s wrong, rather it can be also because our evidence (sampled data / observations) is not very representative of the underlying population.

Dr. Asad Ali works in the fields of Bayesian statistics, astrostatistics, and geostatistics, teaching extensive graduate-level courses on these topics at GREL. He is also a part of the eLisa mission data analysis team to observe gravitational waves from space.
To end this blog  post, I can’t resist sharing this brain-wrecking xkcd p-value joke:

A joke for statisticians only

This entry was posted in Uncategorized and tagged on by .

About WQ

I received my PhD (2013) in Remote Sensing, Earth and Space Science at the Dept. of Aerospace Engineering Sciences, University of Colorado, Boulder, USA, under a Fulbright fellowship. Currently, I'm an Assistant Professor in the Dept. of Space Science at Institute of Space Technology (IST), Islamabad, Pakistan, where I have been a founding member of the Geospatial Research & Education Lab (GREL). My general expertise is in Remote Sensing where I have worked with various remote sensing datasets through my career, while for my PhD thesis I specifically worked on Remote Sensing using SAR (Synthetic Aperture Radar) and Oceanography, working extensively on development of techniques to measure ocean surface currents from space-borne SAR intensity images and interferometric data. My research interests are: Remote sensing, Synthetic Aperture Radar (SAR) imagery and interferometric data processing & analysis, Visible/Infrared/High-resolution satellite image processing & analysis, Oceanography, Earth system study and modelling, LIDAR data processing and analysis, Scientific programming. I am a reviewer for IEEE Transactions on Geoscience & Remote Sensing, Forest Ecosystems, GIScience & Remote Sensing, Journal of African Earth Sciences, and Italian Journal of Agronomy. I am an alumnus of Pakistan National Physics Talent Contest (NPTC), an alumnus of the Lindau Nobel Laureate Meetings, a Fulbright alumnus, and the Pakistan National Point of Contact for Space Generation Advisory Council (SGAC). I was an invited speaker at the TEDxIslamabad event held in Nov., 2014. I've served as mentor in the NASA International Space App Challenge Islamabad events in April 2015 and April 2016.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s