Statistics, as a discipline, has focused almost exclusively on the challenges of the third part, hypothesis testing, termed confirmatory data analysis (CDA). The challenge in testing our hypotheses is that the world is noisy, and it can be very difficult to distinguish the signal - the patterns in our data that are stable from sample to sample - from the noise - the patterns in our data that change randomly from sample to sample. Science often proceeds in roughly three parts: notice a pattern in nature, form a hypothesis about it, and then test the hypothesis by measuring nature. This imbalance is crippling the social sciences. Satan, Hell and Evil were the “active springing from Energy,” wrongly suppressed by the Church, the Dionysian excess that “leads to the palace of wisdom.”ĭata analysis in the social sciences has two forms, one “good” that is highly developed and has many rules that supposedly will lead us to truth, and one “bad” that lives in the shadows, has few if any rules, and is frequently, but wrongly, vilified. God, Heaven, and the Good were the “passive that obeys reason,” our Apollonian side. William Blake, for example, in The Marriage of Heaven and Hell, took aim at the imbalance he saw in Christian theology. If one element gains the upper hand over the other, beware. Many intellectual and moral frameworks are structured around two, often opposing, elements. Plate 4 of The Marriage of Heaven and Hell. Another solution to the problem of data dredging is to use the Bonferroni correction.Good and Evil Angels.

It is now common practice to register clinical trials and specify in advance what the primary endpoints and hypotheses are to avoid the bias of data dredging. They may not be a true relationship and is spurious and any correlation found is by chance.ĭata dredging is also referred to as fishing, p-hacking, significance chasing or data snooping. If you do many and repeated statistical tests (multiple comparisons) on a data set, then some will be statistically significant by chance. This typically happens when a data set is examined too many times with many statistical tests on the data and then only reporting or paying attention to those results that come back with statistical significance. This leads to a spurious excess of false-positive and statistically significant results. Data dredging is the cherry-picking of multiple statistical tests on a data set to demonstrate a promising or attractive finding.

Data dredging is ok