HCI statistics without p-values
Errata
- The p-value for Pill 4 is .012, not .001 (Figures 2, 3, 7 and Table 1). Fortunately this does not change the argument, and even makes it slightly more compelling. This will be corrected in a later revision.
(Added 25 June 2015) Thanks to Geoff Cumming for spotting this. - I revised my definition of estimation, and think it should refer to reporting effect sizes with interval estimates in general, rather than confidence intervals specifically. Confidence intervals are one type of interval estimate, a closely related one is the Bayesian (credible) interval. Credible intervals have many advantages but are harder to master and still not widely used and supported by stats packages. I will briefly discuss differences in a later revision, with refs. For now, my position is that switching from mechanical significance testing to estimation is a larger step forward than switching from frequentist to Bayesian calculation procedures (many would agree, many would also disagree). In typical HCI analyses, which are generally quite simple, confidence intervals often approximate so-called "objective" credible intervals and are thus a perfectly good place to start. HCI researchers who are perfectionists and are willing to invest time can of course directly switch to proper Bayesian estimation, but it is also perfectly acceptable and natural to switch after having gained some experience with estimation using confidence intervals, and/or after Bayesian methods become more firmly established in research practice.
(Added 25 June 2015) Thanks to Dan Simmons and Steven Franconeri for providing me with thought-provoking reading material on this.
New and updated tips
None so far.
Comments
I'll be happy to add questions, comments and commentaries here.
License
 | All material on this page is CC-BY-SA. You can reuse or adapt it provided you link to www.aviz.fr/badstats. |