1. Home
  2. Volume 8 - Isuue 3
  3. Using predictive methods to assess observation and measure importance
New submission
Issues
Journal Information
Aims and ScopeEditorial BoardEditorial Advisory BoardJournal Policies
Resources
For Readers
For Authors
For Editors
For Reviewers
Publication EthicsCall for Special Issue
ISSN
ISSN 2615-9821
ISSN(Previous) 2588-1396

Volume 8 - Isuue 3 | November

Using predictive methods to assess observation and measure importance

William M. Briggs

05/12/2024 8 0 0
Like Buy now 20.000 đ
Bayes factors Hypothesis testing Predictive modeling Replication crisis

Abstract:

Purpose
This study aims to find suitable replacements for hypothesis testing and variable-importance measures.

Design/methodology/approach
This study explores under-used predictive methods.

Findings
The study's hypothesis testing can and should be replaced by predictive methods. It is the only way to know if models have any value.

Originality/value
This is the first time predictive methods have been used to demonstrate measure and variable importance. Hypothesis testing can never prove the goodness of models. Only predictive methods can.

References:

  1. Berger, J.O. and Selke, T. (1987), “Testing a point null hypothesis: the irreconcilability of p-values and evidence”, JASA, Vol. 33 No. 397, pp. 112-122, doi: 10.2307/2289131.
  2. Briggs, W.M. (2016), Uncertainty: the Soul of Probability, Modeling & Statistics, Springer, New York.
  3. Briggs, W.M. (2019), “Everything wrong with p-values under one roof”, in Kreinovich, V., Thach, N.N., Trung, N.D. and Thanh, D.V. (Eds), Beyond Traditional Probabilistic Methods in Economics, Springer, New York, pp. 22-44.
  4. Colquhoun, D. (2014), “An investigation of the false discovery rate and the misinterpretation of p-values”, Royal Society Open Science, Vol. 1 No. 3, pp. 1-16, doi: 10.1098/rsos.140216.
  5. Geisser, S. (1993), Predictive Inference: an Introduction, Chapman & Hall, New York.
  6. Geisser, S. and Eddy, W.F. (1979), “A predictive approach to model selection”, Journal of the American Statistical Association, Vol. 74 No. 365, pp. 153-160, doi: 10.1080/01621459.1979.10481632.
  7. Goodman, S.N. (2001), “Of p-values and Bayes: a modest proposal”, Epidemiology, Vol. 12 No. 3, pp. 295-297, doi: 10.1097/00001648-200105000-00006.
  8. Greenland, S., Senn, S.J., Rothman, K.J., Carlin, J.B., Poole, C., Goodman, S.N. and Altman, D.G. (2016), “Statistical tests, P-values, confidence intervals, and power: a guide to misinterpretations”, European Journal of Epidemiology, Vol. 31 No. 4, pp. 337-350, doi: 10.1007/s10654-016-0149-3.
  9. Harrell, F. (2017), “A litany of problems with p-values”, available at: https://www.fharrell.com/post/pval-litany/
  10. Lu, M. and Ishwaran, H. (2017), “A machine learning alternative to p-values”, arXiv preprint arXiv:1701.04944.
  11. Meyer, M.C. and Laud, P.W. (2002), “Predictive variable selection in generalized linear models”, Journal of the American Statistical Association, Vol. 97 No. 459, pp. 859-871, doi: 10.1198/016214502388618654.
  12. Nguyen, H.T. (2016), “On evidence measures of support for reasoning with integrated uncertainty: a lesson from the ban of p-values in statistical inference”, in Integrated Uncertainty in Knowledge Modelling and Decision Making, Springer, pp. 3-15.
  13. Romano, Y., Patterson, E. and Candes, E. (2019), “Conformalized quantile regression”, in Wallach, H., Larochelle, H., Beygelzimer, A., d’ Alché-Buc, F., Fox, E. and Garnett, R. (Eds), Advances in Neural Information Processing Systems, Curran Associates, Vol. 32.
  14. Trafimow, D., Amrhein, V., Areshenkoff, C.N., Barrera-Causil, C.J., Beh, E.J., Bilgiç, Y.K., Bono, R., Bradley, M.T., Briggs, W.M., Cepeda-Freyre, H.A. and Chaigneau, S.E. (2018), “Manipulating the alpha level cannot cure significance testing”, Frontiers in Psychology, Vol. 9, 699, pp. 1-7, doi: 10.3389/fpsyg.2018.00699.
  15. UCLA (n.d.), “UCLA advanced research computing: statistical methods and data analytics”, available at: https://stats.idre.ucla.edu/stat/data/binary.csv (accessed 10 May 2024).
  16. Wasserstein, R.L. and Lazar, N.A. (2016), “The ASA's statement on p-values: context, process, and purpose”, American Statistician, Vol. 70, pp. 129-132.
  17. William, M.B. and Hung, T.N. (2019), “Clarifying ASA's views on p-values in hypothesis testing”, Asian Journal of Economics and Banking, Vol. 3, pp. 1-16.

Related articles