Validating technique in psycohology
The first section of the article (section Replication) describes in brief what counts as a replication.
In section Simulating Replicability via Cross-Validation Techniques, we introduce the concept of cross-validation and how this technique can be utilized for establishing replicability. Pub Med Abstract | Cross Ref Full Text | Google Scholar © 2018 Koul, Becchio and Cavallo.
A systematic study designed to assess the reproducibility of psychological science (Open Science Collaboration, 2015) found that the mean effect size of the replicated studies was half of that of the originally conducted studies, and, even more strikingly, only 36% of replication studies had statistically significant results.
To counter the concerns generated by these results and improve the quality and credibility of the psychological literature, two sets of strategies have been identified: (i) improvement in research methodologies; (ii) promotion of replication attempts.
Direct replications are replication attempts that aim at reproducing the exact effects as obtained by a previous study, incorporating the exact experimental conditions.
At the same time, it increases the confidence that the effects obtained in a specific study will be replicated, instantiating a simulated replication of the original study.
Unfortunately, replication attempts are not always feasible.
For example, they may not be feasible for large clinical-epidemiological studies.
When due to practical or methodological constraints direct replication and conceptual replication are not feasible or difficult to perform, simulated replication—we contend—provides an alternative approach to put the replicability of research findings to the test.
Simulated replicability can be implemented via procedures that repeatedly partition collected data so as to simulate replication attempts. Cross-validation entails a set of techniques that partition the dataset and repeatedly generate models and test their future predictive power (Browne, 2000).
In this way, cross-validation mimics the advantages of an independent replication with the same amount of collected data (Yarkoni and Westfall, 2017). doi: 10.1037/a0015108 Cross Ref Full Text | Google Scholar Schrouff, J., Rosa, M.