Joseph Weeks (plantcoffee6)
Reproducibility is a confused terminology. In this paper, I take a fundamental view on reproducibility rooted in the scientific method. The scientific method is analysed and characterized in order to develop the terminology required to define reproducibility. Furthermore, the literature on reproducibility and replication is surveyed, and experiments are modelled as tasks and problem solving methods. Machine learning is used to exemplify the described approach. Based on the analysis, reproducibility is defined and three different degrees of reproducibility as well as four types of reproducibility are specified. This article is part of the theme issue 'Reliability and reproducibility in computational science implementing verification, validation and uncertainty quantification in silico'.With the relentless rise of computer power, there is a widespread expectation that computers can solve the most pressing problems of science, and even more besides. TBK1/IKKε-IN-5 IκB inhibitor We explore the limits of computational modelling and conclude that, in the domains of science and engineering which are relatively simple and firmly grounded in theory, these methods are indeed powerful. Even so, the availability of code, data and documentation, along with a range of techniques for validation, verification and uncertainty quantification, are essential for building trust in computer-generated findings. When it comes to complex systems in domains of science that are less firmly grounded in theory, notably biology and medicine, to say nothing of the social sciences and humanities, computers can create the illusion of objectivity, not least because the rise of big data and machine-learning pose new challenges to reproducibility, while lacking true explanatory power. We also discuss important aspects of the natural world which cannot be solved by digital means. In the long term, renewed emphasis on analogue methods will be necessary to temper the excessive faith currently placed in digital computation. This article is part of the theme issue 'Reliability and reproducibility in computational science implementing verification, validation and uncertainty quantification in silico'.Free and open source software (FOSS) is any computer program released under a licence that grants users rights to run the program for any purpose, to study it, to modify it, and to redistribute it in original or modified form. Our aim is to explore the intersection between FOSS and computational reproducibility. We begin by situating FOSS in relation to other 'open' initiatives, and specifically open science, open research, and open scholarship. In this context, we argue that anyone who actively contributes to the research process today is a computational researcher, in that they use computers to manage and store information. We then provide a primer to FOSS suitable for anyone concerned with research quality and sustainability-including researchers in any field, as well as support staff, administrators, publishers, funders, and so on. Next, we illustrate how the notions introduced in the primer apply to resources for scientific computing, with reference to the GNU Scientific Library as a case study. We conclude by discussing why the common interpretation of 'open source' as 'open code' is misplaced, and we use this example to articulate the role of FOSS in research and scholarship today. This article is part of the theme issue 'Reliability and reproducibility in computational science implementing verification, validation and uncertainty quantification in silico'.This article provides the motivation and overview of the Collective Knowledge Framework (CK or cKnowledge). The CK concept is to decompose research projects into reusable components that encapsulate research artifacts and provide unified application programming interfaces (APIs), command-line interfaces (CLIs), meta descriptions and common automation actions for related artifacts. The CK framework is used to
