SciFoo is a wonderful annual gathering of thinkers about science. It’s an unconference and people who choose to speak do so. Here’s my reaction to a couple of these talks.
In Pete Worden’s discussion of modeling future climate change, I wondered about the reliability of simulation results. Worden conceded that there are several models doing the same predictions he showed, and they can give wildly opposing results. We need to develop the machinery to quantify error in simulation models just as we routinely do for conventional statistical modeling: simulation is often the only empirical tool we have for guiding policy responses to some of our most pressing issues.
But the newest I saw was Bob Metcalfe’s call for us to imagine what to do with the coming overabundance of energy. Metcalfe likened solving energy scarcity to the early days of Internet development: because of the generative design of Internet technology, we now have things that were unimagined in the early discussions, such as YouTube and online video. According to Metcalfe, we need to envision our future as including a “squanderable abundance” of energy, and use Internet lessons such as standardization and distribution of power sources to get there, rather than building for energy conservation.
Cross posted on The Edge.
Technology has a history of sweeping scientific enterprise: from Vannevar Bush’s first analog PDE calculators at MIT in the 30’s through the differential analyzers of the 50’s and 60’s to today’s unfinished transition that will end with computation as absolutely central to scientific enterprise. Now computational tools play not only the traditional role of helping scientific discovery, but of facilitating it. On July 26 I’ll be talking about changes to the scientific method that computation has brought — does reproducibility matter? is computation creating a third branch of the scientific method? — at Science 2.0 in Toronto. The conference focuses on how the Internet is changing the process of doing science: how we share code and data, and how we use new communication technologies for collaboration and work tracking. Here’s the abstract for my talk and the URL:
How Computational Science is Changing the Scientific Method
As computation becomes more pervasive in scientific research, it seems to have become a mode of discovery in itself, a “third branch” of the scientific method. Greater computation also facilitates transparency in research through the unprecedented ease of communication of the associated code and data, but typically code and data are not made available and we are missing a crucial opportunity to control for error, the central motivation of the scientific method, through reproducibility. In this talk I explore these two changes to the scientific method and present possible ways to bring reproducibility into today’ scientific endeavor. I propose a licensing structure for all components of the research, called the “Reproducible Research Standard”, to align intellectual property law with longstanding communitarian scientific norms and encourage greater error control and verifiability in computational science.