Category Archives: Reproducible Research

My input for the OSTP RFI on reproducibility

Until Sept 23 2014, the US Office of Science and Technology Policy in the Whitehouse was accepting comments on their “Strategy for American Innovation.” My submitted comments on one part of that RFI, section 11 follow: “11) Given recent evidence … Continue reading

Posted in Economics, Intellectual Property, Law, Open Data, Open Science, OSTP, Reproducible Research, Scientific Method | 2 Comments

Changes in the Research Process Must Come From the Scientific Community, not Federal Regulation

I wrote this piece as an invited policy article for a major journal but they declined to publish it. It’s still very much a draft and they made some suggestions, but since realistically I won’t be able to get back … Continue reading

Posted in Intellectual Property, Law, Open Data, Open Science, OSTP, Reproducible Research | 5 Comments

Data access going the way of journal article access? Insist on open data

The discussion around open access to published scientific results, the Open Access movement, is well known. The primary cause of the current situation — journal publishers owning copyright on journal articles and therefore charging for access — stems from authors … Continue reading

Posted in Intellectual Property, Law, Open Data, Open Science, Reproducible Research, Scientific Method | 6 Comments

Don’t expect computer scientists to be on top of every use that’s found for computers, including scientific investigation

Computational scientists need to understand and assert their computational needs, and see that they are met. I just read this excellent interview with Donald Knuth, inventor of TeX and the concept of literature literate programming, as well as author of … Continue reading

Posted in Open Science, Reproducible Research, Uncategorized | 3 Comments

The nature of science in 2051

Right now scientific questions are chosen for study in a largely autocratic way. Typically grants for research on particular questions come from federal funding agencies, and scientists competitively apply with the money going to the chosen researcher via a peer … Continue reading

Posted in Open Science, Reproducible Research, Statistics | 1 Comment

Regulatory steps toward open science and reproducibility: we need a science cloud

This past January Obama signed the America COMPETES Re-authorization Act. It contains two interesting sections that advance the notions of open data and the federal role in supporting online access to scientific archives: 103 and 104, which read in part: … Continue reading

Posted in Law, Open Science, OSTP, Peer Review, Reproducible Research, Scientific Method, Technology | Leave a comment

Generalize clinicaltrials.gov and register research hypotheses before analysis

Stanley Young is Director of Bioinformatics at the National Institute for Statistical Sciences, and gave a talk in 2009 on problems in modern scientific research. For example: 1 in 20 NIH-funded studies actually replicates; closed data and opacity; model selection … Continue reading

Posted in Open Science, Peer Review, Reproducible Research, Scientific Method, Statistics, Talks | Leave a comment

Open peer review of science: a possibility

The Nature journal Molecular Systems Biology published an editorial “From Bench to Website” explaining their move to a transparent system of peer review. Anonymous referee reports, editorial decisions, and author responses are published alongside the final published paper. When this … Continue reading

Posted in Open Science, Peer Review, Reproducible Research, Technology | 3 Comments

My Symposium at the AAAS Annual Meeting: The Digitization of Science

Yesterday I held a symposium at the AAAS Annual Meeting in Washington DC, called “The Digitization of Science: Reproducibility and Interdisciplinary Knowledge Transfer,” that was intended to bring attention to how massive computation is changing the practice of science, particularly … Continue reading

Posted in Conferences, Intellectual Property, Law, Open Science, Reproducible Research, Scientific Method, Software, Statistics, Talks, Technology | Leave a comment

Letter Re Software and Scientific Publications – Nature

Mark Gerstein and I penned a reaction to two pieces published in Nature News last October, “Publish your computer code: it is good enough,” by Nick Barnes and “Computational Science…. Error” by Zeeya Merali. Nature declined to publish our note … Continue reading

Posted in Open Science, Reproducible Research, Scientific Method, Software, Technology | 8 Comments

Open Data Dead on Arrival

In 1984 Karl Popper wrote a private letter to an inquirer he didn’t know, responding to enclosed interview questions. The response was subsequently published and in it he wrote, among other things, that: “Every intellectual has a very special responsibility. … Continue reading

Posted in Developing world, Intellectual Property, Open Science, Reproducible Research, Scientific Method, Software, Statistics, Technology | 6 Comments

Ars technica article on reproducibility in science

John Timmer wrote an excellent article called “Keeping computers from ending science’s reproducibility.” I’m quoted in it. Here’s an excellent follow up blog post by Grant Jacobs, “Reproducible Research and computational biology.”

Posted in Open Science, Reproducible Research, Scientific Method, Software, Statistics, Technology, Uncategorized | Tagged | Leave a comment

Code Repository for Machine Learning: mloss.org

The folks at mloss.org — Machine Leaning Open Source Software — invited a blog post on my roundtable on data and code sharing, held at Yale Law School last November. mloss.org’s philosophy is stated as: “Open source tools have recently … Continue reading

Posted in Conferences, Open Science, Reproducible Research, Software | Leave a comment

Video from "The Great Climategate Debate" held at MIT December 10, 2009

This is an excellent panel discussion regarding the leaked East Anglia docs as well as standards in science and the meaning of the scientific method. It was recorded on Dec 10, 2009, and here’s the description from the MIT World … Continue reading

Posted in Open Science, Reproducible Research, Scientific Method, Talks | Leave a comment

My answer to the Edge Annual Question 2010: How is the Internet Changing the Way You Think?

At the end of every year editors at my favorite website The Edge ask intellectuals to answer a thought-provoking question. This year it was “How is the internet changing the way you think?” My answer is posted here: http://www.edge.org/q2010/q10_15.html#stodden

Posted in Open Science, Reproducible Research, Scientific Method, shameless self-promotion, Statistics, Technology | Leave a comment

Post 3: The OSTP’s call for comments regarding Public Access Policies for Science and Technology Funding Agencies Across the Federal Government

The following comments were posted in response to the OSTP’s call as posted here: http://www.ostp.gov/galleries/default-file/RFI%20Final%20for%20FR.pdf. The first wave, comments posted here, asked for feedback on implementation issues. The second wave requested input on Features and Technology (our post is here). … Continue reading

Posted in Intellectual Property, Law, Open Science, OSTP, Reproducible Research, Scientific Method, Uncategorized | Leave a comment

Post 2: The OSTP’s call for comments regarding Public Access Policies for Science and Technology Funding Agencies Across the Federal Government

The following comments were posted in response to the second wave of the OSTP’s call as posted here: http://www.ostp.gov/galleries/default-file/RFI%20Final%20for%20FR.pdf. The first wave, comments posted here and on the OSTP site here (scroll to the second last comment), asked for feedback … Continue reading

Posted in Intellectual Property, Law, Open Science, OSTP, Reproducible Research, Scientific Method, Software, Statistics, Technology | 4 Comments

Nathan Myhrvold advocates for Reproducible Research on CNN

On yesterday’s edition of Fareed Zakaria’s GPS on CNN former Microsoft CTO and current CEO of Intellectual Ventures Nathan Myhrvold said reproducible research is an important response for climate science in the wake of Climategate, the recent file leak from … Continue reading

Posted in Intellectual Property, Open Science, Reproducible Research, Scientific Method | Leave a comment

The OSTP's call for comments regarding Public Access Policies for Science and Technology Funding Agencies Across the Federal Government

The following comments were posted in response to the OSTP’s call as posted here: http://www.ostp.gov/galleries/default-file/RFI%20Final%20for%20FR.pdf: Open access to our body of federally funded research, including not only published papers but also any supporting data and code, is imperative, not just … Continue reading

Posted in Intellectual Property, Law, Open Science, OSTP, Reproducible Research, Scientific Method, Uncategorized | 4 Comments

The Climate Modeling Leak: Code and Data Generating Published Results Must be Open and Facilitate Reproducibility

On November 20 documents including email and code spanning more than a decade were leaked from the Climatic Research Unit (CRU) at East Anglia University in the UK. The Leak Reveals a Failure of Reproducibility of Computational Results It appears … Continue reading

Posted in Law, Open Science, Reproducible Research, Scientific Method, Software, Statistics, Technology, Uncategorized | 7 Comments

Software and Intellectual Lock-in in Science

In a recent discussion with a friend, a hypothesis occurred to me: that increased levels of computation in scientific research could cause greater intellectual lock-in to particular ideas. Examining how ideas change in scientific thinking isn’t new. Thomas Kuhn for … Continue reading

Posted in Open Science, Reproducible Research, Scientific Method, Software, Technology | 2 Comments

My Interview with ITConversations on Reproducible Research

On September 30, I was interviewed by Jon Udell from ITConversations.org in his Interviews with Innovators series, on Reproducibility of Computational Science. Here’s the blurb: “If you’re a writer, a musician, or an artist, you can use Creative Commons licenses … Continue reading

Posted in Intellectual Property, Law, Open Science, Reproducible Research, Scientific Method, shameless self-promotion, Statistics, Technology | 2 Comments

Optimal Information Disclosure Levels: Data.gov and "Taleb's Criticism"

I was listening to the audio recording of last Friday’s “Scientific Data for Evidence Based Policy and Decision Making” symposium at the National Academies, and was struck by the earnest effort on the part of members of the Whitehouse to … Continue reading

Posted in Economics, Internet and Democracy, Law, Open Science, Reproducible Research, Scientific Method, Statistics, Technology | 2 Comments

What's New at Science Foo Camp 2009

SciFoo is a wonderful annual gathering of thinkers about science. It’s an unconference and people who choose to speak do so. Here’s my reaction to a couple of these talks. In Pete Worden’s discussion of modeling future climate change, I … Continue reading

Posted in Conferences, Developing world, Economics, Open Science, Reproducible Research, Statistics, Talks, Technology | Leave a comment

Wolfram|Alpha Demoed at Harvard: Limits on Human Understanding?

Yesterday Stephen Wolfram gave the first demo of Wolfram|Alpha, coming in May, what he modestly describes as a system to make our stock of human knowledge computable. It includes not just facts, but also our algorithmic knowledge. He says, “Given … Continue reading

Posted in Open Science, Reproducible Research, Statistics, Talks, Technology | 2 Comments