Comments on: Wolfram|Alpha Demoed at Harvard: Limits on Human Understanding? http://blog.stodden.net/2009/04/29/wolframalpha-demoed-at-harvard-limits-on-human-understanding/ Just another WordPress weblog Mon, 17 May 2010 01:59:26 +0000 hourly 1 http://wordpress.org/?v=3.0.1 By: vcs http://blog.stodden.net/2009/04/29/wolframalpha-demoed-at-harvard-limits-on-human-understanding/comment-page-1/#comment-573 vcs Thu, 30 Apr 2009 21:20:05 +0000 http://blog.stodden.net/?p=51#comment-573 It's something I'm thinking about - and Wolfram has made me consider boundaries to the creation of a human-accessible model of outcomes in the world (what I've always thought of as the grand goal of science). His example doesn't make an argument against reproducibility, but it suggests reproducibility in the sense of comprehensible scripts might not always be feasible. So I'm starting to ask what conditions does a testbed need to satisfy before it can be said to verify code? Another example that removes reproducibility from 'fixer of all problems in science' status (if it ever was there) is examples in climate change modeling where code is open, but models are so complex and researchers so invested in them, that well-known bugs can't be fixed. Can't in the political sense, not in the technical sense. Well, I can't wait until you get a chance to play with Wolfram|alpha either, Garrett. :) It’s something I’m thinking about – and Wolfram has made me consider boundaries to the creation of a human-accessible model of outcomes in the world (what I’ve always thought of as the grand goal of science). His example doesn’t make an argument against reproducibility, but it suggests reproducibility in the sense of comprehensible scripts might not always be feasible. So I’m starting to ask what conditions does a testbed need to satisfy before it can be said to verify code?

Another example that removes reproducibility from ‘fixer of all problems in science’ status (if it ever was there) is examples in climate change modeling where code is open, but models are so complex and researchers so invested in them, that well-known bugs can’t be fixed. Can’t in the political sense, not in the technical sense.

Well, I can’t wait until you get a chance to play with Wolfram|alpha either, Garrett. :)

]]> By: Garrett http://blog.stodden.net/2009/04/29/wolframalpha-demoed-at-harvard-limits-on-human-understanding/comment-page-1/#comment-572 Garrett Thu, 30 Apr 2009 16:18:32 +0000 http://blog.stodden.net/?p=51#comment-572 I can't wait to play with it. Also, this stood out as a good practical observation: "no one actually read the code. So now he advocates using test cases to check the code rather than inspecting the code itself." I wonder if this affects your thinking on publishing reproducible results? Would it be as, or even more, satisfying to be able to play with an algorithm's input and output than to have access to the code? I can’t wait to play with it.

Also, this stood out as a good practical observation:

“no one actually read the code. So now he advocates using test cases to check the code rather than inspecting the code itself.”

I wonder if this affects your thinking on publishing reproducible results? Would it be as, or even more, satisfying to be able to play with an algorithm’s input and output than to have access to the code?

]]>