Scientific communities are grappling with assessing knowledge derived from computational processes: interpretability of machine learning models; generalizability of data science claims; verifiability of computational science. There are two broad categories of “knowing” at play here that distinguish approaches to understanding when a claim can be deemed a contribution to our stock of knowledge. I’m calling these categories Experiential and Inferential computational knowledge in a direct extension of general experiential and logical ways of knowing about our world.
The first general class of problems data science tries to solve are classification-type problems that appear in many online and industry applications such as image classification or recommender systems. The second is process-type problems such as understanding drug effectiveness or the impact of policy changes. The motivation of the researcher is different in these two broad problem classes. One is effectively seeking an engineering solution that makes reliable predictions or outcomes in a circumscribed context with little concern for how those predictions are made, the other is seeking a generalizable science solution where elucidating the underlying mechanisms that generate the outcome is of key interest. In the first case it isn’t a question of whether the model is right but whether it’s giving reliable outputs. In that case we’re learning by absorbing data and finding patterns, similar to learning about the world noncomputationally: learning how people you know act, what’s a broken heart, or even what happens if you slam on the car brakes. Cause and effect reasoning. Explaining why we expect a certain outcome isn’t part of the learning process, but we still feel like we know something given a set of experiences. Arguably this is the most common way we learn.
This is different to inferential or explainable knowledge where I can present the chain of logic that I purport leads to a result and you can learn why that result happens, or is true. That knowledge is directly transferrable by communication, and does not require experience (although it may rely on other knowledge elements). So, is one type of knowledge more valuable? more reliable? or to be prioritized? Possibly. My argument is that we have an analogy in computationally-enabled and data-enabled research and should expect different justifications to convince us why we should accept the result for the two different types of computational knowledge. And we should expect different generalizability for the two types of knowledge. Experiential knowledge generalizes by analogy so understanding the context under which it hold is key, inferential knowledge by explanation so understanding the reasoning that generated the result is key. So what does that mean for computationally generated knowledge? Depends which model it follows.
For inferential computational knowledge to generalize how and why the result holds is an essential part of the communication, and these aspects are in part embedded in the computational steps that generated the result. The code, the inputs (e.g. empirical data, function settings and parameters, machine state variables), expected output and expect variability of the output are unavoidable epistemological components, along with the noncomputational logical reasoning (e.g. statistical tests used). For experiential knowledge this is not necessarily the case. In theory all I need is the black box that produces the output from the inputs and a clear understanding of what outputs different types of inputs will produce (for most machine learning problems, or black box problems in general, this latter component is very hard, making misuse of black boxes almost guaranteed). What are the characteristics of the data that I need to assure before this black box can be applied?
We need to treat both types of computational knowledge differently and not tacitly assume computationally generated output from data is necessarily scientific and therefore generalizable. We need to improve both computational disclosure for computational knowledge we intend as a generalizable result (inferential knowledge) so we have a verifiable computational chain of reasoning as well as a logical chain, and improve contextual disclosure for experientially/data based computational output. Without doing so the ways of computationally knowing are ways of knowing nothing.