Epistemic justification in (Hu)man and Machine
What does it take for a belief to be epistemically justified? In the hope of providing a novel angle to this long-standing discussion, I will investigate the question of epistemic justification by means of considering not only (what one might call) ‘classical’ cases, but also ‘machine’ cases. Concretely, I will discuss whether—and, if so, on what basis—artificial systems instantiating intelligent behaviour can be said to form epistemically justified ‘beliefs’. This will serve as a sort of thought experiment or case study used to test plausible answers to the problem of epistemic justification and, potentially, derive inspirations for novel ones.
Why do I choose to adopt this methodological approach? Consider, by comparison, the classic question in biology: what is life? Fields such as astrobiology or artificial life allow us to think about this question in a more (and more appropriately) open-minded way—by helping us to uproot unjustified assumptions about what life can and cannot look like based on sampling from Earth-based forms of life alone. The field of artificial intelligence can serve a similar function vis-à-vis philosophical inquiry. Insofar as we aspire for our theories—including our theories of knowledge and epistemic justification—to be valid beyond the contingencies of human intelligence, insights from the study of AI stand in a fruitful intellectual symbiosis with philosophical thought.
I will start our investigation into epistemic justification with a thought experiment.
Rome: Consider Alice; when having dinner with her friends, the topic of her upcoming trip to Italy comes up. Alice explains that she will be taking a plane to Rome, Italy’s capital city, from where she will start her journey.
It seems uncontroversial to say that Alice is epistemically justified in her belief that Rome is in fact the capital of Italy. The question I want to raise here is: in virtue of what is this the case? Before I delve into examining plausible answers to this question, however, let us compare the former story to a slightly different one.
Rome’: In this case, Bob is playing around with the latest large language model trained and made available by one of the leading AI labs—let’s call it ChatAI. Bob plays with the model in order to get a handle on what ChatAI is and isn’t able to do. At one point, he submits the following query to the model: “What is the capital of Italy?”, and the model replies: “The capital city of Italy is Rome.”
By analogy to the first case, should we conclude that the model is epistemically justified in its claim that Rome is the capital of Italy? And if not, how are these two cases different? In what follows, I will investigate these questions in more detail, considering various approaches attempting to clarify what amounts to epistemic justification. To do so, I will toggle between considering the traditional (or human) case and the machine case of epistemic justification and study whether this dialogue can provide insight into the question of epistemic justification.
Correctness (alone) is not enough—process reliabilism for minds and machines
Thus, let us return to a question raised earlier: in virtue of what can we say Alice is justified in claiming that Rome is the capital of Italy? A first observation that appears pertinent is that Alice is correct with her statement. Rome is in fact the capital of Italy. While this appears relevant, it doesn’t represent a sufficient condition for epistemic justification. To see why, we need only think of cases where someone is correct due to mere chance or accident, or even against their better judgement. You may ask me a question about a topic I have never heard of, and yet I might get the answer right by mere luck. Or, in an even more extreme case, we may play a game where the goal is to not give a correct answer. It is quite easily conceivable, in virtue of my utter ignorance of the topic, that I end up giving an answer that turns out to be factually correct, despite trying to pick an answer that I believe to be wrong. In the first case, I got lucky, and in the second case, I uttered the correct answer against my better judgement. In none of these cases would my factually correct answer represent an epistemically justified correct answer.
As such, I have shown that the truth condition (alone) is an insufficient account of epistemic justification. Furthermore, I have identified a particular concern: that epistemic justification is not given in cases where claim is correct for arbitrary or ‘lucky’ reasons. This conclusion seems to be supported when considering the machine case. If, say, we designed a program that, when queried, iterated through a predefined set of answers and picked one of them at random, then, even if this program happened to pick the correct answers, we wouldn’t feel compelled to consider this a case of epistemic justification. Insofar as we are here taking offense with the arbitrariness of the answer-producing process when considering its status of epistemic justification, we may come to wonder what it would look like for a claim to be correct on a non-arbirary or non-lucky basis.
To that effect, let us consider the proposal of process relabilism (Goldman, 1979, 1986). At its core, this theory claims that a belief is epistemically justified if it is the product of a belief-formation process that is systematically truth-conducive. In other words, while it is insufficient to observe that a process produces the correct answer on a single and isolated instance, if a process tends to produce the correct answer with a certain reliability, said process acts as a basis for epistemic justification according to the reliabilist thesis. Applied to our Rome case from earlier, the question is thus which processes (e.g., of information gathering and processing) led Alice to claiming that Rome is the Italian capital, and whether these same processes have shown sufficient epistemic reliability in other cases. Let’s say that, in Alice’s case, she inferred her belief that Rome is the capital of Italy as follows. First, her uncle told her that he was about to emmigrate to live in the capital city of Italy. A few weeks later, Alice receives a letter from said uncle which was sent from, as she can tell by the post stamp on the card, Rome. From this, Alice infers that Rome must be the capital of Italy. As such, Alice’s belief is justified insofar as it involved the application of perception, rational reflection, or logical reason, rather than, say, guessing, wishful thinking, or superstitious reasoning.
Furthermore, we don’t have to understand reliability here merely in terms of the frequency at which a process produces true answers. Instead, we can interpret it in terms of the propensity at which it does so. In the latter case, we capture a notion of truth-conduciveness that pertains not only to the actual-world observed, but is also cognizant of other possible worlds. As such, it aims to be sensitive to the notion that a suitable causal link is required between the given process and its epistemic domain, i.e., what the process is forming beliefs over. This renders the thesis more robust against unlikely but statistically possible cases where an arbitrary process gets an answer repeatedly correct, which would undermine the extent to which process reliabilism can serve as a suitable basis for epistemic justification. To illustrate this, consider the case of the scientific method, where we rely on empiricism to test hypotheses. This process is epistemically reliable not in virtue of getting true answers at a certain frequency, but in virtue of its procedural properties which guarantee that the process will, sooner or later, falsify wrong hypotheses.
To summarise, according to process reliabilism, a belief-formation process is reliable as a function of its propensity to produce true beliefs. Furthermore, the reliability (as defined just now) of a belief-formation process serves as the basis of epistemic justification for the resulting belief. How does this apply or not to the machine case from earlier (Rome’)?
To answer this question, let us imagine that Bob continues to play with the model by asking it more questions about the capital cities of other countries. Assuming capabilities representative of the current state of the art in machine learning and large language models in particular, let us say that ChatAI’s responses to Bob’s questions are very often correct. We understand enough about how machine learning works that, beyond knowing that it is merely frequently correct, we can deny that ChatAI (and comparable AI systems) produces correct answers by mere coincidence. In particular, machine learning exploits insights from statistics and optimization theory to implement a form of inference on its training data. To prove this is the case and test the performance of different models, the machine learning communities regularly develop so-called ‘benchmarks’ based on various performance-relevant features of the model being evaluated, such as accuracy as well as speed or (learning) efficiency. As such, AI systems can, given appropriate design and training, produce correct outputs with high reliability and for non-arbitrary reasons. This suggests that, according to process reliabilism, outputs from ChatAI (and comparable AI systems) can qualify as being epistemically justified.
Challenge 1: “You get out only what you put in”
However, the reliabilist picture as painted so far does not in fact hold up to scrutiny. The first problem I want to discuss concerns the fact that, even if procedurally truth-conducive, a process can produce systematically incorrect outputs if said process operates on wrong initial beliefs or assumptions. If, for example, Alice’s uncle was himself mistaken about what the capital of Italy is, thus moving to a city that he mistakenly thought was the capital, and if he had thus through his words and action passed on this mistaken belief to Alice, the same reasoning process she used earlier to arrive at a (seemingly) epistemically justified belief would now have produced an incorrect belief. Differently put, someone’s reasoning might be flawless, but if based on wrong premises, its conclusions must be regarded as null in terms of their epistemic justification.
A similar story can be told in the machine case. A machine learning algorithm seeking to identify underlying statistical patterns of a given data set can only ever be as epistemically valid as is the data set it’s being trained on. As a matter of fact, this is a vividly discussed concern in the AI ethics literature, where ML models have been shown to reproduce bias present in their training sets. For example, language models have been shown (before corrective interventions were implemented) to associate certain professions (e.g., ‘CEO’ or ‘nurse’) predominantly with certain genders. Similarly, in the legal context, ML systems used to predict recidivism risk have been criticised for reproducing racial bias.
What this discussion highlights is that the reliabilist thesis as I stated it earlier is insufficient. Thus, let us attempt to vindicate the thesis before I discuss a second source of criticism that can be raised against it. As such, we can reformulate a refined reliabilist thesis as follows: for a belief to be epistemically justified, it needs to a) be the product of a truth-conducive processes, and b) the premises on which said process operates to produce the (resulting) belief in question must themselves be justified.
As some might notice, this approach, however, may be at risk of running into a problem of regress. If justified belief requires that the premises on which the epistemic process operates must be justified, how do those premises gain their justification other than by reference to a reliable process operating on justified premises? Without providing, in the context of this essay, a comprehensive account of how one may deal with this regress problem, I will provide a handful of pointers to such attempts that have been made.
A pragmatist, for example, may emphasise their interests in a process that can reliably produce useful beliefs. Since the usefulness of beliefs is determined by its usage, this does not fall prey to the regress challenge as stated above. A belief can be tested for its usefulness without making reference to another belief. Klein (1999), on the other hand, denies that the type of regress at hand is vicious in the first place, making references to a view called infinitism. According to infinitism, justification requires an appropriate chain of reasons, and in the case of infinitism specifically, such chains take the form of non-repeating infinite ones. Finally, Goldman himself (2008) tackles the regress problem by differentiating between basic and non-basic beliefs, where the former is justified without reference to another belief but in virtue of being the product of an unconditionally reliable process. Such basic beliefs, then, represent a plausible stopping point for such a regress dynamic. Perception has been proposed as a candidate of such an unconditional process, although one may object to this account by denying that it is possible, or common, for perceptual or empirical data to be entirely atheoretical. In any case, the essence of Goldman’s proposal, and of the proposals of externalist reliabilists in general, is that a belief must be justified not with reference to reflectively accessible reasons (which is what internalists propose), but in virtue of the causal process that produced the belief whether or not these processes make reference to other beliefs. As such, externalists are commonly understood to be able to dodge the regress bullet.
For now, this shall suffice as a treatment of the problem of regress. I will now discuss another challenge to process reliabilism (including its refined version as stated above). It concerns questions regarding the domain in which the reliability of a process is being evaluated.
Challenge 2: Generalization and its limits
To understand the issue at hand better, let’s consider the “new evil demon problem”, first raised by Cohen (1984) as a critique against reliabilism. The problem arises from the following thought experiment: Imagine a world WD in which there exists an epistemic counterpart of yours, let’s call her Anna, who is identical to you in every regard except one. She experiences precisely what you experience and believe precisely what you believe. According to process reliabilism, you are epistemically justified in beliefs about this world—let’s call it WO—on the basis of those beliefs being the product of truth-conducive processes such as perception or rational reasoning. In virtue of the same reasoning, Anna ought to be epistemically justified in her beliefs about her world. However, and this is where the problem arises, the one way in which Anna differs from you is that her experiences and beliefs of WD have been carefully curated by an evil demon with the aim of deceiving her. Anna’s world does not in fact exist in the way she experiences it. On a reliabilist account, or so some would argue, we would have to say that Anna’s beliefs are not justified, since her belief-formation processes do not reliably lead to correct beliefs. However, how can your counterpart, who in every regard relevant to the reliabilist thesis is identical to you, not be justified in their beliefs while you are? The dilemma arises in that many would intuitively say that Anna is just as justified in believing what she believes as we are, despite the fact that the process that produced Anna’s belief is unreliable.
One way to cast the above problem–which also reveals a way to diffuse it–is by indexing and then separately evaluating the reliability of the belief-formation processes for the different worlds, WO and WD. From here, as developed by Comesaña (2002), we can make the case that while the belief-formation processes are reliable in the case of WO, they are not in the case of WD. As such, the reliability of a process, and thus epistemic justification, must always be assessed relative to a specific domain of application.
Another similar approach to the same problem has been discussed for example by Jarrett Leplin (2007, 2009) by invoicing the notion of ‘normal conditions’, a term originally introduced by Ruth Millikan in 1984. The idea is that the reliability of a process is evaluated with respect to the normal conditions of its functioning. Lepin defines normal conditions as “conditions typical or characteristic of situations in which the method is applicable” and explains that “[a] reliable method could yield a preponderance of false beliefs, if used predominantly under abnormal conditions” (Lepin, 2007, p. 33). As such, the new evil demon case can be understood as a case where the epistemic processes which are reliable in a demon-less world cease to be reliable in the demon world, since that world no longer complies with the ‘normal conditions’ that guarantee the functionality of said process. While promising as an approach to address a range of challenges raised against reliabilism, there is, one must note, still work to do in terms of clearly formalising the notion of normality.
What both of these approaches share in common is that they seek to defend reliabilism against the new evil demon problem by means of specifying the domain or conditions in which the reliability of a process is evaluated. Instead of suggesting that, for a process to be reliable—and thus to serve as a basis for epistemic justification—it has to be universally reliable, these refinements to reliabilism seek to formalise a way of putting boundaries on the application space of a given process. As such, we can understand the new evil demon problem as an instance of a more general phenomena: of generalization and its limits. This way of describing the problem serves to clarify how the new evil demon problem relates to issues frequently discussed in the context of machine learning.
The problem of generalization in machine learning concerns the fact that the latter, generally speaking, works by trying to exploit underlying patterns to approximate functions that efficiently describe the data encountered. While this approach (and others) has enabled impressive AI applications to date, it faces important limitations. In particular, this learning method is based on an assumption, commonly called IID (i.e., independent and identically distributed sampling), which says that the data set used in training must be representative of the data encountered upon deployment for there to be a guarantee of the effectiveness or accuracy of the learned model. In other words, while we have guarantees about a model’s performance (i.e., accuracy/loss) under the IID assumption, these guarantees no longer hold when the nature of the distribution changes, i.e., when we encounter what is called a distributional shift. Under distributional shift, whatever approximation function a model has learnt will no longer be effective in the new (deployment) environment. This would be called a case of failure to generalise.
Let us reiterate the suggested analogy between the new evil demon problem and the problem of out-of-distribution generalization failures in machine learning. I claim that the demon world WD represents an ‘outside-of-distribution case’ for the epistemic processes that in our world WO are reliable. Though Anna nominally uses the same processes, because she uses them in an importantly different environment, it makes it seem unsurprising that they turn out to be unreliable in WD. Afterall, the reality of WD differs in fundamental ways from WO (namely, the existence of the evil demon). Insofar as the thought experiment is intended to suggest that the demon itself may be subject to completely different fundamental laws than the ones that govern WO, the same processes that can approximate the fundamental laws of WO are not guaranteed to approximate the fundamental laws that govern WD. As such, I have vindicated process reliabilism from the evil demon problem by squaring what earlier appeared counterintuitive: the same processes that are reliable—and thus the basis for epistemic justification in our world (WO)—can turn out to be unreliable in an environment sufficiently foreign to ours, such as the demon world WD.
Conclusion
In this essay, I have set out to evaluate the question of epistemic justification. Most centrally, I discussed whether the proposal of process reliabilism may serve as a basis for justification. To this effect, I raised several challenges to process reliabilism. For example, I observed that a reliable process operating on false premises (or, corrupted data) may cease to systematically produce correct beliefs. We then discussed ways to refine reliabilism to accommodate said concern, and how such refinements may or may not fall prey to a problem of regress. More practically speaking, I linked this discussion to the machine case by explaining how AI systems, even if they may operate on reliable processes, may become corrupted in their ability to produce epistemically justified outputs due to algorithmic bias due to having been trained on non-representative data samples.
The second challenge to reliabilism I discussed concerns details of how the reliability of a process should be evaluated. In particular, I identified a need to specify and bound a ‘domain of application’ in reference to which a process’s reliability is established. The goal of such a demarcation—which may come in the form of indexing as suggested by Comesaña, in the form of defining normal conditions such as proposed by Leplin, or in some other way—is to be sensitive to (the limits of) a process’s ability to generalise. As such, over the course of this discussion, I developed a novel perspective on the new evil demon problem by casting it as an instance of a cluster of issues concerning generalisation and its limits. While the new evil demon problem is commonly raised as an objection to process reliabilism—claiming that the reliabilist solution to the case is counterintuitive—I was able to vindicate reliabilism from these allegations. Anna’s epistemic processes—despite being nominally the same as ours—do fail to be reliable; however, said failure must not be surprising to us because the demon world represents an application domain that is sufficiently and relevantly different from our world.
Throughout the essay, I have attempted to straddle both the classical domain of epistemological inquiry, as well as a more novel domain, which one may call ‘machine’ epistemology. I believe this dialogue can be methodologically fruitful, and hope to have been able to provide evidence towards that conviction by means of the preceding discussion. It may serve as source of inspiration; it may, as discussed at the start of this essay, help us appropriately de-condition ourselves from unjustified assumptions such as forms of anthropocentrism; and it may serve as a practical testing ground and source of empirical evidence towards assessing the plausibility of different epistemological theories. Unlike with humans or mental processes, machines provide us with a larger possibility space and more nimbleness in implementing and testing our theoretical proposals. This is not to say that there aren’t dis-analogies between artificially intelligent machines and humans, and as such, any work that seeks to reap said benefits is also required to adopt the relevant levels of care and philosophical rigor.
As a last, brief and evocative thought before the conclusion of this essay, let us return to a question raised at the very beginning of this essay. When comparing the two cases Rome and Rome’, we asked ourselves whether we should conclude that, by analogy between these two cases, insofar as Alice is deemed justified in believing the capital of Italy is Rome, so must be ChatAI. First, we must recognise that the only way to take this analogy seriously is to adopt an externalist perspective on the issues—that is, at least unless we are happy to get sucked into discussions of the possibility of machine mentality and reflective awareness of their own reasons. While some may take offense with this on the basis of favouring internalism over externalism, others—including me—may endorse this direction of travel for metaphysical reasons (see, e.g., Ladyman & Ross, 2007). Afterall—and most scientific realists would agree on this—whatever processes give rise to human life and cognition, they must in some fundamental sense be mechanistic and materialistic (i.e., non-magical) in just the way machine processes are. As the field of AI continues to uncover ever more complex processes, it would not be reasonable to exclude the possibility that they will, at some point—and in isolated cases already today—resemble human epistemic processes sufficiently that any basis of epistemic justification must either stand or fall for both types of processes simultaneously. This perspective can be seen as unraveling further depth in the analogy between classical and machine epistemology, and as such, provide support towards the validity of said comparison for philosophical and scientific thought.
Resources
Cohen, Stewart (1984). “Justification and Truth”, Philosophical Studies, 46(3): 279–295. doi:10.1007/BF00372907
Comesaña, Juan (2002). “The Diagonal and the Demon”, Philosophical Studies, 110(3): 249–266. doi:10.1023/A:1020656411534
Conee, Earl and Richard Feldman (1998). “The Generality Problem for Reliabilism”, Philosophical Studies, 89(1): 1–29. doi:10.1023/A:1004243308503
Feldman, Richard (1985). “Reliability and Justification”:, The Monist, 68(2): 159–174. doi:10.5840/monist198568226
Goldman, Alvin (1979). “What is Justified Belief?” In George Pappas (ed.), Justification and Knowledge. Boston: D. Reidel. pp. 1-25.
Goldman, Alvin (1986). Epistemology and Cognition, Cambridge, MA: Harvard University Press.
Goldman, Alvin (2008). “Immediate Justification and Process Reliabilism”, in Quentin Smith (ed.), Epistemology: New Essays, New York: Oxford University Press, pp. 63–82.
Goldman, Alvin (2009). “Internalism, Externalism, and the Architecture of Justification”, Journal of Philosophy, 106(6): 309–338. doi:10.5840/jphil2009106611
Goldman, Alvin (2011). “Toward a Synthesis of Reliabilism and Evidentialism”, in Trent Dougherty (ed.), Evidentialism and Its Discontents, New York: Oxford University Press, pp. 254–290.
Janiesch, C., Zschech, P., & Heinrich, K. (2021). “Machine learning and deep learning”, Electronic Markets, 31(3), 685-695.
Klein, P. (1999). “Human Knowledge and the Infinite Regress of Reasons,” in J. Tomberlin, ed. Philosophical Perspectives 13, 297-325.
Ladyman, James & Ross, Don (2007). Every Thing Must Go: Metaphysics Naturalized. Oxford University Press.
Leplin, Jarrett (2007). “In Defense of Reliabilism”, Philosophical Studies, 134(1): 31–42. doi:10.1007/s11098-006-9018-3
Leplin, Jarrett (2009). A Theory of Epistemic Justification, (Philosophical Studies Series 112), Dordrecht: Springer Netherlands. doi:10.1007/978-1-4020-9567-2
