Commentary on Barbey & Sloman (2007)
Abstract: 38 words
Main Text: 978 words (including footnotes)
References: 123 words
Total Text: 1210 words
One Wrong Does Not Justify Another:
Accepting Dual Processes by Fallacy of False Alternatives
Iris van Rooij
Department of Psychology
Barbey and Sloman advocate a dual-process (two-system) approach by comparing it with an alternative perspective (ecological rationality), claiming that the latter is unwarranted. Rejecting this alternative approach cannot serve as sufficient evidence for the viability of the former.
The target article's title suggests two messages to take home. Current theories of ecological rationality rest on weak grounds (we generally agree), and data patterns of base-rate neglect provide empirical support for dual-process theory (we generally disagree). Barbey and Sloman's (henceforth B&S) analysis is mistaken on two grounds.
First, they commit the fallacy of false alternatives: Demonstrating that 4 out of 5 theoretical accounts are false, does not necessarily imply the truth of the remaining one unless the list of hypotheses is exhaustive, which it is not (we label the 5 accounts considered by B&S as T1 – T5). T1–T5 do not instantiate an exclusive set of (plausible) theoretical possibilities. This becomes evident when analyzing T5 into its conjunctive parts: (1) the hypothesis that explicating nested set relations facilitates Bayesian reasoning (TNESTED), and (2) the hypothesis that the mind has a dual-process architecture with associative processing occurring in parallel to rule-based processing (TDUAL). Clearly, TNESTED and TDUAL are two distinct and separable theoretical claims (hence we already have T1–T6 different accounts, with T6 being equal to TNESTED without TDUAL). There is no reason we can see, nor do the authors provide one, that the nested set hypothesis be married specifically with a dual-process architecture of mind. It seems, a priori at least, as plausible that a single, or multi, process architecture can implement the benefits from nested set representations.
This leads directly to B&S's second fallacy: Even if the non-rejected account (nested sets) has some merits, in no way does it imply or support a dual process (two-systems) perspective. It is well-known that representation and computation can trade spaces, in the sense that computation can be facilitated, or otherwise affected, by changing between (logically equivalent) representational formats (Clark & Thornton, 1997; Marr, 1982). This general cognitive principle has been demonstrated in areas as diverse as problem solving, memory retrieval, and visual imagery. Also the cognitive facilitation afforded by Venn diagrams, and diagrams in general (Larkin and Simon, 1987), is well known (yet, unrelated to dual process theories). Framing effects in decision making also illustrate how changes in representational format affect cognitive judgments. The nested sets facilitation hypothesis, reported by B&S, seems to be yet another (potential) example.1 As such, the hypothesis, though viable, is neither novel nor surprising. Due to its generalized flavor it seems particularly ill-suited as a basis for conjecturing a particular architecture of mind: almost any architecture of mind (whether single-, dual- or multi-process; whether associative, rule-based, both or neither) could accommodate the effect.
Apparently, B&S do consider evidence in favor of the nested set hypothesis as also constituting support for the idea that human minds have a dual-process architecture. Arguing for such a general theoretical position, based on the available performance data alone, is simply trying to do impossible. This is also illustrated by Table 2 in the target article. Close inspection of the table shows that the available data cannot decide between theories that assume modular or non-modular architectures (predictions for T1 and T2 are identical), and cannot decide between theories postulating evolutionary or non-evolutionary adaptations (predictions for T3 and T4 are identical). In the same vein, the available data cannot decide between theories that postulate dual- or single-process architectures. Table 2 may seem to suggests otherwise because the predictions of T5 appear to be unique. However, it should be noted that the Table is missing a column and thus is incomplete. The authors should have included a sixth column listing predictions for T6 identical to the predictions for T5 (granting that T5 is really making the listed predictions, which seems questionable to begin with, yet is insubstantial for our claim that the reviewed findings cannot discriminate between T5 and T6). Having done so may have highlighted that the dual process assumption is superfluous in the authors' explanation of base rate neglect.
Here B&S are confronted with the fact that theoretical frameworks in science generally cannot be justified on the basis of a small set of empirical phenomena (Lakatos, 1977) . Rather, theoretical frameworks derive their explanatory power from making insightful a large corpus of seemingly unrelated findings, that would otherwise be puzzling or anomalous. B&S make no attempt to argue for the explanatory superiority of dual-process architectures (compared to other architectures of mind); and as argued above, effects of representational format (e.g. nested set relations) on cognitive processing are not puzzling in any event.
In short, B&S do not provide any argument for why support for the nested-set hypothesis constitutes evidence for dual process (two-systems) theories. The presumed superiority of dual-process architectures is presumably established by citing other authors who advocate a two system theory (e.g., Evans & Over, 1996; Kahneman & Fredrick, 2002; Sloman, 1996; Stanovich & West, 2000). Indeed, there has recently been an upsurge in theoretical frameworks alluding to the existence of two different processing systems that supposedly operate according to different rules. Recently, we (Keren & Schul 2007) have pointed out to the lack of robust and reliable evidence that would support the two system architecture of the mind. The target article seems to offer arguments that question the viability of the natural frequencies approach, and more generally the ecological rationality framework. Yet, it does not add any forceful evidence in support of the alternative favored by the authors, namely the dual process approach. The possibility that both theoretical frameworks (i.e., ecological rationality and dual processes) are undefendable, cannot be ruled out.
1. B&S's attempt to rule out the possibility that explicating nested set relation simply affords easier computation is questionable. They draw on a study asking participants to judge ease of understanding of different presentation formats. Whether participants have introspective access to the nature and efficiency of their own cognitive processes is highly doubtful (Nisbett & Wilson, 1977).
2. Certainly when the phenomenon under discussion remain controversial (Koehler, 1996) on both theoretical and empirical grounds.
Clark, A & Thornton, C. (1997). Trading spaces: Computation, representation, and the limits of uninformed learning. Behavioral and Brain Sciences, 20(1), 57–92.
Keren, G. & Schul, Y. (2007). Two is not always better than one: A critical evaluation of Two-systems theories. Manuscript under review.
Larkin, J.H. & Simon, H.A. (1987). Why a diagram is (sometimes) worth ten thousand words. Cognitive Science, 11, 65–99.
Marr, D. (1982). Vision:
A computational investigation into the human representation and processing
visual information. W. H. Freeman:
Nisbett, R. E. & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84(3), 231–259.