We have a new paper at Computational Brain & Behavior that describes a joint deep neural network and evidence accumulation modeling approach to human decision-making with naturalistic images. Evidence accumulation models (EAM) have proven to be an invaluable tool in probing the dynamical properties of decisions over recent decades. However, much of this literature has studied decisions utilizing simple stimuli where the experimenter has perfect knowledge and control over stimulus properties. Here we develop and test a new method for studying decisions involving naturalistic stimuli (medical images in this case) where the experimenter has neither perfect knowledge nor control of the stimuli properties. The central challenge in studying such decisions is to extract useful representations of images that can be associated with accumulation or drift rates in EAMs. Here we couple a deep convolutional neural network (CNN) with the diffusion decision model (DDM) to study how expert pathologists and novices make decisions involving the classification of digital images of blood cells as either normal (Non-Blast) or cancerous (Blast). In our approach, the CNN is the basis of a function that translates each image into a drift rate for use in the DDM. Results of fitting the joint CNN-DDM model to choice and response time data demonstrates that 1) both novices and experts demonstrated substantial speed accuracy tradeoffs, 2) both were susceptible to biases introduced by the presentation of pre-stimulus probabilistic cues, and 3) experts were more adept at extracting useful information from images than novices. These results demonstrate that this is a fruitful approach to studying decisions involving complex stimuli that will open new avenues for studying questions not possible with existing methods. Furthermore, this approach is technically feasible and has the potential to be translated into other domains of decision making research.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s