Lawrence Barsalou is a psychologist and a cognitive scientist who worked on perception, memory, and language. One of his most cited works is Perceptual Symbol Systems (1999). In this section, I’m summarizing some of the main concepts in that paper, and reviewing the possibility of using concepts such as simulations to represent cognitive processes, or aspects of them. Further, the aim of this is to contribute into a more comprehensive view of cognitive complexity. This section also includes a review of other concepts of Barsalou such as grounded cognition and ad hoc categories that are explained in other papers of him.
Barsalou originally proposed perceptual symbols as a system to represent what we perceive and think of. Perceptual symbols are modal symbols unlike letters, they rather have features similar to the perceptual state that produced them. It is possible to say that they are closer to emojis than to letters and words (amodal symbols). The main motivation behind creating a modal system is because amodal systems are arbitrary. There’s nothing in terms of perception or cognition which can tell us to represent a seat as a chair (in English) or a silla (in Spanish).
Amodal vs modal symbols systems
Views from early thinkers such as Hume, Luke and Berkeley all hypothesized that thoughts are represented visually in the brain. However, this model started to fall out of interest later in favour of modal systems in the 20th century until Barsalou and others introduced their newer versions of it. The general concept of modal symbols wasn’t adopted only by Barsalou but also by Fodor and others[1]. Varieties of this idea focus on different aspects and assume different rules for systems that organize them in our minds.
Other reasons to favour the modal representations or to unfavour the amodal representations is the ability to use modal representations in finding relationships between objects such as finding similarity. A chair and a table have many similarities if they are viewed visually, but their words has nothing in common apart from what we attribute to them in our cognition. Barsalou shares multiple arguments against the amodal systems including the challenges that neuroscience pose against it.
Views to how concepts are represented in the brain inspire wider representational languages that are common today such as semantic nets, feature lists and connectionism. Equally, we are attempting to create a representational language that can serve the purpose of analysing the way we process complexity in cognitive processes. Representational languages can help to understand that level of higher cognitive capabilities that is derived by interactions between functions such as language, knowledge and memory.
Perceptual Symbols
It’s important to highlight what perceptual symbols construct mean for Barsalou and what it doesn’t. Perceptual symbols are not just emojis or images stored in the brain, and also, they are not amodal symbols such as words. What are they?
Barsalou assumed the existence of Perceptual Symbols as records of the neural states that underlie perception. It’s a layer that would function unconsciously in our thinking, as their basic definitions is within the neural level, something we can’t be aware of. He assumed that the perceptual symbols don’t contain a whole representation of the brain state but rather a schematic aspect of the brain states. Perceptual symbols are dynamic, which means that they depend on the context they’d come with (similar to words that have different meaning based on the position of speech). Despite having patterns, such as the tiger or the zebra strips, the features of perceptual symbols aren’t quantitative. We can recognize a tiger by the pattern but not to remember the number of black strips.
The next important construct that Barsalou introduces is simulators. The perceptual symbols system extracts some parts of the elements from perception schematically and integrates them into simulators. Therefore, the system isn’t a recording system, it’s rather a system of schematic features extraction.
On the other hand, Perceptual Symbols are not like physical pictures. They are not mental images and they don’t look like any form of conscious subjective experience. They’re not a record of everything in a specific brain state that underlies the perception of something.
Perceptual symbols are not discrete. They are rather attractors in a connectionist network, i.e they depend on the context. Important to mention also that they’re not perfectly accurate. They are “partial, sketchy, and never complete”. They are also biased and distorted
Perceptual symbols are organized within simulators so they can be used to construct specific simulations of an entity in its presence or absence (like imagining a car). Introspection is when we represent something in its absence. Important research on introspection shows that it can be viewed from different cultures in the same way. Cross-cultural research shows that a range of verbs such as see, reason, know, guess, and compare are organized within the same dimensions of perceptual / conceptual, certain / uncertain and creative / noncreative.
Simulators and Simulations
As mentioned earlier, Barsalou organizes the perceptual symbols into simulators. It’s the construct that represents cognitive systems ability to construct simulations of an entity or an event in its absence. He assumes that simulators contain a frame that integrates symbols across category instances. Each frame can be consisted of a set of simulations.
Simulators don’t produce complete simulations, but rather partial and sketchy ones. So the process isn’t a typical template matching. It never extracts all the information or matches using a complete set of information. Simulations could also be biased and distorted. Barsalou attributes that to genetic constraint on “processing of space, objects, movement, and emotion”. As there is also “Mechanisms with strong genetic constraints almost certainly play central roles in establishing, maintaining, and running simulators.”. Barsalou cites studies about perception and forming knowledge in infants to explain this genetic difference in the way we make simulations (or our simulators do).
We start developing our simulators since childhood. Simulators develop aspects of sensory experience, proprioception (the body’s ability to sense its position and movement in space), and introspection (compare, memory, happy, hungry). They are also behind our capabilities of making categories and categorizing things. Abstract concepts are based on complex simulations that integrate both physical and introspective events. The primary goal of human learning according to Barsalou is to establish simulators.
What if our simulators fail? False propositions which can happen also when we experience negation or surprise by finding the lack of match between something and our simulation of it. Barsalou finds anger as one other reaction we have towards false simulations.
Selective attention extracts symbols initially from perception, then simulators in long-term memory in turn control our attention. To understand where does this happen, simulations are created in short term memory while simulators reside in the long term memory.
Language
According to Barsalou, linguistic symbols get associated with their perceptual symbols within the same process in which they get integrated into simulators. Words associated with simulators provide linguistic control over the process of forming simulations. Selective attention on words integrate with schematic memories from the perceptual states. Words, hence are no more than a part or an aspect of the perceptual symbol. Barsalou explains:
“Whereas “car” becomes linked to the entire simulator for car, “trunk” becomes linked to one of its subregions. Simulators for words also become associated with other aspects of simulations, including surface properties (e.g., “red”), manners (e.g., “rapidly”), relations (e.g., “above”), and so forth. Within the simulator for a concept, large numbers of simulators for words become associated with its various aspects to produce a semantic field that mirrors the underlying conceptual field”
Concepts and categories
Barsalou defines concepts, an equivalent to a simulator, as the knowledge and accompanying processes that allow an individual to represent some kind of entity or event adequately. Knowledge is represented in the long term memory, where Barsalou assumes simulators are stored, and the short term where simulations are created. Therefore, concepts also are represented in memory, but there he defines temporary representations or permanent knowledge. Not just concepts, but all the functions related to learning and skill acquisition are modelled in a similar way.
Categories are also based on knowledge. Categorization involves associating knowledge with an entity and its structure, history, and behaviour based on what was already associated with the category.
Simulations can have some variance based on many things including the viewpoint. Barsalou gives an example of seeing something like a computer from front or from back, you can see different things and create different simulations based on where are looking from. Assuming that concepts are equivalent to simulators, that means concepts can vary based on the variance of the produced simulations. Some concepts can have high inter-personal variance, which makes them variable concepts or unstable concepts.
Barsalou highlights the importance of motor-sensory parts of the brain in the categorization process. He provides examples such as “damage to the visual system disrupts the conceptual processing of categories whose exemplars are primarily processed visually, such as birds”. Categorization depends on familiarity and novelty of what we experience through our sensory-motor part, but then successful categorization “stores more simulations of the entities categorized” according to Barsalou. Knowledge about categories get enriched thus. Barsalou introduced the concept of grounded cognition to highlight the idea that cognitive processes are deeply rooted in body and the social experience.
Categories for Barsalou are dynamic, and they are based on the perceptual symbols. That means the categorization of an entity happens in the same way the entity was perceived. To categorize something as a chair, we approximate the actual perceptions of chairs. Ad hoc categories of Barsalou represent an example of how dynamic his categories are. They represent temporary, goal-driven categories that people construct spontaneously in specific contexts, rather than relying on pre-existing, stable categories. Ad hoc categories can be a list of the things in the fridge or in the garage rather than a solid category like plants or furniture.
__________________________
[1] Adams, F. & Aizawa, K. (1994) Fodorian semantics. In: Mental representation, ed. S. Stich & T. Warfield.M Blackwell