A few years ago, a rather unusual test was administered. Flown to Madison, Wisconsin, to meet with neuroscientist Giulio Tononi, the purpose was to explore his heavily debated theory of consciousness, known as integrated information theory. A practical outcome of Tononi’s research is a device designed to detect consciousness, employed to assess whether unresponsive patients retain awareness.
Seated in a dental-style chair, electrodes were attached to my scalp to monitor electrical activity. A component resembling a garden hose and nozzle was then positioned near my head. Harmless magnetic pulses were applied to my cranium. The expectation is that a conscious brain will exhibit electrical reverberations. Conversely, if simulating sentience without genuine inner experience—a philosophical zombie—the brain’s response would be akin to a cracked bell’s dull thud.
After a couple of hours, the results confirmed my consciousness. While I harbored a strong suspicion of this already, the test provided external validation.
However, a simple binary measurement reveals nothing about the subjective qualities of conscious experience, known as qualia. These encompass sensations like the peculiar softness of stepping into slush or the predictable rhythm of a dog’s walk. This chasm between internal sensations and measurable neural signals, often termed the “hard problem of consciousness,” appears formidable. Yet, neuroscientists are increasingly refining their methods, drawing finer distinctions between sensations and signals, potentially unlocking this profound mystery.
This “structural” methodology is beginning to indicate whether individuals perceive colors, emotions, and other sensations similarly. It addresses the enduring philosophical question: Is my experience of “red” identical to yours? “We are at the conclusion of the initial phase of consciousness science and the commencement of the second,” stated Johannes Kleiner, a physicist at the University of Bamberg in Germany.
The Philosophical Roots of Structuralism in Experience
Philosophers have long been intrigued by the architecture of experience, particularly how sensations interrelate. “Structure will be central to the science of consciousness,” commented David Chalmers of New York University, who introduced the concept of the hard problem of consciousness in the 1990s. Proponents of structuralism suggest that our spectrum of experiences is holistic; each sensation is contingent upon all others. When we perceive red, it’s in contrast to other colors and distinct perceptual categories. For example, red shares similarities with orange but is divergent from blue, and fundamentally different from pain or joy. “Any individual experience depends on this totality,” explained Holger Lyre, a philosopher at the University of Magdeburg, Germany.
From introspection to Large-Scale Data: The Evolution of Methods
The endeavor to study subjective experience through objective means began with early psychological structural studies in the late 19th century, with the experimental framework remaining largely consistent. Researchers typically present participants with multiple stimuli, such as color pairs, and ask them to rate their similarity. Today’s primary advancements lie in the scale and scope of these investigations. Dedicated participants now evaluate thousands of color combinations, shapes, motion patterns, musical tones, spoken syllables, and emotions.
This abundance of quantitative data has drawn a community of primarily young, mathematically adept researchers from both within and outside the field of neuroscience. Utilizing individuals’ judgments of qualia differences, they construct classification systems for experience, often represented as abstract geometric forms. “Our method involves categorizing all possible relationships between qualia,” said Nao Tsuchiya, a psychologist at Monash University in Melbourne, Australia, and ATR Computational Neuroscience Laboratories in Kyoto, Japan. She leads a significant research initiative called the Qualia Structure Project.
Tsuchiya noted that individuals’ judgments exhibit considerable consistency. Across diverse ages and cultures, people make broadly similar assessments regarding color, sound, and other sensory qualities. In a study published the previous year, Tsuchiya and his colleagues surveyed 247 children aged 3 to 12 in Japan and 29 children aged 6 to 8 in China, alongside 84 adults, regarding their perception of color. The responses showed minimal variation. “The impact of language, culture, and development appears to be relatively small,” he observed. While distinct labels shape our understanding of the world—consider how the Tsimané people in the Bolivian Amazon use a single term for blue and green, or the ancient Greeks’ extensive vocabulary for sea shades—these cultural markers, according to Tsuchiya, do not fundamentally alter immediate sensory experiences.
Interestingly, people’s judgments often diverge from established historical structures like musical scales and color-mixing principles. Most participants do not identify tones an octave apart as identical, a common practice among musicians across cultures. Their color comparisons frequently appear internally inconsistent, not aligning neatly with the spectrum. This suggests their experienced color space extends beyond the three dimensions (redness, greenness, blueness) of the common RGB system. Tsuchiya’s experiments indicate that humans perceive at least seven dimensions of color.
Empirical Investigations of Philosophical Conundrums
The Qualia Structure Project has transformed classic philosophical thought experiments into actionable empirical research. The question of whether my “red” is the same as your “red” is being explored through studies of atypical color vision. Following an online survey, participants were invited to the lab for in-depth testing. Four participants identified as color-blind on a screening test, four indicated their color perception differed without a formal diagnosis, and three had typical color vision. Within each group, participants’ color similarity judgments were consistent, but discrepancies arose between groups. Geometric analysis simplified the data into distinct shapes. The shape corresponding to the second group—those who self-identified as having different color vision—displayed characteristics of the shapes from the other groups, creating an “intermediate shape.”
From a structural perspective, an individual’s perception of “red” is defined by its position within this shape. Thus, individuals within the same group could assert, “My red is the same as your red.” However, what constitutes “red” for one group might effectively be perceived as green by another. Tsuchiya described the intermediate second group as a “bridge” between atypical and typical color perception, capable of comprehending both.
The project is also applying its methods to other sensory experiences. To investigate emotions, participants are shown pairs of videos designed to elicit specific feelings, and then asked to compare their subjective responses. Tsuchiya reported that individuals with alexithymia, a condition characterized by difficulty expressing emotions, make distinctions similar to others. This suggests that the inability to articulate emotions does not necessarily equate to an absence of feeling.
The Periodic Table of Experience and Its Implications
In the 19th century, structural psychologists drew analogies between their analyses and chemistry, suggesting that complex mental states could be parsed into fundamental units much like chemical compounds are broken down into elements. Adopting this metaphor, Tsuchiya aims to organize qualia into a periodic table, with designated places for color, pitch, and other sensory inputs.
He acknowledges this analogy is imperfect. The power and name of chemistry’s periodic table stem from repeating patterns of properties, a feature Tsuchiya does not observe among qualia. Instead, he believes their distinct structures are why sight and sound feel so different. However, Tsuchiya speculates that given the holistic nature of experience, various qualia might share underlying commonalities. “Perhaps there is some form of underlying similarity across different modalities,” he suggested. “It must be the case, I believe.”
By segmenting experience into these categories, researchers gain novel avenues for testing consciousness theories. Moving beyond simple consciousness detectors, they can now seek patterns of brain activity that align with the mapped structures of experience. “We need to ascertain precisely what physically causes something to feel red, or blue, or to be painful or joyful,” stated João Pedro Parreira Rodrigues at the Einstein Center for Neurosciences in Berlin.
Neuroscientists have historically used similarity judgments to pinpoint brain functions, such as specific areas within the visual cortex responsible for color differentiation. “The signals emanating from this brain region, when I analyze collections of stimuli, exhibit the same similarity structure as those derived from behavior,” explained neuroscientist Brian Wandell from Stanford University in California. Tsuchiya’s project is currently expanding this type of research. The objective is to employ the structure of qualia to identify relevant brain activity, measured via fMRI, and then compare these findings against the predictions of various consciousness theories.
Testing the Boundaries: Consciousness and Unconscious Perception
Lucia Melloni, a neuroscientist at Ruhr University Bochum in Germany, expressed general approval of the project but also voiced specific concerns. Tsuchiya and his colleagues collect data by presenting participants with two colors or other stimuli, clearing the screen, and then asking them to rate the similarity on a scale of 1 to 8. This process requires participants to form a judgment, retain it in memory, and assign a numerical value. Melloni worries that this method might inadvertently be testing memory more than perception: “I wonder if he isn’t just testing memory.”
Collaborating with Parreira and Zefan Zheng at Ruhr, Melloni is conducting a foundational reality check on the structural approach. While Tsuchiya asks participants to compare two stimuli they are consciously aware of, her team elicits comparisons of stimuli that participants are not consciously perceiving.
This premise sounds contradictory by definition. How can the brain compare elements it is not consciously registering? Intrigued, I volunteered to participate in the experiment. Parreira guided me through the software installation, which resembled a rudimentary 1980s arcade game. A colored circle flickered, followed by a colored ring; at times, only a ring appeared. My task was to press the left arrow if the ring was red and the right arrow if it was green. This continued with various colors for 40 minutes.
Initially, I believed the circle appeared only half the time. However, Zheng and Parreira clarified that a circle was always presented. If I failed to perceive it, the subsequent rapid appearance of the ring likely confused my visual cortex, preventing the initial stimulus from entering conscious awareness. By manipulating the timing, the experiment controlled which stimuli I consciously registered, allowing for a comparison of my responses.
Even when the circle bypassed conscious detection, it exerted a subtle influence: it primed my brain. When a green circle preceded a green ring, my reaction time was consistently about 50 milliseconds faster than when the preceding circle was red. This indicated that my unconscious mind was actively making color-similarity judgments. Although I couldn’t articulate these judgments, my response times revealed them.
The experiment revealed a significant divergence between my conscious and unconscious color structures. My conscious perception arranged colors in a clear sequence from green to blue, purple, and red. My unconscious perception, however, was akin to a young child scattering crayons, lacking any discernible order. “We found that there is no unconscious color space at all,” stated Zheng. This outcome supports the structuralist perspective. Zheng cautions that the experiment is preliminary and requires further validation, but it suggests that structure effectively distinguishes conscious from unconscious perception. Consequently, it could serve as a tool for identifying markers of consciousness within brain activity. “The structuralist turn represents a highly promising development,” he remarked.
Addressing the Hard Problem of Consciousness Through Structure
Ultimately, Tsuchiya’s goal extends beyond gathering evidence for specific consciousness theories; he aims to confront the hard problem of consciousness directly. He believes that the structure of qualia holds the foundational answers. One way to articulate the hard problem is to suggest that experiences possess an inherent, unanalyzable quality, such as redness or joyfulness. Redness, for instance, appears to possess a distinct subjective quality independent of any associated concepts. “Consider red while consciously disregarding all its relations,” proposed philosopher Hedda Hassel Mørch at the University of Inland Norway. “There is clearly something more to contemplate.”
However, science fundamentally deals with relationships, not unanalyzable qualities. Scientific descriptions, including mathematical equations, are inherently relational. “Most scientific descriptions are formulated in structural terms,” observed Kristjan Loorits, a philosopher at the University of Helsinki in Finland.
Tsuchiya posits that experiences themselves may be entirely structural, lacking any intrinsic qualities. This perspective aligns with Buddhist philosophy and integrated information theory, which equates consciousness with the causal architecture of information-processing networks. “Experience is entirely about relationships,” he stated. What seems like an intrinsic property could simply be a complex network of these connections. If this is the case, such experiences could be described by equations or other mathematical constructs. Tsuchiya has turned to category theory, a branch of mathematics, for a suitable descriptive language. This approach would enable science to explain experience, thereby resolving the hard problem into a tractable issue.
Most philosophers who generally support structuralism do not venture this far. Nevertheless, they agree that experience is, in some manner, analysable, and the central question revolves around why we perceive it as inherently unanalyzable. Ron Chrisley, a cognitive scientist at the University of Sussex, UK, suggests a potential neural mechanism for this impression of intrinsic qualities. The brain’s constant self-monitoring could lead to a recursive loop of self-doubt. “Yes, I believe there is a plate on the table, because it appears to me as this oval shape with this shading,” he explained. “But why do you believe there is that oval shape and that shading? If it always has to provide an answer, it will become trapped in an infinite regression of justification.” At a certain point, the brain must cease analysis and accept its experiences as unanalyzable.
Kristjan Loorits views the perceived unanalyzability of qualia as a significant aspect of human psychology. Masterful works of art often defy expectations, evoking awe. “It is beautiful, and I cannot articulate why,” he said. However, this state is often temporary. Through discussion with peers and engagement with critical analysis, we develop the vocabulary to deconstruct our feelings. “Beauty that seems unanalyzable to me now may become analysable for me,” he added. While experiences are purely structural from an objective, third-person perspective, they can be perceived as unanalyzable from a subjective, first-person viewpoint until a critical distance is adopted, allowing for a structural understanding.
Whether sublime beauty is merely one among innumerable qualia amenable to Tsuchiya’s methodologies remains to be determined. Nevertheless, these efforts to map the interconnections among all forms of experience will undoubtedly reshape our self-perception. If experiences are indeed relative to one another, then altering one’s view of a single element can transform the perception of everything else, according to Lyre. Individuals with synaesthesia, who might experience sensory crossovers like tasting words or smelling colors, perceive these connections directly. However, all humans make such associations. “To a certain extent, we are all synaesthetes,” he concluded.