Unpacking AI's Vision: Why Brain-Like CNNs Still Struggle with Human Texture Perception

The Enigma of AI and Human Texture Perception

      Artificial intelligence has made astounding progress in understanding the visual world, particularly in tasks like object recognition, where sophisticated models can identify countless items with remarkable accuracy. However, a recent academic paper delves into a more nuanced aspect of AI vision: how well these advanced models "see" textures compared to humans. The research, titled "Perceptual misalignment of texture representations in convolutional neural networks" by de Paolis et al., uncovers a surprising disconnect: even AI models considered highly "brain-like" in their ability to recognize objects do not necessarily align with human texture perception. This finding opens critical questions about the fundamental mechanisms AI vision systems employ and points to an underexplored path for their improvement.

      Convolutional Neural Networks (CNNs) are the backbone of modern computer vision, designed to mimic the hierarchical processing of the mammalian visual system. For years, these networks have been instrumental in pushing the boundaries of what machines can see and understand. Yet, as the study highlighted in this paper demonstrates, the journey towards truly human-like visual intelligence is still underway. Understanding this misalignment is crucial for developing AI that can tackle the full spectrum of visual complexities in real-world applications, from detailed industrial quality control to sophisticated security systems.

Understanding Texture: From Julesz to AI's Gram Matrices

      The human ability to perceive texture has fascinated scientists for decades. A foundational concept comes from Hungarian psychologist Béla Julesz, who proposed that human texture perception is largely based on local correlations—how adjacent visual features relate to each other within an image. This insight has guided much of the subsequent research into how our brains process visual patterns.

      In the realm of machine learning, this idea found a powerful operationalization with the work of Gatys et al. They introduced a method that uses CNNs to extract "texture representations" from images. At its core, this involves analyzing the "feature maps" generated by the intermediate layers of a CNN. These feature maps are essentially what the network "sees" at various stages of processing—from basic edges and colors in early layers to more complex patterns in deeper layers. To quantify texture, Gatys et al. employed a mathematical construct called a Gram matrix. Simply put, a Gram matrix summarizes the statistical correlations between these features across different channels within a specific layer, effectively capturing the "style" or "texture statistics" of an image while intentionally discarding precise spatial information. This approach became widely adopted in applications like neural style transfer and texture synthesis, where the goal is to generate new images that replicate the aesthetic or textural qualities of a source.

CNNs as Models of the Visual Brain: A Complex Relationship

      CNNs trained for object recognition have achieved remarkable success as computational models for the ventral visual stream—the part of the brain responsible for identifying "what" we see. Metrics like Brain-Score are used to quantify how well these AI models align with actual brain activity and behavioral responses in vision tasks. Many CNNs have shown strong predictive power for neural activity, suggesting they capture some fundamental aspects of biological vision for object recognition.

      However, this strong alignment for object recognition does not perfectly extend to all aspects of visual perception. While excellent at identifying discrete objects, CNNs have shown less predictive power for behavioral responses to individual stimuli that go beyond simple identification. Texture perception, a complex cognitive function involving the recognition of overall patterns rather than individual objects, falls into this intricate category. This naturally raised a critical question: Do CNNs that are considered "brain-like" in the context of object recognition also process and represent visual textures in a way that aligns with human perception? The answer to this question could reveal whether object recognition and texture perception rely on shared fundamental computational mechanisms within these networks, or if specialized capabilities are required.

The Surprising Disconnect: Brain-Likeness vs. Texture Alignment

      The core of the research involved a meticulous comparison. The scientists took a diverse selection of popular CNNs and evaluated them using two key metrics. First, they used Brain-Score to gauge each CNN's "brain-likeness" in object recognition. Second, to assess texture perception, they applied the Gatys method, computing Gram matrices from the CNNs' feature maps, and then compared these AI-generated texture representations against human-annotated texture classes from the Describable Texture Dataset. The goal was to see if the AI's internal texture organization mirrored how humans categorize textures.

      The findings were, surprisingly, negative. The study found no significant correlation between a CNN's performance on object recognition (as measured by Brain-Score) and its ability to represent textures in a way that aligns with human perceptual categories. In simpler terms, a CNN that was excellent at identifying objects and highly aligned with human brain activity in that task did not automatically mean it would also "see" textures in a human-like manner. This suggests a fundamental limitation in current object-recognition-trained CNNs when it comes to replicating the full richness of human texture perception. The implication is that the features and computations that make a CNN great at object recognition are not fully sufficient for modeling how humans perceive textures, hinting that texture perception might involve distinct neural mechanisms or require the integration of contextual information that current models simply don't capture.

Implications for Next-Generation AI Vision Systems

      This research highlights an "underexplored path" in AI development, emphasizing the need to move beyond object recognition as the sole benchmark for comprehensive visual intelligence. For enterprises across various industries, this has significant implications. Developing AI systems that truly understand textures can unlock new levels of precision and insight. For instance, in manufacturing, advanced texture analysis could improve quality control by detecting subtle surface defects. In healthcare, it could aid in analyzing tissue patterns for diagnostic purposes. In smart cities, it might enhance environmental monitoring by distinguishing between different types of ground cover or material degradation.

      For organizations demanding robust and nuanced visual intelligence, this finding underscores the value of custom AI solutions that can specifically address complex perceptual challenges. Companies like ARSA Technology, an AI & IoT solutions provider experienced since 2018, engineer bespoke systems that can process visual data with high accuracy and reliability. While general-purpose CNNs excel at many tasks, tailored approaches might be necessary to integrate contextual information or develop specialized texture-processing modules. This could involve deploying ARSA AI Box Series for edge-based processing where latency and data privacy are critical, or leveraging AI Video Analytics software that can be customized for specific industry needs, thereby capturing details that generic models might overlook.

Conclusion: Paving the Way for More Human-Like AI Vision

      The research on perceptual misalignment in texture representations serves as a crucial reminder that while AI has achieved impressive feats in computer vision, true human-like perception remains a complex and multifaceted goal. The findings suggest that future AI vision systems may need to incorporate mechanisms beyond those optimized solely for object recognition, potentially focusing on how contextual information is integrated into visual processing.

      As AI continues to evolve, understanding these subtleties will be vital for developing intelligent systems that not only recognize objects but also grasp the intricate details of their surface properties and overall appearance. This deeper level of visual intelligence will be indispensable for advanced applications across various industries, leading to more robust, versatile, and ultimately, more human-like AI capabilities.

      To discuss how specialized AI vision solutions can meet your enterprise's unique operational challenges, contact ARSA today for a free consultation.

      Source: de Paolis, L., Anselmi, F., Ansuini, A., & Piasini, E. (2026). Perceptual misalignment of texture representations in convolutional neural networks. arXiv preprint arXiv:2604.01341. https://arxiv.org/abs/2604.01341