SENTIENT SOCIETIES?
I do not believe that humans are on a sustainable trajectory. Our species tends to distance itself from nature and death while we have simultaneously widened and deepened our global impact, especially over the last five hundred years. There is little check or balance on our ambitions and our powers. Yet, despite this power, we neither feed the hungry nor care for our only home: Earth. As the ancient Greeks understood, it is our greatest strength that is often our undoing; this may be our tragedy: as we understand and manipulate so much with our abstract imaginations, symbolic means of exchange and mechanical tools, we simultaneously distance ourselves from that which actually sustains us moment to moment. As we expand in so many dimensions, we lose sight of so much.
Aside from the birth of life itself, multicellularity likely ranks as the most impactful innovation over our nearly four billion years as an uninterrupted living lineage. Humans may be unique among species in many ways, however, much to our disappointment, these very coveted novelties may be our undoing; we certainly wouldn't be alone as over 99.9% of all species that have ever existed over nearly four billion years have gone extinct. It would be tragic hubris to believe we are immune from this fate. Can we improve our odds?
I believe we can by learning from life itself and its transition to multicellularity. And to do this, I believe we have to understand, implement and scale sentience via novel human institutions and modes of being.
At its core, sentience goes back to the root of life itself. It is the way in which life must navigate the intrinsically paradoxical nature of existence: explore vs exploit; self vs environment; near-term vs long-term focus; risk vs reward; freedom vs security; individual vs collective. Life must face these dichotomies moment to moment in the face of a capricious and mostly mysterious world. As Socrates suggested the answer is to be one's own devil's advocate; we have to be both the adventurer and the home builder. It is both the awareness of these disparate modes and the capacity to embody the tensions between them that is sentience and the source of creativity. As we scale in power and extent, so must sentience. Or, as with every over-extended and under-prepared endeavor, we will fail together.
As one might imagine, this project is non-trivial in its scope. Until it can be an internalized capacity we all share habitually, it must be many externalized projects that come together clumsily in starts and stops. The section on Technology Paradox below is one path through this project. An even broader view of this project can be thought of as three distinct projects, and examples of each:
Architecture - current social practice, interactive sculpture, Technology Paradox below
Sentience Research - publications, Technology Paradox below
Social Theory - publications, Technology Paradox below
At the root of every institution is the human endeavor to re-create the world; architecture embodies this habit. As argued above, this habit of re-creation must resolve the ever-greater tensions they create as they scale both spatially and in complexity; research into cognitive, semiotic and sentient dynamics explores this possibility. And Social Theory explores the human institutions that mediate between these built and biological modes where our capacity to coexist with each other and the only known biosphere will persist ... or not.
TECHNOLOGY PARADOX
Technology can be used for good or ill, but it is rarely, if ever, neutral; it is made, maintained and used to empower some, while disempowering others; all the while, we claim it is what, in fact, it is not: salvation.
An initial intent of this project is to try and understand the question of vitality and justice by sculpting artifacts -- such as architecture, sculptures and technologies -- that problematize the ostensible promise of technology to transcend human limitations, while accepting its existence as intrinsic to our species. To these ends, I want to ground technology in our world as it is by imagining a technics -- the more inclusive and general term for human artifacts, from tools to our institutions that make and use them -- where our human creations can be seen for what they are, not what we wished they could deliver us from. From the telescope to writing to democracy itself, technics can certainly help us expand our understanding of ourselves and our world. However, mostly we use technics to escape our fears of death and irrelevancy. And, to face this uncomfortable human habit of escapism, technology has some intrinsic attributes that must be faced as we inevitably embrace it. I'll focus on three inter-related attributes, each with existential implications, to help set this investigation's context: Functional Over-coherence, Species Insulation and Entropy Maximization.
Technology, to date, is defined by its functional utility. We make tools for reasons, and if the tool does not perform that pre-established purpose, we consider it broken. However, as Greek tragedy points out, humans have a curious habit. As we put more effort into making tools for useful reasons, we double-down on believing past reasons for a tool will be relevant reasons in perpetuity. In other words, we create a positive feedback loop where the tool is made to perform some task useful to the human in a given context, and then we try to convince ourselves - even modifying our environment - to maintain the reason to maintain the tool. Often, this maintenance is in spite of a changing environment. Given our initial investment in a tool and its ostensible success, we tend to invert the logic, exhibiting hubris, to preserve a tool's fixed, if archaic, raison d'être.
This Functional Over-coherence necessitates insulating ourselves from our environment. We use ever more resources to mitigate the growing mismatch between our own dynamics and those in the larger open system. This insulation then allows us to cohere ever harder on the fabricated belief in stability, a belief animated by mechanistic metaphors of fixed functions and determinate outcomes. Ever increasing order and isolated efficiency is both the reward and its own cost. But, being a limited species, we lose track of all the costs we've externalized. We focus in, seeing only the benefits, while entropy accelerates beyond the horizon of our awareness.
This Species Insulation made possible by our technological insulation from change accelerates entropy production. Not only is there waste from making the originally useful tool, there's waste on top of that to maintain into the future an environment conducive to the utility for that original tool. This accelerates waste and risks rendering our species toxic to the wider biosphere, which, of course, renders us toxic to ourselves as Silent Spring foretells.
Some stabilization is healthy. However, as Heraclitus observed, one needs to change to remain the same, i.e. to be a river is for it to flow by definition. This is true for life ontologically. The dilemma is a "bird in the hand vs. two in the bush." Does a society focus on what works now, specializing, or does it diversify its capacities, generalize? Do we cohere on what is known or risk exploration into the unknown? Life is in the balance.
This paradox is captured by the Blind Men and the Elephant. Each has their own perspective, perceiving the world in their own way. To maximize the view of one over the others risks perceiving little of reality. The task is to first acknowledge the reasonable, even if different, view of each and then strive to resolve the contradictions among them. My intent is to resolve this dilemma, and those described above by creating a novel technology to help find overlaps among our diverse understandings, while providing a lens through which this understanding of ourselves in the physical world is expanded.
Below are a series of projects that describe my journey to create this new technology that is not merely mechanical, and whose behavior can adapt and complexify in response to its environment, not unlike sentient creatures. The intent is to better understand our own capacity to navigate the paradoxical terrain into the future by building a technology that doesn't abet our over-coherent, insulated and wasteful relationship with the only known biosphere. It intends to augment our individual and social capacity to perceive the wider existential implications of our choices.
World Trade Center Memorial Competition
World Trade Center Memorial Design Competition Entry; 2003
Design - 3D AutoCAD, 3DS Max, Photoshop
Unbuilt
Design - 3D AutoCAD, 3DS Max, Photoshop
Unbuilt
This rather ambitious task started with a seemingly simple and practical intent. I wanted to make an interactive sculpture that transcends current New Media. More than a mechanical or algorithmic stimulus-response interaction with passersby, I wanted to create an immersive experience where the architectural artifice had a literal sentient relationship with human visitors.
Back in 2003, to do this, I proposed a transparent obelisk that would display a dynamic 3D abstract pattern. It was an interpretation of sensed motion, heat, sound, and other patterns of visitors to the site (upper image below). Individual participants would figuratively see themselves reflected in the proposed 3D display (lower left image); while, many participants would see both themselves and the abstract patterns of others superposed in real-time (lower right image). Based on sensing people, past and present, the dynamic pattern would evolve to create ever-novel and increasingly complex "patterns of patterns".
The intent was for people to imagine themselves among others in a non-judgmental abstract way. I wanted them to see themselves as distinct while inter-related, simultaneously. Over time, patterns would be selected by the system that tended to attract more people, thereby increasing its own impact. This was my response to the acrimony of the 9/11 attack.
Below is a diagram of my nascent attempt at a rigorous description. Creating that diagram generated many more questions than answers and set the stage for a decade of investigation into how exactly a technology could be built that would literally sense its environment and continuously synthesize novel higher-order patterns based on those sensory inputs.
Improbable Monument
Studio project within the Conceptual and Information Arts, San Francisco State Univ.; 2007
Design - AutoCAD, 3Ds MAX and Photoshop
Unbuilt
Design - AutoCAD, 3Ds MAX and Photoshop
Unbuilt
To implement the art installation above, as early as 2004, I was fascinated by artificial neural networks, not unlike those under the hood of current Deep Learning implementations; however, once I read a number of neuroscience papers and investigated some of the critiques of “brain as computer” by John Searle, Hubert Dreyfus and Rodney Brooks, I realized that my goal of sentience wasn’t down any purely digital road. So, I went to many lectures at the Helen Wills Neuroscience Institute at UC Berkeley, and was part of a neuroscience study in Professor Levi’s perception lab.
In 2007, I applied and was accepted to the Conceptual Information Arts program at San Francisco State under Steven Wilson at the time. Although nascent and only intuitive in its nature at that time, I made this video for acceptance to the program describing my artistic medium: SFSU Application Video
But, after a semester I realized that my technology, or "medium", able to implement my proposed "sentient observatories" was not ready. After SF State I recommitted myself to rigorously articulate both the design to build and theory to test what I then called "synthetic cognition". And for technical implementation purposes, between 2007 and 2009, I completed the analog and digital circuit design tracks at San Francisco City College along with JAVA programming.
And, although my semester at SF State didn't render my technology any more rigorous, it did help me refine my artistic intent. The project shown above was an attempt to better integrate a built structure with an immersive and interactive experience. This project links the artistic intent in my World Trade Center Memorial design to the body of work at Studio for Public Spaces, which further develops and realizes my art installation practice.
As an "Improbable Monument", it intended to challenge the idea that the permanence of memory relies on the static and immobile rigidity of a traditional monument. Like Heraclitus's river allegory, actual memories in the brain are paradoxically maintained through change. They are adapted by subsequent experiences to maintain their relevance in a changing world. Visitor stimulation is necessary for this maintenance, just as humans must have stimulation.
S.O.A.R.S.E.
Schematic design for a Self Organizing Autonomous Resonant Superposition Engine; 2011
Design - AutoCAD, 3Ds MAX, Photoshop
Build - analog sensors and physical 3D nodal matrix with an abstract digital data structure
Design - AutoCAD, 3Ds MAX, Photoshop
Build - analog sensors and physical 3D nodal matrix with an abstract digital data structure
My work on the Improbable Monument at SF State helped me realize how much further I still had to go to realize a viable implementation for anything approximating animal sentience. So, from 2007 to the end of 2011, I focused entirely on that implementation, which is summarized briefly below.
To understand its non-exotic, if novel and unfamiliar, nature it helps to understand my original inspirations for SOARSE. The precipitating event was the trivial observation of sunlight reflecting off a baking dish in my kitchen. The light shown on a wall, illuminating undulating waves, a sort of gossamer of dancing light. I marveled at how much information was contained in that simple, yet dynamic, reflection; and, I also realized that this information was, in a sense, a physical proto-interpretation, or representation, of many simultaneous inter-dependent events: the sun's angle; the geometry of the dish and its constraints on the waves; the water properties; perturbation of the dish; and, the 2D wall.
This observation galvanized two other core inspirations: my study of quantum mechanics at UC Santa Barbara and my exposure to Colin Rowe's Transparency: Literal and Phenomenal at CU Denver. The conceptual and practical take-away from these two deep subjects is the concept of superposition. Briefly, the problem of sensing and knowing rests on the capacity to realize not just individual things in the mind, but the interplay between them, simultaneously. In other words, some physical medium must embody and superpose both the distinct patterns and their inter-dependencies, simultaneously. For example, knowing a few things about an apple - like color, shape, feel and taste - and their inter-dependencies is significantly more useful than knowing infinitely many isolated things. These myriad physically embodied inter-dependencies, like the waves in the baking dish, are the scaffold upon which higher-order conception can be accrued.
But, how does one make this?
As mentioned above, I started with recursive neural nets, the basis for Deep Learning today. However, those neural nets, when digitally implemented, are constrained to isolated states, because that's the genius of digital: its noiseless preservation of state. But, the downside is that, by design, digital can never truly be superposed. It can only ever simulate superposition. Nature, on the other hand, is inherently superposed. Nature is fundamentally noisy; and, the brain evolved in and adapted to this reality. As Searle points out "simulation is not duplication", and as Korzybski pointed out "the map is not the territory."
But, what do Searle and Korzybski really mean, and what do we get in return for what amounts to adding noise to our once isolated and stable states? What are we trying to put back into our technologies that current functional machines have bracketed off? How can a technology reform itself to inform itself?
Stable states are not adaptive; so, by adding what I call "structured noise", we get the opportunity to discover novel inter-relations among environmental phenomena. The noise is structured because we are putting relevance back into the machine; in other words, each Primary Node (burgundy above) is uniquely sensitive to the environment, just like people are. Digital transistors, on the other hand, are specifically designed to behave identically; whereas, diverse SOARSE elements resonate with their environment differentially, and then "emote" a signal back into their shared space to create a novel superposed complex pattern, like the baking dish patterns. This so-called "noise" is quite structured because it has inherent physical order that is relevant to the higher-order environment that stimulated it. Now, Secondary Nodes (light blue above) literally sense this interstitial field and harvest the sparse order that emerges. This "harvesting" is the creation of interconnections along the relatively constructive interference in the superposed terrain; and, these connections tend to correlate with the co-stimulation of diverse Primary Nodes by patterns in the environment, e.g. Hebbian Plasticity. In essence, the system re-forms itself to in-form itself, because it accrues network biases that sensitize it to ever-more complex patterns of patterns that are literally resonant with dynamic patterns in its environment.
What was once a dilemma with the diversity of response from the Blind Men, is now a strength as the system leverages its distinct perspectives of a shared reality to embody a higher-order sentience. Like Heraclitus's river, the distinct sentient self, situated in the world, is maintained through continual and dynamic shaping by the complex terrain it is immersed within. Far from being a simulation, a static map or a modular cog with a functionally fixed response, the patterns of patterns that accrue within the sentient agent are unique to its internal structure, tuning, history and physical vantage point.
SOARSE is a technology that implements a self-organizing adaptive resonant superposition engine, a step towards authentic sentience; and, it is my artistic medium, which intends to breath some semblance of "life" into the lifeless technology and architecture that currently insulate us from a more general, adaptive and vital understanding of ourselves in the world.
CULTURE AND TECHNICS
When I first read Gilbert Simondon's essay Culture and technics (1965), I misunderstood his main point. At first, I thought he was arguing for a culture with more control over technics, when, in fact, he was largely arguing for the opposite: a society less about breeding the domesticated citizen towards less complexity and more about proficiency in technics grounded in natural systems, embracing a shared complexity. The irony is that, despite my obvious interest in technics from architecture to synthetic sentience, I still had an intrinsic bias against technics, such that I could not believe, upon my initial reading, that Simondon was advocating for more machines.
Of course, Simondon's argument is more subtle even as he plainly asserts that to liberate ourselves from our self-imposed alienation, we must liberate the machine! How can this possibly make sense?
Technics is a projection of ourselves onto our shared world; it reflects our deep desire for domination over nature, death and our fear of them both. In a trifecta of irony, paradox and tragedy, insofar as we succeed in this endeavor, we only succeed in dominating ourselves, while rendering our viability ever more brittle on this planet. In Culture and technics, Simondon argues that it is our focus on culture that is to blame as it distills subtle, complex and valuable dynamics down to overly simplistic narratives like nationalism and religion that unite some while necessarily dividing many others. He states (p. 23):
The employment of cultural contents acquired in childhood, like national glory, the courage of valorous exploits, or the exigency that the true religion triumph over the infidels, can only serve to keep a sane analysis of the problem at bay: culture, here, as an obstacle to the only adequate technicity, is particularly murderous and noxious; it leads to a regression, to the point where, out of utter exhaustion, one finally adopts a technical solution.
Simondon is not categorically against culture, but he argues that, without a check on its hegemonic tendencies, it will tend to bend technical solutions towards "murderous and noxious" outcomes. On the same page he contextualizes culture and technicity as follows:
Culture, rooted in the invariance of groups, would be perfectly adapted to the resolution of a problem, if that problem were purely human, which is to say if it were posed in terms of relations and attitudes internal to a homogeneous group. Technicity, by contrast, would be best suited to problems concerning the relation between man and the environment;
I interpret Simondon to mean that culture serves to organize, discipline and homogenize, whereas technicity is the means by which humans mediate between themselves and the greater environment. Through the lens of my own project outlined above, these are potentially complementary forces: a force of coherence and distinctness that contrasts the 'signal' of civilization from the 'noise', and a force of broader inter-relation and dependency to both each other and their shared context. Culture is what gives us our identity and often our very purpose, but it is also a source of our hubris and blindness. Technics are the means by which we remain rooted; any number of distinct warring cultures must employ technics that tap into a common ground shared by all, even as the culture may actively endeavor to deny this shared ground because it competes with the very nature of a coherent culture.
This paradox is the core of my own project, which concludes that sentience exists to resolve this paradox by embodying both distinct metastabilities and a common dynamic medium from which such distinctness emerges.* Humans focus on the stabilities, or 'signals' in the 'noise', because that is the nature of our human perception: objectification. The continuous dynamic medium, of which all signals share and from which they all emerge, is intrinsically difficult to characterize symbolically; this relatively ephemeral milieu is sub-symbolic.** And, as suggested earlier, its physical, biological and social nature contradicts the very logic of fixed narratives our minds crave and our cultures demand.
A powerful example that suggests an answer to Simondon's Culture and technics are totem poles. I was in the Pacific Northwest and visited some sites in Vancouver that showcased totem poles from First Nation Tribes. They are cultural insofar as they communicate status and provide a collective identity for a tribe or family. But they are also technics in that they relate humans to the broader milieu of nature they are all immersed. Each tribe draws its power from a common pool that is both physically and spiritually more powerful than any one family or tribe.
There is so much to consider with this and similar examples in our human history, but it is important to avoid overly romanticizing a past that is no longer our set of circumstances. And, as René Girard and Georges Bataille would argue, systems of exchange meant to manage internal and external rivalries all have their dark sides; many of those from the past, like human sacrifice, are totally unacceptable today. Nevertheless, provided these cautions, we can draw from our shared human experience to prototype novel institutions able to sustain our ongoing viability as a species.
My own initial prototypes involve what I am provisionally calling catalytic probes, urban totems, sentient observatories and field stations. They are all works in progress. The most recent examples of which are my Sentient Art Works project and a community based grant application, which are both put in a larger professional and Critical Theoretical context in a recent report entitled Problematizing Social Practice: Interactive Installations and Human Habits of Alienation, which is available upon request.
* In his project of Individuation, Simondon calls this metastable common medium the pre-individual. To adapt to their ever changing circumstances, the living individual must constantly individuate, which means that they must resolve the internal tensions that arise among the many dimensions within themselves that embody many orders of magnitude, just as the plant resolves the cosmic scale of sunlight with the tiny molecules in its surroundings to manifest itself. The metastable pre-individual is a precondition of the individual's capacity to creatively and adaptively individuate.
** I'm using the term symbol here in the semiotic sense inspired by C.S. Peirce. As such, by sub-symbolic, I intend icon-like and index-like sign actions, or semiosis, as opposed to the more familiar symbolic mode of animal communication, which is built on relations of icons and indexes. For my purposes here, an example of icon-like semiosis is resonance, such as a tuning fork responding sympathetically to a particular frequency; and an example of index-like semiosis is contiguity, whereby differently tuned forks -- e.g., fork A and B -- tend to be stimulated at the same time so that a semiotic system -- e.g., a human -- might come to anticipate the stimulation of tuning fork B when fork A is stimulated. These examples are non-symbolic phenomenon and they are also sub-symbolic in the sense that symbolic semiosis emerges upon the often ignored scaffolding of relations between iconic and indexical phenomena.