On extending human understanding of animal sensory worlds through AI
This post is a bit different from my typical fare. I was inspired this last month reading Ed Yong’s “An Immense World: How Animal Senses Reveal the Hidden Realms Around Us” – a book about the diversity and wonder of animal senses, and how their sensory worlds (aka “their Umwelten”) allow them to experience the world in ways humans are unable to fully comprehend. In parallel, I’ve been spending a lot of time thinking about artificial intelligence (AI) so I thought I’d tie these two threads together.
The gist of the post is this: The physical world is far beyond human understanding (and the more we learn, the more we find ways we are limited). Meanwhile, AI advances are showing that humans can create intelligences exceeding our own. While a lot of people talk about the dangers and the nearer-term applications of AI, I’m going to instead reflect on how it might extend our understanding and connection to the world around us.
If this sounds interesting to you, let’s dive in!
UMWELTEN & THE LIMITATIONS OF BEING HUMAN
We humans only perceive a tiny fraction of the world around us. I don’t mean this in a metaphysical way, I mean it literally. The world around us is rich in physical stimuli we can’t tap into.
It’s hard for me to truly describe how limited our human sensory world is is without asking you to spend time reading Ed Yong's book. It’s the kind of book where I highlight so much it ends up being pointless… Every page is rich with exquisite, mind-blowing details about animal biology. To summarize the book, here is Yong’s main thesis (if you’re short on time, just read the highlighted pieces):
Earth teems with sights and textures, sounds and vibrations, smells and tastes, electric and magnetic fields. But every animal can only tap into a small fraction of reality's fullness. Each is enclosed within its own unique sensory bubble, perceiving but a tiny sliver of an immense world.
There is a wonderful word for this sensory bubble Umwelt. It was defined and popularized by the Baltic German zoologist Jakob von Uexküll in 1909. Umwelt comes from the German word for "environment," but Uexküll didn't use it simply to refer to an animal's surroundings. Instead, an Umwelt is specifically the part of those surroundings that an animal can sense and experience-its perceptual world. Like the occupants of our imaginary room, a multitude of creatures could be standing in the same physical space and have completely different Umwelten. […]
This was a radical notion at the time—and in some circles, it might still be. Unlike many of his contemporaries, Uexküll saw animals not as mere machines but as sentient entities, whose inner worlds not only existed but were worth contemplating. Uexküll didn't exalt the inner worlds of humans over those of other species. Rather, he treated the Umwelt concept as a unifying and leveling force. The human's house might be bigger than the tick's, with more windows overlooking a wider garden, but we are still stuck inside one, looking out. Our Umwelt is still limited; it just doesn't feel that way. To us, it feels all-encompassing. It is all that we know, and so we easily mistake it for all there is to know. This is an illusion, and one that every animal shares.
We cannot sense the faint electric fields that sharks and platypuses can. We are not privy to the magnetic fields that robins and sea turtles detect. We can't trace the invisible trail of a swimming fish the way a seal can. We can't feel the air currents created by a buzzing fly the way a wandering spider does. Our ears cannot hear the ultrasonic calls of rodents and hummingbirds or the infrasonic calls of elephants and whales. Our eyes cannot see the infrared radiation that rattlesnakes detect or the ultraviolet light that the birds and the bees can sense.
The Umwelt concept can feel constrictive because it implies that every creature is trapped within the house of its senses. But to me, the idea is wonderfully expansive. It tells us that all is not as it seems and that everything we experience is but a filtered version of everything that we could experience. It reminds us that there is light in darkness, noise in silence, richness in nothingness. […]
I highly recommend Yong’s book if this piques your interest. It has changed how I view animals: I am now hyperaware of how ignorant I am to their Umwelten. The bird, dog, cat, rodent, insect - they all have so many senses I can’t connect with and I can only imagine a fraction of what they perceive.
Beyond increasing our empathy, is there nothing else we humans can do to deepen our connection to animal Umwelten? Could we develop ways of moving beyond our human understanding?
Let’s talk about AlphaGo.
ALPHAGO & CREATING INTELLIGENCES WE DON’T UNDERSTAND
Thinking more about this question, I recently watched the full AlphaGo documentary (free on YouTube). For those that don’t know the context, the movie documents how Deepmind (the Alphabet/Google research lab based out of the UK) created AI to beat the world champion Go player - a feat people thought was far off and/or impossible.
The movie is extremely well done and quite profound. It leaves you feeling daunted by the coming AI wave. There’s no way you watch that movie and feel like things will be the same in the future as they are today. It’s amazing to watch the world-champion Go-players and their coterie move from hubris to shock to fear as they repeatedly lose to AlphaGo.
What struck me most about AlphaGo was how it played moves far beyond the knowledge of the creators of the program. The programmers at Deepmind were not Go champions - many didn’t even know the intricacies of the game. As they watch the games play out, they have no idea what Go is actually doing and yet the program makes moves beyond any seen before in history, leaving the crowd of experts speechless. This is not unique to AlphaGo - the makers of AI models are often not subject matter experts on the subject of the data used for training (e.g., someone working on DALL-E doesn’t need to be a world-class painter).
It’s remarkable to see as AlphaGo plays the Go world champion and the experts struggle to understand non-human intelligence: “The whole game we thought AlphaGo was wrong about the board position” and yet it won. “AlphaGo knew this was an unusual human move but it still chose it - something humans would never do” they say in shock.
Recently with DALL-E and other AI advances, you hear people talk about how AI will enhance human creativity. And while this is definitely true, the more I watch AlphaGo the more I think this is a narrow view of AI’s potential. Naturally when we think of AI we think about it in our human context, but as Ed Yong’s book tells us, this is an incredibly small tranche of the perceivable world. The human Umwelt is tiny.
Steve Jobs famously described computers as “bicycles for the mind.” This analogy works if we are thinking of computers in the same way we think about cars or bikes, as extending existing human capabilities. However, I think this limitation misses the mark for AI - it doesn’t have to just be a bicycle for the mind, but AI could allow us to build completely new ones.
HUMAN LIMITS IN UNDERSTANDING REAL COMPLEXITY
There is this post from Bert Hubert called “Is biology too complex to ever understand?” where he talks about how likely it is that humans will never be fully capable of understanding biology without the help of computers:
[Nowadays] Biology has [turned] into a gigantic fact-finding mission, assembling 100 articles every hour, full of new and recycled observations. This stream of words vastly exceeds what any human being could possibly comprehend, even when concentrating on a sliver.
[We] know that there is too much unsynthesised data for humans to make sense of, it may be better to 1) Not attempt [a physics-style] global understanding, [but instead] 2) Gather everything we learn into first-class quality databases that might enable computers to make sense of what we have learned.
AlphaFold (Deepmind’s foray into protein folding) shows how we’ve already started down this path. Just recently, AlphaFold released predicted structures for 200M proteins - “the entire protein universe”. This is but the start of AI helping us understand the natural world...
EXTENDING THE HUMAN UMWELT THROUGH AI
As AI advances it offers the possibility of extending our Umwelt along the dimensions perceived by other species.
What if we can begin to understand and interact with the star mole, the spider, the snake? What if we can create AI that mimics the way a bird flies - and not just by putting a big piece of metal in the sky but actually “feeling” the micro-air currents and modifying the shape of wings? What if we can create intelligences to feel vibrations the way bugs do? To understand the dynamics and motions of the ocean like a fish? To feel electric signals like a shark?
This isn’t just about communicating with other species (although that is interesting too, e.g., “Towards understanding the communication of sperm whales”) but it’s more about creating AIs that interact with worlds we can’t understand: A theoretical AlphaFish would "feel" the ocean currents. AlphaSpider would "sense" the web’s vibrations. AlphaSnake could "smell" heat…
Reading Yong I can’t help but feel that the animal Umwelten and the physical stimuli they tap into present entire new worlds of engineering potential. People often talk about how “the low hanging fruit of innovation” is over, but it’s impossible to read about animal biology and feel like we are even close to a technological frontier. Look at an ant, a bird, a worm - we can only dream about this level of technology and it’s only now that we start to have the tools to unfold its layers.
There is a quote from Ted Chiang’s story, “The Great Silence,” where the parrot narrator tells humans to look more closely at nature:
The humans use Arecibo to look for extraterrestrial intelligence. Their desire to make a connection is so strong that they’ve created an ear capable of hearing across the universe.
But I and my fellow parrots are right here. Why aren’t they interested in listening to our voices?
There is so much for us to learn just by looking at the world around us... I’ll close with a quote from Yong’s book that captures the opportunity:
This ability to dip into other Umwelten is our greatest sensory skill. [Humans are] the only creature capable of knowing what the others [animals are] sensing and, perhaps, the only one who might care. A bogong moth will never know what a zebra finch hears in its song, a zebra finch will never feel the electric buzz of a black ghost knifefish, a knifefish will never see through the eyes of a mantis shrimp, a mantis shrimp will never smell the way a dog can, and a dog will never understand what it is like to be a bat. We will never fully do any of these things either, but we are the only animal that can even come close. We may not ever know what it is to be an octopus, but at least we know that octopuses exist, and that their experiences differ from ours. Through patient observation, through the technologies at our disposal, through the scientific method, and, above all else, through our curiosity and imagination, we can try to step into their worlds. We must choose to do so, and to have that choice is a gift. It is not a blessing we have earned, but it is one we must cherish.
