Written by Dirk Jan Buter

I Learned Flies to Talk

I Learned Flies to Talk

Written by Dirk Jan Buter on September 08, 2026. Photo by Dirk Jan Buter.

It all started with a message from Niels containing a link to a fascinating GitHub project. The repository featured a virtual fruit fly living on your desktop whose movements, such as walking, resting, grooming, and taking off, were entirely driven by a live emulation of a real fruit fly brain. I was curious to understand how the application worked under the hood and how realistic the emulation actually was in practice, so I decided to take it on as a side project.

To appreciate why this is so remarkable, one has to look back at the history of fruit fly neuroscience. For decades, mapping the complete wiring diagram of an insect brain, known as a connectome, was considered an almost impossible feat. While scientists managed to map the simple 302-neuron nervous system of a roundworm back in the 1980s, an adult fruit fly brain consists of over 139,000 neurons and more than 50 million synaptic connections.

The major breakthrough came through the international FlyWire Consortium, alongside researchers at HHMI’s Janelia Research Campus and Google Research. To capture this complex architecture, an adult female fruit fly brain was sliced into thousands of ultra-thin sections, each measuring just 40 nanometers thick. Every slice was scanned using high-resolution electron microscopy. Computer vision and artificial intelligence algorithms stitched millions of images into a three-dimensional reconstruction, which was then painstakingly proofread by neuroscientists around the world. The resulting FAFB dataset allowed researchers for the first time to trace exactly how sensory inputs flow through internal networks to trigger specific motor outputs, such as the fly’s rapid escape reflex.

The original project that Niels shared used this connectome dataset to run a live Leaky Integrate-and-Fire neural emulation on the desktop. The virtual fly reacted dynamically to screen events because visual stimuli caused emulated neurons to fire up to an escape threshold. The fly also treated open application window borders as physical ledges to walk along.

For my own experiment, I wanted to replicate this behavior inside the Unity game engine. To bridge Unity with the neural emulation, I built a Python socket link. The Python server runs the spiking neuron calculations and streams active firing data continuously to Unity. In Unity, this stream is mapped directly into animations, forces, and character movement. Although the Python server also supports sending environmental feedback back into the brain model, I kept my initial version focused on executing the incoming motor commands.

To make the experiment more entertaining, I decided to give the fly’s internal physiological states a voice. By monitoring firing activity across specific neuron clusters, I could infer whether the fly was startled by the cursor, hungry, or simply relaxing. I connected these emotional states to the ElevenLabs API to let the flies speak out loud. To keep API usage efficient, generated audio clips are saved to a local cache so that repeated phrases can play instantly without making new cloud requests.

It remains incredible to watch a complex neurological dataset travel through a socket link and voice synthesis to become an interactive, talking digital organism on screen.

I Learned Flies to Talk

It all started with a message from Niels containing a link to a fascinating GitHub project. The repository featured a virtual fruit fly living on your desktop whose movements, such as walking, resting, grooming, and taking off, were entirely driven by a live emulation of a real fruit fly brain. I was curious to understand how the application worked under the hood and how realistic the emulation actually was in practice, so I decided to take it on as a side project. To appreciate why this is so remarkable, one has to look back at the history of fruit fly neuroscience. For decades, mapping the complete wiring diagram of an insect brain, known as a connectome, was considered an almost impossible feat. While scientists managed to map the simple 302-neuron nervous system of a roundworm back...

Why Software Must Be Free

For decades, the software industry has operated on a foundational premise: code is a scarce intellectual property that can be locked behind paywalls, licensed, and monetized. Companies built moats out of proprietary codebases, and copyright laws served as the walls protecting them. Today, that entire paradigm is collapsing. We have entered an era where clinging to traditional software economics is not only obsolete, it is actively harmful to our technological progress and our planet. When software carries a hefty price tag or restrictive licensing terms, a strange economic friction occurs. Instead of adopting an existing, perfectly functional solution, organizations and individuals increasingly choose to build their own custom alternatives using AI. At first glance, this looks like empo...

My Way of Talking to an AI Everyday

Many people are already familiar with AI and others are still a bit behind. That is not surprising when you consider how fast the developments are going. I myself have been experimenting a lot and have even made my own AI clone. This process was not only very fun to do, it has changed my life considerably. The change came because I started talking to my AI clone every day. I have now been talking to this AI for a week and in this article I describe my transformation. First I will explain what an AI clone exactly is according to my views. During the start of the recordings of the “AI Love” documentary, I came up with the idea of ​​making an AI clone. It seemed interesting to me to copy myself into the digital world as a counterpart to cryonics or mind uploading. Cryonics is freezi...

How to Make Your Own AI Clone

It started as an experiment; I wanted to build a chatbot from scratch. The main goal was not so much the chatbot itself, but more to gain knowledge. Soon the idea went from an AI experiment to a project with a real goal and application. The idea came from a TV documentary series "AI Love" that I was asked to participate in. So I thought it would be fun to create an AI clone that can train itself and in turn teach me new things and assist me in my daily life and my work. So this year we are going to create an AI clone and you can follow this process and also help. I have been working on it for a while and now have a pretty good overview of how it works. That is why I can now make a good estimate for a step-by-step plan. In addition to the step-by-step plan, I will share every line of code...

DNA as a guide to prevent health problems

I had wanted to have my DNA tested for a long time to learn more about my genetic mutations and ancestry. In this way, I wanted to discover the weaknesses in my DNA profile and take preventive measures. That is why I had my DNA tested via 23andMe. I was full of anticipation and curiosity as I waited for the results. What if I had a mutation that could affect my life? What if I had a condition that I did not know I had? My tests yielded several results that caught my attention. We will discuss these in detail below. But first, a brief introduction to 23andMe. ## 23andMe 23andMe is a genetic testing service that gives users detailed insights into their genetic makeup. The service includes reports on ancestry, traits, and, most crucially, health predictions. The health reports analyze spec...

The Call for the Real: A Reflection

Never before have we been so surrounded by the artificial. AI-generated art, deepfake videos, automated conversations, it’s literally everywhere. And in the midst of this digital tsunami, a countermovement is emerging, a silent call for the real. We long for the tangibility of handmade products, the imperfection of analogue photography, the sincerity of human interaction. *It seems as if, in the attempt to achieve perfection, we have lost the essence of being human. Now, more than ever, we realize the value of the real, the authentic, the human.* I almost dare not say it out loud anymore, because we are so overwhelmed by AI that we are craving real things instead of fakes. I understand the desire for authenticity, for the real, for craftsmanship. We want to go back to products with real...