Fruit fly-inspired AI learns smells quickly with far less memory

A fruit fly brain contains only about 140,000 neurons, yet it can recognize odors with remarkable efficiency. Researchers have now borrowed part of that strategy to build an artificial system that learns smells quickly while using little memory.

The algorithm, called Spi-Fly, uses a spiking neural network modeled on the fruit fly olfactory system. Researchers at the Okinawa Institute of Science and Technology, or OIST, designed it for situations where training examples, memory and computing power are limited.

The system performed especially well when it had only a few examples of each odor. It also learned new categories without substantially forgetting earlier ones.

“It’s one of the best-mapped brains; it’s simple enough that we can study the brain as a whole, but it still achieves complex processing,” said co-author Yang Shen of OIST.

Fruit flies process odors by expanding incoming information into a much larger neural population and then sharply limiting how many cells remain active.

Odor information reaches roughly 2,000 Kenyon cells in a brain structure called the mushroom body. Each Kenyon cell samples only a small number of inputs.

Strong inhibition then silences about 95% of those cells.

Only around 5% remain active for a particular smell, producing a sparse pattern that functions somewhat like a neural barcode.

“It has a particularly interesting olfactory system that implements sparse coding, a computational scheme that helps reduce overlap between different signals without training, so we can efficiently represent odors with lower energy cost,” Shen said.

Spi-Fly recreates those principles with three layers.

Incoming sensor values become electrical spikes. Those input neurons connect sparsely and randomly to a much larger hidden layer, which contained 1,000 neurons in the main experiments.

Competition among hidden neurons leaves only a small fraction active. The network then strengthens connections between those active neurons and the output assigned to the correct odor.

Unlike conventional neural networks, Spi-Fly does not need backpropagation to adjust every part of the network.

Researchers tested Spi-Fly on two odor datasets.

The first contained 200 gas-sensor recordings representing 10 compounds, including acetone, ammonia, methane, methanol, benzene and toluene. Each recording used 72 sensors.

The network contained 72 input neurons, 1,000 hidden neurons and 10 output neurons.

On that dataset, Spi-Fly reached its peak performance after seeing each odor only three times.

A second experiment used synthetic data modeled on fruit fly olfactory receptors. It included 18 odors, five receptor types and 450 electrode traces.

That version of Spi-Fly used five input neurons, the same 1,000-neuron hidden layer and 18 outputs.

It reached its best performance after just one presentation of each odor.

“Spi-Fly was the best performing method for few-shot learning, classifying scents easily after seeing only a few samples,” said co-author Kevin Max of OIST’s Neural Computation Unit.

Conventional training eventually achieved better accuracy on the synthetic dataset after repeated exposure, but rapid learning from scarce examples was Spi-Fly’s main advantage.

The researchers also tested continual learning, a major challenge for artificial neural networks.

Many systems suffer from catastrophic forgetting. When they learn a new task, changes to their internal connections can damage information stored from earlier tasks.

Spi-Fly largely avoided that problem.

The team taught networks pairs of odor classes one after another. After each new pair, the systems were tested on every class learned so far.

Spi-Fly showed no significant performance loss compared with training on all classes together.

A spiking network trained with backpropagation through time, or BPTT, performed much worse and eventually scored only slightly above chance.

Researchers also added elastic weight consolidation, a technique designed to protect important old connections. It did not significantly improve BPTT’s continual-learning performance.

Spi-Fly works differently because adding a new class mainly changes connections leading to that class’s output neuron.

Connections responsible for previously learned odors stay largely untouched.

“It was also good at continual learning, correctly learning to classify new odors without forgetting about past learned ones, a huge issue with standard training methods, known as catastrophic forgetting,” Max said.

Another advantage appeared when researchers reduced the precision of the network’s weights.

Neuromorphic hardware often stores synaptic values using only a few digital bits because memory is limited.

Spi-Fly remained effective at 6-bit precision and lost less performance than BPTT as precision decreased.

On the gas-sensor dataset, it even outperformed BPTT at 4 bits.

The memory difference was larger still.

Backpropagation requires storage for weights, neural activity and gradients during training. Continual-learning methods can require extra information as well.

Spi-Fly mainly needs local learning and a buffer for spike counts.

Researchers estimated that its off-chip memory requirements were about five orders of magnitude lower than BPTT and BPTT with elastic weight consolidation in the gas-sensor experiment.

That reduction could matter for systems designed to operate on small, energy-constrained hardware rather than powerful external computers.

Spi-Fly still has important limitations.

After many repeated training examples, its classification performance falls below stronger conventional machine-learning methods.

The system also does not update connections between its input and hidden layers, limiting how much it can refine internal representations.

Its performance declines at very low weight precision, even though it handles that decline better than BPTT.

The experiments also used individual odors.

Natural environments commonly contain mixtures and background smells, and researchers do not yet know how well Spi-Fly will separate those complicated signals.

The network requires supervised labels during learning, and its capacity depends partly on the size of the hidden layer.

It also does not explicitly use precise spike timing, even though timing contributes to real fruit fly smell processing.

The immediate value of Spi-Fly lies in showing that biologically inspired learning can address several practical limits at once.

The system learns from few examples, resists catastrophic forgetting and uses much less memory than conventional backpropagation.

Researchers are now working toward integrating the algorithm with odor-sensing hardware being developed with collaborators at TU Eindhoven and Kiel University.

“Detecting explosives; assessing allergen levels; ensuring food safety; identifying drugs, there’s a huge range of applications for scent classification,” Max said.

The team ultimately aims to build an artificial nose that combines chemical sensors with brain-inspired processing.

Future work will need to test mixed odors, improve accuracy, refine weight quantization and examine performance on real hardware.

For now, the fruit fly offers a compact lesson: sophisticated learning does not necessarily require a large brain or a massive computing system.

These resources explore the sparse neural coding and continual-learning ideas that inspired systems such as Spi-Fly.

Sparse, decorrelated odor coding in the mushroom body enhances learned odor discrimination: This study showed that inhibitory feedback keeps fruit fly Kenyon cell responses sparse and helps flies distinguish similar odors. (Nature Neuroscience, 2014)

Mushroom body memoir: from maps to models: This foundational review explains how the fruit fly mushroom body supports odor learning and memory through sparse neural representations. (Nature Reviews Neuroscience, 2003)

Drosophila olfactory memory: single genes to complex neural circuits: The review traces how mushroom body circuits and their molecular machinery support odor learning and memory in fruit flies. (Nature Reviews Neuroscience, 2007)

A neural theory for counting memories: This theoretical work examines how sparse random projections in the mushroom body can support efficient memory storage. (Nature Communications, 2022)

A survey and perspective on neuromorphic continual learning systems: This review examines how brain-inspired hardware and algorithms can learn new information while reducing catastrophic forgetting. (Frontiers in Neuroscience, 2023)

Research findings are available online in the journal Neuromorphic Computing and Engineering.

The original story "Fruit fly-inspired AI learns smells quickly with far less memory" is published in The Brighter Side of News.

Like these kind of feel good stories? Get The Brighter Side of News' newsletter.