Neural Network Modeling of the Brain Function of a Fruit Fly and its Connections with 50 Million Nerve Cells (Drosophila melanogaster)
h is a vector of length M representing KC activity, r is a matrix of length N representing the synaptic weights between the KCs and the PNs The number of KCs and ALPNs is denoted by M and N, respectively. In this model, the PN activity is assumed to have zero mean, ({\bar{{\bf{r}}}}{{\rm{P}}{\rm{N}}}=0), and be uncorrelated, (\bar{{{\bf{r}}}{{\rm{P}}{\rm{N}}}\cdot {{\bf{r}}}{{\rm{P}}{\rm{N}}}}={{\bf{I}}}{N}). Here, ({{\bf{I}}}{N}) is an N × N identity matrix and ({\bar{{\bf{r}}}}{{\rm{P}}{\rm{N}}}) denotes the average taken over independent realizations of ({{\bf{r}}}_{{\rm{P}}{\rm{N}}}). The ijth element is included in the covariance matrix of h.
The predictions of K were generated from a simple rate model of neural networks. 5k. KC activity is modelled by
The number of unique odour channels that the KC receives input from was used to calculate K. We looked at what inputs they received and how many were received by a KC. K as reported in Fig. 6 is based on non-thresholded connectivity. We’ve observed that the hemibrain has a lower K even if we filter out weak connections, because it’s stable across thresholds. 7g.
A fruit fly may not be the smartest, but scientists can still learn from it’s brain. Researchers are hoping to do that now that they have a new map — the most complete for any organism so far — of the brain of a single fruit fly (Drosophila melanogaster). The connecting diagram has more than 50 million connections of nervecells, which are found in the wiring diagram.
The dataset published by Buhmann et al. contained ~244 million synapses. The sphinx had to fulfill two criteria to be removed: 1) the pre- or post-synaptic location remained unassigned to a segment, and 2) it had a score of 50. We removed duplicate annotations between the pre- and postsynaptic partners, which were defined by their presynaptic coordinate. We ended up with 130 million of synapses.
To increase accessibility and reach of the annotated FlyWire connectome, meshes of proofread FlyWire cells were skeletonized and imported into a popular web-based tool for collaborative tracing, annotations and analysis. PostGIS is a PostgreSQL extension that is popular in the geographic information system community. This enables us to reuse many existing tools to work with large spatial datasets, for example, indexes, spatial queries and mesh representation.
When a matched type was either missing large parts of its arbours due to truncation in the hemibrain or the comparison with the FlyWire matches suggested closer inspection was required, we used cross-brain connectivity comparisons (see the section below) to decide whether to adjust (split or merge) the type. A merge of two or more hemibrain types was recorded as, for example, SIP078,SIP080, while a split would be recorded as PS090a and PS090b (that is, with a lower-case letter as a suffix). We would record the hemibrain body ID as hemibrain type and assign a CBXXXX to it if we were able to find a match for an untyped hemibrain neuron.
Our comparison was mostly against the right hemisphere because cell types exist only on the left side of the hemibrain.
Skeletonization of neuron meshes and morphological similarity scores for the flywire dataset using NBLAST38 and navis128
We created skeletons for all the neuron we thought were proofread by using the skeletor. In brief, neuron meshes from the exported segmentation (LOD 1) were downloaded and skeletonized using the wavefront method in skeletor. These raw skeletons were further processed to remove false twigs and heal breaks, and to produce downsampled versions using navis128. This skeletonization is implemented in fafbseg.
For some of our analyses, like the FlyWire and the hemibrain datasets, morphological similarity scores were generated with the help of NBLAST38. A point cloud is the term used byNBLAST to describe point clouds with associated tangents. For a given query, we use a k nearest neighbours search to score each nearest-neighbour pair by their distance and dot product from the two point clouds. The final querytarget NLS score is calculated using these summed up. Important to note is that the direction of the NBLAST affects the proliferation of ABBA. Unless otherwise noted, we use the minimum between the forward and reverse NBLAST scores.
The methods described in this section are similar to those in the electron microscopy stack. We applied a transformation to all synapses to map them into the FlyWire FAFB14.1 space. The field had a resolution of 64 64 40 NM.
The NBLAST algorithm is implemented in both navis and the natverse (Table 1). However, we modified the navis implementation for more efficient parallel computation in order to scale to pools of more than 100,000 neurons. For example, the full 139,000 FlyWire neurons alone occupy over 500 MB of memory. The majority of the largesnes took less than a day to complete, thanks to the generous assistance of the MRC LMB Scientific Computing group.
Biological Outliers with Complete Brain Segmentation: Hierarchical Annotation and Multi-Connectome Cell Typing of Drosophila
Central brain associated neurons further include superclasses: visual projection neurons (VPNs), ascending neurons and sensory neurons (but omit sensory neurons with cell class: visual).
Hierarchical annotations include flow, superclass, class (plus a subclass field in certain cases) and cell type. The flow and superclass were generally assigned based on an initial semi-automated approach followed by extensive and iterative manual curation. The definitions and sections of the Supplementary Table 3 can be found here.
Biological outliers range from small additional/missing branches to entire misguided neurite tracks, and were typically assessed within the context of a given cell type and best possible contralateral matches within FlyWire and/or the hemibrain. When biological outliers were suspected, careful proofreading was undertaken to avoid erroneous merges or splits of neuron segmentation.
(2) Some neurons are missing large arbours (for example, a whole axon or dendrite) because a main neurite suddenly ends and cannot be traced any further. Many Pundits cross the brain’s midline in commissures. In some cases, we were able to bridge the gaps, and in others, we were unable to find the missing branch. We classified them as outliers, where the neurons remained incomplete.
Source: Whole-brain annotation and multi-connectome cell typing of Drosophila
Enhanced Box Plots for the Brain: I. Analyses for the Significance of Gradual Variation Inferred from Data and Observations
Enhanced box plots—also called letter-value plots125—in Fig. 5h and Extended Data Fig. 7f are box plots that can be used to represent large samples. They replace the whiskers with a variable number of letter values where the number of letters is based on the uncertainty associated with each estimate, and therefore on the number of observations. The fattest letters are 25th and 75th quantiles and the second fattest letter is 12.5th and 8th quantiles. Note that the width of the letters is not related to the underlying data.
We encountered ambiguous daisy-chains in many cases where A is similar to B, C and D. It is hard to tell A from B, B from C and C from D. But, at the same time, A and D (on the opposite ends of the spectrum) are so dissimilar that we would not expect to assign them the same cell type (Fig. 3k and Extended Data Fig. 4h). The graded or continuous variation has been observed in a number of locations in the nervous system, and is one of the classic problems of cell typing. The only reasonable option, if there is any, is for them to be lumped together because there isn’t compelling information to separate them. The fly brain is the defacto standard of hemibrain cell types, so we have been conservative in making these changes.
The scipy123 Python package is used to perform statistical analyses such as Pearson R or cosine distance. To determine statistical significance, we used either t-tests for normally distributed samples, or Kolmogorov–Smirnov tests otherwise.
Adapting the FAFB-FlyWire Dataset to Address Neuron Inhibition and Implications for Image Analysis and Visualization: I. Identification of the Neurons
The number of input connections to each mixing layer neuron is kept at a constant K for all neurons. It is definitely a simplification that can be corrected by introducing a distribution P(K) but this requires further detailed modelling.
APL’s global inhibition to all of the mixing layer are assumed to take a single value. The number of APL and mixing layer neurons would have an impact on the level of inhibition.
From Fig. A simple model showing the differences between the values of K found in the FlyWire left and FlyWire right datasets shows the shift towards lower values of K when compared with the hemibrain.
Consistency with the standard convention can be found in FlyWire data being mirrored. To facilitate this, we provide tools to digitally mirror FAFB-FlyWire data using the Python flybrains (https://github.com/navis-org/navis-flybrains) or natverse nat.jrcbrains (https://github.com/natverse/nat.jrcbrains) packages (Extended Data Fig. Through 1c.
Owing to the extensive post-processing of the FAFB dataset and derived datasets (for example, transformation fields, image mosaicing and stack registrations to produce aligned volumes, segmentation supervoxels, proofread neuron segmentations, skeletons, meshes and myriad 3D visualizations), which had been undertaken at the time at which this error was discovered, we deemed it impractical to correct this error at the raw data level. The flybrain is usually placed on the viewer’s left with a convention of presentation. The view one has of themselves from the perspective of the fly brain is the same view it places on the viewer. This maintains consistency with past publications. All the labels of left and right in the figures have been corrected, for example, the https://codex.flywire.ai, FAFB/FlyWire CAmAID and associated digital repository have been changed. In these resources, a neuron labelled as being on the left is indeed on the left of the fly’s brain.
Neuron categorization, sensory modality annotations and nerve assignments are described in detail in our companion paper12. One of the three flow classes are afferent, efferent and insturment. Intrinsic neurons had their entire arbor within the FlyWire brain dataset. This also included cells that were projected from the SEZ. Next, each flow class was divided into superclasses in the following way. Afferent is sensory. The visual projection from the central brain to the optic lobes is called visual liquefaction. Efferent is the endocrine, descending motor.
The main branch entering through the neck connective was one of the criteria that ANs were identified on.
To identify the types of DNs described in the ref. 107 in the EM dataset, we transformed the volume renderings of DN GAL4 lines into FlyWire space. Matching closely related neurons was enabled by displaying their cells in the same space. For DNs without available volume renderings, we identified candidate EM matches by eye, transformed them into JRC2018U space and overlaid them onto the GAL4 or Split GAL4 line stacks (named in ref. It is possible to get verification in FIJI for that type. We were able to identify all of them using the methods that we had used before and annotated their cell type with the published nomenclature. All of the otherDNs received a systematic cell type containing their soma location, an ‘e’ and a three digit number. There is a detailed account and analysis of the items on the list.
We classified the operatives based on their soma location in a previous report. In brief, the soma of DNa, DNb, DNc and DNd is located in the anterior half (a, anterior dorsal; b, anterior ventral; c, in the pars intercerebralis; d, outside cell cluster on the surface) and DNp in the posterior half of the central brain. DNg somas are located in the GNG.
For sensory neurons, side refers to whether they enter the brain through the left or the right nerve. In a small number of cases, we couldn’t tell if the side named ‘na’ was nerve entry or not.
If the cell type is a correction and the hemibrain is a refinement, then the cell type should take precedence. This generates the reported total count of 8,453 terminal cell types and includes 3,643 hemibrain-derived cell types (Fig. 3h (right side of the flow chart)) and 4,581 proposals for new types. New types consist of 3,504 CBXXXX types, 65 new visual centrifugal neuron types (‘c’ prefix, for example, cL08), 173 new VPN types (‘e’ suffix, for example, LTe07), 602 new AN types (‘AN_’ or ‘SA_’ prefix, for example, AN_SMP_1) and 237 new DN types (‘e’ suffix, for example, DNge094). There are 229 different cell types, some of which are known from other literature.
Visual sensory neurons (R1–6, R7–8 and ocellar photoreceptor neurons) were identified by manually screening neurons with pre-synapse in either the lamina, the medulla and/or the ocellar ganglia93.
The left hemisphere Johnston’s brain tissue was characterized based on innervation of A,B,C,D, E and F, but not further classified into subzone innervation as shown previously 104. The right hemisphere has other sensory cells such as bristle and taste peg that were identified through a comparison with their mirrored counterpart in the left hemisphere. The antennal lobes has thermosensory and hygrosensory cells, which were identified through their connections to the uniglomerular projection neurons.
In one paper, the researchers used a computer model of the fruit-fly brain to figure out what its connections were to each other. They were able to sense either bitter or sweet tastes with the use of activated cortexes. Motor neurons tied to the fly’s proboscis were triggered by the cascade of signals through the virtual fly’s brain. The signal for extending the proboscis was transmitted as if the insect was getting ready to eat, when the bitter circuit was activated. The team activated the same neurons in a fruit fly. The researchers believed the simulation was more than 90 percent accurate in predicting how the fly would behave.
An ItoLee Nomenclature for the Central Brain. I. hemilineage tracts and (hemi)neuronal lineages
The majority of VPNs (99.6%) and VCNs (98.3%) were assigned to specific types. Only a small amount of VPNs and VCNs were left untyped because they could not confidently assign a cell type.
Hemibrain types split across multiple clusters were double checked (for example, by running a triple-hemisphere connectivity clustering), which often led to a split of the hemibrain type; for example, SMP408 was split into SMP408a–d.
For VPNs the nomenclature follows the format [neuropil][C/T][e][XX], where neuropil refers to regions innervated by VPN dendrites; C/T denotes columnar versus tangential organization; e indicates identification through EM; and XX represents a zero padded two digit number.
Two systems for hemilineage are used in the existing literature. There are some parts of these systems that are not described in the other version of the system. In the main text, we provide (hemi)lineages according to the ItoLee nomenclature for simplicity. Below and in the Supplementary Information, we also provide both names as ItoLee/Hartenstein, and the mapping between the two nomenclatures is provided in Supplementary Data 3. It was expected that a total of around 114 lineages in the central brain were included in the previous literature. We were able to identify all 119 lineages based on light-level clones and tracts. There was only one lineage, which could not be compared to any clone. Thus, together, we have identified 120 lineages.
We can reconcile reports if we thoroughly inspected the hemilineage tracts originally in CATMAID and then in FlyWire. New to the refs. 33,34 (ItoLee nomenclature) are: CREl1/DALv3, LHp3/CP5, DILP/DILP, LALa1/BAlp2, SMPpm1/DPMm2 and VLPl5/BLVa3_or_4—we gave these neurons lineage names according to the naming scheme in refs. 33,34. New to the ref. 31 (Hartenstein nomenclature) are: SLPal5/BLAd5, SLPav3/BLVa2a, LHl3/BLVa2b, SLPpl3/BLVa2c, PBp1/CM6, SLPpl2/CP6, SMPpd2/DPLc6, PSp1/DPMl2 and LHp3/CP5—we named these units according to the Hartenstein nomenclature naming scheme. We did not take the following clones from ref. 33 into account for the total count of lineages/hemilineages, because they originate in the optic lobe and their neuroblast of origin has not been clearly demonstrated in the larva: VPNd2, VPNd3, VPNd4, VPNp2, VPNp3, VPNp4, VPNv1, VPNv2 and VPNv3.
Notably, although light-level clones from refs. 33,34 match very well the great majority of the time, sometimes clones with the same name only match partially. The AOTUv1/DAL cm2 clone seems to have a missing hemilineage. There appears to be a similar situation for the lineages. When there is a conflict, we prefer cloneds. 34.
The coconatfly package provides a streamlined interface to help carry out clustering. For example the following command can be used to see if the types given to a selection of neurons in the Lateral Accessory Lobe (LAL) are robust:
The optional interactive mode is an efficient way to explore a web browser. For further details and examples, see https://natverse.org/coconatfly/.
Clustering Hemibrain Types and the Number of Connections Input into a Class of KCs II: Global Inhibition by APL
A hemibrain type is a mix of two or more; if the clusters contained two or more hemibrain types, they were double checked and corrected.
and how it changes with respect to K, the number of input connections. What are the numbers of connections K into individual KCs that maximize their responses, and how do they do it?
More detailed calculations can be found in a previous report122. Weighted by Randomized and Heterogeneous weights, each row of (BBW) has K elements that are 1 and N K elements that are. The global inhibition by APL is represented by the parameters. The value inhibition was set to be α = A/M, where A = 100 is an arbitrary constant and M is the number of KCs in each of the three datasets. The primary quantity of interest is the dimension of the KC activities defined by122:
CAVE: The Flywire Reservoir: Tracking the Accuracy and Performance of Test Cells in the Convolutional Neural Network
The professional proofreading team received additional proofreading training. Correct proofreading relies on a diverse array of 2D and 3D visual cues. Proofreaders learned about 3D morphology, resulting from false merger or false split without knowing what types of cells they are. The types of ultrastructures studied by proof readers provided valuable 2D cues and were reliable guides for accurate tracing. Each of the professionals practiced on an average of 200 cells before they were admitted to Production. In this dataset, we determined the accuracy of test cells by comparing them to ground-truth reconstructions. To improve proofreading quality, peer learning was highly encouraged.
Any estimate of how long it took to create the Flywire resource is not reliable because of the dispersed nature of the community, the interlacing of analysis and the variability of how it was done. The second public release, version 783, required 3,013,513 edits. We had a measurement of the time it took for us to find the right cell in the central brain. We collected timings and number of edits for 29,135 independent proofreading tasks after removing outliers with more than 500 edits. We were able to determine an average time per edit. However, we observed that proofreading times per edit were much higher for proofreading tasks that required few edits (<5). Our measurement wasn’t representative of the second round of proofreading, because it went over all segments with > 100 synapses. The edits usually require 1–5. We adjusted for that by limiting the computations to a subset of timed tasks, and then estimating the speeds of the two rounds. The average time per edit is the average time for tasks with 1–5 edits. We averaged these times for an overall proofreading time because the number of tasks in each category were similar. The result was an average time of 78 s per edit which adds up to an estimate of 34.1 person-years if we assume a 2,000 h work year.
FlyWire uses CAVE50 for hosting the proofreadable segmentation and all of its annotations. The PyChunkedGraph is a part of CAVE’s system and described in detail elsewhere.
The Locations of the Presynaptic Substructures in the Nervous System. Volume Estimation from the Neuroprint-v1.2.1 Dataset
The location of the presynaptic locations were assigned to the Neuropils. We used the project’s website to calculate whether the location was within the neuropil mesh and assigned it to the sphinx. Some synapses remained unassigned after this step because the neuropils only resemble rough outlines of the underlying data. If the synapse was less than 10 m from it, all remaining sphinxes were assigned to the closest one. The remaining synapses were left unassigned.
We used the mesh of each neuropil to calculate the volume. In the aggregated volumes presented in the paper we assigned the lamina, medulla, accessory medulla, lobula and lobula plate to the optic lobe. The ocellar ganglion was placed in the central brain.
$$\begin{array}{l}P\,=\,\frac{{\rm{TP}}}{{\rm{TP}}+{\rm{FP}}}\ R\,=\,\frac{{\rm{TP}}}{{\rm{TP}}+{\rm{FN}}}\ {\rm{F1}}\,=\,\frac{2\times P\times R}{P+R}\end{array}$$
We retrieved the latest completion rates and synapse numbers for the hemibrain from neuprint (v1.2.1). In some cases, neuropil comparisons were not directly possible because of redefined regions in the hemibrain dataset. The regions were not included in the comparison.
Whether or not a neuron is innate to a region depends on the location of its pontine connections and its arbor. In other words, the neurites of an intrinsic neuron are allowed to exit the region, provided that they do not make synapses after leaving. The diagrams in White et al.31 provided information about C. elegans sphinx locations.
It should be understood that this estimate has ‘error bars’ because of definitional ambiguities. There is some question as to whether motor neurons are intrinsic neurones and could possibly be removed from the list. Or the brain could be enlarged by moving the posterior border further behind the excretory pore, which would add 10 neurons (RIF, RIG, RMG, ADE and ADA). 35 10 intrinsic neurons are what we estimate to make the ambiguities explicit. Of the 302 CNS neurons, 180 make synapses in the brain126. Intrinsic to the brain are between 15 to 25% of brain neurons and 8 to 15% of the central nervous system.
We calculated cell volumes and surface areas using CAVE’s L2Cache50. Volumes were calculated by summing up the voxels within the cell segment and using the voxel resolution as a factor. Area calculations were more complicated and performed by overlap in the voxel space. They pulled the overlap of true and false voxels by shifting the binarized segment. For each dimension, we counted the extracted voxels and multiplied the count by the voxel resolution of the given dimensions. Finally, we added up per dimension area estimates. This measurement will underestimate area slightly, but smoothed estimates are too compute intensive.
The scepter was trained on truth from the CREMI challenge. Three 5 5 5 m cubes are from the calyx of FAFB14 and contained in three CREMI datasets. While the classifier from Buhmann et al. was trained and validated on only this dataset, they evaluated its performance on multiple regions (calyx, lateral horn, ellipsoid body and protocerebral bridge). It’s important to mention that the performance varies by region.
Eckstein et al.10 created a machine learning model to predict neurotransmitter identities for all synapses from Buhmann et al. The imagery was derived through the use of electron microscopy. Each synapse was assigned a probability for one of six neurotransmitters: acetylcholine, glutamate, GABA, serotonin, dopamine and octopamine. They used neurotransmitter identities published for individual neuronal cell types and built a dataset with 3,025 neurons with known transmitter type assuming Dale’s law applies. According to the report, a per neuron accuracy of 94% and a perapse accuracy of 87% were reported.
For each neuron, we calculated the fraction of presynapses in the left and right hemisphere. The hemisphere opposite its dominant input side was named the contralateral hemisphere. Most of the presynapses or most of the postsynapses in the centre region were not excluded.
We used the information flow algorithm to rank each neuron starting with a set of seeds. The algorithm traverses the synapse graph of neurons probabilistically. The likelihood of a neuron being added to the traversed set increased linearly with the fraction of synapses it receives from already traversed neurons up to 30% and was guaranteed above this threshold. We repeated the rank calculation for all sets of afferent neurons as seed as well as the whole set of sensory neurons. The groups we used are: olfactory receptor neurons, gustatory receptor neurons, mechanosensory Johnston’s organ neurons, head and neck bristle mechanosensory neurons, mechanosensory taste peg neurons, thermosensory neurons, hygrosensory neurons, VPNs, visual photoreceptors, ocellar photoreceptors and ascending neurons.
Input seeds were created by combining all the listed modalities and removing the visual sensory groups.
We averaged 10,000 runs for each of the 10 different therapies we performed. We then ordered the neurons according to their rank and assigned them a percentile based on their location in the order. To compute a reduced dimensionality, we treated the vector of all ranks (one for each modality) as neuron embedding and calculated two dimensional embeddings using UMAP129 with the following parameters: n_components=2, min_dist=0.35, metric = “cosine”, n_neighbors=50, learning_rate = .1, n_epochs=1000.
Annotating a Sensory Wiring Circuit: The Discovery of Eight Thousand Neurons in the COVID-19 Epidemic Team
But these tools aren’t perfect, and the wiring diagram needed to be checked for errors. Volunteers were invited to help the scientists because they spent so much time manually looking at the data. According to the co-author, Gregory Jefferis a neuroscientist at the University of Cambridge, more than 3 million edits were made by members and volunteers. He notes that many of this work was done in 2020 when researchers were working from home during the COVID-19 epidemic.
The team was surprised by the ways in which the different cells communicate. For instance, neurons that were thought to be involved in just one sensory wiring circuit, such as a visual pathway, tended to receive cues from multiple senses, including hearing and touch1. Murthy says that the brain is very connected.
But the work wasn’t finished: the map still had to be annotated, a process in which the researchers and volunteers labelled each neuron as a particular cell type. Jefferis compares the task to assessing satellite images: AI software might be trained to recognize lakes or roads in such images, but humans would have to check the results and name the specific lakes or roads themselves. More than anyone had expected, the researchers identified over eight thousand types of neuron. Of these, 4,581 were newly discovered, which will create new research directions, Seung says. “Every one of those cell types is a question,” he adds.
There are nine papers about the data published in Nature today. Mala Murthy and Sebastian Seung are the brains behind FlyWire and they are both at Rutgers University in New Jersey.
Clay Reid, who was not a part of the project but worked on one of the team members, said it was a huge deal. For a long time, the world has been waiting for it.
