Global Highlights: Neuroengineering towards Whole-Brain Emulation and Mind Uploading

Logan Thrasher Collins

Key events in neurotechnology from 2006 to 2018.

Blue Brain Project Cortical Column Simulation¹ (2006)

• Mapped about 10,000 neurons in 2-week-old rat somatosensory neocortical columns with enough resolution to show the rough spatial locations of the dendrites and synapses.
• After constructing a virtual model, algorithmic adjustments refined the spatial connections between neurons to increase accuracy (10,000 neurons and over 10 million synapses).
• Emulated the cortical column using the Blue Gene/L supercomputer.
• The emulation demonstrated high accuracy with respect to experimental data.

Hippocampal Prosthesis in Rats² (2012)

• Theodore Berger and his team developed an artificial replacement for neurons that transmit information from the CA3 region to the CA1 region of the hippocampus.
• This cognitive prosthesis employs a multi-input multi-output (MIMO) model to encode the information in CA3 and send it to CA1.
• The prosthesis was shown to restore and enhance memory in rats (as evaluated by behavioral testing and brain imaging).

In Vivo Superresolution Microscopy for Brain Imaging³ (2012)

• Stefan Hell (2014 Nobel laureate in chemistry) developed stimulated emission depletion microscopy (STED), a type of superresolution fluorescence microscopy which allowed imaging of synapses and dendritic spines.
• STED microscopy uses transgenic neurons which express fluorescent proteins and fluoresce sequentially. Although the wavelength of the light used for imaging would ordinarily limit the resolution (the diffraction limit), STED’s temporal contrast overcomes this limitation.
• Transgenic mice with glass-sealed holes in their skulls over their somatosensory cortices were imaged using STED (they were anesthetized during this process). Synapses and dendritic spines were observed up to fifteen nanometers below the surface of the somatosensory cortex.

Eyewire: Crowdsourcing Method for Retina Mapping⁴ (2012)

• The Eyewire project was initiated by Seung’s group. It is a crowdsourcing initiative for mapping the connectome in the retina and uncovering its neural circuits.
• Laboratories first collect structural data from tissue in the retina using serial electron microscopy as well as functional data using two-photon microscopy.
• In the Eyewire game, slices of imaging data are provided to players. The players then help reconstruct neural morphologies and circuits by “coloring in” the parts of the images that correspond to cells and then stacking many images on top of each other to generate 3D images. Artificial intelligence helps to provide initial “best guesses” and guide the players, but the players ultimately perform the task of reconstruction.
• By November 2013, around 82,000 participants had played the game and its popularity continues to grow. Over time, Eyewire has shown great successes in reconstructing neurons and circuits.

The BRAIN Initiative⁵ (2013)

• The BRAIN Initiative (Brain Research through Advancing Innovative Technologies) provided neuroscience with $110 million in governmental funding and $122 million in funding from private sources such as the Howard Hughes Medical Institute and the Allen Institute for Brain Science.
• The BRAIN Initiative focused on funding research which develops and utilizes new technologies for functional connectomics. It helped to accelerate research on tools for decoding the mechanisms of neural circuits in order to understand and treat mental illness, neurodegenerative diseases, and traumatic brain injury.
• The BRAIN Initiative emphasized collaboration between neuroscientists and physicists. It also pushed forward nanotechnology-based methods to image neural tissue, record from neurons, and otherwise collect neurobiological data.

The CLARITY Method for Making Brains Translucent⁶ (2013)

• Deisseroth and his colleagues developed a method called CLARITY to make samples of neural tissue optically translucent without damaging the fine cellular structures in the tissue. This method was even able to make entire mouse brains transparent.
• Mouse brains were infused with hydrogel monomers (acrylamide and bisacrylamide) as well as formaldehyde and some other compounds for facilitating crosslinking. Next, the hydrogel monomers were crosslinked by incubating the brains at 37°
• Lipids in the hydrogel-stabilized mouse brains were extracted using hydrophobic organic solvents and electrophoresis.
• CLARITY allows antibody labeling, fluorescence microscopy, and other optically-dependent techniques to be used for imaging entire brains. In addition, it renders the tissue permeable to macromolecules, which broadens the types of experimental techniques that these samples can undergo (i.e. macromolecule-based stains, etc.)

Telepathic Rats Engineered Using Hippocampal Prosthesis⁷ (2013)

• Berger’s hippocampal prosthesis was implanted in pairs of rats.
• When “donor” rats were trained to perform a task, the donor rats developed neural representations (memories) that were encoded in their hippocampal prostheses.
• The donor rat memories were processed by the MIMO model and transmitted to the hippocampal prostheses of untrained “recipient” rats. After receiving the memories, the recipient rats performed significantly better on the task that they had not been trained to perform.

Integrated Information Theory 3.0⁸ (2014)

• Integrated information theory (IIT) was originally proposed by Giulio Tononi in 2004. IIT is a quantitative theory of consciousness which may help explain the hard problem of consciousness.
• IIT begins by assuming phenomenological axioms. These include that each experience is specifically characterized by how it differs from other experiences, each experience unified (cannot be reduced to interdependent parts), and the boundaries which distinguish individual experiences are describable as having specific “spatiotemporal grains.”
• From these phenomenological axioms and the assumption of causality, IIT identifies maximally irreducible conceptual structures (MICS) associated with each experience. MICS represent particular patterns of qualia which form unified percepts.
• IIT also outlines a mathematical measure of an experience’s quantity. This measure is called integrated information or φ.

Japan’s Brain/MINDS Project⁹ (2014)

• The Brain/MINDS (Brain Mapping by Integrated Neurotechnologies for Disease Studies) project was initiated in Japan to further understanding of the brain. This project received almost $30 million in funding for its first year alone.
• Brain/MINDS focuses on studying the brain of the common marmoset (a non-human primate that is abundant in Japan), developing new technologies for brain mapping, and understanding the human brain with the goal of finding new treatments for brain diseases.

Openworm¹⁰ (2014)

• The anatomical elegans connectome was originally mapped in 1976 by Albertson and Thomson.
• More data has since been collected on neurotransmitters, electrophysiology, cell morphology, and other characteristics of the elegans connectome.
• Szigeti, Larson, and their colleagues made an online platform for crowdsourcing research on elegans computational neuroscience, with the goal of completing an entire “simulated worm.”
• They also released software called Geppetto. This program allows users to manipulate both multicompartmental Hodgkin-Huxley models and highly efficient soft-body physics simulations (for modeling the worm’s electrophysiology and anatomy).

SyNAPSE Program of DARPA and IBM¹¹ (2015)

• The TrueNorth neuromorphic computing chip was constructed and validated by DARPA and IBM.
• TrueNorth uses circuit modules which mimic neurons. Inputs to these fundamental circuit modules must overcome a threshold in order to trigger “firing.”
• TrueNorth’s neuromorphic architecture enables emulation of up to 1 million neurons with over 250 million synapses.
• The chip requires far less power than traditional computing devices.
• TrueNorth can be programmed to emulate any arrangement of biological neurons.

Human Brain Project Cortical Mesocircuit Reconstruction and Simulation¹² (2015)

• The HBP achieved digital reconstruction of a 0.29 mm³ section of rat cortical tissue (31,000 neurons and 37 million synapses) based on a partial map, morphological data, “connectivity rules,” and additional known datasets.
• This mesocircuit was emulated using the Blue Gene/Q supercomputer and a few accessory hardware components.
• The emulation demonstrated enough accuracy to reproduce emergent neurological processes and yield new insights on how these processes function.

Neural Lace¹³ (2015)

• Charles Lieber’s and his group developed a syringe-injectable electronic mesh with sub-micrometer-thick wiring that can be used for neural interfacing.
• They constructed the meshes from flexible, biocompatible electronics. When injected, the neural lace expands to cover and record from centimeter-scale regions of tissue.
• Neural lace may allow for “invasive” brain-computer interfaces to be less invasive by removing the need for surgical implantation.
• Lieber has continued to develop this technology towards clinical application.

Neural Dust¹⁴ (2016)

• Maharbiz developed implantable, ~1 mm biosensors for wireless neural recording and tested them in rats (this “neural dust” could be miniaturized to less than 0.5 mm using customized electronic components).
• Neural dust motes consisted of two recording electrodes, a transistor, and a piezoelectric crystal.
• The neural dust received external power from ultrasound. Neural signals were recorded by measuring disruptions to the reflection of the ultrasound waves by the piezoelectric crystal. Signal processing techniques enabled precise measurement of activity.

Hippocampal Prosthesis in Monkeys¹⁵ (2016)

• Berger continued developing his cognitive prosthesis. Testing in Rhesus Macaques was performed.
• As with the rats, monkeys with the implant showed substantially improved performance on memory tasks. Other tests supported these results.

The China Brain Project¹⁶ (2016)

• The China Brain Project was launched to improve understanding of cognition’s neural mechanisms, develop brain research technology platforms, develop preventative and diagnostic interventions for brain disorders, and improve brain-inspired artificial intelligence technologies.
• This project will be carried out from 2016 to 2030 with the goal of completing mesoscopic brain circuit maps.
• China’s population of non-human primates and preexisting non-human primate research facilities give the China Brain Project an advantage. The project will focus on studying rhesus macaques.

Somatosensory Cortex Stimulation for Spinal Cord Injuries¹⁷ (2016)

• Gaunt, Flesher, and colleagues found that microstimulation of the primary somatosensory cortex (S1) could partially restore tactile sensations to a patient with a spinal cord injury.
• Electrode arrays were implanted into the S1 regions of a patient with a spinal cord injury. The array performed intracortical microstimulation over a period of six months.
• The patient reported the locations and perceptual qualities of the sensations elicited by microstimulation. The patient did not experience pain or “pins and needles” from any of the stimulus trains. Overall, 93% of the stimulus trains were reported as “possibly natural.”
• Results from this study might be used to engineer upper-limb neuroprostheses which provide somatosensory feedback.

The $100 Billion Softbank Vision Fund¹⁸ (2017)

• Masayoshi Son, the CEO of Softbank (a Japanese telecommunications corporation), announced a plan to raise $100 billion in venture capital to invest in artificial intelligence. This plan involved partnering with multiple other companies in order to raise this enormous amount of capital.
• By the end of 2017, the Vision Fund successfully reached its $100 billion goal. Masayoshi Son has since announced further plans to continue raising money with a new goal of over $800 billion.
• Masayoshi Son’s reason for these massive investments is the Technological Singularity. He agrees with Kurzweil that the Singularity will likely occur at around 2045 and he hopes to help bring the Singularity to fruition. Though Son is aware of the risks posed by artificial superintelligence, he feels that superintelligent AI’s potential to tackle some of humanity’s greatest challenges (such as climate change and the threat of nuclear war) outweighs those risks.

Bryan Johnson Launches Kernel¹⁹ (2017)

• Entrepreneur Bryan Johnson invested $100 million to start Kernel, a neurotechnology company.
• Kernel will develop implants which allow for recording and stimulation of large numbers of neurons at once. The initial goal of the company is to develop treatments for mental illnesses and neurodegenerative diseases, while its long-term goal is to enhance human intelligence.
• Kernel was originally working with Theodore Berger and planning to use his hippocampal prosthesis. Unfortunately, Berger and Kernel parted ways after about six months because Berger’s vision was reportedly too long-range to support a financially viable company (at least for now).
• Kernel was originally a company called Kendall Research Systems. This company was started by a former member of the Boyden lab. Edward Boyden is a professor at MIT who specializes in neuroengineering and synthetic biology. In total, four members of Kernel’s team are former Boyden lab members.

Elon Musk’s Launches NeuraLink²⁰ (2017)

• Elon Musk (CEO of Tesla, SpaceX, and a number of other successful companies) initiated a neuroengineering venture called NeuraLink.
• NeuraLink will begin by developing brain-computer interfaces (BCIs) for clinical applications, but the ultimate goal of the company is to enhance human intelligence in order to keep up with artificial intelligence.
• Though many of the details around NeuraLink’s research are not yet open to the public, it has been rumored that injectable electronics similar to Lieber’s neural lace might be involved.

Facebook Announces an Effort to Build Brain-Computer Interfaces²¹ (2017)

• Facebook revealed research on constructing non-invasive brain-computer interfaces (BCIs) at a company-run conference. This initiative is run by Regina Dugan, Facebook’s head of R&D at division building 8.
• Researchers at Facebook are working on a non-invasive BCI which may eventually enable users to type one hundred words per minute with their thoughts alone. This builds on past research which has been used to help paralyzed patients.
• The building 8 group is also developing a wearable device for “skin hearing.” Using just a series of vibrating actuators which mimic the cochlea, test subjects have so far been able to recognize up to nine words. Facebook intends to vastly expand this device’s capabilities.

DARPA Funds Research to Develop Better Brain-Computer Interfaces²² (2017)

• The U.S. government agency DARPA awarded $65 million in total funding to six research groups.
• The recipients of this grant included five academic laboratories (headed by Arto Nurmikko, Ken Shepard, Jose-Alain Sahel and Serge Picaud, Vicent Pieribone, and Ehud Isacoff) and one small company called Paradromics Inc.
• DARPA’s goal for this initiative is to develop a nickel-sized bidirectional brain-computer interface (BCI) which can record from and stimulate up to one million individual neurons at once.

Hippocampal Prosthesis Algorithm in Humans²³ (2017)

• Dong Song (who was working alongside Berger) tested the MIMO algorithm on human epilepsy patients by using implanted recording and stimulation electrodes. The full prosthesis was not implanted, but these electrodes acted similarly, though in a temporary capacity.
• Although only two patients were tested in this study, many trials were performed to at least partly compensate for the small sample size. More thorough testing will occur soon.
• Hippocampal spike trains from individual cells in CA1 and CA3 were recorded from the patients during a delayed match-to-sample task. The patients were shown various images while these data were recorded and then processed by the MIMO model. Then the patients were asked to recall which image they had been shown previously by picking it from a group of “distractor” images. Memories encoded by the MIMO model were used to stimulate hippocampal cells during the recall phase.
• Compared to controls in which the same two epilepsy patients were not assisted by the algorithm and stimulation, the experimental trials demonstrated a significant increase in successful pattern matching.

Brain Imaging Factory in China²⁴ (2017)

• Qingming Luo initiated the HUST-Suzhou Institute for Brainsmatics, a brain imaging “factory.”
• Each of the numerous machines in Luo’s facility performs automated processing and imaging of tissue samples. The devices make ultrathin slices of brain tissue using diamond blades. The machines also treat the samples with fluorescent stains or other contrast-enhancing chemicals. After sample preparation, the tissue slices are imaged by fluorescence microscopy.
• The institute has already demonstrated its potential by mapping the structure of an extremely long neuron which “wraps around” the entire mouse brain.

Automated Patch-Clamp Robot for In Vivo Neural Recording²⁵ (2017)

• Boyden and his colleagues developed a robotic system to automate patch-clamp recordings from individual neurons. Its data collection yield is similar to that of skilled human experimenters.
• By continuously imaging neural tissue using two-photon microscopy, the robot can adapt to a target cell’s movement and shift the pipette to compensate. This adaptation is facilitated by a new type of algorithm called imagepatching. As the pipette approaches its target, the imagepatching algorithm adjusts the pipette’s trajectory based on the real-time two-photon microscopy.
• This system was tested in vivo using mice. It can be used in vivo so long as the target cells express a fluorescent marker or otherwise fluoresce corresponding to their size and position.

Genome Editing in the Mammalian Brain²⁶ (2017)

• Precise genome editing in the brain has historically been challenging because most neurons are postmitotic (non-dividing) and this state prevents homology-directed repair (HDR) from occurring. HDR is a mechanism of DNA repair which allows for targeted insertions of DNA fragments with overhangs that are homologous to the region of interest (by contrast, non-homologous end-joining is highly unpredictable).
• Nishiyama, Mikuni, and Yasuda developed a technique which allows genome editing in postmitotic mammalian neurons. They used adeno-associated viruses (AAVs) and CRISPR-Cas9 to accomplish this.
• The AAVs delivered ssDNA sequences encoding a single guide RNA (sgRNA) and an insert. Inserts encoding a hemagglutinin tag (HA) and inserts encoding EGFP were both employed. Cas9 was encoded endogenously by transgenic host cells and host animals.
• This method successfully demonstrated precise genome editing in vitro and in vivo with a low rate of off-target effects. The inserts did not cause deletion of nearby endogenous sequences in 98.1% of infected neurons.

 

References

(1) Markram, H. (2006). The Blue Brain Project. Nature Reviews Neuroscience, 7(2), 153-160. doi:10.1038/nrn1848

(2) Berger, T. W., Song, D., Chan, R. H., Marmarelis, V. Z., Lacoss, J., Wills, J.,… Granacki, J. J. (2012). A Hippocampal Cognitive Prosthesis: Multi-Input, Multi-Output Nonlinear Modeling and VLSI Implementation. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 20(2), 198-211. doi:10.1109/tnsre.2012.2189133

(3) Berning, S., Willig, K. I., Steffens, H., Dibaj, P., & Hell, S. W. (2012). Nanoscopy in a Living Mouse Brain. Science, 335(6068), 551-551. doi:10.1126/science.1215369

(4) Marx, V. (2013). Neuroscience waves to the crowd. Nature Methods, 10(11), 1069-1074. doi:10.1038/nmeth.2695

(5) The White House Office of the Press Secretary. (2013, April 2). Fact Sheet: BRAIN Initiative [Press release]. Retrieved from https://obamawhitehouse.archives.gov/the-press-office/2013/04/02/fact-sheet-brain-initiative

(6) Chung, K., Wallace, J., Kim, S., Kalyanasundaram, S., Andalman, A. S., Davidson, T. J.,… Deisseroth, K. (2013). Structural and molecular interrogation of intact biological systems. Nature, 497(7449), 332-337. doi:10.1038/nature12107

(7) Deadwyler, S. A., Berger, T. W., Sweatt, A. J., Song, D., Chan, R. H., Opris, I.,… Hampson, R. E. (2013). Donor/recipient enhancement of memory in rat hippocampus. Frontiers in Systems Neuroscience, 7(120). doi:10.3389/fnsys.2013.00120

(8) Oizumi, M., Albantakis, L., & Tononi, G. (2014). From the Phenomenology to the Mechanisms of Consciousness: Integrated Information Theory 3.0. PLoS Computational Biology, 10(5). doi:10.1371/journal.pcbi.1003588

(9) Okano, H., Miyawaki, A., & Kasai, K. (2015). Brain/MINDS: brain-mapping project in Japan. Philosophical Transactions of the Royal Society B: Biological Sciences, 370(1668). doi:10.1098/rstb.2014.0310

(10) Szigeti, B., Gleeson, P., Vella, M., Khayrulin, S., Palyanov, A., Hokanson, J.,… Larson, S. (2014). OpenWorm: an open-science approach to modeling Caenorhabditis elegans. Frontiers in Computational Neuroscience, 8. doi:10.3389/fncom.2014.00137

(11) Akopyan, F., Sawada, J., Cassidy, A., Alvarez-Icaza, R., Arthur, J., Merolla, P.,… Modha, D. S. (2015). TrueNorth: Design and Tool Flow of a 65 mW 1 Million Neuron Programmable Neurosynaptic Chip. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 34(10), 1537-1557. doi:10.1109/tcad.2015.2474396

(12) Markram, H., Muller, E., Ramaswamy, S., Reimann, M., Abdellah, M., Sanchez, C.,… Schürmann, F. (2015). Reconstruction and Simulation of Neocortical Microcircuitry. Cell, 163(2), 456-492. doi:10.1016/j.cell.2015.09.029

(13) Liu, J., Fu, T., Sheng, Z.,… Lieber, C. M. (2015) Syringe-Injectable electronics. Nature Nanotechnology. 10(7), 629–636, doi:10.1038/nnano.2015.115.

(14) Seo, D., R. M., Neely, K., Shen,… Maharbiz, M. M. (2016). Wireless Recording in the Peripheral Nervous System with Ultrasonic Neural Dust. Neuron. 91(3), 529–539, doi:10.1016/j.neuron.2016.06.034.

(15) Deadwyler, S. A., Hampson, R. E., Song, D., Opris, I., Gerhardt, G. A., Marmarelis, V. Z., & Berger, T. W. (2016). A cognitive prosthesis for memory facilitation by closed-loop functional ensemble stimulation of hippocampal neurons in primate brain. Experimental Neurology. doi:10.1016/j.expneurol.2016.05.031

(16) Poo, M., Du, J., Ip, N., Xiong, Z., Xu, B., & Tan, T. (2016). China Brain Project: Basic Neuroscience, Brain Diseases, and Brain-Inspired Computing. Neuron, 92(3), 591-596. doi:10.1016/j.neuron.2016.10.050

(17) Flesher, S. N., Collinger, J. L., Foldes, S. T., Weiss, J. M., Downey, J. E., Tyler-Kabara, E. C.,… Gaunt, R. A. (2016). Intracortical microstimulation of human somatosensory cortex. Science Translational Medicine, 8(361). doi:10.1126/scitranslmed.aaf8083

(18) Lomas, N. (2017) “Superintelligent AI explains Softbank’s push to raise a $100BN Vision Fund.” TechCrunch, 27 Feb. 2017, https://techcrunch.com/2017/02/27/superintelligent-ai-explains-softbanks-push-to-raise-a-100bn-vision-fund/. Accessed 1 Jan. 2018.

(19) Regalado, A. (2017). “Silicon Valley’s race to develop a brain-computer interface.” MIT Technology Review, 16 Mar. 2017. https://www.technologyreview.com/s/603771/the-entrepreneur-with-the-100-million-plan-to-link-brains-to-computers/. Accessed 28 Aug. 2017.

(20) Etherington, Darrell. “Elon Musk’s Neuralink wants to boost the brain to keep up with AI.” TechCrunch, 27 Mar. 2017, techcrunch.com/2017/03/27/elon-musks-neuralink-wants-to-boost-the-brain-to-keep-up-with-ai/. Accessed 29 Aug. 2017.

(21) Constine, J. “Facebook is building brain-computer interfaces for typing and skin-hearing.” TechCrunch, 19 Apr. 2017, https://techcrunch.com/2017/04/19/facebook-brain-interface/. Accessed 28 Aug. 2017.

(22) Hatmaker, Taylor. “DARPA awards $65 million to develop the perfect, tiny two-way brain-computer interface.” TechCrunch, 10 July 2017, techcrunch.com/2017/07/10/darpa-nesd-grants-paradromics/. Accessed 29 Aug. 2017.

(23) Song, D., She, X., Hampson, R. E., Deadwyler, S. A., & Berger, T. W. (2017). Multi-resolution multi-trial sparse classification model for decoding visual memories from hippocampal spikes in human. 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). doi:10.1109/embc.2017.8037006

(24) Cyranoski, D. (2017). China launches brain-imaging factory. Nature, 548(7667), 268-269. doi:10.1038/548268a

(25) Suk, H., Welie, I. V., Kodandaramaiah, S. B., Allen, B., Forest, C. R., & Boyden, E. S. (2017). Closed-Loop Real-Time Imaging Enables Fully Automated Cell-Targeted Patch-Clamp Neural Recording In Vivo. Neuron, 95(5). doi:10.1016/j.neuron.2017.08.011

(26) Nishiyama, J., Mikuni, T., & Yasuda, R. (2017). Virus-Mediated Genome Editing via Homology-Directed Repair in Mitotic and Postmitotic Cells in Mammalian Brain. Neuron, 96(4). doi:10.1016/j.neuron.2017.10.004