Boulder Future Salon

Transistors can be made out of perovskites, the material people keep saying will be used for solar cells yet we still never see them anywhere. Anyway, scientists "have found a solution to the perovskite problem, enabling the material to work as a transistor at room temperature. The device has been 'tricked' into ignoring the material's ion content by shunting these electrically charged species away from the gate to a different part of the transistor, where they can't interfere with the flow of current."

"We modified the construction of the transistors instead of modifying the material, resulting in a transistor with an extra, auxiliary gate. Ions are then pushed to the auxiliary gate and fixed in position."

"The auxiliary gate was created by depositing a ferroelectric layer onto the transistor. Ferroelectrics (dielectric materials that show stable polarisation when the external electric field is turned off) can induce a large surface charge that attracts ions and holds them in position, thereby freeing up the gate for the flow of electrons."

The article touts this development as "green" but fails to mention that perovskites are usually made with toxic heavy metals such as lead. I don't think they are really "green" but they are touted as "green" in solar cells because they are so much cheaper than silicon solar panels, which should theoretically lead to them being everywhere. My understanding is that the reason this hasn't happened is because perovskite solar cells wear out quickly while silicon lasts for decades.

Still, as CMOS transistors hit their limits, the world is looking for alternatives to silicon, and it's interesting to know it's possible to make transistors with perovskites. I never would've expected that.

"We used a dye to make the brain blood vessels visible under fluorescent light, using a technique known as two-photon microscopy. In this way, we could directly observe the red blood cells in capillaries of the neocortex in non-anesthetized mice."

"There was a massive flow of red blood cells through the brain capillaries during REM sleep, but no difference between non-REM sleep and the awake state, showing that REM sleep is a unique state."

"The research team then disrupted the mice's sleep, resulting in 'rebound' REM sleep -- a stronger form of REM sleep to compensate for the earlier disruption. Blood flow in the brain was further increased during rebound REM sleep, suggesting an association between blood flow and REM sleep strength."

The researchers went one step further and did the experiment on mice with the gene for a brain receptor called adenosine A2a receptors knocked out. You have to wonder how they think of these things. Apparently adenosine A2a receptors are what keep you awake if you drink coffee -- or more precisely, adenosine A2a receptors getting blocked. As you might guess at this point, the knockout mice had less of an increase in blood flow during REM sleep, and less during rebound REM sleep as well.

"The peopling of Polynesia was a stunning achievement: Beginning around 800 CE, audacious Polynesian navigators in double-hulled sailing canoes used the stars and their knowledge of the waves to discover specks of land separated by thousands of kilometers of open ocean. Within just a few centuries, they had populated most of the Pacific Ocean's far-flung islands."

Geneticists "compared the DNA of 430 modern individuals from all across Polynesia, and then eliminated later genetic input from European people. Because the researchers knew Polynesians had journeyed stepwise from island to island, their genetic analysis utilized a genetic phenomenon known as a population bottleneck. When a few dozen to a few hundred individuals from already-isolated island populations settled a new island, and then a subset of that group left to settle an additional island, and so forth, their genetic diversity would have shrunk with each voyage."

"To estimate how many generations went by between each island discovery, the scientists measured the length of shared genomic sequences between founder populations. Together, the data showed who descended from whom. That made it possible to not only show that two populations were related, but which came first."

"Canoes set sail from the shores of Samoa -- more than 2000 kilometers north of New Zealand -- around 800 CE. The explorers arrived first on Rarotonga, the largest island in a chain now called the Cook Islands. Successive explorers moved in all directions, island hopping over the course of centuries and eventually reaching all the way to Rapa Nui, 6500 kilometers from Samoa and 3700 kilometers off the coast of Chile, by 1210 CE."

"Three island cultures known for carving massive stone statues -- Rapa Nui, Raivavae, and the North and South Marquesas -- shared a common founder population in the Tuamotu Islands, even though they are thousands of kilometers apart and geographically closer to other parts of the Pacific."

"Those three islands also hold the earliest genetic traces of Native American ancestry among Polynesians. That suggests ancient Polynesians first contacted the Americas around 1100 CE, when the seafarers were beginning their last, and longest, expeditions."

"Machine learning algorithm revolutionizes how scientists study behavior." Hmm what do they mean by "behavior"?

"Previously, the standard method to capture animal behavior was to track very simple actions, like whether a trained mouse pressed a lever or whether an animal was eating food or not. Alternatively, the experimenter could spend hours and hours manually identifying behavior, usually frame by frame on a video, a process prone to human error and bias."

"Hsu, a biological sciences PhD candidate, realized he could let an unsupervised learning algorithm do the time-consuming work. B-SOiD discovers behaviors by identifying patterns in the position of an animal's body. The algorithm works with computer vision software and can tell researchers what behavior is happening at every frame in a video."

"It uses an equation to consistently determine when a behavior starts. Once you reach that threshold, the behavior is identified, every time."

How does it do all this? It's a combination of a clustering algorithm and a classifier. Well, first you employ a pose estimation algorithm (this system doesn't reinvent pose estimation software, it just uses existing pose estimation software). Then...

"B-SOiD extracts the spatiotemporal relationships between all position inputs (speed, angular change, and distance between tracked points). After embedding these high-dimensional measurements into a low-dimensional space UMAP, a state-of-the-art dimensionality reduction algorithm, a hierarchical clustering method, HDBSCAN, is used to extract dense regions separated by sparse regions. Although defining clusters in low-dimensional spaces is largely sufficient to achieve the desired behavioral identification, doing so is a computationally expensive process. Additionally, behavioral transference in the low-dimensional space is difficult to evaluate, owing partly due to the non-linearity in dimensionality reduction. To overcome both of these issues, we utilized a machine learning classifier that learns to predict behaviors based on the high dimensional measurements. This approach provides greatly improved computational speed (processing time for one hour of 60fps data containing six poses is under five minutes with a 128GB RAM CPU) and a consistent model that enables generalization across data sets within or across labs. Because the classifier is trained to partition pose relationships, not their low-dimensional representations, the defined clusters are further apart from one another, greatly improving consistency over statistical embedding methods (for unsupervised behavioral metrics comparing high vs. low-dimensional behavioral representation). Finally, to improve functionality, we have increased accessibility -- formatting the code into a downloadable app which provides an intuitive, step-by-step user interface."

The machine learning technique for making "embeddings", such as the Word2Vec word "embedding" system, has been adapted for electronic medical records.

"Essentially, a computer was programmed to scour through millions of electronic health records and learn how to find connections between data and diseases. This programming relied on 'embedding' algorithms that had been previously developed by other researchers, such as linguists, to study word networks in various languages. One of the algorithms, called word2vec, was particularly effective. Then, the computer was programmed to use what it learned to identify the diagnoses of nearly 2 million patients whose data was stored in the Mount Sinai Health System."

"Phe" stands for "phenotype", in case you were wondering.

Vision through murky water. You need a polarized light camera, but otherwise the process is automated. "Traditional approaches to underwater imaging use either prior knowledge of the imaging area or the background of an image to calculate and remove scattered light. These methods have limited utility in the field because they typically require manual processing, images do not always have visible backgrounds, and prior information is not always available."

"To overcome these challenges, the researchers combined a traditional polarized imaging setup with a new algorithm that automatically finds the optimal parameters to suppress the scattering light. This not only significantly improves image contrast to achieve clear imaging but can be used without any prior knowledge of the imaging area and for images with or without background regions."

This system doesn't use any machine learning; it's all regular old-fashioned physics and math.

An optical 'transistor' has been created that can switch 1 trillion times per second, between 100 and 1,000 times faster than today's top-notch commercial transistors, and is so energy-efficient it requires no cooling.

"What makes the new device so energy-efficient is that it only takes a few photons to switch."

"The device relies on two lasers to set its state to '0' or '1' and to switch between them. A very weak control laser beam is used to turn another, brighter laser beam on or off. It only takes a few photons in the control beam, hence the device's high efficiency."

"The switching occurs inside a microcavity -- a 35-nanometer thin organic semiconducting polymer sandwiched between highly reflective inorganic structures. The microcavity is built in such a way as to keep incoming light trapped inside for as long as possible to favor its coupling with the cavity's material."

"This light-matter coupling forms the basis of the new device. When photons couple strongly to bound electron-hole pairs -- aka excitons -- in the cavity's material, this gives rise to short-lived entities called exciton-polaritons, which are a kind of quasiparticles at the heart of the switch's operation."

"When the pump laser -- the brighter one of the two -- shines on the switch, this creates thousands of identical quasiparticles in the same location, forming so-called Bose-Einstein condensate, which encodes the '0' and '1' logic states of the device."

All cryptocurrency transactions have been banned in China.

"EpyNN is a production-ready but first Educational Python resource for Neural Networks. EpyNN is designed for Supervised Machine Learning (SML) approaches by means of Neural Networks."

"EpyNN includes scalable, minimalistic and homogeneous implementations of major Neural Network architectures in pure Python/Numpy."

"EpyNN is intended for teachers, students, scientists, or more generally anyone with minimal skills in Python programming who wish to understand and build from basic implementations of NN architectures."

"EpyNN has been cross-validated against TensorFlow/Keras API and provides identical results for identical configurations in the limit of float64 precision."

DeepMind and University College London have updated their reinforcement learning course lectures online which you can watch for free. So if you want to know how DeepMind made AI systems that defeated the world's top players at the Chinese game of Go, this series of videos will explain it all.

What I want to know is where I'm going to get the time to go through these? I got part way through the original lectures from several years ago.

Lab-grown meat may never happen. "The Good Food Institute (GFI)'s imagined facility would be both unthinkably vast and, well, tiny. According to the techno-economic analysis, it would produce 10,000 metric tons -- 22 million pounds -- of cultured meat per year, which sounds like a lot. For context, that volume would represent more than 10 percent of the entire domestic market for plant-based meat alternatives (currently about 200 million pounds per year in the US, according to industry advocates). And yet 22 million pounds of cultured protein, held up against the output of the conventional meat industry, barely registers. It's only about .0002, or one-fiftieth of one percent, of the 100 billion pounds of meat produced in the US each year. JBS's Greeley, Colorado beefpacking plant, which can process more than 5,000 head of cattle a day, can produce that amount of market-ready meat in a single week.

"And yet, at a projected cost of $450 million, GFI's facility might not come any cheaper than a large conventional slaughterhouse. With hundreds of production bioreactors installed, the scope of high-grade equipment would be staggering. According to one estimate, the entire biopharmaceutical industry today boasts roughly 6,300 cubic meters in bioreactor volume. (1 cubic meter is equal to 1,000 liters.) The single, hypothetical facility described by GFI would require nearly a third of that, just to make a sliver of the nation's meat."

"Using large, 20,000 L reactors would result in a production cost of about $17 per pound of meat, according to the analysis. Relying on smaller, more medium-efficient perfusion reactors would be even pricier, resulting in a final cost of over $23 per pound."

"The final product would be a single-cell slurry, a mix of 30 percent animal cells and 70 percent water, suitable only for ground-meat-style products like burgers and nuggets. With markups being what they are, a $17 pound of ground cultivated meat at the factory quickly becomes $40 at the grocery store -- or a $100 quarter-pounder at a restaurant."

Problems include: requirement for pharmaceutical-grade equipment to prevent contamination, use of fetal bovine serum (FBS), and the cost of pharmaceutical-grade amino acids and other macronutrients.

Dendritic spines as the way the brain stores memories. Or at least the neocortex. The idea is that dendritic spines use time-based codes to scan for precise spike patterns in their synaptic inputs. But wait, first we should explain what "dendritic spines" are.

Brains are made of neurons, and to send a signal to another neuron, it has to send an electrochemical signal down what is essentially a "wire" to a synapse, which is the junction where the signal is picked up on the other side and sent up another "wire" to the other neuron. The first "wires", that send signals away from the neuron cell body, are called "axons" and tend to be skinny and go a long way, and the "wires" that pick up the signals from the synapse and send them up to the next neuron's cell body are called "dendrites" and tend to be short and fat.

There are cells in the neocortex called pyramidal cells, and these neurons are completely covered with tiny protrusions known as dendritic spines. For neurons that are "excitatory", that tend to activate other neurons and start processes it the brain, the dendritic spines receive the majority of the input. For neurons that are "inhibitory", that is, when they fire, it suppresses the activity of other neurons and stops processes in the brain, the input is usually connected directly to the body of the dendrite.

Dendritic spines can change their physical structure in seconds. This change in geometry has a direct affect on voltage amplification and the amount of signal passed on to the dendrite body. The strength of incoming signals can be dampened or amplified, depending on the length and width that the spine neck takes.

The theory here is that "there is a molecular unit within the dendritic spine that encodes a temporal pattern in the form of a sequence of 'on' states and 'off' states. Whenever a dendritic spine receives a sequence of inputs, it will compare the spike train's temporal pattern, with its internal temporal code. If the received spike train's temporal pattern is similar to the spine's temporal code, information flows from the spine to its dendrite freely. If on the other hand the spike train's temporal pattern is significantly dissimilar from the spines internal code, the spine apparatus gets notified to increase intracellular calcium levels, which changes the shape of the spine and dampens the incoming signals. Signal processing on a local level is therefore controlled mechanically at the spine neck, the behavior of which depends on the temporal pattern of the incoming spike train." Calcium matters because calcium ions are what the brain uses to send electrochemical signals.

"Every dendritic spine on the apical dendride of a pyramidal neuron computes a distance measure between the temporal pattern of the input signal and that of its internal code. When a neuron receives inputs at its dendritic spines, it is essentially in a competition with all other neurons that receive the same inputs. Neurons with internal codes that are substantially different from the input signals attenuate the signals (by altering the spine necks of their spines) and delay the propagation of the signals from the apical tuft to the soma of the neuron." The "apical" dendrite is the longest dendrite extending from one end of the neuron, that goes the longest distance and is more associated with long-distance communication within the brain. The "soma" just refers to the cell body of the neuron.

"The neuron with the internal codes most similar to the input signals, will allow the signals to move freely from the apical dendrite to the soma. The 'best matching neuron', that is, the neuron whose signals past through the apical dendrite to the soma the fastest, will fire before the other pyramidal neurons. If the neurons are organized as a competitive circuit, the best matching neuron will activate neighboring inhibitory interneurons, which then inhibit the other pyramidal neurons in that participated in the competition." By "interneurons" here they are referring to neurons in the brain that are not pyramidal neurons.

"Whenever a neuron fires an action potential, another impulse from the soma is also generated that propagates backwards through the apical dendrite of the neuron. This process is also known as Neural backpropagation. When a backpropagating action potential contacts previously activated dendritic spines, a superlinear rise in internal calcium levels occurs inside the dendritic spines. Neighboring spines that were not activated prior to the backpropagating signal would be unaffected. I argue that the backpropagating action potential can be viewed as a mechanism to inform dendritic spines that their action resulted in a best matching neuron. That is, it informs the dendritic spines down the apical dendrite that their neuron won the competition. Every dendritic spine will then modify their internal code to become slightly more similar to the temporal code that they received during the competition. In a way the backpropagating signal can be seen as a message to all the dendritic spines -- 'We won! Update your codes slightly so that we can win the next time even faster!' This process can be viewed as a form of competitive learning, which is a variant of Hebbian learning."

One more thing. There are three kinds of dendritic spines: thin, stubby, and mushroom spines. Thin spines have a small head and a long neck. Stubby spines have no neck at all and there are a lot in infant brains. Mushroom spines have a wide neck with a large head and there are a lot in adult brains. Thin spines can dynamically appear and disappear throughout life, while most mushroom spines remain stable for a lifetime. It's thought that the thin spines are for "short term" memory while the mushroom spines are for "long term" memory.

Brain scanning at high resolution and fast time frames at the same time. "The combination of two-photon scanning microscopy and fluorescent tags is the gold standard when it comes to imaging the activity of neurons within less transparent brain tissues, which are prone to scattering light. It involves firing a focused laser pulse at a tagged target. A few nanoseconds after the pulse hits its mark, the tag emits fluorescent light that can be interpreted to give scientists an idea of the level of neuroactivity detected."

"But two-photon microscopy suffers from a fundamental limitation. Neurobiologists need to record simultaneous interactions between the sensory, motor, and visual regions of the brain, but it is difficult to capture the activity in such a broad swath of the brain without sacrificing resolution or speed."

The solution to this is something they decided to call "light beads microscopy".

"The technique involves breaking one strong pulse into 30 smaller sub pulses -- each at a different strength -- that dive into 30 different depths of scattering mouse brain but induce the same amount of fluorescence at each depth. This is accomplished with a cavity of mirrors that staggers the firing of each pulse in time and ensures that they can all reach their target depths via a single microscope focusing lens. With this approach, the only limit to the rate at which samples can be recorded is the time that it takes the fluorescent tags to flare. That means broad swaths of the brain can be recorded within the same time it would take a conventional two-photon microscope to capture a mere smattering of brain cells."

It was tested by "recording of the activity of more than one million neurons across the entire cortex of the mouse brain for the first time."

New research shows "dopamine levels increase in response to stressful stimuli, and not just pleasurable ones, potentially rewriting facts about the 'feel-good' hormone -- a critical mediator of many psychiatric diseases."

"In the press, dopamine is often referred to as a 'pleasure molecule' or a 'reward molecule. In the scientific community, research has helped us understand that dopamine's role in learning and memory is more complex than that, but we did not have a complete and accurate theory that could explain what dopamine actually does in the brain."

"The prevailing model, called the reward prediction error theory, is based on the idea that dopamine signals predictions about when rewards will occur. This theory suggests that dopamine is a tracker of every error we make when we try to achieve rewards. The authors show that reward prediction error theory is only accurate in a subset of learning scenarios by proving that 'while rewards increase dopamine, so do stressful stimuli.'" "We then go on to show that dopamine is not a reward molecule at all. It instead helps encode information about all types of important and relevant events and drive adaptive behavior -- regardless of whether it is positive or negative."

This work was done by fitting a machine learning model to the behavior of dopamine as measured by optogenetics in mice brains.

A drug called trihexyphenidyl, approved as a treatment for Parkinson's disease, has been discovered to reduce PTSD flashbacks and nightmares.

The drug is thought to work by blocking the action of a neurotransmitter called acetylcholine. Acetylcholine's primary function in the nervous system is to activate muscles, but it functions in the brain as well and throughout the nervous system.

In this case, the researchers believe that suppressing acetylcholine in a few particular spots in the brain, namely the basal forebrain, amygdala, and hippocampus, is what leads to the suppression of PTSD flashbacks and nightmares.

The basal forebrain, and in particular a part of the basal forebrain called the nucleus basalis of Meynert, is thought to have circuits that store traumatic memories, and these get activated by acetylcholine during PTSD-type flashbacks and nightmares. Note that this is different from the places in the brain where normal fear emotions are processed and normal memories are stored, which are the amygdalae and hippocampus.

(To read you have to click the grainy screenshot, which will download a PDF that has the full article.)

Tesla AI Day. Yeah, I know, lots of you have already seen the video. So I guess this is for the 3 people who haven't yet.

They think of their AI system as being analogous to the "visual cortex" in biological organisms. The problem they have is fusing the input from multiple cameras. A Tesla car has 8 cameras, which are high dynamic range (HDR) cameras with 1280x960 resolution that operate at 36 frames per second.

The solution to this problem they have opted for is for the neural networks that process the vision to output what they see in the form of 3D vectors, into what they call "vector space" and they can visualize this 3D "vector space" representation on a screen.

The processing first goes through residual networks (resnets), which are convolutional neural networks but the "residual" technique allows them to go much deeper than traditional convolutional networks. They like the fact that they can make the network deeper or shallower as they please to trade off vision processing with latency.

After the resnets, the data goes into something called a BiFPN, which stands for Bi-directional Feature Pyramid Network. They don't say much about what this network outputs, other than that it is "features", not images.

After this the data branches into multiple "heads". Each of the branches does something different: object detection, traffic lights, lane prediction, etc

After this, they do something called "rectification", which takes the vector space output and takes into account each camera's position and orientation and projects its output into the same 3D "vector space". The final fusion process uses a type of neural network called a transformer. These were originally invented for language translation and have an "attention" mechanism that enables the translation system to pay attention do different words in the input as it generates the output. Since then, "vision transformers" have been invented that enable the neural network to focus "attention" on a specific part of a scene. However, Tesla is not using standard vision transformers. They invented their own transformer which operates in "vector space". So it doesn't take images as its input, it takes sets of 3D vectors. What it outputs, at the end of the whole process, is a single unified 3D representation of the scene with curbs, lanes, traffic lights, other cars, pedestrians, and so on, identified.

This system has another trick of its sleeve. Everything up to here is just looking at camera input at a single point in time. But they enabled the system to understand motion over time. This is done with two "cache" systems. One of them is simply time based -- it remembers the last few seconds of whatever the car has seen. The second is space based. So if, for example, the Tesla car sits at a red light, it can remember lane markings it has seen many seconds ago because they are in the "space based" cache and it remembers the space it recently drove past or over.

These "caches" are combined with a recurrent neural network. This combination allows the system to keep track of the structure of the road over time, and the system handles remembering cars when they are temporarily occluded very well.

After all this, the data goes into the planning and control system. For this he shows an example of changing lanes to make a left turn, and says the path planning system does 2,500 path searches in 1.5 milliseconds.

The planning system plans for everything in a scene, including other cars and pedestrians. He shows an example where the car is driving down a narrow street where we can pull aside and yield for another car or they can pull aside and yield for us. If the other car yields, our car knows what to do because it created that plan for the other car.

He shows a visualization of an A* backtracking algorithm, and how it is too computationally expensive and says they are developing a neural network, borrowing the design from AlphaGo, to optimize "Monte Carlo Tree Search", which AlphaGo also does.

You might be surprised that up until this point, the system does not use neural networks, but uses traditional computer science path planning algorithms. In the Q&A section, Elon Musk reveals that these are written in C++. He says neural networks shouldn't be used unless they have to be, and for vision they have to be, but since path planning doesn't have to be it's written in C++.

I would think this system would have trouble working in places with chaotic driving without clear rules, and the presenter acknowledges the system won't work in other places like India, where he himself happens to be from.

Next they talk about data set labelling. Originally they labeled images, but they switched to labeling in 3D vector space. They developed a UI where people can move things in vector space and see the projection in multiple photographs.

He talks about an auto-labeling system, but I didn't really understand how it works. Apparently it can combine data from multiple cars and reconstruct the road surface and walls and other parts of the scene from the video from multiple cars going through the same place. It also does a good job handling occlusions of moving objects such as cars and pedestrians.

They went to the next level by creating a simulator. It makes pretty realistic video. Of course since the simulation is computer-generated the vector space can automatically be correctly labeled and produce massive amounts of training data. The simulation system even simulates the characteristics of the cameras in the cars, such as adding sensor noise and simulating the effect the sun has on the camera. Neural networks are used to enhance the images and make them look even more realistic.

The main purpose of the simulator, though, isn't just to create massive amounts of training data but to create lots of examples of accidents and other edge cases that occur infrequently in real life. Speeding police cars, and so on. Most of the environments are algorithmically created, not created by human artists, so there is a potentially unlimited amount of roads to train from.

Before putting the models in cars, they do extensive testing, with 1 million evaluations/week on every code change. They developed their own debugging tools so you can see the outputs of multiple different revisions of the software side by side.

The rest of the talk is about Dojo, Tesla's upcoming supercomputer.

Basically what they did is create a supercomputer for learning how to drive. They start the process by designing a training node, which is a CPU combined with dedicated hardware for matrix operations (the core operations in any AI system), hardware for parallel floating point and integer math (similar to a DSP chip), SRAM, and communication hardware. The CPU has 4 threads and an instruction set designed specifically for machine learning (so it's not using a general instruction set such as x86 or ARM). 354 of these "training nodes" are manufactured on a single chip, called the D1 chip, with high-speed communication from each node to its adjacent nodes on 4 sides. It has 50 billion transistors on a single 645 millimeter chip manufactured at 7 nm.

With these D1 chips, the plan is to take 500,000 D1 chips and connect them with "Dojo interface processors", which in turn connect to outside computers. The D1 chips are organized into "training tiles". They created their own power supply and cooling systems for these "tiles". The tiles are placed in an "exapod" where 10 cabinets are combined and the walls removed so the tiles can communicate directly with each other without cabinet walls getting in the way.

They made their own compiler to compile PyTorch models and other code for the hardware.

Basically, they created a supercomputer specialized, from the transistors themselves on up, for one specific task, which is training vision neural networks.