| "Human languages are optimally balanced between accuracy and complexity. For example, many languages have a word that denotes the colour red, but no language has individual words to distinguish ten different shades of the colour. These additional words would complicate the vocabulary and rarely would they be useful to achieve precise communication."
By "complexity" here they mean complexity of the language itself. In other words, you can have a language that communicates in a simple, compact way, or a language that communicates precisely, but requires a lot of words, with precise meanings and precise grammar.
This research gives a "color naming task" to neural networks to see if the result is similar. It is. The task is: there are 330 color "chips", and there are two neural networks, called the "speaker" and the "listener". The "language" contains a fixed set of 1,024 words, but the words are not assigned to any colors initially. Chips are picked at random and presented to the speaker, which tells one of the words to the listener, who is presented with two chips: the one the speaker picked and another picked at random, and the listener has to pick which one is closest to the word received from the speaker. The neural networks could give a unique name to every chip, as the number of words is larger than the number of chips, but they don't.
When graphed out with complexity on the "x" axis and accuracy on the "y" axis (both measured by information theory complexity and accuracy measurements), the vocabularies of the both the neural networks and human languages lie right on the line that defines the theoretical upper limit on accuracy for a given complexity.
"This suggests that an efficient categorization of colours (and possibly other semantic domains) in natural languages is not dependent on specific human biological constraints, but is a general property of discrete communication systems."
|Bach-inspired piano & synthesizer composition. She composed her own piano piece and added synthesizers using Bach's Invention in D Minor as a starting point and uses motifs from Bach's original piece.|
|"I asked an AI for video ideas for other YouTubers. It went badly." The AI in question was GPT-3. And it didn't go entirely badly. The AI came up with a few ideas that were good, but mostly did a good job of imitating the clickbaity style of YouTube titles, and coming up with ideas that a good YouTube creator might be able to improve upon into a video.|
|"I made an AI-powered Linux shell using OpenAI". If memory serves, someone at OpenAI already did something like this, but this was done by someone outside using the API. What it came up with is pretty entertaining. A lot of commands work but the OpenAI system has the ability to hallucinate things like URLs.|
| Happy Moore's Law Day everybody!
For the thirty-fifth anniversary issue of Electronics magazine, which was published on April 19, 1965, Gordon E. Moore, who was working as the director of research and development at Fairchild Semiconductor at the time, was asked to predict what was going to happen in the semiconductor components industry over the next ten years. His response was a brief article entitled, "Cramming more components onto integrated circuits". Within his editorial, he speculated that by 1975 it would be possible to contain as many as 65,000 components on a single quarter-inch semiconductor. "The complexity for minimum component costs has increased at a rate of roughly a factor of two per year. Certainly over the short term this rate can be expected to continue, if not to increase. Over the longer term, the rate of increase is a bit more uncertain, although there is no reason to believe it will not remain nearly constant for at least 10 years."
If you hear people saying Moore's Law is dead, well, clock rate increases stopped in the mid-2000s, but semiconductor scaling continues to this day, although as we approach 5 nanometers, it's obviously getting more and more difficult. That difficulty is reflected in the doubling time, which was actually never as short as a year -- for most of Moore's Law's history, it was a little over 2 years, and now is quite a bit longer but I'm not sure how much.
| "Colorization APIs are becoming widespread; AI-colorized historical photos are circulated without caveat. But is AI colorization providing an accurate image of the past? To find out, I digitally desaturated these color photos by Sergey Prokudin-Gorsky, taken between 1909 and 1915. I then colorized the photos using the DeepAI Image Colorization API."
"Here's the AI's attempt to color these photos."
"And here are the original color photos."
| "NVIDIA's new Grace CPU will power the world's most powerful AI-capable supercomputer." "The Swiss National Computing Center's (CSCS) new system will use Grace, a revolutionary Arm-based data center CPU introduced by NVIDIA today."
"Taking advantage of the tight coupling between NVIDIA CPUs and GPUs, Alps is expected to be able to train GPT-3, the world's largest natural language processing model, in only two days -- 7x faster than NVIDIA's 2.8-AI exaflops Selene supercomputer, currently recognized as the world's leading supercomputer for AI by MLPerf."
| "Why AI reduces productivity in radiology -- yet" "It is a well-known paradox that technology adoption, not only related to AI, is leading to a J-curve shaped change in productivity."
The idea is that introducing a new technology initially decreases productivity. Productivity increases later as people learn to change workflows to make productive use of the new technology, and ultimately ends up higher than the status quo before the introduction of the technology. The process can take a long time and can be painful, though.
"The first certified machine learning-based products that are supposed to support physicians have entered the market. Initially though, a doctor's life does not seem to be easier but even more complicated: It is not enough to write a report and communicate one's findings with patient and referring physician. For an early-adopter radiologist, there are not only dozens of images but also PDF-reports - automatically generated by AI - in the PACS system. These reports need to be interpreted in the clinical context, which is additional work. Ah, and what about quality control for the input data to the AI? Is that particular case today even within the specification of the product?"
| "Analysis of 100 weeks of curated AI news" from Skynet Today. Articles related to "robots" grew much faster than other areas. After late 2019, articles regarding facial recognition grew much faster than others, with Covid-19 articles making a sudden entrance starting early 2020. Google dominates the AI news cycle in terms of mentions of institution names. Sentiment analysis shows art, deep learning, and robotics have mostly positive coverage, while other applications such as surveillance have generally negative coverage.
Then, just for kicks, they finetuned a GPT-2 model on AI articles, and came up with fake AI articles like the following gem. (The following headline and opening paragraph are fake.)
A fight for the soul of machine learning
Earlier this month, researchers at Nvidia, one of the biggest artificial intelligence (AI) companies, revealed an unwieldy and potentially game-changing deep learning algorithm called 'neural pre-training.' In essence, it was so pretrained that it learned far better by trial and error than a human who had been explicitly trained on the same data set.The results promised an entirely new way to train AI. In doing so, it essentially taught itself to do things no robots had ever seen.
|"15 graphs you need to see to understand AI in 2021". "We're Living in an AI Summer": between 2000 and 2019, AI papers went from being 0.8% of all peer-reviewed papers to 3.8%. "China Takes Top Citation Honors": In 2017, China took the lead on most peer-reviewed papers on AI, now they are top in citations. "Faster Training = Better AI": In 2018, it took 6.2 minutes to train the best system on ImageNet; in 2020 it took 47 seconds. "AI Doesn't Understand Coffee Drinking": Of the 200 activities of daily life shown in the ActivityNet dataset, AI systems had the toughest time recognizing the activity of coffee drinking in both 2019 and 2020. "Language AI Is So Good, It Needs Harder Tests": Version 2.0 of a reading comprehension test called SQuAD made the task harder by incorporating unanswerable questions. "A Huge Caveat": Language models for tasks like speech recognition and text generation have some specific failings that could derail commercial use unless addressed. "The AI Job Market Is Global": Data from LinkedIn shows that Brazil, India, Canada, Singapore, and South Africa had the highest growth in AI hiring from 2016 to 2020 (though the US and China have the most in absolute terms). "Corporate Investment Can't Stop, Won't Stop": Global corporate investment in AI soared to nearly $68 billion in 2020, an increase of 40 percent. "The Startup Frenzy Is Over": The money is being channeled into fewer AI startups. "The COVID Effect": Private investment in 2020 skewed toward certain sectors that have played big roles in the world's response to COVID-19. "Risks? There are Risks?": When asked in a McKinsey survey what risks they considered relevant, only cybersecurity had registered with more than half of respondents. "PhDs Hear the Siren Call of Industry": Academia still can't absorb the growing number of fresh AI PhDs released into the world each year. "Ethics Matter": The chart shows the rise in ethics-related papers at AI conferences. "The Diversity Problem, Part 1": Women make up only about 20 percent of graduates from AI-related PhD programs in North America. "The Diversity Problem, Part 2": Data from that same survey tells a similar story about race/ethnic identity.|
|Vine robots: Inflatable robots that extend from the tip and that enables them to pass through tight spaces and curvy and twisted passageways and over sticky surfaces, and not be stopped even by spikes.|
| SingularityNET video about their switch from Ethereum to Cardano. I haven't been paying attention to cryptocurrencies other than Bitcoin, so I spent some time today checking out what's going on in the rest of the cryptocurrency world. I thought Bitcoin was in a bubble. I might have been wrong. Well, Bitcoin might be in a short-term bubble, but long-term it's just doing the same increase in value that it's been doing as it becomes more widely adopted, and adoption doesn't happen evenly but it abrupt surges that happen every few years. I was hearing that the Fed was printing dollars and concerned about inflation, so I was watching the price of gold and the price of Bitcoin. For a while in 2020 they went up in lockstep. But then gold went down while Bitcoin surged up. It's all very confusing.
Anyway, it looks like where the cryptocurrency world is going to go from here is to proof-of-stake algorithms rather than proof-of-work algorithms. The problem with proof-of-work algorithms is they use ungodly amounts of energy. Bitcoin uses about 11 gigawatts of electricity, which is about 0.6% of all energy consumed on the planet. Plus Bitcoin is limited in how fast it can do transactions. Proof-of-stake uses a small fraction of the energy and can do tons of transactions fast, making it suitable for a currency consumers would used for frequent, everyday transactions. Bitcoin will probably continue to grow but due to this lack of scalability, be limited to fewer and larger transactions, such as between financial institutions.
Which currency ends up on top probably depends on timing -- exactly when Cardano comes out with smart contracts and exactly when Ethereum switches to proof-of-stake.
Anyway, speculation on the future of cryptocurrency aside, SingularityNET has been incredibly busy the last few years. I wasn't aware, because, I hate to admit, I've been paying attention to precisely the "trillion-dollar big tech companies" that he regards as SingularityNET's competition. SingularityNET is a plan to enable "decentralized" AI, which will better server humanity than the "centralized" AI of the trillion-dollar tech companies that serve the elites. He's definitely right that the centralized trillion-dollar tech companies are advancing AI very fast. You all have seen posts from me about their work in this space on a regular basis. SingularityNET is a bold challenge to those companies and a chance for all the rest of us who don't work for those companies to work together to make AI that works for everybody.
| Theoretical model for neural activity of mouse brain. "The work is based on a physics concept known as critical phenomena, used to explain phase transitions in physical systems, such as water changing from liquid to a gas. In liquid form, water molecules are strongly correlated to one another. In a solid, they are locked into a predictable pattern of identical crystals. In a gas phase, however, every molecule is moving about on its own."
"At what is known as a critical point for a liquid, you cannot distinguish whether the material is liquid or vapor. The material is neither perfectly ordered nor disordered. It's neither totally predictable nor totally unpredictable. A system at this 'just right' Goldilocks spot is said to be 'critical.'"
"In recent decades, some scientists began thinking about the human brain as a critical system. Experiments suggest that brain activity lies in a Goldilocks spot -- right at a critical transition point between perfect order and disorder."
"Researchers wanted to test whether fine-tuning of particular parameters were necessary for the observation of criticality in the mouse brain experiments, or whether the critical correlations in the brain could be achieved simply through the process of it receiving external stimuli."
"We previously created a model that showed Zipf's law in a biological system, and that model did not require fine tuning."
"The model's key ingredient is a set of a few hidden variables that modulate how likely individual neurons are to be active."
"The model was able to closely reproduce the experimental results in the simulations. The model does not require the careful tuning of parameters, generating activity that is apparently critical by any measure over a wide range of parameter choices."
| "It has long been known that obesity is an inflammatory disease..." Wait, what? I didn't know that. I thought "inflammatory" meant firing up the immune system, and bodily responses to physical injury. Didn't think obesity had anything to do with that.
"...i.e. a chronic defensive reaction of the body to stress caused by excess nutrients. Based on this knowledge, a group of researchers ... decided to try to fight obesity by preventing inflammation -- and they succeeded. Their paper ... shows that digoxin, a drug already in use against heart diseases, reduces inflammation and leads to a 40% weight loss in obese mice, without any side effects."
"Digoxin reverses obesity completely: treated mice attain the same weight as healthy, non-obese animals. The mice were also cured of metabolic disorders associated to obesity."
"Digoxin reduces the production of a molecule called interleukin 17A, or IL-17A, which generally triggers inflammation. The study identifies IL-17A as a causal factor of obesity: 'When you inhibit the production of IL-17A or the signalling pathway that this molecule activates, you don't have obesity.'"
"The researchers found that IL-17A acts directly on adipose tissue to cause obesity and severe metabolic alterations associated with body weight gain, the so-called metabolic syndrome, which includes type 2 diabetes, hypertension and cardiovascular diseases."
"The animals, obese due to a high-calorie diet, continued to eat as before when they were taking digoxin. However, they showed activation of their basal metabolism, which results in the burning of excess fat and weight loss."
They say no adverse effects. "The benefits were maintained for at least eight months, suggesting that resistance mechanisms do not develop."
| Aging signatures across diverse tissue cells revealed... in mice. So this organization, the Tabula Muris Consortium, was created to make a giant dataset of mouse aging data, and that data set is called the TMS FACS. TMS stands for Tabula Muris Senis, and FACS stands for fluorescence-activated cell sorting. Fluorescence-activated cell sorting is a flow cytometry technique. A flow cytometer is a machine where cells are passed one at a time in front of a laser beam. The word "cytometry" is supposed to imply measurement of cells. The flow cytometer can count cells and tell various characteristics of them such as their size. In the fluorescence-activated variation, the idea is to use the light scattering and fluorescent characteristics of each cell to sort cells into bins of different cell types. The way the actual sorting works is, once the detector looks at the light scattering and fluorescence and makes a sorting decision, an electrical charge is added to the droplet, or not, and it then falls down through an electrostatic deflection system that will divert droplets with the charge.
Anyway, the TMS FACS data focuses on single-cell RNA from 120 different types of mouse cells from 23 tissue types. The researchers used differential gene expression analysis, which looks at which genes are actually being expressed in the cells, and which aren't. They were looking for genes whose expression varies substantially with age in most of the tissue-cell types.
"The number of significantly age-dependent genes per tissue-cell type ranges from hundreds to thousands. Most tissue-cell types have more downregulated aging-related genes than upregulated aging-related genes, suggesting a general decrease in gene expression over aging."
"We found that most aging-related genes identified in the analysis have monotonic aging trajectories, meaning that their expressions either increased or decreased monotonically during aging. However, a subset of genes in the FACS data (13%), while being upregulated during aging overall, increased from 3 m to 18 m and slightly decreased from 18 m to 24 m; those genes are enriched in brown adipose tissue B cells, large intestine epithelial cells, and mesenchymal adipose tissue mesenchymal stem cells of adipose."
I think "m" here means "months"; mice have much shorter lives that humans. Probably makes them useful for studying aging. "Mesenchymal" means the cells come from the mesoderm. In the embryo, there are three layers, an outside layer called the ectoderm, an inside layer called the endoderm, and a layer in between called the mesoderm. These mesenchymal cells generally become bone, muscle, and cartilage.
"We found that most genes are significantly related to aging in at least one tissue-cell type, consistent with the intuition that aging is a highly complex trait involving many biological processes. The aging-related genes discovered in the FACS data significantly overlap with other important gene sets, including both known human and mouse aging markers as recorded in the GenAge database, senescence genes, transcription factors, eukaryotic initiation factors, and ribosomal protein genes. Some of the top overlapping genes, significantly related to aging in most tissue-cell types, include known mouse aging markers Jund, Apoe, and Gpx4 and known human aging markers Jund, Apoe, Fos, and Cdc42 from the GenAge database, and senescence genes Jund, Junb, Ctnnb1, App, and Mapk1. In addition, we found that each tissue-cell type has around 5% aging-related genes that are shared by the GenAge human aging markers. However, we did not find any tissue-cell types that are specifically enriched with these known human aging markers, suggesting that the conservation between mouse aging and human aging is relatively uniform across tissue-cell types."
"We identified 330 global aging genes in total, among which 93 are consistently upregulated and 190 consistently downregulated (>80% of weighted tissue-cell types); only 47 have an inconsistent directionality (upregulated in 20 -- 80% of weighted tissue-cell types). We found that the aging genes significantly overlap with genes known in aging-related diseases, including strong overlap with genes related to Alzheimer's disease, neuroblastoma, fibrosarcoma, and osteoporosis, and relatively weaker overlap with genes related to Huntington's disease, skin carcinoma, kidney cancer, acute promyelocytic leukemia, acute myeloid leukemia, endometrial cancer, and hypertension."
"We visualized the age coefficients for 10 top up/downregulated aging genes (a consistent direction in >80% of weighted tissue-cell types) that are related to aging in the most number of tissue-cell types. Many of these genes have been previously shown to be highly relevant to aging. For example, the downregulation of Lars2 has been shown to result in decreased mitochondrial activity and increase the lifespan for C. elegans. On the other hand, Jund is a proto-oncogene known to protect cells against oxidative stress and its knockout may cause a shortened lifespan in mice. Moreover, Rpl13a was observed to be upregulated in almost all tissue-cell types. As a negative regulator of inflammatory proteins, Rpl13a contributes to the resolution phase of the inflammatory response, ensuring that the inflamed tissues are completely restored back to normal tissues. It also contributes to preventing cancerous growth of the injured cells caused by prolonged expression of the inflammatory genes. Therefore, it is interesting to observe the upregulation of Rpl13a given that most old mice have severe inflammatory symptoms. Gene Ontology biological pathway enrichment analysis revealed that the 330 aging genes are associated with apoptosis, translation, biosynthesis, metabolism, and cellular organization. These biological processes are highly relevant to aging and are shared across most cell types, consistent with the intuition that aging genes represent the global aging process across tissue-cell types. In addition, the KEGG pathways associated with the aging genes are consistent with the Gene Ontology terms and additionally highlighted immune-related pathways and multiple aging-related diseases. Moreover, the findings were supported by similar analyses on the set of 59 aging genes discovered in the droplet data. We also performed pathway enrichment analysis using the Ingenuity Pathway Analysis software, which confirmed our findings for the biological processes associated with the aging genes. Of note is the finding that the mTOR pathway, a known aging-associated pathway, is predicted to be inhibited given the expression of the aging genes. Interestingly, mTOR downregulation has been shown to promote longevity, a further indication that the aging genes are related to the aging process."
mTOR stands for mammalian target of rapamycin, and is a kinase central to two protein complexes (creatively called mTOR complex 1 and mTOR complex 2), which regulate a boatload of cellular processes, from cell growth, to cell proliferation, cell motility, cell survival, protein synthesis, autophagy, and transcription.
KEGG stands for Kyoto Encyclopedia of Genes and Genomes and is a database of links between genes and their high-level functions.
"Immune cells and stem cells have higher aging gene score effects, while most parenchymal cell types have lower aging gene score effects; such a contrast is also statistically significant. Indeed, immune cells and stem cells are known to undergo the most substantial changes with aging. Specifically, the aging of the immune system is commonly linked to the impaired capacity of elderly individuals to respond to new infections. Also, adult stem cells are critical for tissue maintenance and regeneration, and the increased incidence of aging-related diseases has been associated with a decline in the stem cell function. On the other hand, parenchymal cells like pancreatic cells, neurons, heart myocytes, and hepatocytes have lower aging scores. This could be an indication that these tissue-specialized cell types are more resilient to aging and are able to maintain their functions despite the changes in the animal. We also found that the tissue-cell aging gene score effects are in general positively correlated with the cell turnover rate. For example, short-lived cells like skin epidermal cells, monocytes, and T cells have higher aging gene score effects while long-lived cells like neurons, oligodendrocytes, pancreatic b-cells, liver hepatocytes, and heart atrial myocytes have very low aging gene score effects."
I've quoted a lot at this point so I'll stop here, but there's more, and the full paper and an extra document of figures is all open access.
| Spintronics at room temperature. Spintronics is when, instead of storing information using the presence or absence of electrons, information is stored using the spin of electrons. Researchers "have now constructed a semiconductor component in which information can be efficiently exchanged between electron spin and light -- at room temperature and above."
"One important advantage of spintronics based on semiconductors is the possibility to convert the information that is represented by the spin state and transfer it to light, and vice versa. The technology is known as opto-spintronics. It would make it possible to integrate information processing and storage based on spin with information transfer through light."
"As electronics used today operates at room temperature and above, a serious problem in the development of spintronics has been that electrons tend to switch and randomise their direction of spin when the temperature rises." "It is thus a necessary condition for the development of semiconductor-based spintronics that we can orient essentially all electrons to the same spin state and maintain it, in other words that they are spin polarised, at room temperature and higher temperatures. Previous research has achieved a highest electron spin polarisation of around 60% at room temperature, untenable for large-scale practical applications."
"Researchers at Linköping University, Tampere University and Hokkaido University have now achieved an electron spin polarisation at room temperature greater than 90%. The spin polarisation remains at a high level even up to 110 °C."
"This technological advance ... is based on an opto-spintronic nanostructure that the researchers have constructed from layers of different semiconductor materials. It contains nanoscale regions called quantum dots." "When a spin polarised electron impinges on a quantum dot, it emits light -- to be more precise, it emits a single photon with a state (angular momentum) determined by the electron spin."
"The quantum dots are made from indium arsenide (InAs), and a layer of gallium nitrogen arsenide (GaNAs) functions as a filter of spin. A layer of gallium arsenide (GaAs) is sandwiched between them. Similar structures are already being used in optoelectronic technology based on gallium arsenide, and the researchers believe that this can make it easier to integrate spintronics with existing electronic and photonic components."
"We are very happy that our long-term efforts to increase the expertise required to fabricate highly-controlled N-containing semiconductors is defining a new frontier in spintronics."
The paper is paywalled so that's all I know. Don't know why InAs + GaAs + GaNAs == spin maintained at room temperature. Quantum dots are sometimes thought of as artificial atoms, as they can mimic the electronic states of atoms through quantum effects without actually being atoms.