Boulder Future Salon

Thumbnail
"Tiny Corp decides to make both AMD and Nvidia versions of its TinyBox AI server" but "GeForce version is 67% more expensive".

"The TinyBox went public with much fanfare this February. Tiny Corp described it as an innovation that could help democratize PetaFLOPS-class performance for artificial intelligence. It was convinced of the AMD advantage at the time, explaining that the Radeon consumer-grade GPUs worked with superior 'full fabric' PCIe 4.0 x16 links and peer-to-peer interconnections necessary for large language models (unlike GeForce). Top performers like the AMD Radeon RX 7900 XTX were also much cheaper to buy and easier to find."

I'm always interested in the question of whether Nvidia can get some serious competition. The article goes on to recount how Tiny Corp ran into problems with their AMD GPUs.

"Fast forward to today, and Tiny Corp says that it has found a useful userspace debugging and diagnostic tool for AMD GPUs that offers some hope."

Thumbnail
"They praised AI at SXSW -- and the audience started booing."

"The booing started in response to the comment that 'AI is a culture.' And the audience booed louder when the word disrupted was used as a term of praise (as is often the case in the tech world nowadays)."

"Ah, but the audience booed the loudest at this statement:"

"'I actually think that AI fundamentally makes us more human.'"

"The event was a debacle -- the exact opposite of what the promoters anticipated."

"This is not a passing fad. We have arrived at the scary moment when our prevailing attitude to innovation has shifted from love to fear."

Thumbnail
Survey of 2,700 AI researchers.

The average response placed each of the following within the next 10 years:

Simple Python code given spec and examples
Good high school history essay
Angry Birds (superhuman)
Answer factoid questions with web
World Series of Poker
Read text aloud
Transcribe speech
Answer open-ended fact questions with web
Translate text (vs. fluent amateur)
Group new objects into classes
Fake new song by specific artist
Answers undecided questions well
Top Starcraft play via video of screen
Build payment processing website
Telephone banking services
Translate speech using subtitles
Atari games after 20m play (50% vs. novice)
Finetune LLM
Construct video from new angle
Top 40 Pop Song
Recognize object seen once
All Atari games (vs. pro game tester)
Learn to sort long lists
Fold laundry
Random new computer game (novice level)
NYT best-selling fiction
Translate text in newfound language
Explain AI actions in games
Assemble LEGO given instructions
Win Putnam Math Competition
5km city race as bipedal robot (superhuman)
Beat humans at Go (after same # games)
Find and patch security flaw
Retail Salesperson

...and the following within the next 20 years:

Equations governing virtual worlds
Truck Driver
Replicate ML paper
Install wiring in a house
ML paper

... and the following within the next 40 years:

Publishable math theorems
High Level Machine Intelligence (all human tasks)
Millennium Prize
Surgeon
AI Researcher
Full Automation of Labor (all human jobs)

It should be noted that while these were the averages, the was a very wide variance -- so a wide range of plausible dates.

"Expected feasibility of many AI milestones moved substantially earlier in the course of one year (between 2022 and 2023)."

If you're wondering what the difference between "High-Level Machine Intelligence" and "Full Automation of Labor" is, they said:

"We defined High-Level Machine Intelligence thus: High-level machine intelligence is achieved when unaided machines can accomplish every task better and more cheaply than human workers. Ignore aspects of tasks for which being a human is intrinsically advantageous, e.g. being accepted as a jury member. Think feasibility, not adoption."

"We defined Full Automation of Labor thus:"

"Say an occupation becomes fully automatable when unaided machines can accomplish it better and more cheaply than human workers. Ignore aspects of occupations for which being a human is intrinsically advantageous, e.g. being accepted as a jury member. Think feasibility, not adoption. [...] Say we have reached 'full automation of labor' when all occupations are fully automatable. That is, when for any occupation, machines could be built to carry out the task better and more cheaply than human workers."

They go on to say,

"Predictions for a 50% chance of the arrival of Full Automation of Labor are consistently more than sixty years later than those for a 50% chance of the arrival of High Level Machine Intelligence."

That seems crazy to me. In my mind, as soon as feasibility is reachend, cost will go below human labor very quickly, and the technology will be adopted everywhere. That is what has happened with everything computers have automated so far.

"We do not know what accounts for this gap in forecasts. Insofar as High Level Machine Intelligence and Full Automation of Labor refer to the same event, the difference in predictions about the time of their arrival would seem to be a framing effect."

A framing effect that large?

"Since 2016 a majority of respondents have thought that it's either 'quite likely,' 'likely,' or an 'about even chance' that technological progress becomes more than an order of magnitude faster within 5 years of High Level Machine Intelligence being achieved."

"A large majority of participants thought state-of-the-art AI systems in twenty years would be likely or very likely to:"

1. Find unexpected ways to achieve goals (82.3% of respondents),
2. Be able to talk like a human expert on most topics (81.4% of respondents), and
3. Frequently behave in ways that are surprising to humans (69.1% of respondents)

"Most respondents considered it unlikely that users of AI systems in 2028 will be able to know the true reasons for the AI systems' choices, with only 20% giving it better than even odds."

"Scenarios worthy of most concern were: spread of false information e.g. deepfakes (86%), manipulation of large-scale public opinion trends (79%), AI letting dangerous groups make powerful tools (e.g. engineered viruses) (73%), authoritarian rulers using AI to control their populations (73%), and AI systems worsening economic inequality by disproportionately benefiting certain individuals (71%)."

"Respondents exhibited diverse views on the expected goodness/badness of High Level Machine Intelligence. Responses range from extremely optimistic to extremely pessimistic. Over a third of participants (38%) put at least a 10% chance on extremely bad outcomes (e.g. human extinction)."

Thumbnail
"Furno bets that super-efficient, modular kilns will turn the cement industry upside down."

"Calcium containing minerals, usually from limestone, are sent through the kiln, where they're heated to more than 1,110 degrees F (600 degrees C)."

"Raw materials enter one end of the kiln, which lies horizontally, and are pushed past an intense flame to the other end, where they emerge as clinker."

"Kilns today are not very efficient. A study of one in China showed that only about 30% of the heat in the kiln powered the calcination reaction; the rest was lost."

"Furno's kiln, on the other hand, is far more efficient, reducing emissions from fossil fuels by 70% or more. It accomplishes that by turning the kiln vertical, filling it with ingredients that are ground to prescribed dimensions, and then seeping through it a combustive gas (natural gas, biogas or hydrogen) and setting it ablaze."

Thumbnail
"India's government has stepped back from its plan to require government approval for AI services before they come online."

I posted about this when it was first announced (on March 12), so I feel obligated to post this follow-up.

"That plan, announced in early March, was touted as India grappled with what the Ministry of Electronics and Information Technology described as the 'inherent fallibility or unreliability' of AI."

"But last Friday the ministry issued a widely reported update removing the requirement for government permission, but adding obligations to AI service providers. Among the new requirements for Indian AI operations are labelling deepfakes, preventing bias in models, and informing users of models' limitations. AI shops are also to avoid production and sharing of illegal content, and must inform users of consequences that could flow from using AI to create illegal material."

Thumbnail
"How a solar revolution in farming is depleting the world's groundwater."

"There is a solar-powered revolution going on in the fields of India. By 2026, more than 3 million farmers will be raising irrigation water from beneath their fields using solar-powered pumps. With effectively free water available in almost unlimited quantities to grow their crops, their lives could be transformed. Until the water runs out."

"The desert state of Rajasthan is the Indian pioneer and has more solar pumps than any other. Over the past decade, the government has given subsidized solar pumps to almost 100,000 farmers. Those pumps now water more than a million acres and have enabled agricultural water use to increase by more than a quarter. But as a result, water tables are falling rapidly."

Thumbnail
"Why vertical farms worldwide are failing."

Spoiler: the need for highly skilled labor makes labor costs much higher than regular farms.

Thumbnail
3D AI Studio claims to create 3D models using AI. They say they can do text prompts and image input. The 3D models can be exported in many formats, including OBJ, STL, FBX, USD, and more.

Commercial product with free tier.

They don't say anything about how the system works.

Thumbnail
"Will AI save physicians and clinicians time and from burnout?"

"Copilots for clinicians are also becoming more common. Ambient clinical documentation is a booming business. The technology allows doctors to record conversations with patients to automatically turn them into clinical notes and summaries using AI and is a major topic at Healthcare conferences like HIMSS conference this year, where more than 30,000 health and tech professionals gathered in Orlando, Florida."

"Earlier in March, Salesforce announced Einstein Copilot: Health Actions will allow doctors to book appointments, summarize patient information and send referrals by prompting AI with conversational language."

"Administrative workloads are a major problem for clinicians across the US health-care system. A survey published (via CNBC) by Athenahealth in February found that more than 90% of physicians report feeling burned out on a regular basis, largely because of the paperwork they are expected to complete."

"I used to be part of an admissions committee for a medical school. When I interviewed idealistic young people applying to medical school, 'typing' and 'filling out forms' was never once mentioned as a reason for becoming a physician."

She goes on to describe using AI for prior authorization letters that have to be written to insurance companies. These require a letter to be written to justify the use of a drug or therapy for a specific patient and to contain details of that specific patient and why that patient needs that therapy. These are frequently rejected by the insurance companies and have to be re-written over and over to eventually get approval. "A third of medical offices employ full-time staff to take care of the average 30 prior authorizations per physician per week."

On the flip side, "the insurers have started to use AI to deny claims more quickly."

Another use is referral letters from one physician to another. "Like prior authorization letters, these are pretty formulaic."

But the thing she has the most enthusiasm for is what she calls "ambient scribes". "Ambient scribes" are AI systems that listen in to the conversation between the patient and the physician and create a templated note for the medical record. "This technology allows physicians to avoid looking at a screen and typing while they're trying to connect with a patient."

"I've tried versions from multiple AI scribe companies (including TORTUS AI, which - full disclosure - I consult for) and they do an amazing job of filtering out irrelevant information and putting the information in the right spot."

"Think of the technological challenge inherent in this process: patient visits are often interrupted by clinic staff or phone calls, meander off into conversations about kids and dogs, and use abbreviations and technical jargon. They're often circular, meaning a patient will mention a symptom and the physician won't ask a follow up question about it until several minutes later. These tools produce a full transcript that uses generative AI to find the important information and put it into a form that's indistinguishable from what a physician would actually type. Many of my friends have reported that ambient scribes actually do a better job of including important details than they would have included themselves."

Thumbnail
The Scalable, Instructable, Multiworld Agent (SIMA) from DeepMind plays video games for you. You tell it what you want to do in regular language, and it goes into a 3D environment, including some provided by commercial video games, and carries out keyboard-and-mouse actions.

Before getting into how they did this, might be worth citing some of the reasons they thought this was challenging: Video games can be open-ended, visually complex, and have hundreds of different objects. Video games are asynchronous -- no turn taking like chess or Go, or many research environments, which stop and wait while the agent computes its next action. Each instance of a commercial video game needs its own GPU -- no running hundreds or thousands of actors per game per experiment as has been historically done in reinforcement learning. AI agents see the same screen pixels that a human player gets -- no access to internal game state, rewards, or any other "privileged information". AI agents use the same keyboard-and-mouse controls that humans do -- no handcrafted action spaces or high-level APIs.

In addition to all those challenges, they demanded their agents follow instructions in regular language, rather than simply pursuing a high score in the game, and the agents were not allowed to use simplified grammars or command sets.

"Since the agent-environment interface is human compatible, it allows agents the potential to achieve anything that a human could, and allows direct imitation learning from human behavior."

"A key motivation of SIMA is the idea that learning language and learning about environments are mutually reinforcing. A variety of studies have found that even when language is not necessary for solving a task, learning language can help agents to learn generalizable representations and abstractions, or to learn more efficiently." "Conversely, richly grounded learning can also support language learning."

I figure you're all eager to know what the games were. They were: Goat Simulator 3 (you play the goat), Hydroneer (you run a mining operation and dig for gold), No Man's Sky (you explore a galaxy of procedurally-generated planets), Satisfactory (you attempt to build a space elevator on an alien planet), Teardown (you complete heists by solving puzzles), Valheim (you try to survive in a world of Norse mythology), and Wobbly Life (you complete jobs to earn money to buy your own house).

However, before the games, they trained SIMA in research environments. Those, which you probably never heard of, are: Construction Lab (agents are challenged to build things from construction blocks), Playhouse (a procedurally-generated house), ProcTHOR (procedurally-generated rooms, such as offices and libraries), and WorldLab (an environment with better simulated physics).

The SIMA agent itself maps visual observations and language instructions to keyboard-and-mouse actions. But it does that in several stages. For input, it takes a language instruction from you, and the pixels of the screen.

The video and language instruction both go through encoding layers before being input to a single, large, multi-modal transformer. The transformer doesn't output keyboard and mouse actions directly. Instead, it outputs a "state representation" that gets fed into a reinforcement learning network, which translates the "state" into what in reinforcement learning parlance is called a "policy". A more intuitive regular word might be "strategy". Basically this is a function that, when given input from the environment including the agent's state within the environment, will output an action. Here, the actions are the same actions a human would take with mouse and keyboard.

The multi-modal transformer was trained from scratch. A recent new algorithm called Classifier-Free Guidance (CFG) was used, an algorithm inspired by the algorithms used by diffusion models to "condition" the diffusion model on the text you, the user, typed in.

Even in the research environments, it is hard to automate judging of whether an agent completed its tasks. Instructions may be such things as, "make a pile of rocks to mark this spot" or "see if you can jump over this chasm". The environment may not provide any signal indicating these have been fulfilled. There are some they can handle, though, like "move forward", "lift the green cube", and "use the knife to chop the carrots".

For commercial video games, all the agent gets is pixels on the screen, just like a human player, and has no access to the internal game state of the game. The games generally don't allow any game state to be saved and restored, something researchers like for reproducibility.

For video games, they resorted to detecting on-screen text using OCR. They did this in particular for two games, No Man's Sky and Valheim, "which both feature a significant amount of on-screen text."

Why not just have people look, i.e. have humans judge whether the instructions were followed? Turns out humans were "the slowest and most expensive." They were able to get judgments from humans who were experts at the particular game an agent was playing, though.

For automated judgment, if a task contains a knife, a cutting board, and a carrot, the agent may ascertain the goal ("cut the carrot on the cutting board") without relying on the language instruction. This example illustrates the need to differentiate between following a language task and inferring the language task from "environmental affordances".

How'd SIMA do? It looks like its success rate got up to about 60% for Playhouse, but only about 30% for Valheim. That's the percentage of tasks completed. The ranking goes Playhouse, Worldlab, Satisfactory, Construction Lab, No Man's Sky, Goat Simulator 3, and Valheim.

"Note that humans would also find some of these tasks challenging, and thus human-level performance would not be 100%."

Grouped by "skill category", movement instructions ("stop", "move", "look") were the easiest, while food and resource gathering instructions ("eat", "cook", "collect", "harvest") were the hardest.

For No Man's Sky, they did a direct comparison with humans. Human's averaged 60%, while SIMA had around 30%.

How long til the AIs can beat the humans?

Thumbnail
So 3 days ago (while I was watching "Boeing" videos), Nvidia had a developers conference where Nvidia CEO Jensen Huang gave a 2-hour keynote address. The video, as I write this, has over 16 million views -- maybe you are one of them? Maybe there is no reason for me to talk about any of the items in the keynote address.

If you somehow haven't seen it and are wondering what's in it, here are some of the items that got my attention.

First, he starts off saying, "I hope you realize this is not a concert. You have arrived at a developers conference." It might not be a concert, but you have to admit, Jensen Huang is one heck of a showman. His salesmanship probably accounts for a lot of the success of the company.

Moving on to some of the substance of the presentation, Nvidia is working with Ansys, Synopsys, and Cadence Design Systems to use AI to design chips. Ansys develops physics simulation software and is going to be working with Nvidia to improve its simulation systems. Synopsys and Cadence are both electronic design automation (EDA) companies -- which is to say, they sell software that is used to automate the chip design process. These do such things as logic synthesis, where you give the software high-level logic and it figures out what logic gates you need, and physical layout, where the software figures out where to locate transistors on a chip and where the wiring should go. (I don't think he mentioned, Synopsys is in the process of acquiring Ansys, so Ansys might soon disappear as a separate company.) Cadence and Nvidia are building Cadence's next supercomputer together, and Synopsis is integrating AI into its computational lithography system.

He says large language models (LLMs) are able to double in size every six months due to Nvidia chips.

He shows Nvidia's new GPU, called "Blackwell", with, he says, 208 billion transistors. Named for a mathematician named David Blackwell. He says it is made of two "dies" that are somehow stuck together to form a single "chip". He says 10 terabytes of data goes back and fourth between the two dies per second.

I remember in the mid-2000s when I first heard chips had crossed the 1-billion transistor mark. I thought that was mindblowing. 208 billion is like, I feel like I need a word more mindblowing than "mindblowing".

He says it works with the same infrastructure as their previous chip "Hopper". Can that be true? Surely it must make a lot more heat.

He talks a lot about robotics and Nvidia Omniverse and Isaac Sim.

Isaac Sim is a simulation world for training robots in simulation before they are unleashed in the real world.

Omniverse is all about "digital twins", but "digital twins" are not twins of people -- they're "twins" of factories. Basically the idea is to simulate your whole factory in a 3D simulator. The next step is to fill your real-life factory with cameras and link the factory and the simulation together, so the simulation always knows what's happening in the real factory. And link that to language models so you can ask questions in English about what's going on in your factory. Siemens, a giant industrial equipment company in Germany, is working with Nvidia for automating factories.

It's part of Nvidia's mission to "digitize" everything. Not just factories but proteins and genes, etc. A digital twin of the Earth for predicting weather.

Nvidia is launching a new service called "NIM", which stands for "Nvidia Inference Microservices". The plan is to build AlphaFold 2 for you, and lots of other models for proteins and genes and other medical and scientific application. These models are called "NIMs".

Nemo Retriever is platform for building a "digital human" chatbot for retrieving information about a topic, either serving the public, or as "co-pilots" helping employees on their jobs with information inside the company.

He shows videos of robots learning in simulation in Isaac Sim, then has humanoid robots walk out on stage. Finally he has some Disney robots called BDX robots walk out on stage, which are super cute.

"They're powered by Jetson. Little Jetson robotics computers inside. They learn to walk in Isaac Sim."

Throughout the whole presentation, he's making grandiose predictions for the future about how everything everywhere is going to "be robotic". Nvidia's decades of past success, not just Jensen Huang's showmanship, make such predictions credible.

Thumbnail
Today Google Chrome asks me if I want to try "out experimental AI features?"

(It lives at chrome://settings/ai -- you might see if it's available in yours. If not, it might be on the next update).

"These features use AI, are in early development, and won't always get it right."

I really don't want (even more of) my browser usage sent to Google's servers... but I can't resist, so I turned on the AI features to see what they do.

The features it advertises are:

"Help me write: Helps you write short-form content for things on the web, like reviews. Suggested content is based on your prompts and the content of the web page. To use this feature, right-click on a text box."

"Tab organizer: Automatically creates tab groups based on your open tabs. To use this feature, right-click on a tab and click Organize similar tabs."


To try out "Help me write", I punched in, "Today Google Chrome asks me if I want to try 'out experimental AI features?'".

After selecting "Elaborate" for "Length" and "Formal" for "Tone", I got:

"Today, Google Chrome prompted me to try out 'experimental AI features.' I clicked on the link and was directed to a page that described the features. The descriptions sounded interesting, but I am unsure what the features are or how they would improve my workflow. I am interested in learning whether anyone has tried these features and can share their evaluation of their usefulness."

I clicked on a button, not a link. Ha.

As for "Tab organizer", I clicked on a tab with a video about Boeing and clicked "Organize Similar Tabs".

It created a tab group called "Boeing Drama".

Here are the tabs it put together into the same tab group:

https://www.youtube.com/watch?v=Q8oCilY4szc - Boeing: Last Week Tonight with John Oliver (HBO)

https://www.youtube.com/watch?v=BlmYZ06F-78 - Boeing's killer plane - What went wrong? | ENDEVR Documentary

https://www.youtube.com/watch?v=NDEkH0zd3F8 - Scandal: Boeing's 737 Max disaster - Plainly Difficult

https://www.youtube.com/watch?v=UUuB0C1Nk8U - The SR-71 was much faster than the Air Force will admit - Sandboxx

https://www.youtube.com/watch?v=kLT1QEIIaB4 - Did beleaguered aircraft giant eliminate whistleblower? - Todd Grande

https://www.youtube.com/watch?v=dTmeswV3Ln0 - Boeing whistleblower found dead amid safety concerns and legal battle - dustin dailey

https://www.youtube.com/watch?v=Sdb44vY9VBw - "They Silenced Him." Boeing Whistleblower found dead after testifying | Attorney Ryan Explains

https://www.youtube.com/watch?v=OfoBxa7EoIo - The World with Yalda Hakim: Boeing whistleblower John Barnett found dead - Sky News

https://www.youtube.com/watch?v=mwAtCavQQlA - "Dead after testifying" - Was Boeing whistle blower John Barnett killed to silence him? - Valuetainment

https://www.youtube.com/watch?v=eOffvIaWNm4 - Ex-Boeing Quality Manager Warns of 737 Plane Being Back Air So Soon | TMZ Live - Jan 31, 2024

https://news.ycombinator.com/item?id=39673589 - Boeing whistleblower found dead in US (bbc.com)

https://news.ycombinator.com/item?id=39673589 - Boeing whistleblower found dead in US (bbc.com)


It included the Hacker News link twice. But I had it open twice so maybe it should have done that?

And if you look closely, you'll notice it snuck in one video that's not about Boeing. There's an SR-71 video in there. It kind of makes sense that it's in the group because it's also about aviation, but the label it came up with for the tab group wasn't "Aviation", it was "Boeing Drama". So, there's a little bit of disconnect between the clustering algorithm and the labelling algorithm.

Also, and if you're thinking most of the tabs open on my machine were about Boeing, you'd be wrong. I've got 320 tabs open. So, 13 about Boeing, 307 about other topics. As a percentage, 4% about Boeing. (More on that below.)

And yes, I know you all count on me to bring you insights into the latest AI developments (lol), but I got sucked into the "Boeing news" rabbit hole. (More on that below, too).

Thumbnail
"Neurallambda".

"The Problem: My premise all comes down to 'reasoning', and the lack thereof, in current AI models. I'll provide my working definition of 'reasoning', but for a moment, please bear with a couple examples of reasoning failures."

"Transformer models cannot reason."

"Diffusion models cannot reason."

"AI is currently like a living textbook."

"What is Reasoning? Reasoning is the ability to know true things without having learned them. It is building knowledge/predictions/retrodictions/actions atop principles, instead of evidence."

"What are programs?" "Turing Machines are machines capable of executing programs which can calculate anything calculatable. Your computer is a Turing machine (in the limit of infinite memory and time). I'd also suggest that your conscious mind is Turing Complete."

"Tiers of 'Programming Ability' / 'Reasoning Ability'":

"1. An AI can execute programs"

"2. An AI can verify traits of a program"

"3. An AI can generate novel programs during training"

"4. An AI can generate novel programs post training"

"So far, this library provides an existence proof up to Level 1. It contains code which can execute arbitrary programs, written in a custom lisp dialect, in a fully differentiable setting. (Some tantalizing tests have proven up to Level 3, that AI can learn novel programs to solve toy problems via SGD, but, there are still frontiers of research here)."

By "this library", he is talking about his creation "Neurallambda". What is Neurallambda? It's a dialect of lisp designed to be generated by AI systems but at the same time be human-readable. It also has the important attribute that all the code generated using it is "differentiable". That means the code itself can be incorporated into a stochastic gradient descent model. That's what the "SGD" stands for above. In this form, Neurallambda code can be deterministically translated into tensors, compiled or interpreted, and then the resulting tensors be read back out, and presented in human-readable form.

What do y'all think? Is this a path to computer reasoning?

Thumbnail
"My concerns about the TikTok divestiture bill as a software researcher/developer."

"First, this bill isn't actually aiming to ban TikTok. It really wants to force ByteDance to sell TikTok to someone else who is not controlled by China. It does that by threatening to make it illegal to distribute the TikTok app to Americans unless ByteDance sells TikTok within 180 days of the law taking effect."

"I think what the bill is trying to say is that if ByteDance doesn't sell TikTok, US based app stores have to take down TikTok for US users and US web hosts must not host the TikTok web app for US users. Unfortunately, when I read the text, that's not actually what they have said, because the Internet is complicated and it's hard to write clear rules about it without causing unintended consequences."

"There has been previous attempts by US states, most notably in Montana, to ban TikTok that have been struck down by courts as unconstitutional, because they violate the First Amendment."

"How the bill defines 'hosting services' would seem to reach pretty far into the infrastructure of the Internet to effect its prohibitions."

"The way I read this, I think it would require Verisign to de-platform the domain 'tiktok.com' from the '.com' name servers."

"A related over-broadness concern I have is whether this bill would effect 'peering', or in other words, whether it would prohibit US Internet Service Providers (ISPs) from routing traffic to a TikTok service hosted abroad."

"US law already has the concept of the 'Specifically Designated Nationals list' (SDN list), which is a list of companies and people that US persons and businesses are prohibited from doing business with."

"Another concern I have with this bill is the open-ended nature of the ability to designate other apps as 'foreign adversary controlled applications'."

"I think this definition would cover Telegram, the very popular Russian app whose use is very significant in monitoring ransomware actors and in open-source intelligence of Russian military operations."

Thumbnail
Tor's new WebTunnel bridges mimic HTTPS traffic to evade censorship

The Tor Project has added a "WebTunnel" capability that makes Tor traffic look indistinguishable (or almost indistinguishable) from regular HTTPS web traffic.

"It works by wrapping the payload connection into a WebSocket-like HTTPS connection, appearing to network observers as an ordinary HTTPS (WebSocket) connection. So, for an onlooker without the knowledge of the hidden path, it just looks like a regular HTTP connection to a webpage server giving the impression that the user is simply browsing the web."

Thumbnail
"Race conditions arise when multiple threads attempt to access a shared resource without proper synchronization, often leading to vulnerabilities such as concurrent use-after-free. To mitigate their occurrence, operating systems rely on synchronization primitives such as mutexes, spinlocks, etc."

"Our key finding is that all the common synchronization primitives implemented using conditional branches can be microarchitecturally bypassed on speculative paths using a branch misprediction attack, turning all architecturally race-free critical regions into Speculative Race Conditions, allowing attackers to leak information from the target."

Um. What? That's *crazy*!

"Mutex" here means "mutual exclusion". It is a lock that allows only one concurrent threat to enter a section. "Spinlock" refers to a more primitive technique, where a threat asks "are you unlocked yet?" over and over in a loop until the lock is released and it can acquire it. In modern systems, the hardware and the operating system work together to enable threads to go to sleep and get woken up when their locks are released instead of doing the spinlock thing.

Digging into this further, the researchers say:

"Since 2018, after the discovery of Spectre and Meltdown, transient execution attacks have become an intensively studied area of research."

You know, I remember hearing about Spectre but didn't look into the details of it.

"Whenever a modern CPU implements speculative optimizations (e.g., branch prediction), it speculatively executes a sequence of instructions. The two possible outcome for these instructions are that either they are committed and made visible to the architectural level or they are squashed due to mispeculation (e.g., misprediction) -- leading to transient execution. When the instructions are squashed, the CPU rollbacks the state. Despite the rollback, some microarchitectural side effects are left and can be observed through one of the many side channels available (e.g., data cache, branch target buffer, port contention, etc.) to leak sensitive information."

"Spectre-PHT, also known as Spectre-v1, is the first known attack of this kind, targeting the pattern history table and exploiting a code pattern. The code checks for x to be in-bound before performing a double array access. For exploitation purposes, the attacker can ensure x is out-of-bound and array1_ size is not present in the cache. In this scenario, instead of waiting for array1_size to be loaded from main memory to perform the comparison, the CPU speculates and starts to transiently execute the instructions beyond the comparison. If the comparison has been executed several times before with x in-bound, the CPU is prone to speculate that x is once again in-bound, hence transiently performing the out-of-bound access of array1. When the not cached array2 is accessed using the byte retrieved from the out-of-bound access of array1, the specific accessed location is loaded into the cache. The attacker can complete the 1 byte leak by testing which location of array2 can be accessed faster than the others. Its position within the buffer reveals the secret byte value. Notably, Spectre-PHT remains unmitigated in hardware. Software developers remain responsible to harden potentially vulnerable branches with mitigations (e.g., fencing to prevent speculation), but the extent to which all the 'right' branches have been adequately hardened in large high-value codebases such as the Linux kernel remains an open question."

"Concurrency bugs are a category of bugs which affect multithreaded programs and occur due to the absence or the incorrect use of synchronization primitives. Due to their nondeterministic behavior, concurrency bugs are one of the most elusive and difficult to triage classes of bugs. Under certain conditions, concurrency bugs can also lead to memory error vulnerabilities. In modern operating systems such as the Linux kernel, one of the most common memory error vulnerability caused by concurrency bugs is use-after-free."

"In a use-after-free attack, the first step is generally to free a memory object. This operation invalidates all the pointers to that object, which become dangling. The second step generally involves forcing the allocator to reuse the memory slot of the free object for the allocation of a new object. This step reinitialize the previously freed memory slot. The final step of the attack is generally to force the victim to use one of the dangling pointers, which now points to the newly allocated object. A read from or write to such pointer to controlled data can be used to exploit the bug in a variety of ways."

"When this attack is performed in concurrency settings, and the free step and the use step are executed by distinct threads sharing the underlying object. Such concurrent use-after-free vulnerability is harder to exploit than the single-threaded use-after-free case, since exploitation depends on thread interleaving and the availability of a sufficient race window. While the community has invested significant effort in investigating traditional concurrency bugs and concurrent use-after-free -- e.g., studies demonstrating that more than 40% of the use-after-free vulnerabilities patched in Linux kernel drivers are concurrent use-after-free -- their microarchitectural properties have largely been neglected. In this paper, we study such properties and their security implications for the first time, uncovering a new class of speculative execution vulnerabilities in the process."

They go on to explain their new exploitation technique to precisely interrupt any (kernel) thread and create an architecturally unbounded use-after-free exploitation window. This works by first identifying use-after-free exploitation windows as tiny as eight instructions. Then they employ high-precision hardware timers to interrupt the victim thread at just the right time and amplify the original UAF window. After that, they rely on user interfaces to trigger an interrupt storm to interrupt the victim thread in the amplified window, which has the effect of stretching the UAF window indefinitely. Probably should menton that by "user interfaces", here they mean things like the host controller interface layer of the near field communication (NFC) driver.

Then they go on to exploit speculative race conditions, their new term for speculative execution vulnerabilities "affecting all common synchronization primitives", by which they mean mutexes, spinlocks, etc. "We can consistently trick speculative execution into acquiring a mutex and entering the guarded critical region. Since this is the case regardless of the current (architectural) state of the mutex, we can speculatively acquire a mutex already held by another thread. In other words, the mutex becomes a no-op on the speculative path, leading to a speculative race condition and opening the door to arbitrary concurrency vulnerabilities at the microarchitectural level."

The end result of all this is that they can leak memory from the Linux kernel at a rate of 12 KB/s.

I have to say, I'm amazed people exist who can pull stuff like this off.