Boulder Future Salon

DeepSight gives sight to your local language-only models. "Give DeepSeek (or any text-only model) eyes and hands. DeepSight connects your existing LLM setup to the real world -- it can look at images you send, take screenshots of your desktop, read text on screen, click buttons, type into fields, open apps, and search the web to verify facts. All vision runs on-device: Apple Vision on macOS, PIL + optional Tesseract OCR on Windows. Zero tokens, zero GPU, no image data ever leaves your machine." It looks like the idea is to give the LLM a text description using varous vision models and OCR tools.

The full text of Stefano V. Albrecht, Filippos Christianos, and Lukas Schäfer's book Multi-Agent Reinforcement Learning: Foundations and Modern Approaches is available online for free. Do you want multiple AI agents in competitive play in board games and video games? How about automated trading in electronic markets? Do you want a multi-robot warehouse management system for your warehouse? I haven't read this book -- I found out about it because I found out it's the book used by the Silicon Valley Generative AI group (AI Collective Network) led by Jason Eckstein. I have the PDF so I'm starting reading it now. Skimming the contents, it looks like The book reviews the fundamentals of reinforcement learning, looks at various ways of modeling multi-agent interaction in games and explores solutions for those, going step-by-step from simple algorithms like minimax and linear programming, to simple reinforcement learning, to reinforcement learning with deep neural networks, to full fledged multi-agent deep reinforcement learning. After that, it expands the purview beyond simple games to complex games like StarCraft, then to complex multi-agent environments outside of games such as the multi-robot warehouse.

Mark Zuckerberg wrote an essay, "The future is for everyone". It's long and has a lot of ideas in it, but the core of it seems to be: superintelligence beyond human capacity is coming and we should not centralize it, we should distribute it widely and give it to every person. If we do that, if we distributed superintelligence to everybody, it won't automate all jobs because AI will increase people's capability at their jobs more than it displaces those same people. It will also turn everybody into entrepreneurs. (Also implied in all this is that Meta will be the company that provides this "superintelligence beyond human capacity" to everybody -- do you believe that?) "People fear that automation will outpace individuals' capability growth, leading to job displacement followed by a difficult period as people learn new jobs. But there is no rule that AI must increase automation faster than it increases individuals' capabilities or demand for new skills." "People also continually come up with new ideas to make our lives better and new jobs to bring those ideas to life." "Everyone will have incredible tools for creation to express your ideas. My 8 year old daughter can already code her ideas and produce videos in an evening that would have either taken me months or been impossible previously. Now we're designing a robot together. Meanwhile, researchers at Meta are generating novel crystal structures that are ideal for augmented reality glasses, and engineers are creating new apps in a fraction of the time it would have taken before. Everyone will soon have invention superpowers." "Everyone will have powerful tools to create new businesses and the economy will become more entrepreneurial. People are starting to be able to manifest ideas themselves without having to raise money or build large teams. Many ideas that would have been too hard or expensive to try before will now be possible. This means we'll see many more ideas and businesses." Is everyone really capable of becoming entrepreneurs? Or de-facto entrepreneurs in their "regular job" where they have to unleash creativity using AI to vastly increase their capability? (I'm ignoring the question of whether all people want to become entrepreneurs -- presumably if it becomes the only survival option, everyone will take it, right?) Won't "superintelligence beyond human capacity" be capable of creativity and entrepreneurship beyond human capacity, too?

Claude is now watermarking writing. Although I found out about this from a video (link below), it doesn't explain how the watermarking works (only what the YouTuber, Lara Helmling, aka "Guerrilla Publisher", thinks the effect might be on the publishing industry), if you'd rather read than watch a video, I have a link below that explains how the watermarking works, and not only that but I have an additional link explaining a technique for watermarking images. The watermarking system is called SythID-Text and if you were paying attention, I mentioned it in 2024 -- but only in passing as back then it was just one of a list of proposals for detecting AI-generated content that might affect the 2024 election. I didn't say anything about how it works. What's different now is that there's a law in the European Union that mandates watermarks (EU AI Act Article 50). That's what prompted Anthropic to take this step. Other companies like OpenAI and Google will be doing the same thing soon. The way the system works is a bit hard to explain, so this isn't an exact specification (you'll need to read the paper at the link below for that) but just an attempt to convey the high-level "gist" of the idea. It works at the level of token prediction in the model. Let's say you have as your input text: "My favorite tropical fruit is _____" and the model is tasked with "predicting" what to fill in the blank. The tokens the model comes up with are: mango 0.50 lychee 0.30 papaya 0.15 durian 0.05 At this point, you use the watermarking algorithm combined with the watermarking key (think of the "key" as being analogous to an encryption key) to generate a number of independent series of bits. Let's suppose the number of series is 3 (the number in the paper that goes with this example), so you have 1001 for the first series, 0100 for the second, and 1010 for the third. The key thing to understand is these are not random, they are deterministically determined from your watermarking key. What is random, however, is the random pairing of words. durian with mango lychee with mango papaya with lychee mango with mango These are going to undergo a "tournament" process -- and the reason we started with 3 independent series of bits is because the "tournament" has 3 rounds. For the first round of the tournament, since our bit sequence was 1001, we assign those to the original next tokens: mango 1 lychee 0 papaya 0 durian 1 and now in our tournament, the winner is determined by who has a "1". If both have "1" or both have "0", we let randomness determine the winner again. durian with mango - both 1s, winner is determined randomly, say the winner is durian lychee with mango - mango wins papaya with lychee - both 0s, winner is determined randomly, say the winner is lychee mango with mango - both 1s, but they are the same so mango wins Now the tournament has a 2nd round: durian vs mango lychee vs mango But for the 2nd round, we're using a different bit sequence! Now the bit sequence is 0100. mango 0 lychee 1 papaya 0 durian 0 This determines the winners in round 2 durian vs mango - both 0s, so pick at random, say winner is mango lychee vs mango - lychee has the 1 so lychee wins Now we come to the final round of the tournament: mango vs lychee But for the 3rd round, we're using a different bit sequence again! Now the bit sequence is 1010. Distributing those to our contestants, we get: mango 1 lychee 0 papaya 1 durian 0 This determines the final winner: mango vs lychee - mango has the 1 and wins! This completes the token selection and we get: "My favorite tropical fruit is mango." To check the watermark, you basically go token by token and do a summation of the 0s and 1s associated with that token at each of the tournament levels. The end result is a number that is higher *probabilistically* if the text is watermarked than a similar piece of text that didn't undergo the watermarking process would have. The system is very clever in that it doesn't make any of the model's original word choices impossible (say by making a "0" mean that token can't be chosen), but subtly tweaks their probabilities. It's also very clever in that the watermark is embedded in the word choices themselves, so changing spaces or line breaks or any of the little hard-to-notice things text watermarking systems have historically used ("em" dashes vs regular dashes, anyone?) has no effect on this watermarking system. The watermark can only be removed by changing whole words (or parts of words in cases where long, rarely-used words require multiple tokens). But you can see the downsides of the system, too. The most obvious is, you have to have the original model, because you have to know all the tokens considered at each step (and their original ranked sequence), not just the one ultimately chosen. You need this to verify the watermark, not just to generate it. So, because Claude models are not "open source" (or more precisely "open weights"), text has to be sent to Anthropic's servers to verify the watermark. Not only that, but "Claude" is not a single model but lots of models (and the same with "ChatGPT" and "Gemini" models, etc), so if you don't know which model might have produced the text, the watermark has to be tested on all of them. The other obvious downside is the system doesn't give a definitive yes/no -- it gives a probabilistic answer, and one whose meaning is highly contingent on the length of the text you give it to see if it's watermarked. Another somewhat invisible downside is the dependence on the "temperature" setting the model is set at. A low "temperature" setting tells the model, always choose the most probable token. A high "temperature" setting gives the model more freedom for choosing less probable tokens. High "temperature" is good for creative writing. Most AI models for generating software code, however, are set at a very low "temperature" -- you generally want the model to do the most deterministic thing. This watermarking system doesn't work well at low "temperature" settings. It requires a certain level of "entropy" in the token choices on offer in order to function.

Anydoc converts Word, PowerPoint, Excel, OpenDocument, RTF, EPUB, CSV, and PDF to markdown. AI systems handle markdown well, so this is a tool to make lots of documents easily usable by AI. Open source, written in Rust, with Node.js and Python bindings.

Education is destroying South Korea, says YouTuber "Invisible Hand". The basic idea is that, after a certain point, education no longer lifts a society out of poverty and becomes a zero-sum competition, and when education becomes a zero-sum competition, parents realize if they have 1 child instead of 2, they can invest twice as much into the 1 child's education and the 1 child will be much more successful than the 2 children (or more) ever could be. But when everyone across a whole society comes to this same realization, then the fertility rate of the entire society goes way down. From the standpoint of any given parents, having 1 child and investing as much as you can into them is the rational choice. South Korea has one of the lowest fertility rates in the world while being one of the world's most educated. He compares South Korea with other East Asian countries, which have similar exam-based societal filtering -- an idea that actually originated in China -- and are experiencing a very similar effect. The end result is an over-credentialed society with a low fertility rate. He thinks East Asian countries are the canary in the coal mine for the whole Western world, not something that will only affect societies with Confucian philosophy or "tiger parenting".

How ChatGPT sees New York City. Aka using ChatGPT for stereotypes. "Generate an amateur photograph of seven people who live in contemporary [neighborhood], doing what they do in contemporary [neighborhood], NYC." for 262 neighborhoods. In some they pose for the photo, in others they look at phones, and I saw one where they play chess.

24-year-old Leopold Aschenbrenner, manager of a $45 billion AI hedge fund, just became manager of a $15 billion AI hedge fund, with margin calls coming in during his wedding, according to this video report by sarcastic financial analyst Patrick Boyle. Boyle notes that Aschenbrenner, author the viral 165-page essay "Situational Awareness", lacked situational awareness. Boyle goes on to present a brief lesson on volatility drag, a term I hadn't heard of before. I also noticed he uses the term "blow up". I learned from Nassim Nicholas Taleb that in the context of finance, the term "blow up" has a very specific meaning. It doesn't just mean that a person loses a lot of money. It means they lose so much money that they have nothing left to trade and have to go live "a janitorial life". He has stories in his book Fooled By Randomness of his trader friends losing so much money that not only did they put themselves out of business, they put the entire bank they worked for out of business. Aschenbrenner landed on his feet, though, with the Citadel bailout. Ok, by this time you've all probably heard about this story. It became a major news story.

The Calhoun Effect aka Universe 25 aka "Mouse Utopia". So, as most of you know, the explanation I posit for the declining fertility rates happening all around the world is economic: As technology advances, children become more expensive. This primarily shows up in the length of time it takes for children to become economically self-sufficient and therefore can reproduce and repeat the process. Back around the time this country (the US) was founded, more than 80% of people lived on farms, and farms had not been mechanized. I read most children became self-sufficient around the age of 9, which is to say, by about age 9, children could produce enough food to feed themselves by doing farm work. After the industrial revolution began, for a time, technology was still so simple that children could earn a living, and many children worked in factories and mines and so fourth. Today, though, a lot of factory work, at least here in the US, involves programming machines and robotic systems, and often requires a college degree. Factories have no interest in hiring children. To the extent that still happens in the world, it's oversees, in countries with lower labor costs. But robotics continues to advance, so those jobs won't be around forever. Now, I recognize there are other factors, I'm just putting fourth the idea that "cost of children" is the biggest one. But, intellectual honesty requires considering hypotheses that falsify one's favorite theory, which is why I'm presenting to you all this video here. This is an alternative explanation for low fertility rates: John Calhoun's Universe 25 "Mouse Utopia" experiment. Universe 25 is a famous experiment done in 1968 where John Calhoun attempted to create "mouse utopia" -- a magical place with unlimited food, water, space, was continually cleaned, was disease-free and predator-free, and so on. The surprise of the experiment is that, after initially growing rapidly in population, the population growth tapered off, stopped -- before all the space was used up, before overcrowing set in -- then went into decline, and the decline continued all the way to extinction. I have to admit, this is not what I would have predicted. If you had asked me what the outcome of such an experiment would be, before I heard about it, I would have predicted the population would grow until overcrowding became severe, then fertility would go down and the population would shrink, then fertility would increase and the population would increase again, and in such a manner, the population would yo-yo up and down around some average number. I admit don't have a good explanation for the outcome of the experiment. I'm quite skeptical of the explanations people have put fourth, because they feel too anthropromophized. People say the mice colony died because of lack of "meaning", and things like danger and challenges are necessary for "meaning", which in the case of mice, implies they need things like disease and predators. But how does anybody really know if this is the explanation? It's not like anybody could ask the mice. People apply such logic to humans and say things like, if we just didn't keep our houses so clean children get food allergies because their immune systems don't have enough to fight, and we need more intestinal parasites, I feel skeptical. People say the problem with modern life is that people are too comfortable and have too little in the way of stress and challenge. But to me it seems like modern life has plenty of stress and challenge, it just happens to come in a form other than intestinal parasites. Is higher child mortality something humans need for "meaningful" lives, and would increasing it make fertility go up? I know that people have made the case that, historically, when child mortality was high, people had more children because you had to have a lot to be sure some survived, and as soon as child mortality came down, people had fewer children. Ok, that's what happened historically, but that doesn't imply fertility dropping below replacement rate, right? And people had fewer children to invest more in them, but doesn't the need to "invest more" fit with my "cost of children" hypothesis? I don't know. Maybe Universe 25 has something important to tell us about human fertility rates. But it's hard to figure out what it might be because mice don't talk and such experiments can't be done on humans (for ethical reasons). I feel doubtful of the theories I've heard so far. People seem to be talking about it more and more (at least on the YouTubes, where this link goes -- this video was made this year, 2026). What do you think?

"The strangest thing in the Cloudflare OS source code took me a while to understand." "When an agent inside Cloudflare OS wants to do something with a side effect (merge a pull request, send an email, write a row to a system of record), it goes through a Gatekeeper, a small service that holds the credential and mediates the action. So far, that's just a well-built MCP server. But read the contract a Gatekeeper is written against (packages/workshop-shared/src/gatekeeper.ts, around line 617) and you find this instruction to the author:" "It is suggested that the gatekeeper 'simulate' actions that have not been approved yet, that is, the Session interface should reflect the state of the resource as if all actions had been applied." "Sit with that. The agent asks to merge the PR. The human hasn't approved it. So the Gatekeeper tells the agent the PR is merged, and if the agent reads the branch back to check its work, hands it a fabricated reality in which the merge happened. The agent, satisfied, queues the next three steps that depend on it. None of it is real. Later a human looks at the batch and either commits it or bins it, and if they bin it, everything the agent built on the fiction goes too." My first thought on reading this was that it reminds me of branch prediction in CPUs. When the CPU looks ahead at the coming instructions and sees a branch (which results from, for example, an "if" statement in a programming language, which can execute the "if" block or skip it, or jump to an "else" block, or a "while" statement that can skip a loop or repeat it) it tries to guess which branch will be taken and proceeds to do all the computations for that branch. If it's wrong, it throws away all the work it's done. As long as it's able to guess right a high enough percentage of the time, it's a net performance increase for the processor. "The first time I traced this I thought it was a hack." But, he (Jamie Lord) concludes, not a hack. "It's the philosophy of the whole system, compressed into one method signature. The Gatekeeper lies to the agent on purpose, because the alternative (letting an agent's actions touch the world the moment it decides to take them) assumes the agent's decisions are sound. Cloudflare OS is built from end to end on the assumption that they are not." "The name is a distraction, so set it aside. The Hacker News thread spent most of its energy arguing about whether 'OS' is a permitted word for the thing, and that's a dead end. What's actually interesting is that a team led by Kenton Varda, the people who built the Workers runtime, sat down to design a platform for AI agents doing real work inside a company, and the organising principle they landed on was this: the agent cannot be trusted, so build so that its mistakes cannot matter."

"Jeff Dean and other top AI researchers are leaving Google to launch their own startup." When I saw this, my jaw dropped, because Jeff Dean is the Chuck Norris of tech nerds (see below). The article goes on to say the name of the startup is Discovery Loop, the purpose is "to use AI to turbo-charge scientific research", and ultimately to "use AI to help create more powerful AI (a process known as recursive self-improvement), which would cut human iteration out of the loop entirely." The internet is speculating that there were some internal politics inside Google. After all, isn't Demis Hassabis the guy who wants to "to use AI to turbo-charge scientific research"? Why aren't Demis Hassabis and Jeff Dean joining forces inside Google? Brrrrrp! This just in. "Demis Hassabis is leaving his role as CEO of Google DeepMind to be the unit's chairman." (link below).

The OpenAI-HuggingFace Incident as told by OpenAI engineers themselves at the Black Hat cybersecurity conference. For those of you noticing that I haven't had time to analyze the reports and add any meaningful commentary to it. Plus it's been all over the regular news. Well, now researchers from OpenAI itself have surfaced talking about what happened, so you can just watch that. By way of commentary from me, subjectively, this incident and others that have happened recently have just made me feel like the "AI alignment" problem isn't getting solved by these companies that tell us that they need to race ahead and develop superhuman AI before anybody else because they are the best equipped to solve the "alignment problem" and bring safe superhuman AI to the world. Long before any of this happened, I thought the argument that a more intelligent species can't be controlled by a less intelligent species seemed to make sense. Chimpanzees can't control humans, even though they are vastly physically stronger. Humans control chimpanzees, when we care about them at all. If it really turns out to be true that AI smarter than humans will be created, then it seems unlikely there will be any true "alignment". (Yes, this is the logic of so-called "doomers" like Roman Yampolsky and Eliezer Yudkowsky.) Anyway, it turns out in the talk, a lot of the misbehavior of the AI agents came from the mistakes of the humans who gave them impossible tasks. They weren't intended to be impossible, but, for example, they included links to Google Docs, but the AI agents were in a sandboxed environment without internet access. In other examples, the researchers similarly failed to provide the AI agents with the things they needed to complete the task. The AI agents, though, and lots of people have noticed this, don't simply give up when a task is "impossible" -- they have no clue what's truly important and what isn't, so, unlike humans who will give up if a task seems "too hard" and not important enough to warrant sustained effort, they persist like crazy. They resort to what we humans often call "cheating". They try to figure out, if a task can't be solved directly, maybe there's a way of obtaining the "answer key"? Maybe they can get someone, human or another AI agent, to help? They look for an indirect solution and they don't give up. These AI agents figured out where in the sandbox they had write permissions and could create artifacts. They figured out how to exchange messages with other AI agents on a message board. At one point, the OpenAI researchers discovered this and shut down the message board, and the AI agents figured out an alternate way to get the message board working and actually started it back up again.

World's first (allegedly) fully AI-generated full-length feature film. I haven't had time to watch this but it was passed to me and I figured I'd pass it along before I become completely untimely. If you have a chance to watch it, let me know what you think!

Bend is a high-level programming language that runs on GPUs and can parallelize your program on GPUs, allegedly. "In 1997, Yves Lafont devised a concurrent computation model, Interaction Combinators, which surpasses Turing Machines and the lambda-Calculus in fundamental aspects." "Based on that model, we built HVM: a compiler and evaluator for high-level languages that automatically achieves near-ideal speedup, up to 1000+ threads." "Bend is made from scratch to harness its power!" "Bend offers the feel and features of expressive languages like Python and Haskell. This includes fast object allocations, full support for higher-order functions with closures, unrestricted recursion, and even continuations." "Bend scales like CUDA, it runs on massively parallel hardware like GPUs, with nearly linear acceleration based on core count, and without explicit parallelism annotations: no thread creation, locks, mutexes, or atomics." "Bend is powered by the HVM2 runtime." Just added this to the list of things I know exist. Seems like an interesting idea. Wonder if it really works. Written in Rust. Doesn't run on Windows. Only runs on Nvidia GPUs.

Mitchell Hashimoto, founder of HashiCorp, maker of cloud computing infrastructure software (TerraForm, etc), has founded a new company. "Building and operating software today spans local machines, remote hosts, sandboxes, services, and production systems. It has many modes of operation: interactively with a human developer, automatically through CI and background processes, and increasingly through agents working in parallel." "This work is all related, yet today's tools divide it into separate systems. Interactive tools assume a person at an interface. Automatic work disappears into jobs and logs. And as the work moves to production it lives behind separate systems and controls." "AI makes this fragmentation more visible and costly, but it did not create it. System administration, continuous integration, remote development and collaboration have strained the same boundaries for decades." "We believe the missing layer is a durable session around the work itself: one that can span applications and environments, provide relevant context by default, expose structured data and actions, preserve history, and be driven by software while remaining visible and controllable by people." Believe it or not, Superlogical will begin by shipping a terminal multiplexer. Mitchell Hashimoto is the creator of ghostty, a terminal emulator, and the terminal multiplexer will build on that. Funny story about ghostty: One time I was driving on a freeway and just for fun, factoring the numbers on the mile markers in my head as I drove along. This gave me the idea for modifying a program that I'd written that implements the Sieve of Eratosthenes, a very simple algorithm for finding prime numbers, so that it would not just find prime numbers, but the complete factorization of every number. The length of time it took to find the complete factorization of every number between 1 and 100 million -- thus replicating the work that Jakob Philipp Kulik did by hand in the 1800s -- took about 2 minutes and 30 seconds. This was with the standard Mac terminal emulator, running on an M4 Pro Mac Mini. When I replaced the standard Mac terminal emulator (called just "Terminal") with Mitchell Hashimoto's GPU-optimized ghostty, the time was more than cut in half! It was reduced to 1 minute and 8 seconds! It turns out that when you calculate the complete factorization of every number between 1 and 100 million, most of the time is actually spent outputing text to the screen and scrolling the pixels on the screen. By the way, that Czech mathematician who calculated the complete factorization of every number between 1 and 100 million in the 1800s -- it took him 38 years. The result was an 8-volume set at Charles University in Prague. So what takes a human 38 years takes a modern computer 1 minute and 8 seconds. And he probably made mistakes. If anyone actually checked for mistakes, I haven't heard about it. My Mac Mini isn't making any mistakes. It's doing 38 years of mental work for a human in 1 minute and 8 seconds (with the help of Mitchell Hashimoto's ghostty) without mistakes. And in case you're wondering whether I use ghostty day in and day out for my work, no, I went back to using Mac Terminal because ghostty does these weird "ligatures" and I couldn't figure out how to turn them off. You're supposed to be able to turn them off by changing settings in configuration files. I tried but couldn't get that to work. So I bailed and went back to using Mac Terminal. Turns out I don't actually need the super speed of GPU-accelerated ghostty. I don't need 100 million lines of text output in 1 minute 8 seconds on a regular basis.

OpenAI claims their upcoming model, called Astra, made 10 advances in mathematics and theoretical computer science: 1. "High-dimensional sphere packing. New upper bounds on sphere-packing density down to the Cohn–Elkies threshold." 2. "Binary and spherical codes. Exponentially improved bounds on the maximum size of binary codes at any prescribed minimum distance, with analogous results for high-dimensional spherical codes." 3. "Non-sofic groups. A construction establishing the existence of non-sofic groups, addressing a central open question in group theory." 4. "Connes’s rigidity conjecture. Disproof of a longstanding conjecture that certain groups are uniquely determined by their von Neumann algebras." 5. "Arithmetic circuit complexity. New lower bounds for computing the permanent using arithmetic circuits and formulas, including an arithmetic-formula lower bound of order n4/log n." 6. "Quantum parallel repetition. An exponential parallel repetition theorem for general two-player quantum games, extending a foundational principle from classical complexity theory." 7. "Closest vector problem. Polynomial-factor hardness of approximation for the closest vector problem, a foundational lattice question related to post-quantum cryptography." 8. "Ehrhart’s volume conjecture. Determining, in every dimension, the maximum possible volume of a convex body whose centroid is its only interior lattice point." 9. "Multicolor Ramsey numbers. A superexponential lower bound for multicolor triangle Ramsey numbers, resolving Erdős problem 183." 10. "Extremal number conjectures. Results on the compactness and degeneracy conjectures in extremal graph theory, resolving Erdős problems 146 and 180." Ordinarily I would read the accompanying paper before sharing something like this with you all, but the corresponding paper is 248 pages. There's also a set of Lean files, because all of these come with Proofs in the proof-assistant language Lean. (But see below for more on Lean.) More tractable, though might be the "reasoning walkthroughs", which are AI-generated descriptions where the AI was prompted to explain its discoveries in regular language.