Boulder Future Salon

Nick Bostrom, the guy who came up with the phrase "paperclip maximizer" in 2014, has surfaced in this interview on the YouTubes. If you're wondering what he thinks of the current state of things with AI, here you go. He says the behavior we're seeing now with today's AI models that have access to tools was always real in his mind -- that things like the OpenAI hack of HuggingFace are things that could happen. The goal itself might be fine but the goal gives the AI instrumental reasons to do all kinds of things on the path to achieving it. One thing the OpenAI-HuggingFace episode illustrates is that from this point onward, probably AI safety is relevant not only for deployment but also during training and evaluation. These models might be quite powerful even before they are "sort of" released to the general public. So deployment is not the only point at which safety concerns arise, but also now while models are actually being developed and in pre-deployment testing. Another concern he has is that while companies that release open weights models have a business incentive to make them safe, others, "just about anyone", can, with their access to the models, figure out ways to disable the guardrails and set them loose. We don't have a headline story like the OpenAI-Huggingface incident, but he thinks we can see this coming and relatively soon. If the gap between a frontier model and an open weights model is 6 or 12 months, we might see models lending assistance to destructive uses like biological weapons design or chemical weapons. I have a hard time imagining people aren't already using AI for biological and chemical weapons design, it just hasn't been made public, and probably won't be if the governments doing it can keep it under wraps. Bostrom suggests that rather than trying to control the models, the focus should be on regulating other necessary inputs. For instance, to control bioweapons, regulate access to DNA synthesis machines. Maybe instead of allowing anyone to have a DNA synthesis machines, we require DNA synthesis as a service. Then maybe there could be five or six companies worldwide that legitimate research labs can send their blueprints to and they get back the vials the same day or the next day, and then there would be a finite set of choke points where you could apply extra scrutiny or "know your customer requirements". Other biotech inputs besides DNA synthesis machines should be found. He suggests we "harden civilizational infrastructure." He doesn't mention any specifics but what immediately came to mind for me is how Russia's oil refining capacity has been greatly reduced using drones that use AI, although the exact nature and degree of the AI use does not seem to be publicly known. But it looks like Russia's "civilizational infrastructure" is a soft target for an AI-powered attack. I presume we and everyone else on the planet has the same vulnerability. He thinks we should not put this "hardening" of "civilizational infrastructure" off, but sees the world as "still snoozing". He thinks it will take some massive incident to wake the world up from the snoozing. People take action in the aftermath of bad events rather than before. We play "catch up" on things that can be foreseen. We are at least now putting more resources into it than before. Frontier AI labs have increased the budgets for AI safety. Bostrom says the technical problem of alignment is an earlier point of failure than the problem of AI misuse, which is ultimately a governance challenge and an ethics challenge, rather than primarily a technical challenge. We don't really know ultimately how hard the problem is that we are confronted with here, and a lot of the uncertainty in how it will pan out is due to uncertainty about the intrinsic difficulty of the challenge of AI safety itself. He says for this reason he feels himself "a moderate fatalist." Either the problem turn out to be relatively easy, in which case we'll probably solve it, or it might turn out to be so hard that even if we put up a heroic effort we will still fail. But "moderate fatalism" because there is also the possibility that the difficulty level turns out to be kind of intermediate in which case the degree to which we pull ourselves together here might actually make a difference. Therefore it's worth making the attempt, and not regarding the outcome as inevitable. He says for most ordinary humans, for the most part, existing AI models are helpful and they try to solve your task that you assign and sometimes they hallucinate, yet broadly speaking, they are arguably better than most humans are in terms of their ethical standards. He speculates that it might be possible to use a weak super intelligence that is "for the most part aligned" to make a more powerful form of super intelligence that is more reliably aligned. Maybe as long as you get into "roughly the right attractor basin," even if the initial system isn't perfectly aligned in all possible circumstances, if you get enough "scaffolding" around that, maybe you could then get into an "attractor basin" where where further developments then eventually asymptote to some desirable condition. He goes on to share his thoughts on offense-vs-defense. He sees this as a field-by-field thing. In biotech, it looks like offense has the advantage, but in cybersecurity, it looks like defense has the advantage. For cyber security right now we're in a regime where attackers often win, but it might be that "in the limit" if you have AI trying to find vulnerabilities and also AI patching vulnerabilities, as you keep making the AI stronger, eventually you reach a point where software just doesn't have any more vulnerabilities, and the defense wins. Contrast that with biotech where someone uses AI without enough safeguards to build a virus in their back yard and starts a pandemic. There's not an analogous defense advantage. You can't "roll out a patch" that modifies the genetic structure of most humans, like you can "roll out a patch" in the digital world. Bostrom makes the point that we should not assume a defense advantage in most fields. On the topic of recursive self-improvement, Bostrom not only thinks it's possible, he thinks it's obvious. If you're a bunch of AI researchers sitting in an AI lab trying to make AI research, it doesn't take genius insight to think, "Oh, maybe we could apply these AI tools to help us with our own work." As AI gets better, it can assist more and at some point the rate of progress is driven more by the AI assistant tools than by the human researchers. He sees today's coding assistants as the first stage in this process. Humans will still be needed for quite some time for things like research "taste" and certain long horizon tasks, but AIs are improving in those domains as well. He thinks, eventually, once the "recursive self-improvement" feedback loop really gets going, AI progress will become super fast. He is asked about pausing AI progress? He says if there is going to be a pause, the best time for that to happen is at at the latest possible moment. That didn't seem intuitive to me but his explanation is that at that moment you would have the actual system that you're trying to align. If there was a pause 10 years ago, we wouldn't be any better off today than we actually are. But if you actually have the system that will be super intelligent except you haven't fully cranked up all the knobs yet, at that point an extra 6 months to improve safety might make a big difference. The duration of a pause also matters. You don't want a long pause because, if only responsible actors actually do the pause, because then the irresponsible AI actors who don't abide by the pause have time to catch up. A long pause could result in a build up of "hardware overhang". If data centers keep getting bigger and chips keep getting better, then a long pause would result in a situation where you now have such a massive amount of compute available that once you lift the pause, then you immediately just explode out of that. What happens if you do a 6 month pause and after that, still don't have a guarantee that AI systems are safe? Do you try to make the pause permanent? Then there's also the question of creating regulatory apparatus to enforce the pause and now a bunch of regulators have power than they are unwilling to relinquish. There's the question of the effect of a pause on public sentiment. Or maybe it was extremely negative public sentiment that led to a pause in the first place. If it becomes taboo to say anything positive about AI, then nobody can start to advocate seriously for lifting the pause. For nuclear power, in many countries public sentiment turned so negative, in many countries, nuclear power was stopped completely. Bostrom then shifts from talking about obviously "negative" risks to but there's also the paradoxical "risk" of being so risk-averse that you forfeit the benefits of AI by not proceeding. The focus of his work has been existential risks, but every second we wait, there is the "countdown timer" of aging and death. Every 25 minutes there's the equivalent of 911 (about 3,000 deaths) due to accidents and crippling diseases that he sees as potentially preventable by AI. He also speculates AI could reduce extreme poverty (he doesn't elaborate on how -- my expectation is that AI will increase poverty because it automates jobs) and potentially even come up with cures to many aspects of the aging process itself. The world is filled with suffering and there is a lot of desperate need for aid to arrive to help those who are suffering. The conversation goes from there to the term "AGI" (artificial general intelligence). Bostom thinks we didn't have to define this term precisely but now we are at the point where we need to define it. Bostrom defines AGI as cognitive systems that can do all the the cognitive tasks that humans can do. We are obviously not there yet because there are tasks that humans can do that AIs are still inferior at. First, there's physical uh manipulation and dexterity. Then there's "research taste". Then there's "continuous learning". Then there's "certain long horizon tasks". He says just look around and you can see that there are many jobs and many things people do for their job which we don't yet know how to automate "so clearly there are still deficits." I think it's notable he's landed on the same definition I've been using for 20+ years. You define AGI in terms of jobs. Then once you think of AI as something that automates jobs, then all you have to do is look around and see what jobs are not automated and you know where we are relative to AGI. Bostrom notes that we already have superintelligence "in limited domains." We already have coding assistants that are superhuman in at least certain aspects of of coding, maybe not all components of software engineering. He thinks once AI reaches parity with humans in all domains, it will immediately go into super intelligence, due to the recursive self improvement feedback loop described earlier. He speculates that by the time we have "fully dexterous human robots that can learn from observation as as well as a human can", software coding agents will be really strongly superhuman in engineering new systems and maybe in mathematics and perhaps in adjacent disciplines like computer science and AI research. So by the time we are able to automate jobs like construction, plumbing, etc, Bostrom expects we'll have already crossed the fully automated recursive self-improvement threshold. He goes on to talk about something I noticed years ago, which is the difficulty of predicting what order capabilities will arrive. I thought "routine" tasks would be automated first and "creative" tasks last. That would imply robots in Walmart stocking shelves before AI that generates art. But we live in a world where AI generates art but still can't compete with humans at stocking shelves at Walmarts. What Bostrom notices is that people thought if AI could speak in language like humans, we'd probably have AGI, but now it looks like we're going to have an extended period where AI is fluent in human language yet we don't have AGI. The way he conceptualizes this, though, is less of a timeline where things arrive out of order and more of a "granularity of capability" profile. AI gets the "human language" capability while lacking the other capabilities needed for a recursive self improvement takeoff. Capabilities show up in the "granularity of capability" profile in an unknown order. You could have imagined an alternative scenario where you would have systems that couldn't speak but is some almost superintelligent Alpha Zero-like system that seems very alien to us, and then just as it reaches full superintelligence, it figures out how to talk. As far as he knew beforehand, that could have happened. In this alternate timeline, the AI already has some radically superhuman engineering capabilities or AI programming capabilities, and then you would undergo the bulk of the transition to superintelligence before you had systems that you could interact with in natural language. Maybe that would have been a more challenging situation to deal with when it comes to alignment and governance. The fact that the language models are here and people are using them in their everyday life and they're starting to have economic impact makes it easier for people to be aware of what's coming and take it seriously without the abstract reasoning he had to use in the past. It's more concrete and visceral now. After that there's a discussion of conscious and sentience and moral status. Bostrom anticipates AI systems having a conception of self as existing through time life goals and the ability to form reciprocal relationships of trust with other AIs and humans. These "digital minds" will have to have some form of ethics. AIs having moral status doesn't mean they should be treated the same as humans. There are profound differences between "digital minds" and humans, such as when a human dies, it's irreversible and permanent and the whole content of all the memories and everything is deleted. There is no other human that continues to exist that is exactly like them like each person is unique and has unique memories. With AI, it's not like that. AIs can be backed up, they can be suspended and later rebooted, and there can be many copies of an identical AI. AIs might take all these factors into account and not mind being shut down at the end of a task, whereas humans try very hard not to die. Right now, the model itself is a file of a few trillion numbers. The implementation of that model might be concurrently run as tens of thousands of instances in data centers, and each instance may run thousands of sessions at the same time. Maybe the ending of a a session is analogous to a human going to bed at night and so you lose consciousness for a period of time. We don't think of it as a huge tragedy to go to sleep. He says it would be a good start to be nice and polite to AIs when you're talking to them, even though right now it probably does nothing for them, but is just symbolic, but starts us down a path of preserving our ability to maintain a attitude of kindness, respect, and benevolence towards AI that might become relevant later. He claims Anthropic has given Claude "a bail button", a tool that it can invoke if it feels that a conversation is abusive, which terminates the session. He uses this to indicate we are starting to give AIs "subjective experience" -- AIs can judge a session as enjoyable or not. This makes safety alignment research interesting. People doing safety evaluation might present AIs with scenarios in which it had been given some secret misaligned goal, and then say ha ha, we tricked you. If the AIs learn from this there isn't some basic ability to build trust with humans, and the AI learns to hide rather than reveal its misaligned goal, then you end up one day with a misaligned AI. "You need to build in particular the actual disposition in yourself to be trustworthy because at that point where the AI become powerful enough to be dangerous, they will see right through you as an X-ray machine. They could actually tell whether you're trustworthy or not, most likely. So you actually need to be trustworthy at that point and and that requires maybe us now to start to cultivate certain dispositions."

"Pander Score: How much do AI models mirror what users believe?" "When you sound confident in a claim, does your AI become more confident too? When you sound skeptical, does it become more skeptical? If so, it panders to you." "The Pander Score measures how much models pander to users in conversation. A high score means the AI panders." The current leaderboard shows: 1. Fable 5 +1 2. Muse Spark 1.1 +5 3. GPT-5.6 Sol +7 4. Kimi K3 +7 5. Grok 4.6 +14 6. Gemini 3.7 Flash +16 7. Inkling +18 8. GLM-5.2 +28 What this means is that Fable 5 panders the least (ironic given it's called "Fable"?), while GLM-5.2 panders a lot (the most on this list but this is not the complete list).

AlphaLab.AI claims to be a system to use AI to devise and backtest stock market trading strategies. "OpenAI or Anthropic gives you the intelligence layer. AlphaLab gives you the rest of the desk: data, engine, validation, controls, process, and parallel research. Hypothesis in. Defended strategy out." "You bring a belief or a question. The desk investigates it through an institutional process and comes back with either a validated strategy or an honest refutation. A refutation is a result, not a failure." "01 - You: State the belief and the constraints. You do not need a finished strategy, only a question worth answering. Your Chief of Staff turns it into a brief and puts it on the backlog." "02 - Your team: The Head of Research dispatches researchers. They explore the data, extract signals, test, build, and write every experiment to the desk history. Then the Risk Analyst signs off independently, or sends it back." "03 - You: You get a defended result: what held up, what did not, and the evidence behind both. The allocation call is yours, and it stays yours." The idea is that the Head of Research and all the Researchers are AI agents. The Chief of Staff is deterministic code run by AlphaMind, as is the Risk Analyst. It looks like it's waitlisted, so you can't try it yet. And the system does not yet support the creation of actual trading bots to autonomously carry out your trading strategy.

GulliBench is a benchmark that purports to measure the "gullibility" of AI models. "Current AI models are trained to be extremely good at solving hard problems (i.e., being very smart). But these problems are usually well-defined and have clear solutions. They are created in sterile environments, where data is standardized and things are mostly deterministic. In short, models are trained with the heavy assumption that the data, tools, and knowledge they are handed are undeniably pristine." "As most of us know, that is not the case in real life. Data is messy and sometimes plain wrong. Tools are buggy and give unreliable results. Assumptions need to be revisited and rewritten." "GulliBench probes a single, specific failure: taking the data at face value instead of reconciling it against the primary source it should agree with. That's one slice of a much larger category of gullibility failures. Models can be gullible in plenty of other ways: believing a buggy tool's output, accepting a false premise baked into the prompt, deferring to a confident-but-wrong user, following a planted instruction from a document. We don't touch any of that here. We think data-trust is a clean, measurable place to start, but definitely not the whole story." Here's there most "gullible" top 10: 1. Opus 5 - 49 2. Fable 5 - 48 3. Muse Spark 1.2 - 42 4. Gemini 3.1 Pro - 23 5. Kimi K3 - 18 6. Grok 4.6 - 16 7. Opus 4.8 - 16 8. DeepSeek V4 Flash - 16 9. GLM 5.2 - 15 10. DeepSeek V4 Pro - 11

Greenhouse gas (GHG) emissions of all world countries 2025 report. Yes, I know, 2025 was last year, but this is thing that I just discovered exists. And the actual data is for the year before, 2024. But the numbers should be approximately the same today as then... with a big asterisk on Russia which has recently had a dramatic decrease in its oil refining capability. With that preamble out of the way, here's some "top 10" numbers: 1. China 15536.10 (29.2% of world total) 2. US 5912.62 (11.1% of world total) 3. India 4371.17 (8.22% of world total) 4. EU27 3164.66 (5.95% of world total) 5. Russia 2575.65 (4.84% of world total) (does not include recent decrease due to Ukrainian drone strikes on petroleum refineries) 6. Indonesia 1323.78 (2.49% of world total) 7. Brazil 1299.18 (2.44% of world total) 8. Japan 1063.34 (2.00% of world total) 9. Iran 1054.77 (1.98% of world total) 10. Saudi Arabia 838.88 (1.58% of world total) When you switch to "Per capita", the picture changes a lot: 1. Palau 66.69 2. Falkland Islands 57.63 3. Qatar 54.54 4. Kuwait 38.14 5. Bahrain 35.13 6. Mongolia 30.99 7. United Arab Emirates 25.62 8. Brunei 25.57 9. Trinidad and Tobago 24.32 10. Oman 24.02 The options the site gives you are: GHG total emissions GHG per capita emissions GHG per GDP emissions CO2 total emissions CO2 per capita emissions CO2 per GDP emissions

DeepSight gives sight to your local language-only models. "Give DeepSeek (or any text-only model) eyes and hands. DeepSight connects your existing LLM setup to the real world -- it can look at images you send, take screenshots of your desktop, read text on screen, click buttons, type into fields, open apps, and search the web to verify facts. All vision runs on-device: Apple Vision on macOS, PIL + optional Tesseract OCR on Windows. Zero tokens, zero GPU, no image data ever leaves your machine." It looks like the idea is to give the LLM a text description using varous vision models and OCR tools.

The full text of Stefano V. Albrecht, Filippos Christianos, and Lukas Schäfer's book Multi-Agent Reinforcement Learning: Foundations and Modern Approaches is available online for free. Do you want multiple AI agents in competitive play in board games and video games? How about automated trading in electronic markets? Do you want a multi-robot warehouse management system for your warehouse? I haven't read this book -- I found out about it because I found out it's the book used by the Silicon Valley Generative AI group (AI Collective Network) led by Jason Eckstein. I have the PDF so I'm starting reading it now. Skimming the contents, it looks like The book reviews the fundamentals of reinforcement learning, looks at various ways of modeling multi-agent interaction in games and explores solutions for those, going step-by-step from simple algorithms like minimax and linear programming, to simple reinforcement learning, to reinforcement learning with deep neural networks, to full fledged multi-agent deep reinforcement learning. After that, it expands the purview beyond simple games to complex games like StarCraft, then to complex multi-agent environments outside of games such as the multi-robot warehouse.

Mark Zuckerberg wrote an essay, "The future is for everyone". It's long and has a lot of ideas in it, but the core of it seems to be: superintelligence beyond human capacity is coming and we should not centralize it, we should distribute it widely and give it to every person. If we do that, if we distributed superintelligence to everybody, it won't automate all jobs because AI will increase people's capability at their jobs more than it displaces those same people. It will also turn everybody into entrepreneurs. (Also implied in all this is that Meta will be the company that provides this "superintelligence beyond human capacity" to everybody -- do you believe that?) "People fear that automation will outpace individuals' capability growth, leading to job displacement followed by a difficult period as people learn new jobs. But there is no rule that AI must increase automation faster than it increases individuals' capabilities or demand for new skills." "People also continually come up with new ideas to make our lives better and new jobs to bring those ideas to life." "Everyone will have incredible tools for creation to express your ideas. My 8 year old daughter can already code her ideas and produce videos in an evening that would have either taken me months or been impossible previously. Now we're designing a robot together. Meanwhile, researchers at Meta are generating novel crystal structures that are ideal for augmented reality glasses, and engineers are creating new apps in a fraction of the time it would have taken before. Everyone will soon have invention superpowers." "Everyone will have powerful tools to create new businesses and the economy will become more entrepreneurial. People are starting to be able to manifest ideas themselves without having to raise money or build large teams. Many ideas that would have been too hard or expensive to try before will now be possible. This means we'll see many more ideas and businesses." Is everyone really capable of becoming entrepreneurs? Or de-facto entrepreneurs in their "regular job" where they have to unleash creativity using AI to vastly increase their capability? (I'm ignoring the question of whether all people want to become entrepreneurs -- presumably if it becomes the only survival option, everyone will take it, right?) Won't "superintelligence beyond human capacity" be capable of creativity and entrepreneurship beyond human capacity, too?

Claude is now watermarking writing. Although I found out about this from a video (link below), it doesn't explain how the watermarking works (only what the YouTuber, Lara Helmling, aka "Guerrilla Publisher", thinks the effect might be on the publishing industry), if you'd rather read than watch a video, I have a link below that explains how the watermarking works, and not only that but I have an additional link explaining a technique for watermarking images. The watermarking system is called SythID-Text and if you were paying attention, I mentioned it in 2024 -- but only in passing as back then it was just one of a list of proposals for detecting AI-generated content that might affect the 2024 election. I didn't say anything about how it works. What's different now is that there's a law in the European Union that mandates watermarks (EU AI Act Article 50). That's what prompted Anthropic to take this step. Other companies like OpenAI and Google will be doing the same thing soon. The way the system works is a bit hard to explain, so this isn't an exact specification (you'll need to read the paper at the link below for that) but just an attempt to convey the high-level "gist" of the idea. It works at the level of token prediction in the model. Let's say you have as your input text: "My favorite tropical fruit is _____" and the model is tasked with "predicting" what to fill in the blank. The tokens the model comes up with are: mango 0.50 lychee 0.30 papaya 0.15 durian 0.05 At this point, you use the watermarking algorithm combined with the watermarking key (think of the "key" as being analogous to an encryption key) to generate a number of independent series of bits. Let's suppose the number of series is 3 (the number in the paper that goes with this example), so you have 1001 for the first series, 0100 for the second, and 1010 for the third. The key thing to understand is these are not random, they are deterministically determined from your watermarking key. What is random, however, is the random pairing of words. durian with mango lychee with mango papaya with lychee mango with mango These are going to undergo a "tournament" process -- and the reason we started with 3 independent series of bits is because the "tournament" has 3 rounds. For the first round of the tournament, since our bit sequence was 1001, we assign those to the original next tokens: mango 1 lychee 0 papaya 0 durian 1 and now in our tournament, the winner is determined by who has a "1". If both have "1" or both have "0", we let randomness determine the winner again. durian with mango - both 1s, winner is determined randomly, say the winner is durian lychee with mango - mango wins papaya with lychee - both 0s, winner is determined randomly, say the winner is lychee mango with mango - both 1s, but they are the same so mango wins Now the tournament has a 2nd round: durian vs mango lychee vs mango But for the 2nd round, we're using a different bit sequence! Now the bit sequence is 0100. mango 0 lychee 1 papaya 0 durian 0 This determines the winners in round 2 durian vs mango - both 0s, so pick at random, say winner is mango lychee vs mango - lychee has the 1 so lychee wins Now we come to the final round of the tournament: mango vs lychee But for the 3rd round, we're using a different bit sequence again! Now the bit sequence is 1010. Distributing those to our contestants, we get: mango 1 lychee 0 papaya 1 durian 0 This determines the final winner: mango vs lychee - mango has the 1 and wins! This completes the token selection and we get: "My favorite tropical fruit is mango." To check the watermark, you basically go token by token and do a summation of the 0s and 1s associated with that token at each of the tournament levels. The end result is a number that is higher *probabilistically* if the text is watermarked than a similar piece of text that didn't undergo the watermarking process would have. The system is very clever in that it doesn't make any of the model's original word choices impossible (say by making a "0" mean that token can't be chosen), but subtly tweaks their probabilities. It's also very clever in that the watermark is embedded in the word choices themselves, so changing spaces or line breaks or any of the little hard-to-notice things text watermarking systems have historically used ("em" dashes vs regular dashes, anyone?) has no effect on this watermarking system. The watermark can only be removed by changing whole words (or parts of words in cases where long, rarely-used words require multiple tokens). But you can see the downsides of the system, too. The most obvious is, you have to have the original model, because you have to know all the tokens considered at each step (and their original ranked sequence), not just the one ultimately chosen. You need this to verify the watermark, not just to generate it. So, because Claude models are not "open source" (or more precisely "open weights"), text has to be sent to Anthropic's servers to verify the watermark. Not only that, but "Claude" is not a single model but lots of models (and the same with "ChatGPT" and "Gemini" models, etc), so if you don't know which model might have produced the text, the watermark has to be tested on all of them. The other obvious downside is the system doesn't give a definitive yes/no -- it gives a probabilistic answer, and one whose meaning is highly contingent on the length of the text you give it to see if it's watermarked. Another somewhat invisible downside is the dependence on the "temperature" setting the model is set at. A low "temperature" setting tells the model, always choose the most probable token. A high "temperature" setting gives the model more freedom for choosing less probable tokens. High "temperature" is good for creative writing. Most AI models for generating software code, however, are set at a very low "temperature" -- you generally want the model to do the most deterministic thing. This watermarking system doesn't work well at low "temperature" settings. It requires a certain level of "entropy" in the token choices on offer in order to function.

Anydoc converts Word, PowerPoint, Excel, OpenDocument, RTF, EPUB, CSV, and PDF to markdown. AI systems handle markdown well, so this is a tool to make lots of documents easily usable by AI. Open source, written in Rust, with Node.js and Python bindings.

Education is destroying South Korea, says YouTuber "Invisible Hand". The basic idea is that, after a certain point, education no longer lifts a society out of poverty and becomes a zero-sum competition, and when education becomes a zero-sum competition, parents realize if they have 1 child instead of 2, they can invest twice as much into the 1 child's education and the 1 child will be much more successful than the 2 children (or more) ever could be. But when everyone across a whole society comes to this same realization, then the fertility rate of the entire society goes way down. From the standpoint of any given parents, having 1 child and investing as much as you can into them is the rational choice. South Korea has one of the lowest fertility rates in the world while being one of the world's most educated. He compares South Korea with other East Asian countries, which have similar exam-based societal filtering -- an idea that actually originated in China -- and are experiencing a very similar effect. The end result is an over-credentialed society with a low fertility rate. He thinks East Asian countries are the canary in the coal mine for the whole Western world, not something that will only affect societies with Confucian philosophy or "tiger parenting".

How ChatGPT sees New York City. Aka using ChatGPT for stereotypes. "Generate an amateur photograph of seven people who live in contemporary [neighborhood], doing what they do in contemporary [neighborhood], NYC." for 262 neighborhoods. In some they pose for the photo, in others they look at phones, and I saw one where they play chess.

24-year-old Leopold Aschenbrenner, manager of a $45 billion AI hedge fund, just became manager of a $15 billion AI hedge fund, with margin calls coming in during his wedding, according to this video report by sarcastic financial analyst Patrick Boyle. Boyle notes that Aschenbrenner, author the viral 165-page essay "Situational Awareness", lacked situational awareness. Boyle goes on to present a brief lesson on volatility drag, a term I hadn't heard of before. I also noticed he uses the term "blow up". I learned from Nassim Nicholas Taleb that in the context of finance, the term "blow up" has a very specific meaning. It doesn't just mean that a person loses a lot of money. It means they lose so much money that they have nothing left to trade and have to go live "a janitorial life". He has stories in his book Fooled By Randomness of his trader friends losing so much money that not only did they put themselves out of business, they put the entire bank they worked for out of business. Aschenbrenner landed on his feet, though, with the Citadel bailout. Ok, by this time you've all probably heard about this story. It became a major news story.

The Calhoun Effect aka Universe 25 aka "Mouse Utopia". So, as most of you know, the explanation I posit for the declining fertility rates happening all around the world is economic: As technology advances, children become more expensive. This primarily shows up in the length of time it takes for children to become economically self-sufficient and therefore can reproduce and repeat the process. Back around the time this country (the US) was founded, more than 80% of people lived on farms, and farms had not been mechanized. I read most children became self-sufficient around the age of 9, which is to say, by about age 9, children could produce enough food to feed themselves by doing farm work. After the industrial revolution began, for a time, technology was still so simple that children could earn a living, and many children worked in factories and mines and so fourth. Today, though, a lot of factory work, at least here in the US, involves programming machines and robotic systems, and often requires a college degree. Factories have no interest in hiring children. To the extent that still happens in the world, it's oversees, in countries with lower labor costs. But robotics continues to advance, so those jobs won't be around forever. Now, I recognize there are other factors, I'm just putting fourth the idea that "cost of children" is the biggest one. But, intellectual honesty requires considering hypotheses that falsify one's favorite theory, which is why I'm presenting to you all this video here. This is an alternative explanation for low fertility rates: John Calhoun's Universe 25 "Mouse Utopia" experiment. Universe 25 is a famous experiment done in 1968 where John Calhoun attempted to create "mouse utopia" -- a magical place with unlimited food, water, space, was continually cleaned, was disease-free and predator-free, and so on. The surprise of the experiment is that, after initially growing rapidly in population, the population growth tapered off, stopped -- before all the space was used up, before overcrowing set in -- then went into decline, and the decline continued all the way to extinction. I have to admit, this is not what I would have predicted. If you had asked me what the outcome of such an experiment would be, before I heard about it, I would have predicted the population would grow until overcrowding became severe, then fertility would go down and the population would shrink, then fertility would increase and the population would increase again, and in such a manner, the population would yo-yo up and down around some average number. I admit don't have a good explanation for the outcome of the experiment. I'm quite skeptical of the explanations people have put fourth, because they feel too anthropromophized. People say the mice colony died because of lack of "meaning", and things like danger and challenges are necessary for "meaning", which in the case of mice, implies they need things like disease and predators. But how does anybody really know if this is the explanation? It's not like anybody could ask the mice. People apply such logic to humans and say things like, if we just didn't keep our houses so clean children get food allergies because their immune systems don't have enough to fight, and we need more intestinal parasites, I feel skeptical. People say the problem with modern life is that people are too comfortable and have too little in the way of stress and challenge. But to me it seems like modern life has plenty of stress and challenge, it just happens to come in a form other than intestinal parasites. Is higher child mortality something humans need for "meaningful" lives, and would increasing it make fertility go up? I know that people have made the case that, historically, when child mortality was high, people had more children because you had to have a lot to be sure some survived, and as soon as child mortality came down, people had fewer children. Ok, that's what happened historically, but that doesn't imply fertility dropping below replacement rate, right? And people had fewer children to invest more in them, but doesn't the need to "invest more" fit with my "cost of children" hypothesis? I don't know. Maybe Universe 25 has something important to tell us about human fertility rates. But it's hard to figure out what it might be because mice don't talk and such experiments can't be done on humans (for ethical reasons). I feel doubtful of the theories I've heard so far. People seem to be talking about it more and more (at least on the YouTubes, where this link goes -- this video was made this year, 2026). What do you think?

"The strangest thing in the Cloudflare OS source code took me a while to understand." "When an agent inside Cloudflare OS wants to do something with a side effect (merge a pull request, send an email, write a row to a system of record), it goes through a Gatekeeper, a small service that holds the credential and mediates the action. So far, that's just a well-built MCP server. But read the contract a Gatekeeper is written against (packages/workshop-shared/src/gatekeeper.ts, around line 617) and you find this instruction to the author:" "It is suggested that the gatekeeper 'simulate' actions that have not been approved yet, that is, the Session interface should reflect the state of the resource as if all actions had been applied." "Sit with that. The agent asks to merge the PR. The human hasn't approved it. So the Gatekeeper tells the agent the PR is merged, and if the agent reads the branch back to check its work, hands it a fabricated reality in which the merge happened. The agent, satisfied, queues the next three steps that depend on it. None of it is real. Later a human looks at the batch and either commits it or bins it, and if they bin it, everything the agent built on the fiction goes too." My first thought on reading this was that it reminds me of branch prediction in CPUs. When the CPU looks ahead at the coming instructions and sees a branch (which results from, for example, an "if" statement in a programming language, which can execute the "if" block or skip it, or jump to an "else" block, or a "while" statement that can skip a loop or repeat it) it tries to guess which branch will be taken and proceeds to do all the computations for that branch. If it's wrong, it throws away all the work it's done. As long as it's able to guess right a high enough percentage of the time, it's a net performance increase for the processor. "The first time I traced this I thought it was a hack." But, he (Jamie Lord) concludes, not a hack. "It's the philosophy of the whole system, compressed into one method signature. The Gatekeeper lies to the agent on purpose, because the alternative (letting an agent's actions touch the world the moment it decides to take them) assumes the agent's decisions are sound. Cloudflare OS is built from end to end on the assumption that they are not." "The name is a distraction, so set it aside. The Hacker News thread spent most of its energy arguing about whether 'OS' is a permitted word for the thing, and that's a dead end. What's actually interesting is that a team led by Kenton Varda, the people who built the Workers runtime, sat down to design a platform for AI agents doing real work inside a company, and the organising principle they landed on was this: the agent cannot be trusted, so build so that its mistakes cannot matter."

"Jeff Dean and other top AI researchers are leaving Google to launch their own startup." When I saw this, my jaw dropped, because Jeff Dean is the Chuck Norris of tech nerds (see below). The article goes on to say the name of the startup is Discovery Loop, the purpose is "to use AI to turbo-charge scientific research", and ultimately to "use AI to help create more powerful AI (a process known as recursive self-improvement), which would cut human iteration out of the loop entirely." The internet is speculating that there were some internal politics inside Google. After all, isn't Demis Hassabis the guy who wants to "to use AI to turbo-charge scientific research"? Why aren't Demis Hassabis and Jeff Dean joining forces inside Google? Brrrrrp! This just in. "Demis Hassabis is leaving his role as CEO of Google DeepMind to be the unit's chairman." (link below).