Boulder Future Salon

Boulder Future Salon

Thumbnail
"Every student is cheating with AI" starts a blog post by Ben Borgers. This came out in August but I found more stuff related to it (which I will include here), and have been very busy with work and that delayed me writing any of my own commentary.

"Last month, a Brown professor wrote about discovering that a majority of his students had used AI to cheat on his take-home exam."

That "wrote" was a link to an article I've been hearing people talk about, but haven't seen the original article before.

"For the first time since he started teaching Welfare Economics and Social Choice Theory nearly two decades ago, Brown University economics professor Roberto Serrano gave his students a take-home midterm this spring."

"But by the end of the semester, Serrano regretted the decision. Dozens of students in the class likely used artificial intelligence to cheat and earn perfect or near-perfect scores on their midterm, he said. Serrano in turn made the final exam in-person, which led more than a dozen students to drop the course and even more to fail it."

"Historically the average grade in the midterm of this course has ranged between 65 and 80 [percent], and this exam was harder than the exams I wrote in the past, because ... take-home is an opportunity to challenge the class a little bit more, given that you're giving the students unlimited time,' Serrano said."

"He knew something 'fishy' was going on, and so he and his graders ran the test through ChatGPT. The AI gave answers that mirrored what his students had written, and which were 'kind of correct, but very off and with a very convoluted style,' Serrano said. For example, one question asked students to prove a mathematical statement that could most obviously be done using a 'direct argument,' Serrano explained. ChatGPT -- and many of his students -- used a 'contradiction argument,' which gave the right answer but was 'very contrived' and which Serrano could tell wasn't written by a human."

"I am not declaring [the midterm] void for now. I am going to give the class a chance to prove me wrong. That is, if the distribution of the final exam is roughly similar to the distribution of the midterm, I will count the midterm."

"18 of his students subsequently dropped the class. Nine students remained enrolled but did not take the final exam. And Serrano said the results proved him right; three students earned a zero, and the average score on the final was 48.6 percent -- by far a historic low, he said. Previously, the average final exam score had never dropped below 65 percent."

The key diagram shows midterm scores in orange and final scores in grey, with students numbered 1 through 59, ordered by, it looks like, the minimum of the two scores.

What the diagram reveals is that there were 2 honest students -- student 1 and student 22. Student 1 got a 95.5 on the midterm and a 95 on the final. So student 1 definitely really learned the course material. Student 22 got a 55 on the midterm and a 59 on the final. Student 22 was the only student whose score increased between the midterm and the final.

Could there have been other honest students who just happened to get a lower score on the final than the midterm? It's possible. For example student 2 got a 98 on the midterm and an 88 on the final. But as you go down the chart and see wider and wider gaps, it gets harder and harder to believe. Student 42 got a 100 on the midterm and a 35 on the final. Student 47 got a 100 on the midterm and a 33.5 on the final. Student 54 got a 100 on the midterm and a 16.5 on the final.

"In a follow-up message to students, Serrano announced the midterm was void and the final exam would be worth 80 percent of the students' final grade. Any student that scored a 40 percent or higher on the final earned a passing grade -- previously, he set the passing line at 50 percent or higher. In total, 19 students failed the class."

Flipping back to Ben Borgers, he says:

"I attended a well-ranked university until last year. The truth is: essentially every student is cheating with AI. Every single one. Nobody doesn't cheat."

"Basically every student cheats in some form on take-home exams."

"The Brown professor's article doesn't describe any kind of isolated behavior. Every student at a university today knows that this is the truth."

"Everyone's cheating with AI. There is no stigma."

"(If you could learn everything yourself, and find other people to do it with, you probably wouldn't go to university.)" -- Interesting parenthetical comment. I tell everyone, if you want to learn something you should never go to school. School is for obtaining credentials. Learning and obtaining credentials are different things.

"Professors and universities need to face that every student is cheating with AI. Even the ones you think are wonderful and would never."

"Build the environment around the realistic understanding that -- under the pressure of grades, deadlines in a slew of classes, and with unmonitored take-home exams and assignments -- everyone is bending the rules."

We know it's not every student, though. Student 1 and student 22 didn't "cheat". That's about 3%. Which I guess is why it's important he didn't always say "every", sometimes he had qualifiers like "essentially" and "basically".

A bit of commentary (will have more below): I'm putting the word "cheat" in quotes because, if the purpose of school is to prepare people for work, and work requires people to use AI to get tasks done as fast as possible -- which is in fact the case for me where I work and from what I can tell, is true of most "white collar" work in most workplaces (with exceptions made for various things, work that involves security clearances, for example) -- then it makes sense that students in school should use AI to get tasks done as fast as possible whenever they possibly can and this shouldn't be considered "cheating".

Thumbnail
Riemann hypothesis proven, or at least that is the claim. A mathematician named Levent Alpöge has released a proof in the form of code for Lean, the proof assistant software, in 14 parts (all AI-generated, by Claude Code). Does that mean this proof is already "Lean verified"? It certainly looks so.

"Claude and I have had a great time together with such breadth in recent months, but this time Claude far outpaced me near the finish line. No, Claude flew, and danced, and sparkled near that finish line. I had us on the line of combining two quite different ideas present in a toy setting, one of which was already new and shocking, making very simple one of the shadows of Riemann over which we mathematicians felt most triumph. I guided us to the right situation analogous to that toy setting, and then Claude began to leave me in awe."

Thumbnail
Theres another major US-Israel attack on Iran in the works, according to WarFronts (one of Simon Whistler's YouTube channels). I'm sharing this now so you all can see if this prediction comes true. I try to get predictions out before events because after the fact people can say they predicted anything.

Thumbnail
"When processing and producing non-English text, generative AI burns more energy and sometimes provides less accurate answers."

"The energy disparity is greatest in languages that don't use Latin script and have less digital content available for training. AI models could use 1.4 to 180 times more energy to generate non-English text, with most Western European languages coming in at twice the energy expenditure or less. However, Hindi and Bengali, each spoken by hundreds of millions of people, require 7.9 and 15.3 times more energy than an equivalent English prompt."

Hmm, very interesting, I never would have expected this.

Interesting that they don't mention Chinese, and Chinese in Chinese-made models... oh, but they have a graph. Native speakers is on the x-axis, energy consumed is on the y-axis. They put simplified Chinese at 2.1x English and traditional Chinese at 2.4x English.

The models used were Alibaba's Qwen 3, Google's Gemma 3, and Meta's Llama 3.1. One of those (the Qwen model) is Chinese, but they don't say any more about it.

Thumbnail
"Take the application on your computer that you're most familiar with. Now imagine its creators opening up the product roadmap to 100 randomly selected users. They would build every feature request, idea, and redesign these people would come up with for half a year. Do you think you would still use that tool in two years time?"

"I'm pretty sure I would not. I think it would turn very quickly into an obtuse mess of a product that would do everything badly."

Well, now that AI is here and we can make 20 features in a week, we're going to find out, aren't we?

Obvious solution not mentioned in this piece: instead of adding feature ideas from 100 user to one product, just make 100 products. Is this where we're headed? Instead of one product for everybody, everybody -- maybe not every individual person but every organization or team -- vibe-codes their own custom solution that perfectly fits their workflow.

Thumbnail
Having a "human in the loop" is useless because humans rubber stamp everything, says Mir Arshad Ali. As the AI agent gets better, the humans trust it more, which actually increases rather than decreases the tendency to rubber stamp everything. Supposedly the human is still needed to catch the rare failures, especially if the consequences are severe, but as the AI gets more accurate, the human-in-the-loop is actually less likely to do that. The better the AI, the more difficult vigilance becomes for the human-in-the-loop.

If the volume of decisions to review gets very large (the human-in-the-loop must make hundreds of decisions per day), context is missing (reasoning behind a decision is missing or relevant information is not presented at the right time to the human-in-the-loop), or incentives are wrong (the human-in-the-loop is judged by superiors based on throughput), the problem is compounded.

Thumbnail
Dartmouth University has stopped admissions for the next cohort in its math PhD program. The people already in the math PhD program will be allowed to stay, but new people can't start.

I expected employment for mathematicians to go down now that AI can prove theorems that have defied humans' ability to prove for decades. I didn't expect it to happen this fast.

Having said that, we're not talking about a lot of employment. Dartmouth's math PhD program usually accepts 7-9 new people per year. And they work like slaves for a tiny stipend.

But still.

If you're wondering why the article says "Guarini" instead of "Dartmouth", "Guarini" is the name of the graduate school within Dartmouth.

And yes, I know the article doesn't blame AI for stopping new admissions to the math PhD program. If the article had given some clear alternate explanation, I'd be inclined to go with that, but it didn't, leaving AI proving theorems as "the elephant in the room", as the expression goes.

Thumbnail
"The Filthy Human Hands (FHH) License v1.0"

The terms are short enough I can just quote them all.

"No part of a work under this license may be used or reproduced in any manner for the purposes of training autonomous agents, generating anything created by artificial intelligence (AI), any form of data mining/scraping, or related systems and technologies."

"No part of a work under this license may be used or reproduced in any manner by organizations or businesses with more than fifty (50) employees. An exception is granted to allow ISPs and other hosting services to reproduce the work for the purposes of display and distribution."

"No part of a work under this license may be used or reproduced in any manner which promotes bigotry of any kind. This includes but is not limited to fascism, racism, sexism, misogyny, ableism, transphobia (including erasure and/or denial of nonbinary identities), homophobia, ageism, sane-ism (including denial of multiplicity and/or plurality), and discrimination against non-human identities."

"Any use of any form of a work under this license must include an attribution crediting the original creators for their contribution."

Thumbnail
"The European Space Agency's Euclid mission has released its first major batch of data. With this release, which captures the light of more than a billion objects and covers an area of sky as big as 9500 full Moons, ESA's dark Universe detective lays the foundation for its largest-ever 3D map of the cosmos."

They say, unprecedented numbers of cosmic objects:

"Over 900 million galaxies."
"Around 15,000 strong gravitational lensing candidates."
"Over 60 million quasar candidates, the brilliant cores of distant galaxies."
"The spectra from more than 160 million sources."
"Euclid Wide Survey results from 1900 deg^2 of sky -- a total of 1.7 million GB of data."

By "deg^2" they mean "square degrees", a 2-dimensional patch of a sphere. To give you something to compare the new 1900 figure to, they say:

"This is a change in scale from Euclid's previous releases, which included a few days of observations over around 60 deg^2 of sky."

"The strong gravitational lenses sought by Euclid are extremely rare, and form when the light travelling to us from distant galaxies is significantly distorted by a foreground galaxy. This release contains the largest strong lens catalogue ever -- bigger than all previous catalogues combined."

They go on to describe how Euclid uses specroscopy to measure hydrogen-alpha lines to estimate distance. Euclid uses redshift to estimate the distance to a galaxy protocluster, and its age.

"This data release also includes key information on the faint, diffuse starlight that encircles the brightest galaxy within a cluster, known as intracluster light (a result of gas stripped away from surrounding galaxies over time as they interact and merge). This light acts as a fossil record for how the cluster came to be, and can trace hidden tendrils of dark matter. However, it's very faint, making it difficult to detect -- but Euclid's sensitive eyes have spotted intracluster light in hundreds of clusters out to immense distances."

Thumbnail
HackerTube is YouTube videos ranked based on Hacker News posts and comments. Made by Andrej Karpathy, believe it or not.

"YouTube's recommendations are tuned for watch time. They're very good at finding the next thing you'll click, and not so good at finding the talk you'll still be thinking about next year."

"Meanwhile, for nearly two decades, Hacker News commenters have been pointing each other at exactly those videos: the conference talk that changed how someone writes software, the lecture series that finally made a subject click, the documentary nobody else seems to know about. Those recommendations are scattered across millions of comments. HackerTube collects them in one place and makes them browsable, searchable and personal."

These videos aren't the latest. Instead, they're classic videos going back decades. One of Andrej Karpathy's own videos, "Let's build the GPT tokenizer", shows up.

Here's some of the titles I see scrolling down the home page:

"Best fan placement to move air through the house",
"USENIX ATC '21/OSDI '21 Joint Keynote Address-It's Time for Operating Systems to Rediscover Hardware",
"Simpsons - Monorail Song",
"Let's build the GPT Tokenizer",
"Why The World Relies On ASML For Machines That Print Chips",
"Dr. Robert Zubrin with a brilliant answer to "Why Should We Go To Mars?"",
"James Kunstler: How bad architecture wrecked cities",
"DEF CON 19 - Moxie Marlinspike - SSL And The Future Of Authenticity",
"A Mind Is Born (256 bytes)",
"DIY haptic input knob: BLDC motor + round LCD",
"Casey Muratori | Smart-Pointers, RAII, ZII? Becoming an N+2 programmer",
"Elon Musk's Unbelievably Simple 12-minute Killer Break Down on Climate Change",
"Brainwash: The Gender Equality Paradox",
"Elm Europe 2017 - Richard Feldman - Scaling Elm Apps",
"Kirk McKusick - A Narrative History of BSD",
"Saving Voyager 1! - Bruce Waggoner at !!Con 2024",
"Maciej Ceglowski - Barely succeed! It's easier!",
"Nordic.js 2014 - Douglas Crockford - The Better Parts",
"The Obesity Code Lecture (Why do we get Fat?) Part 1",
"IBM Selectric Typewriter & its digital to analogue converter",
"Will AI outsmart human intelligence? - with 'Godfather of AI' Geoffrey Hinton",
"The Electrowetting Display",
"CppCon 2014: Chandler Carruth "Efficiency with Algorithms, Performance with Data Structures"",
""The Hard Parts of Open Source" by Evan Czaplicki",
"The Future Of Reasoning",
"Erik Jansson - GPU driven Rendering with Mesh Shaders in Alan Wake 2",
"The Secrets Of Cornwall - Part 1 - Communications",
"Alcoholic Vervet Monkeys! - Weird Nature - BBC animals",
"DEF CON 25 - John Sotos - Genetic Diseases to Guide Digital Hacks of the Human Genome",
"3D Gaussian Splatting! - Computerphile",
"The Beauty of Bézier Curves"
"AI can't cross this line and we don't know why.",
"AI Learns Noise Filtering For Photorealistic Videos | Two Minute Papers #215",
"GPUs to Mars: Full-Scale Simulation of SpaceX's Mars Rocket Engine",
"Bret Victor The Future of Programming",
"Practical Deep Learning for Coders: Lesson 1",
"Uptime 15,364 days - The Computers of Voyager by Aaron Cummings"

Thumbnail
"Since late September, every image a ProductAI subscriber generates in the studio gets a second look. A vision model is shown the photo the seller uploaded next to the finished render and asked one question: is this still the same product? The scene, the light and the angle are supposed to change. The product is not."

Evidently, ProductAI is a company for helping people market products, primarily with images and video. They make some interesting observations about how their own product (photos ProductAI itself generates) can go wrong.

"In the first 18 days that check looked at 495 renders and flagged 111 of them, about one in five. We read every one of those 111 notes by hand. This is what they say about where AI product photos go wrong, and what you can do before you press Generate."

"More than a third of all flags were about writing on the product. The failures came in three kinds."

"Numbers that quietly change. A perfume bottle labelled 50 ML came back as 30 ML. A tube marked 118 mL came back as 110 mL, and on another run as 116 mL. A spray whose label states an active ingredient at 9.6% was rendered at 5.6%."

"Words that almost survive. 'MADE WITHOUT PARABENS' became 'MADE WITNONT PARABENS'. 'Silicone-based' lost a letter. A two-word engraving on a pendant changed one letter, twice, in two different ways."

"Labels that go blank. A pharmacy bottle with printed dosage instructions came back with a clean white label on three different models."

"The first kind is the dangerous one. A garbled word looks wrong at a glance. '110 mL' looks perfectly fine, and it is a false statement about your product on your own listing."

My commentary: Hmm I thought these kinds of problems were fixed a long time ago. Maybe not, or maybe ProductAI is lagging far behind the state of the art? You all tell me.

Thumbnail
"The Product Hunt graveyard: 1 in 4 top launches from 2023-2025 is already dead." This was from July but I didn't see it until today.

"We visited the live website of 2,291 products that made Product Hunt's daily leaderboards between January 2023 and April 2025."

"24.4% are hard-dead -- the domain no longer resolves, the page is parked for sale, the security certificate expired, or the homepage returns 404."

"Another 4.5% now redirect to a different company (acquired or quietly folded). Total gone as independent products: 28.9%."

"AI tools die at the same rate as everything else -- 24.5% vs 24.4%. The AI boom changed what gets built, not what survives."

"Launch-day success is real protection: finish in your day's top 5 and the death rate drops to 9%, vs 25.9% for everyone else. But it's no guarantee -- several 600-upvote launches are gone without a trace."

"Death comes fast: the 2025 cohort is already 20% dead after roughly a year."

"This is the floor, not the ceiling: 9.8% of products gave ambiguous answers (anti-bot walls, timeouts, vanished app-store listings) and were excluded rather than counted -- plus a site that still loads isn't necessarily a business that still operates."

Somebody tell me again about how, after AI automates all jobs, we'll have a planet of 8 billion entrepreneurs who use AI to start new businesses?

Thumbnail
"Why is AI-generated language so hard to understand?"

You might be wondering why AIs trained to imitate human text would deviate from human text in ways hard for humans to understand. Spoiler: In the last year or so, the AI companies have been using less reinforcement learning from human feedback (RLHF) and more reinforcement learning from verifiable rewards (RLVR), which compares the AI output with known facts and deterministically computed mathematical and coding results, in an effort to make the AI systems "hallucinate" less and give factually correct and logically reasoned answers more. This has had the side effect of more precise but less typically "human" language.

If you're wondering how these researchers determine the "humanness" of text, they do it using the following 6 "linguistic features of interest": n-gram statistics, dependency length, object relative clauses, stacked noun compounds, denominal verbs, and inanimate subjects.

If you're wondering what an "n-gram" is, first you have words, then word pairs, then what do you call it when you have a group of 3 words? A 3-gram. And 4 words is a 4-gram, and so on. Hence n-grams. You'd expect AIs trained on human text to use any given n-gram with the same probability humans do. But now, thanks to RLVR, improbable n-grams are used more by AI, increasing cognitive load on human readers.

Dependency length refers to the distance between an adjective and the noun it modifies, an adverb and the verb it modifies, a verb and the subject it refers to, and so on. AI systems are now generating text with longer distances than text written by humans.

Object relative clauses are those clauses that start with "that" or "which", but the "that" or "which" can often be left out, but it's still an "object relative clause" even if the "that" or "which" has been left out.

Stacked noun compounds refers to when a noun acts as an adjective modifying another noun that comes after it. You can stack lots of nouns in front of another noun.

Denominal verbs are nouns used as verbs.

Inanimate subjects are when an inanimate object is used with a verb that typically only goes with an animate subject.

Examples:

An improbable 3-gram: "verified red against". "Every behavioral test was verified red against the original source ..."

Dependency length: "A cat who can always get up and away is a much calmer cat." The word "is" is 8 words away from the word "cat" (the first word "cat" which is the subject of the verb "is").

Object relative clause: "Or a first name he'd never hear anyone use." The word "that" is omitted ("Or a first name that he'd never hear anyone use.")

Stacked noun compound: "You've basically installed a horror movie soundtrack generator in your skull ..." The words "horror", "movie", "soundtrack", and "generator" are all nouns.

Denominal verbs: "Footnote it". The noun "footnote" is turned into a verb. "Calendar it". "Calendar" is normally a noun.

Inanimate subjects: "is_file() swallows that and returns false." (Software functions can swallow?) "Puzzle patterns where this asymmetry bites ..." (Puzzle patterns can bite?)

Thumbnail
"The great software IP writedown." Martin Alderson ponders whether it's not just the cost of new software that's gone down 90%, it's the value of everything already built.

"The final thing that is really nagging at me is just how easy it is to copy software now. You can take any agent, give it some screenshots (or if you are even more lazy, a browser tab for it to control) and tell it to clone software you think you like."

"We've already seen this on an industrial scale with the ArtCraft suite of clones of everything from Excel to Photoshop."

"While I'm sure they are not quite as good or stable as the 'genuine' product, this has been the biggest shock to me. Arguably already their Photoshop clone has more features than the unfortunately named GIMP open source clone of Photoshop, which has had 30 years of human effort aimed at it."

"For the past decade or so, the standard defence of loss-making software companies has been that R&D isn't really an expense. It's an investment. You spend heavily on engineering today, and you get a long-lived asset -- the product -- that keeps paying out for years.

"That logic only holds if the R&D actually builds something durable. If a competitor -- or your own customer -- can rebuild what you spent five years on in a few weeks with agents, then that spend isn't building an asset at all. It's much closer to a running cost you have to keep paying just to stand still. Which means the underlying economics of a lot of software companies are quite a bit worse than the adjusted numbers suggest."

"A year ago I was asking whether the cost of building software had dropped 90%. I think the more uncomfortable question now is whether the value of what's already been built has dropped by a similar amount."

My commentary: I don't think it's as easy as he thinks, but it probably will be before too long as AI gets smarter, so it seems to me that his conclusion that the value of existing software is going down still stands.

Thumbnail
"Since late September, every image a ProductAI subscriber generates in the studio gets a second look. A vision model is shown the photo the seller uploaded next to the finished render and asked one question: is this still the same product? The scene, the light and the angle are supposed to change. The product is not."

Evidently, ProductAI is a company for helping people market products, primarily with images and video. They make some interesting observations about how their own product (photos ProductAI itself generates) can go wrong.

"In the first 18 days that check looked at 495 renders and flagged 111 of them, about one in five. We read every one of those 111 notes by hand. This is what they say about where AI product photos go wrong, and what you can do before you press Generate."

"More than a third of all flags were about writing on the product. The failures came in three kinds."

"Numbers that quietly change. A perfume bottle labelled 50 ML came back as 30 ML. A tube marked 118 mL came back as 110 mL, and on another run as 116 mL. A spray whose label states an active ingredient at 9.6% was rendered at 5.6%."

"Words that almost survive. 'MADE WITHOUT PARABENS' became 'MADE WITNONT PARABENS'. 'Silicone-based' lost a letter. A two-word engraving on a pendant changed one letter, twice, in two different ways."

"Labels that go blank. A pharmacy bottle with printed dosage instructions came back with a clean white label on three different models."

"The first kind is the dangerous one. A garbled word looks wrong at a glance. '110 mL' looks perfectly fine, and it is a false statement about your product on your own listing."

Thumbnail
Anduril has a business model that is completely different from large incumbent military contractor companies.

Normally, "The military would make some specs and design requirements describing what they want, and assign a certain amount of money they are willing to pay for it. A few large incumbent companies would compete for this contract by presenting designs, timeliness, ideas."

But with Anduril: "Instead of trying to win a contract, they build things before the government even awards them the contract, taking all the risk on themselves. Once they have made the actual tech, then the military can choose to buy this."

"Specifically what anduril builds is a variety of drones, fighter jets, submarines, etc. All of these plug into a software called LatticeOS which is an ai native platform to analyze all this data and coordinate all the hardware."

"Their thesis is that as their product catalog increases, latticeOS becomes more and more powerful."

"So far Anduril's Thesis has been performing extremely well, the company has been awarded many contracts, and they've been shooting up the setter 30."

The Setter 30 is a list of "highly sought-after pre-IPO companies in the global secondary market" made by venture capital firm Setter Capital.

"In addition, LatticeOS has reportedly been performing extremely well, and on the hardware side Anduril has a massive factory called Arsenal-1 in Ohio."