Glasswing, Glass Houses, Glass Onions, and Attention

Originally published on LinkedIn: https://www.linkedin.com/pulse/glasswing-glass-houses-onions-attention-jonathan-sander-kwfbe/

If you’ve been in security long enough, you’ve definitely been involved in lots of code reviews. I have seen my fair share. Like many of you I’m sure, I’ve signed so many NDAs it’s surprising I can talk at all. So I won’t get too specific. But I’ve led team efforts and been the guy grinding line by line, function by function, object by object to focus attention on the odd code quirks at the heart of vulnerabilities. Sometimes it’s a bug. Just as often it’s abusing a feature. It always feels surprising that things you find weren’t noticed before. Do this enough, and you realize the only thing standing between your code and the bad guys trying to exploit it is attention - someone’s, anyone’s, focused on the right detail at the right time.

The mythos of Mythos

People are talking about Mythos like it’s a conspiracy theory. There’s the full range of emotional reactions from denial to panic. We’re going to set all that aside and focus on what we know for sure:

1. Anthropic used a new model they’re calling Mythos to do the super detailed work of code reviews for a staggering number of open source projects and found some IRL exploitable vulnerabilities.
2. The projects affected included every major browser (directly or indirectly), the Linux kernel, and the headline grabbing 27 year old vulnerability in the famously security first operating system OpenBSD.
3. Anthropic has now formed a partnership named “Glasswing” with a number of heavy hitters in the tech and business world with the purpose of doing scans on their code to make sure they aren’t as exposed as these OSS projects.

If that was everything, I wouldn’t be writing about it. (OK, maybe I would because that is already pretty nuts!) It’s all the reactionary talk about all this that I find most interesting.

Unavoidable attention

Glasswing is named after a butterfly wing which is transparent. It hides in plain sight; it hopes it won’t catch your attention. The project named Glasswing has gotten a lot of attention, though. So much that skeptics say this is the whole point of it. So much that some say it means this is all just marketing and should be ignored. These people point to 2019 when OpenAI claimed GPT-2 was too dangerous to release after it started writing fake wikipedia articles. (Given the times we live in now, doesn’t that seem quaint?) You only have to recall that the people steering Anthropic (and Glasswing specifically) are the same people who were then at OpenAI making similar claims about GPT-2 being dangerous. That’s more than enough for many people to dismiss this Mythos as noise. Just marketing moves they say. But that means ignoring a lot of the experts saying these results are real (even if they’re also hyped). See Bruce Schneier’s piece on it from just a couple days before I’m writing this.

Attention is the only weapon you need

Today’s LLMs are built on a fundamental technology called the transformer. The transformer was famously outlined in a paper called “Attention is All You Need.” Attention in the context of LLMs is the mechanism deciding what details and relationships get both micro- and macro-focus as data is processed to generate the output. The core capability of today’s AI platforms is focusing and harnessing attention at scale. Tell the LLM to spot a list of a hundred specific needles in a farm’s worth of haystacks and it will do its best to obey. It will definitely do about as well as some double-digit number of reasonably competent humans, but the humans will take exponentially more time.

So what happens when you take the very latest model’s capabilities to focus attention and turn it to the problem of code review? That’s where we are right now. There’s no magic here. Everyone who’s ever done code review already knew that any codebase under enough scrutiny by enough competent eyes would yield vulnerabilities and exploits. When people point to the OpenBSD example sitting there in plain view for 27 years, the reason is that OpenBSD is an example of some of the industry’s most serious security minds paying very close attention but still missing something over and over. The real takeaway for me is this: LLMs harness brute force attention tirelessly, without bias, and indifferent to how long it takes to get the job done. It’s the code review reckoning every security pro knew may come at some point.

Party Like It’s 1999

There’s a window. It’s right now. LLMs will continue to review code, pump out vulnerabilities, and we will have to keep up. The tech industry has seen time pressure before. What’s funny to me is that many also said Y2K was nothing but hype. But there were really systems with two digits to track the year which were absolutely going to burn to the ground when the clocks passed midnight on the last day of 1999 (setting the year to 00 which the code could not handle). I know they were real. I touched them in the frenzy to make sure we fixed everything before the clock struck 12 (literally). We did it back then and we did not have AI to help us write the fixes for the decades of problems we made for ourselves. I would have welcomed LLMs to help during Y2K, and I’m glad they’re here to help now.

Some have said this is the death knell for open source software because there’s no way it can keep up with its code hanging out in the wind. I strongly disagree. OpenBSD didn’t get so secure by closing its code away. “Many hands make light work” has always been one of the founding ideas of the OSS community. The adaptation will be to use artificial hands alongside the human ones. Open code will actually have the advantage I think. With so many depending on it and all of them armed with both the cleverest humans and hardest working LLMs, the code everyone can pay attention to and fix will likely be the best maintained.

Rock gardens in glass houses

As LLMs continue to weaponize attention to deliver results we would have needed armies to achieve before, we will need to re-train ourselves to pay attention to different details. We’ve always had an attention issue in security. What are bug bounties if not bribing competent people to pay attention to places we think need attention? Leveraging these new tools in automated processes is where we need to focus now.

Everyone throwing rocks at Anthropic over all this better check their walls for glass. The vulnerabilities are coming whether we like it or not. While we’re in this window of time where the stuff in production is the same stuff LLMs are writing exploits for, there will be big bloody breaches. Even when the patching is done (surely with the help of those same LLMs), there will be laggards who will pay the price. Right now, we should all invest energy in coming up with the next generation of our security programs. Am I saying it won’t be hard? Nope. Do I think those breaches will cause material harm? I sure do. But security has never been a place for the faint of heart. You only have so much attention to give. Pay attention to the real challenges.

Peeling Back Layers of the Glass Onion

The boys from Liverpool were trolling the world. Were the clues people kept finding in the lyrics real? Was it all just hype to sell records? Were the strange lyrics truth or fiction? Lennon’s answer was: why choose? Are Glasswing and Mythos hype? Sure. Did Mythos find real issues? Absolutely. It’s both real and hype. The layers of this onion are transparent. You can see them all at the same time if you pay attention to the right details.

Peel back the cynicism: real capability underneath. Peel back the hype: real fear underneath. Peel back the fear: an industry that’s been hoping nobody ever had the patience to look closely at every line of its foundations. The onion has always had a lot of layers. The difference now is we finally have something that doesn’t get tired of peeling.