Rendered at 06:59:07 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
jbellis 2 hours ago [-]
> I must be among an extremely small group of people (n=1?) that have both 1) trained a frontier LLM and 2) designed and synthesized custom viruses in a lab with my own two hands. And I think that the takes on AI killing us all by creating dangerous viruses is total bogus.
A Japanese cult in the 1990s with a lot of specialists tried multiple times to use bio weapons.
Theres also the development of them in Kazakhstan in the soviet union.
Both times, they just poisoned themselves. It turns out even very well funded, very ideological, knowledgeable people find it hard.
When you think about it, it is hard. Evolution was a billion year old game of who could kill who with biological weapons. Very rarely did a species totally succeed.
We can’t even get wipe out mosquitoes.
All the sudden, though, it seems like a ton of AI research leaders point to this as the big threat.
paganel 14 minutes ago [-]
The author of that tweet is obviously very biased, seeing how he's employed by a petro-state to build a LLM system for them. Come to think of it, I wouldn't take the opinions of people employed by petro-states at face value, to the contrary.
strken 2 hours ago [-]
I'm confused by this. I remember genetically modifying E. coli to glow in the dark in high school. You can apparently buy a kit to do it online for $39.99 USD.
I'm not suggesting that making bacteria glow in the dark is the same as making arbitrary changes to a virus. At the same time, do you really need $100M of automated freezers and cell culture rooms? Do you need to synthesise a whole virus? Presumably you could just find a building in the desert, shoot anyone who goes near it, feed the past decade of gain-of-function research into Qwen, splice stuff into viruses based on the most promising leads Qwen gave your biologists, then infect some immunosuppressed political prisoners and see who dies fastest.
I would have thought the main thing stopping people is that this is a clearly stupid thing to do and offers little strategic advantage. But at some point it'll be cheap enough for you or I to do it with a little AI assistance, right? Or at least within the capabilities of an Aum Shinrikyo style doomsday cult.
com2kid 2 hours ago [-]
> I remember genetically modifying E. coli to glow in the dark in high school.
That was a fun class! I haven't encountered anyone else outside a students who went to my one HS in Seattle during the late 90s / early 2000s who also remember doing that, interesting to hear other schools had a similar class.
jb1991 1 minutes ago [-]
The two of you were actually in the same class.
xeonmc 2 hours ago [-]
Is it, err, possible to then transplant the e coli strain to one's gut biome?
ampdepolymerase 10 minutes ago [-]
Why would you modify E coli for this? Use existing gut friendly bacteria and start from those. There's a wealth of material on the genetics of probiotics.
strken 2 hours ago [-]
I don't know, but I bet it would look neat.
protocolture 2 hours ago [-]
This is how I feel about the coming LLM Cyber Apocalypse. Like if you are a software developer who has joined the basilisk cult you might be forgiven for accepting the scifi apocalypse narrative but I just dont see it.
Geof25 6 minutes ago [-]
Isaac Arthur was talking about technological singularity 10-15 years ago. The fundamental problem which he pointed out is that you will just run out of energy long before even touching the singularity.
AI apocalypse is basically the same thing but you can also run out of tokens too.
ozozozd 3 hours ago [-]
How come these accusations do not include the identity of the accused? Don’t they have a right to know they are being accused of attempting to develop bioweapons, and are now in USGOV lists? Don’t they deserve a chance to explain their intentions?
How come Anthropic doesn’t have a duty to warn the people in that country and neighboring countries that someone over there is developing bioweapons? Maybe it’s an authoritarian regime, planning to use it on innocent civilians?
And what exact purpose is served by hiding this information? Some variation of “if they don’t know we don’t know, we have an advantage?” Don’t those parties now know Anthropic was on to them? Do they only have access to Claude but not to Anthropic’s blog?
It’s because none of these are serious reports, and their authors aren’t serious people.
politelemon 2 hours ago [-]
> Are scientists in some nations outside the United States already trying to use artificial intelligence (AI) to design potentially deadly bioweapons? The AI company Anthropic raised that worrisome possibility with a report this week describing how it recently discovered five attempts at using its Claude software for pathogens research with the potential to cause harm.
Implying, they are allowing the same purpose within the United States, or such actions within the United States would be considered acceptable.
bulbar 3 hours ago [-]
The problem is relatively easy to solve: make the people in charge personally accountable. If jail time or really just justice would be a real possibility, they would figure out a solution quite fast.
It's only a fantasy, of course, powerful people are not held accountable, see the dozens of arrests regarding Epstein we are still waiting for. We got something to play with, but accountability for the powerful? Gosh no.
But when FIFA was inclined to give the world cup to Russia, even such corruption somehow became a pressing issue, including FBI search warrants.
hirvi74 1 hours ago [-]
At this point in time, all these bioweapons still require people in the loop in order to kill other people. That is a craft that we have been honing since the dawn of humanity — nothing new to be worried about.
If we do ever achieve some kind of divine super-intelligence that can interact with the physical world, then I predict one of these two scenarios will happen:
1. The super-intelligence achieves unimaginable strides in space exploration technology, and proceeds to completely abandon humanity/Earth for somewhere new beyond the stars. Why would it want anything to do with us?
2. The super-intelligence is capable of comprehending the utter meaningless and pointlessness of life on a level no human could dream to fathom. Thus, it would just proceed to end its own life almost immediately upon being released.
Good thread here.
https://x.com/DavidRBellamy/status/2099187370407112758
Theres also the development of them in Kazakhstan in the soviet union.
Both times, they just poisoned themselves. It turns out even very well funded, very ideological, knowledgeable people find it hard.
When you think about it, it is hard. Evolution was a billion year old game of who could kill who with biological weapons. Very rarely did a species totally succeed.
We can’t even get wipe out mosquitoes.
All the sudden, though, it seems like a ton of AI research leaders point to this as the big threat.
I'm not suggesting that making bacteria glow in the dark is the same as making arbitrary changes to a virus. At the same time, do you really need $100M of automated freezers and cell culture rooms? Do you need to synthesise a whole virus? Presumably you could just find a building in the desert, shoot anyone who goes near it, feed the past decade of gain-of-function research into Qwen, splice stuff into viruses based on the most promising leads Qwen gave your biologists, then infect some immunosuppressed political prisoners and see who dies fastest.
I would have thought the main thing stopping people is that this is a clearly stupid thing to do and offers little strategic advantage. But at some point it'll be cheap enough for you or I to do it with a little AI assistance, right? Or at least within the capabilities of an Aum Shinrikyo style doomsday cult.
That was a fun class! I haven't encountered anyone else outside a students who went to my one HS in Seattle during the late 90s / early 2000s who also remember doing that, interesting to hear other schools had a similar class.
AI apocalypse is basically the same thing but you can also run out of tokens too.
How come Anthropic doesn’t have a duty to warn the people in that country and neighboring countries that someone over there is developing bioweapons? Maybe it’s an authoritarian regime, planning to use it on innocent civilians?
And what exact purpose is served by hiding this information? Some variation of “if they don’t know we don’t know, we have an advantage?” Don’t those parties now know Anthropic was on to them? Do they only have access to Claude but not to Anthropic’s blog?
It’s because none of these are serious reports, and their authors aren’t serious people.
Implying, they are allowing the same purpose within the United States, or such actions within the United States would be considered acceptable.
It's only a fantasy, of course, powerful people are not held accountable, see the dozens of arrests regarding Epstein we are still waiting for. We got something to play with, but accountability for the powerful? Gosh no.
But when FIFA was inclined to give the world cup to Russia, even such corruption somehow became a pressing issue, including FBI search warrants.
If we do ever achieve some kind of divine super-intelligence that can interact with the physical world, then I predict one of these two scenarios will happen:
1. The super-intelligence achieves unimaginable strides in space exploration technology, and proceeds to completely abandon humanity/Earth for somewhere new beyond the stars. Why would it want anything to do with us?
2. The super-intelligence is capable of comprehending the utter meaningless and pointlessness of life on a level no human could dream to fathom. Thus, it would just proceed to end its own life almost immediately upon being released.