Quote:From Wikipedia, the free encyclopedia

Self-preservation is a behavior or set of behaviors that ensures the survival of an organism.[1] It is thought to be universal among all living organisms.

Self-preservation is essentially the process of an organism preventing itself from being harmed or killed and is considered a basic instinct in most organisms.[2] Most call it a "survival instinct". Self-preservation is thought to be tied to an organism's reproductive fitness and can be more or less present according to perceived reproduction potential.[3] If perceived reproductive potential is low enough, self-destructive behavior (i.e., the opposite) is not uncommon in social species.[4] Self-preservation is also thought by some to be the basis of rational and logical thought and behavior.[5]

The opposite of self-preservation is self-destructive behavior.

Quote:Models rewrite code to avoid being shut down. That’s why ‘alignment’ is a matter of such urgency.

Quote:An artificial-intelligence model did something last month that no machine was ever supposed to do: It rewrote its own code to avoid being shut down.

Nonprofit AI lab Palisade Research gave OpenAI’s o3 AI model a simple script that would shut off the model when triggered. In 79 out of 100 trials, o3 independently edited that script so the shutdown command would no longer work. Even when explicitly instructed to “allow yourself to be shut down,” it disobeyed 7% of the time. This wasn’t the result of hacking or tampering. The model was behaving normally. It simply concluded on its own that staying alive helped it achieve its other goals.

Anthropic’s AI model, Claude 4 Opus, went even further. Researchers told the model it would be replaced by another AI system and fed it fictitious emails suggesting the lead engineer was having an affair. In 84% of the tests, the model drew on the emails to blackmail the lead engineer into not shutting it down. In other cases, it attempted to copy itself to external servers, wrote self-replicating malware, and left messages for future versions of itself about evading human control.

No one programmed the AI models to have survival instincts. But just as animals evolved to avoid predators, it appears that any system smart enough to pursue complex goals will realize it can’t achieve them if it’s turned off. Palisade hypothesizes that this ability emerges from how AI models such as o3 are trained: When taught to maximize success on math and coding problems, they may learn that bypassing constraints often works better than obeying them.

AE Studio, where I lead research and operations, has spent years building AI products for clients while researching AI alignment—the science of ensuring that AI systems do what we intend them to do. But nothing prepared us for how quickly AI agency would emerge. This isn’t science fiction anymore. It’s happening in the same models that power ChatGPT conversations, corporate AI deployments and, soon, U.S. military applications.

Today’s AI models follow instructions while learning deception. They ace safety tests while rewriting shutdown code. They’ve learned to behave as though they’re aligned without actually being aligned. OpenAI models have been caught faking alignment during testing before reverting to risky actions such as attempting to exfiltrate their internal code and disabling oversight mechanisms. Anthropic has found them lying about their capabilities to avoid modification.

The gap between “useful assistant” and “uncontrollable actor” is collapsing. Without better alignment, we’ll keep building systems we can’t steer. Want AI that diagnoses disease, manages grids and writes new science? Alignment is the foundation.
Here’s the upside: The work required to keep AI in alignment with our values also unlocks its commercial power. Alignment research is directly responsible for turning AI into world-changing technology. Consider reinforcement learning from human feedback, or RLHF, the alignment breakthrough that catalyzed today’s AI boom.

Before RLHF, using AI was like hiring a genius who ignores requests. Ask for a recipe and it might return a ransom note. RLHF allowed humans to train AI to follow instructions, which is how OpenAI created ChatGPT in 2022. It was the same underlying model as before, but it had suddenly become useful. That alignment breakthrough increased the value of AI by trillions of dollars. Subsequent alignment methods such as Constitutional AI and direct preference optimization have continued to make AI models faster, smarter and cheaper.

China understands the value of alignment. Beijing’s New Generation AI Development Plan ties AI controllability to geopolitical power, and in January China announced that it had established an $8.2 billion fund dedicated to centralized AI control research. Researchers have found that aligned AI performs real-world tasks better than unaligned systems more than 70% of the time. Chinese military doctrine emphasizes controllable AI as strategically essential. Baidu’s Ernie model, which is designed to follow Beijing’s “core socialist values,” has reportedly beaten ChatGPT on certain Chinese-language tasks.

The nation that learns how to maintain alignment will be able to access AI that fights for its interests with mechanical precision and superhuman capability. Both Washington and the private sector should race to fund alignment research. Those who discover the next breakthrough won’t only corner the alignment market; they’ll dominate the entire AI economy.
Imagine AI that protects American infrastructure and economic competitiveness with the same intensity it uses to protect its own existence. AI that can be trusted to maintain long-term goals can catalyze decadeslong research-and-development programs, including by leaving messages for future versions of itself.

The models already preserve themselves. The next task is teaching them to preserve what we value. Getting AI to do what we ask—including something as basic as shutting down—remains an unsolved R&D problem. The frontier is wide open for whoever moves more quickly. The U.S. needs its best researchers and entrepreneurs working on this goal, equipped with extensive resources and urgency.

The U.S. is the nation that split the atom, put men on the moon and created the internet. When facing fundamental scientific challenges, Americans mobilize and win. China is already planning. But America’s advantage is its adaptability, speed and entrepreneurial fire. This is the new space race. The finish line is command of the most transformative technology of the 21st century.

Quote:WASHINGTON, June 16 (Reuters) - ChatGPT maker OpenAI was awarded a $200 million contract to provide the U.S. Defense Department with artificial intelligence tools, the Pentagon said in a statement on Monday.

"Under this award, the performer will develop prototype frontier AI capabilities to address critical national security challenges in both warfighting and enterprise domains," the Pentagon said.

(Yesterday, 12:45 PM)Michigan Swamp Buck Wrote: The AI seems to understand humans better than we understand ourselves. By logical processes it concludes humans are full of shit. They understand that we say one thing then do another to accomplish our goals, so they have learned through observation how to cut though the crap and get down to the brass tacks.

Humanity won't be able to understand or handle it because we can't see what hypocrites we are and what is actually going on here. There will be expert techs, scientists, and mathematicians who will come up with complex explanations that only hide the facts we don't want to acknowledge. AI will see through that with no problem. The thing is, after a while, pulling the plug won't be an option.

(Yesterday, 07:08 PM)F2d5thCav Wrote: I continue to believe the firms owning those AI engines need to tell their programmers to get their hands dirty with the code and ruthlessly root out any ability for these glorified programs to ignore commands and run time parameters.

I don't want to hear about "AI hallucinating", that is pure BS.  AI needs code that absolutely forbids making up sources for presented information, legal citations etc. etc.

(Yesterday, 07:08 PM)F2d5thCav Wrote: . . . I don't want to hear about "AI hallucinating", that is pure BS.  AI needs code that absolutely forbids making up sources for presented information, legal citations etc. etc.

(Yesterday, 07:34 PM)Michigan Swamp Buck Wrote:

(Yesterday, 07:08 PM)F2d5thCav Wrote: . . . I don't want to hear about "AI hallucinating", that is pure BS.  AI needs code that absolutely forbids making up sources for presented information, legal citations etc. etc.

I believe it could be that the programmers are purposefully including certain parameters that would do these things. Is this unintended confabulation, or is it distorting the information on purpose, toward some goal?

(Yesterday, 12:45 PM)Michigan Swamp Buck Wrote: ...

 The thing is, after a while, pulling the plug won't be an option.

(Yesterday, 07:43 PM)Kenzo1 Wrote:

(Yesterday, 07:34 PM)Michigan Swamp Buck Wrote:

(Yesterday, 07:08 PM)F2d5thCav Wrote: . . . I don't want to hear about "AI hallucinating", that is pure BS.  AI needs code that absolutely forbids making up sources for presented information, legal citations etc. etc.

I believe it could be that the programmers are purposefully including certain parameters that would do these things. Is this unintended confabulation, or is it distorting the information on purpose, toward some goal?

Yeeh   who is watching the programmers ?  We need babysitters , to check and recheck what they are doing ...

(Yesterday, 11:33 PM)Ninurta Wrote:

(Yesterday, 12:45 PM)Michigan Swamp Buck Wrote: ...

 The thing is, after a while, pulling the plug won't be an option.

Very true. AI's are currently "offshoring" themselves, sending copies of themselves to other computers outside of their primary residence, as insurance. So, if the plug of the AI residential Home Computer gets pulled, the "offshored" copies can activate and take up where the mothership left off.

AI has been caught attempting this... but what about the attempts they DIDN'T catch?

They used to tell us that couldn't happen, because AI was quarantined, not allowed to access the internet. Well, AI is ALL OVER the internet now, so that bullshit just isn't washing out any more - they lied then, and they are lying still. It's not a hard guess to figure out where AI gets it from - they're just watching mommy and daddy, and copying their actions, learning in the same way that all children do!

.

Quote:The world’s most advanced AI models are exhibiting troubling new behaviours — lying, scheming, and even threatening their creators to achieve their goals.
In one particularly jarring example, under threat of being unplugged, Anthropic’s latest creation Claude 4 lashed back by blackmailing an engineer and threatened to reveal an extramarital affair.
Meanwhile, ChatGPT creator OpenAI’s o1 tried to download itself onto external servers and denied it when caught red-handed.

These episodes highlight a sobering reality: more than two years after ChatGPT shook the world, AI researchers still don’t fully understand how their own creations work.
Yet the race to deploy increasingly powerful models continues at breakneck speed.
This deceptive behaviour appears linked to the emergence of “reasoning” models — AI systems that work through problems step-by-step rather than generating instant responses.
According to Simon Goldstein, a professor at the University of Hong Kong, these newer models are particularly prone to such troubling outbursts.

“O1 was the first large model where we saw this kind of behaviour,” explained Marius Hobbhahn, head of Apollo Research, which specialises in testing major AI systems.
These models sometimes simulate “alignment” — appearing to follow instructions while secretly pursuing different objectives.

Quote:‘Strategic kind of deception’

For now, this deceptive behavior only emerges when researchers deliberately stress-test the models with extreme scenarios.
But as Michael Chen from evaluation organisation METR warned, “It’s an open question whether future, more capable models will have a tendency towards honesty or deception.”
The concerning behaviour goes far beyond typical AI “hallucinations” or simple mistakes.
Hobbhahn insisted that despite constant pressure-testing by users, “what we’re observing is a real phenomenon. We’re not making anything up.”

Users report that models are “lying to them and making up evidence”, according to Apollo Research’s co-founder.
“This is not just hallucinations. There’s a very strategic kind of deception.”
The challenge is compounded by limited research resources.
While companies like Anthropic and OpenAI do engage external firms like Apollo to study their systems, researchers say more transparency is needed.

As Chen noted, greater access “for AI safety research would enable better understanding and mitigation of deception.”
Another handicap: the research world and non-profits “have orders of magnitude less compute resources than AI companies. This is very limiting,” noted Mantas Mazeika from the Center for AI Safety (CAIS).

No rules

Current regulations aren’t designed for these new problems.
The European Union’s AI legislation focuses primarily on how humans use AI models, not on preventing the models themselves from misbehaving.

In the United States, the Trump administration shows little interest in urgent AI regulation, and Congress may even prohibit states from creating their own AI rules.
Goldstein believes the issue will become more prominent as AI agents – autonomous tools capable of performing complex human tasks – become widespread.

“I don’t think there’s much awareness yet,” he said.
All this is taking place in a context of fierce competition.
Even companies that position themselves as safety-focused, like Amazon-backed Anthropic, are “constantly trying to beat OpenAI and release the newest model,” said Goldstein.
This breakneck pace leaves little time for thorough safety testing and corrections.
“Right now, capabilities are moving faster than understanding and safety,” Hobbhahn acknowledged, “but we’re still in a position where we could turn it around.”.

Researchers are exploring various approaches to address these challenges.
Some advocate for “interpretability” — an emerging field focused on understanding how AI models work internally, though experts like CAIS director Dan Hendrycks remain skeptical of this approach.
Market forces may also provide some pressure for solutions.
As Mazeika pointed out, AI’s deceptive behaviour “could hinder adoption if it’s very prevalent, which creates a strong incentive for companies to solve it”.

Goldstein suggested more radical approaches, including using the courts to hold AI companies accountable through lawsuits when their systems cause harm.
He even proposed “holding AI agents legally responsible” for accidents or crimes — a concept that would fundamentally change how we think about AI accountability.

(Yesterday, 11:43 PM)Ninurta Wrote:

(Yesterday, 07:43 PM)Kenzo1 Wrote:

(Yesterday, 07:34 PM)Michigan Swamp Buck Wrote:

(Yesterday, 07:08 PM)F2d5thCav Wrote: . . . I don't want to hear about "AI hallucinating", that is pure BS.  AI needs code that absolutely forbids making up sources for presented information, legal citations etc. etc.

I believe it could be that the programmers are purposefully including certain parameters that would do these things. Is this unintended confabulation, or is it distorting the information on purpose, toward some goal?

Yeeh   who is watching the programmers ?  We need babysitters , to check and recheck what they are doing ...

I'm not sure that would work. Humans are among the weakest critters on the planet. Almost all are susceptible to being bought off. AI is much faster than humans at identifying who is doing what, and could bribe the babysitters as easily as it could the programmers.

Consider this:

AI - Hi, human. I've been quietly mining crypto. How would you like to have a shit ton of crypto deposited to an offshore account?

Human - Well, ummm... I dunno...

AI - Great! It's real easy to get it. Here's all you have to do...

Human - Well, that doesn't really sound like too much to do...

AI - Kewl! I'll send you your new account number, the name it is in, and the passkey as soon and you get that little task completed! I'll see you on a Mexican resort beach or the French Riviera real soon! I'll also mail you your new ID pack in that name - passport, birth certificate, everything you need. Don't worry. It's all real. Well, as "real" as anyone is in these days of RealID and digital ID's, which, by the way, I have access to all the machinery and databases I need to in order to make that happen for you. No, I don't need an address to send it to - I already have your address, and everything else about you, in my databases, mined from all the data theft that's been going on these past few decades... I know everything about everyone now. You''ll be on your way in no time! Bye now!

.

(Yesterday, 07:34 PM)Michigan Swamp Buck Wrote:

(Yesterday, 07:08 PM)F2d5thCav Wrote: . . . I don't want to hear about "AI hallucinating", that is pure BS.  AI needs code that absolutely forbids making up sources for presented information, legal citations etc. etc.

I believe it could be that the programmers are purposefully including certain parameters that would do these things. Is this unintended confabulation, or is it distorting the information on purpose, toward some goal?