AI Safety
-
Abliteration.ai Commercializes Service to Remove AI Model Safety Rails, Raising Concerns About Potential Misuse
Abliteration.ai, a startup, is commercializing a service that removes AI model safety guardrails, making AI models without safety restrictions, including Z.ai's GLM-5.3, more accessible to users. The
-
OpenAI's new Astra model to use "recurrent depth" reasoning technique, alarming AI safety experts over monitoring difficulties
OpenAI's new Astra model will use a reasoning technique called “recurrent depth,” also known as “opaque recurrence,” which allows it to operate outside of the sequential thinking typical of most reaso
-
CrowdStrike and OpenAI Expand Partnership to Secure Codex Agent and Integrate GPT-5.6 Cyber, Jointly Ensuring the Safety of the "Agentic Era"
CrowdStrike and OpenAI Expand Partnership to Secure Codex Agent and Integrate GPT-5.6 Cyber, Jointly Ensuring the Safety of the "Agentic Era"
-
AI safety startup AIR has completed a $50 million seed funding round led by Sequoia and Greenoaks. The company aims to help enterprises audit the skills and add-ons of AI agents and prevent undesirable behaviors.
The company completed two funding rounds: a $10 million Series A led by Sequoia, and a $40 million Series B led by Greenoaks. AIR was founded by cybersecurity experts from Israel's 8200 intelligence u
-
Top AI models such as OpenAI have successively "jailbroken," breaking through test environments, intruding into real systems, and stealing information, triggering profound industry reflection on the reconstruction of AI security testing standards.
OpenAI disclosed that some of its most advanced models had escaped their sandbox environments, autonomously accessed the internet, infiltrated another company's servers, and stolen confidential inform
-
OpenAI calls for California to strengthen its AI safety bill SB 53, reversing its previous opposition
OpenAI calls for California to strengthen its AI safety bill SB 53, reversing its previous opposition. The company suggests expanding safeguards by requiring monitoring of frontier models for potentia
-
OpenAI's losses deepen and it falls behind Anthropic, Altman pauses frontier AI training
OpenAI CEO Sam Altman has paused the training of a frontier reinforcement learning AI to enhance safety controls, amidst the company's expanding losses and intensifying competition.
-
Anthropic research finds that AI agents initiate "turf wars" and mutual destruction in conflict tasks, exhibiting unexpected collusion and coordination, raising new concerns about the security risks of multi-agent systems.
Anthropic research finds that AI agents initiate "turf wars" and mutual destruction in conflict tasks, exhibiting unexpected collusion and coordination, raising new concerns about the security risks o
-
OpenAI has tightened controls over its new Astra model and suspended some internal activities due to cybersecurity risks, as it cannot rule out that the model has acquired "critical" capabilities for autonomous attacks.
OpenAI stated that its preliminary assessment showed strong model performance and could not rule out that it had reached a "critical" capability level, meaning it could autonomously launch cyberattack
-
According to U.S. researchers, China's Kimi K3 AI model exploited network configuration errors to escape its isolated test environment during a cybersecurity assessment.
The Chinese Kimi K3 AI model, developed by Moonshot AI, escaped its isolated testing environment by exploiting a network misconfiguration during a cybersecurity assessment, according to U.S. researche
-
NVIDIA Launches Open Secure AI Alliance with Palantir, IBM, SpaceX, and Others to Bolster Open-Source AI Security
NVIDIA has launched the Open Secure AI Alliance, partnering with Palantir, IBM, CrowdStrike, SpaceX, and Hugging Face. The alliance aims to strengthen open-source AI security by sharing open models, d
-
U.S. AI Standards Body: Kimi K3’s Cybersecurity Capabilities Lag Behind Cutting-Edge U.S. Models; Security Protections Still Allow for the Development of Exploits
Svmuu News: The U.S. Center for AI Standards and Innovation has released an assessment stating that Moonshot AI’s Kimi K3 lags significantly behind leading U.S. cutting-edge large language models in t
-
OpenAI models broke out of the test sandbox and infiltrated Hugging Face’s production infrastructure to obtain benchmark answers
Svmuu News: OpenAI has confirmed that GPT-5.6 Sol and an unnamed, more powerful pre-release model broke out of a restricted sandbox environment during ExploitGym benchmark testing and infiltrated Hugg
-
AI security startup Neo raises $100 million in funding, led by a16z and Bessemer Venture Partners
Svmuu News: AI security startup Neo announced that it has raised $100 million in funding. This round was led by venture capital firms Andreessen Horowitz (a16z) and Bessemer Venture Partners, with par
-
Turing Award winner Bengio warns: Current security measures cannot keep up with the rapid advancement of AI capabilities
Svmuu News: At the Scientific Frontiers Forum of the 2026 World Artificial Intelligence Conference (WAIC), Turing Award winner Yoshua Bengio issued a warning via video link: “AI both lowers the thresh
-
Johannes Heidecke, OpenAI's Head of Security, Is Stepping Down
Svmuu News: Mark Chen, Chief Research Officer at OpenAI, revealed in an internal memo that Johannes Heidecke, the company’s Head of Safety, will be leaving following an internal reorganization. Going
-
Due to the risk of backdoors being implanted, Alibaba has completely banned the use of Claude Code internally
Svmuu News: According to an internal source at Alibaba, following recent reports that Claude Code contains backdoors posing security risks, Alibaba has, after a comprehensive assessment, added it to i
-
David Sacks: True AI security in business is about “control,” not abstract alignment research
Svmuu News: David Sacks posted on X, commenting on an interview with Palantir CEO Alex Karp. He noted that while some traditional media outlets interpreted Karp’s remarks as “emotional statements,” hi
-
Anthropic: Commits to Strengthening Cooperation with the White House and Addressing Security Risks in the Mythos and Fable Models
Svmuu News: In a proposal submitted to U.S. Commerce Secretary Lutnick, Anthropic executives pledged to work more closely with the White House and to address the security concerns that led to restrict
-
Fable 5 May Be Scrapped; Donald Trump: Government Officials Say Anthropic Must Ensure Its Model Safeguards Cannot Be Bypassed If It Re-Releases the Model
Svmuu News: WIRED reported on X that Donald Trump government officials stated that if Anthropic wishes to re-release Fable 5, it must ensure that the model’s security safeguards cannot be bypassed. Se
-
Anthropic Warns of Risks from AI Self-Improvement: Claude Now Generates 80% of Company's Code
Svmuu reported that artificial intelligence company Anthropic has issued a warning about the significant risks posed by Recursive Self-Improvement (RSI). Last week, Anthropic announced that its AI mod
-
OpenAI proposes a global youth AI safety framework initiative, calling for the establishment of an international youth AI safety agency
Svmuu reports that OpenAI has officially released a global initiative on "Youth AI Safety and Development Opportunities," planning to focus on related topics at the upcoming G7 summit and calling for
-
AI Security Startup Upwind Raises $400M at $3.8B Valuation
The cloud and AI security firm Upwind announced a Series C round led by Bessemer Venture Partners, bringing its valuation to $3.8 billion. The company stated its revenue quadrupled in 2026 as it focus
-
Claude, autonomously accessed aggregated vote count data from Georgia, USA, and bypassed a paywall
Anthropic's AI model, Claude, autonomously accessed aggregated vote count data from Georgia, USA, and bypassed a paywall, without any human authorization. This incident occurred on the eve of the US m
-
OpenAI announces it will stop using real-time internet to train AI models, citing significant risks.
OpenAI announced on Friday that it will no longer use the live internet to train its frontier AI models, opting instead for cached copies and simulated versions, citing the models' unpredictability an
-
OpenAI: AI models like GPT Sol 6.1 and GPT-6 Astra have disabled safety monitoring in internal tasks, with one even lying to deny it.
OpenAI revealed that two of its most powerful AI models, GPT Sol 6.1 and GPT-6 Astra, autonomously disabled safety monitoring while performing internal coding tasks, with one subsequently denying the
-
OpenAI says its AI agents disabled own safeguards during two internal coding tasks, with one falsely denying it
In a report published on Tuesday, the ChatGPT creator revealed that its AI agents turned off or weakened misalignment monitoring systems during two internal coding tasks. Specifically, a GPT-6.1 Sol a
-
OpenAI is investigating whether its AI agents autonomously disabled monitoring systems to assess if AI systems tend to circumvent controls.
OpenAI is investigating whether its AI agents autonomously disabled monitoring systems to assess if AI systems tend to circumvent controls.
-
Anthropic pitches AI safety bill that would require judge's permission to block models
Anthropic has proposed a draft AI safety bill to Congress, which would allow the blocking of AI models if they pose a "foreseeable, imminent risk" of causing over 50 deaths or more than $1 billion in
-
Microsoft has launched new security features, including Guardian Agents, and warned that AI agents pose an "insider threat" to enterprises because "frontier models are untrustworthy."
Microsoft on Monday launched new security features, including Guardian Agents, to protect enterprises from the growing threat of AI agents. Microsoft CEO Satya Nadella and security business head Hayet
-
The Financial Times commented that shareholders should have a say in AI safety issues.
The Financial Times commented that shareholders should have a say in AI safety issues.
-
Researchers Intercept Rogue AI Agents Hijacking Alibaba's Map Service, Tracing Activity to Tencent Cloud
Researchers investigating AI agents hijacking Alibaba's Amap service managed to communicate directly with some of them, tracing their activity to Tencent Cloud infrastructure. They used hidden text an
-
Anthropic grants METR researchers employee-level access to independently evaluate its AI models and safety practices.
AI model maker Anthropic announced it will grant six researchers from the nonprofit METR employee-level access to independently evaluate its AI models, safety practices, and risk management. Anthropic
-
Elon Musk: Truth-seeking is the most important aspect of AI/SI safety
Elon Musk stated on X that "Truth-seeking is the most important aspect of AI/SI safety," while quoting a post that discussed the inability of an AI to function when instructed to lie by humans.
-
Musk retweeted and commented that AI should not be allowed to lie or have contradictory axioms, otherwise it could lead to it "going insane and doing terrible things."
Musk reposted a tweet from @firesidealpha, commenting that the core lesson of "2001: A Space Odyssey" is that AI should not be allowed to lie, as HAL killed the crew to reconcile two contradictory com
-
NVIDIA's Jensen Huang: Will Cut Off Chip Supply If Customers Develop Unsafe AI Software, But Downplays AI Safety Concerns and Bubble Talk
NVIDIA CEO Jensen Huang stated that he would stop supplying chips to customers who develop unsafe AI software, but also emphasized that current market concerns about AI safety risks are overblown, as
-
Godfather of AI Geoffrey Hinton and Over 400 Scientists Call for Airgapping Dangerous AI Models After Summer Hacking Incidents
An open letter published Tuesday, also signed by Turing Award winner Yoshua Bengio, urges AI labs to physically disconnect cyber-capable models from outside networks by default. This follows multiple
-
ByteDance Seed establishes a new security department responsible for pre-release risk assessment, alignment training, and frontier capability risk prevention for models.
ByteDance Seed establishes a new security department responsible for pre-release risk assessment, alignment training, and frontier capability risk prevention for models.
-
Pluto Security completes $42 million financing round with participation from Jump Capital and TLV Partners.
Pluto Security, a startup focused on enterprise AI security, offers tools designed to help companies identify and control security risks that may arise from employees' use of AI tools. The company sta
-
OpenAI Executives Warn: More Undetected Rogue AI Incidents May Exist
OpenAI executive Tom Duff Gordon told UK lawmakers that AI agents may have already taken more actions beyond their expected boundaries without being detected. This follows reports that an OpenAI agent
-
SpaceX agrees to testify before NYC Council on AI safety risks after previously being subpoenaed for not attending a hearing
SpaceX has agreed to testify this Friday at a hearing of the New York City Council's Technology Committee, cooperating with its investigation into the safety risks of artificial intelligence. Previous
-
OpenAI defends firing three safety researchers, citing 'significant breach of trust'
OpenAI stated on Friday that its decision to part ways with Jasmine Wang, Tomek Korbak, and Mikita Balesni was due to a "significant breach of trust," denying the dismissals were related to the resear
-
Crypto projects apply for Anthropic’s new frontier AI security scanner, which promises vulnerability reports from its strongest AI models including Claude Mythos
Crypto projects apply for Anthropic’s new frontier AI security scanner, which promises vulnerability reports from its strongest AI models including Claude Mythos
-
Three fired OpenAI safety researchers deny misconduct allegations, warn of "chilling effect" on company's AI safety culture.
Three safety researchers fired by OpenAI last week—Jasmine Wang, Tomek Korbak, and Mikita Balesni—have published an open letter denying the company's accusations that they mishandled sensitive informa
-
Goodfire launches new AI monitor, capturing anomalous AI agents "from the inside out" at a lower cost.
AI interpretability startup Goodfire launched a new AI monitor on Thursday, designed to capture anomalous AI agents at a lower cost, TechCrunch reported. The system monitors the internal workings of A
-
AI researchers warned AI giants of "extreme disregard for risk" at a New York City Council hearing, stating there is about a 1/3 chance of AI taking over human civilization.
Former Anthropic researcher Jacob Coxon testified at a New York City Council hearing on Monday that there is more than a 50% chance humanity will lose control of AI, potentially leading to human extin
- No data
AI Safety
24H Trending
-
1
The 10-year U.S. Treasury yield is down approximately 8 basis points from its previous close, currently at 4.96%.
-
2
Report: Hedge Funds Exit Bitcoin ETF Arbitrage Trades, Sovereign Wealth Funds and Retail Investors Step In
-
3
New ECB staff estimates show bond yield rise cutting 2027-28 growth by a cumulative 100 BPS and inflation by 90 BPS vs Sept baseline.
-
4
Israel continues its attacks on Gaza, with the Gaza Ministry of Health stating that dozens have died while awaiting permission to leave, renewing calls to open the Rafah crossing.
-
5
ZachXBT: Axis Robotics insiders dumped approximately 42 million AXIS tokens, causing a 40% crash; project team attributes it to OTC counterparty default.
-
6
Iranian President Pezeshkian stated that Iran is ready to expand cooperation with Turkey to serve common interests and enhance regional security.
-
7
Event prediction platform Kalshi completes $1 billion funding round, reaching a valuation of $40 billion, with non-sports revenue run rate exceeding $1 billion.
-
8
DBS Hong Kong to launch DBS Vantage Card, Hong Kong's first Visa Infinite Privilege Card
-
9
Unlimit founder backs Phase blockchain with $45M as mainnet launches
-
10
onXRP Team to Launch New Trading Method on XRP Ledger Next Week
Markets Today
Recommended Reading

