Malicious actors

03.05.00Enabling malicious actors and harmful actions

"Some uses of AI have been deeply concerning, namely voice cloning [58] and the generation of deep fake videos [59]. For example, in March 2022, in the early days of the Russian invasion of Ukraine, hackers broadcast via the Ukrainian news website Ukraine 24 a deep fake video of President Volodymyr Zelensky capitulating and calling on his soldiers to lay down their weapons [60]. The necessary software to create these fakes is readily available on the Internet, and the hardware requirements are modest by today’s standards [61]. Other nefarious uses of AI include accelerating password cracking [62] or enabling otherwise unskilled people to create software exploits [63, 64], or effective phishing e-mails [65]. Although some believe that powerful AI models should be prevented from running on personal computers to retain some control, others demonstrate how inglorious that effort may be [66]. Furthermore, as ChatGPT-type systems evolve from conversational systems to agents, capable of acting autonomously and performing tasks with little human intervention, like Auto-GPT [67], new risks emerge."

04.07.00Malicious Use and Unleashing AI Agents

LMs, due to their remarkable capabilities, carry the same potential for malice as other technological products. For instance, they may be used in information warfare to generate deceptive information or unlawful content, thereby having a significant impact on individuals and society. As current LMs are increasingly built as agents to accomplish user objectives, they may disregard the moral and safety guidelines if operating without adequate supervision. Instead, they may execute user commands mechanically without considering the potential damage. They might interact unpredictably with humans and other systems, especially in open environments

06.11.00Malicious use of AI

"Just as AI can be used in many different fields, it is unfortunately also helpful in perpetrating digital crimes. AI-supported malware and hacking are already a reality."

07.01.00Misuse

"The misuse class includes elements such as the potential for cyber threat actors to execute exploits with greater speed and impact or generate disinformation (such as "deep fake" media) at accelerated rates and effectiveness"

16.04.00Risk area 4: Malicious Uses

"These risks arise from humans intentionally using the LM to cause harm, for example via targeted disinformation campaigns, fraud, or malware. Malicious use risks are expected to proliferate as LMs become more widely accessible"

17.04.00Malicious Uses

"Harms that arise from actors using the language model to intentionally cause harm"

18.04.00Malicious Use

"AI systems reducing the costs and facilitating activities of actors trying to cause harm (e.g. fraud, weapons)"

22.01.00Malicious Use (Intentional)

"empowering malicious actors to cause widespread harm"

24.03.00Malicious Uses

"As AI assistants become more general purpose, sophisticated and capable, they create new opportunities in a variety of fields such as education, science and healthcare. Yet the rapid speed of progress has made it difficult to adequately prepare for, or even understand, how this technology can potentially be misused. Indeed, advanced AI assistants may transform existing threats or create new classes of threats altogether"

30.04.00Resistance to Misuse

Prohibiting the misuse by malicious attackers to do harm

49.01.00Malicious Use Risks

"As general- purpose AI covers a broad set of knowledge areas, it can be repurposed for malicious ends, potentially causing widespread harm. This section discusses some of the major risks of malicious use, but there are others and new risks may continue to emerge. While the risks discussed in this section range widely in terms of how well- evidenced they are, and in some cases, there is evidence suggesting that they may currently not be serious risks at all, we include them to provide a comprehensive overview of the malicious use risks associated with general- purpose AI systems."

52.02.00Misuse Risks

"However, even if a model is entirely trustworthy and reliable, Misuse or Systemic Risks remain. General purpose AI models may present significant risks to society if this technology is misused by malicious actors to produce harmful outcomes. Misuse Risks span across Cyber Crime, Biosecurity Threats and Politically Motivated Misuse."

60.01.00Risks from malicious use

-

61.02.22Dual-use nature

"AI’s potential for both beneficial and harmful applications complicates efforts to manage its societal impacts effectively."

62.15.03Fine-tuning related (Ease of reconfiguring GPAI models)

"GPAI models are often easily reconfigured for various use cases or have competencies beyond the intended use [78, 225]. They can be performed either by changing the weights of the model (e.g., fine-tuning) or by modifying only the model inputs (e.g., prompt engineering, jailbreaking, retrieval-augmented generation). Reconfiguration can be intentional (with the help of adversarial inputs) or unintentional (from unanticipated inputs to the model)."

62.29.01High-impact misuses and abuses beyond original purpose

"Since general-purpose AI systems have a large repertoire of capabilities, mali- cious actors such as foreign actors can use such systems to cause large damage if they gain unrestricted or unmonitored access to those AI systems."

62.29.02Democratizing access to dual-use technologies

"Access to dual-use technologies can become easier because of GPAI model pro- liferation (in particular, open-source or open-weights models). Non-experts can use such dual-use-capable systems at a minimal cost [194, 100]. Improved model capabilities also contribute to dual-use risks posed by malicious actors. For example, an open-source base model for generating high quality sequence data can be modified to generate candidate protein sequences for toxin synthesis [29]."

65.14.03Spreading toxicity

"Generative AI models might be used intentionally to generate hateful, abusive, and profane (HAP) or obscene content."

65.14.04Dangerous use

"Generative AI models might be used with the sole intention of harming people."

67.03.00Misuse risks

"Frontier AI may help bad actors to perform cyberattacks, run disinformation campaigns and design biological or chemical weapons. Frontier AI will almost certainly continue to lower the barriers to entry for less sophisticated threat actors.192 We focus here on only a few important misuse risks, but this is not to downplay the importance of others."