Artificial Intelligence (AI) is transforming our way of life from new forms of social organization and scientific discovery to defense and intelligence. This explosive progress is especially apparent in the subfield of machine learning (ML), where AI systems learn autonomously by identifying patterns in large volumes of data.[1] Indeed, over the last five years, the fields of AI and ML have witnessed stunning advancements in computer vision (e.g., object recognition), speech recognition, and scientific discovery.[2], [3], [4], [5] However, these advances are not without risk as transformative technologies are generally accompanied by a significant risk profile, with notable examples including the discovery of nuclear energy, the Internet, and synthetic biology. Experts are increasingly voicing concerns over AI risk from misuse by state and non-state actors, principally in the areas of cybersecurity and disinformation propagation. However, issues of control – for example, how advanced AI decision-making aligns with human goals – are not as prominent in the discussion of risk and could ultimately be equally or more dangerous than threats from nefarious actors. Modern ML systems are not programmed (as programming is typically understood), but rather independently developed strategies to complete objectives, which can be mis-specified, learned incorrectly, or executed in unexpected ways. This issue becomes more pronounced as AI becomes more ubiquitous and we become more reliant on AI decision-making. Thus, as AI is increasingly entwined through tightly coupled critical systems, the focus must expand beyond accidents and misuse to the autonomous decision processes themselves.
The principal mid- to long-term risks from AI systems fall into three broad categories: risks of misuse or accidents, structural risks, and misaligned objectives. The misuse or accident category includes things such as AI-enabled cyber-attacks with increased speed and effectiveness or the generation and distribution of disinformation at scale.[6] In critical infrastructures, AI accidents could manifest as system failures with potential secondary and tertiary effects across connected networks. A contemporary example of an AI accident is the New York Stock Exchange (NYSE) “Flash Crash” of 2010, which drove the market down 600 points in 5 minutes.[7] Such rapid and unexpected operations from algorithmic trading platforms will only increase in destructive potential as systems increase in complexity, interconnectedness, and autonomy.
The structural risks category is concerned with how AI technologies shape the social and geopolitical environment in which they are deployed. Important contemporary examples include the impact of social media content selection algorithms on political polarization or uncertainty in nuclear deterrence and the offense-to-defense balance.[8],[9] For example, the integration of AI into critical systems, including peripheral processes (e.g., command and control, targeting, supply chain, and logistics), can degrade multilateral trust in deterrence.[10] Indeed, increasing autonomy in all links of the national defense chain, from decision support to offensive weapons deployment, compounds the uncertainty already under discussion with autonomous weapons.[11]
Continue reading: https://www.hstoday.us/featured/artificial-intelligence-critical-systems-and-the-control-problem/
The principal mid- to long-term risks from AI systems fall into three broad categories: risks of misuse or accidents, structural risks, and misaligned objectives. The misuse or accident category includes things such as AI-enabled cyber-attacks with increased speed and effectiveness or the generation and distribution of disinformation at scale.[6] In critical infrastructures, AI accidents could manifest as system failures with potential secondary and tertiary effects across connected networks. A contemporary example of an AI accident is the New York Stock Exchange (NYSE) “Flash Crash” of 2010, which drove the market down 600 points in 5 minutes.[7] Such rapid and unexpected operations from algorithmic trading platforms will only increase in destructive potential as systems increase in complexity, interconnectedness, and autonomy.
The structural risks category is concerned with how AI technologies shape the social and geopolitical environment in which they are deployed. Important contemporary examples include the impact of social media content selection algorithms on political polarization or uncertainty in nuclear deterrence and the offense-to-defense balance.[8],[9] For example, the integration of AI into critical systems, including peripheral processes (e.g., command and control, targeting, supply chain, and logistics), can degrade multilateral trust in deterrence.[10] Indeed, increasing autonomy in all links of the national defense chain, from decision support to offensive weapons deployment, compounds the uncertainty already under discussion with autonomous weapons.[11]
Continue reading: https://www.hstoday.us/featured/artificial-intelligence-critical-systems-and-the-control-problem/
