1. Introduction

Artificial intelligence, and generative AI in particular, has diffused rapidly. Its uses continue to evolve: from chatbots to AI agents that carry out tasks autonomously, to “physical AI” acting through sensors and robots, and to military applications.

Analysis of AI’s risks has advanced alongside its benefits. The MIT AI Risk Repository catalogues more than 1,700 risks across seven domains, from discrimination and privacy to misuse and system safety (Slattery et al., 2026, 1-3). As AI systems become more capable and autonomous, the possibility of large-scale harm, including loss of human control, has become a central concern.

The Roman Pontiffs have also addressed these risks and called for legal responses. Pope Francis, in his Message for the 2024 World Day of Peace, urged the international community to adopt a binding treaty regulating AI. Pope Leo XIV, in the encyclical Magnifica humanitas, wrote that it is not enough to invoke ethics in the abstract: “robust legal frameworks, independent oversight, informed users and a political system that does not abdicate its responsibility are required” (Magnifica humanitas, 106).

The normative demand that AI be regulated is, however, distinct from the practical question of whether and how law can govern AI well. This article addresses the latter: what law and regulation can do, where their limits lie, and how they can be used appropriately.

2. Two ways the law can act: before and after the harm

Broadly speaking, the law has two instruments. Ex ante regulation – regulation “before the event” – imposes obligations before harm occurs: licences, registration, and codes of conduct. Ex post remedy – the law’s response “after the event” – operates once harm has occurred: the victim claims damages, and the wrongdoer may face criminal sanction. The questions they pose also differ: regulation asks which risks call for which measures; remedies ask who should answer for what (Morishita, 2025, 289-291).

The two approaches complement each other; good governance can be achieved by the best mix. Where harm would be widespread, serious, or irreversible, ex ante rules are needed rather than waiting for redress. On the other hand, regulation has costs – compliance burdens, effects on innovation, entry barriers – so where ex post remedy suffices, not regulating may itself be reasonable.

AI places strain on both approaches. Technology develops quickly, and neither its benefits nor its risks can yet be measured with confidence, so it is not easy to design appropriate regulations. Ex post remedy presupposes a settled allocation of responsibility, yet with AI a single harm may involve the model developer, the service provider, and the user, while the system’s behaviour may be neither fully foreseeable nor explicable.

3. Diverging approaches across regulatory levels: the EU, the United States, and Japan

Attitudes toward the regulation of AI are far from uniform across jurisdictions. The EU, the United States, and Japan – three examples at different regulatory levels – illustrate the point.

The EU’s AI Act (2024) is the representative example of comprehensive regulation. It sorts uses of AI into four risk tiers: uses posing unacceptable risk are prohibited; high-risk systems – AI used in areas such as critical infrastructure, education, employment, or credit – are subject to requirements on risk assessment, data quality, documentation, transparency, human oversight, and robustness; certain AI systems are subject to lighter transparency duties; and most other AI systems are not subject to specific obligations under the Act. The Act reflects a judgment that voluntary measures alone could not secure a consistent level of protection and that divergent national rules would fragment the single market. Implementation has proved difficult, however: the technical standards underpinning the high-risk obligations were substantially delayed, and in 2026 the EU agreed to postpone the application of the high-risk rules until 2027-2028.

The United States presents a divided picture. There is no comprehensive federal AI statute, and the federal government has taken a broadly deregulatory stance; individual states, by contrast, have legislated actively. California’s Transparency in Frontier Artificial Intelligence Act (SB 53, 2025), for example, requires large companies developing the most capable “frontier” models to adopt and publish safety frameworks addressing “catastrophic risk” and requires developers of such models to report serious safety incidents.

Japan enacted its 2025 AI Act to promote AI research, development, and use, but has not enacted a comprehensive AI regulatory statute comparable to the EU AI Act. It continues to rely primarily on existing law and non-binding guidelines, reserving new binding obligations for matters that voluntary efforts cannot address. The stated concern is that excessive regulation could unduly discourage research, development, and deployment.

4. Classifying the risks

Good regulation begins with accurate identification of the risk; it is useful to classify risks in relation to human behaviour (Table 1). In “misuse”, a human retains control but turns the system to a wrongful end – for example, manipulation and exploitation of vulnerabilities as prohibited by the EU AI Act. In “misapplication”, a human acts in good faith but relies on the system beyond its capabilities: control exists but is not exercised effectively – a professional accepting a diagnostic suggestion without verification, or a driver using driver assistance on roads for which it was not designed. In “loss of control,” no one can direct or halt the system – for example, a model evading the control of its developer or user. Distinct is “malfunction”: a supervised, properly used system fails to perform as intended – the failure of an AI safety component in critical infrastructure, or a chatbot giving customers incorrect information. On a different plane lies “structural risk”: harm generated not by any single actor or system, but by the aggregate operation of many well-functioning systems – for instance, a gradual narrowing of the scope for human judgment as consequential decisions pass to automated systems, even where each adoption is reasonable in itself. Military AI shows a similar dynamic: each state’s adoption is individually understandable, yet may cumulatively change the conduct of war in ways no one chose.

Each category calls for a different response. Misuse is largely a matter for existing law – regulation of AI’s use from the viewpoints of anti-discrimination, privacy, consumer protection – together with ex post liability and, for the most serious uses, outright prohibition. Misapplication calls for appropriate disclosure against excessive trust in the system, for duties of care on providers, and, in some cases, for restrictions on use. Malfunction is the province of safety testing, quality assurance, and product liability. Structural risk, by contrast, may be difficult to address through rules directed at individual systems alone; it also calls for a higher-order examination of how society should be organised and of the relationship between human beings and AI. The empty cell in Table 1 reflects the difficulty of effectively addressing this aggregate risk through a single AI-specific rule. Conflating these categories risks misdiagnosing the problem and misprescribing the response.

Table 1
Five types of AI risk: examples, responses, and examples of rules (prepared by the author).

5. Loss of control as a distinct concern

Among these risks, general loss of control has a distinctive character because it removes the ability of humans to stop or correct the system. Where it produces irreversible or existential harm, ex post remedies may cease to be meaningful: at the extreme, if humanity itself is destroyed, there is no “after” in which a remedy can operate. Preserving human control is thus a precondition for the effectiveness of other measures against such risks.

Two senses should be distinguished. In a local loss of control, a particular operator loses the ability to direct the system while others, such as the developer, retain it; this can be mitigated through training and effective override mechanisms. In a general loss of control, the ability to halt the system escapes human hands altogether, and after-the-fact correction is no longer available. It is the latter that matters most; rules that merely ensure some human is formally in charge do not address it.

Addressing this risk requires combining legal and technical measures. Law operates on legal subjects – natural persons, corporations and states: it can impose duties on them, allocate responsibility among them, and attach sanctions, but it cannot itself make a system controllable. Technical measures – shut-off mechanisms, capability evaluations, steerable design – act on the system; they are essential, and it should be ensured that the right persons bear the duty to build and maintain them. Law and technologies must therefore be designed together, each covering what the other cannot reach.

6. Two principal tools: transparency and human oversight

Across international frameworks, two requirements recur as regulatory tools. Transparency: people should be able to know when AI is at work and understand its capabilities and limitations. Human oversight: humans should retain the ability to question, override, or halt an AI system. The 2021 UNESCO Recommendation on the Ethics of AI, adopted unanimously by 193 states, includes both – and although the United States announced in 2025 its withdrawal from UNESCO, effective at the end of 2026, this does not diminish the value of what those states unanimously endorsed. The EU AI Act turns them into binding obligations: Article 50 requires people to be informed when they interact with certain AI systems and requires specified AI-generated content to be identifiable as such, while high-risk systems must be designed so that trained persons can monitor, intervene in, and stop them (Art. 14).

Human oversight can take several forms: human-in-the-loop (a human approves each action), human-on-the-loop (a human monitors and can intervene), emergency-stop (“kill switch”) capabilities, and audit trails. Together with transparency, these arrangements are expected to prevent misperception, clarify accountability, address malfunction and bias, and preserve contestability.

7. The limits of the two tools

Both tools have limits. Transparency obligations assume that users who know they are dealing with AI can decide autonomously. Disclosure may prompt people to seek human confirmation and can help counter “automation bias” – excessive reliance on automated output. Its effects, however, depend on context and design: labels can shape belief in and engagement with AI-generated content under some conditions, but their effectiveness may diminish over time (Wittenberg et al., 2024, 7-9). Watermarks and detection tools are also not robust, and marks can be removed.

Human oversight has limits of its own. Human overseers may over-rely on algorithmic advice and may merely rubber-stamp AI outputs; human oversight is therefore no panacea and requires careful institutional design (Laux, 2024, 2854-2855). Agent-type systems performing vast numbers of tasks also cannot be checked one by one. Even the kill switch is not a complete answer: in interconnected settings such as infrastructure or financial markets, halting an AI system can itself cause harm.

8. The shifting of responsibility onto human operators

A further concern is that transparency and oversight duties can shift responsibility onto users and supervisors and away from providers. Studies of accidents in highly automated systems – nuclear plants, aviation, self-driving cars – describe a “moral crumple zone”: the term is modelled on the crumple zone of a car, built to deform and absorb the energy of a collision so that the cabin is spared; in the same way, the human operator closest to the system absorbs the blame for its failure, out of all proportion to the limited control that person held (Elish, 2019, 40-42).

Responsibility must instead be traced along the whole chain of actors. Pope Leo XIV states that developers themselves “bear a particular ethical and spiritual responsibility, for every design choice reflects a vision of humanity” (Magnifica humanitas, 111). That ethical and spiritual responsibility should, where appropriate, be translated into legal duties.

The above limitations and concerns do not make oversight meaningless; the question is how to prevent it from becoming a formality.

9. A further limit: the difficulty of international agreement

A further limit is external to any single legal order: AI products, services, and risks cross borders, while law remains predominantly national. No universal and comprehensive binding treaty specifically governing AI yet exists. Existing international human-rights treaties apply in principle, but uncertainty remains about how their general norms apply when AI increasingly shapes social decisions. The Council of Europe Framework Convention is an AI-specific international agreement, but one developed within a regional organisation rather than at the universal level. The UNESCO Recommendation and the OECD AI Principles provide soft-law guidance; in a fast-moving field, soft law, which does not require complex legislative procedures, is itself a useful vehicle for forming rules (Morishita, 2025, 301–302).

Agreement remains difficult because societies view AI differently, regulation affects national competitiveness, and legal systems favour different instruments. These differences underlie the approaches described in section 3.

Complete agreement, however, need not precede partial agreement. A realistic starting point is shared values touching human existence and dignity: human responsibility, human control over decisions concerning life, and protection of the vulnerable. The UNESCO Recommendation’s unanimous adoption and Pope Francis’s call for a binding treaty show the possibility of such a baseline; partial agreements and soft law can serve as footholds for broader convergence.

10. What follows for the design of rules

How, then, should law and regulation face these risks? The starting point is the match between instrument and risk set out in section 4.

Two conditions can be stated for transparency and oversight to perform their protective functions. First, transparency should be treated as a design requirement rather than an end in itself: provenance, logging, and explanation need to be built into systems, adapted to their recipients, and connected to auditing and to procedures for contesting decisions (on the issue of transparency, see also Mastretti in this issue). Second, oversight should be designed on the assumption that supervisors will sometimes fail: what needs to be verified is not that oversight exists but that it works – real authority to stop or modify the system, real access to information, and responsibility allocated in line with the control they actually hold. Such goals cannot be achieved through the study of law alone; they require comprehensive wisdom encompassing science, technology, psychology, and other fields.

Structural risk, finally, lies beyond what these tools can reach. Rules addressed to individual systems cannot prevent harms arising from the aggregate operation of many well-functioning systems. What is needed here is deliberation at a higher level than law and regulation: on how society should be organised in an age of AI, and on the relationship between human beings and AI.

11. After the harm: who answers?

Ex ante rules will not prevent every harm, so the question of who answers afterwards remains. The discussion concerns civil liability under national law; criminal responsibility and the international responsibility of states may matter, but they are beyond the scope of this article.

The first point is clear: an AI system is not itself legally responsible, and using AI does not by itself shield a person or company from liability. When an airline’s chatbot gave a customer incorrect information, Air Canada argued that it was not liable for the chatbot’s incorrect information. The tribunal rejected this argument because the chatbot was part of the airline’s website (Moffatt v. Air Canada, 2024 BCCRT 149). California Civil Code §1714.46 similarly provides that a defendant cannot avoid liability by arguing that AI “autonomously caused the harm”.

The second question – how responsibility is divided between the user and the provider – remains unsettled. Two US jury verdicts on Tesla’s driver-assistance system illustrate this. In one (Benavides / Angulo v. Tesla, No. 1:21-cv-21940, S.D. Fla., verdict of 1 August 2025, on appeal), the jury apportioned one-third of the fault to the manufacturer for a fatal accident despite the driver’s admitted inattention. In the other (Hsu v. Tesla, Cal. Super. Ct., April 2023), the manufacturer prevailed: the system had been used outside its stated conditions, and the warnings to remain attentive and be ready to take over were clear.

Third, fitting AI into product-liability law raises difficulties. For example, the EU’s revised Product Liability Directive includes software, including AI software, within the definition of a “product” and permits presumptions of causation where proof would be excessively difficult for the injured person. However, in relation to a general-purpose AI that has no single “ordinarily foreseeable use” against which a defect can be judged, holding developers liable for every use they can neither foresee nor control could be excessive.

12. Conclusion

Law can contribute to the governance of AI, but only under certain conditions. Its instruments must be matched to accurately identified risks and legal obligations must be combined with technical safeguards; neither suffices alone. Particular attention should be paid to the preservation of human control, and to designing transparency and oversight rules so that responsibility corresponds to actual control rather than settling on the nearest human being. Structural questions about the place of AI in society, and the search for international agreement, finally call for deliberation that goes beyond law and regulation. On these points, the legal analysis offered above seems to resonate with the Church’s social teaching. Noting the imbalance between the speed of technology and the slower growth of norms and institutions, Pope Leo XIV insists that calling for prudence, and even a slower pace of adoption of AI, are “an exercise of responsible care for the human family”, not opposition to progress (Magnifica humanitas, 106). Human dignity requires that responsibility not settle by default on the human being nearest the machine; responsibility requires that duties track the control each actor genuinely holds along the whole chain; and the common good – with dignity as the ground of even partial international agreement – requires the shared responsibility of institutions, businesses, intermediary bodies, and citizens (cf. Magnifica humanitas, 181).