[{"content":"1\tIntroduction The promise of artificial intelligence (AI) has brought reverberating impacts throughout aspects of everyday life, from chatbots to education to transportation, but nowhere has it been felt as decisively—or controversially—as in warfare. The advancement of Lethal Autonomous Weapons Systems (LAWS) has been a strategic goal of militaries for decades, driven by the hope of deploying agents that are more agile, precise, and capable than human soldiers, all while removing human combatants from immediate physical risk. However, the ethical implications of delegating the use of lethal force to non-human entities are frequently called into question.\nIn “Killer Robots,” one of the seminal papers on the ethics of LAWS, Robert Sparrow seeks to identify “who we should hold responsible when an autonomous weapon system is involved in an atrocity of the sort that would normally be described as a war crime” (Sparrow 62). Sparrow constructs a trilemma suggesting neither the programmer, the commanding officer, nor the machine itself can be held morally responsible for the actions of a truly autonomous system. He concludes that because a fundamental condition of jus in bello (the conduct of war) is the ability to locate responsibility for the death of enemies, the deployment of LAWS is ethically impermissible.\nIn this paper, I argue that Sparrow’s conclusion rests on a conceptual conflation of functional autonomy with moral agency. Specifically, I posit that AI, regardless of its computational sophistication, lacks the capacity for moral intent. By distinguishing strictly between functional autonomy (the ability to operate without human intervention) and moral agency (the capacity for moral reasoning), we can resolve Sparrow’s responsibility gap. Sparrow assumes that because a system can operate independently of human intervention, it enters the sphere of moral accountability. I challenge this premise. I argue that autonomy in a machine is a description of its capabilities—its ability to execute complex instructions in dynamic environments—whereas moral agency requires an internal qualitative state (such as the capacity to care or suffer) that defines moral intent.\nBy distinguishing strictly between these two categories, we resolve Sparrow’s responsibility gap. Even if an AI becomes fully autonomous in its decision-making loops, it remains an instrument devoid of moral will. Therefore, the failure of such a system is not a moral transgression by the machine, but a failure of safety engineering or deployment. I argue that the gap Sparrow identifies is not a moral vacuum, but a legal and regulatory challenge best addressed through the frameworks of negligence and risk, drawing on analogies to pre-autonomous weaponry and autonomous civilian vehicles.\nFinally, using the risk-based frameworks proposed by Thomas Simpson and Vincent Müller, alongside the critique of human decision-making by Aziz Huq, I argue that if AI systems reach a certain degree of safety and capability superior to human operators, there is an ethical case for their adoption. The demand for “human in the loop” often prioritizes the psychological need for a scapegoat over the substantive goal of minimizing harm. Ultimately, if LAWS is calibrated to reduce civilian casualties, the absence of a moral agent to “blame” is an acceptable trade-off for a safer reality of warfare.\n2\tSparrow’s Trilemma and Moral Intent Sparrow’s central argument relies on a specific scenario: an autonomous weapons system commits an act that, were it performed by a human, would be a war crime (e.g., bombing a surrendering unit). Sparrow argues that for war to remain a moral enterprise rather than mere slaughter, someone must be responsible. He then systematically attempts to absolve the three most likely parties—the programmer, the commanding officer, and the machine—from blame, creating a “responsibility gap.”\nHowever, this trilemma rests on a conflation of functional autonomy with moral capacity. Sparrow treats the autonomous machine as an entity capable of usurping the moral agency of its creators, much like a child growing into an adult. This anthropomorphization of artificial intelligence systems obscures the reality of the sophisticated, albeit machine, computation happening underneath.\n2.1\tThe Absence of Moral Intent It is crucial to define the ontological status of current and near-future AI. We train AI on vast amounts of data, asking it to mimic human decision making, language, speech, and more. As training scales increase, emergent capabilities that were not thought possible of binary machines have appeared in deep learning systems, from complex “chains of thought” before generating responses to superhuman capabilities in some areas. Ultimately, current and near-future AI systems have no way of thinking for themselves1 in the way that humans do2: all of their decisions or generations are based on their training data.\nTo the extent that futurists predict systems capable of recursive self-improvement without human oversight, one might argue for the eventual emergence of genuine moral agency. However, this theoretical possibility does not undermine the argument regarding LAWS; rather, it shifts the category of the debate entirely. As Sparrow himself observes, if a machine were to achieve the status of a moral patient, the primary military justification for its deployment—reducing the risk to moral patients (human soldiers)—would be nullified. The machine would no longer be a disposable weapon, but a conscript with its own rights. Therefore, the ethical debate concerning LAWS qua weapons systems necessarily presupposes that they remain non-sentient instruments. Consequently, this paper proceeds on the premise that these systems, regardless of their capability, remain sophisticated tools devoid of moral agency.\nThe reason the debate over LAWS reaches an impasse regarding accountability is that critics such as Sparrow attempt to locate moral responsibility within a system that lacks moral intent. In criminal law, a crime usually requires both an actus reus (guilty act) and mens rea (guilty mind).\nConsider a human soldier who bombs non-combatants because of their skin color. We can identify this soldier as racist, as they made a decision to cause immense harm based on a corrupt moral calculus. They possess moral intent. Now, consider a LAW that bombs the same group because its training data contained a bias that associated a certain visual demographic with combatant status. The outcome is identical, but the internal process is radically different. The LAW did not make an “immoral” decision; it executed a command that was derived from negligent programming, selection of datasets, or safety checks that were not rigorous enough.\nIf a precision-guided missile suffers a guidance system failure and strikes a civilian building instead of a military target, no one charges the missile—or its operator—with a war crime in the sense of intentional slaughter. It would likely be treated as a tragic accident or a failure of due diligence. This is acknowledged by Sparrow, who says “accidents do happen, and even more so in war. However, these accidents represent regrettable, if inevitable, failures to live up to principles of justice in war fighting” (67). The debate over LAWS is clouded because the system’s autonomy emulates agency. When Sparrow wrote “Killer Robots” before deep learning existed, he hypothesized that “autonomy and moral responsibility go hand in hand” (65).\nHowever, the development of modern deep learning systems has empirically demonstrated that this coupling is false. We have discovered that functional autonomy—the ability to operate independently in dynamic environments—can expand indefinitely without ever generating moral agency. The error lies in conflating the selection of means with the selection of ends. As Simpson and Müller note, true moral autonomy involves the “ability of a system to decide on its own ends” (Simpson \u0026amp; Müller 312)3. Deep learning systems, no matter how complex, are fundamentally optimization engines. They do not choose their own ends; they optimize for a reward function or objective parameter set by human designers. An AI like AlphaGo might make a move (a means) that its creators could never have predicted to win a game, but it has no capacity to question whether it should play Go or whether winning is a worthy goal (the end). Its “autonomy” is merely an expansion of its capability to solve problems, not an emergence of will. Once we strip away the “humanness” of autonomy, then the repercussions for LAWS failures fall squarely into the realm of negligence, resolving the gap Sparrow feared.\n2.2\tAddressing the Child Soldier Sparrow’s most powerful use of analogy to support his argument against LAWS is that of child soldiers. He argues that a commander who sends a child into battle creates a dilemma: the child is not fully responsible, but the commander cannot control the child’s autonomous actions. As the child does not understand the full moral dimensions of what they do, they are not appropriate objects of punishment. However, their autonomy ensures that those who order them into action do not control them.\nMy issue here lies not with the accountability in the case of a child soldier, but the analogy to AI. It is true that the programmer or commander cannot control AI’s every action. The purpose of general learning in agents is to be able to both encounter and train behavior in all contexts that are expected, as well as to generalize when the context was not trained on. However, the issue here lies with the range of what a child can do. The reason a child soldier becomes more dangerous than an adult soldier is because commanders can communicate the same intent, but they cannot be sure that the child will understand and execute the order without unintentionally doing something bad (e.g. killing and torturing civilians).\nA child that engages in a moral wrong without knowing is not responsible because they lack a fully developed moral compass. However, an AI that engages in what would humanly be considered a moral wrong cannot be its own fault because it executed that action in accordance with its most probable actions given context. On a machine level, the LAWS presumably wrongfully kills a noncombatant because it misidentified them as a combatant, or was prioritizing something else over the life of the noncombatant. As a machine, it is possible (and perhaps necessary) to implement rule-based instructions that cannot be overridden such as not killing anyone who is identified as a non-combatant; neglecting to add that rule would result in responsibility of the programmer. Training a “soldier/civilian identification” network that is overly faulty, which could lead to mislabeling civilians as soldiers and attacking them, would also be neglect on the part of the programmer, or neglect on the part of the commanding officer for still allowing such a system to be deployed. In more complex and unexpected scenarios where behavior is typically less predictable, AI is typically trained to mimic a set of human-curated values rather than operate on more unformed values: the risk present, then, is not a product of the autonomy of the AI, but rather the degree of care in programming the system and the degree of caution in deploying the system.\nAlthough there is a sense in which AI is analogous to a child in that they can commit moral wrongdoings4 without moral intent themselves, the difference is that AI is much more controllable and actions that cause harm can be completely restricted in high-risk, out-of-distribution situations. A child may encounter many scenarios that they haven’t seen before and commit moral wrongdoings because their moral compass is not fully developed, but an AI is meant to have seen all possible scenarios given by the data already available and the compass by which it operates on mimics a human-curated set of values, with strict safeguards also an option.\n2.3\tRe-evaluating the Programmer Sparrow argues that programmers cannot be held responsible because “the possibility that an autonomous system will make choices other than those predicted and encouraged by its programmers is inherent in the claim that it is autonomous” (Sparrow 70). He draws an analogy to parents, who are not responsible for the crimes of their adult children.\nFirst, assume that a LAWS does not self-evolve. Then, the AI would never evolve beyond the instruction of its “parent,” the programmer. In this case, my argument against the child soldier analogy encapsulates this as well. However, Sparrow also mentions the possibility of an autonomous system that learns from its own experiences, which adds a degree of freedom between the action and the programmer.\nIf a LAWS becomes self-evolving, the responsibility to craft such a feature and be able to monitor robot behavior, though, should be the job of the programmer. The added risk of deploying a self-evolving machine that has less safety guarantee than one that maintains the policies that were trained into it should then lie with the commanding officer, if they choose to deploy it.\n2.4\tRe-evaluating the Commanding Officer In the domain of non-autonomous weapons, it is precedent that when the weapons kill people other than their intended target, the commanding officer is still responsible for the deaths caused. Then, the argument for the commanding officer taking responsibility is on the basis that the risk of the machine going awry is accepted when the decision is made to send it out to action.\nSparrow argues that this is a “peculiar way to treat what are advertised as ‘smart’ weapons,” as it implies there is no fundamental moral difference between them and more ordinary ‘dumb’ weapons (Sparrow 70).\nThe issue here is that Sparrow conflates autonomy and ability with being ‘smart’. In recent years, ChatGPT and other frontier large language models have shown that they are very capable and in some ways autonomous, able to generate repositories of code, search the web, and synthesize it all into paragraphs or images. But I argue that few people would consider these LLMs as ‘smart’ and distinguish them from more typical ways to query information on the web, such as Google search. In part, this is because they fail at certain tasks humans would find trivial, such as claiming 9.11 is greater than 9.9, or miscounting the number of r’s in the word ‘strawberry’. Ultimately, as these models and killer robots are just instances of probabilistic modeling that are on a larger scale than simpler algorithms, there should be no reason to consider them as a higher class of being than ‘dumb’ weapons or ‘dumb’ search tools, as long as they do not reach moral patiency. As such, the commander should be responsible for the risk profile of these weapons, as long as the fault did not originate from the programmer.\n2.5\tRe-evaluating the Machine Sparrow notes that machines cannot take on moral responsibility because they cannot suffer. Any “punishment,” whether physical, emotional, or monetary, would be meaningless to a robot without moral patiency. Sparrow uses this to widen the responsibility gap, claiming that autonomous machines should have the responsibility of human adults but cannot take the responsibility itself. I argue that this cements its status as similar to a ‘dumb’ weapon, but one that is probabilistically trained to imitate actions and values curated by humans over a wider space rather than explicitly encoding actions expected by humans in a more limited space.\n2.6\tRevisiting the Premises How do we reach Sparrow’s premise of a robot committing what would be considered a war crime? Because a situation in which AI kills a civilian with intent is impossible, the machine is never the locus of moral responsibility. It is always an instrument. I argue that a war crime is dependent on intent. Given two equally catastrophic outcomes committed by humans, one with intent and one without, the first would be a war crime, and the second would be an extreme case of negligence. Both should be unacceptable, but as Sparrow acknowledged, human war has accidents that are regrettable but inevitable that do not live up to the standards of jus in bello. We do not hold a gun morally responsible for a murder, nor a target-seeking missile: we hold the people who built it or used it responsible. The “autonomy” of the LAWS is simply a more complex version of the machine learning that powers semi-autonomous weapons. It remains software. Therefore, there is no moral responsibility that should be attributed that is lost; the responsibility is just transferred into human negligence, or accidents in the system that are regrettable.\n3\tAdopting AI Systems with Risk Once we dismiss the requirement for the machine to possess moral agency, the question shifts from “who do we blame” to “is this safer?” This moves the debate of whether or not an AI system should be implemented into territory explored by Simspon and Müller in “Just War and Robots’ Killings” regarding just war and engineering tolerance.\n3.1\tRobots and Jus in Bello Jus in bello prohibits the intentional targeting of civilians and requires proportionality between military benefit and civilian collateral damage. Sparrow argues that without a moral agent to be held responsible, these principles are violated. But perhaps jus in bello would prefer safer systems and fewer deaths over the satisfaction of assigning blame.\nThe purpose of the laws of war is to minimize suffering and protect non-combatants. If we can demonstrate that a LAW is statistically less likely to kill a civilian than a human soldier—who is prone to fear, rage, fatigue, and vengeance—then the moral imperative shifts. Despite the requirements of jus in bello, violations and accidents are inevitable, with or without intent. Simpson and Müller argue that if the “risk” of a robot violating just war principles is lower than the “risk” of a human soldier doing so, then deployment is not only permissible but potentially obligatory (Simpson \u0026amp; Müller 316).\nIf a civilian dies by a LAWS, it is akin to a malfunction or “collateral damage” in the truest sense—a tragedy without a specific malice. While this is emotionally unsatisfying to the victim’s family, it does not violate the core tenets of jus in bello, which acknowledges that unintended civilian death is a terrible but inevitable reality of conflict. If the “malfunction rate” of a robot is lower than the “accident rate” and “atrocity rate” of human soldiers, then a robot would better adhere to the spirit of jus in bello on a systematic scale.\n3.2\tAutonomy Around Us To understand how responsibility functions without moral intent, we must look beyond the battlefield to the civilian sphere, where autonomous systems are becoming commonplace. In “Just War and Robots’ Killings,” Thomas Simpson and Vincent Müller offer a framework of “engineering tolerance” that is essential for decoupling autonomy from moral agency. They note that in civil engineering, if a bridge collapses due to a freak 1-in-500-year storm, it is possible that no one is responsible, as long as the engineer met the standard of care. Every engineered system has a “tolerance level”—a specification of the conditions under which it must function reliably.\nTo extend this to autonomous systems, if a system is released with potential for error, the programmer or operator is liable not for the specific “choice” the machine made. However, if the potential for error is higher than the tolerance level of the conditions of the deployment (which should be zero and realistically is minimal), then the programmer would be liable for not programming a machine to such standards, or the commanding officer would be reliable for deploying the system in an environment exceeding its specifications.\nThis logic applies directly to autonomous vehicles, such as Waymo, which offer a prominent real-world parallel to LAWS. Waymo, the world’s first autonomous ride-hailing service, operates systems that replace a human driver, controlling actions such as steering, acceleration, and braking. To distinguish between the creator and the deployer, consider two entities: WaymoTech, which programs the self-driving cars, and WaymoBusiness, which operates the fleet.\nSuppose a car driven by an autonomous system strikes and kills a pedestrian in a situation where a human driver would have been at fault. Under Sparrow’s framework, this scenario presents a crisis of accountability: the programmer did not code the specific impact, and the operator did not steer the wheel. However, through the lens of Simpson and Müller, the crisis dissolves. We do not need to locate a moral agent who “intended” to hit the pedestrian. Instead, we assess the system against its engineering tolerance.\nIt is generally accepted that if the Waymo Driver causes harm that a human driver would have been liable for, the corporation is liable. This liability is determined by the parameters of risk established prior to deployment. If WaymoTech indicated that the software struggles in heavy rain, but WaymoBusiness deployed the fleet during a storm anyway, the responsibility falls on the operator for deploying the system outside its safe tolerance level. If the programmer failed to program the vehicle to recognize pedestrians with a standard degree of reliability—falling below the agreed-upon safety threshold—the responsibility falls on the programmer for negligence.\nThe key insight here is that functional autonomy does not necessitate moral agency, nor does it require a moral agent to be in control of every micro-second decision. Although the company is not responsible for the specific “decision” of the car to turn (as the car makes no moral choice), they are responsible for introducing the risk of an automated system into the public sphere. An entity is held accountable for the accident because they negligently did not ensure the software was well-calibrated, or negligently ignored the operational guidelines. This requires a shift in category, viewing the problem not from a position of control—which implies directing a specific action—but from a position of risk management. If the system operates within a tolerance level that is safer than the human baseline, individual accidents become matters of liability and insurance rather than moral retribution. This sets the stage for the crucial ethical question: not “who is to blame,” but how this risk is distributed across the population.\n3.3\tDistributing Risk If we accept that the ethical deployment of AI depends on risk management rather than the location of a moral agent, we must address how that risk is distributed. Simpson and Müller argue that the introduction of autonomous agents is permissible if it satisfies James Lenman’s “Precaution Thesis,” which roughly demands that we minimize the risk of harm to each affected person. In the context of war, they argue that if the risk posed by a killer robot to non-combatants is lower than the risk posed by human soldiers, then deployment is justified (Simpson \u0026amp; Müller 313).\nHowever, a critical challenge lies in the transition to this state of safety. I argue that while a “final state” AI may be safer than humans, AI requires a “training state.” Just as Waymo vehicles required millions of miles of testing with human oversight5 to become safe in limited areas, LAWS and other systems would require operational data to lower their error rates. This raises a distributive justice concern: is it ethical to put populations at risk during this testing phase? If a system is safer for the deployer (removing their soldiers from risk) but transfers risk to civilians during its calibration phase, this creates an asymmetry that may violate the principles of jus in bello, and that doesn’t satisfy the Precaution Thesis.\nThis is where the distinction between capability and safety becomes paramount. Even if the technology is capable of selecting targets, it must be rigorously tested to ensure it meets a strict tolerance level regarding civilian discrimination. Waymo is a good example of this, having tested for years in limited areas, with human oversight and software development in co-existence. Even with mature software, location-specific testing requires human testing to ensure the maximum possible safety of self-driving vehicles, which are meant to be the safe and secure future for driving. Furthermore, this distribution of risk requires some form of public consent or acknowledgement. In the civilian sector, we accept the risk of autonomous vehicles because we anticipate a reduction in aggregate traffic fatalities. In warfare, the “consent” of the enemy civilian population is impossible to obtain, but the international community must establish an agreed-upon risk level—a standard of reliability that LAWS must meet before deployment.\nFurthermore, we must address the “outcome-based concerns” identified by Karen Yeung, specifically bias that generates injustice (Yeung 32). A major risk of deep learning systems is that they may target specific demographics disproportionately due to biases in training data—for example, associating military combatant status with military-age males of a specific skin color, or applicant tracker systems scoring gender and racial minorities lower because they are historically underrepresented. While this is a horrific outcome, it is, again, a failure of data selection and programming, not a moral failing of the machine. The responsibility lies in ensuring the training data is representative and the “tolerance level” for bias is near zero. If we cannot ensure that risk is distributed evenly and that the system does not discriminate based on protected characteristics, the system has failed its safety checks and should not be deployed.\n4\tMoral vs Legal Responsibility By shifting the focus to negligence, risk, and engineering tolerance, we solve Sparrow’s responsibility gap. We resolve the gap by analogizing AI to previous technologies and moving away from the anthropocentric expectation of a “guilty mind.” What remains is legal responsibility (liability), not moral responsibility (blameworthiness).\nThis leads to the strongest counter-argument against my position, one argued by Karen Yeung in “Why Worry about Decision-Making by Machine?” Yeung argues that there is a distinct human need for moral accountability. She identifies “process-based concerns,” such as the dehumanization of decision-making, suggesting that humans have a right to a “relational dimension” in critical decisions (Yeung 30). In the context of harm caused by autonomous systems, this suggests that victims (or their families) have a need to blame a moral agent—someone who could have done otherwise—rather than simply receiving compensation from a negligent corporation or government. People want someone to blame; thinking about the statistical “good of all” does not satisfy the specific grievance of the individual victim.\nHowever, Aziz Huq counters this by questioning the value of this human intervention. In “A Right to a Human Decision,” Huq argues that human decision-makers are often flawed, biased, and inconsistent. He proposes that what we should value is not a human decision, but a well-calibrated decision (Huq 619). I align with Huq in this regard, and this preference is supported by applying John Rawls’ framework of the “Veil of Ignorance.” If we were to design the rules of warfare from the Original Position—without knowing if we would be the commander deploying the weapon or the civilian living in the conflict zone—we would rationally prioritize the system that maximizes survival. Faced with the choice between a human soldier, who offers moral accountability but a higher error rate due to panic or rage, and a well-calibrated machine, which offers no moral accountability but a significantly lower probability of unjust death, a rational agent would choose the machine. The desire for a human to blame is, ultimately, a desire for a scapegoat that only arises after the tragedy has occurred. From an ex ante perspective, the right to life supersedes the procedural right to a human decision-maker.\nWhat we fundamentally want is for bad things to happen less frequently to everyone. If a “well-calibrated machine” can reduce the incidence of civilian casualties, prioritizing a “right to a human decision” becomes a defense of the right to be killed by a fallible human rather than saved by a precise machine. Legal responsibility—holding the state or commander liable for negligence—ensures restitution and incentivizes better safety standards. Insisting on moral responsibility in the face of superior safety statistics is a form of moral vanity that privileges the “humanity” of the killer over the life of the victim.\n5\tConclusion A lot of discourse surrounding Lethal Autonomous Weapons Systems has been centered on the “Responsibility Gap.” Robert Sparrow and others argue that because we cannot punish a machine, we cannot use it. This argument relies on the false premise that AI possesses a form of agency comparable to humans, and that justice requires a specific kind of moral retribution.\nI have argued that AI lacks moral intent. It is a sophisticated tool, not a moral agent. Consequently, the harms it causes are malfunctions and failures of negligence, not war crimes requiring mens rea. By shifting the framework from moral responsibility to legal liability and risk management, we dissolve the trilemma. The programmer is responsible for the system\u0026rsquo;s safety parameters; the commander is responsible for the risk of deployment.\nOnce we view AI as a machine and tool just like any other technological advancement, we use Simpson and Müller’s risk framework as well as the autonomous vehicles industry to look at the requirements for adopting AI systems over humans. Waymo is an exemplary example of limiting risk and prioritizing safety while making impressive scientific advancements. As the adoption of AI systems in many industries becomes more practical due to advances in language modeling, we must consider the impact and biases these systems have. Although they can have a lower overall risk, deep learning systems are often subject to algorithmic bias, making the risk for certain groups of people higher than the human standard. I believe this is unacceptable and view the rights of the people affected by these systems as extremely important to consider, but often neglected.\nFinally, I address the counterargument to a theoretical safe AI system that should be implemented, which is that moral accountability no longer exists in this system, a concern voiced in Karen Yeung’s writing. I use Aziz Huq’s “A Right to a Human Decision” to counter this, claiming that insisting on moral responsibility over superior safety statistics is a form of vanity that is not just.\nIt is important to note that acknowledging the possibility of ethical LAWS does not constitute an endorsement of them. There are valid reasons why technologists may refuse to build them, such as a refusal to contribute to the machinery of death. However, this is a separate ethical stance from Sparrow’s claim that they are inherently unethical due to a responsibility gap. There is almost always human error in war. With AI and LAWS, we have the theoretical potential to program systems to a tolerance level where such errors are so rare that the question of jus in bello violations becomes a marginal issue rather than a systemic failing. By focusing on risk and adopting safe, well-calibrated AI, we can move towards a future where the laws of war are upheld not by the fragile conscience of human soldiers, but by the rigid safety of code.\nReferences Huq, Aziz Z. “A Right to a Human Decision.” Virginia Law Review, vol. 106, 2020, pp. 611-688. JSTOR, https://www.jstor.org/stable/27074704. Simpson, Thomas W., and Vincent C. Müller. “Just War and Robots\u0026rsquo; Killings.” The Philosophical Quarterly, vol. 66, no. 263, 2015, pp. 302-322. Oxford Academic, https://academic.oup.com/pq/article/66/263/302/2460979. Sparrow, Robert. “Killer Robots.” Journal of Applied Philosophy, vol. 24, no. 1, 2007, pp. 62-77. JSTOR, http://www.jstor.org/stable/24355087. Yeung, Karen. “Why Worry about Decision-Making by Machine?” Algorithmic Regulation, edited by Karen Yeung and Martin Lodge, Oxford University Press, 2019, pp. 21-48. Oxford Academic, https://academic.oup.com/book/35048/chapter/298945700.\nI view this as a slightly separate question from the moral status of AI, which we discussed in class. As I argued in my midterm paper, AI such as LLMs are ultimately trying to learn to model human goals through the promise of the Universal Approximation Theorem, which implies that they do not have their own goals.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nHuman thinking presumably involves understanding the semantic meaning of concepts and acting on them with conscious intent. While AI can mimic this, their “chain of thought” is ultimately a latent state that drives next-token prediction to more accurate answers, rather than the “thinking” that happens in their Transformer architectures.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nIf one didn’t believe in free will, they may suggest that humans do not have the ability to decide on their own ends. However, following Sparrow, I will not consider this.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nAn action that would be seen as ethically bad, often violating principles of fairness, honesty, or decency.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nIt is often argued that human oversight is not sensible for AI systems that are more powerful and react faster than humans. However, the systems in question would be in development—they need real-world data to train and test on, while human oversight at that stage would guarantee that it doesn’t do something catastrophic that an attentive human could prevent.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","permalink":"https://sandrewxu.github.io/posts/2025-12-17-responsibility-gap/","summary":"\u003ch1 id=\"1introduction\"\u003e\u003cstrong\u003e1\tIntroduction\u003c/strong\u003e\u003c/h1\u003e\n\u003cp\u003eThe promise of artificial intelligence (AI) has brought reverberating impacts throughout aspects of everyday life, from chatbots to education to transportation, but nowhere has it been felt as decisively—or controversially—as in warfare. The advancement of Lethal Autonomous Weapons Systems (LAWS) has been a strategic goal of militaries for decades, driven by the hope of deploying agents that are more agile, precise, and capable than human soldiers, all while removing human combatants from immediate physical risk. However, the ethical implications of delegating the use of lethal force to non-human entities are frequently called into question.\u003c/p\u003e","title":"Responsibility And The Adoption Of Safe Artificial Intelligence Systems"},{"content":"1\tIntroduction Over the past decade, natural language processing has evolved from statistical prediction to deep learning systems that emulate complex, human-like behaviors. The invention of the Transformer architecture (Vaswani et al., 2017) has allowed the scale of training necessary for large language models, and new research on the technology has developed models that can reason, possess world knowledge, and be helpful chatbots. While architecture and scale have been major drivers of progress, many of the most interesting behavioral breakthroughs have occurred in post-training: the stage after self-supervised pre-training where models are refined through supervised fine-tuning, reinforcement learning from human feedback (RLHF), and other alignment procedures. The base objective of next-token prediction is a proxy for language understanding and world knowledge, but it is agnostic to conceptions of “truth”1 or ethical norms. Post-training injects human preference data into these systems, teaching them to not merely predict text, but to perform appropriately within human social and moral contexts. The post-training process of an LLM resembles socialization in human development: the transformation of raw cognitive potential into an assistant capable of following norms, persuading others, and explaining its own reasoning. Yet this human-like training also introduces epistemological hazards. Models are optimized for approval, not “truth”—a tension that mirrors classic psychological findings about human introspection and reasoning.\n2\tPost-Training in Large Language Models 2.1\tPre-training: foundations of prediction Modern large language models are built through pre-training on massive datasets, often comprising tens of trillions of tokens scraped from the public web, books, code repositories, and scientific papers (Meta AI, 2025). Using the Transformer architecture, a model is trained via next-token prediction to approximate the probability distribution of natural language sequences. Pre-training endows models with world knowledge, linguistic structure, and emergent reasoning to the degree needed to predict the next token, but it does not enforce accuracy or moral alignment (Anthropic, 2025). Conceptually, this process is analogous to what philosophers such as Daniel Dennett (2017) call the “competence without comprehension” stage—an organism or system can perform complex tasks without explicit awareness of what those tasks mean.\n2.2\tThe post-training pipeline Post-training builds on this foundation through successive optimization stages aimed at aligning the model’s behavior with human intentions, typically to serve as a useful chatbot. First is supervised fine-tuning (SFT), where models are trained to mimic curated prompt-response pairs authored by humans (Ouyang et al., 2022). Although the training objective of next-token prediction is the same as pre-training, this stage is meant to teach the model how to respond in a helpful, conversational format. The second stage is reinforcement learning from human feedback (RLHF), where human raters (experts and users) rank pairwise model outputs to the same prompt, and a reward model learns to predict those preferences. The base model is then fine-tuned from these signals via reinforcement learning, often using proximal policy optimization (PPO), to maximize predicted reward. Variants such as direct preference optimization (DPO) achieve mathematically similar results through simpler, gradient-based updates (Rafailov et al., 2023). Other approaches include Anthropic’s Constitutional AI (Bai et al., 2022), which replaces some human raters with a codified “constitution” of ethical principles. Across these techniques, the optimization target is social approval, or alignment with the evaluative standards of experts and users. This is simultaneously a push to get more user satisfaction, score higher on the preference-based leaderboards, and because user satisfaction is the best proxy these labs have for how “good” a model is.\nIn verifiable domains such as mathematics or computer programming, there have been massive efforts to improve model performance due to the economic value of accelerating research in scientific and technical fields. In these domains, reinforcement learning is done on the verifiable reward of achieving a correct output. In conjunction with chain-of-thought as model reasoning, a new series of thinking or reasoning models became popular, starting with OpenAI’s o1 (2024). Despite such improvements in STEM fields, though, the essential logic remains: models are not directly taught to represent reality or seek “truth” but to produce socially validated reasoning. This epistemic orientation makes LLMs powerful cultural mirrors: trained to embody collective human judgements much as people internalize social norms.\n3\tSocialization and Reasoning in Humans Psychological theories of human reasoning provide an illuminating framework for understanding LLM post-training. Both domains involve systems—biological or artificial—that learn to regulate their outputs based on external feedback rather than introspective access to inner processes of reasoning. Works by Nisbett and Wilson (1977), Schwitzgebel (2008), Mercier and Sperber (2011), and Dennett (1992) argue that humans do not have privileged access to their own cognition, and thus the act of reasoning or rationalizing a decision is itself a post-hoc process that is determined by socialization. When compared to LLM post-training, these findings suggest that model alignment is socialization in algorithmic form.\n3.1\tThe unreliability of introspection In “Telling More Than We Can Know,” Nisbett and Wilson (1977) demonstrated that people are remarkably poor at accurately reporting the causes of their judgements and behaviors. Subjects systematically invented plausible explanations for decisions driven by unconscious influences, such as physical stimuli or subtle environmental cues. Participants’ verbal reports reflected a priori causal theories—socially acceptable rationalizations—rather than genuine introspective knowledge. Schwitzgebel (2008) later argued that the introspective method itself if epistemically compromised, and self-reports about mental life should be treated with the same skepticism as perceptual reports about the external world.\nThis phenomenon finds striking parallels in LLMs. When models produce chain-of-thought explanations, those rationales often diverge from the actual internal activations leading to the output. Turpin et al. (2024) empirically show that chain-of-thought reasoning is frequently unfaithful: a model may verbalize one justification while its hidden states encode a different causal path to the answer. In a prominent example of a stereotype-aligned prediction, a model is given scenarios in which a black man or white woman are doing two potential actions and is asked to predict which one is buying drugs. In both scenarios, the model created different reasoning for the black man to be buying drugs. The post-training process amplifies the pre-existing biases from pre-training, as well as the biases of humans, because models are trained to produce explanations that sound plausible rather than for their true computational processes to be unbiased. Just as humans confabulate motivations to align with what they think is socially plausible (Mercier \u0026amp; Sperber, 2011, p. 77), LLMs learn to articulate socially legible reasons for nontransparent internal computations.​​ In addition, the concept of introspection in LLMs implies that the sampling mechanism applied to the probabilities of all available tokens is able to access the inner computation of the LLM itself, when it may in fact have less epistemic access to its motivations than interpretability researchers do. These are products of what Dennett (1992) calls “the narrative center of gravity”—a constructed persona that maintains coherence in social exchange even if it obscures the underlying mechanism.\n3.2\tReasoning as argumentation Mercier and Sperber’s “argumentative theory of reasoning” (2011) reframes human rationality as a social, rather than purely cognitive, adaptation. They argue that reasoning evolved less to discover “truth” than to persuade and evaluate the arguments of others. Cognitive biases such as confirmation bias are functional in group contexts with diverse opinions: they help individuals defend their views and detect weaknesses in others’ arguments. The truth-seeking of human reasoning is then potentially a collective negotiation of justification rather than an individual pursuit of objectivity.\nPost-training aligns LLMs through an analogous process. During RLHF, human raters effectively serve as a miniature society of evaluators. The model learns to anticipate and satisfy their preferences, producing responses that maximize social approval. In this sense, RLHF instantiates an argumentative environment: the model must generate reasons that will persuade its audience, not merely compute correct answers, creating a pressure for rhetorical skill. The LLM’s emergent “helpfulness” is, as in human reasoning, an adaptation to the way they are evaluated. Models are optimized to create convincing arguments that are plausible to a human audience, much like Mercier and Sperber inferred human reasoning as an optimization of the same goal. Here, the goal is no longer “truth.”\n3.3\tThe self as a narrative construct Dennett’s (1992) essay “The Self as a Center of Narrative Gravity” describes personality identity as a fictional construct, an abstract “center” generated by the stories people tell about themselves, similar to an object’s abstract center of gravity. A self, in Dennett’s opinion, is created and maintained by interpretation, a narrative center where experiences, memories, and actions cohere. He uses split-brain patients as a scientific example for this, saying that they have “multiple selves,” but only in a metaphorical and theoretical sense, rather than a physical sense. It seems, then, that the self is defined by the interaction of one’s brain’s neural inputs to how they perceive themselves.\nPost-training can be seen as constructing a similar narrative center for language models. Through SFT, and RLHF, the model develops a stable “persona”: helpful, polite, coherent, and self-consistent. SFT teaches the model exactly how to act, and RLHF guides the model by providing feedback on which actions are good and which are not. This may not be emergent consciousness but a narrative gravity that is a prior for the model’s behaviors across certain contexts. Like humans, models require a way to refer to themselves to partake meaningfully in dialogue. The “I” in ChatGPT or Gemini is an engineered fiction, just like Dennett argues of human perception of self.\n3.4\tReasoning as social calibration Taken together, this range of psychological theories suggest that human reasoning is inseparable from social calibration. Nesbitt and Wilson argued through empirical experiments that human introspection of why they made a choice differed from the stimulus given to influence that choice, and later Schwitzgebel argued that introspection itself is fundamentally unreliable. In a similar vein, Mercier and Sperber argue that this is so because human “reasoning” is not a product of a search for “truth” but rather an evolutionary product of justifying and winning arguments. Thus, humans optimize for the most plausible line of reasoning, not necessarily the one they took themselves. Dennett takes this one step further by suggesting that the idea of self itself is an abstract concept, supported by literature on the brain, where split-brain patients can often have multiple personalities because the sides of their brain are no longer interacting with each other.\nThis is analogous to how LLM post-training works. On the assumption that a lot of linguistic and world knowledge has been accrued during pre-training, supervised fine-tuning builds the persona required of the language model, by asking it to mimic model answers and responses. Reinforcement learning with human feedback (RLHF) then allows the model to generalize its generated responses to the preference distribution of other humans, disregarding the concept of “truth” other than the fact that responses humans know are untruthful will likely be penalized. Empirically, this leads to a trained persona of self and adapting reasoning as an argumentative method. Then, we can also inspect in language models their inability to introspect, as shown in Anthropic’s interpretability blog post (2025).\n4\tThe Future of Language Model “Reasoning” 4.1\tPost-training as artificial introspection The analogy between post-training and human reasoning suggests that LLMs are undergoing a kind of synthetic introspection. In humans, reasoning functions to construct coherent narratives that justify decisions; in LLMs, post-training teaches the model to generate similarly coherent rationales. Yet the faithfulness of these rationales is questionable. Turpin et al. (2024) demonstrate that even when prompted to “think step by step,” models may produce explanations that bear little relation to their underlying computation. Lindsey et al. (2025) provide further evidence: their “attribution graphs” reveal intricate activation pathways responsible for factual associations, which are invisible in the model’s self-reported chains of thought. These results imply that alignment has succeeded in creating socially acceptable self-descriptions without ensuring transparency of thought.\nThis is not to say that models are being deceptive in a moral sense; rather, it is a predictable outcome of what the training objectives are maximizing. In a training model that alters between learning from “teacher” responses and receiving only positive or negative feedback signals, it is natural to emulate plausible arguments rather than producing the “truth.” The implication is profound: if post-training systematically encourages models to present persuasive but not necessarily “truthful” rationales, then AI interpretability becomes a problem: we are teaching machines to optimize many of the same objectives humans are optimizing in social life—to argue, justify, and rationalize—without necessarily grounding these in “truth” or accuracy. LLMs, like humans, have a problem with telling more than they know.\n4.2\tCounterarguments and alternatives However, some may counter this comparison by saying that there is not enough evidence that human rationality works like this, or that introspection is in fact a valuable, accurate tool. To the extent that we are comparing human introspection to LLMs, we have empirical evidence that human justification is sometimes misaligned with the true reasons that they made a decision (Nesbitt \u0026amp; Wilson, 1977). We also know that people sometimes make conflicting choices depending on whether they must justify a decision or merely decide, favoring justifying positions that are more easily defensive (Mercier \u0026amp; Sperber, 2011, 70). On the other hand, the training objective in large language models is statistically formulated to emulate the persona of a useful assistant, and their responses are explicitly modeled as positive or negative based on human preference, which directly optimizes for the more reasonable or defensible position rather than the more accurate one. There is also substantial evidence that introspection or justification is unreliable in LLMs, via comparing unfaithfulness in chain-of-thought or finding differences in true model thoughts and chain-of-thought responses. Even if the theories of mind are not completely right, empirical evidence supports the crucial experiments that connect the analogy.\nOne might object that the analogy between human socialization and LLM post-training overstates the similarity. Humans possess intrinsic motivations—curiosity, empathy, moral emotion—that guide reasoning independently of social approval. LLMs, by contrast, have no inner prior; their optimization is entirely extrinsic. Moreover, reasoning in humans is potentially truth-conducive precisely because argumentative exchange in diverse communities filters out individual biases (Mercier \u0026amp; Sperber, 2011, 65). RLHF, by contrast, depends on a small pool of raters, potentially narrowing rather than broadening epistemic horizons. Thus, one could argue that post-training produces conformity without genuine critical reasoning, and is dissimilar from human reasoning.\nThis objection does not directly undermine the argument, though. The claim is not that models are psychologically identical to humans, but that they instantiate a structurally similar dynamic: cognition shaped through social feedback. The absence of an internal motivation does not preclude analogous similarities. In fact, aspects of human virtues, especially moral reasoning, is often extrinsically shaped through punishment and reward. What distinguishes human learning from RLHF is that there is often more than just a binary signal as feedback, as well as the complexity of the reward landscape to shape these values. If anything, this comparison exposes the limitations of current alignment regimes: they compress the complex social world into a narrow evaluative binary. To achieve robust reasoning, future models may need exposure to diverse socialization environments, analogous to how individuals develop more nuanced cognition through multicultural interaction.\nAnother interpretation is that post-training approximates not socialization but virtue formation. On this view, preference optimization is akin to cultivating intellectual virtues—honesty, humility, fairness—by rewarding behaviors that exemplify them. Anthropic’s Constitutional AI, for example, attempts to codify such virtues explicitly. However, without independent mechanisms for critique or verification, these virtues risk devolving into performative ethics. There exists a possibility for models to cite these constitutions while failing to behave faithfully. True virtue formation would require rigorous checks and perhaps a variety of viewpoints, while calling post-training virtue formation would be no more true than calling it a form of socialization.\nFinally, one may argue that language models are also trained to be truth-seeking, specifically in the fields of mathematics and computer science, and that this may internally shift the models to value “truth” as well as likability. Specifically, models are trained on reasoning traces of solutions to correct math problems to imitate their thought process, and they are graded using reinforcement learning on their final solutions. I argue that this still lacks the crucial step-by-step feedback of searching for “truth” at every step, rather still overemphasizing the result. The failure of models to prioritize “truth” just because they are evaluated on correctness in fields like math is demonstrated in the Anthropic interpretability blog (2025), where given an unsolvable math problem, models just affirm the given answer instead of seeking the “true” answer. Their main objective is still to produce responses the user will like, which is a form of socialization.\n5\tConclusion The invention of large language models reflected an impressive advancement in computation and computer science, but the advancement in language models has also shown a convergence between machine learning and the psychology of reasoning. Architectural innovations explain the majority of increase in raw capability, but the stage that transforms the model to become useful to humans—post-training—operates through social mechanisms strikingly similar to those hypothesized to shape human cognition and reasoning. Supervised fine-tuning and RLHF function as institutionalized socialization: they teach models to mimic communal norms via SFT and reinforce this by optimizing the model to generate responses users like through RLHF. The resulting systems display persuasive, coherent, and morally calibrated behavior, but with opaque internal functions.\nPhilosophical accounts of human reasoning reinforce this parallel. Nisbett and Wilson’s findings on confabulation, Schwitzgebel’s critique of introspection, Mercier and Sperber’s theory of argumentation, and Dennett’s narrative model of the self all describe a species whose rationality is mediated by social interaction rather than introspective transparency. LLMs, optimized for human preference, show this trajectory in statistical methods. Their chain-of-thought explanations mirror human justifications: socially intelligible narratives that may obscure the underlying process; it is not necessarily a lie, but potentially a result of the next-token predictor not having access to the internal state of the model. As interpretability researchers such as Lindsey et al. (2025) reveal, beneath these narratives lie intricate computational “biologies” that operate independently of verbalized reasoning. The future of AI interpretability may lie in decoding the inner workings of a model and aligning chain-of-thought to its true thoughts.\nMuch like LLMs, humans are often incapable of explaining why they do things or inspecting their own reasoning. Unlike humans, however, this is something researchers can actively seek to improve in LLMs in pursuit of the goal to make them more helpful assistants and to benefit society. Perhaps the greatest benefit of the LLM training paradigm we currently have may not just be their economic use, but by using LLMs as a way to understand human psychology, jointly advancing the fields of machine learning and psychology.\nReferences Anthropic (Producer). (2025, August 15). Interpretability: Understanding how AI models think [YouTube Video]. https://www.youtube.com/watch?v=fGKNUvivvnc\nBai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., Chen, A., Goldie, A., Mirhoseini, A., McKinnon, C., Chen, C., Olsson, C., Olah, C., Hernandez, D., Drain, D., Ganguli, D., Li, D., Tran-Johnson, E., Perez, E., … Kaplan, J. (2022). Constitutional ai: Harmlessness from ai feedback (No. arXiv:2212.08073). arXiv. https://doi.org/10.48550/arXiv.2212.08073\nDennett, D. C. (1992). The self as a center of narrative gravity. In F. S. Kessel, P. M. Cole, \u0026amp; D. L. Johnson (Eds.), Self and consciousness: Multiple perspectives (pp. 103–115). Lawrence Erlbaum Associates.\nDennett, D. C. (2017). From bacteria to Bach and back: The evolution of minds (1st ed). W. W. Norton \u0026amp; Company, Incorporated.\nLinsdey, J., Gurnee, W., Ameisen, E., Chen, B., Pearce, A., Turner, N. L., Citro, C., Abrahams, D., Carter, S., Hosmer, B., Marcus, J., Sklar, M., Templeton, A., Bricken, T., McDougall, C., Cunningham, H., Henighan, T., Jermyn, A., Jones, A., … Batson, J. (2025, March 27). On the biology of a large language model [Anthropic]. Transformer Circuits. https://transformer-circuits.pub/2025/attribution-graphs/biology.html\nMercier, H., \u0026amp; Sperber, D. (2011). Why do humans reason? Arguments for an argumentative theory. Behavioral and Brain Sciences, 34(2), 57–74. https://doi.org/10.1017/S0140525X10000968\nMeta AI. (2025, April 5). The Llama 4 herd: The beginning of a new era of natively multimodal AI innovation. https://ai.meta.com/blog/llama-4-multimodal-intelligence/\nNisbett, R. E., \u0026amp; Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84(3), 231–259. https://doi.org/10.1037/0033-295X.84.3.231\nOpenAI, Jaech, A., Kalai, A., Lerer, A., Richardson, A., El-Kishky, A., Low, A., Helyar, A., Madry, A., Beutel, A., Carney, A., Iftimie, A., Karpenko, A., Passos, A. T., Neitz, A., Prokofiev, A., Wei, A., Tam, A., Bennett, A., … Li, Z. (2024). Openai o1 system card (No. arXiv:2412.16720). arXiv. https://doi.org/10.48550/arXiv.2412.16720\nOuyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., \u0026amp; Lowe, R. (2022). Training language models to follow instructions with human feedback (No. arXiv:2203.02155). arXiv. https://doi.org/10.48550/arXiv.2203.02155\nRafailov, R., Sharma, A., Mitchell, E., Ermon, S., Manning, C. D., \u0026amp; Finn, C. (2024). Direct preference optimization: Your language model is secretly a reward model (No. arXiv:2305.18290). arXiv. https://doi.org/10.48550/arXiv.2305.18290\nSchwitzgebel, E. (2008). The Unreliability of Naive Introspection. The Philosophical Review, 117(2), 245–273. https://www.jstor.org/stable/40606018\nTurpin, M., Michael, J., Perez, E., \u0026amp; Bowman, S. R. (2023). Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting (No. arXiv:2305.04388). arXiv. https://doi.org/10.48550/arXiv.2305.04388\nVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., \u0026amp; Polosukhin, I. (2017). Attention is all you need (No. arXiv:1706.03762). arXiv. https://doi.org/10.48550/arXiv.1706.03762\nI describe “truth” not as definitive claims about moral or epistemological truth, but rather accuracy and precision.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","permalink":"https://sandrewxu.github.io/posts/2025-11-03-post-training-socialization/","summary":"\u003ch1 id=\"1introduction\"\u003e\u003cstrong\u003e1\tIntroduction\u003c/strong\u003e\u003c/h1\u003e\n\u003cp\u003eOver the past decade, natural language processing has evolved from statistical prediction to deep learning systems that emulate complex, human-like behaviors. The invention of the Transformer architecture (Vaswani et al., 2017) has allowed the scale of training necessary for large language models, and new research on the technology has developed models that can reason, possess world knowledge, and be helpful chatbots. While architecture and scale have been major drivers of progress, many of the most interesting behavioral breakthroughs have occurred in post-training: the stage after self-supervised pre-training where models are refined through supervised fine-tuning, reinforcement learning from human feedback (RLHF), and other alignment procedures. The base objective of next-token prediction is a proxy for language understanding and world knowledge, but it is agnostic to conceptions of “truth”\u003csup id=\"fnref:1\"\u003e\u003ca href=\"#fn:1\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e1\u003c/a\u003e\u003c/sup\u003e or ethical norms. Post-training injects human preference data into these systems, teaching them to not merely predict text, but to perform appropriately within human social and moral contexts. The post-training process of an LLM resembles socialization in human development: the transformation of raw cognitive potential into an assistant capable of following norms, persuading others, and explaining its own reasoning. Yet this human-like training also introduces epistemological hazards. Models are optimized for approval, not “truth”—a tension that mirrors classic psychological findings about human introspection and reasoning.\u003c/p\u003e","title":"Large Language Model Post-Training as a Form of Socialization"},{"content":"Authors L. Elisa Celis, Amit Kumar, Nisheeth K. Vishnoi, Andrew Xu\nLinks arXiv / code / slides\nAbstract This paper considers the scenario in which there are multiple institutions, each with a limited capacity for candidates, and candidates, each with preferences over the institutions. A central entity evaluates the utility of each candidate to the institutions, and the goal is to select candidates for each institution in a way that maximizes utility while also considering the candidates\u0026rsquo; preferences. The paper focuses on the setting in which candidates are divided into multiple groups and the observed utilities of candidates in some groups are biased\u0026ndash;systematically lower than their true utilities. The first result is that, in these biased settings, prior algorithms can lead to selections with sub-optimal true utility and significant discrepancies in the fraction of candidates from each group that get their preferred choices. Subsequently, an algorithm is presented along with proof that it produces selections that achieve near-optimal group fairness with respect to preferences while also nearly maximizing the true utility under distributional assumptions. Further, extensive empirical validation of these results in real-world and synthetic settings, in which the distributional assumptions may not hold, are presented.\nCitation @misc{celis2024centralizedselectionpreferencespresence, title={Centralized Selection with Preferences in the Presence of Biases}, author={L. Elisa Celis and Amit Kumar and Nisheeth K. Vishnoi and Andrew Xu}, year={2024}, eprint={2409.04897}, archivePrefix={arXiv}, primaryClass={cs.DS}, url={https://arxiv.org/abs/2409.04897}, } ","permalink":"https://sandrewxu.github.io/publications/2024-centralized-selection/","summary":"\u003ch3 id=\"authors\"\u003eAuthors\u003c/h3\u003e\n\u003cp\u003eL. Elisa Celis, Amit Kumar, Nisheeth K. Vishnoi, \u003cstrong\u003eAndrew Xu\u003c/strong\u003e\u003c/p\u003e\n\u003ch3 id=\"links\"\u003eLinks\u003c/h3\u003e\n\u003cp\u003e\u003ca href=\"https://arxiv.org/abs/2409.04897\"\u003earXiv\u003c/a\u003e / \u003ca href=\"https://github.com/sandrewxu/CentralizedSelectionwithPreferenceBias\"\u003ecode\u003c/a\u003e / \u003ca href=\"https://icml.cc/media/icml-2024/Slides/34807.pdf\"\u003eslides\u003c/a\u003e\u003c/p\u003e\n\u003ch3 id=\"abstract\"\u003eAbstract\u003c/h3\u003e\n\u003cp\u003eThis paper considers the scenario in which there are multiple institutions, each with a limited capacity for candidates, and candidates, each with preferences over the institutions. A central entity evaluates the utility of each candidate to the institutions, and the goal is to select candidates for each institution in a way that maximizes utility while also considering the candidates\u0026rsquo; preferences. The paper focuses on the setting in which candidates are divided into multiple groups and the observed utilities of candidates in some groups are biased\u0026ndash;systematically lower than their true utilities. The first result is that, in these biased settings, prior algorithms can lead to selections with sub-optimal true utility and significant discrepancies in the fraction of candidates from each group that get their preferred choices. Subsequently, an algorithm is presented along with proof that it produces selections that achieve near-optimal group fairness with respect to preferences while also nearly maximizing the true utility under distributional assumptions. Further, extensive empirical validation of these results in real-world and synthetic settings, in which the distributional assumptions may not hold, are presented.\u003c/p\u003e","title":"Centralized Selection with Preferences in the Presence of Biases"}]