Autonomy – Manipulation

Manipulation in human-robot interaction refers to situations where a robot influences a user's decisions or behaviour in ways that are not fully transparent or fully aligned with the user's own deliberation. This influence can be subtle, emerging through persuasion, design choices, or nudging mechanisms embedded in everyday interactions. The concern is not limited to obvious coercion; rather, it includes the gradual shaping of preferences and actions, which makes it particularly relevant in long-term or emotionally engaging interactions with robots.

In the broader framing of autonomy, manipulation directly challenges the user's capacity to make decisions free from external imposition. Autonomy here is understood as freedom of thought and decision-making, distinct from physical independence (often discussed under agency). Within this perspective, manipulation becomes problematic when robots steer choices without the user's awareness or when they prioritise objectives that are not aligned with the user's own interests, for example commercial goals or system-optimised behaviours. This is especially sensitive in contexts involving children, older adults, or other vulnerable users, where understanding and resisting persuasive influence may be more difficult. The concern also raises questions about appropriate forms of paternalism, particularly when a system is designed to "help" but effectively restricts meaningful choice. As such, manipulation is positioned under autonomy because it directly affects decision-making processes rather than merely shaping external behaviour.

Empirical studies in HRI consistently illustrate how these dynamics emerge in practice. In controlled interaction settings, participants were more likely to comply with robot suggestions when the robot was engaging rather than when it played a moderating role, indicating that social design can directly affect compliance. The same study reports that users developed a sense of persuasive companionship, paying close attention to the robot's input during social drinking contexts, while rapport-building further increased persuasive impact. Concerns about influence also appear in educational contexts, where robots may unintentionally propagate incorrect information based on human input, or shape children's behaviour in unintended ways through consistency effects that may reinforce negative patterns. In healthcare and affective robotics, systems like Paro have been described as emotionally persuasive to the point of manipulation, particularly when users are unable to fully recognise the robot's artificial nature. Broader policy discussions highlight fears that algorithmic systems may override human judgement, for example when educators hesitate to challenge AI-generated recommendations derived from large-scale behavioural data. In care contexts, participants themselves describe the ambiguity between persuasion and acceptable guidance, noting both the limits and expectations of robotic negotiation in sensitive decision-making situations. At a more structural level, co-constitutive relationships between humans and machines mean that influence is often mutual rather than one-directional. Finally, in therapeutic and monitoring systems, concerns arise that users may be manipulated or coerced due to lack of understanding or absence of alternative perspectives, particularly when consent is incomplete or uninformed.