Between autonomy and oversight: Trust calibration and human controllability in agentic AI systems

The deployment of agentic AI systems, meaning large language model-based agents capable of autonomous planning, tool use and sequential decision-making with real-world consequences, has surfaced a governance challenge that practitioners and researchers are still working to understand; how much decision-making authority should these systems hold and what mechanisms ensure humans remain meaningfully in control? This paper argues that the answer is neither simple nor uniform. Drawing on a focused review of 34 empirical, technical and regulatory sources published between 2019 and 2026, we examine how trust forms and miscalibrates in human-agent interaction, analyze the design space for human-in-the-loop oversight and identify persistent gaps between regulatory intent and technical implementation. Our primary contribution is the Tiered Controllability Framework (TCF), a four-tier model that maps oversight requirements to task risk, action reversibility and agent autonomy scope. We validate the TCF against three documented enterprise deployment cases and conduct a structured regulatory gap analysis covering the EU Artificial Intelligence Act and the NIST AI Risk Management Framework. Our findings indicate that trust miscalibration, encompassing both over-reliance and under-reliance, constitutes the most prevalent failure mode in deployed agentic systems and that current transparency tools remain inadequate for supporting informed human oversight at operational scale.