© 2019, © 2019 Taylor & Francis Group, LLC. This article presents a qualitative driving simulator study designed to understand the experience of giving up control to automated processes in semiautonomous driving systems. The study employed an experience prototyping methodology, with 12 drivers (4 female) completing 2 sessions in a high-fidelity driving simulator. Condition A simulated a normally functioning car, while Condition B simulated a semiautonomous system that monitors driver behavior and takes evasive action when danger is detected. The simulator experience was used to ground wider discussion of automation and the experience of driving, which was explored through a semistructured interview. Results identify design challenges for autonomous driving systems; the loss of user agency and confidence, and handling the change between manual and automated control. Opportunities were identified; in augmenting rather than removing human abilities, and in providing new learning opportunities for drivers.