User-Driven Constraints for Layout Optimisation in Augmented Reality

Aziz Niyazov, Barrett Ens, Kadek Ananta Satriadi, Nicolas Mellado, Loïc Barthe, Tim Dwyer, Marcos Serrano

Automatic layout optimisation allows users to arrange augmented reality content in the real-world environment without the need for tedious manual interactions. This optimisation is often based on modelling the intended content placement as constraints, defined as cost functions. Then, applying a cost minimization algorithm leads to a desirable placement. However, such an approach is limited by the lack of user control over the optimisation results. In this paper we explore the concept of user-driven constraints for augmented reality layout optimisation. With our approach users can define and set up their own constraints directly within the real-world environment. We first present a design space composed of three dimensions: the constraints, the regions of interest and the constraint parameters. Then we explore which input gestures can be employed to define the user-driven constraints of our design space through a user elicitation study. Using the results of the study, we propose a holistic system design and implementation demonstrating our user-driven constraints, which we evaluate in a final user study where participants had to create several constraints at the same time to arrange a set of virtual contents.

Niyazov, A., Ens, B., Satriadi, K.A., Mellado, N., Barthe, L., Dwyer, T., Serrano, M. 2023. User-Driven Constraints for Layout Optimisation in Augmented Reality. To appear at CHI ’23: ACM Conference on Human Factors in Computing Systems, April 23 - April 28 2023, Hamburg, Germany.