Adaptique: Multi-objective and Context-aware Online Adaptation of Selection Techniques in Virtual Reality
Chao-Jung Lai, Mauricio Sousa, Tianyu Zhang, Ludwig Sidenmark, Tovi Grossman
Abstract
Selection is a fundamental task that is challenging in virtual reality due to issues such as distant and small targets, occlusion, and target-dense environments. Previous research has tackled these challenges through various selection techniques, but complicates selection and can be seen as tedious outside of their designed use case. We present Adaptique, an adaptive model that infers and switches to the most optimal selection technique based on user and environmental information. Adaptique considers contextual information such as target size, distance, occlusion, and user posture combined with four objectives: speed, accuracy, comfort, and familiarity which are based on fundamental predictive models of human movement for technique selection. This enables Adaptique to select simple techniques when they are sufficiently efficient and more advanced techniques when necessary. We show that Adaptique is more preferred and performant than single techniques in a user study, and demonstrate Adaptique’s versatility in an application.
System
We define the problem of adapting the interaction technique as follows: given a virtual environment with all inferred selection targets, the system will choose the interaction technique that maximizes selection task performance in terms of four objectives: speed, accuracy, comfort, and familiarity. Speed, accuracy, and comfort are three common metrics used to evaluate interaction technique performance. However, while advanced interaction techniques improve performance in selection tasks, they often come with trade-offs such as increased complexity in control and higher levels of abstraction. Overcoming these drawbacks necessitates user familiarity. Therefore, we consider it as one of the inputs in our system. We quantify these metrics and give objective scores to aid in our optimization process. The system would then post-process the data and switch the interaction technique for the user. Technically, the system works in the following steps as illustrated in the pipeline of the figure:
- Acquire the targets within the interaction space.
- Extract contextual information, including user postures, target positions, sizes, and so on.
- Calculate and aggregate the objectives for each technique.
- Switch the technique if a more optimal one shows a consistent improvement in overall performance.