Human-machine interaction

Designing interfaces for virtual worlds of tomorrow

Human-machine interaction

Designing interfaces for virtual worlds of tomorrow

Overview

The experience provided by today’s virtual reality equipment is a powerful tool capable to create great emotions and feelings of excitement to users who try it. Recent advances in technology have provided affordable and reliable virtual reality systems, which are expected to have an important impact on consumers and industries. Augmented reality technologies benefited as well from hardware advances, and in few years will be able to augment the user experience by empowering users with ever more effective ways to interact with the real and virtual worlds.

Goals

Despite the relevance of the above achievements, developments are still required especially in the context of human-machine interaction in order to make users feel fully immersed and present in the virtual or augmented environments.

VR@POLITO is actively exploring this domain, by focusing on various interaction perspectives, with the aim of further enhancing user experience with next-generation systems.

Results

In particular, studies have been carried out to improve hand gesture-based interaction in wearable settings. For instance, a custom upper limb motion tracking device has been developed, which exploits inertial measurement units to enable interactions outside the field of view of common inside/out sensors available, e.g., on virtual and augmented reality headsets like the HTC Vive, or the Microsoft Hololens. Activities have been carried out in collaboration with the University of Ontario, Institute of Technology (UOIT). Results have been reported in a work which was presented at the IISA 2018 conference, where it won the Best student paper award.

Further activities have been devoted to locomotion. In particular, efforts are aimed at charactering the functionality of various approaches and devices which have been proposed so far for moving in virtual spaces (while possibly interacting with / manipulating objects). More specifically, arm swinging-based techniques (which are quite common in today’s systems) are compared with innovative approaches based on the “walk in place” paradigm as well as with omnidirectional treadmills which are appearing on the market. A work has been published already reporting the results of a preliminary user study with a limited set of experiments which was carried out in collaboration with SiTI – Istituto Superiore sui Sistemi Territoriali per l’Innovazione. A more in-depth analysis is under development.

A further area which is currently explored is that of haptic interaction. In particular, the VR@POLITO will participate in the challenge organized by the 26th IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR 2019) conference, by proposing a solution based on affordable physical passive objects and consumer devices like virtual reality headsets, gloves and other tracking equipment.

 

Multimedia

Publications

Title On the usability of consumer locomotion techniques in serious games: Comparing arm swinging, treadmills and walk-in-place / Calandra, Davide; Lamberti, Fabrizio. Proc. 9th IEEE International Conference on Consumer Electronics, Berlin, Germany, September 8-11, 2019
Abstract When we refer to locomotion in Virtual Reality (VR) we subtend a vast and variegated number of investigations, solutions and devices coming from both research and industry. Despite this richness, a consolidated methodology for evaluating the many locomotion techniques available is still lacking. The present paper extends a previous work in which authors performed a user study-based comparison between two common locomotion techniques, i.e., Arm Swinging, and an omni-directional treadmill with a containment ring. In the study, users were engaged in a realistic immersive VR scenario depicting a fire event in a road tunnel. Remaining adherent to the previously defined methodology, the current work widens the comparison to consider two other locomotion methods (keeping results obtained with the former technique above for reference purposes), namely, a different treadmill constraining the user through a top-mounted independent support structure, and Walk-in-Place, a technique which allows the user to move in the virtual environment by performing a natural marching gesture by exploiting two sensors placed on his or her legs. less
Related project or activity: Human-machine interaction
Title Investigating tangible user interaction in mixed-reality robotic games / Prattico', Filippo Gabriele; Baldo, Piero; Cannavo', Alberto; Lamberti, Fabrizio. Proc. 9th IEEE International Conference on Consumer Electronics, Berlin, Germany, September 8-11, 2019, (In press), pp. 1-6.
Abstract Among the emerging trends in Human-Robot Inter- action, some of the most frequently used paradigms of interaction involve the use of Tangible User Interfaces. This is especially true also in the field of robotic gaming and, more specifically, in application domains in which commercial off-the-shelf robots and projected Mixed Reality (MR) technology are combined together. The popularity of such interfaces, also in other domains of Human-Machine Interaction, has led to an abundance in the number of gestures that can be used to perform tangible action using these interfaces. However, there are not sufficient pieces of evidence on how these different modalities can impact the user experience, in particular when interacting with a robot in a “phygital play” environment. By moving from this consideration, this paper reports on the efforts that are ongoing with the aim to investigate the impact of diverse gesture sets (which can be performed with the same physical prop) on the perception of interaction with the robotic system. It also presents preliminary insights obtained, which could be exploited to orient further research about the use of such interfaces for interaction in MR- based robotic gaming and related scenarios.
Related project or activity: Human-machine interaction
Title Building trust in autonomous vehicles: Role of virtual reality driving simulators in HMI design / Morra, Lia; Lamberti, Fabrizio; Prattico', Filippo Gabriele; La Rosa, Salvatore; Montuschi, Paolo. In: IEEE Transactions on Vehicular Technology. ISSN 0018-9545 (In press), pp. 1-12.
Abstract The investigation of factors contributing at making humans trust Autonomous Vehicles (AVs) will play a fundamental role in the adoption of such technology. The user's ability to form a mental model of the AV, which is crucial to establish trust, depends on effective user-vehicle communication; thus, the importance of Human-Machine Interaction (HMI) is poised to increase. In this work, we propose a methodology to validate the user experience in AVs based on continuous, objective information gathered from physiological signals, while the user is immersed in a Virtual Reality-based driving simulation. We applied this methodology to the design of a head-up display interface delivering visual cues about the vehicle' sensory and planning systems. Through this approach, we obtained qualitative and quantitative evidence that a complete picture of the vehicle's surrounding, despite the higher cognitive load, is conducive to a less stressful experience. Moreover, after having been exposed to a more informative interface, users involved in the study were also more willing to test a real AV. The proposed methodology could be extended by adjusting the simulation environment, the HMI and/or the vehicle's Artificial Intelligence modules to dig into other aspects of the user experience.
Related project or activity: Human-machine interaction
Title User perception of robot's role in floor projection-based Mixed-Reality robotic games / Prattico', Filippo Gabriele; Cannavo', Alberto; Chen, Junchao; Lamberti, Fabrizio (In press), pp. 1-6, Proc. IEEE 23rd International Symposium on Consumer Technologies (ISCT 2019), Ancona, Italy, June 19-21, 2019.
Abstract Within the emerging research area represented by robotic gaming and, specifically, in application domains in which the recent literature suggests to combine commercial off-the-shelf (COTS) robots and projected mixed reality (MR) technology in order to develop engaging games, one of the crucial issues to consider in the design process is how to make the player perceive the robot as having a key role, i.e., to valo
Related project or activity: Human-machine interaction
Title Building reconfigurable passive haptic interfaces on demand using off-the-shelf construction bricks. Calandra, Davide; Prattico', F. Gabriele; Cannavo', Alberto; Micelli, Luca; Lamberti, Fabrizio (In press), pp. 1-2. Proc. 26th IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR 2019), Osaka, Japan, March 23-27, 2019.
Abstract Although passive haptic interfaces have been shown to be capable to enhance the user’s sense of presence in Mixed Reality experiences, their use is still constrained by the need to rely on exact replicas of virtual objects or on custom-made devices mimicking the original ones. Unfortunately, the former are not flexible enough in terms of reconfigurability, whereas the latter may be difficult to reproduce. To tackle these issues, this paper explores the possibility to build passive haptic interfaces using off-the-shelf toy construction bricks. Bricks can be assembled to provide the intended feedback in more than one task. Moreover, they may be reassembled in another application to mimic completely new objects and support totally different tasks.
Related project or activity: Human-machine interaction
Title Extending upper limb user interactions in AR, VR and MR headsets employing a custom-made wearable device. Micelli, Luca; Acosta, David; Uribe-Quevedo, Alvaro; Lamberti, Fabrizio; Kapralos, Bill (In press), pp. 1-4. Proc. 9th IEEE International Conference on Information, Intelligence, Systems, and Applications (IISA 2018), Zakynthos, Greece, July 23-25, 2018.
Abstract Upper limb interactions play an important role in scenarios employing virtual, augmented, and mixed reality. Nu- merous sensors (e.g., optical, magnetic, mechanical, myography), have been employed to provide natural interactions, in some cases requiring additional hardware. Recently, virtual, augmented, and mixed reality headsets have started embedding sensors to avoid the use of external hardware that can be cumbersome, thus increasing their ease of use and setup. Microsoft’s recent approach to embedded sensors is known as inside/out tracking, comprised of integrated cameras that allow interactions through fingers and eye tracking with the Hololens, or controller tracking with the mixed reality headsets. However, the motion capture area of the inside/out sensors is limited to the field of view of the cameras. In this paper, we introduce a custom upper limb motion tracking device that extends the user interactions while employing a VR, AR, or MR headset. Our 3D motion tracking system is a compact wireless wearable prototype employing inertial measurement units that provide orientation and position data that provides upper limb user interaction outside the field of view of the inside/out sensors.
Related project or activity: Human-machine interaction
Title Arm swinging vs treadmill: A comparison between two techniques for locomotion in virtual reality / Calandra, Davide; Bill, Michele; Lamberti, Fabrizio; Sanna, Andrea; Borchiellini, Romano. (In press), pp. 1-4. Proc. Eurographics 2018, Delft, The Netherlands, April 16-20, 2018.
Abstract When it comes to locomotion in Virtual Reality (VR), a wide range of different techniques has been proposed in the scientific literature or as commercial products. However, the best choice for a specific application is still not immediate, being each technique characterized by different advantages and drawbacks. The present work reports on the results of a user study-based comparison between two methods: a locomotion treadmill, which supports omni-directional movements through walking in place over a hardware device, and Arm Swinging, which recognizes movement from the swinging back and forth of the user's arms (e.g., gathered by sensors embedded in hand controllers). Experiments have been carried out in a realistic immersive VR scenario, which requested users to respond to a fire emergency in a road tunnel by moving and interacting with the environment.
Related projects or activities: Human-machine interaction, Education and training

Contact Person

Fabrizio Lamberti