Publications

Storytelling in Intelligence: Theoretical Foundations

This article provides a theoretical framework with analyses of key concepts, including narratives, narrative truth, narrative plausibility, and explanation.

Keywords: storytelling,intelligence

Strollytelling: Coupling Animation with Physical Locomotion to Explore Immersive Data Stories

We propose and explore 'strolly'telling, a novel data storytelling technique that maps the story progression with the user/audience’s physical locomotion. Inspired by the conventional web-based technique for scrolling-based stories (i.e. scrollytelling), our technique tightly couples the user’s position in physical space to the animation frame of the data story. This technique leverages the natural tendency of humans to 'walk and talk' while telling a story and requires users to engage with the content actively.

Keywords: chi,storytelling,immersive storytelling

Investigating the Effects of Physical Landmarks on Spatial Memory for Information Visualisation in Augmented Reality

While spatial memory has been studied in physical environments (wall displays) and virtual reality environments, there has been little research on how physical surroundings might affect memorisation of virtual content in a mixed environment like AR. Therefore, we provide the first AR study of spatial memory, comparing two different room settings with two different situated layouts of virtual targets on an abstract spatial memory task.

Keywords: spatial memory, immersive analytics, view man-agement, physical landmark, augmented reality, mixed reality

SADAS: A Dialogue Assistant System Towards Remediating Norm Violations in Bilingual Socio-Cultural Conversations

In today's globalized world, bridging the cultural divide is more critical than ever for forging meaningful connections. The Socially-Aware Dialogue Assistant System (SADAS) is our answer to this global challenge, and it's designed to ensure that conversations between individuals from diverse cultural backgrounds unfold with respect and understanding. Our system's novel architecture includes: (1) identifying the categories of norms present in the dialogue, (2) detecting potential norm violations, (3) evaluating the severity of these violations, (4) implementing targeted remedies to rectify the breaches, and (5) articulates the rationale behind these corrective actions.

Keywords: computational cultural understanding, augmented reality

That's Rough! Encoding Data into Roughness for Physicalization

While visual channels (e.g., color, shape, size) have been explored for visualizing data in data physicalizations, there is a lack of understanding regarding how to encode data into physical material properties (e.g., roughness, hardness). This understanding is critical for ensuring data is correctly communicated and for potentially extending the channels and bandwidth available for encoding that data. We present a method to encode ordinal data into roughness, validated through user studies.

Keywords: chi,physicalization,visualization

Context-Dependent Memory in Situated Visualization

Situated visualization presents data alongside their source context (physical referent). While environmental factors influence memory recall (known as Context-Dependent Memory or CDM), how physical context affects cognition in real-world tasks such as working with visualizations in situated contexts is unclear. This study explores the design space of information memorability in situated visualization through the lens of CDM.

Keywords: context-dependent-memory,situated visualization

Active Proxy Dashboard: Binding Physical Referents and Abstract Data Representations in Situated Visualization through Tangible Interaction

We introduce a conceptual design space defining four quadrants as combinations of two dimensions: spatial to abstract representation; and passive to active proxy. Our focus is the novel quadrant defined by active proxies and abstract representations. Designing active proxy techniques is non-trivial as users are accustomed to common interaction modalities.

Keywords: chi,dashboard,late-breaking work,scale model,situated visualization,tangible

Defining Embodied Provenance for Immersive Sensemaking

We propose the concept of embodied provenance, the use of three-dimensional space and embodied interactions in supporting recalling, reproducing, annotating and sharing analysis history in immersive environments. We highlight a set of design criteria for analytic provenance drawn from prior work and propose a conceptual framework for embodied provenance.

Keywords: analytics provenance,chi,embodied,late-breaking work

Embodied Provenance for Immersive Sensemaking

We propose the concept of embodied provenance, the use of three-dimensional space and embodied interactions in supporting recalling, reproducing, annotating and sharing analysis history in immersive environments. We propose a conceptual framework for embodied provenance by highlighting a set of design criteria for analytic provenance drawn from prior work and identifying essential properties for embodied provenance.

Keywords: analytics provenance,embodied,iss

ProxSituated Visualization: An Extended Model of Situated Visualization using Proxies for Physical Referents

We propose an extended model of situated visualization that encompasses Immediate Situated Visualization and ProxSituated (Proxy of Situated) Visualization. Our model describes a set of key entities involved in proxSituated scenarios and important relationships between them. From this model, we derive design dimensions and apply them to existing situated visualization work.

Keywords: chi,design space,situated visualization,theory

User-Driven Constraints for Layout Optimisation in Augmented Reality

In this paper we explore the concept of user-driven constraints for augmented reality layout optimisation. With our approach users can define and set up their own constraints directly within the real-world environment. We first present a design space composed of three dimensions: the constraints, the regions of interest and the constraint parameters.

Keywords: augmented reality,chi,layout,optimisation

Visualization Placement for Outdoor Augmented Data Tours

We explore the impact of visualization placement on user experience for outdoor augmented data tours, the tours that rely on situated data visualization in augmented reality (AR). Little is known about the optimal placement of outdoor AR information visualizations. Inspired by an existing guided tour about the environmental impact of buildings, we investigated three potential placements for data visualizations: on buildings, on the floor in the user’s immediate area, and on a virtual map displayed in front of the user.

Keywords: data tour,energy data,situated visualization,sui

Augmented Scale Models: Presenting Multivariate Data Around Physical Scale Models in Augmented Reality

We present augmented scale models, immersive visualisations that place charts of multivariate data via Augmented Reality (AR) registered to physical 3D models. We identified two main factors for presenting AR charts in the limited display space around the models: 1) how charts are laid out (Slides vs Dashboard), and 2) how the chart views are arranged in the 3D space (On Scale Model, On Table, On Virtual Board).

Keywords: augmented reality,dashboard,ismar,layout,scale model,situated visualization,tangible

Tangible Globes for Data Visualisation in Augmented Reality

We explore the design space and use cases of tangible globe visualisation in AR. We created custom tangible globes that can be tracked in 3D space using a combination of a motion capture system and HoloLens 2. The globe has internal infrared LEDs, as opposed to standard marker-based motion tracking. This allows for a clean view of geographical information on the globe surface.

Keywords: augmented reality,chi,design space,geovisualisation,globe,tangible

TimeTables: Embodied Exploration of Immersive Spatio-Temporal Data

We propose a novel prototype to support data exploration, using embodiment with space-time cubes in virtual reality. TimeTables uses multiple space-time cubes on virtual tabletops, which users can manipulate by extracting time layers or individual buildings to create new tabletop views. The surrounding environment includes a large space for multiple linked tabletops and a storage wall.

Keywords: embodied,ieeevr,spatio-temporal,virtual reality

Embodied Gesture Interaction for Immersive Maps

Our work explores the use of embodied interaction for immersive maps. We propose four design considerations for embodied maps and embodied gesture interaction with immersive maps: object presence, consistent physics, human body skills, and direct manipulation. We present an example of an immersive flow map with a series of novel embodied gesture interactions, which adhere to the proposed design considerations.

Keywords: cagis,embodied,geovisualisation,gesture,maps

Quantitative Data Visualisation on Virtual Globes

We evaluated five different globe visualisation idioms and then proposed a new visualisation design. Geographic data visualisation on virtual globes is intuitive and widespread, but has not been thoroughly investigated. We explore two main design factors for quantitative data visualisation on virtual globes: i) commonly used primitives (2D bar, 3D bar, circle) and ii) the orientation of these primitives (tangential, normal, billboarded).

Keywords: chi,globe,perception,virtual globe

Elicitation Study Investigating Hand and Foot Gesture Interaction for Immersive Maps in Augmented Reality

While immersive maps can offer unprecedented engaging experiences, the way to perform panning, zooming and other basic map interaction is not obvious. This limitation may hamper widespread adoption of immersive maps. We therefore conducted an elicitation study to identify commonly suggested hand and foot gestures for interacting with large immersive AR maps placed on the floor. Study participants were shown simulations of interaction operations, and asked to design gestures to trigger these operations.

Keywords: augmented reality,cagis,elicitation,gesture,maps

Maps Around Me: 3D Multiview Layouts in Immersive Spaces

We conducted a user study to obtain a better understanding of how users arrange immersive multiview maps. Visual exploration of maps often requires a contextual understanding at multiple scales and locations. Multiview map layouts, which present a hierarchy of multiple views to reveal detail at various scales and locations, have been shown to support better performance than traditional single-view exploration on desktop displays.

Keywords: geovisualisation,iss,layout,maps,virtual reality

Augmented Reality Map Navigation with Freehand Gestures

We evaluated freehand gestures design space for AR map navigation. Mid-air hand gesture interaction has long been proposed as a ‘natural’ input method for Augmented Reality (AR) applications, yet has been little explored for intensive applications like multiscale navigation. In multiscale navigation, such as digital map navigation, pan and zoom are the predominant interactions.

Keywords: augmented reality,gesture,ieeevr,maps

Interacting with maps in virtual and augmented reality

Keywords:

Immersive Isosurface Visualisation for Engineering Datasets

In this paper, we present the study and the design of a method and workflow to display isosurfaces in heterogeneous immersive environments. In particular, we address the problem of scale and structure inherent to this type of data. We provide a set of tools to process large volume datasets to eventually display it in 3D immersive environments.

Keywords: bdva,engineering,isosurface,scalar field