Project description

Individuals with upper extremity motion disabilities find it difficult to precisely interact with devices that rely on touch interaction, such as mobile smart phones, tablets, and consumer self check-out systems.

To investigate an interaction solution to this problem, we received a grant in amount of $149,864 from NSF SBIR Phase 1, as the PI of a Sub-Award, "Adapting Devices for Motion Disabled Users".

Prior to that, we received a NSF SBIR Phase 0 grant, "Adapting Touchscreen Devices for Motion Disabled Users" to conduct preliminary work in this area.

We have designed and developed a novel accessory device for assisting people with upper extremity motor disabilities to be able to interact with touch devices.

Our work focuses on evaluation of this device as well as improving the design of hardware and interaction techniques.

Results and Future Work

This work has already resulted in a number of patent applications as well as a budding small business in Wyoming.

Future work on this will expand to military solutions and exploration in hybrid 2D/3D space.

The potential impact of this work is to assist users with motion disabilities to better interact with touch-based systems and partially increase independence for an aging population.

A bit of background

Participants in realistic Virtual Environments (VEs) tend to react closer to how they would in real life. This enables researchers to influence their real life behaviours by immersing them in specially crafted VEs.

More realistic VEs can be created by allowing the users touch the virtual objects. Even if the virtual objects' shapes or materials don't quite match what the users are feeling with their hands. As long as the discrepancies between the seen objects and the touched objects is not too large, the discrepancy will not become apparent.

In the following image you can see what the user sees while in the VE. That is their real hand (captured with the help of a LEAP Motion controller), and the texture they are seeing looks like denim, while the actual texture they were touching was cotton:

Our experiment

Our goal was to study how the speed and the direction of the movement of objects in VEs affect the threshold at which discrepancies become evident.

We designed an experimental study in which participants ranked the similarity between the objects they were seeing and the objects they were touching, for both static and moving objects. Then, we increased/decreased the speed and direction discrepancy of the visual objects, and recorded the impressions of the participants for each speed/direction combination.

Our hypothesis for this study, was that the speed of the objects will affect the threshold of noticeable discrepancies. The higher the speed, the less noticeable the discrepancies will be.

Our results

This is actually my latest work, so I don't have results yet, as of Sept 2016. I will update here as results become available.

Why?

This is a program I made for the 3DIA Lab in order to showcase the tracking technology we have there. It also shows how the addition of passive haptic feedback improves the sense of presence inside the virtual world. The original idea comes from a study in University of North Carolina about passive haptic feedback.

What?

The setup consists of a pair of wooden planks in the floor, and an HTC Vive with its two base stations.

The idea behind the program is to evoke the inherent fear of heights we have (a cued gestalt). In the PitDemo, players can look over the edge of the platform they are on, and they act just as if it were real life (they are very cautious not to fall):

About the passive haptic feedback, in our facilities we have a pair of planks we locate in the middle of the Vive's tracking area, and we have them in the same position on the PitDemo's scene. So when a person steps on the virtual planks, they also feel how they are stepping in the real planks:

How to evoke people's fears?

Now, to really evoke the fear of people for heights, we ask them to step on the planks, and we cause an explosion that removes the big platform, leaving only the virtual planks the user is stepping on. This progressive transition into the virtual world adds to the sense of presence in that virtual environment. We ask them to walk from one plank to the next, and this always scares them as the planksare real, if they don't step correctly they will trip (I'm always beneath them in case they lose balance):

Aftermath

Finally, I ask people to walk off the plank to explore the bottom of the world (if they are not too afraid of stepping into the void of course). The fall feels somewhat real, and succeeds at causing a vertigo sensation. This is how the bottom of the PitDemo looks like:

Being able to look up to where they came from also surprises them:

Future work

I have received several suggestions of what to add to this scene. These are the ideas they have given me and that I will be adding soon:

• They want to see their feet and hands in the scene. Well, hands are easy with the LEAP Motion Controller, so I will add that in the upcoming days.
• When they fall they don't see the splash in the water. Ups, my bad, I will add that too.
• They want fish swimming around the player. This is not actually necessary, but I loved the idea :) so I will be adding that too.