UIST2015: Sensing Techniques

Tracko: Ad-hoc Mobile 3D Tracking Using Bluetooth Low Energy and Inaudible Signals for Cross-Device Interaction

http://dl.acm.org/citation.cfm?id=2807475

While current mobile devices detect the presence of surrounding devices, they lack a truly spatial awareness to bring them into the user’s natural 3D space. We present Tracko, a 3D tracking system between two or more commodity devices without added components or device synchronization. Tracko achieves this by fusing three signal types. 1) Tracko infers the presence of and rough distance to other devices from the strength of Bluetooth low energy signals. 2) Tracko exchanges a series of inaudible stereo sounds and derives a set of accurate distances between devices from the difference in their arrival times. A Kalman filter integrates both signal cues to place collocated devices in a shared 3D space, combining the robustness of Bluetooth with the accuracy of audio signals for relative 3D tracking. 3) Tracko incorporates inertial sensors to refine 3D estimates and support quick interactions. Tracko robustly tracks devices in 3D with a mean error of 6.5 cm within 0.5 m and a 13 cm error within 1 m, which validates Tracko’s suitability for cross-device interactions.

EM-Sense: Touch Recognition of Uninstrumented, Electrical and Electromechanical Objects

http://dl.acm.org/citation.cfm?id=2807481

Most everyday electrical and electromechanical objects emit small amounts of electromagnetic (EM) noise during regular operation. When a user makes physical contact with such an object, this EM signal propagates through the user, owing to the conductivity of the human body. By modifying a small, low-cost, software-defined radio, we can detect and classify these signals in real-time, enabling robust on-touch object detection. Unlike prior work, our approach requires no instrumentation of objects or the environment; our sensor is self-contained and can be worn unobtrusively on the body. We call our technique EM-Sense and built a proof-of-concept smartwatch implementation. Our studies show that discrimination between dozens of objects is feasible, independent of wearer, time and local environment.

study #1
9 objects, 2 location, trained on 1 person tested on 12 people 6weeks after training> 96.1% accuracy
study#2
24 objects > 97.9% accuracy
study #3
multiple objects of similar category
study#4
identical objects (5 imacs, 4 conf room schedules)
imacs 100%, room scheduler 98%)
>object libraries
study#5
detect object states? (off, low, mid, high of dremel)

usecase
toothbrush > start timer
touch refrigerator and cooking> start radio
touch room door> receive messages
touch wood piece with dremel > instructions advance

limitations: not all objects emit EMI signals, some environments are noisy, doppler shifts.

Tomo: Wearable, Low-Cost Electrical Impedance Tomography for Hand Gesture Recognition

http://dl.acm.org/citation.cfm?id=2807480

smart watch + hand gesture

MRI > magnetic fields and radio waves
CT scan > X-ray
tomo > electric signal

We present Tomo, a wearable, low-cost system using Electrical Impedance Tomography (EIT) to recover the interior impedance geometry of a user’s arm. This is achieved by measuring the cross-sectional impedances between all pairs of eight electrodes resting on a user’s skin. Our approach is sufficiently compact and low-powered that we integrated the technology into a prototype wrist- and armband, which can monitor and classify gestures in real-time. We conducted a user study that evaluated two gesture sets, one focused on gross hand gestures and another using thumb-to-finger pinches. Our wrist location achieved 97% and 87% accuracies on these gesture sets respectively, while our arm location achieved 93% and 81%. We ultimately envision this technique being integrated into future smartwatches, allowing hand gestures and direct touch manipulation to work synergistically to support interactive tasks on small screens. 

use case:
grasp and answer phone call, open and dismiss.

Corona: Positioning Adjacent Device with Asymmetric Bluetooth Low Energy RSSI Distributions

http://dl.acm.org/citation.cfm?id=2807485

We introduce Corona, a novel spatial sensing technique that implicitly locates adjacent mobile devices in the same plane by examining asymmetric Bluetooth Low Energy RSSI distributions. The underlying phenomenon is that the off-center BLE antenna and asymmetric radio frequency topology create a characteristic Bluetooth RSSI distribution around the device. By comparing the real-time RSSI readings against a RSSI distribution model, each device can derive the relative position of the other adjacent device. Our experiments using an iPhone and iPad Mini show that Corona yields position estimation at 50% accuracy within a 2cm range, or 85% for the best two candidates. We developed an application to combine Corona with accelerometer readings to mitigate ambiguity and enable cross-device interactions on adjacent devices.

Disclaimer: The opinions expressed here are my own, and do not reflect those of my employer. -Fumi Yamazaki