UIST2015: Touch Input

Biometric Touch Sensing: Seamlessly Augmenting Each Touch with Continuous Authentication

http://dl.acm.org/citation.cfm?id=2807458

Current touch devices separate user authentication from regular interaction, for example by displaying modal login screens before device usage or prompting for in-app passwords, which interrupts the interaction flow. We propose biometric touch sensing, a new approach to representing touch events that enables commodity devices to seamlessly integrate authentication into interaction: From each touch, the touchscreen senses the 2D input coordinates and at the same time obtains biometric features that identify the user. Our approach makes authentication during interaction transparent to the user, yet ensures secure interaction at all times. To implement this on today’s devices, our watch prototype Bioamp senses the impedance profile of the user’s wrist and modulates a signal onto the user’s body through skin using a periodic electric signal. This signal affects the capacitive values touchscreens measure upon touch, allowing devices to identify users on each touch. We integrate our approach into Windows 8 and discuss and demonstrate it in the context of various use cases, including access permissions and protecting private screen contents on personal and shared devices. 

-user authentication

-continuous authentication

-detect people based on shoes

-data transfer to touchscreen

-conversation with Atmel

-go through body -> can do with ankle

Push-Push: A Drag-like Operation Overlapped with a Page Transition Operation on Touch Interfaces

http://dl.acm.org/citation.cfm?id=2807457

A page transition operation on touch interfaces is a common and frequent subtask when one conducts a drag-like operation such as selecting text and dragging an icon. Traditional page transition gestures such as scrolling and flicking gestures, however, cannot be conducted while conducting the drag-like operation since they have a confliction. We proposed Push-Push that is a new drag-like operation not in conflict with page transition operations. Thus, page transition operations could be conducted while performing Push-Push. To design Push-Push, we utilized the hover and pressed states as additional input states of touch interfaces. The results from two experiments showed that Push-Push has an advantage on increasing performance and qualitative opinions of users while reducing the subjective overload.

Out > hover > touch > press

Exploring and Understanding Unintended Touch during Direct Pen Interaction

The user experience on tablets that support both touch and styli is less than ideal, due in large part to the problem of unintended touch or palm rejection. Devices are often unable to distinguish between intended touch (i.e., interaction on the screen intended for action) and unintended touch (i.e., incidental interaction from the palm, forearm, or fingers). This often results in stray ink strokes and accidental navigation, frustrating users. We present a data collection experiment where participants performed inking tasks, and where natural tablet and stylus behaviors were observed and analyzed from both digitizer and behavioral perspectives. An analysis and comparison of novel and existing unintended touch algorithms revealed that the use of stylus information can greatly reduce unintended touch. Our analysis also revealed many natural stylus behaviors that influence unintended touch, underscoring the importance of application and ecosystem demands, and providing many avenues for future research and technological advancement.

-distinguish intentional / unintentional touch
-palm rejection - skin input variations, hand posture, method of moving stylus
-right/left handed users behave differently
-importance of pre-touch information and continuous stylus location

Pin-and-Cross: A Unimanual Multitouch Technique Combining Static Touches with Crossing Selection

http://dl.acm.org/citation.cfm?id=2807444

We define, explore, and demonstrate a new multitouch interaction space called “pin-and-cross.” It combines one or more static touches (“pins”) with another touch to cross a radial target, all performed with one hand. A formative study reveals pin-and-cross kinematic characteristics and evaluates fundamental performance and preference for target angles. These results are used to form design guidelines and recognition heuristics for pin-and-cross menus invoked with one and two pin fingers on first touch or after a drag. These guidelines are used to implement different pin-and-cross techniques. A controlled experiment compares a one finger pin-and-cross contextual menu to a Marking Menu and partial Pie Menu: pin-and-cross is just as accurate and 27% faster when invoked on a draggable object. A photo app demonstrates more pin-and-cross variations for extending two-finger scrolling, selecting modes while drawing, constraining two-finger transformations, and combining pin-and-cross with a Marking Menu.

Disclaimer: The opinions expressed here are my own, and do not reflect those of my employer. -Fumi Yamazaki

UIST2015: Fabrication 2: Flexible and Printed Electronics

Foldio: Digital Fabrication of Interactive and Shape-­Changing Objects With Foldable Printed Electronics

http://dl.acm.org/citation.cfm?id=2807494

- folding
- edge touch slider
- shape sensing/control open//close
- actuation

Foldios are foldable interactive objects with embedded input sensing and output capabilities. Foldios combine the advantages of folding for thin, lightweight and shape-changing objects with the strengths of thin-film printed electronics for embedded sensing and output. To enable designers and end-users to create highly custom interactive foldable objects, we contribute a new design and fabrication approach. It makes it possible to design the foldable object in a standard 3D environment and to easily add interactive high-level controls, eliminating the need to manually design a fold pattern and low-level circuits for printed electronics. Second, we contribute a set of printable user interface controls for touch input and display output on folded objects. Moreover, we contribute controls for sensing and actuation of shape-changeable objects. We demonstrate the versatility of the approach with a variety of interactive objects that have been fabricated with this framework.

uniMorph - Fabricating Thin Film Composites for Shape-Changing Interfaces

http://dl.acm.org/citation.cfm?id=2807472

3 components:
-environmental actuation
-computational control
-sensing and control

Researchers have been investigating shape-changing interfaces, however technologies for thin, reversible shape change remain complicated to fabricate. uniMorph is an enabling technology for rapid digital fabrication of customized thin-film shape-changing interfaces. By combining the thermoelectric characteristics of copper with the high thermal expansion rate of ultra-high molecular weight polyethylene, we are able to actuate the shape of flexible circuit composites directly. The shape-changing actuation is enabled by a temperature driven mechanism and reduces the complexity of fabrication for thin shape-changing interfaces. In this paper we describe how to design and fabricate thin uniMorph composites. We present composites that are actuated by either environmental temperature changes or active heating of embedded structures and provide a systematic overview of shape-changing primitives. Finally, we present different sensing techniques that leverage the existing copper structures or can be seamlessly embedded into the uniMorph composite. To demonstrate the wide applicability of uniMorph, we present several applications in ubiquitous and mobile computing.

Printem: Instant Printed Circuit Boards with Standard Office Printers & Inks

http://dl.acm.org/citation.cfm?id=2807511

Printem film, a novel method for the fabrication of Printed Circuit Boards (PCBs) for small batch/prototyping use, is presented. Printem film enables a standard office inkjet or laser printer, using standard inks, to produce a PCB: the user prints a negative of the PCB onto the film, exposes it to UV or sunlight, and then tears-away the unneeded portion of the film, leaving-behind a copper PCB. PCBs produced with Printem film are as conductive as PCBs created using standard industrial methods. Herein, the composition of Printem film is described, and advantages of various materials discussed. Sample applications are also described, each of which demonstrates some unique advantage of Printem film over current prototyping methods: conductivity, flexibility, the ability to be cut with a pair of scissors, and the ability to be mounted to a rigid backplane.

Capricate: A Fabrication Pipeline to Design and 3D Print Capacitive Touch Sensors for Interactive Objects

http://dl.acm.org/citation.cfm?id=2807503

3D printing is widely used to physically prototype the look and feel of 3D objects. \ Interaction possibilities of these prototypes, however, are often limited to mechanical parts or post-assembled electronics. In this paper, we present Capricate, a fabrication pipeline that enables users to easily design and 3D print highly customized objects that feature embedded capacitive multi-touch sensing. The object is printed in a single pass using a commodity multi-material 3D printer. To enable touch input on a wide variety of 3D printable surfaces, we contribute two techniques for designing and printing embedded sensors of custom shape. The fabrication pipeline is technically validated by a series of experiments and practically validated by a set of example applications. They demonstrate the wide applicability of Capricate for interactive objects.

Disclaimer: The opinions expressed here are my own, and do not reflect those of my employer. -Fumi Yamazaki

UIST2015: Sensing Techniques

Tracko: Ad-hoc Mobile 3D Tracking Using Bluetooth Low Energy and Inaudible Signals for Cross-Device Interaction

http://dl.acm.org/citation.cfm?id=2807475

While current mobile devices detect the presence of surrounding devices, they lack a truly spatial awareness to bring them into the user’s natural 3D space. We present Tracko, a 3D tracking system between two or more commodity devices without added components or device synchronization. Tracko achieves this by fusing three signal types. 1) Tracko infers the presence of and rough distance to other devices from the strength of Bluetooth low energy signals. 2) Tracko exchanges a series of inaudible stereo sounds and derives a set of accurate distances between devices from the difference in their arrival times. A Kalman filter integrates both signal cues to place collocated devices in a shared 3D space, combining the robustness of Bluetooth with the accuracy of audio signals for relative 3D tracking. 3) Tracko incorporates inertial sensors to refine 3D estimates and support quick interactions. Tracko robustly tracks devices in 3D with a mean error of 6.5 cm within 0.5 m and a 13 cm error within 1 m, which validates Tracko’s suitability for cross-device interactions.

EM-Sense: Touch Recognition of Uninstrumented, Electrical and Electromechanical Objects

http://dl.acm.org/citation.cfm?id=2807481

Most everyday electrical and electromechanical objects emit small amounts of electromagnetic (EM) noise during regular operation. When a user makes physical contact with such an object, this EM signal propagates through the user, owing to the conductivity of the human body. By modifying a small, low-cost, software-defined radio, we can detect and classify these signals in real-time, enabling robust on-touch object detection. Unlike prior work, our approach requires no instrumentation of objects or the environment; our sensor is self-contained and can be worn unobtrusively on the body. We call our technique EM-Sense and built a proof-of-concept smartwatch implementation. Our studies show that discrimination between dozens of objects is feasible, independent of wearer, time and local environment.

study #1
9 objects, 2 location, trained on 1 person tested on 12 people 6weeks after training> 96.1% accuracy
study#2
24 objects > 97.9% accuracy
study #3
multiple objects of similar category
study#4
identical objects (5 imacs, 4 conf room schedules)
imacs 100%, room scheduler 98%)
>object libraries
study#5
detect object states? (off, low, mid, high of dremel)

usecase
toothbrush > start timer
touch refrigerator and cooking> start radio
touch room door> receive messages
touch wood piece with dremel > instructions advance

limitations: not all objects emit EMI signals, some environments are noisy, doppler shifts.

Tomo: Wearable, Low-Cost Electrical Impedance Tomography for Hand Gesture Recognition

http://dl.acm.org/citation.cfm?id=2807480

smart watch + hand gesture

MRI > magnetic fields and radio waves
CT scan > X-ray
tomo > electric signal

We present Tomo, a wearable, low-cost system using Electrical Impedance Tomography (EIT) to recover the interior impedance geometry of a user’s arm. This is achieved by measuring the cross-sectional impedances between all pairs of eight electrodes resting on a user’s skin. Our approach is sufficiently compact and low-powered that we integrated the technology into a prototype wrist- and armband, which can monitor and classify gestures in real-time. We conducted a user study that evaluated two gesture sets, one focused on gross hand gestures and another using thumb-to-finger pinches. Our wrist location achieved 97% and 87% accuracies on these gesture sets respectively, while our arm location achieved 93% and 81%. We ultimately envision this technique being integrated into future smartwatches, allowing hand gestures and direct touch manipulation to work synergistically to support interactive tasks on small screens. 

use case:
grasp and answer phone call, open and dismiss.

Corona: Positioning Adjacent Device with Asymmetric Bluetooth Low Energy RSSI Distributions

http://dl.acm.org/citation.cfm?id=2807485

We introduce Corona, a novel spatial sensing technique that implicitly locates adjacent mobile devices in the same plane by examining asymmetric Bluetooth Low Energy RSSI distributions. The underlying phenomenon is that the off-center BLE antenna and asymmetric radio frequency topology create a characteristic Bluetooth RSSI distribution around the device. By comparing the real-time RSSI readings against a RSSI distribution model, each device can derive the relative position of the other adjacent device. Our experiments using an iPhone and iPad Mini show that Corona yields position estimation at 50% accuracy within a 2cm range, or 85% for the best two candidates. We developed an application to combine Corona with accelerometer readings to mitigate ambiguity and enable cross-device interactions on adjacent devices.

Disclaimer: The opinions expressed here are my own, and do not reflect those of my employer. -Fumi Yamazaki

UIST2015 Fabrication 1: Augmentation

Encore: 3D Printed Augmentation of Everyday Objects with Printed-Over, Affixed and Interlocked Attachments

http://dl.acm.org/citation.cfm?id=2807498

One powerful aspect of 3D printing is its ability to extend, repair, or more generally modify everyday objects. However, nearly all existing work implicitly assumes that whole objects are to be printed from scratch. Designing objects as extensions or enhancements of existing ones is a laborious process in most of today’s 3D authoring tools. This paper presents a framework for 3D printing to augment existing objects that covers a wide range of attachment options. We illustrate the framework through three exemplar attachment techniques – print-over, print-to-affix and print-through, implemented in Encore, a design tool that supports a set of analysis metrics relating to viability, durability and usability that are visualized for the user to explore design options and tradeoffs. Encore also generates 3D models for production, addressing issues such as support jigs and contact geometry between the attached part and the original object. Our validation helps to illustrate the strengths and weaknesses of each technique. For example, print-over is stronger than print-to-affix with adhesives, and all the techniques’ strengths are affected by surface curvature.

Print-Over
-extrude and add attachment and connector
-support structure
Print-to-Affix
-print new parts affixable
Print-Through
-use physics simulation

Patching Physical Objects

http://dl.acm.org/citation.cfm?id=2807467



How to keep sustainability of 3D printed objects?
> Not to reprint every time, but to patch/edit the printed objects.
Print > re-scan the old 3D model to patch.

Personal fabrication is currently a one-way process: Once an object has been fabricated with a 3D printer, it cannot be changed anymore; any change requires printing a new version from scratch. The problem is that this approach ignores the nature of design iteration, i.e. that in subsequent iterations large parts of an object stay the same and only small parts change. This makes fabricating from scratch feel unnecessary and wasteful. 

In this paper, we propose a different approach: instead of re-printing the entire object from scratch, we suggest patching the existing object to reflect the next design iteration. We built a system on top of a 3D printer that accomplishes this: Users mount the existing object into the 3D printer, then load both the original and the modified 3D model into our software, which in turn calculates how to patch the object. After identifying which parts to remove and what to add, our system locates the existing object in the printer using the system’s built-in 3D scanner. After calibrating the orientation, a mill first removes the outdated geometry, then a print head prints the new geometry in place. Since only a fraction of the entire object is refabricated, our approach reduces material consumption and plastic waste (for our example objects by 82% and 93% respectively). 

ReForm: Integrating Physical and Digital Design through Bidirectional Fabrication

http://dl.acm.org/citation.cfm?id=2807451

Digital fabrication machines such as 3D printers and laser-cutters allow users to produce physical objects based on virtual models. The creation process is currently unidirectional: once an object is fabricated it is separated from its originating virtual model. Consequently, users are tied into digital modeling tools, the virtual design must be completed before fabrication, and once fabricated, re-shaping the physical object no longer influences the digital model. To provide a more flexible design process that allows objects to iteratively evolve through both digital and physical input, we introduce bidirectional fabrication. To demonstrate the concept, we built ReForm, a system that integrates digital modeling with shape input, shape output, annotation for machine commands, and visual output. By continually synchronizing the physical object and digital model it supports object versioning to allow physical changes to be undone. Through application examples, we demonstrate the benefits of ReForm to the digital fabrication process.

Makers’ Marks: Physical Markup for Designing and Fabricating Functional Objects

http://dl.acm.org/citation.cfm?id=2807508

To fabricate functional objects, designers create assemblies combining existing parts (e.g., mechanical hinges, electronic components) with custom-designed geometry (e.g., enclosures). Modeling complex assemblies is outside the reach of the growing number of novice ``makers'' with access to digital fabrication tools. We aim to allow makers to design and 3D print functional mechanical and electronic assemblies. Based on a formative exploration, we created Makers' Marks, a system based on physically authoring assemblies with sculpting materials and annotation stickers. Makers physically sculpt the shape of an object and attach stickers to place existing parts or high-level features (such as parting lines). Our tool extracts the 3D pose of these annotations from a scan of the design, then synthesizes the geometry needed to support integrating desired parts using a library of clearance and mounting constraints. The resulting designs can then be easily 3D printed and assembled. Our approach enables easy creation of complex objects such as TUIs, and leverages physical materials for tangible manipulation and understanding scale. We validate our tool through several design examples: a custom game controller, an animated toy figure, a friendly baby monitor, and a hinged box with integrated alarm.

Disclaimer: The opinions expressed here are my own, and do not reflect those of my employer. -Fumi Yamazaki

Hackaday workshop: How to solder

I joined the Hackaday workshop on how to solder today :)



It was organized by Matt Berggren. Thank you so much Matt!



We made 3 boards today:

Board #1 – Simple LED Breakout Board



This is the first board of the day... very simple :)



Soldered all of the resistors and LEDs.



Works!



Board #2 – Configurable Breadboard Power Supply

First step: solder the USB port. It's so tiny and hard!



Solder all other parts.



Here is the magic. On P1, which connects to left rail on the breadboard, by putting jumper on left 2 slots, we direct to 5V. By putting jumper on right 2 slots, we direct to adjustable power.

This is important, since when you are using Arduino, power is 5V, but other things you might want to use might be 3.3V. So this board allows us to use 5V on one side and other voltage on the other side :)

Testing that 5V mode works.



Testing that the adjustable mode works.



Similarly on P2, which connects to right rail on the breadboard, by putting jumper on left 2 slots, we direct to 5V. By putting jumper on right 2 slots, we direct to adjustable power.



The middle 3 slots directs where the power comes from. Left 2 will indicate power source is from USB, if we change the jumper to the right one, it directs to DC power source.



Board #3 – BGA-PCB test Platform

Here comes the hard part- we're doing BGA (ball grid array)!

This is the parts we used.



Schematics of resistors and LEDs.



These are the stencils we used.





This is the boards- you can see it's tiny. Imagine how hard soldering on the back side of the board (right) is...!



Place the board under the stencil, put the solder paste on the stencil.



Place the resistors and LEDs on the solder paste. Heat it till the solder turns silver and shiny.



Turn it over and place the board under the stencil again, put the solder paste on the stencil- this is really really hard. I had to do it over many many many many times.

When you finally get the solder paste properly on the board, heat the board again, but be careful- if you overheat, the parts on the other side of the board will move.

I couldn't take any pictures of this process since my hands were dirty with the solder paste.

After both sides are done, put solder paste on the bigger board as well, and place the smaller board on top of the bigger one.



Put the board in the IC heater.





Done! And all LED works as intended...





And we're done!!

Recommended tools by Matthew:

Thanks again Matt, and for those of you interested, his slides etc can be found here:
https://hackaday.io/matt

Disclaimer: The opinions expressed here are my own, and do not reflect those of my employer. -Fumi Yamazaki

October #sfhtml5 on Accelerated Mobile Pages

I joined October SFHTML5 last night, topic was Accelerated Mobile Pages Project.

AMP Project Page:
https://www.ampproject.org/
Google's Blog Post:
Introducing the Accelerated Mobile Pages Project, for a faster, open mobile web
It's an open source project, GitHub Page is here:
https://github.com/ampproject



We talk a lot about performance... we all know performance matters. But what does "slow" mean? "Slowness" is contextual... but as Google's philosophy "Ten things we know to be true" says, we should "focus on user and all else will follow" - so developers should really think about "What does the user feel?" using the service you have built.



One thing we experience is this clash of "monetization and user acquisition" vs "user experience".

This slide got a big reaction! Very true - devs know ad cruft hurts users #sfhtml5 #webdev pic.twitter.com/6JuEw2okwy

— Alan Hogan (@AlanHogan) October 24, 2015

AMP is trying to solve that problem, building a win-win-win for users, publishers and advertisers.

This slide explains why AMP (Accelerated Mobile Pages) is a winning proposition #sfhtml5 @google #webdev pic.twitter.com/dGCb5io4bh

— b01dface (@b01dface) October 24, 2015

AMP project consists of AMP HTML, AMP JS and AMP CDN. No user-authored JS, subset of tags and selectors, but instead it consists of advanced components like amp-img tag, amp-ad tag which is a custom tag for ads, amp-twitter tag that shows embedded tweet, etc.

It's still an early stage project, and the team welcomes you all to join the project, try it out, give feedback and contribute to the project.

Videos:

RAIL: Putting the User at the Center of Performance with Paul Irish



Perf audit: Loading performance

#sfhtml5 Wikimedia team printed Chrome DevTools & analyzed what is slowing down page load, ended up doubling speed! https://t.co/OYZHYxVw9B

— Fumi (@Fumi) October 24, 2015

AMP: Accelerated Mobile Pages with Paul Bakaus



AMP Anatomy with Malte Ubl



Q&A with Paul Irish, Malte Ubl, Jordan Adler, and Paul Bakaus



Disclaimer: The opinions expressed here are my own, and do not reflect those of my employer. -Fumi Yamazaki

Tried "Dominator" of Psycho-pass

I had the opportunity to play with "Dominator" of Psycho-pass!



And of course the video :)

This Dominator is created by a Japanese hardware startup Cerevo, after a famous Japanese science fiction animation "Psycho-pass". If you have never seen Psycho-pass, you should go to Hulu.com or Funimation and watch :)

http://www.hulu.com/psychopass
http://www.funimation.com/shows/psycho-pass/

Story:
In the future, even just thinking about a crime is enough to make you guilty – and justice is dispensed from the barrel of a gun. Detectives work in teams made up of Enforcers and Inspectors. Enforcers take out the bad guys, and Inspectors make sure their partners don’t cross the fine line between good and evil. The great equalizer in the war against thugs is the Dominator, a futuristic weapon that can read minds and assess the probability that a citizen will turn criminal. There are no trials. Only Enforcers, Inspectors, and the Dominator. Commit a crime in your mind – and the Dominator will make sure you pay the ultimate price.

Disclaimer: The opinions expressed here are my own, and do not reflect those of my employer. -Fumi Yamazaki