Cross-Device Interaction by Sensor Correlation


Although users today frequently interact with a diverse set of devices, many software applications are still programmed to run mostly on single devices alone. What if we change this fundamental assumption, where instead systems will by default run on multiple devices? How will this change the way we design interactive systems? What new interactive experiences can we bring to the users?

My research goal is to explore these questions by building interactive systems that span and leverage a combination of multiple networked devices. To achieve this goal, I have been taking a sensor correlation approach to enable novel user experience of cross-device interaction. Specifically, by correlating sensors across devices, we can capture rich information about users’ behavior, intent and context, which was not accessible using each device alone. I will describe a framework for design-time sensor correlation, as demonstrated in three systems that span wearable, mobile, to room-scale devices. I will also discuss a run-time sensor correlation approach, which allows users to customize their own cross-device interaction via simple programming by demonstration.


Xiang 'Anthony' Chen is a Research Scientist at Tableau Research, focusing on enabling users to interact with data on various digital platforms. He recently graduated from his PhD working with Scott Hudson and Stelian Coros in the School of Computer Science, Carnegie Mellon University. His research develops technical and design approaches to build novel human-computer interfaces that enhance users' physical interactivity with ubiquitous computers, or enable their creativity in fabricating physical objects of their design (e.g., using 3D printing). Anthony is an Adobe Research Fellow in Human-Computer Interaction. Frequently collaborating with industrial research labs (Microsoft, Autodesk, and Google), he has published 17 papers in top-tier HCI conferences and journal (CHI, UIST, and TOCHI) and has received three best paper and honorable mention awards.