Seer: Layered Representations for Learning and Inferring Office Activity from Multiple Streams of Information

Abstract

In this project we develop hierarchical probabilistic representations for modeling activities of people. We describe how to use our representation to do sensing, learning, and inference at multiple levels of temporal granularity and abstraction. The approach centers on the use of a Layered Hidden Markov Models (LHMMs), using parameters that are learned from data. LHMMs provide a robust means for modeling diverse human activities. We illustrate the application of LHMMs within an office-awareness situation. We describe the ability to correctly classify in real-time typical office activities such as talking on the phone, being in a meeting with someone else, giving a presentation or just performing work within an office setting.

 

Images from SEER debut  at keynote presentation by Bill Gates, IJCAI 2001. After Eric Horvitz presents Priorities, he introduces Nuria Oliver and Ashutosh Garg for a demonstration of the SEER context-sensing system.

 

Related Papers

A Comparison of HMMs and DBNs for Recognizing Office Activities. Nuria Oliver & Eric Horvitz. Proceed. User Modeling 2005 (UM'05). Edimburgh, July 2005

Selective Perception Policies for Guiding Sensing and Computation in Multimodal Systems: A Comparative Analysis. Nuria Oliver & Eric Horvitz. Computer Vision and Image Understanding Journal (CVIU'05). Volume 100. Issue 1-2

"Selective Perception Policies for Limiting Computation in Multimodal Systems: A Comparative Analysis". Nuria Oliver and Eric Horvitz
Proceedings of Int. Conf. on Multimodal Interfaces (ICMI'03). Vancouver, CA. Nov 2003.

 'Layered Representations for Human Activity Recognition', Nuria Oliver, Eric Horvitz & Ashutosh Garg. Paper presented at ICMI 2002 (Pittsburgh, October 2002)

Paper presented at CVPR2001 (Cues in Communication Workshop), Nuria Oliver, Eric Horvitz & Ashutosh Garg

Videos

Live demonstration during Bill Gates invited speech at IJCAI2001