Mobile augmented reality systems afford a host of intriguing engineering challenges and research problems at the intersection of distributed signal processing, coding, and system architecture. For object recognition on mobile devices, a visual data base is typically stored in the cloud. Hence, for a visual comparison, information must be either uploaded from, or downloaded to, the mobile over a wireless link. The response time of the system critically depends on how much information must be transferred in both directions, and efficient compression is the key to a good user experience. We review recent advances in mobile visual search, using compact feature descriptors, and show that dramatic speed-ups and power savings are possible by considering recognition, compression, and retrieval jointly. For augmented reality applications, where image matching is performed continually at video frame rates, interframe coding of SIFT descriptors achieves bit-rate reductions of 1-2 orders of magnitude relative to advanced video coding techniques. We will use real-time implementations for different example applications, such as recognition of landmarks, media covers or printed documents, to show the benefits of implementing computer vision algorithms on the mobile device, in the cloud, or both.

Bernd Girod is Professor of Electrical Engineering in the Information Systems Laboratory of Stanford University, California, since 1999. Previously, he was a Professor in the Electrical Engineering Department of the University of Erlangen-Nuremberg. His current research interests are in the area of networked media systems. He has published over 500 conference and journal papers and 6 books, receiving the EURASIP Signal Processing Best Paper Award in 2002, the IEEE Multimedia Communication Best Paper Award in 2007, the EURASIP Image Communication Best Paper Award in 2008, the EURASIP Signal Processing Most Cited Paper Award in 2008, as well as the EURASIP Technical Achievement Award in 2004 and the Technical Achievement Award of the IEEE Signal Processing Society in 2011. As an entrepreneur, Professor Girod has been involved in several startup ventures, among them Polycom, Vivo Software, 8x8, and RealNetworks. He received an Engineering Doctorate from University of Hannover, Germany, and an M.S. Degree from Georgia Institute of Technology. Prof. Girod is a Fellow of the IEEE, a EURASIP Fellow, and a member of the German National Academy of Sciences (Leopoldina). He currently serves Stanford’s School of Engineering as Senior Associate Dean for Online Learning and Professional Development.

This talk gives an overview of some recent work at NVIDIA Research on mobile visual computing. We give an overview of Tegra, the NVIDIA family of mobile processors, and in particular look into its visual processing capabilities. We then discuss several recent research projects: 2D registration of moving scenes for computational photography, 3D tracking and modeling for augmented reality, a processing architecture for hierarchical image processing, and interactive editing of a live scene on a camera viewfinder which provides a WYSIWYG experience for computational photography.

Kari Pulli is a Senior Director at NVIDIA Research where he heads the Mobile Visual Computing Research team and works on topics related to cameras, imaging, and vision on mobile devices. Before NVIDIA he was at Nokia where he was the 6th Nokia Fellow and a Member of CEO's Technology Council. Kari has worked on standardizing mobile media APIs at Khronos and JCP and wrote a book on Mobile 3D Graphics. Kari was a visiting scientist at MIT and research associate at Stanford University. Kari has a B.Sc. from the University of Minnesota, M.Sc. and Lic. Tech. from the University of Oulu (Finland), and Ph.D. from the University of Washington (Seattle), all in Computer Science / Engineering; and an MBA from the University of Oulu.