The phone in your pocket has the sense of sight (camera), sound (microphone), touch and location (still working on taste and smell). Through live demos we’ll show off some new mobile hardware (could your phone really be replaced by AR glasses?) and what developers and marketers will be able to create with these new tools (speech recognition, computer vision, etc.)

• 1:02 PM: AndreG PRESENTERS
  Ted Power – Google
  Nicholas Jitkoff – Google
  Matt Biddulph – Dopplr
  Ben Averch – Microvision
• 1:02 PM: AndreG Current smartphones are way more than mini computers. They are full of sensors and collect tons of data all the time.
• 1:03 PM: AndreG Steve Mann – MIT/UofT is a pioneer in the field
• 1:04 PM: AndreG 4GB per hour x 24 hrs x 365 – 35TB of video/sound data….
• 1:05 PM: AndreG In 10 year 35TB a year will cost approx $3. Collecting and saving all that data will cost basically nothing.
• 1:06 PM: AndreG Phone enabled senses come along with a huge list of privacy questions. Important as society and people evolve.
• 1:06 PM: AndreG Sound and Voice
• 1:07 PM: AndreG Phones are great at holding data but there is a challenge with accessing it. Old school tiny keyboards are not answer
• 1:08 PM: AndreG Google is currently working on farming out voice and sound analysis to “the cloud” so they can interpret more and more input
• 1:08 PM: AndreG Hard part now is training people on what they can do and try. Humans are not yet accustom to talking to their computers
• 1:09 PM: AndreG The machine needs to be constantly learning and adapting to human input
• 1:11 PM: AndreG All about making things much more convenient. Just like one-click activation. Voice should work well with eyes closed and total lack of UI. Gestures help as a quick way to access the device
• 1:12 PM: AndreG You don’t want an interface that gets in front of you when you are using voice. Needs to be smart enough to understand input and just do the right thing.
• 1:13 PM: AndreG With an effective voice app you can cut down a complex multi step interaction into a handful of very simple interactions. Voice commands while driving are way better than having to navigate a keyboard and touch interaction
• 1:14 PM: AndreG Location Based Interactions….
• 1:14 PM: AndreG Current geo apps are making amazing things happen with very minimal data – lat/long
• 1:15 PM: AndreG Proximity…
• 1:17 PM: AndreG Wiimotes and phones are not that different. Touch and physical feedback are driving haptic interactions across almost every device nowadays.
• 1:17 PM: AndreG Haptics have been around forever and there is lots of really good research behind it.
• 1:18 PM: AndreG Typing on screen, phone can buzz slightly and give the sense of friction on a frictionless surface.
• 1:18 PM: AndreG Pixel addressable surfaces are around the corner for the mainstream
• 1:19 PM: AndreG Touch works well as a secondary backup interaction for a traditional UI
• 1:19 PM: AndreG Haptic feedback can increase typing accuracy by large amounts
• 1:22 PM: AndreG Humans are starting to feed haptic information into their everyday lives. It’s a way to augment human senses and giving people more senses. Human are really good at using secondary information. A quick pocket buzz is all you might need. (example of a device that buzzes when user turns to the north – made map and wayfinding skills much better very quickly)
• 1:22 PM: AndreG Sensing digital information – eg Oyster Card
• 1:23 PM: AndreG Sight
• 1:24 PM: AndreG Pico projectors are finding their ways into phones and tiny devices. Lasers can now create displays with novel capabilities
• 1:25 PM: AndreG Laser based displays were originally built for military purposes. Now filtering down to consumer products
• 1:27 PM: AndreG Augmented reality – overlaying additional digital information over a camera image from a mobile phone camera. Lots of buzz right now
• 1:27 PM: AndreG So far no one has come up with a valuable killer app for augmented reality
• 1:27 PM: AndreG Currently a bit of a novelty
• 1:31 PM: AndreG In the future AR is capable of providing everyone with a totally personalized, metadata enriched experience. Might or might not come from a traditional phone experience. Could be and might need to be built into glasses or some other form factor.
• 1:32 PM: AndreG Having AR in glasses (or contact lenses?) is key to allowing constant AR overlay of location based metadata.
• 1:33 PM: AndreG The first augmented reality tool may have been the Sony Walkman
• 1:36 PM: AndreG Apps like Yelp and Foursquare are another example of augmented reality in that a user can be out and about and easily access metadata about that location (restaurant reviews, etc…). You don’t need a crazy funky UI for AR.
• 1:40 PM: AndreG Google doing a lot to move services like translate, image recognition, etc. into the cloud and allowing users to access their services through their phone. Take a photo of a menu, send photo to cloud, OCR, translate, send back to user. – Google Goggles.