The Microsoft Build conference opened this morning. Some of the announcements were technologies that had been previously announced, but are now more generally available (such as Hololens); Some were brand new technologies, such as Bash on Windows.

I was most excited about the Microsoft Bot Framework that allows you to build intelligent bots that interact with Cortana to act as a personal assistant within your app; and Cognitive Services, which includes many of the APIs included in Project Oxford.

Below are the raw notes I took during the keynote. Please let me know (politely) if you discover any typos.

Edge: Biometric authentication for supporting web sites

Fingerprint login (USAA web site)

Ink

Sticky notes: Cortana recognizes words like "tomorrow" as date in appointments or reminder

Draw lines on a map: Distance automatically calculated.

Add labels to map: Labels stay in place when map is rotated in 3D space

Low latency: Ink from pen

Virtual ruler on screen to draw straight line

Developer

GPU Effects, e.g., Blur background synchronized with touch movement

control

control: Allow users to mark up your screen

Over 1000 new APIs and features

Released today: VS 2015, Update 2

Announced: Bash shell is coming to Windows (native Linux binaries)

Gaming

UWP: Achieve 4k at 60 frames/second

Xbox games running on desktop

  • Take advantage of Windows features
  • Consistent input experience

Xbox developer mode

  • Run, test, debug game on Xbox
  • Preview available today

Cortana on Xbox One

Direct X 12

Hololens

First & only Untethered holographic computer

Hololens shipping today

Releasing app and source code

Apps

  • Holographic anatomy: View internal organs &nerves; collaborate with remote partners
  • Destination Mars by NASA: Virtual tour of another planet

Cortana

Works on lockscreen

Add Cortana to your app.

Insights: When Cortana surfaced in app

Users must give permission for Cortana to their data

Windows, Android, Web Sites, and (soon) iOs

Skype

Interaction with Cortana

Bots: Remembers context from conversation

  • Location of events, friends in locations
  • Bots available today

Skype for Hololens

Microservices

Microsoft Bot Framework

https://dev.botframework.com/

Build conversational bots for your application

Rule-based natural language processing

UI to teach bot to understand your jargon/ terms/ slang

Cognitive Services

22 APIs

https://www.microsoft.com/cognitive-services/

Many of them used to in Project Oxford

Custom Recognition Intelligent Service (CRIS):

  • Understand speech and language
  • Customize speech-to-text

Demo

Smart Phone / Smart glasses app for blind people

Facial recognition and analysis

Read menu, including headings