# Friday, 14 September 2018

A chatbot provides an electronic exchange between a user and a computer. For example, each exchange might consist of a question from the user and an answer from the computer or a question from the computer and an answer from the user.

As with most interactions that take place across the Internet, a chatbot is, by default, stateless - meaning it remembers nothing from one exchange to the next.

Often, we would like to remember answers from one question to the next or even from one conversation to the next. If the bot asks a person's name, that name could be used to respond to a question asked later in the same conversation; or even in a different conversation.

This is why we implement State in our bots.

There are two key issues when managing state:

  • What is the scope of the data?
  • Where is stateful information stored?

In this article, we will focus on the first question and store all state data in memory.

What is the scope of the data?

The first question we ask ourselves is: When we save the data, when and where will it be seen?

We have 3 options, corresponding to 3 properties of the context (IDialogContext Interface)

  • ConversationData. Data stored here is available to anyone within the current conversation.
  • PrivateConversationData. Data stored here is available to the current user within the current conversation.
  • UserData. Data stored here is available to the current user in any conversation. This means he/she can come back the next day and the bot will remember his/her data.

Each of the 3 options above is a property of the IDialogContext interface. The context object, which is passed into the MessageReceivedAsync, implements this interface, so we can read and write to the appropriate storage by reading and writing to objects stored in context.

Below is a simple example that reads to and writes from the three different storage options.

private async Task MessageReceivedAsync(IDialogContext context, IAwaitable<IMessageActivity> result)
{
	var message = await result;
	var channelId = message.ChannelId;
	var conversationId = message.Conversation.Id;
	var userId = message.From.Id;

	var currentTime = DateTime.Now;

	DateTime firstLoginTimeThisConversationAnyUser = DateTime.MinValue;

	context.ConversationData.TryGetValue<DateTime>("FirstLoginTime", out firstLoginTimeThisConversationAnyUser);
	if (firstLoginTimeThisConversationAnyUser == System.DateTime.MinValue)
	{
		firstLoginTimeThisConversationAnyUser = System.DateTime.Now;
		context.ConversationData.SetValue<DateTime>("FirstLoginTime", firstLoginTimeThisConversationAnyUser);
	}

	DateTime firstLoginTimeEverForCurrentUser = DateTime.MinValue;
	context.UserData.TryGetValue<DateTime>("FirstLoginTime", out firstLoginTimeEverForCurrentUser);
	if (firstLoginTimeEverForCurrentUser == System.DateTime.MinValue)
	{
		firstLoginTimeEverForCurrentUser = currentTime;
		context.UserData.SetValue<DateTime>("FirstLoginTime", firstLoginTimeEverForCurrentUser);
	}

	DateTime firstLoginTimeCurrentConversationCurrentUser = DateTime.MinValue;
	context.PrivateConversationData.TryGetValue<DateTime>("FirstLoginTime", out firstLoginTimeCurrentConversationCurrentUser);
	if (firstLoginTimeCurrentConversationCurrentUser == System.DateTime.MinValue)
	{
		firstLoginTimeCurrentConversationCurrentUser = currentTime;
		context.PrivateConversationData.SetValue<DateTime>("FirstLoginTime", firstLoginTimeCurrentConversationCurrentUser);
	}

	var output = $"User: {userId}\nConversation: {conversationId}\nChannel: {channelId}\n\n";
	output += $"A user logged into this converstaion at {firstLoginTimeThisConversationAnyUser}.\n";
	output += $"You first logged into any conversation at {firstLoginTimeEverForCurrentUser}.\n";
	output += $"You first logged into this conversation at {firstLoginTimeCurrentConversationCurrentUser}.";
	await context.PostAsync(output);

	context.Wait(MessageReceivedAsync);
}
  

  Listing 1

Notice that we are checking each time to see if any new data is returned from a name-value pair ("FirstLoginTime"). If nothing is stored in a value, it will return DateTime.MinValue, which signifies an empty date. In this case, we set the value to the current time.

You can test this with the Bot Framework Emulator (available here and described here).

You can download a Visual Studio Solution that uses the code in Listing 1 from the SavingStateDemo of the Bot-Framework-Demos on my GitHub page.

Launch your bot from Visual Studio and connect to it with the Emulator. Type any message into the textbox and press [ENTER], as shown in Fig. 1.

Fig01-BotEmulator1
Fig. 1

You should see information about the user, channel, and conversation; along with the value just stored for the when users (including you) logged into this (or another) conversation. Notice all 3 values are the same because this is your first conversation and you are the first user in this conversation.

Again, type any message in the textbox and press [ENTER]. The values have not changed because they were set as stateful data last time (Fig. 2)

Fig02-BotEmulator2
Fig. 2

You can start a new conversation by clicking the ENPOINT on the left, opening a new tab. Type something into this tab's textbox and press [ENTER] (Fig. 3)

Fig03-BotEmulatorNewConversation
Fig. 3

Notice that the user is the same, but the Conversation ID is different.

Therefore, the bot updated the ConversationData and the PrivateConversationData values, but not the UserData value.

In this article, I showed you how to save stateful data from a conversation using the Microsoft Bot Framework.

Friday, 14 September 2018 09:42:00 (GMT Daylight Time, UTC+01:00)
# Thursday, 13 September 2018

GCast 13:

Azure Media Services: Closed Captioning

Generate and add closed captioning to your video with Microsoft Azure Media Services.

Thursday, 13 September 2018 07:11:00 (GMT Daylight Time, UTC+01:00)
# Wednesday, 12 September 2018

In the previous articles, I showed how to create a new chatbot using the Microsoft Bot Framework and I explained the relevant code in the sample chatbot you created.

In this article, I will show how to extend your bot by adding a new dialog.

Launch Visual Studio and create a new project based on the "Bot Builder Echo Bot", as shown in Fig. 1

Fig01-NewBotProject
Fig. 1

From the Visual Studio menu, select Build | Rebuild Solution.

In the Visual Studio Solution Explorer, right-click the "Dialogs" folder and select Add | Class from the context menu, as shown in Fig. 2.

Fig02-AddClassMenu
Fig. 2

A dialog displays, allowing you to name and add a new class, as shown in Fig. 4.

Fig03-AddClass
Fig. 3

Enter a name for this dialog class. I chose "TimeDialog.cs", because I plan to add code to display the current time. Click the [Add] button to create this class.

An editor with the default code in it displays. The default code is in Listing 1.

namespace MyBotWithDialog.Dialogs
{
    public class TimeDialog
    {
    }
}
  

Listing 1

Above the class declaration, add the [Serializable] tag.

After the class name, add the following to derive from the IDialog interface.

: IDialog<object>

The IDialog interface requires one method: StartAsync. Add this method as shown in Listing 2.

        public Task StartAsync(IDialogContext context)
        {
            context.Wait(MessageReceivedAsync);

            return Task.CompletedTask;
        }
  

Listing 2

Notice that StartAsync asynchronously calls MessageReceivedAsync, so you will have to add this method, as shown in Listing 3.

        private async Task MessageReceivedAsync(IDialogContext context, IAwaitable<object> result)
        {
            var activity = await result as Activity;

            // Calculate something for us to return
            int length = (activity.Text ?? string.Empty).Length;

            // Return our reply to the user
            await context.PostAsync($"The time is now {System.DateTime.Now}!");

            context.Wait(MessageReceivedAsync);
        }
  

Listing 3

Note that the output sent back to the channel is a string containing the current time.

Finally, you will need to add using statements at the top to resolve any errors. Add the following to the top of the file.

using Microsoft.Bot.Builder.Dialogs;

using Microsoft.Bot.Connector;

Listing 4

When you are finished, the class should look like Listing 5 below.

using System;
using System.Threading.Tasks;
using Microsoft.Bot.Builder.Dialogs;
using Microsoft.Bot.Connector;

namespace MyBotWithDialog.Dialogs
{
    [Serializable]
    public class RootDialog : IDialog<object>
    {
        public Task StartAsync(IDialogContext context)
        {
            context.Wait(MessageReceivedAsync);

            return Task.CompletedTask;
        }

        private async Task MessageReceivedAsync(IDialogContext context, IAwaitable<object> result)
        {
            var activity = await result as Activity;

            // Calculate something for us to return
            int length = (activity.Text ?? string.Empty).Length;

            // Return our reply to the user
            await context.PostAsync($"The current time is {System.DateTime.Now}!");

            context.Wait(MessageReceivedAsync);
        }
    }
}
  

Listing 5

Next, modify the controller to point to your new Dialog. Open MessageController.cs and find the following line of code.

await Conversation.SendAsync(activity, () => new Dialogs.RootDialog());

Replace this line with the following:

await Conversation.SendAsync(activity, () => new Dialogs.TimeDialog());

Now, build and run your bot. When it loads, use the Bot Framework Emulator (available here and described here) to test it.

Connect to your Bot from the Emulator and enter any text in the "Type your message" textbox and press ENTER.

The bot should respond by displaying the current date and time, as shown in Fig. 3.

Fig04-BotFrameworkEmulator
Fig. 4

In this article, I showed how to create and use a new Dialog class to an Bot Builder Echo Bot project. You can find this code in the NewDialogDemo of the Bot-Framework-Demos on my GitHub page.

Wednesday, 12 September 2018 09:41:00 (GMT Daylight Time, UTC+01:00)
# Tuesday, 11 September 2018

An Azure storage account gives you the ability to create, store, and manage tables, queues, blobs, and files.

Azure Storage Services

Available Azure storage services are:

Blobs

A blob is any unstructured or semi-structured object that you want to store. Examples include videos, images, and text files. Blob storage is flexible for storing an object without the system having to know much about the object.

Files

Azure Files allow you to store files to Azure and access them as a file share using the standard SMB protocol. Files are built on top of Azure Blob Storage.

Tables

Azure Table Storage provides a simple NoSQL database for your application. Data in an Azure Table is stored as rows. Each row contains a key and one or more properties. You do not need to pre-define these properties beforehand and you can define different properties for different rows in the same table.

Queues

A queue is a popular mechanism for designing asynchronous applications. One application can drop a message onto a queue and another application can pick up that message later and process it. This decoupling allows for scalability, flexibility, and faster response times.

Creating an Azure Storage Account

To create a new Azure Storage Account, navigate to the https://portal.azure.com.

Click the [Create a resource] button; then select Storage | Storage account - blob, file, table, queue from the menu, as shown in Fig. 1

Fig01-NewResource
Fig. 1

The New Storage Account blade displays, as shown in Fig. 2.

Fig02-NewStorage
Fig. 2

At the "name" field, enter a unique name for your storage account. You will be able to access this account through a REST API by sending requests to

https://accountname.core.windows.net

where accountname is the name you enter for your account.

At the "Deployment model" radio button, select "Resource manager".

At the "Account kind" dropdown, select "Storage (general purpose)"

At the "Location" dropdown, select a location in which to create your account. To minimize latency, you should create an account either near you or near the users and applications that will access the data in this account.

At the "Resource Group" field, click the "Create new" link to display the "Create Resource Group" dialog, as shown in Fig. 3.

Fig03-NewResourceGroup
Fig. 3

Enter a unique name for your new resource group and click the [OK] button to close the dialog and return to the previous blade.

Review your Storage Account settings and click the [Create] button to create your new storage account.

After a few seconds, you will be able to access the properties of your Storage Account, manage the account, and connect to the account.

The "Overview" tab (Fig. 4) displays information about the account, along with links to create and manage Blobs, Files, Tables, and Queues.

Fig04-StorageOverview
Fig. 4

Click the "Blobs" link in the "Services" section of the "Overview" tab to create and manage blobs and containers, as shown in Fig. 5.

Fig05-BlobsBlade
Fig. 5

Click the "Files" link in the "Services" section of the "Overview" tab to create and manage files, as shown in Fig. 6.

Fig06-FilesBlade
Fig. 6

Click the "Tables" link in the "Services" section of the "Overview" tab to create and manage tables , as shown in Fig. 7.

Fig07-TablesBlade
Fig. 7

Click the "Queues" link in the "Services" section of the "Overview" tab to create and manage queues, as shown in Fig. 8.

Fig08-QueuesBlade
Fig. 8

Azure Storage gives you many option for storing data in the cloud.

Tuesday, 11 September 2018 09:37:00 (GMT Daylight Time, UTC+01:00)
# Monday, 10 September 2018
Monday, 10 September 2018 09:29:00 (GMT Daylight Time, UTC+01:00)
# Sunday, 09 September 2018

I may be the last person in America to see Billy Joel in concert. Despite not releasing an album in over two decades, Joel tours almost constantly, routinely filling stadiums and arenas. Recently, he performed a record 100th time at Madison Square Garden alone.

And Friday night, he performed for the seventh time at Wrigley Field in Chicago - also a concert record for this venue.

And I was there.

IMG_2164I bought my first Billy Joel album ("The Stranger") in 1977 and played it until the grooves ran out. It was one of the first albums I bout and I memorized every song. Joel must have known I was coming because he played six songs ("Movin' Out (Anthony's Song)", "Scenes from an Italian Restaurant", "Vienna", "Only the Good Die Young", and "She's Always a Woman") from "The Stranger". Interestingly, he did not play "Just the Way You Are", one of the biggest hits of the year.

In fact, he drew from nearly every one of his 12 studio albums. With 33 Top 40 hits and 23 Grammy nominations, he could not play all his hits; but he did play for two and a half hours, delighting a full crowd at the baseball stadium. He combined big hits like

and a surprising aria written by Puccini and sung by his guitarist Mike DelGuidice. Joel also brought background singer Crystal Taliefero to the front to sing "RESPECT" an Otis Redding composition made famous by the late Aretha Franklin.

He left the audience feeling appreciated by not only mentioning Chicago, but by playing parts of Sinatra's "Chicago (My Kind of Town)" twice during his set. His drummer wore a Cubs jersey and shouted "Good luck in the playoffs!" as he left the stage.

For me, the highlights were a moving rendition of "New York State of Mind", a song I've always believed that Joel never got enough credit for; and "River of Dreams" featuring its complex rhythms - a song Joel surprised us with after a series of pop songs in the 1980s.

Everyone sang along and rocked back and forth as the piano man sang his classic "Piano Man" before bowing and waving Good Night. But, of course he returned to the stage and delivered 4(!) upbeat songs for his encore.

Two and a half hours is a long time for anyone to play, particular for a singer in his 60's; but Billy Joel's voice sounds as strong as ever. And his piano playing is still outstanding. And with a solid group of musicians behind him, the crowd left satisfied.

And now that I've seen Billy Joel, I suppose that everyone has.

Sunday, 09 September 2018 07:01:00 (GMT Daylight Time, UTC+01:00)
# Saturday, 08 September 2018

The Microsoft Bot Framework provides a template that creates a simple bot: The "Bot Builder Echo Bot". It is named this because it echoes back any text you send it, along with the length of that text.

The bot is simply an ASP.NET Web API application that exposes web services to Channel clients. As such, it uses the default routing for Web API applications of "api/controllername"

where controllername is the name of the Controller class with the word “Controller” removed.

A channel will submit an HTTP POST to a URL containing "api/message", which will call the Post method in the MessagesController class. Let's take a look at this class.

    [BotAuthentication]
    public class MessagesController : ApiController
    {
        /// <summary>
        /// POST: api/Messages
        /// Receive a message from a user and reply to it
        /// </summary>
        public async Task<HttpResponseMessage> Post([FromBody]Activity activity)
        {
            if (activity.GetActivityType() == ActivityTypes.Message)
            {
                await Conversation.SendAsync(activity, () => new Dialogs.RootDialog());
            }
            else
            {
                HandleSystemMessage(activity);
            }
            var response = Request.CreateResponse(HttpStatusCode.OK);
            return response;
        }

        private Activity HandleSystemMessage(Activity message)
        {
            string messageType = message.GetActivityType();
            if (messageType == ActivityTypes.DeleteUserData)
            {
                // Implement user deletion here
                // If we handle user deletion, return a real message
            }
            else if (messageType == ActivityTypes.ConversationUpdate)
            {
                // Handle conversation state changes, like members being added and removed
                // Use Activity.MembersAdded and Activity.MembersRemoved and Activity.Action for info
                // Not available in all channels
            }
            else if (messageType == ActivityTypes.ContactRelationUpdate)
            {
                // Handle add/remove from contact lists
                // Activity.From + Activity.Action represent what happened
            }
            else if (messageType == ActivityTypes.Typing)
            {
                // Handle knowing that the user is typing
            }
            else if (messageType == ActivityTypes.Ping)
            {
            }

            return null;
        }
  

Listing 1

A channel posts an activity to the service. If the activity type is anything other than "Message", it calls the HandleSystemMessage method. Notice that nothing is actually implemented in this method. You will have to write this yourself.

However, if the activity type is "Message", the Post method asynchronously calls the Dialogs.RootDialog class. Here is that class:

    [Serializable]
    public class RootDialog : IDialog<object>
    {
        public Task StartAsync(IDialogContext context)
        {
            context.Wait(MessageReceivedAsync);

            return Task.CompletedTask;
        }

        private async Task MessageReceivedAsync(IDialogContext context, IAwaitable<object> result)
        {
            var activity = await result as Activity;

            // Calculate something for us to return
            int length = (activity.Text ?? string.Empty).Length;

            // Return our reply to the user
            await context.PostAsync($"You sent {activity.Text} which was {length} characters");

            context.Wait(MessageReceivedAsync);
        }
    }
  

Listing 2

The StartAsync method is called first, which waits for a message to be completely received; then, calls the MessageReceivedAsync method.

This method reads the text of the activity and forms a string that contains the original message text and the length of that text. It then Posts this back to the channel with the line below:

await context.PostAsync($"You sent {activity.Text} which was {length} characters");

Finally, it waits for the next message to come in via the following line

context.Wait(MessageReceivedAsync);

The rest of this project is Web API plumbing, setting up things like routing and where to store state information.

As you can see, this is a fairly simple architecture built on an established framework (Web API). This makes it easier to understand and to modify.

This simple bot is a good starting place that demonstrates the structure of a bot. You can modify this dialog, add new dialogs, or modify the logic as suits your purposes.

Saturday, 08 September 2018 09:15:00 (GMT Daylight Time, UTC+01:00)
# Friday, 07 September 2018

Microsoft Visual Studio makes it very simple to create a new bot with the Microsoft Bot Framework.

Launch Visual Studio 2017 (You can download a free version here)

From the menu, select File | New | Project.

The "New Project" dialog displays, as shown in Fig. 1.

Fig01-NewProjectDialog
Fig. 1

Create a new "Bot Builder Echo Bot" project. From this dialog, you find this template by either expanding Visual C# | Bot Framework in the navigation tree on the left; or by entering "Bot Builder" in the search box at the top right.

Enter a name and location for your project and click the [OK] button.

When the project loads, select Build | Rebuild Solution from the menu to compile the solution and validate all references.

This simple bot receives a message and echoes back the same message, along with the length of that message. The relevant code is in the MessageReceivedAsync method of the RootDialog class, which is shown in Listing 1.

private async Task MessageReceivedAsync(IDialogContext context, IAwaitable<object> result)
{
  var activity = await result as Activity;

  // Calculate something for us to return
  int length = (activity.Text ?? string.Empty).Length;

  // Return our reply to the user
  await context.PostAsync($"You sent {activity.Text} which was {length} characters");

  context.Wait(MessageReceivedAsync);
}
  

Listing 1

After the solution is rebuilt, select Debug | Start Debugging from the menu to launch the bot. This will host a web service and a web page on your local machine. The web page will display in your browser, as shown in Fig. 2.

Fig02-BotRunning
Fig. 2

Note the URL of this page, which will look something like

http://localhost:nnnn/

where nnnn is the port number on which this web application is running.

You can call the bot's web service from many sources; but the easiest way to test it is with the Bot Framework emulator. You can find detailed information on this emulator here. You can download the latest version of the emulator from here, as shown in Fig. 3.

Fig03-DownloadBotFramework
Fig. 3

Download and run the most recent EXE file from this site to install the emulator on your local PC.

Launch the emulator to view the form shown in Fig. 4.

Fig04-BotFrameworkEmulator
Fig. 4

Click the [Open Bot] button on the "Welcome" tab of the emulator. A JSON file containing configuration about your bot was created for you and saved in the Visual Studio project folder. Its root name will be the same as the project, but it will have a ".bot" extension. Select this file and click the [Choose file] button, as shown in Fig. 5.

Fig05-OpenBotConfiguration
Fig. 5

The format of this file looks like the sample in Listing 1.

{
  "name": "MyBot",
  "secretKey": "",
  "services": [
    {
      "appId": "",
      "id": "http://localhost:3978/api/messages",
      "type": "endpoint",
      "appPassword": "",
      "endpoint": "http://localhost:3978/api/messages",
      "name": "MyBot"
    }
  ]
}
  

Listing 2

NOTE: When you return to the emulator, you can click the [Open Bot] button or click a shortcut link below the button to re-load the configuration.

After you save (or open) a configuration file, a "Live Chat" tab displays in the emulator, as shown in Fig. 6

Fig06-BotEmulatorRunning
Fig. 6

To test the bot, type a message into the "Type your message" textbox and press [ENTER]. Your message will display. A short time later, a response will display with your message, followed by the number of characters.

You can find this code in the SimpleBotDemo project of the Bot-Framework-Demos on my GitHub page.

In future articles, I will show you how to enhance your bot, deploy it to a public site, such as Azure, and call it from a variety of clients. But this is the simplest way to create, run, and test a bot.

Friday, 07 September 2018 09:05:00 (GMT Daylight Time, UTC+01:00)
# Thursday, 06 September 2018

GCast 12:

Azure Media Services: Live Streaming

Learn how to Upload, Encode, and Share a Video using Azure Media Services.

Thursday, 06 September 2018 09:22:00 (GMT Daylight Time, UTC+01:00)
# Wednesday, 05 September 2018

The Bots are taking over.

A chatbot is an automated process that can respond to questions or commands via voice or text. They are useful for simulating a conversation and handling some of the tasks that would ordinarily require a call center staffed with people. Uses include processing orders, checking status, and answering questions. If you can understand the question and figure out the answer, you can probably write a chatbot to handle it.

Advantages of Chatbots

Chatbots have some advantages over human interactions.

Operating Cost. Labor costs tend to be among the highest cost of any business and operators in a call center can be expensive. Once deployed, a chatbot's cost tends to be a fraction of the cost of paying a person.

Availability. Chatbots can scale up to increase demand, so that users, customers, and potential customers do not need to wait for the next available operator. If activity rises unexpectedly, it is far easier and faster to deploy new resources to support a chatbot than it is to bring in more people to answer calls.

Speed. A computer can often look up information much faster than a human can.

Flexibility. A chatbot can be wired up to a variety of communication channels, allowing you to use the same code base to connect to different clients in different ways. For example, you can write a chatbot and connect to users via Facebook, SMS, a web page, or any number of other platforms.

Disadvantages of Chatbots

Of course, chatbots will not solve all your problems. Here are some disadvantages.

Startup cost. There tends to be a larger startup costs for chatbots. They must be programmed, tested, and deployed. This cost is only justified if they will yield sufficient savings over the lifetime the chatbot is deployed. Tools like the Microsoft Bot Framework can reduce development time; but that early cost still exists.

Impersonal. Chatbots are not people and some customers prefer talking to a human being over talking to a machine. The cost of human interaction is worth it when you wish to cultivate a personal relationship with your customer or partner. 

Too human. With advances in natural language processing, it can be difficult for people to determine whether they are interacting with a person or a computer. Some people are turned off by this ambiguity. They feel cheated when a computer represents itself as a person.

Real-time flexibility. For many interactions, it is impossible to anticipate every potential question or command from your customers. Humans can react to these very well. Generally, you need to program a chatbot to anticipate each question or command.

Finally

There is no question that chatbots are becoming more and more important as companies interact with their customers. But how your company uses them can determine how successful your chatbot strategy is.

Some scenarios call for starting an interaction with a chatbot; then bringing in a person only if the chatbot cannot adequately help the customer.

Understanding the needs of your customer and the problem you are trying to solve is a key to effective use of chatbots.

Wednesday, 05 September 2018 09:02:00 (GMT Daylight Time, UTC+01:00)