# Tuesday, June 18, 2019

CTRL+V has been in Windows since the beginning: After copying something to the Windows clipboard (via CTRL+C or some other method), hold down the CTRL key and press V to insert that something at the current cursor location.

But I learned today about a new feature: WINDOWS + V.

Hold down the WINDOWS key (Fig. 1) and press V.

wv01-WindowsKey
Fig. 1

This will bring up a context menu, listing the last few items added to the clipboard, as shown in Fig. 2.

wv02-ContextMenu
Fig. 2

You can then select from this list which item to insert at the current cursor position.

The context menu even lists the time the item was added to the clipboard.

This is useful if you need to copy several items before pasting them. But, the most useful use case is when you accidentally copy something to the clipboard without thinking you might overwrite a previous item copied there. Now you have some time to still use that previously overwritten item.

I'm unclear how long items stay in this clipboard list, but I like this advantage.

Tuesday, June 18, 2019 2:11:00 AM (GMT Daylight Time, UTC+01:00)
# Monday, June 17, 2019

Episode 568

Heather Wilde on Anticipatory Design

Heather Wilde discusses how to machine learning with user interfaces and user experience to craft a more personalized experience between a person and the products and services they use.

https://twitter.com/heathriel

Monday, June 17, 2019 8:21:00 AM (GMT Daylight Time, UTC+01:00)
# Sunday, June 16, 2019

ArrowOfGodArrow of God is part of Chinua Achebe's African Trilogy. Although it is Achebe's third novel, chronologically, it is second in the trilogy. The first book - Things Fall Apart - took place as the English colonizers were arriving in west Africa; book 2 - No Longer At Ease - takes place near the end of the colonization period; and "Arrow of God"'s story happens in the 1920's - at the height of European colonization.

Unlike the first two books, this one does not focus on Okonkwo or his descendants.
Arrow of God tells the story of Ezeulu, a high priest of the god Ulu in colonial Nigeria. Ezeulu's tribe goes to war with another tribe - a result of a perceived insult and a dispute over land ownership. The British governor steps in, ends the conflict, and burns everyone's guns. It is this assertion of control by the British over the local population that is at the heart of the novel.

The British are in Africa to implement their legal system and their culture and their religion on the native population and they use any means to do so. They implement "Local Rule", installing an African in a position of authority, but controlling that man so he works in their interests.

The British government bureaucracy makes them terribly inefficient, making their job harder; but the infighting among individuals, villages, and tribes of the Africans makes them easier to manipulate. As part of England's efforts at influencing the local population, missionaries are trying to convert the natives to Christianity. Many resist this change in culture; but they are aided when a failure of the yam crop leads to famine and praying to the god Ulu does not help. The religious conversions are a microcosm of the colonizers' efforts to implement their culture on the Africans, subsuming the existing culture.

Ezeulu stands in the center of it all. He wants to better understand the British and sends his son to work with them to gain more information; but he rejects their offer to make him a Local Ruler. Still, his own people are suspicious of his motives, feeling he has given too much to the white invaders. There is conflict between Ezeulu and

Arrow of God is the story of conflict and influence; of people trying to hold onto their traditions in the face of a tidal wave of change; and of the hope on which people place their faith; of how culture was supplanted in a short time. It is the story of toe loss of the Igbo cultural identity.

Achebe was probably the first English language writer to portray events in Africa from the point of view of the Africans.

The book is told in an eloquent style, peppered with many African proverbs. This one sums it up best:

"When brothers fight to death a stranger inherits their father's estate"

Sunday, June 16, 2019 9:06:00 AM (GMT Daylight Time, UTC+01:00)
# Saturday, June 15, 2019

ShelteringSkySometimes, I dream about traveling to new places with no responsibilities and no worries about money.

The Sheltering Sky by Paul Bowles may have cured me of that dream.

"The Sheltering Sky" tells the story of 3 idle rich people - husband and wife, Port and Kit, along with their friend Tunner - who decide to explore north Africa after World War II. They don't have a plan and so drift from city to city seeing the sites and trying to overcome their boredom. They insist this makes them "travelers", rather than "tourists". They drift through a land of sweltering temperatures and dust storms and bedbugs and lice and biting flies.

The story turns tragic as the trio splits up, Port falls gravely ill, and Kit becomes lost in the desert. But this book isn't defined by plot twists or great character developments. It is an existential tale about people who appear to be drifting through life, with no purpose and no direction. What did this trio contribute to the world on their travels? What did they gain for themselves? What meaning did their lives have? They are searching for something, but even they don't know what that is. And none of them ever finds it.

Every character seems bent on self-destruction in this story. Almost everyone we meet is unlikable - from the 3 main characters to the constantly-arguing-and-probably-incestuous young man and his mother they encounter to a traveler who kidnaps and rapes Kit before adding her to his harem. Port and Kit are each unfaithful to one another during their trip, but only Kit feels any guilt about it.

Some will take offense at Kit's reaction to her rape (she accepts it and even comes to enjoy it), but I saw this as evidence of her decreasing mental stability.

If you are looking for a good travelogue of Algeria and Morocco, skip this one. If you want a commentary on the state of the human condition, this is a pretty good one.

Saturday, June 15, 2019 9:46:00 AM (GMT Daylight Time, UTC+01:00)
# Friday, June 14, 2019

In the last few articles, I introduced the OCR Service of the Cognitive Services Computer Vision API. The OCR service is a general-purpose tool for detecting text in an image. But this tool is only useful if you want to do something with that text. Often it is easier to figure out how to process recognized text if you know something about the image.

Enter receipt-api - an open source project that builds on the Cognitive Services API to recognize information in a store receipt.

You can download this project at https://github.com/nzregs/receipt-api.

Compile and run it in Visual Studio and you have a web service that you can call by submitting an HTTP POST to the following URL:

http://localhost:xxxxx/api/values

where xxxxx is the port number on which the web service is running. You can spot this port easily because a browser launches when the app runs and the port number is in the URL, as shown in Fig. 1.

ra01-Browser
Fig. 1

Before Testing

In order to test the API, you will need an image of a receipt. You can take a photo with your phone and copy it to your computer.

You will also need to create a Computer Vision service in Azure, as described here.

Finally, you will also need to make a change to the receipt-api project. Open the solution and rename Sample-Secrets_cs.txt to Secrets.cs. The code in this file is shown in listing 1.

Listing 1:

public class Secrets

{
    // rename this file to Secrets.cs
    // update the constants below with your API Key and API Endpoint
    public const string apikey = "28737opek;jwlbjksgui3y2[pik";
    public const string apiendpoint_ocr = @"https://australiaeast.api.cognitive.microsoft.com/vision/v1.0/ocr";
}
  

Replace the key and api endpoint with the key and endpoint in the service associated with your cognitive service.

Testing the Receipt API

A simple way to test any API is with Postman, a free tool available at https://www.getpostman.com/.

Download, install, and run Postman.

With the receipt-api service running, create a new request in Postman consisting of a POST to the receipt-api service URL, as shown in Fig. 2.

ra02-Postman
Fig. 2

On the "Headers" tab, add a header row with NAME = Content-Type and VALUE = application/octet-stream, as shown in Fig. 3.

ra03-Headers
Fig. 3

On the "Body" tab, click the [Select File] button and select the photo of the receipt from your computer, as shown in Fig. 4.

ra04-Body
Fig. 4

Click the [Send] and wait for a response to appear. If all goes well, you will see something like Fig. 5.

ra05-Response
Fig. 5

If you have used the OCR service, you will notice that this response looks identical to the response from that service. But scroll to the bottom, as shown in Fig. 6 and you will see information specific to receipts.

ra06-Response
Fig. 6

This is from the receipt shown in Fig. 7.

ra07-Receipt
Fig. 7

How it works

The solution works by first calling the Cognitive Services OCR service; then, looping through each line and word, looking for patterns. It uses regular expressions to find these patterns. Below is the code to find the date in the recognized text:

Listing 2:

static string ExtractDate(string line) 
 { 
    string receiptdate = ""; 
    // match dates "01/05/2018" "01-05-2018" "01-05-18" "01 05 18" "01 05 2018" 
    string pat = @"\s*((31([-/ .])((0?[13578])|(1[02]))\3(\d\d)?\d\d)|((([012]?[1-9])|([123]0))([-/ .])((0?[13-9])|(1[0-2]))\12(\d\d)?\d\d)|(((2[0-8])|(1[0-9])|(0?[1-9]))([-/ .])0?2\22(\d\d)?\d\d)|(29([-/ .])0?2\25(((\d\d)?(([2468][048])|([13579][26])|(0[48])))|((([02468][048])|([13579][26]))00))))\s*"; 
    foreach (Match in Regex.Matches(line, pat)) 
    { 
        receiptdate = match.Value.Trim(); 
            receiptdate = receiptdate.Replace("-", "/"); 
         receiptdate = receiptdate.Replace(".", "/"); 
        receiptdate = receiptdate.Replace(" ", "/"); 
    }

    // didnt find date?  now we'll try searching with month names.  03 OCT 2017, 03 October 2017 etc 
    if (receiptdate == "") 
    { 
        pat = @"((31(?![-/ .](Feb(ruary)?|Apr(il)?|June?|(Sep(?=\b|t)t?|Nov)(ember)?)))|((30|29)(?![-/ .]Feb(ruary)?))|(29(?=[-/ .]Feb(ruary)?[-/ .](((1[6-9]|[2-9]\d)(0[48]|[2468][048]|[13579][26])|((16|[2468][048]|[3579][26])00)))))|(0?[1-9])|1\d|2[0-8])[-/ .](Jan(uary)?|Feb(ruary)?|Ma(r(ch)?|y)|Apr(il)?|Ju((ly?)|(ne?))|Aug(ust)?|Oct(ober)?|(Sep(?=\b|t)t?|Nov|Dec)(ember)?)[-/ .]((1[6-9]|[2-9]\d)\d{2})";

        foreach (Match in Regex.Matches(line, pat, RegexOptions.IgnoreCase)) 
        { 
            receiptdate = match.Value.Trim(); 
            receiptdate = receiptdate.Replace("/", "-"); 
            receiptdate = receiptdate.Replace(".", "-"); 
            receiptdate = receiptdate.Replace(" ", "-"); 
         } 
    }

    return receiptdate; 
}
  

Limitations

This tool is not perfect.

It is incomplete. Although the model supports a Business Name and a Tax Total, it looks like the logic to extract this information has not yet been written.

Note that it is an Open Source project and you are welcome to contribute and submit a Pull Request. If this logic is important to your project, write it and share it with the world.

The solution is also limited by the capabilities of the OCR service it calls. However, my experience is that this service becomes more accurate as time goes on.

The results are best with a clear, high-contrast receipt. If your receipt is wrinkled or faded or has a watermark, the OCR will be degraded, effecting any analysis of the recognized text.

Strengths

The receipt-api project does provide several advantages:

  • It is simple to use.
  • It can scale when deployed to Azure.
  • It is an open source project, so other developers (including you) can improve it.
  • It is free.
  • It has an MIT license and can be used without restriction.

The receipt-api open source project provides a simple way to extract data from a receipt.

Friday, June 14, 2019 9:22:00 AM (GMT Daylight Time, UTC+01:00)
# Thursday, June 13, 2019

GCast 52:

Using hilite.me to Format Code as HTML

The online tool at hilite.me allows me to format code in just about any language and paste it into my blog.

GCast | HTML5 | Screencast | Video | Web
Thursday, June 13, 2019 9:03:00 AM (GMT Daylight Time, UTC+01:00)
# Wednesday, June 12, 2019

In a previous article, I showed how to use the Microsoft Cognitive Services Computer Vision API to perform Optical Character Recognition (OCR) on a document containing a picture of text. We did so by making an HTTP POST to a REST service.

If you are developing with .NET languages, such as C# Visual Basic, or F#, a NuGet Package makes this call easier. Classes in this package abstract the REST call, so can write less and simpler code; and strongly-typed objects allow you to make the call and parse the results more easily.


To get started, you will first need to create a Computer Vision service in Azure and retrieve the endpoint and key, as described here.

Then, you can create a new C# project in Visual Studio. I created a WPF application, which can be found and downloaded at my GitHub account. Look for the project named "OCR-DOTNETDemo". Fig. 1 shows how to create a new WPF project in Visual Studio.

od01-FileNewProject
Fig. 1

In the Solution Explorer, right-click the project and select "Manage NuGet Packages", as shown in Fig. 2.

od02-ManageNuGet
Fig. 2

Search for and install the "Microsoft.Azure.CognitiveServices.Vision.ComputerVision", as shown in Fig. 3.

od03-NuGet
Fig. 3

The important classes in this package are:

  • OcrResult
    A class representing the object returned from the OCR service. It consists of an array of OcrRegions, each of which contains an array of OcrLines, each of which contains an array of OcrWords. Each OcrWord has a text property, representing the text that is recognized. You can reconstruct all the text in an image by looping through each array.
  • ComputerVisionClient
    This class contains the RecognizePrintedTextInStreamAsync method, which abstracts the HTTP REST call to the OCR service.
  • ApiKeyServiceClientCredentials
    This class constructs credentials that will be passed in the header of the HTTP REST call.

Create a new class in the project named "OCRServices" and make its scope "internal" or "public"

Add the following "using" statements to the top of the class:

using Microsoft.Azure.CognitiveServices.Vision.ComputerVision;
using Microsoft.Azure.CognitiveServices.Vision.ComputerVision.Models;
using System.IO;
  


Add the following methods to this class:

Listing 1:

internal static async Task<OcrResult> UploadAndRecognizeImageAsync(string imageFilePath, OcrLanguages language) 
 { 
    string key = "xxxxxxx"; 
    string endPoint = "https://xxxxx.api.cognitive.microsoft.com/"; 
    var credentials = new ApiKeyServiceClientCredentials(key);

    using (var client = new ComputerVisionClient(credentials) { Endpoint = endPoint }) 
    { 
        using (Stream imageFileStream = File.OpenRead(imageFilePath)) 
        { 
             OcrResult ocrResult = await client.RecognizePrintedTextInStreamAsync(false, imageFileStream, language); 
            return ocrResult; 
        } 
    } 
}

internal static async Task<string> FormatOcrResult(OcrResult ocrResult) 
{ 
    var sb = new StringBuilder(); 
    foreach(OcrRegion region in  ocrResult.Regions) 
    { 
        foreach (OcrLine line in region.Lines) 
        { 
             foreach (OcrWord word in line.Words) 
            { 
                 sb.Append(word.Text); 
                sb.Append(" "); 
            } 
            sb.Append("\r\n"); 
        } 
         sb.Append("\r\n\r\n"); 
    } 
    return sb.ToString(); 
}
  

The UploadAndRecognizeImageAsync method calls the HTTP REST OCR service (via the NuGet library extractions) and returns a strongly-typed object representing the results of that call. Replace the key and the endPoint in this method with those associated with your Computer Vision service.

The FormatOcrResult method loops through each region, line, and word of the service's return object. It concatenates the text of each word, separating words by spaces, lines by a carriage return and line feed, and regions by a double carriage return / line feed.

Add a Button and a TextBlock to the MainWindow.xaml form.

In the click event of that button add the following code.

Listing 2:

private async void GetText_Click(object sender, RoutedEventArgs e) 
{ 
    string imagePath = @"xxxxxxx.jpg"; 
    OutputTextBlock.Text = "Thinking…"; 
    var language = OcrLanguages.En; 
    OcrResult ocrResult =  await OCRServices.UploadAndRecognizeImageAsync(imagePath, language); 
     string resultText = await OCRServices.FormatOcrResult(ocrResult); 
    OutputTextBlock.Text = resultText; 
 }
  


Replace xxxxxxx.jpg with the full path of an image file on disc that contains pictures of text.

You will need to add the following using statement to the top of MainWindow.xaml.cs.

using Microsoft.Azure.CognitiveServices.Vision.ComputerVision.Models;
  

If you like, you can add code to allow users to retrieve an image and display that image on your form. This code is in the sample application from my GitHub repository, if you want to view it.

Running the form should look something like Fig. 4.

od04-RunningApp
Fig. 4

Wednesday, June 12, 2019 9:46:00 AM (GMT Daylight Time, UTC+01:00)
# Tuesday, June 11, 2019

In a previous article, I described the details of the OCR Service, which is part of the Microsoft Cognitive Services Computer Vision API.

To make this API useful, you need to write some code and build an application that calls this service.

In this article, I will show an example of a JavaScript application that calls the OCR web service.

If you want to follow along, you can find all the code in the "OCRDemo" project, included in this set of demos.

To use this demo project, you will first need to create a Computer Vision API service, as described here.

Read the project's read.me file, which explains the setup you need to do in order to run this with your account.

If you open index.html in the browser, you will see that it displays an image of a poem, along with some controls on the left:

  • A dropdown list to change the poem image
  • A dropdown list to select the language of the poem text
  • A [Get Text] button that calls the web service.

Fig. 1 shows index.html when it first loads:

oj01-WebPage
Fig. 1

    Let's look at the JavaScript that runs when you click the [Get Text] button. You can find it in script.js

    print 'hello world!'$("#GetTextFromPictureButton").click(function () {
         var outputDiv = $("#OutputDiv");
         outputDiv.text("Thinking…");
         var url = $("#ImageUrlDropdown").val();
         var language = $("#LanguageDropdown").val();
    
        try {
             var computerVisionKey = getKey();
         }
         catch(err) {
             outputDiv.html(missingKeyErrorMsg);
             return;
         }
    
        var webSvcUrl = "https://westcentralus.api.cognitive.microsoft.com/vision/v2.0/ocr";
        webSvcUrl = webSvcUrl + "?language=" + language;
        $.ajax({
            type: "POST",
            url: webSvcUrl,
            headers: { "Ocp-Apim-Subscription-Key": computerVisionKey },
            contentType: "application/json",
            data: '{ "Url": "' + url + '" }'
        }).done(function (data) {
            outputDiv.text("");
    
            var regionsOfText = data.regions;
            for (var r = 0; r < regionsOfText.length; h++) {
                var linesOfText = data.regions[r].lines;
                for (var l = 0; l < linesOfText.length; l++) {
                    var output = "";
    
                    var thisLine = linesOfText[l];
                    var words = thisLine.words;
                    for (var w = 0; w < words.length; w++) {
                        var thisWord = words[w];
                        output += thisWord.text;
                        output += " ";
                    }
                    var newDiv = "<div>" + output + "</div>";
                    outputDiv.append(newDiv);
    
                }
                outputDiv.append("<hr>");
            }
    
        }).fail(function (err) {
            $("#OutputDiv").text("ERROR!" + err.responseText);
        });
      

    This code uses jQuery to simplify selecting elements, but raw JavaScript would work just as well.

    On the page is an empty div with the id="OutputDiv"

    In the first two lines, we select this div and set its text to "Thinking…" while the web service is being called.

        var outputDiv = $("#OutputDiv");
        outputDiv.text("Thinking…");

    Next, we get the URL of the image containing the currently displayed poem and the selected language. These both come from the selected items of the two dropdowns.

        var url = $("#ImageUrlDropdown").val(); 
        var language = $("#LanguageDropdown").val();
      

    Then, we get the API key, which is in the getKey() function, which is stored in the getkey.js file. You will need to update this file yourself, adding your own key, as described in the read.me.

        try { 
            var computerVisionKey = getKey(); 
        } 
        catch(err) { 
            outputDiv.html(missingKeyErrorMsg); 
            return; 
        }
      

    Now, it's time to call the web service. My Computer Vision API service was created in the West Central US region, so I've hard-coded the URL. You may need  to change this, if you created your service in a different region.

    I add a querystring parameter to the URL to indicate the slected language.

    Then, I call the web service by submitting an HTTP POST request to the web service URL, passing in the appropriate headers and constructing a JSON document to pass in the request body.

        var webSvcUrl = "https://westcentralus.api.cognitive.microsoft.com/vision/v2.0/ocr";
        webSvcUrl = webSvcUrl + "?language=" + language;
        $.ajax({
            type: "POST",
            url: webSvcUrl,
            headers: { "Ocp-Apim-Subscription-Key": computerVisionKey },
            contentType: "application/json",
            data: '{ "Url": "' + url + '" }'
      

    Finally, I process the results when the HTTP response returns.

    JavaScript is a dynamic language, so I don't need to create any classes to identify the structure of the JSON that is returned; I just need to know the names of each property.

    The returned JSON contains an array of regions; each region contains an array of lines; and each line contains an array of words.

    In this simple example, I simply loop through each word in each line in each region, concatenating them together and adding some HTML to format line breaks.

    Then, I append this HTML to the outputDiv and follow it up with a horizontal rule to emphasize that it is the end.

        }).done(function (data) { 
            outputDiv.text("");
    
            var regionsOfText = data.regions; 
            for (var r = 0; r < regionsOfText.length; h++) { 
                var linesOfText = data.regions[r].lines; 
                for (var l = 0; l < linesOfText.length; l++) { 
                     var output = "";
    
                    var thisLine = linesOfText[l]; 
                    var words = thisLine.words; 
                     for (var w = 0; w < words.length; w++) { 
                         var thisWord = words[w]; 
                        output += thisWord.text; 
                        output += " "; 
                    } 
                     var newDiv = "<div>" + output + "</div>"; 
                     outputDiv.append(newDiv);
    
                } 
                outputDiv.append("<hr>"); 
            }
      

    I also, catch errors that might occur, displaying a generic message in the outputDiv, where the returned text would have been.

        catch(err) { 
            outputDiv.html(missingKeyErrorMsg); 
            return; 
        }
      

    Fig. 2 shows the results after a successful web service call.

    oj02-Results
    Fig. 2

    Try this yourself to see it in action. The process is very similar in other languages.

    Tuesday, June 11, 2019 9:11:00 AM (GMT Daylight Time, UTC+01:00)
    # Monday, June 10, 2019

    Episode 567

    Elton Stoneman on Docker

    Elton Stoneman describes how to manage containers using Docker on a local machine and in the cloud.

    Monday, June 10, 2019 9:52:00 AM (GMT Daylight Time, UTC+01:00)
    # Sunday, June 9, 2019

    ClaudiusTheGod"Claudius the God" continues Robert Graves's fictional translation of the Roman Emperor Claudius's autobiography that he began in "I, Claudius".

    The first novel ended with the assassination of Nero and with Claudius being unexpectedly and reluctantly elevated to Emperor.

    Claudius was born handicapped and undersized and weak and stuttering, so he remained mostly overlooked during his pre-Emperor years, avoiding the (often fatal) power struggles exercised by his family. So, he seems completely unprepared and unqualified for his role as supreme leader. Despite his initial reluctance, Claudius embraces the role, implementing his policies to modernize the Empire and sometimes executing his enemies without trial. During his 13-year reign, he successfully undoes much of the damage caused by the mad Caligula. Overall, he performed better than almost anyone expected.

    At first glance, this seems like a good deal for Claudius.

    But his life is damaged by his relationships with women - particularly with Messalina - his young and beautiful wife - who conspires against him and is unfaithful with literally hundreds of other men. After four marriages, Claudius remains unhappy.

    Once again, Graves does a good job of bringing to life the political intrigue and personal dramas of ancient Rome and her key players. Nearly everyone in this world with ambition is assassinated or plots assassinations or both. Claudius tries to stay above this and is mostly successful for most of his life.

    Previous Roman Emperors had embraced their role as god-king; but Claudius resisted this idea, insisting that he was not a god. In fact, he dreams of Rome eventually rejecting its recent incarnation as a dictatorship and returning to a Republic. His childhood friend the Hebrew king Herod Agrippa takes a different approach, declaring himself to be the prophesied Messiah and is ultimately killed for his arrogance.

    But, Claudius changes with time, as happens to so many with ultimate power. Near the end of his life, Claudius accepted his deification when he discovered the Britons were building temples to him. He begins plotting the succession of the throne - without the best interests of the Empire.

    The full title of this book is "Claudius, the God and his wife, Messalina" and Claudius's full name is Tiberius Claudius Drusus Nero Germanicus. Much like these names, Robert Graves shortens and simplifies the 13-year reign of Emperor Claudius.

    Sunday, June 9, 2019 9:11:00 AM (GMT Daylight Time, UTC+01:00)