# Friday, June 14, 2019

In the last few articles, I introduced the OCR Service of the Cognitive Services Computer Vision API. The OCR service is a general-purpose tool for detecting text in an image. But this tool is only useful if you want to do something with that text. Often it is easier to figure out how to process recognized text if you know something about the image.

Enter receipt-api - an open source project that builds on the Cognitive Services API to recognize information in a store receipt.

You can download this project at https://github.com/nzregs/receipt-api.

Compile and run it in Visual Studio and you have a web service that you can call by submitting an HTTP POST to the following URL:

http://localhost:xxxxx/api/values

where xxxxx is the port number on which the web service is running. You can spot this port easily because a browser launches when the app runs and the port number is in the URL, as shown in Fig. 1.

ra01-Browser
Fig. 1

Before Testing

In order to test the API, you will need an image of a receipt. You can take a photo with your phone and copy it to your computer.

You will also need to create a Computer Vision service in Azure, as described here.

Finally, you will also need to make a change to the receipt-api project. Open the solution and rename Sample-Secrets_cs.txt to Secrets.cs. The code in this file is shown in listing 1.

Listing 1:

public class Secrets

{
    // rename this file to Secrets.cs
    // update the constants below with your API Key and API Endpoint
    public const string apikey = "28737opek;jwlbjksgui3y2[pik";
    public const string apiendpoint_ocr = @"https://australiaeast.api.cognitive.microsoft.com/vision/v1.0/ocr";
}
  

Replace the key and api endpoint with the key and endpoint in the service associated with your cognitive service.

Testing the Receipt API

A simple way to test any API is with Postman, a free tool available at https://www.getpostman.com/.

Download, install, and run Postman.

With the receipt-api service running, create a new request in Postman consisting of a POST to the receipt-api service URL, as shown in Fig. 2.

ra02-Postman
Fig. 2

On the "Headers" tab, add a header row with NAME = Content-Type and VALUE = application/octet-stream, as shown in Fig. 3.

ra03-Headers
Fig. 3

On the "Body" tab, click the [Select File] button and select the photo of the receipt from your computer, as shown in Fig. 4.

ra04-Body
Fig. 4

Click the [Send] and wait for a response to appear. If all goes well, you will see something like Fig. 5.

ra05-Response
Fig. 5

If you have used the OCR service, you will notice that this response looks identical to the response from that service. But scroll to the bottom, as shown in Fig. 6 and you will see information specific to receipts.

ra06-Response
Fig. 6

This is from the receipt shown in Fig. 7.

ra07-Receipt
Fig. 7

How it works

The solution works by first calling the Cognitive Services OCR service; then, looping through each line and word, looking for patterns. It uses regular expressions to find these patterns. Below is the code to find the date in the recognized text:

Listing 2:

static string ExtractDate(string line) 
 { 
    string receiptdate = ""; 
    // match dates "01/05/2018" "01-05-2018" "01-05-18" "01 05 18" "01 05 2018" 
    string pat = @"\s*((31([-/ .])((0?[13578])|(1[02]))\3(\d\d)?\d\d)|((([012]?[1-9])|([123]0))([-/ .])((0?[13-9])|(1[0-2]))\12(\d\d)?\d\d)|(((2[0-8])|(1[0-9])|(0?[1-9]))([-/ .])0?2\22(\d\d)?\d\d)|(29([-/ .])0?2\25(((\d\d)?(([2468][048])|([13579][26])|(0[48])))|((([02468][048])|([13579][26]))00))))\s*"; 
    foreach (Match in Regex.Matches(line, pat)) 
    { 
        receiptdate = match.Value.Trim(); 
            receiptdate = receiptdate.Replace("-", "/"); 
         receiptdate = receiptdate.Replace(".", "/"); 
        receiptdate = receiptdate.Replace(" ", "/"); 
    }

    // didnt find date?  now we'll try searching with month names.  03 OCT 2017, 03 October 2017 etc 
    if (receiptdate == "") 
    { 
        pat = @"((31(?![-/ .](Feb(ruary)?|Apr(il)?|June?|(Sep(?=\b|t)t?|Nov)(ember)?)))|((30|29)(?![-/ .]Feb(ruary)?))|(29(?=[-/ .]Feb(ruary)?[-/ .](((1[6-9]|[2-9]\d)(0[48]|[2468][048]|[13579][26])|((16|[2468][048]|[3579][26])00)))))|(0?[1-9])|1\d|2[0-8])[-/ .](Jan(uary)?|Feb(ruary)?|Ma(r(ch)?|y)|Apr(il)?|Ju((ly?)|(ne?))|Aug(ust)?|Oct(ober)?|(Sep(?=\b|t)t?|Nov|Dec)(ember)?)[-/ .]((1[6-9]|[2-9]\d)\d{2})";

        foreach (Match in Regex.Matches(line, pat, RegexOptions.IgnoreCase)) 
        { 
            receiptdate = match.Value.Trim(); 
            receiptdate = receiptdate.Replace("/", "-"); 
            receiptdate = receiptdate.Replace(".", "-"); 
            receiptdate = receiptdate.Replace(" ", "-"); 
         } 
    }

    return receiptdate; 
}
  

Limitations

This tool is not perfect.

It is incomplete. Although the model supports a Business Name and a Tax Total, it looks like the logic to extract this information has not yet been written.

Note that it is an Open Source project and you are welcome to contribute and submit a Pull Request. If this logic is important to your project, write it and share it with the world.

The solution is also limited by the capabilities of the OCR service it calls. However, my experience is that this service becomes more accurate as time goes on.

The results are best with a clear, high-contrast receipt. If your receipt is wrinkled or faded or has a watermark, the OCR will be degraded, effecting any analysis of the recognized text.

Strengths

The receipt-api project does provide several advantages:

  • It is simple to use.
  • It can scale when deployed to Azure.
  • It is an open source project, so other developers (including you) can improve it.
  • It is free.
  • It has an MIT license and can be used without restriction.

The receipt-api open source project provides a simple way to extract data from a receipt.

Friday, June 14, 2019 9:22:00 AM (GMT Daylight Time, UTC+01:00)
Comments are closed.