# Tuesday, 26 June 2018

A Text Template Transformation Toolkit (or "T4") template is a text generation tool that mixes raw text and code to generate text as output.

Visual Studio contains tools to assist with the creation and execution of T4 templates.

To add a T4 template to your project, right-click on the project, and select Add | New Item from the context menu.

In the "Add New Item" dialog, the T4 templates are listed under the General category; but you can find them quickly by typing "T4" in the "Search Installed Templates" textbox.

There are two kinds of T4 templates: a Text Template and a Runtime Text Template. Each of these starts with a file with ".tt" extension, but is designed to generate at least one other file.

The Text Template regenerates the final output file every time you save the ".tt" file. This is a quick way to generate simple files, but it lacks flexibility.

The Runtime Text Template generates a VB or C# class that you can call from your code to generate the output file. We will focus on the Runtime Text Template in this article.

A Runtime Text Template consists of raw text (which will be exactly duplicated in the output file) and executable code, which the template will run. Any output of that code will be included in the output file. Code is delimited from the raw text by placing it between the <# and #> symbols. If you just want to output the results of an expression, you can surround that expression with the <#= and #> symbols.

For example, to output the current date and time, include the following line in your template.

<#=System.DateTime.Now #>
  

You can combine this with text as in the following example:

(This file was generated at <#=System.DateTime.Now #>!)
  

which would output something like the following:

(This file was generated at 06/25/2018 20:33:14!)

You can include more complex code between the <# and #> symbols, as in the following example:

<#
for (System.Int32 I = 0; I < 5; I++)
{
	// WriteLine is a utility function that outputs text. 
	WriteLine(I);
}
#>
  

The result of the above code is:

0
1
2
3
4

When you save a Runtime Text Template, Visual Studio generates code for a public .NET class with the same name as your tt file. You can instantiate this class as you would any other public class and call the TransformText() method to generate the output as a string.

For example, if I created a Runtime Text Template named "myT4Template.tt", the following code would generate the output and save it to a variable named "outputText".

var tt = new myT4Template();
string outputText = tt.TransformText();
  

T4 templates give you the flexibility to generate text and text files quickly and the flexibility to change the template without worrying about the associated data.

Tuesday, 26 June 2018 08:50:00 (GMT Daylight Time, UTC+01:00)
# Monday, 25 June 2018
Monday, 25 June 2018 09:10:00 (GMT Daylight Time, UTC+01:00)
# Sunday, 24 June 2018

MySQL is a popular relational database. Although the product is owned by Oracle, which offers several versions for purchase, it is an open source project and they offer a free "Community Edition.

There are several ways to install and start running MySQL.

You can download a copy from https://www.mysql.com/downloads/. The community edition offers install files for various platforms (an MSI file for Windows) that allows you install the database engine locally.

My preferred option is to install a database on Azure and store my data in the cloud. Because Azure offers MySQL as a service, I don't need to create a Virtual Machine or install the database engine anywhere. To create a MySQL service is Azure, navigate to the Azure portal and select Create a Resource | Databases | Azure Database for MySQL; then complete the blade that displays.

Whether using a cloud service or installing locally, you will be prompted to create an admin user and password. Remember these.

Once you have access to MySQL, you can use the command line to create and manage databases. On a local machine, open a command prompt; in Azure, you can click the "Cloud Shell" button at the top of the Azure portal.

To begin working with MySQL, you need to connect to the database engine with the following command:

mysql -h hostname -u username -p
  

where hostname is the name of the MySQL instance and username is the name of the admin user.

For example, if I have an Azure MySQL instance named "dgmysql" and a user named "dgiard", I would type the following:

mysql -h dgmysql.mysql.database.azure.com -u dgiard@dgmysql -p
  

When prompted, enter the appropriate password for the user.

The following commands are usfule for working with databases.

# Create a new database
CREATE DATABASE databasename;

# List all databases available
SHOW DATABASES;

# Switch to a specific database
USE publicspeaking;
  

Now you can start using standard SQL commands to work with database objects and data in the current database.

For example,

# Create a new database
CREATE DATABASE databasename;

# List all databases available
SHOW DATABASES;

# Switch to a specific database
USE publicspeaking;
  

For the most part, MySQL follows ANSI SQL syntax.

If you are familiar with another relational database, such as Microsoft SQL Server, working with MySQL will feel natural.

Sunday, 24 June 2018 07:16:00 (GMT Daylight Time, UTC+01:00)
# Saturday, 23 June 2018

On May 19, I delivered a presentation titled "How Cloud Computing Empowers a Data Scientist" at the Chicago AI & Data Science Conference.

I described ways that the cloud has accelerated the fields of data science, machine learning, and artificial intelligence; and I gave examples of Azure tools that facilitate development in these fields.

You can watch the video below or at https://youtu.be/H19IW6nykZo

Saturday, 23 June 2018 08:17:00 (GMT Daylight Time, UTC+01:00)
# Friday, 22 June 2018

Often, I need to restart an ASP.NET web site. There are many reasons but the most common reason (for me) is that I have updated a page and ASP.NET is displaying a cached version of that page, which does not reflect my changes.

There are multiple ways to restart an ASP.NET site; but, I always found it easiest to copy any file into the site's "bin" folder. In fact, I typically keep an empty text file named "_a.txt" in my local Documents folder for just this purpose. It is a bit of a hassle to open my FTP client (I use Filezilla), just to copy 1 file; so I recently decided to automate this process.

The Windows command prompt has an "FTP" command that that takes a "-s" argument with which you can provide a file containing FTP commands.

So I created the following text file named "RestartBlog.ftp"

open blahblahblah.ftp.azurewebsites.windows.net
myusername
mypassword
put _a.txt /site/wwwroot/bin/_a.txt
close
quit
  

In my file, I replaced blahblahblah.ftp.azurewebsites.windows.net with the FTP Host Name of my site, myusername with the FTP User Name of my site, and mypassword with the password associated with that username.

This opens a connection to my FTP site, logsin in and copies a local file named "_a.txt" to the site's bin folder. It's possible that your site's "bin" folder is in a different path, so you may need to change this. Next, it closes the connection and exits FTP mode.

After saving this file, I created another file in the same folder containing the following text:

echo ".">_a.txt
ftp -s:RestartBlog.ftp
del _a.txt
pause
  

This creates a small text file named "_a.txt"; then calls my file with FTP commands. After executing the FTP commands, this batch deletes the text file it created.

I added the "pause" at the end so that the command window would not immediately close and I could view the results and any errors that occurred. This is optional.

While building these scripts, I found this FTP Comand reference very useful: http://www.nsftools.com/tips/MSFTP.htm.

Now, I have a shortuct on my start menu that I can click whenver I want to quickly restart my ASP.NET web site.

ASP.NET | Web
Friday, 22 June 2018 09:50:00 (GMT Daylight Time, UTC+01:00)
# Thursday, 21 June 2018

GCast 3:

Creating and Deploying a Predictive Web Services with Azure ML Studio

Building on Episode 2, I show you how to create, publish, and call a predictive web service from a trained model built with Azure ML Studio.

Thursday, 21 June 2018 08:26:00 (GMT Daylight Time, UTC+01:00)
# Wednesday, 20 June 2018

Parsing information from a web page is not a trivial task. Fortunately, HTML has a defined structure and libraries exist to help us navigate that structure.

One such library for C# is HTML Agility Pack or HAP.

You can add this library to a project via NuGet. Simply right-click your project in the Visual Studio Solution Explorer, select "Manage NuGet Packages" (Fig. 1); then search for "HTML Agility Pack" and install the package (Fig. 2).

hap01-MangeNuGet
Fig. 1

hap02-NuGet
Fig. 2

Once the package is installed, you can load your document into an HtmlAgilityPack.HtmlDocument and begin working with it.

There are 3 ways to load a web page into an HtmlDocument: from a file on disk; from a string of HTML, from a URL, and from whatever document is loaded in a browser.

Below are examples of each (taken from the HAP web site).

// From File
var doc = new HtmlDocument();
doc.Load(filePath);

// From String
var doc = new HtmlDocument();
doc.LoadHtml(html);

// From Web
var url = "http://html-agility-pack.net/";
var web = new HtmlWeb();
var doc = web.Load(url);

// From Browser
var web1 = new HtmlWeb();
var doc1 = web1.LoadFromBrowser(url, o =>
{
	var webBrowser = (WebBrowser) o;

	// WAIT until the dynamic text is set
	return !string.IsNullOrEmpty(webBrowser.Document.GetElementById("uiDynamicText").InnerText);
});
  

Listing 1

I am interested in this project because I have a web page that lists all my public presentations and I want to make this page data driven, so I won't have to update a text file every time I schedule a new presentation.

To populate a database with this information, I could either type in every presentation or grab it from my web page and parse out the relevant information in an HTML table. Since I have delivered nearly 500 presentations, re-typing each of these individually seemed like way too much work.

Here is a screenshot of my "Public Presentations" page:

image
Fig. 3

and here is a partial listing of the source HTML for that page.

<table id="talklist">
 <tr style="height:15.0pt">
  <th style="height:15.0pt;width:62pt" width="83">
   Date</th>
  <th width="458">
   Topic</th>
  <th width="311">
   Event</th>
  <th width="64">
	Location</th>
 </tr>
        
 <tr style="height:15.0pt">
 <td>
  Sep 20, 2018
 </td>
 <td>
  Adding Image and Voice Intelligence to Your Apps with Microsoft Cognitive Services
 </td>
 <td>
  VSLive
 </td>
 <td>
  Chicago, IL
 </td>
 </tr>

 <tr style="height:15.0pt">
 <td>
  Sep 20, 2018
 </td>
 <td>
  Effective Data Visualization
 </td>
 <td>
  VSLive
 </td>
 <td>
  Chicago, IL
 </td>
 </tr>

 <tr style="height:15.0pt">
 <td>
  Jun 22, 2018
 </td>
 <td>
  Effective Data Visualization
 </td>
 <td>
  Beer City Code
 </td>
 <td>
  Grand Rapids, MI
 </td>
 </tr>
  

Listing 2

I used the following code to load the data into an HtmlDocument:

var url = "http://www.davidgiard.com/Schedule.aspx";
Console.WriteLine("Getting data from {0}...", url);
var web = new HtmlWeb();
var doc = web.Load(url);
  

Listing 3

The HtmlDocument has a DocumentNode property, which returns the root element of the HTML document as a node. Most of the time, I found myself working with nodes and collections of nodes. Each node has a SelectSingleNode and a SelectNodes method which returns a node and a collection of nodes, respectively. These take an XPATH argument, with which I was familiar from my days working with XML documents.

The following code retrieves a nodelist of all the <tr> row nodes within the "talklist" table, shown in Listing 2.

var documentNode = doc.DocumentNode;
var tableNode = documentNode
			.SelectSingleNode("//table[@id='talklist']");
var rowsNodesList = tableNode.SelectNodes("tr");
  

Listing 4

Finally, because each <tr> node contains 4 nodes of <td> cells, I can iterate through each node, find all the cells, and the innertext of each cell. For good measure, I stripped off any non-printing characters. Because the title row contains <th> cells, instead of <td> cells, I want to check for this before extracting information.

The code for this is in Listing 5.

var rowCount = 1;
foreach (var row in rowsNodesList)
{
	var cells = row.SelectNodes("td");
	if (cells != null && cells.Count > 0)
	{
		var date = cells[0].InnerText;
		date = date.Replace("\r\n", "").Trim();
		var topic = cells[1].InnerText;
		topic = topic.Replace("\r\n", "").Trim();
		var eventName = cells[2].InnerText;
		eventName = eventName.Replace("\r\n", "").Trim();
		var location = cells[3].InnerText;
		location = location.Replace("\r\n", "").Trim();

		Console.WriteLine("Row: {0}", rowCount);
		Console.WriteLine("Date: {0}", date);
		Console.WriteLine("Topic: {0}", topic);
		Console.WriteLine("Event: {0}", eventName);
		Console.WriteLine("Location: {0}", location);
		Console.WriteLine("--------------------");
		rowCount++;
	}
}
  

Listing 5

Here is the full code listing for my console app that retrieves the text of each cell.

using HtmlAgilityPack;
using System;

namespace TestHAP
{
    class Program
    {
        static void Main(string[] args)
        {
            var url = "http://www.davidgiard.com/Schedule.aspx";
            Console.WriteLine("Getting data from {0}...", url);
            var web = new HtmlWeb();
            var doc = web.Load(url);

            var documentNode = doc.DocumentNode;
            var tableNode = documentNode
                        .SelectSingleNode("//table[@id='talklist']");
            var rowsNodesList = tableNode.SelectNodes("tr");

            var rowCount = 1;
            foreach (var row in rowsNodesList)
            {
                var cells = row.SelectNodes("td");
                if (cells != null && cells.Count > 0)
                {
                    var date = cells[0].InnerText;
                    date = date.Replace("\r\n", "").Trim();
                    var topic = cells[1].InnerText;
                    topic = topic.Replace("\r\n", "").Trim();
                    var eventName = cells[2].InnerText;
                    eventName = eventName.Replace("\r\n", "").Trim();
                    var location = cells[3].InnerText;
                    location = location.Replace("\r\n", "").Trim();

                    Console.WriteLine("Row: {0}", rowCount);
                    Console.WriteLine("Date: {0}", date);
                    Console.WriteLine("Topic: {0}", topic);
                    Console.WriteLine("Event: {0}", eventName);
                    Console.WriteLine("Location: {0}", location);
                    Console.WriteLine("--------------------");
                    rowCount++;
                }
            }

            Console.ReadLine();
        }
    }
}
  

Listing 6

You can download this solution from my GitHub repository.

Although I don’t intend to use it for this, HAP also supports modifying the HTML you select with node methods like AppendChild(), InsertAfter(), and RemoveChild().

This tool will help me to retrieve and parse the hundreds of rows of data from my web page and insert them into a database.

C# | HTML5 | Web
Wednesday, 20 June 2018 08:39:00 (GMT Daylight Time, UTC+01:00)
# Tuesday, 19 June 2018

Years ago, I created a list of my public speaking. In order to make it more readable, I added a bit of jQuery to style alternating rows with a light gray color. I used jQuery because, at that time, CSS did not support the ability to apply a style to every other row in a table.

I liked the results, but the code always struck me as awkward.

image

Eventually, CSS3 provided this ability, but not every browser immediately supported this style. So I waited. And then, I forgot about it. It was slow, but it worked.

It has been a few years and I had a bit of time, so today I removed the jQuery reference and JavaScript from this page and achieved the same results using the CSS

Here is the table opening tag. I applied a style to it only to make it easy to find, in case there are other tables on the same page.

<table class="tiger-stripe">
  

Each row has no background-color styling, because I applied that later. Here is a sample row.

 <tr style="height:15.0pt">
  <td>
   Jun 11, 2018
  </td>
  <td>
   Building and Training your own Custom Image Recognition AI
  </td>
  <td>
   Norwegian Developers Conference
  </td>
  <td>
   Oslo, Norway
  </td>
 </tr>
  

Originally, I had all this within the page's <head>:

 <style type="text/css">
  table {width:90%; border:1px solid gray;}
  .oddrow {background-color:#E5E5E5;}
 </style>
 <script type="text/javascript" src="http://code.jquery.com/jquery-latest.min.js">
 </script>

 <script type="text/javascript">
  $(function(){
   $("table.tiger-stripe tr:even").addClass("oddrow");
 });
 </script>
  

I was able to accomplish the same results using the "even" argument of the "nth-child" selector. I replaced the code within the <head> above with the following:

<style type="text/css">
table {width:90%; border:1px solid gray;}
table#presentationlist tr:nth-child(even) {background-color:#E5E5E5;}
</style>
  

I decided to replace the table’s class (“tiger-stripe”) with an id (“presentationlist”), since the class only exists to identify this table. Strictly speaking, this was not necessary, but it felt cleaner this way. Here is the new table tag.

<table id="presentationList">
  

Each row has no background-color styling, because I applied that later. Here is a sample row.

By eliminating the custom JavaScript and the jQuery reference, the code is much simpler and the page loads faster. I did not need to modify any HTML. CSS selectors take care of all the styling.

This is an example of using declarative programming, instead of imperative programming to accomplish a task. In other words, the new system tells the browser the results I want without explicitly specifying how I want it done.

You can see the results here

HTML5 | Web
Tuesday, 19 June 2018 20:27:18 (GMT Daylight Time, UTC+01:00)
# Monday, 18 June 2018
Monday, 18 June 2018 03:46:00 (GMT Daylight Time, UTC+01:00)

IMG_0860Most of the Norway I saw is defined by towering cliffs, the result of glaciers gouging their way through the country thousands of years ago. Many of these filled with water to became the famous fjords of Norway.

This was the Norway I experienced when I made my first trip here this week.

I was invited to speak at the The Norwegian Developer Conference (“NDC”), so I flew to Oslo after speaking at IT Camp in Romania. I arrived in Oslo on a rainy Sunday night after a week in Romania.

IMG_0920The first 2 days in Oslo, I worked on a Machine Learning project for Bane Nor - the Norwegian national railway. This was a great experience for me, as Microsoft flew in engineers from all over the world and I had an experience to learn about the train industry from the customer and about data science from several experts.

NDC began on Wednesday, so I arrived bright and early to experience it all. The conference was amazing. Hundreds of speakers from all over the world come to Oslo each year for some high-quality sessions. I knew some of the speakers and I had the opportunity to meet many more.

IMG_0873Wednesday evening, the conference organizers treated all the speakers to a boat ride around the islands near Oslo, which was a great chance to meet new people.

IMG_0932My presentation - Building and Training your own Custom Image Recognition AI - was the last one of the day. I was happy to get it over with on Day 1, but I spent much of Wednesday preparing for it. In the end, it went very well. The bright stage lights prevented me from seeing the audience, but I received several good questions afterward, so I think the audience enjoyed it as much as I did.

IMG_0993The day after the conference, I booked a trip to Bergen. Oslo is near the eastern border of Norway and Bergen is on the west coast, so this all-day trip took me across the entire country. It consisted of 3 trains, a bus, and a boat. The boat ride was the most impressive as we traveled through the fjords of central Norway. The trip was designed to be more scenic than efficient and it took us from Oslo to Myrdal to Flåm to Gudvangen to Voss to Bergen. The fjord boat cruise took me to the northernmost point I have ever been, edging out my trip to Upsala, Sweden 3 years ago.

IMG_1028I only had one day in Bergen and I was exhausted from 2 weeks on the road. But I did a lot of walking around the city, and visited 2 art museums, and drank some local beers, and ate reindeer stew and whale steak, and sat by the harbor to watch the sun set at midnight. Scandinavian daylight last for over 20 hours this time of year, making it very difficult for me to pace myself.

IMG_1035I missed Father's Day in America (most European countries celebrate in March), but I will make time with my boys in the next few weeks.

IMG_1049In a few hours, I fly home, tired but content from 2 weeks abroad traveling thousands of miles. I feel like I need to return to Norway and see all the places I missed. Hopefully, NDC will make that happen next year.

Monday, 18 June 2018 00:21:00 (GMT Daylight Time, UTC+01:00)