# Friday, 29 June 2018

The .NET framework provides the FtpWebRequest class to assist you in managing FTP commands.

A common use of this class is to upload a file to an FTP site.

To do this in C#, we create an FtpWebRequest object (the WebRequest object has a static factory method to do this) on the object, set our intended method, and specify the FTP site's login credentials, as shown below.

string ftpUrl = @"ftp://blah.ftp.blah.com";
ftpUrl = ConfigurationManager.AppSettings["ftpUri"];
string userName = ConfigurationManager.AppSettings["userName"];
string password = ConfigurationManager.AppSettings["password"];

string ftpDestinationFolder = "site/wwwroot/content/Giard";
string localFolderSource = @"C:\Test\_source";
string fileName = "testfile.txt";

string sourcePath = @"C:\Test\_source\testfile.txt";
string destinationPath = @"ftp://blah.ftp.blah.com/folder1/testfile.txt";

FtpWebRequest request = (FtpWebRequest)WebRequest.Create(destinationPath);
request.Method = WebRequestMethods.Ftp.UploadFile;
  

A common mistake is to set only the site and folder name (and skip the filename) in the destinationPath.

To send a file via FTP, we need to read the file from disk and convert it into a Stream.

byte[] fileContents;
using (StreamReader sourceStream = new StreamReader(sourcePath))
{
	fileContents = Encoding.UTF8.GetBytes(sourceStream.ReadToEnd());
}
  

We then write this StreamReader to the FTP request object

request.ContentLength = fileContents.Length;
using (Stream requestStream = request.GetRequestStream())
{
	requestStream.Write(fileContents, 0, fileContents.Length);
}
  

Finally, we retrieve the FTP response.

using (FtpWebResponse response = (FtpWebResponse)request.GetResponse())
{
	var responseText = response.StatusDescription;
	Console.WriteLine("Completed uploading file {0}!", fileName);
	Console.WriteLine("Status={0}", responseText);
}
  

The full code of my console app is here:

using System;
using System.Configuration;
using System.IO;
using System.Net;
using System.Text;

namespace DemoFtp
{
    class Program
    {
        static void Main(string[] args)
        {
            string ftpUrl = @"ftp://blah.ftp.blah.com";
            // Note: I stored some values in app.config
            // I had to set a reference to System.Configuration to read these values
            ftpUrl = ConfigurationManager.AppSettings["ftpUri"];
            string userName = ConfigurationManager.AppSettings["userName"];
            string password = ConfigurationManager.AppSettings["password"];

            string ftpDestinationFolder = "site/wwwroot/content/Giard";
            string localFolderSource = @"C:\Test\_source";
            string fileName = "testfile.txt";

            string sourcePath = Path.Combine(localFolderSource, fileName);
            string destinationPath = Path.Combine(ftpUrl, ftpDestinationFolder, fileName);

            FtpWebRequest request = (FtpWebRequest)WebRequest.Create(destinationPath);
            request.Method = WebRequestMethods.Ftp.UploadFile;

            request.Credentials = new NetworkCredential(userName, password);

            byte[] fileContents;
            using (StreamReader sourceStream = new StreamReader(sourcePath))
            {
                fileContents = Encoding.UTF8.GetBytes(sourceStream.ReadToEnd());
            }

            request.ContentLength = fileContents.Length;
            using (Stream requestStream = request.GetRequestStream())
            {
                requestStream.Write(fileContents, 0, fileContents.Length);
            }

            using (FtpWebResponse response = (FtpWebResponse)request.GetResponse())
            {
                var responseText = response.StatusDescription;
                Console.WriteLine("Completed uploading file {0}!", fileName);
                Console.WriteLine("Status={0}", responseText);
            }

            Console.ReadLine();
        }
    }
}
  

You can find this console app in my github repository at https://github.com/DavidGiard/Demo-FTP-CSharp

.Net | C# | Web
Friday, 29 June 2018 07:38:00 (GMT Daylight Time, UTC+01:00)
# Thursday, 28 June 2018

GCast 4:

Azure Notebooks

Learn how to create a free library and Jupyter notebooks hosted in Azure.

Thursday, 28 June 2018 07:45:00 (GMT Daylight Time, UTC+01:00)
# Wednesday, 27 June 2018

A T4 ("Text Template Transformation Toolkit") template is a text generation tool that mixes raw text and code to generate text as output.

In my last article, I showed how to create a Runtime Text Template and generate output from it via the following lines of C# code:

var tt = new myT4Template();
string outputText = tt.TransformText();
  

But sometimes, it is helpful to pass parameters from your C# code and use them in your template.

The steps to do this are:

Add a T4 Runtime Text Template to your Visual Studio project  In the T4 template:

    1. Add one or more parameter directives  
    2. Use the parameters in your template

In your C# code

  1. Instantiate your T4 template object
  2. Set the Session property to a new Dictionary
  3. Pass the parameters in the Session object
  4. Initialize the T4 template object
  5. Generate the output text with the T4 template object's TransformText() method.

An example will help illustrate this.

I have a T4 Runtime Text Template named "ParamTemplate1.tt" and I want to add 2 parameters: firstName and lastName (both strings) and use them in my template.

At the top of the template file, add the following directive:

<#@ parameter name="firstName" type="System.String" #>
<#@ parameter name="lastName" type="System.String" #>
  

Notice that we included the Namespace in the parameters' data types.

Now, use the parameters in your template, as in the following code:

Hello <#=firstName #> <#=lastName #>!
  

In your C# program, add the following code

var pt1 = new ParamTemplate1();
pt1.Session = new Dictionary<string, object>();
pt1.Session["firstName"] = "David";
pt1.Session["lastName"] = "Giard";
pt1.Initialize();
var outputText1 = pt1.TransformText();
  

This passes the values "David" and "Giard" to the firstName and lastName parameters, respectively.

We can even pass in a custom data type, based on a type we create.

Imagine I have created a Person class with a FirstName and a LastName properties (both strings) and I want to pass a parameter of this type into my T4 template named "ParamTemplate2.tt" and use that parameter inside the template.

Here is an example of my custom Person class:

namespace DemoT4Parameters
{
    public class Person
    {
        public string FirstName { get; set; }
        public string LastName { get; set; }
    }
}
  

In my T4 template, I again add a parameter directive, as shown below:

<#@ parameter name="person" type="DemoT4Parameters.Person" #>
  

Notice the custom namespace in the Type.

Now, I can use the parameter and its properties in my template, as shown below:

Hello <#=person.FirstName #> <#=person.LastName #>!
  

My C# code is very similar to the first example.

var pt2 = new ParamTemplate2();
pt2.Session = new Dictionary<string, object>();
var person = new Person()
{
	FirstName = "Satya",
	LastName = "Nadella"
};
pt2.Session["person"] = person;
pt2.Initialize();
var outputText2 = pt2.TransformText();
  

You can find this code in my GitHub repository at https://github.com/DavidGiard/Demo-T4-Parameters.

In this article, I showed how to pass parameters into a T4 Runtime Text Template.

Wednesday, 27 June 2018 08:44:00 (GMT Daylight Time, UTC+01:00)
# Tuesday, 26 June 2018

A Text Template Transformation Toolkit (or "T4") template is a text generation tool that mixes raw text and code to generate text as output.

Visual Studio contains tools to assist with the creation and execution of T4 templates.

To add a T4 template to your project, right-click on the project, and select Add | New Item from the context menu.

In the "Add New Item" dialog, the T4 templates are listed under the General category; but you can find them quickly by typing "T4" in the "Search Installed Templates" textbox.

There are two kinds of T4 templates: a Text Template and a Runtime Text Template. Each of these starts with a file with ".tt" extension, but is designed to generate at least one other file.

The Text Template regenerates the final output file every time you save the ".tt" file. This is a quick way to generate simple files, but it lacks flexibility.

The Runtime Text Template generates a VB or C# class that you can call from your code to generate the output file. We will focus on the Runtime Text Template in this article.

A Runtime Text Template consists of raw text (which will be exactly duplicated in the output file) and executable code, which the template will run. Any output of that code will be included in the output file. Code is delimited from the raw text by placing it between the <# and #> symbols. If you just want to output the results of an expression, you can surround that expression with the <#= and #> symbols.

For example, to output the current date and time, include the following line in your template.

<#=System.DateTime.Now #>
  

You can combine this with text as in the following example:

(This file was generated at <#=System.DateTime.Now #>!)
  

which would output something like the following:

(This file was generated at 06/25/2018 20:33:14!)

You can include more complex code between the <# and #> symbols, as in the following example:

<#
for (System.Int32 I = 0; I < 5; I++)
{
	// WriteLine is a utility function that outputs text. 
	WriteLine(I);
}
#>
  

The result of the above code is:

0
1
2
3
4

When you save a Runtime Text Template, Visual Studio generates code for a public .NET class with the same name as your tt file. You can instantiate this class as you would any other public class and call the TransformText() method to generate the output as a string.

For example, if I created a Runtime Text Template named "myT4Template.tt", the following code would generate the output and save it to a variable named "outputText".

var tt = new myT4Template();
string outputText = tt.TransformText();
  

T4 templates give you the flexibility to generate text and text files quickly and the flexibility to change the template without worrying about the associated data.

Tuesday, 26 June 2018 08:50:00 (GMT Daylight Time, UTC+01:00)
# Monday, 25 June 2018
Monday, 25 June 2018 09:10:00 (GMT Daylight Time, UTC+01:00)
# Sunday, 24 June 2018

MySQL is a popular relational database. Although the product is owned by Oracle, which offers several versions for purchase, it is an open source project and they offer a free "Community Edition.

There are several ways to install and start running MySQL.

You can download a copy from https://www.mysql.com/downloads/. The community edition offers install files for various platforms (an MSI file for Windows) that allows you install the database engine locally.

My preferred option is to install a database on Azure and store my data in the cloud. Because Azure offers MySQL as a service, I don't need to create a Virtual Machine or install the database engine anywhere. To create a MySQL service is Azure, navigate to the Azure portal and select Create a Resource | Databases | Azure Database for MySQL; then complete the blade that displays.

Whether using a cloud service or installing locally, you will be prompted to create an admin user and password. Remember these.

Once you have access to MySQL, you can use the command line to create and manage databases. On a local machine, open a command prompt; in Azure, you can click the "Cloud Shell" button at the top of the Azure portal.

To begin working with MySQL, you need to connect to the database engine with the following command:

mysql -h hostname -u username -p
  

where hostname is the name of the MySQL instance and username is the name of the admin user.

For example, if I have an Azure MySQL instance named "dgmysql" and a user named "dgiard", I would type the following:

mysql -h dgmysql.mysql.database.azure.com -u dgiard@dgmysql -p
  

When prompted, enter the appropriate password for the user.

The following commands are usfule for working with databases.

# Create a new database
CREATE DATABASE databasename;

# List all databases available
SHOW DATABASES;

# Switch to a specific database
USE publicspeaking;
  

Now you can start using standard SQL commands to work with database objects and data in the current database.

For example,

# Create a new database
CREATE DATABASE databasename;

# List all databases available
SHOW DATABASES;

# Switch to a specific database
USE publicspeaking;
  

For the most part, MySQL follows ANSI SQL syntax.

If you are familiar with another relational database, such as Microsoft SQL Server, working with MySQL will feel natural.

Sunday, 24 June 2018 07:16:00 (GMT Daylight Time, UTC+01:00)
# Saturday, 23 June 2018

On May 19, I delivered a presentation titled "How Cloud Computing Empowers a Data Scientist" at the Chicago AI & Data Science Conference.

I described ways that the cloud has accelerated the fields of data science, machine learning, and artificial intelligence; and I gave examples of Azure tools that facilitate development in these fields.

You can watch the video below or at https://youtu.be/H19IW6nykZo

Saturday, 23 June 2018 08:17:00 (GMT Daylight Time, UTC+01:00)
# Friday, 22 June 2018

Often, I need to restart an ASP.NET web site. There are many reasons but the most common reason (for me) is that I have updated a page and ASP.NET is displaying a cached version of that page, which does not reflect my changes.

There are multiple ways to restart an ASP.NET site; but, I always found it easiest to copy any file into the site's "bin" folder. In fact, I typically keep an empty text file named "_a.txt" in my local Documents folder for just this purpose. It is a bit of a hassle to open my FTP client (I use Filezilla), just to copy 1 file; so I recently decided to automate this process.

The Windows command prompt has an "FTP" command that that takes a "-s" argument with which you can provide a file containing FTP commands.

So I created the following text file named "RestartBlog.ftp"

open blahblahblah.ftp.azurewebsites.windows.net
myusername
mypassword
put _a.txt /site/wwwroot/bin/_a.txt
close
quit
  

In my file, I replaced blahblahblah.ftp.azurewebsites.windows.net with the FTP Host Name of my site, myusername with the FTP User Name of my site, and mypassword with the password associated with that username.

This opens a connection to my FTP site, logsin in and copies a local file named "_a.txt" to the site's bin folder. It's possible that your site's "bin" folder is in a different path, so you may need to change this. Next, it closes the connection and exits FTP mode.

After saving this file, I created another file in the same folder containing the following text:

echo ".">_a.txt
ftp -s:RestartBlog.ftp
del _a.txt
pause
  

This creates a small text file named "_a.txt"; then calls my file with FTP commands. After executing the FTP commands, this batch deletes the text file it created.

I added the "pause" at the end so that the command window would not immediately close and I could view the results and any errors that occurred. This is optional.

While building these scripts, I found this FTP Comand reference very useful: http://www.nsftools.com/tips/MSFTP.htm.

Now, I have a shortuct on my start menu that I can click whenver I want to quickly restart my ASP.NET web site.

ASP.NET | Web
Friday, 22 June 2018 09:50:00 (GMT Daylight Time, UTC+01:00)
# Thursday, 21 June 2018

GCast 3:

Creating and Deploying a Predictive Web Services with Azure ML Studio

Building on Episode 2, I show you how to create, publish, and call a predictive web service from a trained model built with Azure ML Studio.

Thursday, 21 June 2018 08:26:00 (GMT Daylight Time, UTC+01:00)
# Wednesday, 20 June 2018

Parsing information from a web page is not a trivial task. Fortunately, HTML has a defined structure and libraries exist to help us navigate that structure.

One such library for C# is HTML Agility Pack or HAP.

You can add this library to a project via NuGet. Simply right-click your project in the Visual Studio Solution Explorer, select "Manage NuGet Packages" (Fig. 1); then search for "HTML Agility Pack" and install the package (Fig. 2).

hap01-MangeNuGet
Fig. 1

hap02-NuGet
Fig. 2

Once the package is installed, you can load your document into an HtmlAgilityPack.HtmlDocument and begin working with it.

There are 3 ways to load a web page into an HtmlDocument: from a file on disk; from a string of HTML, from a URL, and from whatever document is loaded in a browser.

Below are examples of each (taken from the HAP web site).

// From File
var doc = new HtmlDocument();
doc.Load(filePath);

// From String
var doc = new HtmlDocument();
doc.LoadHtml(html);

// From Web
var url = "http://html-agility-pack.net/";
var web = new HtmlWeb();
var doc = web.Load(url);

// From Browser
var web1 = new HtmlWeb();
var doc1 = web1.LoadFromBrowser(url, o =>
{
	var webBrowser = (WebBrowser) o;

	// WAIT until the dynamic text is set
	return !string.IsNullOrEmpty(webBrowser.Document.GetElementById("uiDynamicText").InnerText);
});
  

Listing 1

I am interested in this project because I have a web page that lists all my public presentations and I want to make this page data driven, so I won't have to update a text file every time I schedule a new presentation.

To populate a database with this information, I could either type in every presentation or grab it from my web page and parse out the relevant information in an HTML table. Since I have delivered nearly 500 presentations, re-typing each of these individually seemed like way too much work.

Here is a screenshot of my "Public Presentations" page:

image
Fig. 3

and here is a partial listing of the source HTML for that page.

<table id="talklist">
 <tr style="height:15.0pt">
  <th style="height:15.0pt;width:62pt" width="83">
   Date</th>
  <th width="458">
   Topic</th>
  <th width="311">
   Event</th>
  <th width="64">
	Location</th>
 </tr>
        
 <tr style="height:15.0pt">
 <td>
  Sep 20, 2018
 </td>
 <td>
  Adding Image and Voice Intelligence to Your Apps with Microsoft Cognitive Services
 </td>
 <td>
  VSLive
 </td>
 <td>
  Chicago, IL
 </td>
 </tr>

 <tr style="height:15.0pt">
 <td>
  Sep 20, 2018
 </td>
 <td>
  Effective Data Visualization
 </td>
 <td>
  VSLive
 </td>
 <td>
  Chicago, IL
 </td>
 </tr>

 <tr style="height:15.0pt">
 <td>
  Jun 22, 2018
 </td>
 <td>
  Effective Data Visualization
 </td>
 <td>
  Beer City Code
 </td>
 <td>
  Grand Rapids, MI
 </td>
 </tr>
  

Listing 2

I used the following code to load the data into an HtmlDocument:

var url = "http://www.davidgiard.com/Schedule.aspx";
Console.WriteLine("Getting data from {0}...", url);
var web = new HtmlWeb();
var doc = web.Load(url);
  

Listing 3

The HtmlDocument has a DocumentNode property, which returns the root element of the HTML document as a node. Most of the time, I found myself working with nodes and collections of nodes. Each node has a SelectSingleNode and a SelectNodes method which returns a node and a collection of nodes, respectively. These take an XPATH argument, with which I was familiar from my days working with XML documents.

The following code retrieves a nodelist of all the <tr> row nodes within the "talklist" table, shown in Listing 2.

var documentNode = doc.DocumentNode;
var tableNode = documentNode
			.SelectSingleNode("//table[@id='talklist']");
var rowsNodesList = tableNode.SelectNodes("tr");
  

Listing 4

Finally, because each <tr> node contains 4 nodes of <td> cells, I can iterate through each node, find all the cells, and the innertext of each cell. For good measure, I stripped off any non-printing characters. Because the title row contains <th> cells, instead of <td> cells, I want to check for this before extracting information.

The code for this is in Listing 5.

var rowCount = 1;
foreach (var row in rowsNodesList)
{
	var cells = row.SelectNodes("td");
	if (cells != null && cells.Count > 0)
	{
		var date = cells[0].InnerText;
		date = date.Replace("\r\n", "").Trim();
		var topic = cells[1].InnerText;
		topic = topic.Replace("\r\n", "").Trim();
		var eventName = cells[2].InnerText;
		eventName = eventName.Replace("\r\n", "").Trim();
		var location = cells[3].InnerText;
		location = location.Replace("\r\n", "").Trim();

		Console.WriteLine("Row: {0}", rowCount);
		Console.WriteLine("Date: {0}", date);
		Console.WriteLine("Topic: {0}", topic);
		Console.WriteLine("Event: {0}", eventName);
		Console.WriteLine("Location: {0}", location);
		Console.WriteLine("--------------------");
		rowCount++;
	}
}
  

Listing 5

Here is the full code listing for my console app that retrieves the text of each cell.

using HtmlAgilityPack;
using System;

namespace TestHAP
{
    class Program
    {
        static void Main(string[] args)
        {
            var url = "http://www.davidgiard.com/Schedule.aspx";
            Console.WriteLine("Getting data from {0}...", url);
            var web = new HtmlWeb();
            var doc = web.Load(url);

            var documentNode = doc.DocumentNode;
            var tableNode = documentNode
                        .SelectSingleNode("//table[@id='talklist']");
            var rowsNodesList = tableNode.SelectNodes("tr");

            var rowCount = 1;
            foreach (var row in rowsNodesList)
            {
                var cells = row.SelectNodes("td");
                if (cells != null && cells.Count > 0)
                {
                    var date = cells[0].InnerText;
                    date = date.Replace("\r\n", "").Trim();
                    var topic = cells[1].InnerText;
                    topic = topic.Replace("\r\n", "").Trim();
                    var eventName = cells[2].InnerText;
                    eventName = eventName.Replace("\r\n", "").Trim();
                    var location = cells[3].InnerText;
                    location = location.Replace("\r\n", "").Trim();

                    Console.WriteLine("Row: {0}", rowCount);
                    Console.WriteLine("Date: {0}", date);
                    Console.WriteLine("Topic: {0}", topic);
                    Console.WriteLine("Event: {0}", eventName);
                    Console.WriteLine("Location: {0}", location);
                    Console.WriteLine("--------------------");
                    rowCount++;
                }
            }

            Console.ReadLine();
        }
    }
}
  

Listing 6

You can download this solution from my GitHub repository.

Although I don’t intend to use it for this, HAP also supports modifying the HTML you select with node methods like AppendChild(), InsertAfter(), and RemoveChild().

This tool will help me to retrieve and parse the hundreds of rows of data from my web page and insert them into a database.

C# | HTML5 | Web
Wednesday, 20 June 2018 08:39:00 (GMT Daylight Time, UTC+01:00)