# Monday, 04 June 2018
Monday, 04 June 2018 09:38:00 (GMT Daylight Time, UTC+01:00)
# Sunday, 03 June 2018

Today I am grateful for a walk along the Fox River in St. Charles yesterday

Today I am grateful to get my photos and videos transferred to and organized on my new computer.

Today I am grateful for the NBA Finals and the NHL Finals

Today I am grateful for a hot bath last night.

Today I am grateful for Taco Tuesday.

Today I am grateful for a walk around Irving Park last night.

Today I am grateful for a boat ride around Vancouver, BC yesterday

Today I am grateful to participate in a successful Imagine Cup Finals yesterday.

Today I a m grateful to make it safely to Vancouver after a full day of travel yesterday.

Today I am grateful to spend time this week with my boys while Nick was in town.

‪Today I am grateful to Nick for helping me set up my TV and DVD player yesterday. ‬

Today I am grateful to watch the Cubs-Indians game at Wrigley Field last night.

Today I am grateful for online training courses

Today I am grateful to speak at the Chicago Data Science conference this weekend at the University of Chicago

Today I am grateful for an afternoon in Kenosha, WI

Today I am grateful for a long walk along the river in Chinatown last night.

Today I am grateful for
-Dinner with Carol last night
-Seeing the Chicago Symphony Orchestra for the first time

Today I am grateful for the Marvel Cinematic Universe

Today I am grateful for my first Personal Training session in months.

Today I am grateful to watch a series of storms yesterday through a large window from the shelter of my living room.

Today I am grateful to celebrate Mother's Day with someone else's mother.

Today I am grateful to see Freddy Cole at the Jazz Showcase last night.

Today I am grateful to attend and present at my first Toastmasters meeting in months.

Today I am grateful to finally catch up on my sleep.

Today I am grateful for all these free drinks and food that Starbucks now owes me.

Today I am grateful for:
-Dinner with Tim last night
-5 days in California

David Giard updated his status.
Today I am grateful for:
-The hospitality and generosity of Greg and Sharon and their family
-Lunch with Gail yesterday
-An afternoon with my Uncle Bill and Aunt Jean
-A drive to the Pacific Ocean at Point Reyes National Seashore

Today I am grateful to Kim for a ride to San Rafael yesterday.

Sunday, 03 June 2018 22:53:37 (GMT Daylight Time, UTC+01:00)
# Friday, 01 June 2018

After creating an Azure Virtual Machine (VM), you may wish to add one or more disks. Maybe you want to increase your VM's storage; capacity or maybe you want to separate some data from the operating system disk. Either way, open your Virtual Machine properties in the Azure Portal and select "Disks", as shown in Fig. 1.

Fig. 1

The Disks blade displays, as shown in Fig. 2.

Fig. 2

Under the "Data Disks" section, click the "Name" dropdown of the first empty line; and select "Create Disk", as shown in Fig. 3.

Fig. 3

The "Create Managed Disk" blade displays (Fig. 4).

Fig. 4

Provide a disk name, resource group, region, type, and size. I generally prefer to create a disk in the same resource group and region as the VM. You can create a new empty disk, or one based on an existing snapshot.

Click the "Create" button to begin creating the disk, which will take a few minutes.

When finished, you should see your new disk listed in the VMs' Disks blade, as shown in Fig. 5.

Fig. 5

Repeat this process to add any more disks you like.

When you connect to the VM (via Remote Desktop for a Windows VM and via SSH for a Linux VM), you will see the extra disk and you will be able to read to and write from this disk.

Azure | IAAS
Friday, 01 June 2018 10:09:00 (GMT Daylight Time, UTC+01:00)
# Thursday, 31 May 2018

MagicianApprenticePug is an orphan on the Midkemia world. He is apprenticed to the wizard Kulgan, learns a bit of magic, saves a princess, and is granted a lordship. Then a war breaks out between the people of his homeworld of Midkemia    and the Tsurani of Kelewan, who travel to his world through a "rift" in space.

"Magician: Apprentice" Tells the story of Pug, of his friend Tomas, who receives a magic gift from a dying dragon, and of the war between the invading Tsurani and the people of Midkemia. The stories are straightforward tales of high fantasy, in the tradition of Lord of the Rings, but simpler. Major themes are Pug's coming of age and the contrasting cultures of the two warring societies.

I enjoyed reading it, but found the book unsatisfying by itself.  This book and "Magician: Master" were originally published as a single volume titled simply "Magician". Following the success of "Magician", Feist expanded the story and released it as two separate novels. But it the two together form a single story and Book 1 does not hold up well on its own. To appreciate the story, you must read both volumes.

Still, it is a good start to a good story and it is well told.

Thursday, 31 May 2018 18:04:00 (GMT Daylight Time, UTC+01:00)
# Wednesday, 30 May 2018

Creating a virtual machine (VM) may be the simplest thing to do in Azure. Azure VMs are very similar to on-premises VMs; in fact, they are built using the same technology - Microsoft Hyper-V.

You can create an Azure VM with just about any operating system, software, and data that you want. Hundreds of templates are available, pre-installed with a variety of configurations.

Azure supports Windows machines and Linux machines. (Did you know that about a third of the VMs in Azure are running Linux?)

The first step in creating an Azure VM is the same step when we create any Azure resource - Navigate to the Azure Portal, log in, and click the [Create a resource] button (Fig. 1) at the top left of the portal.

Fig. 1

From the list of resource categories, select the "Compute" category; then select the template on which you want to base your VM, as shown in Fig. 2.

Fig. 2

If you don't see the VM you want, you can search for it by typing a partial description in the search box. This will display matching templates from which you can select, as shown in Fig. 3.

Fig. 3

After selecting a template, a blade with a brief description displays. Fig. 4 shows the blade for an Ubuntu server. Click the [Create] button at the bottom of this blade.

Fig. 4

The "Create virtual machine" dialog displays with the first tab - "Configure basic settings" - active, as shown in Fig. 5. Complete the forms on this tab.

Fig. 5

  • At the name field, enter a unique name for your VM. Choose one that will make it easy for you to identify it when you return to the portal to manage this VM.
  • At the "VM disk type" dropdown, select "SSD" for a Solid State Drive or "HDD" for a standard hard drive.
  • At the "User name" field, enter a name for the local administrator account on this VM.
  • At "Authentication type", select either "SSH public key" or "Password" to specify how you want to log into this VM. I have selected "Password", which causes the "Password" fields to appear, requiring me to enter a password twice. Passwords must meet complexity requirements, including a length of at least of at least 12 characters and inclusion of numbers, alpha characters, and special characters.
  • If you have an Azure AD configured, you can configure your VM to use this for authentication by selecting "Enabled" under "Login with Azure Directory".
  • You will likely have only one Azure subscription; but if you have more than one, you can select the subscription with which to associate this VM.
  • At the "Resource group" fields, select an existing resource group or create a new one. Resource groups are a logical grouping of resources that you want to manage together. If there are existing Azure resources that will interact with this VM, you may want to place them all in the same Resource Group.
  • At the "Location" dropdown, select the region in which you want to create this VM. Location considerations include proximity to the people managing this VM; proximity to people using this VM; and proximity to any other resources that will interact with this VM.

When the tab is complete, click the [OK] button at the bottom.

The second tab - "Size" - will display, as shown in Fig. 6.

Fig. 6

On the "Size" tab, select the size machine you want to create. This is a long list, but you can filter or search to make it more manageable. Note that not every size is available in every region.

After highlighting the desired size, click the [Select] button at the bottom of the tab.

The "Settings" tab will display as shown in Fig. 7.

Fig. 7

Azure provides default values for every setting on this tab, so it is not necessary to change anything. However, you can configure your VM to connect to a specific virtual network or subnet or assign a specific IP address or change any of the settings you like.

When you are satisfied with the settings, click the [OK] button at the bottom of the tab.

The "Summary" tab displays what you have configured, in case you want to change anything at the last minute. It also validates that all your selections are complete and consistent. If the validation lists any problems, you should go back and correct these. 

After validation passes and you are satisfied with your selection, click the [Create] button at the bottom of the blade.

After a few minutes, Azure will create and deploy the VM based on your configuration.

Once it is created, it is your VM and you can connect to it as you would any local VM. Use Remote Desktop to connect to a Windows VM or use SSH to connect to a Linux VM.

Azure | IAAS
Wednesday, 30 May 2018 21:46:00 (GMT Daylight Time, UTC+01:00)
# Monday, 28 May 2018
Monday, 28 May 2018 10:13:00 (GMT Daylight Time, UTC+01:00)
# Sunday, 27 May 2018

IMG_0659Two weeks ago, I attended my first Imagine Cup event - the US Finals in San Francisco. Yesterday, I attended my second.

The Canadian Imagine Cup Finals took place in Vancouver, BC. Six finalists competed for the right to advance to the International Finals in Redmond in July.

This event was smaller and shorter than the corresponding US event; but equal in energy. Every one of the teams showed great creativity, strong technical skills, and impressive presentations.

I was excited that all 6 finalists came from schools with which I work. First place went to SmartArm - a project that provides affordable prosthetic arms for amputees - a team I helped mentor at the UTHacks hackathon in January in Toronto.

I'm getting used to these Imagine Cup projects and competitions and I'm looking forward to the world finals.

Sunday, 27 May 2018 17:35:17 (GMT Daylight Time, UTC+01:00)
# Friday, 25 May 2018

One of the challenges of working with data is what to do with missing data.

A missing column in a dataset can be, but is not limited to the following:

  • No text or numbers between 2 column delimiters
  • An empty string ("")
  • A blank string (e.g., "   ")
  • The number 0
  • A special indicator, such as "NA" or "NONE"
  • An inconsistent data type, such as a number where a string is expected
  • A value that makes no sense in the context of the data.

The last one requires some domain knowledge about the data, so it is often difficult to spot.

There are two strategies for dealing with missing data

  1. Delete or ignore the entire row
  2. Replace the column with a reasonable value.

If only a few rows contain missing data, it may be efficient to simply delete these rows.

But if many rows contain missing data, it probably makes sense to keep them as other columns may contain valuable information. In this case, we will want to replace the missing data with a reasonable value.

But what is a reasonable value?

Options include replacing the column with an average value, such as the mean or median of the non-missing values. Of course, this is only valid for numeric data that is ordinal, that is data in which higher numbers indicate a higher value and not simply a discrete category.

The Pandas library contains some simple functions for deleting rows and replacing values. The fillna function is the simplest way to do this.

# Replace all missing values with 0

# Replace all missing values with the string 'Missing'

You can delete invalid or missing rows by overwriting a dataset with a filtered version of that set, as in the following examples

# Delete all rows with area = 0 
df = df[df.area != 0]

# Delete all rows with null area 
df = df[df.area.notnull()] 

But for values that are not missing, but are inappropriate (e.g., using 0 to represent a missing data point, when 0 could be a valid measurement), we can use the map function.

Below, we use the map function to want to replace any value in the 'area' column that has a value of 0 with the mean value for this column.

# Replace 0 area with the mean
mean_area = df['area'].mean()   
df['area'] = df['area'].map({0: mean_area})    

In this article, we discussed ways to use Pandas handle missing data in a dataframe.

Friday, 25 May 2018 00:44:34 (GMT Daylight Time, UTC+01:00)
# Monday, 21 May 2018
Monday, 21 May 2018 16:37:00 (GMT Daylight Time, UTC+01:00)
# Friday, 18 May 2018

I was working with a dataset in Azure ML Studio and I needed to replace values in a column.

Reasons for replacing value include:

  • Replacing codes with a more readable word or words
  • Consistency when combining 2 sets of data
  • Converting to numeric values in order to assign values to discrete strings
  • Converting to numeric values in order to work with an algorithm that only accepts numeric values

There is no built-in shape to do this, but you can do so with a couple lines of code.

I can demonstrate by creating a new ML studio experiment and dragging the "Automobile Price Data" sample dataset onto the experiment design surface, as shown in Fig. 1.

Fig. 1

If we click on this shape and select "Visualize" (Fig. 2), we can see the data in the dataset. Click the "drive-wheels" column to see details about the data in that column (Fig. 3).

Fig. 2

Fig. 3

You can see from the visualization that the "drive-wheels" column contains 3 distinct values: "fwd", "rwd", "4wd"
Imagine I wanted to replace these with "FRONT", "REAR", and "FOUR", respectively. (Maybe to be consistent with a second dataset I plan to merge with this one.)

Drag an "Execute Python Script" shape to the  Experiment and connect its input to the output of the data shape (Fig. 4).

Fig. 4

In the Properties of the "Execute Python Script" shape, replace the existing code with the following:

import pandas as pd
def azureml_main(dataframe1 = None):
    dataframe1['drive-wheels'] = dataframe1['drive-wheels'].map({'fwd': 'FRONT', 'rwd': 'REAR', '4wd': 'FOUR'})
    return dataframe1,

The azureml_main function is required by ML Studio. It accepts one parameter - a dataframe, which we name “dataframe1”

The first line of code maps the 3 existing drive-wheels values to 3 new values for every row and saves these 3 new values back to the dataset. By returning that dataframe, we make this updated data the output of this shape, so it can be used by later steps in our experiment.

After running this experiment, we can click the script shape and Visualize the output and see that each value in "drive-wheels" has been replaced, as shown in Fig 5.

Fig. 5

This article shows a simple way to replace values in a dataframe column with new values in Azure ML Studio.

Friday, 18 May 2018 22:15:08 (GMT Daylight Time, UTC+01:00)