Lois McMaster Bujold's "Labyrinth" is a novella set in the Vorkosigan universe - one she created to hold the adventures of her hero Miles Vorkosigan.

While on a mission to the corrupt planet Jackson's Whole, Miles discovers Taura - the failed result of a genetic experiment to create a super-soldier. Taura is eight feet tall, capable of great violence and her fangs and claws give her an appearance that is more than a little frightening; but Miles senses a gentleness and sensitivity in Taura and takes pity on her mistreatment and imprisonment. He also needs some of the genetic material hidden in her calf to complete his mission. When Miles is captured, he and Taura form an unlikely couple as they plan their escape.

A version of this story appears within the novel "Borders of Infinity", but I had forgotten most of it in the five years since reading that collection, so it seemed fresh to me. In any event, the story stands on its own.

It shows off Miles's spirit of adventure, his resourcefulness, and his character. In later stories, Taura shows fierce loyalty to Miles because he was faithful to her. He not only saw past her outward appearance to the beauty within, but he kept his promise to rescue her.

This is a brief but significant story in the saga.


Episode 707

Beijie Zhang on Technical Interviews

Beijie Zhang has conducted dozens of interviews with developers applying for tech jobs. He describes the structure of these interviews and how a candidate can prepare.

Link:

BetterLogic channel


May 2022 Gratitudes

Comments [0]

5/2
Today I am grateful to sit in a coffee shop and read a book for a couple hours yesterday afternoon.

5/3
Today I am grateful for 500 subscribers to my GCast YouTube channel

5/4
Today I am grateful for an informative conversation with Dave yesterday.

5/5
Today I am grateful for coffee with Adam yesterday.


5/6
Today I am grateful to see Joe Lovano perform on my first visit to the Village Vanguard.

5/7
Today I am grateful for my first visit to the Museum of Modern Art yesterday

5/8
Today I am grateful to all the excellent mothers who make this world a better place

5/9
Today I am grateful to the hotel that mailed back to me the library book I left in my room.

5/10
Today I am grateful for supportive conversations with Shahed, Nick, and Tony

5/11
Today I am grateful to talk about the tech community yesterday with Eric

5/12
Today I am grateful for warm weather in Chicago

5/13
Today I am grateful to arrive safely in Tampa.

5/14
Today I am grateful to meet so many of my current and future family yesterday.

5/15
Today I am grateful for a rehearsal lunch and an evening get-together last night in Tampa.

5/16
Today I am grateful
-to attend the wedding of my son Nick
-to welcome Adriana into our family
-for a wonderful 4 days in Tampa

5/17
Today I am grateful
-to arrive safely in Seattle after a full day of flying
-for dinner last night with my team

5/18
Today I am grateful for a day of hacking in Seattle

5/19
Today I am grateful for dinner with Glenn and his daughter last night.

5/20
Today I am grateful for an evening at the Seattle Aquarium

5/21
Today I am grateful for dinner with Ted last night.

5/22
Today I am grateful for breakfast with Josh yesterday

5/23
Today I am grateful for breakfast yesterday with Dave, Sue, Debora, and Gary

5/24
Today I am grateful for a gift of Hello Fresh meals from my son and daughter-in-law

5/25
Today I am grateful for a conversation with Tim yesterday for the first time in years

5/26
Today I am grateful to connect and talk with Tiberiu yesterday.

5/27
Today I am grateful to see the SteelDrivers in concert last night.

5/28
Today I am grateful to accept an offer of a new job at Microsoft.

5/29
Today I am grateful to see a production of "Spring Awakening" yesterday at the Ruth Page Center for the Arts.

5/30
Today I am grateful to all the men and women who gave their lives in defense of our country.

5/31
Today I am grateful for a 3-day weekend

6/1
Today I am grateful for a fresh start

6/2
Today I am grateful for many good wishes the past few days

6/3
Today I am grateful for pizza with Tim last night

6/4
Today I am grateful to see a performance of "Rasheeda Speaking" at Theater Wit last night.

6/5
Today I am grateful to sit on the patio at Fitzgerald's listening to live music yesterday afternoon.


Sometimes, the last few pages overshadow the rest of a book - even when those pages have nothing to do with the rest of the story.

In "Cryoburn", Lois McMaster Bujold continues the adventure of Miles Vorkosigan, the diminutive galactic Lord and Imperial Auditor.

After being drugged and kidnapped while investigating corruption on the planet Kibou-daini, Miles awakens in a semi-abandoned building to discover a plot to cover up corporate bungling that will result in the death of thousands. He is rescued by Jin, an orphan and runaway, who is hiding in an underground operation that uses technology to "freeze" the terminally ill until a cure for their disease can be found.

McMaster takes us on a fun journey as Miles tries to unravel the conspiracy on this planet. He is assisted by Jin and by armsman Roic, Miles's right-hand man.

Although Miles is the focus of much of the story, the point of view switches mostly between Jin and Roic. The narrative is in the third person, but the language changes depending on the point of view. Most notably, Roic, refers to Miles as m'lord, while Jin calls him Miles-san.

This is the first time I remember an ethnicity or culture from Earth influencing the story, but it is clear from the names and the language that Kibou-daini is populated by the descendants of the people of Japan.

Bujold drops a bombshell at the end of the story that probably deserves more buildup; but her technique mirrors the random ways that major life events often strike in our lives, so it works.

"Cryoburn" works as a detective story, an adventure story, a science fiction story, and a story of corruption and class struggles.

But it is the final chapter that stays with me.


Episode 706

Dasith Wijesiriwardena on OpenTelemetry

OpenTelemetry is a set of standards, SDKs and tools that allow us to implement tracing in a distributed system.

Microsoft Engineer Dasith Wijesiriwardena describes how we can use OpenTelemetry to improve observability and make it easier to analyze distributed applications.


SpringAwakening"Spring Awakening" breaks a lot of rules and pushes the envelope on others. It includes topics of child molestation, masturbation, teenage sex, abortion, and suicide; and it expresses many of these through the catchy melodies of Duncan Sheik and the lyrics of Steven Sater.

This is the story of a group of young people at a restrictive school in 19th-century Germany. Most of them have been mistreated by adults in their life, including their parents and they bear the scars of this mistreatment.

Local Chicago company Porchlight Theatre produced an excellent rendition of this musical at the Ruth Page Center for the Arts in Chicago's Gold Coast.

The play focuses on Melchior (played by Jack DeCesare) and Wendla (played by Maya Lou Hlava) - two star-crossed lovers struggling to find their identities without the benefit of role models.

But Quinn Kelch steals the show as Moritz, a marginal student driven to despair by the thought of failing out of school. His wild-eyed neuroses and magnificent voice energized the show. Early in the play, Moritz surprises the audience in the middle of a mournful soliloquy when he pulls a microphone from his schoolboy uniform, jumps up on a chair, and launches into the rebellious anthem "Bitch of Living". The rest of the cast joins in, taking turns railing against their maltreatment or announcing their angst and sexual frustration.

The show is full of surprises. The same two people play every adult role: Michael Joseph Mitchell for the men and McKinley Carter for the women. The context and some slight costume changes make it clear who is who.

The music takes the audience on an emotional roller coaster, swinging from anger to sadness to hope.

This was a low-budget production staged entirely with local actors, directors, and staff. It lacked the sets and glitter of a Broadway show. The focus was entirely on the story and the characters. I was moved by it.


SteeldriversSometimes the loss of a major talent destroys a band; and sometimes, that band keeps moving forward and continues producing excellent material. The SteelDrivers experienced the latter.

In 2010, founding member Chris Stapleton left the group and went on to a successful solo career. The band has had three lead singers since then, but the lineup has remained relatively stable otherwise and has earned multiple Grammy nominations and one Grammy Award. Current singer Matt Dame has the voice to drive forward the band's music.

Thursday night, The SteelDrivers performed at The City Winery, delighting the crowd with their songs and their energy.

In addition to Dame, the current lineup consists of Tammy Rogers (fiddle and vocals), Richard Bailey (banjo), Mike Fleming (bass and vocals), and Brent Truitt (mandolin).

Rogers took the reins of this show, introducing the songs and charming the audience with humorous stories. The show was scheduled as part of a tour that should have happened two years ago to promote their "Bad for You" album. That album was released a few weeks before a global pandemic shut down much of the world, including the SteelDrivers tour. Rogers informed us that SteelDrivers fans are known as "Steelheads" and she hoped we would all leave as Steelheads tonight.

The band takes pride in performing only their own songs, steering clear of covering other songwriters' material; and they have an impressive list of songs on their five albums from which to choose. By their own admission, the songs tend to be autobiographical, but they leave it to the audience to decide what is fact and what is fiction.

They play bluegrass music, but it is bluegrass with a mix of blues and country - a good combination that empowered the music.

The band sounded better and better as the night went on. The audience drew spirit from the band and the band from the audience. Highlights included the emotional "Ghosts of Mississippi", the traditional country song "Lonely and Being Along", and "Blue Side of the Mountain" - a song recently covered by a contestant on American Idol. They returned for one encore: "Where Rainbows Never Die", a fan favourite.

I was a casual fan at the start of the evening. By the end of the concert, I was a Steelhead.


GCast 124:

Customizing Spelling and Grammar Checks in Microsoft Word

Learn how to customize the spelling and grammar checker in Microsoft word: Enable and disable grammar checking, set the language and dialect, and ignore spellchecker in parts of your document.


When I was a boy, my parents took me to see Doctor Dolittle - a charming musical film in which Rex Harrison played a globetrotting veterinarian who had the ability to talk with animals in their own language. It quickly became my favourite movie and I watched it every time it was on TV. I was vaguely aware that the title character was based on a series of novels, but I never read these books. Until now.

Hugh Lofting's 1920 novel "The Story of Doctor Dolittle" introduced the title character. He was an M.D. but he had so many pets in his home that his patients refused to visit, and his sister eventually moved out, leaving no one to care for him. His pet parrot Polynesia taught the Doctor the languages of other animals and soon he developed a reputation as the most effective veterinarian in England. His reputation spread to Africa, where he was asked to come and cure a colony of sick monkeys.

On his journey, he was kidnapped by an African king, hunted by pirates, and rescued an old man from a cave. Most of his success was due to the help of the local animals.

It is worth noting that at least one scene does not age well. When we first encounter the African king, he is angry at the white Europeans thanks to the exploitation he experienced from previous imperialist visitors. These seemed progressive for a book written a hundred years ago; but a few chapters later, the king's son asks the Doctor to fulfill his dream of becoming a "White Prince". This scene has been cut from some editions, but it was left in the one I read, and it will not sit well with most modern readers. It appears that some racial epithets were removed from this edition.

Despite that, the story is fun, even for a grown-up like me. Lofting leads us from adventure to adventure and it is Dolittle's kindness to animals that is his greatest strength.


Episode 705

Douglas Starnes on Python and Azure

Douglas Starnes discusses how Azure, Visual Studio Code, GitHub Codespaces, and other Microsoft tools support Python and Linux tools.


Robert Jordan's "The Dragon Reborn" continues the adventures of the heroes of "The Wheel of Time" series. As in the first two books, volume three has the characters split up and travel across the world Jordan has created.

It was not a well-kept secret, but the previous book ended with an astral projection of Rand al'Thor's battle with Ba'alzamon Now virtually everyone in this world knows that Rand is The Dragon Reborn - the powerful male mystic, who is destined to change the world in a way that will either save or destory it; and he will likely be driven mad by the power he wields.

Rand sets out on a quest to find his destiny. Moiraine, Lan, Loial, and Perrin depart soon after to search for him. Egwene, Nynaeve, and Elayne move forward with their training to become full sisters of the mystical female Aes Sedai order; then go searching for the Black Ajah Liandran, who betrayed the Aes Sedai. Perrin wrestles with his growing connection with wolves, while Mat is finally cured of the evil that infected him when he stole a cursed blade.

In the end, the major players converge in the province of Tear and Rand again fights the dark lord Ba'alzamon, believing again this is the final battle between the two.

Oddly, the title of this book refers to Rand; we hear far less of him in these pages than of his companions. This worked, as the reader gets more development of the other characters - particularly Mat and Perrin. All these quests, characters, and subplots can confuse the reader. Fortunately, in this novel, Jordan stays longer with each group and place, allowing the reader to gain a comfort level with the plot thread he is exploring.

The Wheel of Time is a marathon and not a sprint and it is best to keep this in mind as Jordan builds his world and his characters. We learn more about his characters as they learn more about themselves.


Episode 704

Shannon Kuehn on Workload Identity Federation

Shannon Kuehn describes how to use Microsoft Workload Identity Federation to simplify authentication across Azure and other systems.

Links:
Shannon's blog

GitHub Federated Identity (Microsoft documentation)

GitHub Federated Identity (GitHub documentation)

Gallant GitHub Code (Az-CLI script)


"More About Paddington" is Michael Bond's second collection of stories about Paddington the Bear. As in the first collection, each story relates an incident or adventure in the life of the good-hearted anthropomorphic bear and the English family that has taken him in.

Each story follows a similar pattern: Paddington has an idea and sets out with the best of intentions; things go wrong and sometimes things get messy; everything works out well in the end. They are cute stories aimed at children, but they entertain enough to appeal to adults.

The stories in this collection are more cohesive than in the previous book. Each tale references the one before and they all take place over a period of a few months at the end of the year.

Paddington is quickly becoming one of my favourite characters in children's literature.


TomHarrellTom Harrell was born before the founding of Jazz Showcase, the southside club at which he performed Saturday night. Harrell looks every bit of his 75 years. He is old and frail and incapable of sitting erect in his chair. But he can still play, and his playing is still impressive.

Harrell has played with Stan Kenton, Woody Herman, Dizzy Gillespie, and a host of other jazz legends over the years; and he still loves the music enough to climb onto a stage and lead his quartet for two sets. The quartet was a terrific set of talent that included Ugonna Okegwo on bass and Adam Cruz on drums. But it was Venezuelan pianist Luis Perdomo who stole the show with his outstanding technique and solos.

Highlights of the evening included "Sea", a beautiful melody, which Harrell dedicated to the late Joe Siegel, who founded Jazz Showcase three-quarters of a century ago; and his version of John Coltrane's classic "Moment's Notice".

Although there was not much visually to the performance, the music was enough to satisfy a full house.



Episode 703

Mary Grygleski on Event Streaming and Processing with Apache Pulsar

Mary Grygleski describes how to use the Apache Pulsar open source product and its connectors to build scalable applications in pieces.

Links:
https://pulsar.apache.org/
https://pulsar-neighborhood.github.io/


April 2022 Gratitudes

Comments [0]

4/4
Today I am grateful for lunch in Little India with Nick yesterday.

4/5
Today I am grateful for a tune-up for my bicycle.

4/6
Today I am grateful to mentor Chicago high school students on their STEM project for the fourth year in a row.

4/7
Today I am grateful to see Sean Hayes in "Good Night, Oscar" at the Goodman Theatre last night.

4/8
Today I am grateful to get some legal protection against the person who has been threatening me for the past year.

4/9
Today I am grateful to see "King James" at the Steppenwolf Theatre last night with family and friends.

4/10
Today I am grateful that my sons took me to a Cubs game yesterday as a late birthday gift.

4/11
Today I am grateful for my first visit to the Field Museum in years.

4/12
Today I am grateful for a conversation with Glenn last night for the first time in too long.

4/13
Today I am grateful for 700 episodes of #TechnologyAndFriends and to all those who made it possible.

4/14
Today I am grateful for dinner last night with Nick and Tim before they flies out of town for Nick's bachelor party.

4/15
Today I am grateful for a long conversation yesterday with Jennifer and for the help, advice, and support she gave me.

4/16
Today I am grateful to celebrate Seder with friends last night for the beginning of Passover.

4/17
Today I am grateful for lunch with Debbie and her family yesterday.

4/18
Today I am grateful to celebrate a Virtual Easter with family and friends yesterday.

4/19
Today I am grateful
-to arrive safely in California
-that my abstinence from meat and alcohol is at an end
-to file my taxes on time

4/20
Today I am grateful to finally meet in person the people I have been working with for the past 6 months.

4/21
Today I am grateful for:
-a visit to 3 vineyards in central California
-coffee with Neha yesterday morning

4/22
Today I am grateful to successfully wrap up a customer project this week.

4/23
Today I am grateful for:
-Coffee with Christine yesterday morning
-Lunch with John yesterday
-My first visit to the newly-opened Microsoft office in Mountain View, CA

4/24
Today I am grateful to see the Tom Harrell Quartet in concert at Jazz Showcase last night.

4/25
Today I am grateful to bring out the cushions yesterday for the chairs on my balcony.

4/26
Today I am grateful for my fourth COVID vaccination shot.

4/27
Today I am grateful to those who offered support to me yesterday and for those who shared their own struggles.

4/28
Today I am grateful for co-workers who say and write nice things about me.

4/29
Today I am grateful to attend the ISTC STEM Challenge Showcase yesterday at the Chicago Cultural Center.

4/30
Today I am grateful that I no longer have to live paycheck to paycheck.

5/1
Today I am grateful to hear some excellent stories at The Moth Grand Slam last night.


KingJamesAmerican men are funny. So many of us like sports and talk about sports and even build friendships primarily on the basis of our shared love of a game or a team.

Rajiv Joseph's play "King James" follows two Cleveland Cavaliers basketball fans, who meet during LeBron James's rookie season. The city of Cleveland has not won a championship in over 50 years and sports fans are energized by the possibility of James - a generational talent drafted by the Cavaliers - changing that. Shawn (Glenn Davis) and Matt (Chris Perfetti) bond over their fandom and become best friends. So much of their friendship is based on their love of basketball and the Cavs. They debate, celebrate, and mourn as LeBron's career and decisions impact the team and its fans.

But the play is not about sports and the expectations that fans have of their athletic heroes.

But ultimately, sports was a red herring; it was nothing more than a reason to bring together the two friends. "King James" is more about relationships and friendships and how people understand one another and communicate. I saw myself in both Shawn and Matt and I felt their struggles. It was a bromantic comedy that even a bro would enjoy.

I left with a feeling of hope - hope that Cleveland sports fans must have felt following the 2016 NBA Finals.


Articles

A Sample JavaScript App Using the Bing Spell Check API

Calling the Bing Spell Check Service

Creating a Bing Spell Check Service

Calling the "Recognize Text" Cognitive Service from a .NET Application

Passing a binary file to a web service from a .NET app

Converting Images to Text with the "Recognize Text" API

Calling Cognitive Services OCR Service from a .NET Application

Calling Cognitive Service OCR service from JavaScript

Using the Cognitive Services OCR Service

Getting Started with the Cognitive Services Computer Vision API

Introducing Cognitive Services and Computer Vision

Natural Language Processing with LUIS

Cognitive Services Optical Character Recognition

Using the Cognitive Services Emotion API

Using Cognitive Services to Generate a Thumbnail Image

Generating a Cognitive Services API Key

Cognitive Services Make It Easy to Use Machine Learning in Your Application

Screencasts


Introducing Cognitive Services and Computer Vision

Creating Applications with the Analyze Image Cognitive Services API

OCR with Cognitive Services

Handwriting OCR with Cognitive Services

Cognitive Services Text Recognition service

Text Recognition Cognitive Service with Binary Images

Text Recognition C# Demo

Sentiment Analysis Cognitive Service

Sentiment Analysis JavaScript Demo

Using the Video Indexer AI Tool

Interviews

Sam Nasr on Cognitive Services

Martin Kearn on Document Recognition and Knowledge Extraction

Hamayal Choudhry and Samin Khan on SmartArm

Presentations

"Building and Training your own Custom Image Recognition AI" presentation at NDC-Oslo

Talking about Cognitive Services on the Eat Sleep Code podcast


GCast 123:

Ingesting Into an ADX Table From an Azure Storage Blob

In this video, you will learn to ingest data from an Azure Storage Blob into an ADX table. This is useful when you have a large amount of data to ingest.


After ten years working on Agile projects, I have discovered that just about everyone has their own spin on how to do it effectively.

Esther Derby and Diana Larsen emphasize frequent retrospectives, which they describe in their appropriately titled book "Agile Retrospectives: Making Good Teams Great".

I have been on many projects that waited until they ended before doing any kind of retrospectives - an event often referred to as a "Post-mortem". I have found these to be of little value. We spend a lot of time analyzing what went right and what went wrong, and we document it thoroughly and we file it away where no one reads it.

"Agile Retrospectives" suggests a more agile approach - conducting an analysis periodically throughout the project and using the information gathered to adjust the team's behavior and goals going forward.

Following a brief introduction describing the history and purpose of retrospectives, most of the book is devoted to a set of activities that one can organize and participate in during a retrospective. Each activity is broken down into the following:

  • Purpose
  • Time Needed
  • Description
  • Steps
  • Materials and Preparation
  • Examples

In addition, the authors sometimes list variations on the steps and description of an activity.

Each chapter reads like a set of recipes and this cookbook format makes it simple to select and follow each "recipe". While this book focuses primarily on retrospectives throughout a project - after each sprint, for example - many of the ideas and activities could also be executed at the end of a project or major deliverable.

One would not and should not attempt to involve their team in every activity described. There simply is not enough time and you will find that some activities are more relevant to your team than others. However, "Agile Retrospectives" contains enough good ideas to help you improve the next iteration by learning from the previous one. Read them all and pick those that will help your team.

I have always viewed Agile methodologies as a way to get and react to feedback as quickly as possible. The activities on this book will help your team do this.


Episode 702

Joe Kunk on Windows UI Testing

Joe Kunk describes how he uses the FlaUI open source tool to automate the testing of his WPF, Windows forms and modern apps.

https://github.com/FlaUI/FlaUI

I Will Remember Bogu

Comments [0]

BoguUntil I purchased my current condominium four years ago, I had always been the one responsible for cleaning my own home. But shortly after I moved in, I decided to hire someone to clean my place regularly. Bogu was referred to me by a friend, so I hired her to come every two weeks. The only exceptions were the times I was traveling.

For the next four years, she worked for me. She was punctual, flexible, hard-working, efficient, and thorough. I told her what to do and she did it, spending about 2.5 hours on each visit and keeping my home in good shape.

We never became close personally. Bogu spoke very little English, and I tried to stay out of her way when she was working. But I appreciated her work and her attitude, and we always greeted one another with a smile.

After this past Christmas, she stopped coming. I received no phone call or text message, and she did not respond to any of my messages. This was unusual, as she had always been very responsive. I called my friends who had referred her, and they knew nothing. I did not know any of Bogu's friends or family, so I had no way of knowing her situation.

Earlier this month, my friend called with the news. In early January, Bogu had died in her sleep. One morning, she simply did not wake up.

Bogu was a Polish immigrant who worked hard her entire life and never had a chance to enjoy retirement. I never heard her complain about anything. There are many such people in our country and in our world and most of them are forgotten.

I will remember Bogu.


GoodNightOscarIn 1958, Oscar Lavant was well known as a pianist, actor, and humorist. It was his sarcastic wit that set him apart and the reason that Jack Parr invited him as a guest on The Tonight Show. Unknown to Parr, Lavant's wife had him committed to a hospital to treat him for his drug addictions and temper. Luckily (or unluckily) he was released for a few hours to perform on Parr's show.

"Good Night, Oscar" - a play by Doug Wright - tells the story of that night.

The story moves seamlessly from hilarious one-liners to poignant moments between husband and wife to the tragedy of a man battling his demons. It is a roller coaster ride of emotions.

Sean Hayes, famous as Jack in the long-running TV show "Will & Grace" - is brilliant as Lavant. He captures a talented, but tortured soul, who craves an audience but deflects praise. He is the personification of imposter syndrome, forever comparing himself to his late friend George Gershwin to whom he can never live up. Portraying mental illness on stage or screen is always risky, but Hayes pulls it off successfully.

At the end, Hayes surprises us all with a vibrant rendition of Gershwin's "Rhapsody in Blue".

I have no idea how much of this story is true, but it was true for me. I felt the pain of the characters and I felt Lavant's need to salve his inadequacies with his caustic wit.

I came away more than satisfied.


Episode 701

Mihai Tataran on the Microsoft Azure Well Architected Framework

Mihai Tataran describes the Microsoft Azure Well-Architected Framework and how you can use its guidance to build a better cloud application - whether you are starting from scratch or migrating an existing application.

https://docs.microsoft.com/azure/architecture/framework/

Somehow, I missed Paddington as I was growing up. I missed him again as my children were growing up. I did not want to wait around for grandchildren, so I picked up "A Bear Called Paddington" - the first of fourteen novels written by Englishman Michael Bond about an anthropomorphic bear discovered by the Brown family in London's Paddington Station.

It was delightful!

The Browns discover a bear in Paddington Station and learn that he has stowed away on a ship from "Darkest Peru". They take him home and name him after the station in which they found him. He quickly becomes part of the family. Although Paddington is honest, polite, and kind, his curiosity and curious nature often get him into trouble.

This book does not contain an overall plot; rather, each chapter is a self-contained short story, and they are all loosely tied together. Each story relates an incident in which the bear finds himself in an unexpected predicament because, as Paddington admits "Things are always happening to me. I’m that sort of bear." It takes only a few pages for things to work themselves out - sometimes by luck and sometimes thanks to Paddington's positive attitude and usually for the best.

Although marketed as a children's book, this novel will appeal to readers of all ages. It is honest and funny and fun and full of adventure. It's that sort of book!


When I create a new software project, I often create a folder structure to hold components that I intend to write later. Sometimes thes folders are empty in early iterations - a reminder to me and others of where things should go.

This presents a problem when working with git source control. Empty folders are ignored by git, so they never make their way into the source control system.

The solution is to add a file to the folder, so it is no longer empty. Many people will add an empty text file named ".gitkeep". If a folder contains any file (even an empty one), git will not ignore it.

Technically this is a hack, but it's common enough that we can think of it as a psuedo-standard. Naming the file ".gitkeep" lets people know the purpose of this file and that it can be removed when useful files are added to the folder.


GCast 122:

Azure Data Explorer Materialized Views

Materialized Views are a features of Azure Data Explorer (ADX) that allow you to pre-aggregate data, making queries much faster. This video shows you how to create and use a Materialized View.


Overview

When designing a cloud application, there are many options. Even if you have settled on a cloud provider, the number of services from which to choose can be overwhelming. It helps if you group cloud services into three categories: Software as a Service (SAAS), Platform as a Service (SAAS),  and Infrastructure as a Service (SAAS). There may be some overlap between these services but thinking of them in this way can simplify your options.

Software as a Service (SAAS)

Software as a Service (or "SAAS" for short) describes services that are pre-built. You can sign up for them and start using them immediately with little or no configuration. Examples include email services, such as Gmail and Customer Relationship Management systems, such as Microsoft CRM. Many of these systems allow you to customize them, but that is your choice. If they match your needs, you can sign up and start using them very quickly.

These are the simplest services to use, as they require the least amount of custom code.

Platform as a Service (PAAS)

Platform as a Service (or "PAAS" for short) describes services you can consume or build on within your applications. They provide some basic functionality, such as a website, a web service, a database, or machine learning models. But they require some custom code on your part to make them useful. These services free you from implementing part of the application, so you can focus on the business logic.

For example, if you want to implement custom vision recognition in your software, you could create and train your own model or you could simply call the Custom Vision API of Microsoft Cognitive Services. The latter is much simpler because much of the data collection and compute processing has been done for you to create an existing model. You can then take the results of that model, use it to identify objects in a photo or video, and build the business logic of your application with that identification information. 

Infrastructure as a Service (IAAS) refers to software components that are pre-built

Infrastructure as a Service (or "IAAS" for short) describes the deployment of your data and code to a virtual machine or a container in the cloud. With this category, you are responsible for all the code and data, but the networking, infrastructure, and hardware are abstracted away from you. Those things are handled by the cloud provider automatically.

On-Premises

The final category used to be our only option. In years past, we had to buy, install, configure, and maintain our own hardware. In addition, we were responsible for installing and patching the operating system and any packaged software we are using. Each of the cloud solutions described above takes that responsibility away from us.

How to decide

As a general rule, it is more cost-effective to buy or rent a solution that fits your needs than to build that system yourself. This may be offset by the amount of customization you need to do, so take this into account when you choose your services.

We can think of the categories above as existing on a continuum, as shown below:

SAAS -> PAAS -> IAAS -> On-Premises

As we move from left to right along this continuum, we gain more control over our system; but we are required to do more of the work ourselves. Doing the work ourselves tends to be expensive because we cannot share that development cost among many customers, as a cloud provider can. On the other hand, we may require flexibility that is not offered by a given service, so we may need to move to the right and absorb that cost to gain that flexibility.

My recommendation when creating an application or service is to begin on the left side of the continuum above and consider if a service in that category will meet your needs. If it does, then stop and consider using that service. If not, move to the right and consider if the next category meets your needs, and so on.

If you can find a SAAS service that you can use, you should strongly consider that. It may be that no such service exists or that all are prohibitively expensive. If that is the case, look at PAAS services and try to find one or more that will meet your needs. If no such service exists, you may need to write your own. In this case, it may be possible to deploy your solution to a different PAAS service (Microsoft App Services and Microsoft CosmosDB, for example). However, if no PAAS service meets your needs, you may need to install and deploy everything yourself. The cloud can still help as you can deploy to IAAS services, using Virtual Machines and containers. You are still freed from maintaining the hardware and underlying infrastructure of the network.

Finally, there may be reasons you cannot deploy your application or data to the cloud. Accessing data or services across the Internet may result in unacceptable latency for your application or local laws may require you to store data in a country in which no cloud data center exists. In these cases, you can purchase and maintain your own servers in your own data center (or use a non-cloud data center). This requires the most work on your part and tends to be the most expensive option, which is why so many companies are moving to the cloud.

As you can see by the above example, there are no answers that fit every scenario. You need to consider your needs, the available cloud services, and each category.

Keep in mind that you can split your application into multiple services and deploy some as SAAS, some as PAAS, and some as IAAS. Doing this requires that your application is divided into component services that can be easily split.

If that is not the case, be aware that you can change your cloud deployment over time. For example, you may choose initially to "lift and shift" an on-premises application - that is, move it from a local server to a Virtual Machine in the cloud. Over time, you can refactor your code, dividing your application into smaller services that can be deployed as PAAS services. This should reduce the amount of code you need to maintain and reduce your costs.

Generally speaking, you should avoid "reinventing the wheel". Take advantage of work that has already been done. And take advantage of services that do things not part of your core business. Every hour you spend patching an O/S, installing a software update, and replacing hardware is an hour that you are not spending enhancing and refining your core business logic.


Episode 700

A Celebration of Friends!

At the end of each Technology and Friends episode, I ask my guests to say something that included the words "Technology" and "Friends". Here is a compilation of all their responses from the last 99 episodes!


Daryl HallTodd Rundgren, Daryl Hall, and their music were a big part of my life growing up. I saw Rundgren perform at a small club in Newport, KY almost 20 years ago; but I had never seen Hall in concert until Friday evening at the Auditorium Theatre.

Rundgren began the evening, performing for over an hour. He played his biggest hits - "Hello, It's Me", "I Saw the Light", "It Wouldn't Have Made Any Difference", and "We Gotta Get You a Woman", along with some deeper cuts from his five-decade recording career. He also delivered moving renditions of two Motown classics - Smokey Robinson's "Ooh Baby Baby" and Marvin Gaye's "I Want You".

After a brief intermission, Daryl Hall took the stage and brought his energy to a mix of originals from his solo career and songs made famous by other artists. He drew the biggest cheers while performing the music of "Hall & Oates" - a dynamic duo that cranked out hit after hit in the 70s and 80s.

For the encore, Rundgren joined Hall on stage, where they sang together for several songs.

ToddRundgrenBoth bands shared the same backing band - an unusual arrangement for a main act and warmup act. The band was electric - especially the saxophone player, who excited the crowd with his solos. Hall is still a known commodity to mainstream audiences, thanks to his performances on his long-running online show "Daryl's House". He tried to model the stage after the show’s set and kept the atmosphere as relaxed as the show with casual banter between songs. Rundgren still tours and occasionally records but has not maintained the strong public presence of Hall.

Hall and Rundgren are now in their mid-70s, and it is surprising how well their voices have held up. Hall was most famous for his range while singing the soulful melodies as the lead vocalist of Hall & Oates, but Rundgren has always shown great vocal range as well. The years have done little to diminish these skills.

The relationship between these two men goes far back. They both grew up in Philadelphia, they have recorded and toured together before, and Rundgren produced one of H&O's albums. Their closeness was apparent as they harmonized on stage.


March 2022 Gratitudes

Comments [0]

3/7
Today I am grateful to see "The Batman" at an IMAX theater yesterday.

3/8
Today I am grateful for a new shower curtain.

3/9
Today, I am grateful for an Indian lunch this week, courtesy of my employer.

3/10
Today I am grateful for dozens of new Twitter followers this week.

3/11
Today I am grateful for a decent sleep 3 nights in a row.

3/12
Today I am grateful to see a live performance of "West Side Story" in Lincolnshire last night.

3/13
Today I am grateful my flight arrived safely, even if it was 24 hours later than planned.

3/14
Today I am grateful for:
-Lunch yesterday with Stephen and Patrice
-My first visit to Staten Island
-An exciting Islanders-Ducks hockey game last night

3/15
Today I am grateful for an evening playing Top Golf in New Jersey

3/16
Today I am grateful to meet my teammates in person this week after working with them virtually for months.

3/17
Today I am grateful for 3 days in New York and New Jersey

3/18
Today I am grateful for an uneventful flight to Florida last night.

3/19
Today I am grateful for a day with new friends.

3/20
Today I am grateful for my first visit to the Everglades yesterday.

3/21
Today I am grateful for a visit to the Naples Botanical Garden yesterday.

3/22
Today I am grateful for the generosity and hospitality of Sean and Emilie

3/23
Today I am grateful for people who care about me.

3/24
Today I am grateful that my co-worker wasn't upset that I overslept and showed up 15 minutes late for our meeting yesterday morning.

3/25
Today I am grateful to finish preparing my taxes.

3/26
Today I am grateful to see Joseph in concert last night

3/27
Today I am grateful for brunch with friends in Wicker Park yesterday.

3/28
Today I am grateful for new slippers

3/29
Today I am grateful to work from the local Microsoft office for the first time in over 2 years.

3/30
Today I am grateful to receive a care package filled with nuts from my employer.

3/31
Today I  am grateful for dinner with my sons last night

4/1
Today I am grateful that Americans are focused on the world's most important issues, like: who was the biggest jerk at the Oscars

4/2
Today I am grateful to see Daryl Hall and Todd Rundgren at the Auditorium Theatre last night - my first concert at that venue.

4/3
Today I am grateful for exciting basketball games.


Episode 699

Jean Lange on the History of JavaScript

Jean Lange details how JavaScript grew from a scripting language to the language of the web and how studying its history helped her to make better technology choices.


Joseph in concertIt is unusual for a singer to warm up for her own band; but that is what Natalie Schepman did Friday night at the Old Town School of Folk Music. Schepman is one-third of Joseph - a vocal group from Oregon consisting of her and her two sisters: twins Allison and Meegan Closner.

Natalie had a solo career prior to forming Joseph, so she performed songs from that era, accompanying herself on acoustic and electric guitar, before taking a short break and returning to the stage with her sisters.

Each of the three ladies took turns singing lead on their songs performed throughout their 90-minute set. Their voices are distinct from one another, but each of them possesses an impressive vocal range on their own and together their harmonies are amazing! The only accompaniment came from Natalie's guitar playing, but that was enough.

I learned when I arrived that the show was all-request - fans were encouraged to suggest songs on the group's website and they played only those suggested by the fans. Although I was unfamiliar with Joseph before attending the concert, the crowd was not. Many came wearing shirts with the band's logo and large numbers sang along to the songs. It was a fun atmosphere and at least two couples were inspired enough to get engaged during the concert.

I arrived curious and I came away a fan of this trio.


In the universe of Robert Jordan's epic "The Wheel of Time" series, events repeat themselves with each turn of time's wheel - A turn that lasts thousands of years and spans many ages. As the Wheel turns, people are reborn to fulfill a destiny predetermined in a previous age.

One such person was The Dragon - a man who wielded the power of the universe to defeat the forces of darkness but was driven mad by that power.

It is now thousands of years later, and prophecies have foretold the coming of a new Dragon. Rand al'Thor, the protagonist of "The Eye of the World", has wielded this same power to temporarily defeat the villain Ba'alzamon and he now realizes that he is the Dragon Reborn. Rand is a shepherd from an isolated village who is unprepared for this responsibility and fearful of the madness that may consume him.

"The Great Hunt" is volume 2 of tWoT series and takes us with Rand as he and his companions seek the Horn of Valere - a sacred artifact stolen by Dark friends from those who follow the Light.

As in tEotW, we follow the travelers on their quest as they battle magic, demons, and other obstacles until they conclude their quest. Along the way, we get a closer look into the characters that make up the series and the world they inhabit. They grow up a little, hone their skills a little, and evolve a little. The action comes swiftly, but the characters evolve slowly - presumably to make the arc last for over a dozen books in the series.

The pacing is better in this novel than in Book 1, which dragged at times. Topics introduced in the first novel are explored further in Book 2. We see rivalries between factions of the Ais Sedai - a cult of female warriors / sorceresses capable of channeling the universe's power. We see women betrayed and enslaved by other women; we see Perrin tapping into his power to communicate with wolves; and we see Mat bonding himself to more magical devices with tragic consequences.

Jordan continues to draw ideas from Tolkien (calling upon an army of ghosts to fight your battle is an idea lifted straight out of Lord of the Rings), but he includes more of his own ideas in this story.

This is a worthy sequel and an inspiration for me to continue to Book 3.


Episode 698

Godfrey Nolan on Drone SDKs

Godfrey Nolan has been building applications that access the power of drones. He explains the current and future practical uses of these applications and how to get started using SDKs from drone vendors.

https://riis.com
https://www.meetup.com/drone-software-meetup-group


So many of us try to do everything. When an opportunity presents itself, we take it. When someone asks us to do something, our first instinct is to say "yes".

Greg McKeown advises us against this. In his 2014 book "Essentialism: The Disciplined Pursuit of Less", he tells us to think before saying "yes" or taking on more than we can handle. We can simplify our lives by focusing our energy only on those things that bring the most rewards. By eliminating non-essential things from our lives, we can focus more on what we value most.

McKeown provides some practical advice on how to become more focused. He tells us to block off time for ourselves; to take time to consider what in our life is most important and what we can eliminate; to take care of our bodies by getting sufficient sleep; to recognize when things are going poorly and know when to step away; to build buffer time into our schedules; and to learn how to politely decline requests.

Essentialism seems like common sense, but we are all guilty of overextending ourselves at least some of the time. McKeown's advice is sound. When I sold my house and moved to a small apartment in Chicago 8 years ago, I was able to rid myself of a lot of physical clutter. But I am still burdened with mental clutter. I am among those who are easily distracted and stretched too thin. Prioritizing my life would add value to it; especially if I could drop those things at the bottom.


Episode 697

Russ Fustino on Blockchain

Algorand Technical Evangelist Russ Fustino teaches us about Blockchain, how it works, its components, its uses, and how you can get started developing for it.

https://developer.algorand.org


JohnPaulWhiteCan one be happy listening to sad songs all evening? John Paul White proved that one can when he performed Friday night at the Old Town School of Folk Music in Lincoln Square.

His career has spanned over a decade and he has released solo albums, as well as those from Civil Wars - his Grammy-winning collaboration with Joy Williams.

But on this night, he focused on his solo songs, which he performed accompanied only by himself on guitar. And, of course, he focused on his sad songs from "The Hurting Kind" to "The Once and Future Queen", and "My Dreams Have All Come True", a song about a man whose dream of a painful breakup turns into reality.

At one point, he reassured the audience: "In case you're wondering, I'm ok", explaining that he wrote his best sad songs when he was happy.

White's voice has great pitch and range, reminiscent of Roy Orbison and some of the best Country & Western singers. His songs challenge that voice and it rose to that challenge.

The opening act was also a delightful surprise. I was unfamiliar with Parker Millsap, but his excellent songwriting, voice, and guitar picking won me over.

White's show was appreciated by an audience familiar with his music. In addition to his own songs, he played a handful of covers, including Electric Light Orchestra's "I Can't Get Her Out of My Head" with which he closed the evening.

It was a sad evening. But it made us happy.


In a previous article, I showed how to ingest data inline, listing each individual row to ingest. For large datasets, this is not a practical way to initialize data in a table. Another option is to ingest from CSV data stored in an Azure Storage Blob.

The syntax is

.ingest into table tablename
(
     h'blob_url_and_sas_token'
)

where:

  • tablenameis the name of the ADX table into which you want to ingest data
  • blob_url_and_sas_token is the URL of blob, followed by the SAS token of the Azure Storage Account containing that blob. There are no spaces between the 2 strings.

Each row in the CSV file will create one row in the ADX table. The format of the CSV file must match the schema of the ADX table.

Demo

For the demo below, we will use a table defined by the following ADX code:

.create table testdata(
     timeStamp: datetime,
     someNumber: long,
     someString: string,
     someJson: dynamic
     )

and we will create a blob containing the following text:

2022-02-08T21:25:04.811Z,1,foo1,{'foo':'bar1'}
2022-02-08T21:25:07.838Z,2,foo2,{'foo':'bar2'}
2022-02-08T21:25:10.912Z,3,foo3,{'foo':'bar3'}
2022-02-08T21:25:13.829Z,4,foo4,{'foo':'bar4'}
2022-02-08T21:25:15.415Z,5,foo5,{'foo':'bar5'}

Blob and Storage Account Configuration

In an Azure Storage Account, create a blob container and upload to the blob container a file containing the comma-delimited text above.

NOTE: For more information on working with Azure Storage Accounts and Blobs, see this article

After uploading the blob file, you can the Storage Browser browser blade by clicking the [Storage Browser] button (Fig. 1) in the left menu; then, navigating to the container containing your import CSV file, as shown in Fig. 2.

storage browser button

Fig. 1

Blobs in Container

Fig. 2

Click your import file blob in the list to view the properties, as shown in Fig. 3.

Blob Properties

Fig. 3

Copy the URL of this file and save it for later.

Next, generate an SAS Token for the storage account. In the left menu, under the "Security + networking" section (Fig. 4), click the [Shared access signature] button (Fig. 5)

Security and Network menu

 

Fig. 4

SAS Button

Fig. 5

The SAS dialog displays, as shown in Fig. 6.

SAS blade

Fig. 6

At the "Allowed resource types", check the "Object" checkbox.

The valid start and end date times default to the current datetime through 8 hours from now. Adjust these if you want to access the blob beyond the end time or will not begin the import until significantly later.

Click the [Generate SAS and connection string] button to generate a SAS token. Generated data will display below the button, as shown inf Fig. 7.

Generated SAS and connection string

Fig. 7

Copy the value in the SAS token field and save it for later.

Ingesting Data into ADX Table

Open the Azure Data Explorer interface, log in and select the database containing your import table.

From here, execute the ingest command described above. For our sample, this is:

.ingest into table testdata
(
     h'https://dgteststorage.blob.core.windows.net/data-import/TestData.csv?sv=2020-08-04&ss=bfqt&srt=o&sp=rwdlacupitfx&se=2022-03-11T03:57:56Z&st=2022-03-10T19:57:56Z&spr=https&sig=x21QHp4nXaZp%2Bfc9h8cZD41grOOtd%2F73lY%2Fyk5hZsA4%3D
)

You should now be able to query the table with its data via the following KQL command:

testdata

Troubleshooting

If data is not imported, error information can be retrieved via the following command:

.show ingestion failures 
| order by FailedOn

Look at the row matching the time of the import (most likely the most recent row). Common issues are an invalid or expired SAS token and data that does not match the table schema.

Other Data Formats

ADX supports ingestion from other file formats than CSV. For example, Parquet, Avro, and JSON files can also be ingested. You can find a complete list here.


Episode 696

Matt Eland on How Humans Learn

Matt Eland is an instructor at Tech Elevator. He discusses the different ways that students learn and how to be a better teacher by understanding these ways.


February 2022 Gratitudes

Comments [0]

2/7
Today I am grateful for those who trust me enough to tell me their troubles.

2/8
Today I am grateful to the two people who sent flattering emails about me yesterday.

2/9
Today I am grateful to receive a response to a message I wrote to a friend in prison last week.

2/10
Today I am grateful to deliver an internal presentation on Azure Data Explorer and to those who told me they enjoyed it.

2/11
Today I am grateful to see 10,000 Maniacs in concert last night.

2/13
Today I am grateful to see America in concert last night for the first time.

2/14
Today I am grateful that the Chicken Piccata I made for friends yesterday turned out well, even though it was the first time I made it.

2/15
Today I am grateful to catch up with William yesterday.

2/16
Today I am grateful for a conversation with Paul yesterday for the first time in a long time.

2/17
Today I am grateful to co-present on Blockchain with Russell Fustino at the ETHDenver conference yesterday

2/18
Today I am grateful that this year's NFL playoffs were so exciting.

2/19
Today I am grateful to those who reached out to me when they sensed I was feeling low.

2/20
Today I am grateful to see Billy Branch & The Sons of the Blues at Evanston SPACE last night.

2/21
Today I am grateful to see a production of "The Virginian" at the City Lit Theater yesterday.

2/22
Today I am grateful for my first bike ride of 2022.

2/23
Today I am grateful for the arrival of my home COVID tests

2/24
Today I am grateful:
-for an unexpected visit from Nick last night.
-to present at the Valley .NET User Groups last night

2/25
Today I am grateful that the back pain that plagued me for months is almost entirely gone.

2/26
Today I am grateful to see Enter the Haggis in concert last night.

2/27
Today I am grateful for Celtic music.

2/28
Today I am grateful to see the play "The Moors" at Red Orchid Theatre in Old Town yesterday.

3/1
Today I am grateful for:
- an enjoyable and informative conversation with Laurent last night
- another decade around the sun

3/2
Today I am grateful for a birthday dinner with family and friends last night.

3/3
Today I am grateful I have been able to maintain a consistent gym schedule this past month.

3/4
Today I am grateful to present at Code Camp Romania yesterday.

3/5
Today I am grateful to see John Paul White in concert last night in Lincoln Square.

3/6
Today I am grateful to Tim and Natale, who took me out for a birthday sushi dinner last night in Wicker Park.


EnterTheHaggisEvery band should include bagpipes. And two fiddlers. And a fife and a recorder.

It works for the Canadian Celtic Band Enter the Haggis, who delighted the crowd at City Winery Friday night.

For two hours, ETH played a mix of ballads and rock songs - but mostly of Celtic or Celtic-influenced arrangements. The band originated in Toronto, Ontario 25 years ago, but they made their name by writing and recording songs influenced by the music of Scotland and Ireland.

They delighted the audience with the melodic "Down With the Ship" and the rousing "One Last Drink" and closed their encore set with the upbeat "Shangri-La". 

I saw ETH years ago when they performed at the CodeMash conference in Sandusky, OH. After the show, I had a chance to speak with band members Craig Downie, Brian Buchanan, and Trevor Lewington. I was delighted to hear that each of them remembered CodeMash fondly.

Perhaps not every band would benefit from the addition of bagpipes and a fife; but they could all benefit from the enthusiasm shown by Enter the Haggis on a chilly winter night in Chicago.

More photos


In this article, I showed you how to create a user-defined ADX function that returns tabular data. In this article, I will show you how to create a user-defined ADX function that returns a single scalar value.

Setup

This article assumes that you have an Azure subscription, an ADX cluster, and an ADX database. See the previous articles in this series to learn how to create an ADX cluster and/or database.

For the examples in this article, we will use a table created with the following ADX commands:

.create async materialized-view 
.drop table customers

.create-merge table customers
(
FullName:string, 
LastOrderDate:datetime,
YtdSales:decimal,
YtdExpenses:decimal,
City:string,
PostalCode:string
)

.ingest inline into table customers <| 
'Bill Gates', datetime(2022-01-10 11:00:00), 1000000, 500000, 'Redmond', '98052'
'Steve Ballmer', datetime(2022-01-06 10:30:00), 150000, 50000, 'Los Angeles', '90305'
'Satya Nadella', datetime(2022-01-09 17:25:00), 100000, 50000, 'Redmond', '98052'
'Steve Jobs', datetime(2022-01-04 13:00:00), 100000, 60000, 'Cupertino', '95014'
'Larry Ellison', datetime(2022-01-04 13:00:00), 90000, 80000, 'Redwood Shores', '94065'
'Jeff Bezos', datetime(2022-01-05 08:00:00), 750000, 650000, 'Seattle', '98109'
'Tim Cook', datetime(2022-01-02 09:00:00), 40000, 10000, 'Cupertino', '95014'
'Steve Wozniak', datetime(2022-01-04 11:30:00), 81000, 55000, 'Cupertino', '95014'
'Scott Guthrie', datetime(2022-01-11 14:00:00), 2000000, 1000000, 'Redmond', '98052'
'David Giard', datetime(2022-01-02 09:01:00), 1.50, 1, 'Chicago', '60605'

Syntax

Use the .create-or-alter function command to create a new function or modify one that already exists. The syntax is:

.create-or-alter function with (docstring = description, folder=folder_name] name_of_function ( parameter_list ) { KQL_Script }

where:

  • description is a brief description of the function. This is optional, but it is useful to help others understand the purpose of your function.
  • folder_name is a logical folder in which to store the function. This is optional, but it can help to organize your functions if you have many of them.
  • parameter_list is a list of input parameters to the function.
  • KQL_Script is the KQL code to execute when the function is called.

Parameters

Parameters are passed to a function as a comma-separated list of name/data type pairs, for example:

.create async materialized-view 
startDateTime:datetime,
endDateTime:datetime,
You can make a parameter optional by adding a default value, as shown below:
.create async materialized-view 
timeBinLength:timespan = 1h
This creates an optional param named "timeBinLength" of type timespan. If this parameter is not passed to the function, it will default to 1 hour.

Sample

Here is an example of an ADX function that accepts as revenue and expense as input parameters and calculates the profit from these values:

.create async materialized-view 
.create-or-alter function
with (docstring = 'Points drone passed through rolled up by time period', folder='Samples')
Profit(
revenue:decimal,
expenses:decimal
)
{
revenue - expenses
}

Calling the Function

Calling your function requires only the function name, followed by parameters in parentheses.

The code below calls the DroneRoute function, setting the timespan to 30 minutes.

.create async materialized-view 
customers
| extend profit = Profit(YtdSales, YtdExpenses)

The results of that call are in Fig. 1.

Results of calling Profit function

Fig. 1

Conclusion

In this article, you learned how to create and call a User-Defined Scalar Function in Azure Data Explorer. See this article for information on creating a user-defined ADX function that returns a data table.


Kusto Query Language (KQL) ships with dozens of functions that you can call from within your queries. However, you may also write your own functions and call those.

Setup

This article assumes that you have an Azure subscription, an ADX cluster, and an ADX database. See the previous articles in this series to learn how to create an ADX cluster and/or database.

For the examples in this article, we will use a table created with the following ADX commands:

.drop table droneData

.create-merge table droneData
(
DroneId:int,
TimeStamp:datetime,
Longitude:decimal,
Latitude:decimal,
BatteryLife:decimal
)

.ingest inline into table droneData <|
1, datetime(2022-01-10 10:45:00), -73.988888888888888, 40.742222222222222, 100.0
1, datetime(2022-01-10 11:00:00), -73.988888844444444, 40.742222215555222, 100.0
1, datetime(2022-01-10 11:15:00), -73.978820800000000, 40.741862453111111, 99.9
1, datetime(2022-01-10 11:30:00), -73.978800000000000, 40.741862453112222, 99.8
1, datetime(2022-01-10 12:45:00), -73.970000000000000, 40.741862453122222, 99.7
1, datetime(2022-01-10 12:00:00), -73.960555555555555, 40.741862453122222, 99.6
1, datetime(2022-01-10 12:15:00), -73.960000000000000, 40.741862453122222, 99.5
1, datetime(2022-01-10 12:30:00), -73.960000088888888, 40.741862453122222, 99.4
1, datetime(2022-01-10 12:45:00), -73.960000055555555, 40.741862453122222, 99.4
1, datetime(2022-01-10 13:00:00), -73.960000011111111, 40.741862453122222, 99.3
1, datetime(2022-01-10 13:15:00), -73.955555555555555, 40.741862453122222, 99.2
1, datetime(2022-01-10 13:30:00), -73.950000000555555, 40.741862453122222, 99.2
1, datetime(2022-01-10 13:45:00), -73.960000000000000, 40.741862453122222, 99.0
1, datetime(2022-01-10 14:00:00), -73.960555555555555, 40.741862453122222, 98.9
1, datetime(2022-01-10 14:15:00), -73.960555555555555, 40.741862453122222, 98.8

Syntax

Use the .create-or-alter function command to create a new function or modify one that already exists. The syntax is:

.create-or-alter function with (docstring = description, folder=folder_name] name_of_function ( parameter_list ) { KQL_Script }

where:

  • description is a brief description of the function. This is optional, but it is useful to help others understand the purpose of your function.
  • folder_name is a logical folder in which to store the function. This is optional, but it can help to organize your functions if you have many of them.
  • parameter_list is a list of input parameters to the function.
  • KQL_Script is the KQL code to execute when the function is called.

Parameters

Parameters are passed to a function as a comma-separated list of name/data type pairs, for example:

startDateTime:datetime,
endDateTime:datetime,

You can make a parameter optional by adding a default value, as shown below:

timeBinLength:timespan = 1h

This creates an optional param named "timeBinLength" of type timespan. If this parameter is not passed to the function, it will default to 1 hour.

Sample

Here is an example of an ADX function that accepts as input parameters a start and end time and (optionally) a timespan that defines a bin size. It returns a dataset of drone locations between the start and end time, but only returns one row per timespan defined by the timeBinLength parameter.

.create-or-alter function
with (docstring = 'Points drone passed through rolled up by time period', folder='Samples')
DroneRoute(
startDateTime:datetime,
endDateTime:datetime,
timeBinLength:timespan = 1h
)
{
droneData
| where TimeStamp between (startDateTime .. endDateTime)
| summarize arg_max(TimeStamp, Longitude, Latitude) by bin(TimeStamp, timeBinLength)
| order by TimeStamp asc 
| project TimeStamp, Longitude, Latitude
}

Calling the Function

Calling your function requires only the function name, followed by parameters in parentheses.

The code below calls the DroneRoute function, setting the timespan to 30 minutes.

DroneRoute('2022-01-10 11:00:00', '2022-01-10 13:00:00', 30m)

The results of that call are in Fig. 1.

Results of calling droneRoute function

Fig. 1

The code below also calls the DroneRoute function, but omits the optional timeBinLength parameter, so the default of 1 hour is used.

DroneRoute('2022-01-10 11:00:00', '2022-01-10 13:00:00')

The results of that call are in Fig. 2.

Results of calling droneRoute function and not passing optional parameter

Fig. 2

Conclusion

In this article, you learned how to create and call a User-Defined Tabular Data Function in Azure Data Explorer.


Episode 695

Chris Judd on Technical Skills That Are Most In Demand

We are currently in a good job market for those with technical skills. But some skills are more in demand than others. Chris Judd is CTO and Partner at Manifest Solutions, which puts him in an ideal situation to see which skills customers are seeking.


BillyBranchHarmonica player Billy Branch is a connection to the great Chicago blues artists of the past. He has played and recorded with Chicago legends Willie Dixon, Lou Rawls, Koko Taylor, Syl Johnson, and Junior Wells, as well as a host of others.

He founded and led The Sons of the Blues almost 50 years ago and brought the current incarnation of that group to Evanston's SPACE nightclub Saturday night. The venue was completely full, including some who watched while standing in the back.

Branch brought with him many friends - some of whom were descendants of great bluesmen like Willie Dixon and Little Walter. Little Walter had a special place in tonight's concert. Branch and the S.O.B.s recently released an album of Little Walter songs.
https://amzn.to/3h2mXQZ

Like Walter, Branch sings and plays the harmonica, and he delighted the crowd in doing so on this night. He drew heavily from this album, "Nobody But You", "Mellow Down Easy", "Juke"

The Sons' current lineup featured top-notch musicians Andrew "Blaze" Thomas. (drums), Marvin Little (bass), Giles Corey (guitar), and Sumito Ariyoshi (keyboards), along with Branch. Little occasionally stepped to the front to engage the audience with humorous banter, but it was Ariyoshi's playing that really shined - especially on his solos. The sweetness of his melodies contrasted with the blistering of Branch's harmonica.

At 70 years young, Branch lacks the stamina of his youth. When he told a story, he joked that he did so for a chance to catch his breath; He also stepped away from the stage to allow his younger bandmates to play a couple of their own songs; and the performance lasted only about 100 minutes. But it was enough to remind us how much we love the blues and how much we appreciate those keeping alive the legacy of blues in Chicago.



GCast 121:

Analyzing Geographic Data with KQL

Kusto Query Language (KQL) is ideal for analyzing geographic location data stored in Azure Data Explorer (ADX). This video shows how to use some of the features of KQL to implement this analysis.


If you find that you are often querying the same aggregation query of ADX data, it may be useful to create a Materialized View. A Materialized view performs the aggregation in advance as data is added to the table. We can then query the Materialized View, rather than the table, eliminating the need for our query to perform aggregation.

Setup

For the examples in this article, we will use a table created with the following ADX commands:

.drop table customers

.create table customers
(
FullName:string,
LastOrderDate:datetime,
YtdSales:decimal,
City:string,
PostalCode:string 
)

.ingest inline into table customers <| 
'Bill Gates', datetime(2022-01-10 11:00:00), 1000000, 'Redmond', '98052'
'Steve Ballmer', datetime(2022-01-06 10:30:00), 150000, 'Los Angeles', '90305'
'Satya Nadella', datetime(2022-01-09 17:25:00), 100000, 'Redmond', '98052'
'Steve Jobs', datetime(2022-01-04 13:00:00), 100000, 'Cupertino', '95014'
'Larry Ellison', datetime(2022-01-04 13:00:00), 90000, 'Redwood Shores', '94065'
'Jeff Bezos', datetime(2022-01-05 08:00:00), 750000, 'Seattle', '98109'
'David Giard', datetime(2022-01-02 09:01:00), 1.50, 'Chicago', '60605'
See this article for information on managing tables with ADX commands.

You can run the examples in this article in either the ADX Data Explorer web page or in Kusto.Explorer - a rich client Windows application that you can download for free from here.

Creating a Materialized View

Here is some of the syntax for creating a Materialized View, using the features that we found most useful:

.create async materialized-view
with (backfill=backfill_status, docString='description' )
materialized_view_name
on table source {
aggregation_query
}

where:

  • backfill_status is true, if you want to calculate the aggregation for all existing rows in the table. For large tables, this can take a long time. Set this to false if you only want to aggregate data inserted after the view is created.
  • description is a brief description of the view, making it easier for others to identify its purpose.
  • materialized_view_name is the name of the view to create
    source is the name of the source table (or another Materialized View)
  • aggregation_query is a KQL query and/or set of commands that returns a dataset. This query must include aggregated data, such as avg, min, or max.

You must add the async keyword if you set backfill=true.

You can find the full syntax here.

Here is an example:

.create async materialized-view 
with (backfill=true, docString='Summary customer sales' ) 
myMaterializedView
on table customers { 
customers
| summarize numCustomers=count(), minSales=min(YtdSales), maxSales=max(YtdSales), avgSales=avg(YtdSales) by PostalCode
}
Once we have this materialized view, we can query it as we would query a table, as in the following example:
myMaterializedView
| where avgSales > 100000
The results of this query are shown in Fig. 1

Fig. 1

Altering a Materialized View

Use the .alter materialized-view to modify an existing View. The syntax is nearly identical to the .create materialized-view command.

For example, the following command will remove the minSales aggregated column in the view I created above.

.alter materialized-view
myMaterializedView
on table customers { 
customers
| summarize numCustomers=count(), maxSales=max(YtdSales), avgSales=avg(YtdSales) by PostalCode
}

Removing a Materialized View

You can remove a materialized view with the .drop materialized-view command, as in the following example:

drop materialized-view myMaterializedView

Conclusion

Although there is an initial performance hit when rows are inserted, using a Materialized View can speed up your queries considerably.


Happy birthday to the .NET Framework that turned 20 years old earlier this month.

My first introduction to .NET was a rocky one. I remember hearing about it in 2001, shortly before its release, and learning about it in 2002 shortly after. I was working as a consultant at a Microsoft partner in Cincinnati that staffed me at the local gas and electric utility company, where I spent my days writing Oracle stored procedures.

One Saturday, my company put on a full day of training to introduce all to .NET; but I found the training confusing and got little out of it. It was delivered remotely at a time when remote online training was suboptimal, and it was delivered by people with little training experience. To further complicate things, Microsoft had not yet released an IDE for the product, so we were typing into a command prompt and wondering about the results.

A few months later, we secured one .NET project, but I was needed at the utility, so I watched from afar with a bit of jealousy for my colleagues who were learning something new.
 
My first real project came after I rolled off the Oracle project and was assigned to work on an ERP system a customer was building entirely in .NET. The customer was 160 miles away and their slow Internet connection made it very difficult to learn a new technology while working effectively on a project. Refreshing and building my source code each morning took over an hour; builds were long and often timed out; and it was difficult to tell if an error was caused by my code, by a network error. It was not uncommon for a developer in the other city to break an interface by checking in only some of their code. The client grew frustrated with my slow performance and removed me from the project.

I asked some of my colleagues to suggest a good book to help me learn .NET and several of them recommended Jeffrey Richter's "Applied Microsoft® .NET Framework Programming". This turned out to be the wrong choice for a novice. In his book, Richter spent far more time explaining the inner workings of the Intermediate Language and Garbage collection than he did on building a web form or connecting to a database - the more practical skills I was seeking. This was a book for someone already familiar with the basics - not for a beginner like me.

Resolved to learn the alchemy that was .NET, I bought a domain name and began working on a personal website devoted to Michigan State University athletics. It was a good project because I was an MSU fan then and now; and it was a great opportunity to learn and apply ASP.NET, create a site with templates and dynamic content. Soon, I had a site that automatically refreshed when I updated a database. I enhanced the site's features and content for 10 years until the domain name expired (I made the mistake of registering it with a work e-mail. A rookie error)

This is how I learned .NET and it was not long before I was passing certification examines and working on real-world projects and successfully helping customers. Within a year, I was teaching classes on building .NET applications. In 2011, I even co-authored a book on the topic.

Fast forward 20 years. .NET is still around. Its latest incarnation can run on Linux and Mac as well as its original Windows home. I have worked on dozens (maybe hundreds) of projects using this platform. I have built a solid career around it. I don't know how long it would have taken me to get started if I had waited for a good project or good training from my employer, instead of building something on my own to force myself to learn.

Here are the lessons I learned from that experience:

- Learn to recognize which technologies have a chance at making an impact. To me, it was clear early on that .NET was the future of software development on Windows. Sometimes, it is not as obvious. Listen to others, read what is written in journals and influential blogs, and see who is adopting a new technology.
-Do not be discouraged if your first attempts are unsuccessful. I was able to rebound from a frustrating beginning.
-When something new comes along that you think will be relevant for years to come, find a way to learn it. Don't wait for an opportunity - create one!
-Most important: Maintain a passion for learning! Continual learning is the best and worst part of this career I have chosen.


<< Older Posts | Newer Posts >>