John Zumbrum

Product Engineer

Read this first

A weekend with Google AutoML Beta

Some time ago I built a site called slackernews.io to take the top-rated submissions from HackerNews and categorize them into newspaper sections. In this way I could focus on the ones I care to read while blithely ignoring the sections I don’t (cough election news cough).

This categorization was initially built using a machine learning classification model on IBM Watson. Unfortunately, IBM decided to make a breaking change to their API that would require me to rebuild the model and call it with brand-new endpoints (I thought the lesson from the success of Microsoft was to maintain backwards compatibility come hell or high water?).

Instead, I decided to spend the weekend hacking away on the newly announced (Jul 14, 2018) Google AutoML Natural Language Platform.

Key Learnings

  • The price is right: FREE!
  • AutoML has the feature-set I need.
  • For my use-case AutoML didn’t come close to...

Continue reading →


Limitations of SQL Server histogram based row count estimates

Recently, I came across a situation where a query with n-joins was occasionally slow. Here’s how I tracked down the problem…

Profile Overall Application

First, profile the application with Glimpse. This gives a baseline for expected performance. Preliminary stats show total request time around 6 seconds, the server action method was taking around 1.5 seconds to process, and database queries of 250ms.

After that I replicated the issue, which was a database timeout. Glimpse doesn’t load on error pages in our application by default so I moved on to the next level of profiling.

Profile Database Access

SQL Server Profiler. By setting it to only capture completed SQL batches and stored procedures, I isolated the offending query.

Running the procedure in SSMS and including the actual execution plan showed a time to complete of 2.5 minutes (well over the 30 second timeout...

Continue reading →


Why do my asp.net views take so long to render?

I’ve optimized my database queries with indexes and am caching output for 10 minutes at a time, but occasionally I have to recalculate the page, and it’s taking way longer than I like. Tools like firebug only show that the request spent a long time waiting on the server. What I really need is a tool to help me locate the bottleneck.

Enter Glimpse, an open-source tool to analyze load times end-to-end for C asp.net applications. It times everything from javascript, web requests, controller actions, view rendering, database queries, ajax calls, and more.

2016-08-09 07_23_06-Window.png

This image shows that the total time to render the page was 478ms. Of that, 351ms was spent server-side, and 198ms of that was on rendering the view. That means not the database access, but compiling the cshtml with embedded C at runtime.

How to reduce that render time? Precompile the views at build time!

Now that the views...

Continue reading →


Designing Infrastructure for SlackerNews Launch

Background

On Monday, I’m officially launching SlackerNews.io, the best of HackerNews, powered by artificial intelligence.

In this post I describe the steps I took to ensure the burst of activity on Monday morning would be handled gracefully.

Goal

Support up to 5,000 users clicking once every 10 seconds, with a median load time below 1 second.

Throughput:
5,000*6 = 30,000 pages/min -> 500 pages/sec.

Median Load Time:
<1 second

Tools

  • jmeter - most established reputation, painful UI, steep learning curve to get a simple test running
  • LoadImpact - SAAS provider, interesting platform, limited scope without subscribing. Initial results here
  • pingdom - gives page load time information, but not load testing. Results
  • Visual Studio Online Testing - easiest to get started with a simple test, more below

Test Details

Test Actions

Simply load the front page
2016-07-30 12_29_16-Settings.png

Settings

Constant load
2016-07-30 12_29_40-Settings.png

...

Continue reading →


SQL Server In-memory OLTP - Project Hekaton

Key Takeways

  • Uses different underlying data structures for data and index storage, thereby reducing or eliminating locks and contention
  • No more memory pages or extents
  • Memory is byte-addressable, whereas disk is block-addressable, Hekaton takes advantage of that difference
  • In-memory tables must have an index, rows are not contiguously located, even within the Primary Key; rows from different tables could be located side by side
  • Index data is not persisted to disk or transaction log; all indexes are rebuilt when the server is restarted
  • Enables natively compiled stored procedures
  • Can selectively migrate individual tables to in-memory to limit risk
  • Replication not supported!

Architectural Overview

https://blogs.technet.microsoft.com/dataplatforminsider/2013/07/22/architectural-overview-of-sql-server-2014s-in-memory-oltp-technology/

Limitations

https://msdn.microsoft.com/en-us/librar...

Continue reading →


Upgrading EntityFramework

Does ApplyCurrentValues() ring a bell for you? If so you’re probably as excited as I am with the latest version of EntityFramework and the simplified APIs for interacting and manipulating object state.

At Wheelhouse almost all of our applications leverage EntityFramework in some way, so I recently embarked on a mission to upgrade us to the latest production release: EF6.1.3. Here are my chronicles.

Upgrade Methodology

For All Versions

Steps

  • Upgrade/Install NuGet package EF6.1.3
  • Update POCOGenerator.tt
  • Update POCOGenerator.Context.tt
  • Open the .edmx file and update from database

From EF4.*

Code Changes

  • context.ContextOptions -> context.Configuration
  • .AddObject() -> .Add()
  • .RemoveObject() -> .Remove()
  • ObjectQuery<> -> IQueryable<>
  • SqlCommand/SqlConnection family of methods moved to context.Database This one is debatable, as we don’t actually need to drop a reference to...

Continue reading →


User research on UBER

Dear readers,

Frequently when I fly, I take a taxi to and from the airport.

Several reasons make this more compelling than driving myself:

  • I don’t have to worry about the security of my vehicle parked at the airport
  • I’m only 10 miles from the airport so the cost is low
  • I have a flat rate so I know exactly how much it will cost me
  • I can schedule the taxi to arrive at 5:30am
  • The taxi has never been late (so far) in picking me up from my residence

However, there are a few downsides:

  • Cleanliness of the vehicle
  • Age of the vehicle
  • Variability in drivers (safety, hygiene, friendliness)

To that end I’ve decided to look into alternatives, and figured it would be a fun thought experiment to view UBER through the lens of a product manager.

Rider interviews

First, I asked a few people I knew what they thought of UBER, and to describe their experiences. Then I decided to take the...

Continue reading →


Building Blocks of the C# ASP.NET MVC Web Stack

Summary

The ASP.NET web stack is Microsoft’s mature platform for creating feature-rich, powerful web applications.

Rather than explaining how to write code for the ASP.NET stack, this article intends to walk through each of the major building blocks, providing information necessary to successfully deploy, upgrade, and administer the ASP.NET Web Stack in your organization.


The Building Blocks

  • C Language
  • Compiler
  • MSBuild
  • .NET Framework/CLR
  • .NET Core
  • ASP.NET
  • MVC
  • NuGet
  • Visual Studio
  • IIS

C Language

  • C is a high-level language that gets transformed into MSIL (Microsoft Intermediate Language by the compiler
  • MSIL is then further compiled down to machine language by the Runtime
  • MSIL is platform/language agnostic. C, VBasic, F all compile to the same MSIL
  • MSIL is executed by the .NET Common Language Runtime (CLR)
  • The C language version is not explicitly tied to the version of...

Continue reading →


C# Changes for the ASP.NET MVC Developer

C 6

Released Jul 20, 2015

Along with Roslyn Compiler, .NET 4.6, and Visual Studio 2015

Summary

C6 focuses on streamlining common issues that require clumsy work-arounds that clutter the code.

Null-conditional operator, string interpolation, getter only auto-properties, static using statements, expression-bodied methods, and index initializers all contribute significantly towards increasing readability of C code, reducing the number of lines and complexity of code, and improving the elegance and ease of expressing the programmer’s intent.

List of Features

  • Null-conditional Operators

    int? length = customers?.Length

  • String interpolation

    var s = $“{p.Name} is {p.Age} year{{s}} old”;

  • Getter-only auto-properties and initializers for auto-properties

    public string First { get; } = “Jane”;

  • Nameof Expressions

    if (x == null) throw new ArgumentNullException(nameof(x));

  • Static...

Continue reading →


Visual Studio Changes for the ASP.NET MVC Developer

Visual Studio 2015 - Jul 20, 2015

What strikes me most about VS2015 is how eagerly javascript/html/css frameworks and build tools are being incorporated and supported by Visual Studio. These tools change so rapidly that I think it will be hard for Microsoft to keep up with them, but it is a welcome change. Definite shift in attitude towards bring-your-own, instead of our-way-or-the-highway.

Significant Changes for ASP.NET Developers

Significant improvements in every area. Especially note: CodeLens, Git, Lambdas in Debugger Windows, Conditional Breakpoints, improved Editor features for javascript/html/css, NuGet Gallery, Diagnostic Tools Window.

Editor Improvements

  • Lightbulb next to line numbers with suggested improvements
  • Refactoring: Inline temporary variable, and Introduce local
  • Renaming now highlights all instances, and renames all at once
  • Live code analysis and automatic...

Continue reading →