Debugging Sails Applications with WebStorm

Lately I’ve been experimenting with a Rails-like web framework based on node and express called Sails. While it’s still missing a few convenient features from Rails like view scaffolding, manual migrations, and model associations, it’s been fun to work with. I’ve been using the fantastic WebStorm editor from JetBrains, which has some great features for node development, including an interactive debugger.

What’s not straightforward, though, is how to enable debugging for a Sails app. Normally, you start a Sails application by running the sails lift command at the command line. This presents a problem when debugging with WebStorm, as the IDE assumes that you’ll be running the node executable, passing in the name of the startup file of your app. This is easy to fix, though.

First, create a new Run/Debug configuration in WebStorm, based on the default “Node.js” configuration. The “Node interpreter” and “Working directory” fields should already be filled in for you. Check to be sure the working directory is set to the root of your Sails app. In the “JavaScript file” field type “app.js” or choose it from the file explorer dialog by clicking the “…” button to the right of the “JavaScript file” field. Sails generates this file when you create a new project, but you probably wouldn’t notice it unless you were looking for it.

sails debug 1

Next, you’ll need to install the Sails npm module directly into your application, rather than relying on the module you likely installed globally if you were following the instructions on the sails.org website when getting started. To do this, type “npm install sails --save” at the command line while in the root directory of your app. Note the lack of the “-g” option; this stands for “global”, and it’s not what we want. The “--save” option just adds a dependency to your package.json file, so that if you clone the project from source control later, “npm install” will automatically download the Sails package.

You should now be able to use this configuration to run and debug your Sails app from WebStorm.

That’s great, but what if you want to use CoffeeScript in your Sails project? Both Sails and WebStorm support CoffeeScript, but the above setup will result in a CoffeeScript parsing error when you try to start up your app with any *.coffee files present. If you run “sails lift” at the command line, everything works just fine, but you’re losing the ability to debug your application. Frankly, I’m not sure what the difference is between the state of the environment created by “sails lift” and the one created by running the node executable against app.js. Fortunately, I found a workaround.

By installing the Sails module locally, you actually ended up with a copy of the code that runs when you invoke “sails lift“. All we need to do is tell WebStorm to use that instead of app.js. To do this, open up your run configuration in WebStorm and change the “JavaScript file” field to “node_modules/sails/bin/sails.js“. In the “Application parameters” field, type “lift“.

sails debug 2

This will allow your app to run via the WebStorm runner. To enable automatic generation of JavaScript files from your CoffeeScript files and the mapping files that will enable CoffeeScript debugging, we need to create a CoffeeScript file watcher. To do this, go into your Project Settings -> File Watchers, click the plus sign, and choose “CoffeeScript”. The default settings should work.

sails debug 3

Now, when you debug with WebStorm by clicking on the little green beetle, you should be able to set breakpoints in your CoffeeScript code and debug it line-by-line.

Entity Framework Code-First Performance Issue with String Queries

TL;DR – This StackOverflow post explains the observed issue and the resolution

The Problem

I noticed today that a particular lookup in our application was performing quite poorly, so I dug into it with DotTrace and Entity Framework Profiler (we’re using EF Code-First on .NET Framework 4.0 in this particular app). When I zeroed in on the query that was causing the slowdown, I was surprised to find that it was a very simple one that looked something like this:

context.MyTables.Where(m => m.SomeStringProp == stringVar);

Ruling Things Out

Looking at EF Prof, the query it was generating also looked quite simple:

SELECT [Extent1].[ID], [Extent1].[SomeStringProp], [Extent1].[SomeOtherProp],
...
FROM [MyTable] as [Extent1]
WHERE [Extent1].[SomeStringProp] = '1234567890'

The SQL query itself ran almost instantaneously when run through SQL Server Management Studio. Since it wasn’t the query (I thought), I figured it must be the initialization of the EF context, the generation of the SQL query, or the materialization of the MyTable object that was taking so much time.

I ruled out context initialization by adding another EF query before the problem one. This new query, now the first one executed against the context, ran fairly quickly, and the slow one was still slow. Then, I ruled out object materialization by substituting the Where() call with a SqlQuery() call, like so:

context.MyTables.SqlQuery("SELECT [Extent1].[ID] ... WHERE [Extent1].[SomeStringProp] = @param", stringVar);

Doing it this way completely eliminated the performance problem, but SqlQuery() returns fully-tracked entities, just like Where() would, so the bottleneck couldn’t be entity materialization.

The Real Culprit

At this point, I was pretty stumped. Surely the generation of such a simple query from that LINQ statement couldn’t be that slow?

Grasping at straws, I looked at the schema for the table in question, and I noticed something a bit out of the ordinary. The SomeStringProp column was defined as a varchar, not nvarchar like most of the other string columns in the database. Since .NET strings are Unicode, could that have been the problem? I tweaked the SQL query I was running with SSMS to the following to test out the theory:

SELECT [Extent1].[ID], [Extent1].[SomeStringProp], [Extent1].[SomeOtherProp],
...
FROM [MyTable] as [Extent1]
WHERE [Extent1].[SomeStringProp] = N'1234567890'

Bingo! This query ran just as slowly as the LINQ to Entities query.

The Solution

So, how do we tell EF to use the right kind of string when generating the query? It’s as simple as a single data annotation. I changed my entity class like this:

public class MyTable
{
    ...

    [Column(TypeName="varchar")]
    public string SomeStringProp { get; set; }

    ...
}

With that, the performance problem disappeared.

Hopefully this will help someone else who runs into the same issue!

Displaying Database-stored Images in ASP.NET MVC with Data URLs

I’ve worked on quite a few websites which featured user-uploaded images as part of the content. To implement a feature like this, we obviously have to store the image somewhere. One way to do that is to store all the uploaded images directly in the filesystem, then storing the name of the files for the corresponding record in the database. This has always struck me as a bit clunky, since we’re storing part of the data for a Contact (say) in the database, and part of it on the filesystem. I’d much rather store the image in the database and have everything all in one place.

The problem in a web context is that the normal way of displaying images is to render an <img> tag in the HTML and have the browser make a subsequent request to the server at the URL contained in the tag. If you stored your images in the database, this would mean that you’d need a separate action method that would query the database again and write the image content directly to the response stream. But using an HTML feature called Data URLs, we can actually render the content of the image itself to the markup on the page, and not need to make another server request.

The way to do this is to transform the bytes of your image into a base 64-encoded string, put a special prefix on it, and set that as the value of the src attribute of your <img> element.

So, assuming you had a Entity Framework Contact object that looked like this:

public class Contact
{
    public int Id { get; set; }
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public byte[] Photo { get; set; }
}

And a view-model like this:

public class ContactViewModel
{
    public int Id { get; set; }
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public string PhotoString { get; set; }
}

You could grab it from the database and send it down to the view like this:

public ActionResult Index(int id)
{
    var c = _contactEntities.Find(id);
    var vm = new ContactViewModel 
                 { 
                     Id = c.Id, 
                     FirstName = c.FirstName, 
                     LastName = c.LastName, 
                     PhotoString = "data:image/png;base64," + Convert.ToBase64String(c.Photo)
                 };
}

And render the view:

@model ContactViewModel;

...

<ul>
    <li>
        <span>First Name:</span> @Model.FirstName
    </li>
    <li>
        <span>Last Name:</span> @Model.LastName
    </li>
    <li>
        <span>Photo:</span> <img src="@Model.PhotoString" />
    </li>
</ul>

Keep in mind that this will increase the download size of your page, so you’ll have to weigh that against the convenience of storing images in the database. Also be aware that this won’t work in IE 7 and below, and data URLs are limited to 32k in IE 8. For more info, see the wikipedia article on Data URIs.

An Update After a Long Hiatus

It’s been almost two years (!) since my last post, and a lot of things have happened since then. Rather than dwell on the reasons, I’m just going to jump right in and give an update on what I’m up to. I want to get back into posting technical content, but I feel I’d be remiss if I didn’t mention the big changes in my life here.

Alice

This past April we welcomed our third daughter, Alice Josephine, into our family. She’s a wonderfully laid-back baby for the most part, but a dependable sleep schedule has still eluded us, which makes doing after-hours work like blogging a bit difficult.

Why is a raven like a writing desk?

 

MVP

A bit further back, about a year ago, I was awarded the Most Valued Professional award from Microsoft in the Data Platform Development area. At the beginning of 2011, I had started doing a lot of presentations at user groups and conferences about Entity Framework. Speaking was my main community contribution, along with leading the Shreveport .NET User Group.

This summer, after I’d turned in my contributions for the year for my potential MVP renewal in October, I received word from my MVP lead that the Data Platform Development expertise was being retired. Thankfully, they were willing to count my DPD contributions toward another area that I’d like to move into. I ended up choosing ASP.NET, and crossed my fingers; the bar for MVP is pretty high for that expertise. After a very nervous October 1st morning, I got my renewal email, so I’ll be an ASP.NET MVP for the next year. That means that I’ll probably shift the focus of my conference presentations to more web stuff.

 

Improving

At the MVP summit last year, Devlin Liles and Tim Rayburn cornered me and gave me the hard sell on coming to work with them at Improving Enterprises. They were very open and candid, and made me think seriously enough about the idea to talk to my wife about it. I thought she’d be pretty resistant to the idea of moving to the Dallas area, but that turned out not to be the case. Long story short, with her support, I successfully went through the interview process at Improving and took a position as a Senior Consultant at the Addison, TX this past July. We’ve moved to Frisco, which puts us only about a five minute drive from my mom’s house (which she is ecstatic about).

It’s been an awesome experience so far, and I’m getting to work on some interesting projects with some very smart people. I’m really looking forward to the Las Vegas retreat that’s happening in a couple of weeks!

 

Moving Forward

I realize that this post has been pretty terse, but I felt like I had to do it that way, or I’d never get through it! This was mainly a speed bump that I had to get over in order to move on to what I really want to talk about.

As a new ASP.NET MVP, I’m going to need to step up my contributions in that area in order to have a good chance of being renewed next year. Thusly, I’m going to try to start posting some ASP.NET-related content on here in the coming weeks and months.

Besides that, in order to simply exercise my writing muscle, I may post some non-tech-related content as well.  Expect things about coffee, parenthood, and World of Warcraft, most likely. ;-)

 

Video recording of “Taking the Pain Out of Web Deployments with MSDeploy”!

Shawn Weisfeld, the omnipresent “video guy” from INETA, was there to record my presentation at the Northwest Arkansas Code Camp, and the video has been posted to the INETA site.  You can check it out here.

I haven’t watched it yet because, like most people, I hate the sound of my recorded voice.  I’ll have to eventually, since it would be a great way to help me pick out my public speaking quirks, but I’m kind of dreading it. :-P

In any case, thanks to Shawn and INETA for recording and hosting the video!

Speaking Update

Just a quick note here about some recent and upcoming speaking engagements for me. 

A couple of weeks ago,  I gave my Mercurial presentation at Houston Tech Fest.  This conference gets better and more varied every year.  I saw presenations on everything from Xaml styling to the Core Data API for iOS.  I also got to meet Markus Egger, and spoke to him about writing for the project postmortem series in Code Magazine.  I’m working on a pretty cool project at work right now that I think would be interesting to Code readers, so hopefully I’ll be able to get started on that soon.

Last week I presented at both the Tyson Developers’ Conference and the Northwest Arkansas Code Camp.  I really enjoyed hanging out with some friends from the Northwest Arakansas area, as well as some of the other guys and gals I’m beginning to think of as the “Road Crew”.  No matter where I’m speaking, I’m getting to the point where I know at least a few of the other speakers, not from having worked with them or even lived in the same area, but from speaking at other conferences together before.  It’s kind of cool to start to feel a part of the “Brotherhood of the Demo”.  ;-)

Coming up soon, I’ll be speaking in Hattiesburg, MS at the Hub City .NET User Group, run by my friend Keith Elder.  I’m putting together an intro presentation on Silverlight for that group, which will be an interesting change of pace.  Most of my previous presentations have been fairly advanced or at least niche subjects, but I think having an intro-level talk in my repetoire will advantageous.

Over the weekend, I also found out that one of my submissions got accepted for CodeMash!  This will be the farthest I’ve ever travelled to speak at a conference, and the trip will end up being pretty expensive, but I’m really looking forward to this one.  The organizers have built up a stellar reputation for CodeMash, evidenced by the fact that all 800 tickets were sold in less than 4 days!  At this one, I’ll be presenting on the Entity Framework Code-First API that was released as a part of the Community Tech Preview 4 of EF.  Since the API is similar in some respects to Fluent NHibernate, I’ll be able to port some of the samples I used in my talk on that to the one on EF.

Hope to see you at an event soon!

Blank Page When Viewing an ASP.NET MVC Web Application

Since I recently purchased a new laptop for my personal dev machine, I’ve been working with a pretty fresh install of Windows 7.  As I was prepping a presentation for Tyson Dev Con next week, I ran into an odd and frustrating problem.  My presentation involves an ASP.NET MVC application, specifically served from IIS rather than the ASP.NET development server, so I enabled IIS and ASP.NET in the “Add/Remove Windows Features” dialog.

I successfully deployed the app to IIS, but when I hit the site, all I got was a blank page.  What was odd was that when I deployed a Web Forms app to IIS, it worked just fine.  This led me to think it was a problem with my MVC installation, but that wasn’t actually the case.

When you install IIS with the default features enabled, one of the things that is off by default is HTTP Redirection.  The first thing the default MVC template application does when you request the root of the app is redirect to Home/Index.  Enabling this IIS feature fixed my problem immediately.

Hopefully this will help someone who runs into the same problem!

Speaking at Houston TechFest

I’ll be speaking at the Houston Tech Fest this Saturday.  I’ll be doing the same presentation that I did at DevLink, “Introduction to Distributed Version Control with Mercurial.”  Planning to get some extra rehearsing done in the next couple of days so it’ll have a little more polish than last time.

I’m looking forward to the Community Leader Town hall the evening before on the Microsoft campus in Houston, led by my friend, Jay Smith. Always great to get together with other South Central community leaders!

Hope to see you there!

DevLink Retrospective

I had a great experience at DevLink last week.

For one thing, I got to hang out with a bunch of really smart people, some of whom I only knew through Twitter or their blogs.  I noticed during the dinner conversations and various Open Spaces sessions was that these guys read… a lot.  I used to be a pretty voracious reader, but that seems to have changed in the last few years.  Personal programming projects, research, and experimentation have taken the place of books in my evening routine.  While those activities may help me stay up-to-date technically, I think I’ve been missing out on bigger picture concepts that will only come from reading.  One of the takeaways from the conference for me was that I need to work on the balance between the two types of learning.  I plan to start going to bed earlier so I can read more.

Another thing that really started to crystallize in my mind at DevLink was that the primary value for me at conferences is networking and discussion with peers.  That’s not to say that the sessions weren’t valuable. On the contrary, there was a ton of great content, and I wish I had been able to see more of it.  But I’m a smart enough guy that I can usually pick up what I need about technical topics online through blog posts, articles, and tutorials.  What I can’t do every day is meet my peers face-to-face and begin to form relationships that are not only personally rewarding, but that can lead to career enrichment as well.  The various Open Spaces sessions I attended played a big part in this mind shift for me.  Kudos to Alan Stevens for helping facilitate such a great experience for the DevLink attendees.

I also got to hang out with some former co-workers from Praeses.  They took me to a restaurant where I watched my first UFC fight.  It wasn’t nearly as barbaric as I had expected, and I honestly enjoyed it a lot.  I may have to start attending the UFC-watching gatherings that those guys have from time to time.

My Mercurial session went pretty well, I think.  Based on the feedback, it definitely needs some polish in places, but I have a little bit of time to do that before I give it again at the Northwest Arkansas Code Camp and Houston Tech Fest.  If anybody who attended my session would like the slide deck, you can find it on my BitBucket account here.

All in all, it was definitely worth the trip up to Nashville, and I hope to make it back next year.

Speaking in August

This month, I’m speaking at not one, but two conferences!

The first will be DevLink, next week on the 5th thru the 7th in Nashville, TN.  I’ll be giving a presentation on “Distributed Version Control with Mercurial”.  I’m looking forward to catching up with a bunch of friends who are planning to attend.  If you’re going to be there, come see my session at 11:30 on Saturday!

The next one will be SQL Saturday in Baton Rouge on the 14th, where I’ll be giving my “Introduction to NHibernate and Fluent NHibernate” presentation again at 2:45.  I’ve done that one so many times lately, I think I could just about do it in my sleep at this point.  ;-)

Hope to see you there!