Technical Interviewing - You're Doing It Wrong

Last week at the ALM Summit, Microsoft colleague Jonathan Wanagel and I presented a talk titled “Technical Interviewing – You’re Doing it Wrong!” and it was one of many highlights of the event. Download the slides here.

We identified eight anti-patterns that people should avoid. As Jonathan stated in the talk, “many of them were likely created at Microsoft” – and while I can’t say that, I can say that I’ve seen all of them at many companies around the world.

What we did was break them down into why these are bad and what to do about it using a forecast and the candidate experience.

The patterns we identified

• The Riddler
• The Disorienter
• The Stone Tablet
• The Knuth Fanatic
• The Cram Session
• Groundhog Day
• The Gladiator
• Hear No Evil

Here is a quick summary of a few of them. I’ll post a link to the video and audio once it’s up.

The Riddler

This is the person who asks puzzle questions to “see how you think”. An example of this would be the “messed up interview question” post on my site.

The Disorienter

Similar to the Riddler, this is someone who asks programming questions that have nothing to do with the job at hand. This could be writing Sudoku puzzles or my favorite, parsing roman numerals.

The Stone Tablet

This is the person who has a candidate write code on a white board. Really? A white board? I mean, I can understand it because that’s how all modern development is done and it’s obviously the best way to test someone’s skills.

The Cram Session

This is the person who “studies” for an interview, either by asking friends or by looking up common interview questions. The funny thing here is all one needs to do is answer the right questions and they have the job. What the skills are don’t really matter.

The Problems

The problem with each of these techniques is that they don’t create a realistic picture of the candidate’s skills or competencies. They create a poor forecast as well.

Forecast

To forecast is to make a prediction in advance. When we hire people, we make predictions based on written and verbal communication as to whether a given person will be a good fit, both culturally and skillfully, in the company over a period of time. Although, some organizations do this by having a trial period, most companies in the tech industry hire solely based on the interview. That means the forecast made in the interview has to be fairly accurate. Yet this forecast is only as good as the common understanding of why and how they are hiring.

Skills, Competencies, or Both?

Let’s say your reason to hire is that you need more C# programmers. Most companies would screen prospective candidates based on that particular skill: Do they know C# or not? While that makes sense on the surface, it might not be the best way to make a hiring decision. Skills, after all, are easy to learn and can be picked up in a matter of months. A developer who knows java but has never tried C# will be able to pick up C# after a few months, especially when paired with an experienced C# programmer. This assumes, of course that the developer has an ability and willingness to learn, to ask questions, to share his strengths and weaknesses; to put his ego aside and have the courage to learn a new language in order to best help his team.

Let’s reverse that, then, and imagine we hire a very experienced C# programmer who is uncomfortable working in pairs. He doesn’t want to improve or branch out from C#, so can never be taught new languages. And he doesn’t enjoy mentoring others, so he cannot share his extensive skill set. No matter how senior or experienced this developer is, he would not be a good fit at Scott’s company, or on any agile team, because of the misalignment of values.

Based on those two scenarios, screening candidates should always include competencies—how a person thinks, how that person interacts with others, and what that person values. These will impact how quickly he can assimilate with a team and how rapidly he can learn new skills. After all, you might need that same person to learn yet another new language a few projects down the road. In general, you should hire for long-term fit, not for short-term expertise.