I've been interviewing a lot of people lately. Trying to find truly great employees is such an interesting process. I find it difficult to really put my finger on what predicts a future epic employee. I've interviewed a large number of people over many years. I've also hired a good number, and then gone on to see how they performed.
Over the years I have constructed a mathematical model to try to help me gauge different attributes of engineers and ultimately score them. Its not a terribly fancy model, but I do have a number of attributes I rank a candidate on after an interview; these scores are then combined to give me a final candidate score. In many ways, this is a ridiculous thing to do -- try and distill a human being down to a simple number between 0 and 1. What I find, is that I am constantly playing with the model to see which attributes really matter and in what relative weight to one another. I often go back and forth with numbers until I can align them w/ my intuition; I realize this is a silly game, but I find the process helps me think through the many issues and realize ultimately what is important. So I put the scores of the absolute best people I have worked with, and the worst, along with the scores I think I would have given them after interviewing them; this helps give me a little perspective. Some really key lessons stand out for me.
1. Programming problems and puzzles only take you so far. I like to ask a lot of tough questions, but the one lesson that sticks out in my head, is of the hundreds of people I have interviewed over the years, the one person who stood out as the uncontested best at answering my technical questions, turned out to be the absolute worst employee. As a candidate, I recall being utterly impressed by this persons answers. I kept drilling him from all angles, and then said, ok, these answers are simply great and I have to hire the candidate. I did, and then I realized what a mistake it was. It turns out, this person was a complete Bartelby the Scrivener, cleverly refusing to do any work. He simply would not do anything, other than reinstall various operating systems on his machine, and answer other people's questions. It turns out, he was really quite good at answering peoples questions, but a "mere" oracle was not what we needed.
2. Sort of the opposite of my first lesson, is that someone who my instincts tell me is worth hiring, but I may still be hesitant on, may still be great. I barely hired at least one of my greatest all time hires.
3. Motivation is a hard thing to gauge, but an essential part of any great contributor. People that have something to prove, can be incredibly motivated. For example, I once hired someone who had never programmed anything more than some toy applications based on a reading of "Java in 21 days." I knew I was gambling, but I was just so impressed with how much this person knew about something he knew nothing about only 3 weeks prior. Obviously he had many other great technical attributes, a strong math and physics background, etc. In any event, he was able to rewrite 6 months of work from a 15 year veteran in 5 long days and have the system really shining within a few short weeks. A few months later he was the star programmer amongst a strong team of 10.
Okay, I'll end this post, as I'm not sure what my conclusions are, other than, its not a straightforward thing to hire the best of the best.