Do. Reflect. Learn. Repeat!

What prompted this post was this job ad for an experienced web developer by Netpositive and the discussions that it prompted - and the realization that there is no way I can explain my view within twitter's limitations.

I liked the ad because as a prerequisite for being invited to an interview, applicants are required to write a little web application (regularly read from an RSS feed, store it locally, display the posts on a page (with paging), and add Facebook's "like" functionality to the page).

Being able to conduct an interview based on code the candidate written at her own time before the interview has the following benefits:

• the interview-fever problem is eliminated - some smart people can't solve simple problems during the interview they would be able to do in a minute under normal conditions
  
• those that can talk the talk but can't apply the theoretical knowledge to real problems don't get past this filter
• those that cannot sell themselves during the interview but are good programmers can be considered
  
• as an interviewer, I can focus on what the candidate knows, can ask them to suggest ways to implement new features in an application they already familiar with
  
• it is more fair for people who might not know the jargon and terminology (though they certainly should learn it later), but are good at programming
• you can learn a lot that might not be uncovered in a regular interview, e.g.: how likely the candidate is to reinvent the wheel rather than looking for existing solutions
• the interviewer can screen applicants for requirement analysis if needed - just give an ambiguous enough spec
  
• those candidates, who just want to get a job rather than a job at the given company are likely not going to apply because of the extra effort required here. Some great candidates will not apply either; however, I think that is an OK risk to take.
  

The hardest thing with this approach is picking the problem to be solved -

• should be big enough, not just a one off script, but something where some design is required, so the interviewers can learn about the candidate
  
• should be small enough so a good candidate can complete it in only a few hours;
• should be a problem relevant to the job the opening is for, not another hello world program;
• should be such that it's not a problem if it's googleable - we all google all the time, the important part that the candidate should demonstrate an understanding of the googled solution
  
• should obviously not be real work for the company - I would not want to apply to a company that wants to use my work for free to deliver to their customers.
  

I'm not even going to attempt to give a silver bullet solution that satisfies all the above, because - as everything in the software field - it depends on your context. However, the below ideas could be used as starting points:

• problems from online programming competition archives, e.g.: UVa Online Judge, Sphere Online Judge, etc.
• dedicated online screening applications, like Codility
• using tasks (bugfix, new features, etc.) from OSS projects. Yes, it is free work in a sense, but it contributes to the applicant's resume and makes the world a better place! :)
  

---

1. DCifer on 2010/08/09 13:52:57: Just as I mentioned on Twitter, that I'm strongly disagree with this concept. The main reason is time.
   
   When searching for the right candidate I want to interview as much as I can. If I'm waiting for their application it gives a really extra overhead to check each solution. Most candidate will be hired by other companies, while I get to the end.
   
   Just remember in college, how long did it take to receive the grade for your programming exercise.
   
   Both candidate and worker on the same goal here. Get into position as fast as possible. If the Candidate found 10 interesting position and need to write 10 different application after the 4th all position will be closed until he got there.
   
   Remember that these tests are no standards either. So one is want to see your knowledge in Hibernate other in Toplink, than some of them just in JPA. Or maybe a completely different area like JSF/JSP ...etc. It takes lot of effort to look after each one for each test.
   
   To have a good programmer, there are certified tests. For example I had good experience with people having Sun JAVA certificates.
   
   But the certificates actually worth nothing most of the time. The candidate will go through no matter what on a technical interview. Because we want to know we can work with him.
   
   For example a candidate with really good programming skills still not fit, if he favour inheritance over delegation in a team, where every one is delegate type.
   
   The write a program for our company only works in ideal environment. Where there is only one company to choose from, the company wants the candidate. The candidate wants the company. But this technique fails if there are many companies and many candidates on the market.
   
   Just think about it. 11 candidates work 4 hour to create your test application, one is hired. 40 hour lost. As a candidate I would begin to think about, that the management does understand that the project requires both time and money. Many manager fail to understand, that even the employees receive standard salary the project still depend on the time resource.
   
   But to be a little negative, actually I found cases where this is work. Mostly on outsourced companies, where the funds is not enough to hire highly skilled developers.
   
   These companies are searching for candidates, which no other companies wants. Because the candidates on the wrong side of the Demand and Supply they can go through on long interview process. The usual process is to write a small application, which can be shown to the mother company that the candidate has some programming skills.
   
   As the whole business not focus on completion, just do something which looks like IT, it is perfectly enough. During the interview process the company don't have to be afraid that the candidate taken away before the end. On the other side the candidates have the time as nobody bother him with other offers.
   
   My experience is, that the number highly skilled IT professional is low. On the other hand the demand is high. It does happen very often, that when you finally found the right candidate the next day he is taken away.
   
   If a company want to win the race for highly skilled programmers, he has to favour process, where he can interview as much as possible as fast as it can and a decide a quick as he can, or other better driven HR will take them away.
2. Peter Zsoldos on 2010/08/10 08:40:52: @DCifer if I understood correctly, you raised mainly two problems, namely
   
   1. it takes a long time to complete and evaluate a programming exercise, causing numerous problems for both sides
   2. you cannot cover everything in it - team fit, every technology/topic
   
   Not covering everything in a single exercise isn't a problem, these problems - IMHO - are used as a filter instead of a CV to be called into the interview, when team fit and other aspects of the candidate can be evaluated.
   
   
   I don't have experience to make any comments on the supply/demand/quality argument, so I will not :)
   
   Most jobs have an apply before date, so there is no race with time. If you want to apply to 10-20 companies (positions), then yes, this would take an awful lot of time, this process wants to bring in people who likely would like to work for *the* company, not work at *any* company as said. One goal of this process is to ensure that the ratio of quality applicants vs. all applicants should be high - not just technically, but from the point of view of how likely they are to accept an offer if made.
   
   Sizing the problem is the key to ensure you get enough quality applications. But having such a problem might help you actually filter out 199 out of 200 - http://www.codinghorror.com/blog/2007/02/why-cant-programmers-program.html , who won't even apply.
   
   This process might loose great candidates who will not apply, as said in the original post, but that might not be the end of the world.
   
   As for the company's side, evaluating the applications requires time from the company, but that's not wasted time in itself - reading and understanding code is a valuable skill, practicing that shouldn't hurt :)
3. Peter Zsoldos on 2010/09/29 23:43:58: It was a pleasant surprise to find another programming contest with a potential hiring offer added on top of the normal pricing (sorry, Hungarian only) http://www.legjobbfejleszto.com/