Interviewing candidates for a development position can be a daunting task. The consequences of making the wrong decision can be expensive and difficult to correct. One potential solution adopted by many employers is the pop-quiz interview. In a pop-quiz interview the interviewer asks technical questions with a very specific predetermined answer. Employers need a good method for filtering out the candidates who don’t have the requisite skills or knowledge required by the position, so they resort to testing candidates on their “basic knowledge” using a pop-quiz as a way to weed out those who can’t cut it.

There is a real problem being addressed here. Those making the hiring decision are often inundated with resumes, most of which are from people who are a poor match for the position being advertised. Thus there is a need for a certain amount of filtering to happen. One of the first things organizations do is look for a certain amount of education that they think is necessary or look for certain keywords on the resume. This practice has problems of its own, but it is almost a necessary evil. Once this level of filtering is done, there is nothing left but to talk to the candidates. But how does one person assess the technical abilities of another in a 30 minute interview? This is a difficult problem, and some see quizzes as an easy solution.

As an example, one company that I know of asks candidates “What are three ways that session state can be stored?” This is a bad question. First, storing session state is the application server’s problem, not the developer’s. Second, session state is something that developers should normally treat as a black box lest they write brittle code that is dependant on a specific version of an app server. There are a few instances when knowing how a particular app server handles session state can be useful; such as when clustering is being used and you need to mitigate the risks associated with session replication. However, on a day to day basis there is no need for a developer to know off the top of their head three ways that session state might be stored. While this information may ultimately be good for a J2EE developer to have, it is a poor test of the developer’s J2EE skills.

Many time these pop-quiz questions come straight out of popular Java books. These questions are an especially poor litmus test, as they are testing the candidate’s knowledge of a specific book instead of their knowledge of the broader solution space. Another company that I am familiar with asks candidates “What are two ways to implement the Front Controller pattern?” The answer, of course, is in Core J2EE Patterns where the Front Controller pattern is described. Anyone who has not read the book might be at a loss for the specific answer even if they have a working knowledge of the pattern. It is even debatable that knowing the Front Controller pattern is even all that important in these days of J2EE MVC frameworks like Struts and Spring WebMVC. Again, this is “nice to have” knowledge, but J2EE is a large specification and companies risk losing good candidates just because they are unfamiliar with one pattern.

There are situations where it is appropriate to ask pointed questions in an interview. If a candidate comes in and claims to be an EJB “expert” then a few technical questions are probably in order. Especially if the candidate doesn’t seem comfortable discussing their EJB experiences. The main problem is when these questions are used to eliminate otherwise viable candidates.

So, what should an interviewer look for when talking with a candidate? It is important to remember that almost any specific technical knowledge can be obtained. Therefore, it is the intangible qualities that make a good developer that are most important. Problem solving skills should be one of the first traits that an interviewer looks for. If a developer throws up his hands at the first roadblock, he won’t be of much value to the team. Speaking of teams, teamwork and the ability to work as part of a team are also critical skills. Perhaps the most important criteria is communication skills. Despite the stereotype of developers as antisocial, it is crucial for developers to understand the business problems that they are being given to solve and to be able to describe their solution.

Every candidate says that they have good communication skills and can work as part of a team. They also claim to be subject matter experts in their particular field. So how do you determine the veracity of their claims without resorting to pop-quizzes? By exercising some of those communication skills mentioned earlier. Often, it is sufficient to have the candidate talk about their previous developement experiences and listen carefully. Instead of quizzing the candidate on EJBs, consider using a line questioning like the following:

1. Did you use EJBs on your last project?
2. Were you involved in the decision to use EJBs?
3. What was the basis for that decision?
4. What were the benefits of using EJBs on that project?
5. Were there any drawbacks?
6. Would you do it again?

If you listen carefully to the answers to these questions the candidate will tell you everything you need to know about their knowledge of EJB. If the candidate cannot answer these questions in detail, that is all you need to know. They either do not understand the technology or are lacking the skills to communicate that understanding.

Finding good developers is a challenge. Companies need all the tools they can get to seperate the wheat from the chaff. Unfortunately, pop-quiz interviews are usually not a good tool. In the end, there is no substitute for sitting down with a candidate and listening to what they have to say.

Tags: ejb, interviews, java, jobs