editione1.0.8
Updated August 24, 2022You’re reading an excerpt of The Holloway Guide to Technical Recruiting and Hiring, a book by Osman (Ozzie) Osman and over 45 other contributors. It is the most authoritative resource on growing software engineering teams effectively, written by and for hiring managers, recruiters, interviewers, and candidates. Purchase the book to support the author and the ad-free Holloway reading experience. You get instant digital access, over 800 links and references, commentary and future updates, and a high-quality PDF download.
Technical interviews can be conducted in person or remotely, with one interviewer or a small group, synchronously or asynchronously, on a whiteboard, a laptop, or as a take-home test. Which technical interview formats will be most effective—and how those interviews should be conducted—depends on what signals the company finds most useful to gather. Each format collects different information, and each has pros and cons, pitfalls, and associated time and effort investment from both sides. When choosing a format, the company should consider whether it can be scaled and still conducted in a way that is fair to and useful for the candidates.
Deciding on formats can be a balancing act between time savings for the company and interviewers, quality of the signals that are truly predictive, and the need for a positive candidate experience.
Many processes will involve a combination of several formats for technical interviews:
Phone screens or video call screens.
Take-home tasks or assignment.
One or more onsite visits with in-person meetings, one-on-one interviews, pair or panel interviews, or working sessions. Onsites can be technical or nontechnical, or both.
Fully remote meetings.
Generally, formats move from screening (which help disqualify poor fits) to assessments (to qualify and measure candidate-company fit). Assessments include the technical or nontechnical. As a candidate advances through the interview process, the interview can ask more theoretical questions. Along the way, companies sell to candidates—but efforts to sell usually increase as a candidate moves through the pipeline, all the way to the onsite. Effort on both sides increases the further a candidate gets. But be wary of adding too much to the interview loop in the hopes of further defining the signal. There is a point of diminishing returns, and strong candidates are unlikely to stay through a never-ending process. If you aren’t getting the signal you need, it’s time to look carefully at each format you’ve chosen and figure out what’s not working.
| Synchronous | Asynchronous | |
|---|---|---|
| Onsite | Onsite conversation, Whiteboarding, Onsite coding | Onsite task or project |
| Remote | Phone screens, Remote conversation, Remote coding | Take-home coding project, Portfolio or prior work sample |
| Format | Effort and signal | Strengths | Weaknesses |
|---|---|---|---|
| Qualification call (general) | Low | Orients candidate; Sells them on process | May lose good candidates who need more selling or personal touch up front |
| Phone screen (technical) | Low | Time-efficient on both sides for disqualifying a candidate, especially if interviewer is technical and trained | False negatives high if interviewer is not technical (HR or recruiter); May lose good candidates who need more selling or personal touch up front; Difficult to do well, i.e. minimize both false positives and false negatives |
| Remote conversation | Medium | Logistically easier; Less costly; Ideal for remote workers | Less warm and personal for candidate; Less signal for company on personality and team chemistry |
| Onsite conversation | Medium | Largest volume of signal; Good for assembling signal from many interviewers at once; If done well, effective and time-efficient on both sides | Need a properly trained interviewing team and processes; Logistical costs |
| Onsite Whiteboard | Medium | Good for architectural questions; Can work for coding as well without requiring language-specific setup | Whiteboards are not representative of actual coding Whiteboard coding is controversial and disliked by many candidates |
| Remote coding (Coderpad style) | Medium | Much more representative of real programming than whiteboard coding Better signals on knowledge of a specific programming language | Fewer human cues make it more impersonal and stressful; Coding in a new environment can be awkward and stressful for some |
| Onsite coding or pair programming | Medium | Similar to Coderpad style, but can be more friendly or personal | Requires skill and preparation from the interviewers for consistency |
| Onsite task or project | High | Giving the candidate a little more time and a project can be highly representative of real work | Harder to structure and do well; Can be confusing or stressful for candidates |
| Take-home coding project | High | Much more representative of real programming than whiteboard coding; Optionally can pay candidate for their time | Demands a lot of time from candidate; Hard to scale and avoid cheating Biased against those without flexibility to do take-homes |
| Portfolio or prior work sample | Medium to high | Can be an excellent signal if it exists; Open source work can also show popularity and utility of the work | Most candidates do not have this |
Based on data from their experience with over 900 engineers, TripleByte found that interviewing practices and formats vary quite a bit across companies.
Source: TripleByte
controversyThe “best” interview format for coding questions is a notoriously controversial topic. While many companies rely primarily on onsite, face-to-face coding interviews, a significant fraction of engineers consider “whiteboard coding” to be intimidating, stressful, and not predictive of job performance. On the other hand, alternatives like take-home tasks have drawbacks as well.
So how do you decide which style to use? Given the trade-offs, the answer may differ by company, by role, and sometimes even by candidate. For example, an internship-style arrangement might work for an entry-level candidate without a lot of work experience, while a laptop-based interview may help a more experienced engineer showcase their skills. Some companies combine methods (for instance, having a whiteboard interview as well as a laptop-based one). You might also offer candidates a choice of which assessment method they prefer, though that significantly increases the company’s work to develop, maintain, and evaluate the different assessment methods in a fair way.
importantWhen choosing or evaluating formats, consider how each format relates to general hiring and interviewing principles.
According to Scott Woody, Dropbox used to do a lot of take-homes, because they found they got some of the strongest signal possible on a candidate’s ability to solve problems in unique ways. But they had to start moving away from them because they couldn’t contain the cheating.* If we apply this back to the principles, we see how it was easy (if disappointing) to make the decision to switch formats—it’s not fair because some people are cheating, and it’s not effective because you can’t assess people’s work properly or comparatively. The company had gone about their commitment to the format in the right way: they built up a robust process around take-homes, wrote and used detailed rubrics, conducted blind assessments to minimize bias, and despite all of this, encountered an unavoidable pitfall. They wisely pivoted.
In reality, no technique is perfect, and the bulk of ineffectiveness in the industry results from poor adherence to principles in both design and execution. A well-designed and properly conducted whiteboard interview is more effective than a sloppy laptop interview or a take-home exercise that frustrates a candidate. Select whatever format will cover the skills that correspond to the requirements of the job. This will protect efficiency, effectiveness, and the candidate experience. When in doubt, you may wish to revisit both the hiring principles (of candidate experience, accurate assessment, fairness, and efficiency) as well as the interviewing principles (of minimizing bias, being predictive of on-the-job work, avoiding questions that can be gamed, and being mindful of both your and your candidates’ time).
A technical phone screen is an early conversation between a candidate and a current employee of a hiring company with technical ability. Most commonly, technical phone screens are conducted by engineers at the hiring company, but they may also be conducted by others, such as recruiters with deep technical knowledge. The content of a technical phone screen can range from a simple conversation to live coding.
confusion Note that for some candidates—particularly high-value candidates, referrals, and outbound candidates—interviewers may begin with a more general first conversation to get to know the candidate and gauge their interest. Depending on their experience, these candidates may skip technical phone screens altogether.
importantIn general, technical phone screens are better for disqualifying obviously poor candidates than for qualifying them. Secondarily, they can help identify potential superstars whom interviewers will want to focus additional energy on courting.