Take-homes

You’re reading an excerpt of The Holloway Guide to Technical Recruiting and Hiring, a book by Osman (Ozzie) Osman and over 45 other contributors. It is the most authoritative resource on growing software engineering teams effectively, written by and for hiring managers, recruiters, interviewers, and candidates. Purchase the book to support the author and the ad-free Holloway reading experience. You get instant digital access, over 800 links and references, commentary and future updates, and a high-quality PDF download.

A take-home assignment (take-home or takehome) is a coding task given to technical candidates to complete on their own time. Candidates are typically given a day to several days to complete a take-home.

​controversy​ Take-homes are controversial. While there are many pros for the companies assigning them, they are less valuable in terms of the candidate experience. Nonetheless, they do have some advantages for candidates.

Benefits of Take-homes

​candidate​Take-homes remove a lot of the stress associated with onsite challenges. Candidates get to use their own tools and work in the style they would if they were on the job. They can review and iterate on their work, take time away to think or rest, and rewrite. One senior engineer put it this way: “Most employees ‘take home’ their work if you think about it. You get work, you go away and think, you do it, you sleep, you come back and review it. That’s how our jobs work.”

For companies, take-homes have arguably the lowest false negative rate of any interview format—“the truest signal,” as Scott Woody, former Director of Engineering at Dropbox, put it to Holloway. A few factors account for this:

• Take-homes give the candidate enough time to do the work in an environment that they’re comfortable with, so you eliminate the noise of a whiteboard interview.

• It’s very hard to hide weaknesses in coding ability in a take-home, and follow-up conversations can tell you almost everything you need to know about how a candidate thinks through problems.

• Follow-ups also allow you to weed out and correct for any negative signal or false signal—for example, if a candidate cheats, talking through their work will help you figure that out.

Follow-ups are an important part of the take-home evaluation; in this respect, take-homes are the first step of a larger conversation. A good take-home will mimic assignments the candidate might reasonably be asked to do on the job and will give you practical signal on their abilities, creativity, and style.

Here’s how a sample assignment might progress. Let’s say the take-home is something like “Build a simple web-based calculator app” or “Build an AI version of Tetris.” When the candidate returns, you might read the code and interrogate it together, bringing the initial asynchronous assessment into a synchronous evaluation:

1. You ask the candidate to critique their work.

2. You may then ask them what they would do given another 20 hours; or say, “How would you invest 10 more hours on this?” Questions like “What feature would you remove?” and “What feature would you add?” will further refine your understanding of their work.

3. An additional useful question is, “What shortcuts did you take, and why did you think that was the right shortcut?”

Downsides of Take-homes

​caution​ The major downside of take-homes is the time commitment they require. Because the market is so competitive, asking a strong candidate to give up their weekend for a take-home can lead them to drop out of your process. Senior candidates may feel that being asked to do a take-home is a waste of their time. For these reasons, take-homes usually make sense further along in the funnel. They require a lot of engineering time and investment from interviewers and candidates, so they aren’t ideal for screening. Some companies opt to pay candidates for the time they spend on these assignments, but this doesn’t always make a difference. A senior manager at Dropbox told us that before the company pivoted from take-homes, 20% of candidates would simply not complete them. Less-competitive candidates were more likely to complete the assignment, because they didn’t have competing offers. The pass-through rate was close to 10%. If you’re asking candidates to invest 15 hours, and their chance of passing through is 10%, the value asymmetry is strong.

Despite the high signal achieved, interviewers, too, spend hours of their time designing take-homes, scaffolding in multiple languages, and reviewing code, and with such a low pass-through rate, this hardly pays out.

​danger​ The other major con of take-homes is that they explicitly discriminate against people who have families or adverse financial situations, or who work more than one job. If these candidates are in the hiring pipeline at more than one company, they may receive multiple take-home assignments at once, making them impossible to complete, and this may cause the candidates to drop out of your process.

Take-homes also open up the possibility that a candidate may cheat by asking friends to help or collaborate. Additionally, with a take-home, you’re asking candidates to work in a vacuum, which doesn’t match to most work environments, where you’re hopefully able to ask questions and get feedback as you progress.

​caution​ Take-homes are difficult to timebox and thus difficult to assess fairly. If candidates have been given 48 hours to turn in an assignment, you might be comparing candidates who spent 40 hours with those who spent only 2.

One way to avoid this false signal is to pay people an hourly rate for the assignment; but candidates can lie, either saying they worked more hours so as to receive the pay or fewer hours because they want to look impressive.

There are situations in which the pros of take-homes outweigh the cons. Smaller companies may find it easier to assign take-homes than to expend the time and resource investment in a longer pipeline, where multiple interviews would be needed to get the same signal. Younger engineers trying to break into the industry may prefer take-homes because they provide a chance to demonstrate skills they haven’t yet had a chance to prove on the market. If there’s a candidate you haven’t gotten clear signal from yet, adding a take-home to their pipeline will usually tell you one way or another whether the bet will pay off. Scott Woody, former Director of Engineering at Dropbox, told us that people who tend to shine on take-homes have nontraditional backgrounds: “They’re hackers, or they never took CS in college, and they’d fail out of our normal process. But we can see they’ve been doing all this practical work on the side, so let’s give them this practical thing and they’re going to build something singular.”

Take-home Tips

Using a tool like Takehome.io can help with timeboxing take-homes. It might seem like the option of timeboxing would help solve a lot of the cons of this format, but many engineers hold the opinion that time limits introduce further artificiality that compromises what could otherwise be a clear signal. There are tradeoffs any way you approach it.

If you do choose to give take-homes, it’s important to be clear with the candidate that the results will only be used for evaluation and not to produce work for the company. You might also provide an upper bound on the amount of time a candidate should spend on the take-home. When sending candidates the assignment, it’s important to let them know what it is that you will be evaluating—the code? the creativity? the speed? This will help ensure that they don’t waste time on something that won’t translate as much to the assessment and the eventual job. You likely also will want to avoid noting things as nice-to-haves unless they are truly necessary for the assessment.

One idea worth noting to help make the candidate experience better is to replace the take-home with a project that is done in the office during the onsite. Such a project still requires a logistical burden, but has the benefit of feeling like a symmetric exchange of time, particularly if it replaces multiple interview questions. The goal is to mimic the benefits the take-home has for candidates—let them work alone.

​candidate​ Understanding the reasons why a company might choose a take-home problem can help candidates prepare. This guide from Jane Phillips has a host of practical suggestions for tackling take-home coding challenges, along with an FAQ on common scenarios, like needing more time or what to do if you’re not familiar with a language or framework in the take-home problem.