editione1.0.8
Updated August 24, 2022You’re reading an excerpt of The Holloway Guide to Technical Recruiting and Hiring, a book by Osman (Ozzie) Osman and over 45 other contributors. It is the most authoritative resource on growing software engineering teams effectively, written by and for hiring managers, recruiters, interviewers, and candidates. Purchase the book to support the author and the ad-free Holloway reading experience. You get instant digital access, over 800 links and references, commentary and future updates, and a high-quality PDF download.
Anyone who’s run interview processes for enough candidates will be able to recall instances in which a smart manager made a bad call. Whether it’s hiring or passing, there’s no way to know whether this decision is the right one for the long run, but somehow a decision has to be made. It’s worth recognizing that no matter how careful and structured your hiring process is, it can never involve 100% certainty, because it is a human process and we are neither mind-readers nor oracles. But the more thoughtful you are about how decisions are made and what factors to consider, the more confident you can be.
candidate Knowing how a company you’re interviewing with makes hiring decisions will help you have a sense of your timeline with that company, and make the (often stressful) process less opaque. For instance, Google tends to take longer to extend an offer, since offers require both hiring committee and senior leader approval. When interviewing with a company, you can ask your recruiter or interviewer how that company makes hiring decisions and why.
How a company makes hiring decisions is usually related to the company’s decision-making philosophy in general. Netflix has a culture that values trust, freedom, and “diffuse accountability,”* and thus allows teams (and managers) to make their own hiring decisions. Google, on the other hand, believes that well-designed committees can make better decisions than giving a manager unilateral power. This philosophy usually extends beyond hiring to promotions and even technical and product decisions. Because hiring philosophy and strategy may be aligned with how a company or team runs day-to-day, it’s often a good idea (and fair) to share with the candidate how this decision will be made—it helps set them up for success and gives them important insight into the company.
Before you make an offer to someone, think about whether you’d like to have 10 times as many people like them in your company.Patrick Collison, co-founder and CEO, Stripe*
No two companies make hiring decisions in exactly the same way. Nor should they. Companies tend to design their decision-making processes in a way that reflects both their goals and their philosophy on recruiting.
At many smaller startups, the decision-making process tends to be relatively informal. As a company grows and management layers start to form, there is a risk that hiring quality can deteriorate. Hiring managers, faced with an urgency to build out their team, may be less selective about who they extend an offer to, racking up diversity debt in the process. Hiring managers may also be less aware of the company’s culture or do not participate in it fully, which can lead to cultural dilution. This is why understanding how hiring decisions are related to company philosophies and are part of maintaining company values is essential—the risks for losing those things over time is great. At the same time, as a company grows, strategies have to change. It wouldn’t make sense for a company of 100 to involve every employee in a hiring decision, even if that was the practice when the company was 8 people.
We’ve put together a list of (somewhat fictitious) company archetypes to help illustrate how different goals, philosophies, and company values can impact how hiring decisions are made.
ConsensusCo is a startup that believes in decision-making by consensus. Tyranny is the enemy—everyone should be involved and have a say. ConsensusCo might have a culture that is conflict-averse and prone to groupthink, leading them to “default to no” when hiring. Or it might have a culture that values healthy debate and disagreement a little too much. Any interviewer or team member can veto the candidate. Overall, ConsensusCo tends to be slow and conservative when hiring, and might hire engineers who are similar in personality and skill set to its current team. But, when someone is hired, the team is fully invested in the newcomer’s success.
candidate Candidates who interview with ConsensusCo should expect to meet a large part or the entirety of the team.
At AgileCo, everyone might get a say, but the hiring manager (often the founder) makes the ultimate decision about who gets hired. In some extreme circumstances, AgileCo might not tell its employees what position a candidate is interviewing for or allow interviewers to share their feedback with each other (especially if the candidate is a potential executive). In theory, AgileCo holds a “hire slow, fire fast” mentality. In practice, this means many people in the company privately will talk about hires who weren’t right or weren’t qualified. That said, the team understands that speed is important and that some risks might need to be taken, and most of the time believes management is taking reasonable risks.
CommitteeCo believes that “false positives,” or bad hires, should be avoided at all costs. CommitteeCo is highly scientific about its hiring practices and has collected significant academic research and in-house people analytics that show that interviewers and hiring managers are often prone to cognitive biases and poor decision-making. At CommitteeCo, a candidate’s application must go through a highly calibrated, impartial hiring committee and multiple senior reviews before an offer can be extended. This slows down the hiring process, but CommitteeCo believes it can quantitatively prove that this process is superior. Since CommitteeCo is hiring for company fit over team fit, it views software engineers in a somewhat commoditized fashion and expects to be able to easily move them between teams and managers. In fact, many new engineers at CommitteeCo don’t find out their actual team assignment until after joining.
CommitteeCo is a highly desirable employer and uses its selective employment process as a marketing point. That said, hiring managers there are secretly frustrated, and in fact, the more senior they get, the more likely hiring managers are to try to subvert parts of the process. Candidates interviewing at CommitteeCo are also frustrated, but often a lot less secretly.
AccountabilityCo values individual responsibility, trust, and decentralized decision-making. Hiring managers are empowered to make their own decisions and can make them quickly. This results in more flexibility, and a team at AccountabilityCo can hire a candidate with some rough edges if they are a good fit for the team. AccountabilityCo is liberal about firing. An engineer that ends up not being a fit can expect to quickly be out of a job (perhaps with a severance package). A manager that makes poor hiring decisions can expect the same.
AccountabilityCo is very critical about introducing any steps that might undermine managers’ autonomy. With this increase in decision-making freedom comes a cost—hiring managers “own” a lot of the recruiting process and, in addition to making a final decision, have to dedicate large chunks of time to everything from crafting roles to sourcing candidates.
Often, AgileCo’s grow up to be PragmaticCo’s. PragmaticCo straddles the spectrum between AccountabilityCo and CommitteeCo. PragmaticCo is generally anti-process, but understands that some process can be good. The company believes that it must have some amount of checks and balances and oversight on hiring decisions, but finds pragmatic ways of doing so (for instance, requiring a senior executive to review any hiring decision).
The process at PragmaticCo might be very fluid and evolve as the company faces different growing pains. For instance, as the company expands, it might realize that it’s no longer working well to have a senior exec review hiring decisions, as the gap between executive responsibilities and the day-to-day needs of teams grows. They experiment with other methods, like having a “bar raiser” on every interview loop.
Source: Holloway
While any of the above archetypes can be highly successful, each has pitfalls, and the way a company makes hiring decisions can impact the candidate experience and the type of people that get hired. The techniques employed to make hiring decisions are often driven by an underlying philosophy about who should be empowered to make decisions and how those decision-makers should be held accountable.
As you’re building your process, you might use these techniques:
In a recruiting debrief session (or huddle), interviewers, recruiters, and the hiring manager meet to discuss all the feedback from a candidate’s interview process and discuss the candidate’s performance. In advance of the session, the hiring manager should circulate interviewers’ written feedback for the entire group to review. The discussion in the debrief session itself is led by a moderator, often the hiring manager.
As this Guide uses the terms, debrief sessions differ from postmortem meetings in that debrief sessions are about assessing candidates’ performance while postmortem meetings are about assessing the recruiting and hiring process for a role.
At many companies, the moderator is the hiring manager, but not always. For example, at some companies, an impartial moderator is assigned to avoid any authority bias that might affect more junior interviewers (Dropbox used this technique in the past). Others start with the most junior team members to try to avoid biasing them with the opinions of more senior members.
In a debrief, every interviewer should have an answer to the question: “Should we hire this person?”* Healthy debate should be encouraged to avoid groupthink (to see examples of this in action, you can read about the Asch conformity tests). By the end of the meeting, it should be clear whether to reject the candidate, move them forward, or if additional information might be required.
A good debrief session can serve multiple purposes:
Assemble and combine feedback. By collecting and discussing all the feedback, the hiring manager can create a clearer view of how the candidate performed. It’s good to probe for “patterns in the noise.” For instance, maybe multiple interviewers detected a (positive or negative) personality trait in the candidate, but didn’t detect it with enough confidence to weigh it heavily. Or maybe the candidate was rude or condescending to interviewers (or recruiters) of a particular gender, age, or perceived status. There might be an area that wasn’t thoroughly tested and requires further information.
Discuss uncertainties and disagreements. A healthy debate can help surface where perceptions differ and help elicit how interviewers really feel about a candidate. The goal is not to come to universal agreement, but to be welcoming of opinions and explicit about uncertainties and about where people disagree, and why.
Calibrate and learn. Group discussions can help interviewers learn from each other and better calibrate how they interview and evaluate candidates. It’s often a good practice to tell everyone that a secondary goal of every debrief session is to help each other improve at future interviewing and hiring decisions.
Build teamwork on decisions. The verbal discussion can help every interviewer feel heard and bought into the final decision. It helps reinforce the fact that recruiting is a team effort. If the previous three goals are handled well, it can make the whole team more confident that hiring decisions are being made well and with the right inputs.
contribute We’re looking for more resources on how to run a great interview debrief session. If you know of any we’ve missed, or if you are interested in writing on the topic, please let us know!
At many large companies, before a hiring manager can extend an offer, a more senior leader of the company must review a candidate’s feedback packet and sign off on the offer. The assumption is that senior leaders will have a broader view of what the organization needs, be better-calibrated, and be more impartial toward the immediate hiring pressures the team is facing. At some companies, the senior review might just be the hiring manager’s own manager. At other companies, it might be someone higher up the chain of command or even from another team. For instance, Google co-founder Larry Page signed off on all offers until this became impractical.
In technical recruiting and hiring, a hiring committee is a group of employees who did not participate in a candidate’s interview loop and are tasked with deciding whether to extend an offer based on the interviewer’s feedback and the candidate profile. The members of a hiring committee are often from a range of job levels, which can make the decision process feel less hierarchical than requiring senior approval and may produce better decisions.
confusion Hiring committees don’t fit this description in all industries. In academia, for instance, the members of the hiring committee conduct interviews but may not have final say on extending offers.
Google, a strong proponent of hiring committees, points to research suggesting that a diverse group of employees reviewing offers might result in better, less-biased decision-making.
controversy The effectiveness of hiring committees isn’t without controversy, so it’s good to understand the trade-offs.
There are different types of hiring committees. The ones employed by Google are composed of impartial “peers and managers” from different teams. At Facebook or Dropbox, the hiring committee might be composed of directors or other senior leaders.
Since hiring committees, by design, introduce decision-makers who don’t interact directly with the candidate, there are a couple of prerequisites to making them successful. First, interviewers should be well-calibrated and must write very clear and consistent feedback (this is good practice, regardless of whether you’re using hiring committees). Second, hiring committee members should have enough context on the role to enable them to make the proper assessments. Given all this, even companies that use hiring committees in general sometimes break out of that model for roles that are highly specialized.
controversyIt’s rumored that Google once had a hiring committee review anonymized versions of their own interview packets, and the committee rejected themselves.
There can be lighter-touch approaches to introduce a decision-maker from outside the immediate team into the mix. For instance, at Amazon, every interview loop must contain a special interviewer known as a “bar raiser.” Bar raisers are interviewers who hail from outside the team with the immediate hiring need—to ensure that they are as objective as possible. They receive special interview training and calibration so that they are more effective at assessing things like values alignment. They also hold veto power over hiring any candidate and may often moderate the interview debrief. Coinbase also uses the bar-raiser method. Microsoft uses a similar system, known as the “As Appropriate” interview. As Appropriate interviewers are senior leaders who conduct a candidate’s final interview and work with the hiring manager to make a final decision.
Considering how little we have centralized, we use the bar-raiser group as a type of glue across organizations. We select bar raisers from the pool of experienced folk at Amazon, not just those who can interview well, but more importantly—those who deeply understand our leadership principles. As bar raisers, we then try to hire people who can understand and act on our principles.David Anderson, General Manager, Amazon*
No matter the strategy your company employs, it should align with your hiring goals and your company values. To that end, the following questions will be helpful to discuss among your team:
It’s worth considering the trade-offs between the different methods of decision-making. These should be consistent with your company’s culture, and some cultures are inherently more consensus-driven than others.
Requiring consensus will probably mean more selective hiring, since doubt from any interviewer can mean rejecting the candidate. This may be even more important for candidates for a leadership role. On the other hand, allowing some discretion for the hiring manager might be useful in certain cases. After all, the entire process is noisy and a judgment call might be needed. Or, in some cases, the hiring manager may want to augment the team with skills or traits that the current team is undervaluing. Assessing candidate-company fit or candidate-team fit requires an objective sense of the team’s strengths and weaknesses—it’s hard for teams to reflect on what they lack.
Most of the decision-making techniques described above prevent managers from making unilateral hiring decisions. But when managers can’t freely choose who will join their team, they may feel it is because the company doesn’t trust them to make their own hiring decisions, and feel less empowered on all fronts. As we mentioned earlier, Netflix’s culture of “diffused accountability” allows managers (and teams) to make hiring decisions independent from the rest of the company and without a lot of structured process. The company strives to have managers who feel like they “own the hiring decision every step of the way.” Imposing any friction or external decision-making could undermine their sense of accountability.
caution However, managers can be reluctant to take ideas from the team for the wrong reasons, often because they are tasked by their managers to focus on short-term outcomes.* In hiring, this can be particularly problematic when individual decision-makers or hiring managers override disagreement among the team related to interpersonal or values-related concerns—for example, underrepresented interviewers being ignored by hiring managers when they raise concerns about a candidate’s attitude or views. If anyone on the team is worried that bringing someone on will affect their safety or ability to thrive at the company, the decision-maker needs to listen.
It’s important that the team agrees on who is responsible for making a decision and understands the hiring structures in place. Even if the hiring manager (or committee) is right about the candidate’s skills and potential, the candidate will still have to rely on the rest of the team to succeed, especially in the critical first few months.
One way to think about this is whether you prefer hiring for each team or hiring for the company overall. If every team can make local decisions, it can be hard to ensure consistency in hiring, especially at larger companies. This can impact a company’s ability to move engineers internally and even allow cultural silos to form. Many companies with team-based decision-making also are also quick to let employees go if they are no longer a fit for the team they were hired for.
On the other hand, using techniques like hiring committees, which introduce decision-makers who won’t work closely with the candidate or team, will result in decisions that weigh the candidate’s fit for the company as a whole.
One of the core principles is to keep the recruiting process as fast as possible, while balancing speed with the principles of effectiveness and fairness. Efficiency is extremely important for the candidate experience, especially when a candidate is anxiously waiting on the final offer decision and may also be hearing back from other competitors.
But speed must be balanced with the principles of effectiveness and fairness. Having one person responsible for a decision will certainly be fast—but with only one person’s perspective, will it be fair? Is there enough confidence in that person’s decision that the candidate will thrive at the company and be in it for the long term?
A more structured process designed to emphasize effectiveness and fairness will usually lead to longer decision times. Companies with multiple layers of approval, typically large companies, are notorious for taking a significant amount of time to make a final decision. In Facebook’s early days, the company tried to work against this trend by having hiring committees meet on a daily basis for fifteen minutes to ensure faster turnaround times. There is always a tension and a balance to be found between these principles, and the more experience a company has in hiring, the more likely they are to figure out strategies that can meet that balance.
Now you’re set up to navigate the final decision about whether to make a candidate an offer. You have developed a philosophy and strategy around hiring, decided what you’re looking for ahead of time, designed and administered structured interviews, and agreed on who should be making the final decision. And sometimes, the decision can be straightforward. But often, things are not so clear cut—after all, you’re working with limited (and noisy) information.
Here are some tips for making the final call:
Stick to your hiring criteria as best you can. Did the candidate actually demonstrate the skills required for the role? How did they perform against a calibrated rubric? Take into account all the positions you had on strategic issues, such as how strong of a fit you need for this role. Putting all of that together, is this someone who will be successful once hired?
Look for patterns as you put all the feedback together. Sometimes, a behavior that might seem minor in one interview could be a red (or green) flag if it shows up as a common theme. For instance, did the candidate exhibit a bit of arrogance across all interviews? A single interviewer may have interpreted this quality as confidence, but if it shows up several times, it might be something else. It’s worth noting that interview debriefs can help surface these patterns more readily, but absent them, the decision-maker (hiring manager or hiring committee) will need to look deeply at the feedback and possibly follow up with interviewers. Reference check are also a good way to surface or dig into potential patterns.
danger Be conscious of what pressures you are subjected to and how they might bias you. Your personal disposition (risk tolerance) and your situation (hiring urgency) may push you to commit different types of hiring errors. The decision-making techniques we discussed above can help mitigate these errors, but it’s still worth being aware of how these pressures might manifest themselves:
If you are a hiring manager with an urgent need to fill a role, you might be prone to overlook a candidate’s weaknesses out of desperation.
If you are risk averse, you might find excuses to reject any candidate just to play it safe.
If you are hiring based on consensus, you might hire for lack of weakness over presence of strength.
important If you find that you’re frequently struggling to decide on candidates, that could be a sign that your interview process is not well designed. Make sure you have defined what you’re looking for concretely and communicated that to (or designed it with) the team. Ensure that your interviews actually test for those things, and that you have calibrated rubrics to help you evaluate candidates. Be aware of the cognitive biases that can impact people’s judgements, and check that comprehensive feedback is being independently gathered, written, and relayed to decision-makers.
This whole process leads to a weighty decision that will have a significant role in the course of someone’s life. It can feel overwhelming to anyone. One way to lower the stress of this high-stakes decision is to realize that everyone wants the same thing—to find the right fit. While the candidate may be really excited about your company, they might be excited about a company they have in their head and not the actual company you work for with the day-to-day challenges and requirements of its engineers. If the job requirements are beyond their skills, that day-to-day experience will be awful for them as well as the company. Coming to an objective conclusion about a candidate is to everyone’s advantage.
If the decision is to reject a candidate, the right thing to do is to deliver that decision with grace, respect, and speed. At this point, candidates have probably invested a large amount of time and effort in your recruiting process. Whether they receive an offer or rejection may have emotional and material repercussions, impacting their self-esteem as well as their career.
caution How you handle this interaction will also affect your future recruiting prospects. Bitter candidates may not apply again, even if the right role comes along in the future. They may also speak poorly of your company to their friends and colleagues, or on forums like Glassdoor, Blind, or Reddit, thereby deterring other potential candidates and affecting your brand. You might feel like you’d rather spend time on active candidates rather than rejected ones, but consider this an investment in your company’s brand and in upholding your company’s values—as well as your personal reputation. Indeed, as is so often true, doing the right thing is good for business.
Deliver the decision promptly. The longer you wait, the more pent up emotions the candidate might develop. If your decision-making process might take some time, you should set expectations for the candidate about when they should expect to hear back from you. If you’re still deliberating, reach out and give the candidate an update. If you have already decided but are stalling because you are dreading letting the candidate down, realize that the longer you wait the worse it will feel for both you and the candidate. If the candidate is reaching out to ask you for an update about your decision, you probably haven’t managed this properly.
Be considerate. Dismissing a candidate by firing off a generic email, quick text, or voicemail might feel quick and painless to you but will make a candidate feel disrespected. More direct communication is better. A personal email is better than a generic note, but a call is often best, if feasible. Getting on a call to break the news shows courtesy, even if the conversation might feel uncomfortable for you. On Twitter, Jennifer Kim offers some great advice about what language to use. (Of course, if it’s taking a lot of back and forth and several days to schedule a call with a candidate just to deliver a rejection, this is inefficient and even unfair for both sides.)
Hold yourself accountable. By making a decision reasonably quickly and delivering it in a considerate, direct way, you are respecting your position by owning the relationship and the decision with the candidate. You’re making an important decision in someone’s life, and neglecting a rejected candidate—or, worse, ghosting them—means you’re not doing your job.
important If you’re doing this well, candidates walk away from your process wishing they could still work with you, even after being rejected. A tangible sign of this is if candidates who you have rejected later refer other candidates to your company.
controversy Many companies don’t give rejected candidates any specific feedback, citing legal risk, lack of time, and angry candidates as the primary reasons. This can be frustrating for candidates, who, upon asking for more specific feedback, are told that the company has a policy against doing so. But many managers who try offering feedback learn that candidates often react negatively when given concrete reasons for rejection, and eventually move to a “no feedback” policy.
While there are costs and risks associated with giving feedback, it can be helpful and can generate goodwill with candidates. If you’d like to give feedback to candidates, make sure you understand how to avoid the legal risks. Only offer feedback if you can phrase it in a way that’s both specific and constructive. TripleByte offers more advice on when and how to offer feedback. There may still be a subset of candidates that react negatively. But, if you believe in the value of feedback, don’t let a few negative instances trick you into generating bureaucratic scar tissue (that is, superficial reactions to one or a small set of negative events).
As a final note for the hiring manager: in addition to candidate goodwill, having a norm of providing feedback to candidates can, as a side effect, force you to be more rigorous about your decision-making. If you find yourself constantly resistant to giving feedback to candidates, it might be a sign of a broader failure in your process. Ask yourself:
Is your decision-making structured enough? Do you have a clear method of assessing and evaluating candidates that will allow you to objectively determine when and why they aren’t a fit?
Are you failing to build enough trust with candidates earlier in the process? A relationship of trust with candidates should make it easier for you to offer (and for them to accept) constructive feedback.
Are you rejecting out of default, laziness, or busyness without having reasons you can articulate and communicate?
Are you rejecting candidates for reasons that could have been easily determined earlier in your recruiting process? Often, you might hesitate to give a candidate feedback if they are missing an obvious requirement for the job, and informing them of that reason would expose that you just wasted your and their time by not realizing that sooner.
Many hiring managers and recruiters aim to have their offer acceptance rate be 70% or higher. You have hopefully built up enough trust and rapport with a candidate by now that you have some idea of what their answer will be—but how you go about extending the offer can be the difference between a yes! and a no, thanks.
This section was written by Jose Guardado.