editione1.0.8
Updated August 24, 2022You’re reading an excerpt of The Holloway Guide to Technical Recruiting and Hiring, a book by Osman (Ozzie) Osman and over 45 other contributors. It is the most authoritative resource on growing software engineering teams effectively, written by and for hiring managers, recruiters, interviewers, and candidates. Purchase the book to support the author and the ad-free Holloway reading experience. You get instant digital access, over 800 links and references, commentary and future updates, and a high-quality PDF download.
Aline Lerner provided major contributions to this section.
Most recruiters spend 7.4 seconds on a resume before deciding whether to advance a candidate. This is troubling because a resume is a collection of weak and imperfect signals—some positive, some negative, and some just informative. The purely informative signals can’t wisely be used for screening, but they do provide more information about a candidate for later discussion.
A well-formed resume is like a map of where someone can dig later in the interview process.Scott Woody, former Director of Engineering, Dropbox
Gleaning useful information from resumes requires reading them differently than many companies typically do. There are a variety of signals to look for, and interpreting them in an effective way can take practice.
danger In general, not only is the set of signals in a resume noisy, but every signal is imperfect! This means it’s important not to qualify or reject any resume from just one or two signals. Taking a holistic view and noting positive and negative signals will allow you to compare them across the pool of candidates you’re currently looking at.
Resumes don’t provide reviewers with enough information to make thorough and truly fair decisions. The key challenge is, how well can you filter candidates within tight time constraints? How can you make the best of what you can learn from resumes?
Resume reviews have three goals:
Screening. To filter out obviously unqualified candidates—those that are least likely to make it further in the hiring process. This can be a large fraction of candidates for roles with popular public job postings.
Prioritizing. Candidates that look especially well suited may be put on the “top of the pile” for the next stage of the funnel.
Informing the hiring process. For candidates you do speak with, resumes help highlight areas of interest and potential concerns to cover in future conversations.
Every signal you look at on a resume can help with one or more of these goals.
Based on the job description and known needs, are there specific types of expertise, experience, and interests that are essential? These are deal-breakers—things you can’t budge on. Some requirements are pragmatic; others will have been determined when you aligned on the role with your team.
Location. Must the person work onsite? If so, and they are not local, are they willing to relocate?
H1B or other visa sponsorship. Especially if you’re a smaller company, sponsoring a new visa can be logistically prohibitive because you have to wait over a year before the candidate can start. In general, H1B transfers are less demanding. TN and H1B1 visas are also less onerous.*
Domain expertise. Does the candidate have the specific background needed for the role? For software engineers, this might include web or fullstack app development, mobile app development, backend engineering, or data science or machine learning. This also includes the general area of work, such as SaaS and enterprise software, healthcare or biology, finance or accounting, and so on.
Previous roles. Do the nature and scope of past roles demonstrate that the candidate is prepared for or likely to succeed in this role? Has the candidate done the same type of job before? Must the person have management or product experience, or experience solving technical problems with customers? If all previous roles differ greatly from what the candidate would be responsible for in this role, it’s probably not a fit.
Seniority. Do the breadth and scope of experience reflect the level of seniority required for this role? Are you willing to hire someone into their first management position, or do you require that they have management experience?
Technology experience. If the new role requires a significant level of depth in a specific technology—be it React, AWS, Java EE, Keras and Tensorflow, Java memory management, high-performance C++, or web performance optimization—then it makes sense to filter on that. This is more likely to apply to rarer skills that require years to develop, such as GIS systems, embedded systems, or particular hardware architectures. But if your company is willing to mentor engineers or you expect the candidate to learn on the job, specific technical skills are less relevant.
Don’t ask the candidate for things you don’t really need. Any friction you add at the earliest stages of the filtering process will ultimately hurt you.Aline Lerner, co-founder and CEO, interviewing.io
Presentation may seem less important than content, but in fact indicates an understanding of audience (and even empathy for others), clarity in writing, and attention to detail.
A study at TrialPay* tried to find patterns between all candidates who had been interviewed within a year’s period, and everyone who had been offered a position at the company. What mattered in a resume when it came to on-the-job performance? Which school they attended? Seniority or advanced degrees? Side projects? Number of languages they’d mastered?
Far and away, what mattered most was the number of typos and grammatical errors on someone’s resume—most people who got an offer had two errors or fewer. This study also revealed that where people went to school had no effect on their performance. The only other factors that predicted on-the-job performance that could be found on a resume were the clarity with which candidates described what they worked on at previous jobs, and whether they had worked at a top company (this mattered least).*
Language, spelling, and punctuation. Writing a full-page resume that is cleanly formatted with no detectable errors in grammar, spelling, or punctuation always requires attention to detail. Does the resume have conspicuous errors, like multiple typos, spelling errors, or inconsistency in formatting and punctuation? If someone can’t write a page of text without typos, it’s not unreasonable to expect typos to appear in other material they write—like in their code.
Technical clarity and accuracy. Clarity in role descriptions is actually quite rare on resumes. Are the descriptions of roles or projects unusually hard to follow? Most engineers aren’t great writers, or are not focused on business context, so this is only a negative signal if the descriptions themselves are quite unclear or technically suspect.
Context and impact. Does the description of roles include enough context to understand the role? Does it include impact in terms of product relevance or business metrics? Engineers fairly rarely describe this well, so this is mainly a positive signal for senior candidates.
Degrees from academically exclusive schools are a notable signal, because getting into and graduating from an elite institution like MIT reflects success with well-known curriculum and level of rigor. When assessing how rigorous a computer science program is, it can help to know rankings of these programs in the United States as well as those in other countries; for example, the IIT schools and the related JEE rankings reflect competitive admissions across India.
However, it’s a common pitfall to look mainly for “brand name” schools or obsess on school rankings. (For more detail on how academic background can be a source of bias, see Diversity and Inclusion.) Interviewing.io looked at whether the school that candidates attended had any bearing on interview performance. In a study of 1,000 college students, it turned out there wasn’t any statistically significant difference in performance between students who went to elite universities and students who went to other schools (at least among those who had decided to sign up for an interview practice platform of their own volition).*
For many general programming and engineering roles, a higher degree is not required. But some roles, like machine learning or algorithmic roles, do require graduate-level knowledge.
dangerSchool rankings is probably the one signal that’s most overweighted. Exceptional technical talent can come out of any school, or from no school at all. In another study* from interviewing.io, researchers cross-referenced interview performance data from 3,000 candidates with attributes listed on their resumes. This time, what mattered most was whether a candidate had taken programming classes via MOOCs like Udacity and Coursera.
Has the person achieved something that requires unusual talent or dedication to achieve? This may include side projects that achieved some recognition or significant use, talks, industry awards, or the like.
One of the things we’ve seen from all our data crunching is that GPAs are worthless as a criteria for hiring, and test scores are worthless—[there is] no correlation at all except for brand-new college grads, where there’s a slight correlation.Laszlo Bock, former SVP of People Operations, Google*
controversyComputer science MS degrees can be a confusing signal, especially when achieved after a non-computer science undergraduate degree. Some recruiters find that people returning to school for an MS degree in CS may signal inadequate programming experience, which could lead to poor technical interview performance. One theory as to why this occurs is that CS fundamentals instruction tends to happen in undergraduate computer science courses, and some MS programs are geared toward students who have never done any programming either—such programs serve professionals eager to return to school and learn some basic technical skills. At certain schools it is possible to get an MS in CS without ever taking an algorithms or data structures class. Another point is that MS degrees tend to increase income by about $10K per year. But someone already employed and successful as a software engineer would probably earn more by staying in their job, rather than leaving work to return to school for an MS degree.
Next are the signals you can collect from all of a candidate’s past roles.
Getting an offer and holding a job at a company that has a well-known, high hiring bar is a notable positive signal. For some companies, the stage at which a person joins may indicate level of difficulty in being admitted; for example, it was probably harder to get a job at Google in 2005 than in 2015. The same caveats apply to name-brand companies as to name-brand schools. Key here are unique high-responsibility roles. Someone who is tech lead, architect, or manager of a team with significant importance to a company is a particularly strong signal.
Notable achievements can include open-source projects, businesses or products built, or patents. This includes traction. Do any businesses or projects (open-source or otherwise) have objective signs of interest and value to others? GitHub stars? Numbers of users? Revenue?
The set of job transitions a person has had can indicate one of two patterns:
Job hopping. Does the person tend to stay at some jobs for a longer period, or do they tend to move on after a year or so? How much of a signal this is depends on the length of someone’s career so far. For more senior candidates, the pattern can be more obvious.
Very few job changes. Has the candidate been in their most recent job 10+ years? Were there signs of career advancement? This may not be a negative signal, unless it’s most of the person’s career and that past work differs greatly from the new role. If the candidate has only had one role, and that was in a very different environment or domain than the new role, there may not be a fit.
controversy How much you can infer about someone based on “job hopping” is a controversial topic. Switching jobs once or twice quickly is not a negative signal. If a candidate has had six positions, and never stayed at any one of them more than a year or two, that indicates a pattern worth considering carefully. This may mean that they are difficult to work with or have had poor performance in the past. That said, URG employees do leave companies at higher rates, often due to unwelcoming work environments or discrimination, and that’s important to note as another plausible cause for frequent transitions. Overall, it’s important to learn more and discuss with the candidate before judging.
When evaluating a candidate’s career trajectory, a number of factors are worth considering. How far has someone come in their career? How have their roles evolved, and how quickly? Where have their roles shifted?
Career trajectory is one of the more powerful signals a resume carries. However, it can be difficult to tease apart real career growth from flowery embellishments, and to put growth in the context of the candidate’s circumstances. One of the surest positive signals comes from consistent increase in responsibility over a number of years, both within a single company and at company transitions. In a healthy and growing company, such as a mid-stage startup, a strong employee will take on more responsibility every year or two.
The most immediate and obvious way to spot growing responsibility is to look at job titles, but it’s not always that straightforward. Titles are typically not standardized—startups often provide much broader exposure to problems, higher levels of ownership, and more rapid growth opportunity given the distribution of problems to the people able to own them. Reading leveling documents, like Square’s open-source framework, might also give you a good idea of what to look for, but it’s important to remember that most startups won’t be at the point where they have a formalized leveling system. (See Setting Levels and Titles for more.)
Note that at more mature companies, lack of visible promotion may not be a negative signal. At large companies, it can get very difficult to get a promotion, especially when working in hyper-mature product environments. For example, when working on infrastructure at a huge search engine, the only way to get a promotion might be through squeezing nanoseconds of latency out of returned results, after hundreds of other brilliant minds have already tried. Certain types of engineers might be well-suited to this, but others may have a much more rapid trajectory in a broader, more exploratory environment.
Of course, the candidate may have moved between companies often as well, so it’s worth considering if their role expanded or evolved at each transition.
Candidates can also stand out in the way they describe their experience. A candidate’s ability to write about their work with clarity is highly valuable, and is likely to translate to good communication skills in their work.
It’s easy to forget what clear communication looks like because it’s rare; when you read a bunch of bad descriptions in a row, they start to seem normal. And it’s hard to deprioritize or even reject candidates who have gone to great schools or have worked at top companies just because they didn’t describe what they did at their last job particularly well.
importantWhen looking at an actual resume, the ultimate test for whether a description of a role is well-written is to read it, and then try to explain it out loud, in your own words. If you can’t do it, the description isn’t effective.
Especially with nontraditional candidates, good writing is everything, because the rest of their resume will likely not have as much helpful information—either they’re junior or they’ve worked at places you haven’t heard of. In these situations, how well they understand their work and how much they care about it is the best signal you have.
A well-written role description:
Includes quantifiable, results-driven descriptions of the impact of the candidate’s work.
Demonstrates an understanding of the nuance really required to own a function.
Has simple and direct language that’s not fluffy or riddled with obscure or confusing phrasing.
Goes beyond lists of tools to give details of the candidate’s work in the larger context of their team or the company.
candidate When describing past work, focus on giving the reader as much context as they need to understand your role and its impact on the company. What were you responsible for and why did it matter? If there is a role you’re particularly interested in, you may also choose to highlight experiences that represent your interest in that space, or any relevant tools or techniques that demonstrate skill and context. Visualize explaining your work to someone who is intelligent but knows nothing of what you do. Where do you start and where do you end? You probably wouldn’t jump into the middle, with lots of internal jargon or specific bug fixes; you’d go “top down” and explain why your work matters and what makes it interesting or difficult, include any key results, and omit extraneous details.
Note that the same rules may not apply to LinkedIn profiles, as those are in the public sphere, whether a candidate is looking or not, and often candidates who are in high demand will purposely keep their LinkedIns sparse to discourage recruiter spam.
Considering how a candidate got to where they are gives insight into their abilities. Some people call this “grit,” or “distance traveled,” or “affirmative meritocracy.” Where did this person begin, and where are they now? Have they come further than one might expect? Are there signs they have overcome significant obstacles in life through hard work or creativity? Candidates with a less advantaged background may have had to work very hard to get to the same position another candidate attained more easily. A job at an elite tech company is much easier to land after a Stanford CS degree than for someone from a small community college far from any technology hubs.
story Many years ago, when I was working as a software engineer at a small company, one of the existing engineers referred a close friend of his from high school. Though they had taken many of the same classes and spent a good amount of time building projects together, their paths diverged pretty wildly after graduation. The employee graduated from a top 10 computer science school. His friend, on the other hand, attended a much lower-tier college for a semester before dropping out, and at the time of the referral, he was doing data entry. When we first looked at the referral’s resume, we had no idea what to do with it, and most people on the team wanted to pass on him. But, one thing on his resume stood out—he had spent several seasons teaching programming at a prestigious summer program for gifted high school students. This type of leadership and initiative were largely at odds with what we expected a candidate with his profile, and ultimately that’s what tipped us into hiring him. To this day (close to a decade later), he works at the aforementioned company, and has ownership over much of the application’s backend architecture. —Aline Lerner, co-founder and CEO, interviewing.io
While rarely of use for filtering, how a candidate became a programmer can give a lot of insight into their personal journey and technical temperament. There are many ways to become a good engineer, but these backgrounds can cultivate different strengths and styles, and the story can help you get to know someone better. The most common paths to becoming a programmer are:
Self-taught at a young age (perhaps they started with an interest in building video games or taking apart and rebuilding computers)
Switched course mid-career to become a developer, perhaps going back to school or a coding bootcamp
A background in business or finance that led to a focus on programming
A role in IT or system administration that led to software engineering
Desire to build a product or business (often a young entrepreneur)
Formal college or graduate education (in computer science or computer engineering, math, or a field of science where computation was involved).
In addition to engineering skills, many other areas of drive and skill are relevant for technical roles. If an unusual combination of skills is relevant for the role, these kinds of signals are powerful and excellent for prioritizing resumes. By specifically looking for these signals, you can turn a tedious resume screening process into a huge value add by finding rare candidates with exceptional combinations of skills.
Entrepreneurial focus. Has this person started a business before? This can often be seen by project descriptions—does the candidate focus on technical concerns only, or do they include business impact? A purely technical engineering role doesn’t need a proclivity for entrepreneurship or an obsession with revenue, but senior roles or roles at smaller companies can greatly benefit from these qualities.
Growth or marketing focus. Does the candidate talk a lot about moving metrics? Are they focused on quantifiable impact? Have they had marketing-affiliated roles?
Customer focus. Have past roles included talking to customers or external people? Solutions engineers and sales engineers often need to do this.
Product focus. Do they describe what the value of a product is to its end-users, or why they built certain pieces of software? This is great for product engineering roles.
Design focus. A visually polished resume, with carefully selected typography or web design, shows the candidate probably has a focus on design. You can also see this from projects or role descriptions.
A few more technical areas to consider:
Data science or machine learning focus. Is there evidence this person been excited by problems that involve data challenges, noisy data, or extracting insights from datasets using a variety of tools?
Mathematical or algorithmic focus. Is the candidate focused on technical problems involving algorithmic or mathematical challenges? Have they published research papers? Typically these are people who’ve developed these skills in an academic setting.
Writing and documentation focus. Does the candidate have experience and skill with written communication, internally or externally? Is this a skill they value? You can also tell by how they write about past work.
Past work environment. This is more subjective, and usually isn’t useful for filtering, but can be good to know when discussing past work.
Company size. Has the person worked only at small, growth stage, or large companies? It’s not unusual to see an engineer who has only worked at companies of 1,000 or more people. That can mean a big shift in working style compared to being an engineer at early or growth-stage startups. Exceptionally talented engineers can excel in one environment and flounder in the other.
Enterprise vs. consumer product experience. Engineers used to enterprise products, or building purely internal tooling for use within a large company, tend to be used to different objectives than consumer-facing product engineers. Another variation is people who’ve worked primarily on internal tooling, which is often closer to enterprise experience.
Experimentation vs. focus. Does the candidate work on side projects or seem to do a lot of different things, or stick to their one job? If the role involves a lot of novel or creative prototyping of new work, a track record of building new things could be a strong asset. But for some roles, this is not required.
importantIt might be useful to have a list of things you don’t screen for.
Given the current hiring market for technical roles, it’s essential to consider early on whether a candidate is reasonably likely to be interested in your company, and leave their current role. Switching to a selling rather than a vetting mindset might include prioritizing their interest in your space (have they blogged about something relevant, do they have relevant work on GitHub?) over their pedigree. This is also an excellent opportunity for arbitrage—if you can spot something in prospective candidates that others cannot, your odds of closing them will be much higher, because those candidates won’t be bombarded by requests from the open market. In other words, if you can spot grit and passion in lieu of traditional markers like pedigree, you’ll wind up making much more efficient use of your limited time.
Most companies hire using a mix of sources or channels, and these evolve as a company grows and matures.
In the early days of a small startup, the founders often use their personal networks to hire people they have a pre-existing relationship with. They might start to tap into some candidate flow from recruiting platforms and marketplaces. As their hiring needs grow, hiring managers might start doing outbound sourcing for really specific roles, and working with external recruiters, either on a contingency or contract basis. Eventually, as a company scales and matures, it usually makes sense to start hiring internal recruiters to take on some of the responsibility of generating candidate flow. At scale, most companies rely on a mix of employee referrals, inbound applicants, and outbound sourcing.
startup Although data is hard to come by, Lightspeed Ventures’ Startup Hiring Trends report cited that startups at different stages seemed to maintain “the rule of happy thirds”: relying for about a third each of their hires on sourced candidates, inbound candidates, and referrals.