Interpreting Reference Feedback

You’re reading an excerpt of The Holloway Guide to Technical Recruiting and Hiring, a book by Osman (Ozzie) Osman and over 45 other contributors. It is the most authoritative resource on growing software engineering teams effectively, written by and for hiring managers, recruiters, interviewers, and candidates. Purchase the book to support the author and the ad-free Holloway reading experience. You get instant digital access, over 800 links and references, commentary and future updates, and a high-quality PDF download.

When interpreting reference feedback, consider where the person is coming from. If the reference trusts you, they are more likely to give you candid feedback. If they are closer to the candidate, they may not want to risk any negative feedback being attributed to them, or they may feel the relationship colors their ability to provide objective feedback.

In general, references will be more positive than negative. Some recommend mentally compensating by discounting positive comments by 30% and amplifying negative comments by 30%. While this is an arbitrary number, it does highlight the importance of perspective. You can take outrageously positive or negative comments with a grain or two of salt.

Next, think about the context in which the candidate and reference knew each other. How closely did the reference and the candidate work together? How equipped is the reference to judge the candidate’s work? For instance, “I’ve managed over a dozen engineers and Anne is the strongest I’ve had on my team by far,” is a lot more compelling than, “Anne has been great to work with, but that was my first job out of school and I’ve only been here a few months.”

You also may consider whether the reference’s criteria for assessment is consistent with your own. For instance, traits that lead a candidate to success (or hold them back) at a large company may differ significantly from traits that lead to success at a much smaller company.

​important​The limited nature of reference checking makes it more subject to possible noise and bias than interviews. You may have spoken to the reference on a bad day. People grow and change—would you give yourself a glowing reference for every job you’ve held, and would those references all be reflective of the type of work you can do now?

Find out how veteran recruiter Jose Guardado learned the hard way about the power of a (slightly) negative reference comment in “How Reference Checks Can Go Wrong: Managing subjectivity in reference feedback,” up on the Holloway blog.

As with any part of the hiring process, references aren’t a perfect source of information. Ultimately, a negative reference for a candidate should definitely give you pause, but if you believe in the candidate based on other data points, it doesn’t need to completely disqualify them. You can always follow up with the candidate to see if you can figure out if anything has changed since the time they knew the reference.