editione1.0.8
Updated August 24, 2022You’re reading an excerpt of The Holloway Guide to Technical Recruiting and Hiring, a book by Osman (Ozzie) Osman and over 45 other contributors. It is the most authoritative resource on growing software engineering teams effectively, written by and for hiring managers, recruiters, interviewers, and candidates. Purchase the book to support the author and the ad-free Holloway reading experience. You get instant digital access, over 800 links and references, commentary and future updates, and a high-quality PDF download.
It’s beyond the scope of this Guide to develop a bank of nontechnical questions, especially because these questions depend so heavily on your company’s goals, values, and mission—not to mention the specific details of the role for which you’re hiring. We’ve included a few categories of questions you can consider, however, and resources to help you dive deeper into which questions will work best for your needs. New Relic has an excellent post on evaluating potential managers that covers their approach to designing nontechnical questions and their rubrics for evaluating them.
story “You have to let the candidate surprise you. It makes the process of coming up with these questions easier, because you don’t have to come up with all the possible answers. You can look at who are the super productive and thoughtful people on your team, the very valuable people on your team, seniors, lead people, and then ask something specific about them: ‘What do they excel at?’, ‘In what situation did they do something really great?’ Then try and turn that into a question. One example might be that ‘a senior person stood up for someone else in a daily standup meeting.’ Now come up with a related behavioral question. Identify situations where senior people really shone a positive light on the kind of behaviors you’re looking at. On top of that, look at your company’s values and build questions from there. ‘You are the kind of person who values feedback.’ Ok, now you know you can ask, ‘What kind of feedback cycles do you have in your work? How do you give feedback?’ Determine what you optimize for, and build questions from those behaviors.” —Benjamin Retizammer, freelance CTO
When thinking about values alignment, it is critical to know your company and team’s values, and then use your nontechnical interview questions to assess how the candidate has previously demonstrated—or undermined—those values.
important Remember, values alignment is not the same as culture fit. If someone has behaved unethically in the past or demonstrated a lack of integrity, that’s a good reason not to hire them; if someone doesn’t seem fun, that’s not.
Values alignment can often be determined through behavioral or situational interviews that test for how a candidate makes decisions, passes judgments, or how they react to making mistakes. This is not about delivering “correct” answers—the point is to assess whether a candidate’s values related to work style, communication, and mission align with the company’s.
The most effective questions focus on sussing out a candidate’s values and allow you to evaluate them based on how well they line up with company values:
If a company values humility/humbleness. Does the candidate demonstrate ownership for their mistakes or have examples of taking feedback from others?
If a company values biasing for action. Does the candidate have examples of making decisions in situations where the path forward was unclear?
If a company values depth of expertise. Does the candidate have examples of going the extra mile to understand something?
If a company values working as a team. Does the candidate have examples of supporting teammates through a difficult time or putting the team’s goals ahead of their own?
| Prompt | Evaluation |
|---|---|
| “Tell me about when you gave a compliment to a team member.” | How the person feels about giving positive feedback. Do they think about the word appreciation, is it about positive feedback, or very technical things like, “I like your code.” |
| “Imagine you join and everything here has gone wildly successful. Three years from now, what would you want to look back on and be proud of?” | Mission alignment and passion. Are they excited about the core mission? How much do they care about being a great manager and supporting their team? Do they only focus on their own accomplishments? |
| “Name some improvements you’ve made or experiments you’ve tried.” | Self-awareness and what they view as improvement. Again, do they only focus on their own improvement and/or narrow improvements (“I made function X faster”), or do they have a broader view on the value of experimentation and process improvements? |
Another thing to screen for is agency—how much responsibility does the candidate feel comfortable or eager to take on when it comes to solving problems? This is especially important as the company gets bigger.
When assessing agency, it’s important to ask both situational and behavioral questions. Not every candidate will have had the same opportunity to demonstrate agency in their careers, especially if they’ve been in unsupportive work environments—but they should be able to tell you what they would do in a hypothetical situation.
There are questions that can give you signal on how a candidate feels agency that don’t require them to have had a leadership position. For example, a lot of people come to the manager to complain about something at the company. But what you want is someone who will come to you and say, “I saw this problem, and here’s how I can make it better.”
Foundry Group co-founder Jason Mendelson screens for agency with the following question: “Can you share a moment from your career where you felt like you’d been slighted?” Everyone’s got one.
Next, ask them what they did about it. Do they throw their boss under the bus, do they burn their peers or those below them? Do they go higher?
story “I interviewed a salesperson who delivered a great answer to the agency question. He had a shared quota with a salesperson, and she went on maternity leave. So he went and asked the boss, ‘Since she’s on maternity leave, I’m not responsible for the whole quota, right?’ And the boss said, ‘No, you need to hit all of it.’ And he’s like, ‘That’s bullshit.’ And so I asked him, ‘What did you do?’ And he said, ‘I fucking hit the quota.’” —Andy Sparks, co-founder and CEO, Holloway
| Prompt | Evaluation |
|---|---|
| “What got you interested in Acme Inc.?” | General alignment. This rarely totally weeds out a candidate, but sometimes raises red flags where the candidate is interested in totally different things than what the company is focused on. Superstar candidates will also have a chance here to distinguish themselves by the energy they answer with and will often have way more to offer here than average candidates. |
| “Imagine I was a new engineer joining your team at [their current/previous company], what are some key things you’d want me to know right away about the customers of that product and how it shapes the way it should be built?” | Curiosity, bias for understanding customer and product needs. Great engineers won’t just leave it to the product team to figure this out; they’ll actively be curious about what makes their customers tick and be able to communicate it to their team. Weaker candidates here will often just confuse you with a jumble of information about different market segments or platitude type statements. Star candidates help you right then and there to understand the key types of users of the product, what each one needs, and often one or two special details about their psychology that wouldn’t be obvious. |
| “What’s a recent fairly large project you worked on at your last company? Roughly when did the project start and finish? At the halfway point of that timeline, tell me a bit about how you were sizing the project up? What percent complete did you feel you were?” | Risk assessment and ability to tell if a project is really on track or not. Some candidates will only be able to tell you basic stuff here (“Well we finished half the stories in the SCRUM plan, so we must have half way, right?”). Strong candidates understand how to look back at their project plan, but they go much further. They could talk about things like: Is the customer scenario actual coming together? Are we able to show it to people and see it work for them? Are we running into new tech issues we couldn’t foresee that we’ve proactively raised with affected parties like product? They might talk about how they front loaded the riskiest part, and they’re able to qualitatively speak to how they’re seeing it really behave now. |
| Follow-up: “Towards the tail end of that project, what factors were you considering when thinking about whether it was time to ship?” | A standard answer would be something like “the stories are all implemented so we’re done” or “all the tests passed.” Stronger candidates certainly get those things, but they add in more nuance: Is the customer scenario or the key original objective really coming together? They thought about whether they should also add some other element, but realized they needed to just get it out there so they could get feedback. They had a solid plan for deployment and operations. They accounted for how future development experience would work for others (e.g. wiki is updated with how to pull the repo and setup dev boxes). |
One of the most important signals to get from nontechnical interviews is how a candidate communicates. No matter what kind of role you’re hiring for or at what level, communication is key. If an employee struggles to communicate an idea with their peers, boss, or junior engineers, they’re going to be frustrated. They might blame everyone else for not understanding them, or even worse, be disappointed and begin to feel isolated. The vast majority of technical hires need to be able to collaborate and coordinate. To do that, not only do they have to be able to share ideas, thoughts, and challenges—they have to be able to do so quickly.
To assess a candidate’s communication style, Kevin Morrill came up with a high-signal assignment:*
I want you to explain something to me. Pick any topic you want: a hobby you have, a book you’ve read, a project you worked on—anything. You’ll have just five minutes to explain it. At the beginning of the five minutes you shouldn’t assume anything about what I know, and at the end I should understand whatever is most important about this topic. During the five minutes, I might ask you some questions, and you can ask me questions. Take as much time as you want to think it through, and let me know when you want to start.
There is a wide distribution of possible topics here. How to braid hair, how to install drip irrigation, the difference between precision and recall in search quality assessment, the basics of brewing beer, quantum mechanics, the function of reduplication in John Milton’s Paradise Lost, or training a dog to sit.
While what the candidate chooses to explain to you is definitely interesting, the signal you gather is from how they do it. First off, how much time did they take—if any—to come up with a topic and sketch out their five minutes before letting the interviewer know they were ready? Did they write themselves an outline? Did they set something up on their laptop or a whiteboard? Some of this is presentation style, but whatever they do (or don’t do) before they start talking can tell you how much they value organization, planning, structure, and preparation. The bottom line: those who prepare always do better.
The candidate’s delivery style can tell you about their empathy. Do they pause to check if you’re keeping up, asking, “Does that make sense? Are you with me so far?” Are they trying hard to reach the audience where they are, by using analogies or posing questions? If you try to sidetrack them or throw them off a bit, do they say, “That’s not what I meant” or “Perhaps I didn’t explain that well,” and try again? Of course, be careful not to play mind games. Some candidates won’t feel comfortable pushing back on the interviewer.
Another thing to check for is whether the candidate sets themselves a timer. Some people don’t need to because they did debate in high school or have a great internal clock, and others might not have heard the bit about a five-minute limit or choose to ignore it. If a candidate is neither of these, they should set themselves a clock.
controversy Some people who like to use this question are looking for whether the candidate sticks to the five-minute mark, docking them if they go over. Kevin Morrill, the author of this question, says the interviewer shouldn’t stop them at five, so they can see how closely the candidate was listening or how much they overestimate their abilities to self-regulate. Others may be more forgiving of the time limit if a candidate seems excited by their subject or thinks the interviewer wants to hear more. A candidate is likely to expect the interviewer to cut them off when they’re satisfied.
Candidates may stop before the five-minute mark; if they’ve communicated the most important things to you about their topic, they’ve done their job. The purpose of the five-minute communication question is to tell whether, if someone has a great idea about improving the product or speeding up a system, they can communicate it to you in less than five minutes.
Less strong candidates tend to:
jump into a presentation without a plan for what they want to get across
finish either way too fast (so that the interviewer didn’t really learn much) or go way overtime (demonstrating a lack of planning)
not ask questions to learn about the audience
never check in to see if you’re following an explanation
make tons of assumptions about what you already know or understand.
Stronger candidates tend to:
think a bit about what topic makes sense to tackle
come up with an outline for what they want to cover
ask you what you already know about the topic
draw analogies to the things you already know
check in with you: “Does this make sense to you?”
use the whiteboard to express themselves
get to something that’s actually interesting about the topic
give you back a summary of what was covered
finish on time.
Whether someone will be a good manager or not is the subject of countless books, courses, and blog posts (and perhaps a future Holloway Guide!). In lieu of covering this in detail, we provide a few suggested questions to include for assessing management ability and suggest additional recommended reading.
| Prompt | Evaluation |
|---|---|
| “Tell me about your management and leadership philosophy.” | The basic answer here is something like “My job is to make the goals clear, and then get out of the way.” Star candidates know not to be needlessly overbearing, but they have tons more to offer than that. “I work with my team to understand what talents they have and also what motivates them. When someone works on a project, I make sure I know their blind spots they don’t understand, and I always have some mitigation to compensate (e.g. more frequent check-ins, training, get them a mentor, etc.) I make sure that my team gets feedback. I make sure the things that matter are in my team’s control (e.g. great tools, monitoring/instrumentation so they can see what’s going on)” |
| “Have you ever had to fire someone? What led up to it? What did you do when you realized things were off track?” | There are few “wrong” answers here, you’re more looking for the depth of insight the person has accumulated here. Examples of what you might hope to get: Understanding where they could have given feedback sooner; Understanding where the employee wasn’t getting a clear enough message on impact, and how they personally could have done better here; Understanding of how they’d improve hiring process; Understanding of how they’d improve their own feedback process; Understanding of how to go through a performance improvement process. |
| “In our initial conversation, we talked about the immediate needs of the team. Do you have any questions about that? Tell me how you would go after these challenges in the first 90 days?” | The answers to this question will depend heavily on the role—the degree of strategy and proactivity you’re looking for will vary significantly if you’re looking for a development manager or a VP of Engineering. But this person should demonstrate awareness of what is required of their level (or ask good clarifying questions to get there), and provide ideas and tactics accordingly. |
| “When a problem arises on the team, how do you decide at what point to step in and address it?” | Some candidates might not admit to any problems, or a tell a story about when their manager solved a problem for them and how they’d emulate that. Stronger candidates acknowledge that no team is without problems, and has an example of how they constructively resolved a conflict with respect and empathy. |
First Rounds’ 40 favorite interview questions
Workable’s question bank
Devskiller’s list of 45 behavioral questions