{"id":4088,"date":"2022-02-07T08:22:39","date_gmt":"2022-02-07T16:22:39","guid":{"rendered":"https:\/\/coderpad.io\/?p=4088"},"modified":"2022-06-14T07:01:41","modified_gmt":"2022-06-14T14:01:41","slug":"6-ways-customers-improved-their-technical-interviewing-with-playback-mode","status":"publish","type":"post","link":"https:\/\/coderpad.io\/blog\/interviewing\/6-ways-customers-improved-their-technical-interviewing-with-playback-mode\/","title":{"rendered":"6 Ways Customers Improved Their Technical Interviewing with Playback Mode"},"content":{"rendered":"\n<p>Most people can only hold a handful of things in their memory at the same time\u2013<a href=\"https:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC2657600\/\" target=\"_blank\" rel=\"noopener\">famously 7, plus or minus 2<\/a>. (Personally, I\u2019m lucky to hit 3). Technical interviewing requires keeping track of dozens of elements, from the logistics of conducting the interview to evaluating the code. You\u2019re likely to lose some important details.&nbsp;<\/p>\n\n\n\n<p>That\u2019s why CoderPad created <a href=\"https:\/\/coderpad.io\/resources\/docs\/interview\/pads\/playback-mode\/\">Playback Mode<\/a> to help you review your candidates\u2019 code submission and rewatch how they arrived at their solution after the interview. Playback improves your ability to review the final code by letting you watch the interview back. You can do so in real time or by jumping to specific moments. (Protip: Playback skips from edit to edit, compressing any \u201cdead\u201d time, in turn saving you time.) This feature is also useful for reviewing an interview someone else conducted.<\/p>\n\n\n\n<p>We\u2019re evolving Playback for some exciting product updates (TBA, but really, really exciting, I promise!). So I talked with our customers to learn more about how they use Playback now and how returning to the interview helped them improve the hiring process.&nbsp;<\/p>\n\n\n\n<p>My hope is that, by sharing what I found in user research, you\u2019ll be able to refine your own technical interviews. Here are the benefits that respondents described.<\/p>\n\n\n<aside class=\"\n    cta-banner\n     cta-banner--bg-green      cta-banner--has-media \"\ndata-block-name=\"cta-banner\">\n    <div class=\"inner\">\n        <div class=\"content\">\n                            <h2 class=\"headline\">Ready to start interviewing?<\/h2>\n            \n                            <div class=\"cta-buttons\">\n                                    <a href=\"\/sales\/\" class=\"button  js-cta--get-a-demo\"  data-ga-category=\"CTA\" data-ga-label=\"Ready to start interviewing?|Get a demo\">Get a demo<\/a>\n                                <\/div>\n                    <\/div>\n                    <div class=\"media\">\n                <img loading=\"lazy\" decoding=\"async\" width=\"238\" height=\"146\" src=\"https:\/\/coderpad.io\/wp-content\/uploads\/2022\/08\/illustration-of-two-people-chatting-in-app-windows.png\" class=\"attachment-large size-large\" alt=\"\" \/>\n            <\/div>\n            <\/div>\n<\/aside>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"1-jogging-their-memory\">1. Jogging their memory<\/h2>\n\n\n\n<p>Across several chats with customers, I kept hearing the phrase \u201c<em>to jog my memory<\/em>.\u201d Frequently, there was some lag time between the interview and when feedback was written, whether because of meetings, other work, or simply wanting to sit with what happened for a bit. Sometimes a hiring manager or hiring committee would ask follow-up questions days later, or a candidate would interview with the company a second time.&nbsp;<\/p>\n\n\n\n<p>In each case, viewing the code and\/or how it evolved allowed them to recall details and impressions from the interview.<\/p>\n\n\n\n<p>A senior engineer for a large media company says he doesn\u2019t start with the final code, but rather from the beginning. This serves as a way to jog his memory even days or months later.&nbsp;<\/p>\n\n\n<div class=\"\n    quote-block\n                \n\" data-block-name=\"coderpad-quote\">\n\n\n    <figure class=\"the-quote-figure\">\n        <blockquote class=\"the-quote\">\n            <p>I remember code really well. It helps me remember the conversation.<\/p>\n\n        <\/blockquote>\n\n                    <figcaption class=\"the-attribution\">\n               Senior Engineer\n            <\/figcaption>\n            <\/figure>\n<\/div>\n\n\n\n<p>A principal developer on an R&amp;D team says he tries to refresh his memory by walking through how the candidate solved the problem.<\/p>\n\n\n<div class=\"\n    quote-block\n                \n\" data-block-name=\"coderpad-quote\">\n\n\n    <figure class=\"the-quote-figure\">\n        <blockquote class=\"the-quote\">\n            <p>You can see what they did, recall the thought process [they used for] problem solving.<\/p>\n\n        <\/blockquote>\n\n                    <figcaption class=\"the-attribution\">\n               Principal Developer\n            <\/figcaption>\n            <\/figure>\n<\/div>\n\n\n\n<p>Interviewers also use the timeline to spot-check specific moments during the interview, such as when the candidate did something unexpected, struggled, or when the interviewer\u2019s attention was elsewhere.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"2-digging-into-key-moments\">2. Digging into key moments<\/h2>\n\n\n\n<p>Interviewers find Playback helpful to return to specific moments when there is a gap in their memory or notes, when the candidate\u2019s actions were unclear, or when they want to understand the candidate\u2019s thought process better.<\/p>\n\n\n<div class=\"\n    quote-block\n                \n\" data-block-name=\"coderpad-quote\">\n\n\n    <figure class=\"the-quote-figure\">\n        <blockquote class=\"the-quote\">\n            <p>Sometimes if I&#8217;m writing my notes and I realize, oh, the candidate iterated through some versions here, but I don&#8217;t quite remember where they went off track. Did they use the wrong data structure, or did I remember correctly that they made this mistake\u2026 then I&#8217;ll go back.<\/p>\n\n        <\/blockquote>\n\n                    <figcaption class=\"the-attribution\">\n               Data Scientist\n            <\/figcaption>\n            <\/figure>\n<\/div>\n\n\n<div class=\"\n    quote-block\n                \n\" data-block-name=\"coderpad-quote\">\n\n\n    <figure class=\"the-quote-figure\">\n        <blockquote class=\"the-quote\">\n            <p>I want to give more importance to the thought process\u2026 [In the case of one candidate] the solution was not done. He was not able to complete it, but his thought process was good.<\/p>\n\n        <\/blockquote>\n\n                    <figcaption class=\"the-attribution\">\n               Principal Engineer\n            <\/figcaption>\n            <\/figure>\n<\/div>\n\n\n\n<p>Returning to pivotal moments in the interview can unlock a clearer picture of the candidate\u2019s capabilities.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"3-noticing-things-they-missed\">3. Noticing things they missed&nbsp;<\/h2>\n\n\n\n<p>Interviewers are splitting their focus between the candidate, the code, and the logistics of running the interview. As such, it\u2019s common for them to want to hone in on <em>just<\/em> the code after the interview is over.<\/p>\n\n\n<div class=\"\n    quote-block\n                \n\" data-block-name=\"coderpad-quote\">\n\n\n    <figure class=\"the-quote-figure\">\n        <blockquote class=\"the-quote\">\n            <p>While you&#8217;re in the interview session, you\u2019re thinking about a lot of things: you&#8217;re looking at a resume, you&#8217;re thinking about all the other things that you want to ask, like the behavioral question and the technical questions, and then they&#8217;re solving a problem. So a lot of things are going on in your mind, but\u2026 after the interview, when you&#8217;re looking at this, your mind is focused towards [the] problem, only. That&#8217;s something that the playback helps with.<\/p>\n\n        <\/blockquote>\n\n                    <figcaption class=\"the-attribution\">\n               Principal Software Developer\n            <\/figcaption>\n            <\/figure>\n<\/div>\n\n\n\n<p>With the ability to focus on the details without competing priorities, interviewers can take even more considerations into account, such as:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>How long did it take the candidate to get through different stages of the interview?<\/li><li>How much help did they really get from the interviewer?<\/li><li>What was left unaddressed, and how far from a working or complete solution was a candidate?<\/li><li>How was the readability, maintainability, and overall quality of code and documentation?<\/li><li>Where did the candidate go wrong?<\/li><li>Did I miss any red flags, e.g. signs of cheating, inefficient solutions, or poor coding practices?<\/li><\/ul>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"4-settling-disputes-amongst-the-hiring-panel\">4. Settling disputes amongst the hiring panel<\/h2>\n\n\n\n<p>Occasionally the hiring panel may disagree over a candidate. In these cases it\u2019s extremely valuable to return to the hard evidence. With Playback, they were able to replay all or a portion of a candidate\u2019s interview to strengthen their case.<\/p>\n\n\n<div class=\"\n    quote-block\n                \n\" data-block-name=\"coderpad-quote\">\n\n\n    <figure class=\"the-quote-figure\">\n        <blockquote class=\"the-quote\">\n            <p>I remember once that we had a dispute [about one candidate during the debriefing meeting]. Some people really liked him. But in my case, the code really was not working. They replayed [my interview with the candidate, and saw] &#8216;Hey, this is really not a good one.&#8217; So CoderPad, in this situation, played a really vital role for deciding whether the candidate is good or not.<\/p>\n\n        <\/blockquote>\n\n                    <figcaption class=\"the-attribution\">\n               Engineering Interviewer\n            <\/figcaption>\n            <\/figure>\n<\/div>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"5-including-supporting-evidence-in-candidate-feedback-reports\">5. Including supporting evidence in candidate feedback reports<\/h2>\n\n\n\n<p>To back up their assessment of a candidate, many interviewers use Playback to copy portions of the candidate\u2019s code into their feedback report.<\/p>\n\n\n\n<p>One Senior Machine Learning Engineer talks about what he includes in his evaluation writeup: \u201cThis is the question I used, this is the reason for my decision, and give them concrete examples [from the candidate\u2019s code].\u201d<\/p>\n\n\n<div class=\"\n    quote-block\n                \n\" data-block-name=\"coderpad-quote\">\n\n\n    <figure class=\"the-quote-figure\">\n        <blockquote class=\"the-quote\">\n            <p>Coding style is another very important factor to make sure that they&#8217;re writing clean code, not messy code. And normally we ask questions that can be resolved within 50 lines. 100 lines of code? That is really a negative signal. It means that they&#8217;re going the wrong route.<\/p>\n\n        <\/blockquote>\n\n                    <figcaption class=\"the-attribution\">\n               Senior Manager, Interviewer\n            <\/figcaption>\n            <\/figure>\n<\/div>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"6-minimizing-biases\">6. Minimizing biases<\/h2>\n\n\n\n<p>We\u2019re all trying to find the candidates who are going to thrive in our organization. At the same time, we want to avoid wasting time by running an interview that\u2019s biased by our own shortcomings and causes us to miss out on quality candidates.&nbsp;<\/p>\n\n\n<div class=\"\n    quote-block\n                \n\" data-block-name=\"coderpad-quote\">\n\n\n    <figure class=\"the-quote-figure\">\n        <blockquote class=\"the-quote\">\n            <p>You [might] want to hear some keyword from that candidate, and if they say that keyword, you will be happy[&#8230;], But sometimes what happens, the candidate is not able to say that keyword, but he or she is correct about that particular area. [Then, it] actually makes a lot of sense to revisit our interview session, to understand that the candidate was correct, and I was thinking on a \u2018monologue.\u2019<\/p>\n\n        <\/blockquote>\n\n                    <figcaption class=\"the-attribution\">\n               Data Engineer\u00a0\n            <\/figcaption>\n            <\/figure>\n<\/div>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"why-some-interviewers-don-t-use-playback-and-why-they-should\">Why some interviewers don\u2019t use Playback\u2014and why they should<\/h2>\n\n\n\n<p>Some interviewers only utilized the notes they took, what was in their memory, or the impressions and conclusions formed during the interview. Their reasoning was that:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>their initial assessment was solid<\/li><li>their memory was crisp<\/li><li>their notes (if they took any) were sufficient<\/li><\/ul>\n\n\n\n<p>Therefore, they felt that they didn\u2019t need to revisit the \u201craw evidence.\u201d One interviewer said he doesn\u2019t return to the code because:<\/p>\n\n\n\n<p>\u201cI&#8217;ve already assessed the candidate, and I&#8217;ve made my pronouncement, and then I clear my mind\u2026 I just build an image of what the candidate\u2019s capabilities are right away.\u201d&nbsp;<\/p>\n\n\n\n<p>Skipping Playback might be a reasonable approach when assessing an interview with minimal or no coding, such as one focused on systems architecture or a coding question that\u2019s used as a mechanism to learn how someone thinks about a problem. Usually, however, relying exclusively on our initial judgments is a strategy prone to bias.<\/p>\n\n\n\n<p>And in the tech industry, there are still numerous factors that stand in the way of fair hiring. Mthree\u2019s <a href=\"https:\/\/www.mthree.com\/site\/assets\/files\/2689\/diversity_in_tech_2021_us_report_by_mthree.pdf\" target=\"_blank\" rel=\"noopener\">2021 \u2018Diversity in Tech\u2019 report<\/a> found that 45% of businesses haven\u2019t yet invested in anti-bias training for hiring managers. Our hope is that Playback can serve as a way for customers to proactively create deliberate, thorough hiring best practices and ultimately reduce unconscious bias.&nbsp;<\/p>\n\n\n\n<p>Playback\u2019s utility to \u201cjog memory\u201d fills a gap for many interviewers, and seeing customers\u2019 real-world usage gave us a vital perspective on how our living coding environment can continue to evolve. We invite you to continue <a href=\"https:\/\/coderpad.io\/blog\/interviewing\/technical-interviews-are-pretty-terrible\/\">improving technical interviews<\/a> with us.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Most people can only hold a handful of things in their memory at the same time\u2013famously 7, plus or minus 2. (Personally, I\u2019m lucky to hit 3). Technical interviewing requires keeping track of dozens of elements, from the logistics of conducting the interview to evaluating the code. You\u2019re likely to lose some important details.&nbsp; That\u2019s [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":4102,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[12],"tags":[],"persona":[],"blog-programming-language":[],"keyword-cluster":[],"class_list":["post-4088","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-interviewing"],"acf":[],"_links":{"self":[{"href":"https:\/\/coderpad.io\/wp-json\/wp\/v2\/posts\/4088","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/coderpad.io\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/coderpad.io\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/coderpad.io\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/coderpad.io\/wp-json\/wp\/v2\/comments?post=4088"}],"version-history":[{"count":11,"href":"https:\/\/coderpad.io\/wp-json\/wp\/v2\/posts\/4088\/revisions"}],"predecessor-version":[{"id":7886,"href":"https:\/\/coderpad.io\/wp-json\/wp\/v2\/posts\/4088\/revisions\/7886"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/coderpad.io\/wp-json\/wp\/v2\/media\/4102"}],"wp:attachment":[{"href":"https:\/\/coderpad.io\/wp-json\/wp\/v2\/media?parent=4088"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/coderpad.io\/wp-json\/wp\/v2\/categories?post=4088"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/coderpad.io\/wp-json\/wp\/v2\/tags?post=4088"},{"taxonomy":"persona","embeddable":true,"href":"https:\/\/coderpad.io\/wp-json\/wp\/v2\/persona?post=4088"},{"taxonomy":"blog-programming-language","embeddable":true,"href":"https:\/\/coderpad.io\/wp-json\/wp\/v2\/blog-programming-language?post=4088"},{"taxonomy":"keyword-cluster","embeddable":true,"href":"https:\/\/coderpad.io\/wp-json\/wp\/v2\/keyword-cluster?post=4088"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}