{"id":29631,"date":"2023-01-30T11:27:44","date_gmt":"2023-01-30T19:27:44","guid":{"rendered":"https:\/\/coderpad.io\/?p=29631"},"modified":"2024-08-28T02:55:19","modified_gmt":"2024-08-28T09:55:19","slug":"chatgpt-and-the-future-of-technical-interviews-addressing-concerns-of-increased-cheating","status":"publish","type":"post","link":"https:\/\/coderpad.io\/blog\/hiring-developers\/chatgpt-and-the-future-of-technical-interviews-addressing-concerns-of-increased-cheating\/","title":{"rendered":"Generative AI and the Future of Technical Interviews: Addressing Concerns of Increased Cheating"},"content":{"rendered":"\n<p>The current popularity of generative AI (Gen AI) like ChatGPT, Claude, and other large language models (LLMs) has certainly raised no shortage of questions for all of us. These questions seem mostly to be grounded in fear \u2013 anxiety that candidates will use these technologies to deceive recruiters about their skills and lead to low-performing new hires within your organization.<br><br>Due to this, many of the questions we get asked at CoderPad have to do with how to keep ChatGPT from messing up the recruiting process.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>How do we ensure candidates aren\u2019t cheating by using ChatGPT in an interview?<\/li>\n\n\n\n<li>What measures are we planning to take to ensure ChatGPT isn\u2019t used in the code submitted in assessments?<\/li>\n<\/ul>\n\n\n\n<p>At CoderPad, we believe in embracing new technologies, including the notorious ChatGPT. In the world of software development, we don\u2019t consider using Google or StackOverflow to help look up a function or using autocomplete and <a href=\"https:\/\/github.com\/features\/copilot\" target=\"_blank\" rel=\"noopener\">GitHub Copilot<\/a> for efficiency to be cheating. In fact, these tools increase efficiency and make your hard-working developers even more efficient.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">These technologies are a critical part of a developer\u2019s tool kit. <\/h2>\n\n\n\n<p>They help developers to code smarter and more efficiently than they could have otherwise. <br><br>They prevent developers from recreating the wheel and instead allow them to move projects forward by creating the next wheel better and faster. <\/p>\n\n\n\n<p>We are not saying that the job of developers is to be an expert copy and paster, but rather to leverage tools available to develop more creative, well-thought out solutions in the limited time that they have. <br><br>Developers utilizing StackOverflow and Google for guidance on technical issues was also once seen as a threat to the recruiting process, as this was seen as a way for candidates to have an unfair advantage over candidates who did not know where to look for technical guidance.&nbsp; <br><\/p>\n\n\n\n<blockquote class=\"wp-block-quote\">\n<p>We find the modern conversation about the validity of using co-pilot and ChatGPT as a tool in interviewing as a similar conversation to the one we were having decades ago about StackOverflow and Google.&nbsp;&nbsp;&nbsp;<\/p>\n<cite>Amanda Richardson &#8211; CEO of CoderPad<\/cite><\/blockquote>\n\n\n\n<p>Knowing how to use technologies is a skill that developers are expected to have and one that should be assessed for. We see models like ChatGPT as the next set of technologies &#8211; and these technologies help developers be better, but can\u2019t replace problem solving, creativity and logic. <\/p>\n\n\n\n<blockquote class=\"wp-block-quote\">\n<p>\ud83d\udd16&nbsp;<strong>Further reading:<\/strong> <a href=\"https:\/\/coderpad.io\/blog\/hiring-developers\/dos-and-donts-cheating-prevention-nathan-sutter\/\">\u201cDon\u2019t Expect Candidates to Have Everything Memorized\u201d, the Dos and Don\u2019ts of Cheating Prevention With Nathan Sutter<\/a><\/p>\n<\/blockquote>\n\n\n\n<h2 class=\"wp-block-heading\">How should employers hire for developer talent in the world of GenAI?&nbsp;<\/h2>\n\n\n\n<p>While there&#8217;s no doubt GenAI has been and will be used for cheating, this concern can be mitigated with the right interview and assessment practices. In fact, by integrating AI into your process, you can gain a more accurate understanding of candidates&#8217; skills and save valuable time.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Play the long game, build trust<\/h3>\n\n\n\n<p>We believe that building trust and transparency throughout the screening process is the most effective solution on multiple levels. This approach not only distinguishes your process from irrelevant or stressful hiring methods but also reduces the likelihood of candidates resorting to dishonest tactics by making them feel respected and valued. Therefore, you should:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Clearly communicate the purpose of the assessment.<\/li>\n\n\n\n<li>Offer appropriate support at all levels of the process &#8212; the candidates should not be afraid to ask you questions!<\/li>\n\n\n\n<li>Foster an environment that reflects genuine collaboration. Show the candidate you care about their growth and success.<\/li>\n<\/ul>\n\n\n\n<p>By integrating these best practices, you can minimize cheating while also enhancing the overall candidate experience, ensuring a fair and effective assessment process.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Create better questions <\/h3>\n\n\n\n<p>Complex, realistic work-related questions not only resist AI manipulation but also offer a far more engaging experience.<\/p>\n\n\n\n<p>Develop questions that address your business\u2019s current challenges and mirror real-world job scenarios. This approach will attract candidates who:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Understand how to address your key problems.<\/li>\n\n\n\n<li>Are genuinely motivated by the work you do.<\/li>\n\n\n\n<li>Align with your company\u2019s mission.<\/li>\n<\/ul>\n\n\n\n<p>By crafting complex, iterative questions that require multiple steps or deeper understanding\u2014rather than simple, one-step questions\u2014you can significantly reduce the risk of cheating. These questions should involve:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Critical thinking<\/li>\n\n\n\n<li>Problem-solving<\/li>\n\n\n\n<li>Creativity<\/li>\n<\/ul>\n\n\n\n<p>These are areas where AI typically struggles to provide comprehensive answers without human-like context or reasoning. This strategy not only makes the assessment more relevant to the actual work the candidate would perform but also minimizes the chances of AI providing complete responses.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><a href=\"https:\/\/coderpad.io\/wp-content\/uploads\/2024\/08\/AI-resistant-coding-interview-questions.png\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"513\" src=\"https:\/\/coderpad.io\/wp-content\/uploads\/2024\/08\/AI-resistant-coding-interview-questions-1024x513.png\" alt=\"The image is titled &quot;AI 'resistant' questions&quot; and discusses the concept of creating questions that are more resistant to cheating by AI tools during assessments or screenings. It suggests that more complex and iterative questions are harder for AI to solve compared to simpler, single-pass questions. The key points are:\n\nLess Resistant Questions: These include &quot;Algo questions,&quot; &quot;Single pass,&quot; and &quot;Single file exercise.&quot;\nMore Resistant Questions: These are described as &quot;Complex questions,&quot; &quot;Iterative questions,&quot; and &quot;Multi-file exercise.&quot;\nThe implication is that moving towards more complex, iterative, and multi-faceted question structures can make it more difficult for AI to cheat or provide correct answers without truly understanding the problem. This approach is recommended to minimize the chances of AI-assisted cheating in screening processes.\" class=\"wp-image-40635\" srcset=\"https:\/\/coderpad.io\/wp-content\/uploads\/2024\/08\/AI-resistant-coding-interview-questions-1024x513.png 1024w, https:\/\/coderpad.io\/wp-content\/uploads\/2024\/08\/AI-resistant-coding-interview-questions-300x150.png 300w, https:\/\/coderpad.io\/wp-content\/uploads\/2024\/08\/AI-resistant-coding-interview-questions-768x385.png 768w, https:\/\/coderpad.io\/wp-content\/uploads\/2024\/08\/AI-resistant-coding-interview-questions-18x9.png 18w, https:\/\/coderpad.io\/wp-content\/uploads\/2024\/08\/AI-resistant-coding-interview-questions.png 1292w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/a><\/figure>\n<\/div>\n\n\n<p>With that said, Gen AI can still be an effective and important part of this process, especially if your developers will be using it in their day-to-day jobs. <\/p>\n\n\n\n<p>So how do you go about implementing this while addressing potential cheating concerns?<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Gen AI and the asyncronous technical assessment<\/h3>\n\n\n\n<p>During the asynchronous technical assessment phase, allow candidates to use AI models to come up with answers. You can then follow up with written questions asking the candidate to critique the code produced by the model.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>How does it work? Write a readme file on how the code works.<\/li>\n\n\n\n<li>What would they have done differently to solve the problem?<\/li>\n\n\n\n<li>What could improve the code performance?&nbsp;<\/li>\n<\/ul>\n\n\n\n<p>Another option available to you with CoderPad is to use an<a href=\"https:\/\/coderpad.io\/features\/cheating-prevention\/\" data-type=\"link\" data-id=\"https:\/\/coderpad.io\/features\/cheating-prevention\/\"> AI-generated and validated follow-up question<\/a>. If it detects a suspicious answer, our own ChatGPT integration will generate a follow-up question asking a candidate to explain a piece of their code and then validate the answer to ensure they understood what they were typing or pasting into the answer box.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Allowing Gen AI during a live interview<\/h3>\n\n\n\n<p>During a live interview, we recommend spending time digging into the why behind the candidate\u2019s work. <br><br>Ask questions that help you to assess how the candidate is thinking and problem solving. <a href=\"https:\/\/coderpad.io\/blog\/hiring-developers\/hiring-managers-you-should-want-candidates-to-use-chatgpt-to-pass-your-technical-interviews-heres-why\/\">Allow candidates to use ChatGPT<\/a> or other Gen AI to assist in problem solving and then talk through how you would approach a code review. <br><br>If you have concerns that the candidate used an AI model to develop code and just copied it, you can determine so by using CoderPad\u2019s playback feature. This allows the interviewer to playback the candidate\u2019s keystrokes and will make it obvious if code was copied and pasted in from an AI model.&nbsp;<\/p>\n\n\n\n<p>Ultimately, <a href=\"https:\/\/interviewing.io\/blog\/how-hard-is-it-to-cheat-with-chatgpt-in-technical-interviews\" target=\"_blank\" rel=\"noopener\">ChatGPT is not an excuse for a developer not to think or a replacement for understanding how code works<\/a> &#8211; in the same way that using a calculator is not a replacement for a math student knowing how to subtract or divide. However, it is a way to get things done in a new and faster way. And when coupled with problem solving, creativity, and logic &#8211; it can make a great developer into an even stronger one.<\/p>\n\n\n<div class=\"coderpad-container-buster-block\" style=\"margin: 0 calc(50% - 50vw);\"><div style=\"max-width: 90rem; margin: 0 auto;\">\n<section class=\"\n\tsurvey-report-signup-form-block\n\t\t\n\" data-block-name=\"coderpad-survey-report-signup-form\" data-block-bg-color=\"black\" data-block-padding-bottom=\"medium\" >\n\n\t<div class=\"inner\">\n\t\t\t\t\t<h2 class=\"title\">Like what you\u2019re reading?<\/h2>\n\t\t\n\t\t\t\t\t<p class=\"sub-title\">Why not subscribe to our monthly newsletter?<\/p>\n\t\t\n\t\t<div class=\"content\">\n\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"920\" height=\"507\" src=\"https:\/\/coderpad.io\/wp-content\/uploads\/2024\/08\/coderpad-newsletter-signup-illustration.png\" class=\"attachment-large size-large\" alt=\"\" srcset=\"https:\/\/coderpad.io\/wp-content\/uploads\/2024\/08\/coderpad-newsletter-signup-illustration.png 920w, https:\/\/coderpad.io\/wp-content\/uploads\/2024\/08\/coderpad-newsletter-signup-illustration-300x165.png 300w, https:\/\/coderpad.io\/wp-content\/uploads\/2024\/08\/coderpad-newsletter-signup-illustration-768x423.png 768w, https:\/\/coderpad.io\/wp-content\/uploads\/2024\/08\/coderpad-newsletter-signup-illustration-18x10.png 18w\" sizes=\"auto, (max-width: 920px) 100vw, 920px\" \/>\n\n\t\t\t<div class=\"important-parts\">\n\t\t\t\t\t\t\t\t\t<div class=\"description\"><p>Once a month, you\u2019ll receive a hand-picked selection of tech recruitment content: blog posts, checklists, podcasts, videos, studies and statistics\u2026 including insights from our own developer and HR communities! Just drop us your email below.<\/p>\n<\/div>\n\t\t\t\t\n\t\t\t\t<div class=\"\n    hubspot-form\n            hubspot-form--single-input\n            \n\" >\n\t\t<div\n\t\tid=\"coderpad-form-kyetid\"\n\t\tclass=\"hubspot-form-container js-coderpad-hubspot-form\"\n\t\tdata-cp-form-options=\"{&quot;portalId&quot;:&quot;26307476&quot;,&quot;formId&quot;:&quot;b426bea5-fad2-4747-a539-20fae02248e2&quot;,&quot;region&quot;:&quot;eu1&quot;}\"\n\t>\n\t<\/div>\n<\/div>\n\n\t\t\t<\/div>\n\t\t<\/div>\n\t<\/div>\n<\/section>\n\n<\/div><\/div>","protected":false},"excerpt":{"rendered":"<p>The invention of ChatGPT has raised no shortage of questions. These questions seem mostly to be grounded in fear \u2013 anxiety that ChatGPT will replace the need for our jobs, our unique skills, and our need to think.<\/p>\n","protected":false},"author":1,"featured_media":39455,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[23],"tags":[],"persona":[27,28],"blog-programming-language":[],"keyword-cluster":[],"class_list":["post-29631","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-hiring-developers"],"acf":[],"_links":{"self":[{"href":"https:\/\/coderpad.io\/wp-json\/wp\/v2\/posts\/29631","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/coderpad.io\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/coderpad.io\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/coderpad.io\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/coderpad.io\/wp-json\/wp\/v2\/comments?post=29631"}],"version-history":[{"count":9,"href":"https:\/\/coderpad.io\/wp-json\/wp\/v2\/posts\/29631\/revisions"}],"predecessor-version":[{"id":40734,"href":"https:\/\/coderpad.io\/wp-json\/wp\/v2\/posts\/29631\/revisions\/40734"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/coderpad.io\/wp-json\/wp\/v2\/media\/39455"}],"wp:attachment":[{"href":"https:\/\/coderpad.io\/wp-json\/wp\/v2\/media?parent=29631"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/coderpad.io\/wp-json\/wp\/v2\/categories?post=29631"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/coderpad.io\/wp-json\/wp\/v2\/tags?post=29631"},{"taxonomy":"persona","embeddable":true,"href":"https:\/\/coderpad.io\/wp-json\/wp\/v2\/persona?post=29631"},{"taxonomy":"blog-programming-language","embeddable":true,"href":"https:\/\/coderpad.io\/wp-json\/wp\/v2\/blog-programming-language?post=29631"},{"taxonomy":"keyword-cluster","embeddable":true,"href":"https:\/\/coderpad.io\/wp-json\/wp\/v2\/keyword-cluster?post=29631"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}