{"id":31550,"date":"2023-03-08T07:34:26","date_gmt":"2023-03-08T15:34:26","guid":{"rendered":"https:\/\/coderpad.io\/?p=31550"},"modified":"2024-08-05T10:03:49","modified_gmt":"2024-08-05T17:03:49","slug":"how-to-design-your-coding-interviews-to-outsmart-chatgpt","status":"publish","type":"post","link":"https:\/\/coderpad.io\/blog\/interviewing\/how-to-design-your-coding-interviews-to-outsmart-chatgpt\/","title":{"rendered":"Worried About ChatGPT in Coding Interviews? Here’s 3 Ways to Continue Running Great Interviews"},"content":{"rendered":"\n

Remember when you only had to worry about candidates copying and pasting code from StackOverflow?<\/p>\n\n\n\n

Now there\u2019s a new robot kid on the block, and boy, is it making interviewers and technical recruiters nervous.<\/p>\n\n\n

\n
\"Three<\/a><\/figure>\n<\/div>\n\n\n

And rightfully so \u2013 it opens a whole new way for candidates to pull the wool over your eyes and fake skills they don\u2019t have, meaning you\u2019ll have a bunch of useless new hires that will cost you hundreds of thousands of dollars.<\/p>\n\n\n\n

Or not.<\/p>\n\n\n\n

Let me remind you of this piece of wisdom I doled out in April of 2022<\/a>:<\/p>\n\n\n\n

We at CoderPad do not philosophically believe in \u201canti-cheat\u201d tactics. That\u2019s an arms race that only produces temporary winners \u2013 and it\u2019s never good to start an interview coming from a place of distrust.<\/strong><\/em><\/p>\n\n\n\n

\u201cBut unknown CoderPad blogger,\u201d you say (my coworkers call me Ken, by the way), \u201cThat\u2019s all well and good for you to say, but I\u2019m the one staking my company\u2019s reputation on the line with these phony candidates. Do you just expect me to ignore ChatGPT?\u201d<\/p>\n\n\n\n

Of course not. First and foremost one of the best things you can do is to be clear with candidates of expectations and the consequences of using ChatGPT in an interview.<\/p>\n\n\n\n

There\u2019s also a spectrum of interview designs to help you mitigate ChatGPT\u2019s influence. On one end, you can embrace it and create interview questions around its use<\/a> to letting candidates know it\u2019s okay to use it<\/a> (like an open-book test). On the other end, you can work on creating questions that are ChatGPT-resistant. This blog post will focus on the latter \u2013 though I\u2019d encourage you to consider the other two options.<\/p>\n\n\n\n

Get \u2018em talkin<\/h2>\n\n\n\n

Nothing puts a stop to using someone else\u2019s solution than being asked to explain the logic behind it.<\/p>\n\n\n\n

It\u2019s unusual for a candidate to skip straight to the most ideal answer. Humans think through complex problems on an evolutionary path, starting simple then abandoning all but the most ideal branch. Eventually the best candidates do arrive at the right answer, but you can see their methodical thought process.<\/p>\n\n\n\n

We\u2019ve talked extensively about the benefits of making your interviews a collaborative discussion<\/a>, but one of the lesser-known benefits these days is encouraging your candidates to explain the large chunk of pasted code you just saw pop up in your interview screen:<\/p>\n\n\n\n

Nothing fishy here….<\/figcaption><\/figure>\n\n\n\n

Instead of waiting for the interview playback to notice their massive copypasta and terminating their candidacy because of perceived cheating, ask them about the code they copied and pasted.<\/p>\n\n\n\n

They may have a good reason \u2013 maybe they know it would be more efficient to use their time to outsource the trivial code to ChatGPT and focus instead on the lines that take more critical thinking. Or perhaps they knew the basic idea of what they wanted to code but needed a little help verifying the implementation \u2013 help like what they\u2019d usually receive from a coworker. <\/p>\n\n\n\n

Or maybe they don\u2019t have a good reason and lied about their coding skills. Whether or not their reasons (or lack thereof) are red flags is ultimately up to you. But you won\u2019t know if the copy-paste you just witnessed is in good or bad faith until you get the candidate to open up about it.<\/p>\n\n\n\n

The more you talk with them and make the interview experience collaborative, the more comfortable the candidate will be and the more likely they\u2019ll explain their thought process to you.<\/p>\n\n\n\n

You\u2019ll learn more about their skill set by asking them how they think than just watching them code anyways. Or you can watch them squirm when ChatGPT answers their question incorrectly<\/a>.<\/p>\n\n\n\n

Create questions with nuance<\/h2>\n\n\n\n

One of the things we\u2019re learning about current iterations of AI-powered tools like ChatGPT is that it doesn\u2019t do so well at understanding nuance or the technical intricacies of specific topics. <\/p>\n\n\n\n

Take this question about creating palindromes from date formats:<\/p>\n\n\n\n

\n

I was asking ChatGPT about palindrome dates for this century and found several algorithmic errors\/bugs. Can you spot them? pic.twitter.com\/fCgIFu9PKE<\/a><\/p>— Anthony Mooz (@AnthonyMooz) February 24, 2023<\/a><\/blockquote>