Please also see Part 1: Rethinking the Backend Engineering Interview Take-Home Challenge and Part 2: The Recruiting Perspective and Results.
In the last two blog posts, we talked about how we rethought our backend engineering challenge as a way of improving numbers for recruitment. In the same vein, this post talks about how we changed our hiring strategies to adapt to hiring remotely during COVID-19.
Onsite engineering interviews at SoundCloud include whiteboarding, pair programming, barkeeper,1 product manager, and engineering manager interviews. Because of COVID-19, SoundCloud switched to working completely remotely in March of 2020. However, we didn’t let this new setup put a stop to our recruiting efforts.
In the past, we had all of a candidate’s interviews scheduled onsite for the same day at our SoundCloud office space. As we moved to a full remote work environment, we made all the interviews virtual. Virtual interviews made us more flexible, as we could spread the interviews across multiple days to better accommodate the availability of the candidates and interviewers.
Out of the five aforementioned interview types, the latter three types were the simplest to make work in a remote setup because they mostly consist of a verbal dialogue. To carry out these interviews, we set up video calls between the interviewers and the candidates. The candidates were given details about this in advance so that they could try out the setup and make sure they had good audio and video connectivity.
When doing this, we discovered we could get similar signals about a candidate’s proficiency through a video call as we would in an in-person interview. The interviewers also found it was easier to take notes during the call without it being awkward. And the candidates seemed more comfortable taking part in the interview in spaces they were familiar with.
We used Slack to coordinate with one another internally when someone finished an interview to make sure we allowed enough time for the candidate to take a quick break and refresh before the next interview began. This was also a helpful tool for communicating with recruiters — for example, if the candidate couldn’t join on time or had to drop off the call because of any connectivity issues. There was even an instance where we sent the incorrect VC link to the candidate, and having a running conversation on Slack helped us quickly communicate with the recruiter and thereby the candidate.
Whiteboard and pairing interviews were a bit more challenging to adapt to a remote setting, as a big portion of our technical assessment happens over these sessions. Running these interviews in a remote context forced us to invest in better tools, familiarize the interviewers with them, and send out material to candidates in advance to allow them to also understand the tools.
For the system design interview, we used Miro as a whiteboard tool. The candidates could use it to sketch out the system design and walk through their ideas. The first time, we ran into unanticipated permission issues that had to be resolved just before the call. We quickly learned from this and started setting up the Miro board and sharing it with candidates in advance. These virtual boards also made it easier for interviewers to go back and review the work if necessary before filling out the feedback for candidates.
For the code pairing session, we opted to use video conferencing along with screen sharing. This allowed candidates to use the IDE of their choice. However, the interviewer missed out on being able to contribute or point to specific lines of code. It was also difficult when candidates paused and explained before or while making their code changes. As a result, the interviewers needed to be explicit about asking probing questions and clarifying things to be able to keep up with a candidate’s train of thought.
In addition to us learning directly through this experience, we tailored our post-interview feedback forms from the candidates to get their feedback on the interviewing experience. We also collected feedback from the interviewers to make possible improvements and reevaluate the tools.
The entire experience of running interviews has been an ongoing learning process for us and we are still trying to find optimal solutions to some of the challenges. For example, we haven’t found a good replacement for the casual lunch session we had in our onsite interview day. We tried having a casual video call as an alternative, but we haven’t found it to be as equally informal as having lunch together.
Another challenge has been to find a best practice for sharing access to our remote interview tools with the candidates within a reasonable timeframe and in a way that makes it clear how to set up and use them.
When initially faced with the challenges of moving our interviews to a remote setting, it seemed like the overhead of setting up tools and video conferencing in advance would be a lot of work on our part. However, through feedback and iteration, this became a seamless part of our hiring flow. As a result, we were able to continue to hit our hiring goals, even in the midst of a global pandemic.
Overall, running our full onsite interview days in a completely remote setup has been a huge learning experience. It allowed us to be more flexible, empathetic, and inclusive, and we will continue to use these learnings to improve our hiring moving forward.
Barkeepers bring an outside perspective to the hiring team. They represent the technical excellency, culture, and mindset of SoundCloud. They have a history of hiring effective engineers, experience running interviews, and technical depth in multiple areas.
↩