SoundCloud for Developers

Discover, connect and build

We use cookies for various purposes including analytics and personalized marketing. By continuing to use the service, you agree to our use of cookies as described in the Cookie Policy.

Backstage Blog June 5th, 2018 Project Management Project Necromancy By Christine Patton

Or, how to raise a project from the dead with tools you probably have lying around at home.

Background

In a perfect world, every project would flow smoothly — from requirement gathering, to implementation, to acceptance by stakeholders, to launch. Sadly, we don’t live in a perfect world. Sometimes projects hit bumps in the road due to changing business requirements, market conditions, or a host of other factors. This post describes our experiences managing a project that was deprioritized during implementation and lay dormant for an extended period of time before resuming in a different form.

The Life and Death of Project Mark I

The original incarnation of the project in question would have added a feature to an internal tool with significant use across different areas of SoundCloud.

For this project, we added a new microservice to our backend architecture. We deployed and integrated this service without any functionality to begin with, and then we set about building (though not yet deploying) the service’s feature set.

We suffered a minor setback in feature development when a crucial, overlooked edge case came to light and forced a partial redesign of one service component. This resulted in our having to throw away the code we’d written for the component and start again. It was not a significant development in and of itself, but it later informed our thinking about this component. We felt we’d learned an important lesson from the initial mistake and any further changes to the component would risk reintroducing the design flaw.

Otherwise, the project was going fine… Until the day it was suddenly canceled. Our product roadmap had shifted. The functionality we were building was no longer important. The team’s energy needed to focus elsewhere.

We discussed removing our new, currently non-functional service altogether, but we decided to leave it in place. After all, it wasn’t doing any harm, and perhaps the project would resume one day. We also preserved our partially finished feature code in a git branch, just in case.

Then we moved on and largely forgot about our seemingly doomed project.

Interlude

The next several months saw our team merge with another team and several individual team members move to unrelated areas within SoundCloud. Meanwhile, our product roadmap continued to evolve, and the functionality we’d started to deliver with Project Mark I, previously unimportant, eventually became important again. Or rather, certain aspects of the functionality became important again.

Four months after the original project’s cancellation, we were ready to resume developing a subset of its features with a team that contained almost none of the original engineers.

The Project Reanimated

We had to make a decision. Should Project Mark II build on the work we’d done for Project Mark I or not? Project Mark II contained only some of the planned Mark I features. Was there enough commonality to resume the old project, or would it be easier to start afresh?

We decided to reuse as much of the Mark I code and documentation as possible. And after an abbreviated requirements phase, we set to work building Mark II on top of the existing Mark I codebase and data model.

Picking up unfinished code written by engineers who were no longer on the team presented challenges, but not the typical challenges of working with legacy code. We had no problems with documentation, technical debt, or understanding the existing code. Instead, we developed a cognitive anchor effect around choices the previous engineers had made. We were reluctant to question design decisions because: “They must have done that for good reason.” This was especially the case given that we knew an earlier design mistake had necessitated a partial code rework. We thought we were incorporating earlier learnings by leaving things as they were and that we risked repeating the earlier mistake if we changed things.

The engineers who wrote the initial code were all still at SoundCloud and would have been happy to answer questions, but they were also busy with their new teams, and we didn’t want to bother them. As it turned out, we should have bothered them more.

Another group we should have bothered more was our set of stakeholders in SoundCloud’s product organization. This became apparent when we demonstrated Mark II’s functionality to them only near the end of the project, and at that demo, learned a crucial fact: a major feature we’d built for them was not needed. Unfortunately, the concept behind this feature was “baked into” the data model we’d reused, unchanged from the original Mark I project where it had been a hard requirement. As we’d failed to question the original data model, we’d also failed to question the need for this complicated feature that we had spent a considerable amount of time building.

Oops.

It didn’t make sense to support this feature if it wasn’t needed, especially as it complicated the user interface. We therefore made the somewhat painful decision to push back our launch while we removed the feature and simplified the data model. While the delay was frustrating, we felt it was important to deliver the right project, even if that meant delivering later. Today, Project Mark II has many happy users, so we believe we made the right decision.

Lessons Learned

Difficult or frustrating situations nevertheless yield valuable knowledge for the future. That said, here are some of the things we discovered over the lifetime of these projects.

Reanimated Projects Are Changed Projects

We thought of Project Mark II as a direct continuation of Project Mark I. This was only partially accurate. The decision to reuse the Mark I code and data model introduced an implicit assumption into Mark II: “When in doubt, old decisions are right.” While reusing the existing work certainly saved us substantial time and effort, the cognitive bias against questioning the existing work also cost us substantial time and effort.

Requirements Are Even More Important Than Usual

Getting requirements right is important in any project. It’s also hard, and it’s even harder when the requirements have a high chance of becoming outdated or being misunderstood.

When we started Project Mark II, we revalidated some of the Mark I requirements, but we should have rerun the project’s entire requirements phase afresh. This would have forced us to examine requirements that had already become code. It would have also forced us to engage with our stakeholders, who were scattered across multiple teams and multiple geographic locations, and to have conversations we didn’t know we needed to have.

Show Your Work

Finally, given the project’s unusual lifecycle, we should have demoed our progress much sooner than we did. We don’t generally do sprint demos in our team, but in this case, it would have saved us a lot of effort.

TL;DR

When resurrecting an unfinished, dormant project:

  • Revisit your requirements. All of them.
  • Revisit your existing code. All of it.
  • Be conscious of cognitive biases that say, “But someone decided that earlier.”
  • Adopt a flexible mindset. Be prepared to change course.
  • Communicate with stakeholders even more than usual.