Scaling Agile Development
Adapting Your Methodology as You Grow
Apr 09, 2017

Holy wars are pretty common in software development. Tabs vs Spaces. Same line vs Next line. Agile vs Waterfall (kidding!). We have a lot of these debates in jest, at least that’s what we tell ourselves (the whiteboard below appeared after a few days of drawing redrawing our git branching strategy).

Whiteboard illustration of a parody git branching strategy

But some of these debates are serious, like How do you balance innovation and risk? or Is our agile methodology working? The decisions you make have real consequences to development efficiency, team cohesion, and even personal career satisfaction. At Asymmetrik, we try to mix elements of Scrum- or Kanban-based agile development methodologies with the principles of “The Lean Startup” by Eric Reis. Development cycles stay short so we can validate our ideas rapidly with real users. Process and overhead are kept to a minimum so we spend most of our time building and validating. And, we emphasize communication and collaboration. This is why we have an open work environment and leverage tools like Slack and the Atlassian suite.

The decisions you make can have real consequences – to development efficiency, team cohesion, and even personal career satisfaction

All that is great, but the devil is always in the details. This article explore some of the challenges that we’ve faced adapting our agile development methodology as our projects and teams have grown.

Scrum vs Kanban vs Scrumban?

Maybe by default, we started running our team of 3-4 developers using Scrum. We estimated tasks, maintained a backlog, had daily standups, and boxed our development cycles to two week sprints. This worked really well at first, but our team grew pretty quickly to 8 developers. At that point, we faced a major challenge with pure Scrum in dealing with the transition between sprints. There always seemed to be a period of several days during which some team members struggled to finish their tasks, while others were done and looking for new things to start. This created a predictable chaos every couple weeks and developers were growing restless and felt underutilized as a result.

We were losing sight of the bigger picture because we always had our heads down working on the queue of tickets

To deal with this issue, we tried switching to a continuous agile development process based on Kanban in which we limited work in progress, but otherwise didn’t box our work into sprints. This eliminated the awkward lulls between sprints (cause they no longer existed). Developers were able to continuously transition from task to task without having to wait for the rest of the team. But, the lack of well-defined sprints created a sense of never-ending development. No one really knew when to do planning or when to release. We were losing sight of the bigger picture because we always had our heads down working on the queue of tickets.

As a compromise, we tried an agile approach based loosely on Scrumban. Scrumban combines Kanban’s continuous development process and limits on work in progress, with Scrum’s mechanics of planning and prioritizing.

How we did Scrumban

  • We start by populating a prioritized list of tasks that are ready to work in the To Do column.
  • Tasks move to In Progress when work starts.
  • When a task is completed, it moves to the Done column.
  • We perform planning on-demand when the To Do list gets too short.
  • We perform releases on-demand when the Done list grows too large.

Sounds fairly straightforward, right? Scrumban balances planning and predictability, solving a lot of the smaller issues we were running into with Scrum and Kanban. The agile plugins for Atlassian Jira support configuring Kanban boards with column rules that trigger when there are too few or too many tasks. This made it much easier to implement and maintain our new approach.

How do you Scale an Agile Methodology?

We were doing great with Scrumban. Cranking out features, fixing bugs, and everything was awesome. But then, cracks started to show. We noticed that we were always planning. And, when we planned we always felt overwhelmed – buried in tickets. Constantly organizing and prioritizing hundreds of tasks was grueling. Tracking progress across a team of a dozen developers was impossible. As our team grew, these issues became unmanageable. We were spending all our time managing our process and not solving customer problems. Agile development is easy when teams are small. Communication is quick and efficient. Everyone knows what’s going on. Meetings are focused. People feel empowered. But, when a team grows communication gets harder. Coordination and planning less efficient.

As our team grew, these issues were becoming unmanageable – we were spending all our time managing our process and not solving customer problems

The challenge comes when you show success and suddenly, you have a team of 10 to 20. Now you’re spending most of your time just trying to keep the team organized and on track. More people means more opinions, leading to longer and more contentious meetings, code reviews, and pull requests. So many moving parts makes it hard to maintain a consistent vision. Developers then become restless because they’ve lost the autonomy, influence, and efficiency they once had. Communication suffers and people feel out of the loop. Team leads are miserable spending all their time managing and planning instead of building stuff.

Scrum of Scrums

Finally, we had a breakthrough. We divided the team into sub-teams with specific focus areas (things like “data flow”, “analytics”, and “web application”) and implemented a “Scrum of Scrums”, using Scrumban as our actual methodology.

How we ran our Scrum of Scrums

  • Each sub-team nominates a lead who manages his/her own task board and backlog and can run things how he/she chooses.
  • The overall project lead maintains a high-level functional task board that prioritizes and tracks the status of the overall project goals.
  • When a new functional task is started, the relevant sub-teams meet to work out the overall requirements, to subdivide the work, and to define expectations, responsibilities, and interfaces.
  • The sub-team leads and overall lead meet at least three times a week for a Scrum of Scrums Standup where progress is tracked and issues and priorities are discussed.
  • While we aren’t strictly doing Scrum, we still hold retrospectives to make sure we’re still gathering feedback and improving our processes.

Observations

Using Scrumban with the Scrum of Scrums approach, we’re finding that we’re spending less time managing our process and more time building stuff. Everyone is more satisfied, and the lead has a better sense of where the team is overall. For us, the right size for each functional team is around 3 to 5 people. More than that, and the same issues start creeping in that led us down this path in the first place. Fewer than that and it’s process overkill.

One challenge with our new approach is making sure people stay up to date with the high-level goals and direction of our project. You do have to stay disciplined about holding periodic retrospective feedback meetings and be willing to continue evolving your process for the better. Also, pointing out successes and discussing long-term project plans with the entire team once in awhile helps keep everyone in the loop.