Can "Agile in the Large" Succeed?
Agile Development isn’t perfect, but it got something right. Right enough, at least, to gain enough momentum to become a buzzword -- and for consultants to latch on by selling “Agile” to big enterprises. The result is “Agile in the Large”.
Agile in the Large ranges from the Half-Arsed Agile Manifesto to Cargo Cult practices. Too often we simply bolt on a few agile practices without understanding how they provide value. Adding scrums and iteration reviews to your current process and expecting improvement is like building a runway in your back yard and expecting planes to land.
Whether it’s labelled agile or not, a cohesive development team works for a couple reasons. It creates a focused team fully aligned to a common goal. It avoids “Us and Them” mentalities, enabling everyone to adapt to meet the goal. It offers a self-correcting strategy for the thousands of micro-decisions when building software: It must be testable, “You Ain’t Gonna Need It”, and it creates quick feedback to make sure you’re solving the actual problem.
Agile in the Large is flawed because it often fails to achieve these basic ingredients. Take Scrum of Scrums, for example. It tries to coordinate and resolve technical dependencies between teams by having representatives from each team get together and talk through them. It’s better than nothing, but an emerging system has too much ambiguity: the interaction between components is still being defined, bugs can lead to finger pointing, and the important sense of ownership is lost. Everyone feels like cogs in a machine rather than someone solving an important problem for a real user. Our ability to adapt the microdecisions that build great software is lost.
In fact, Agile in the Large seems doomed from the beginning. Once you’re beyond some number of people in a single project, it’s impossible to create that sense of shared ownership and adaptability. Great software is created by small, dedicated teams.
So, what then? It seems the only way out is to make Agile in the Large more like Agile in the Small. A team should be working on one and only one project, and include everything necessary for that project to be successful. How we do that depends on where we are in a project’s maturity curve.
New vs. mature systems
New development is the most easily handled. Look at the problems at hand, and make sure the team is equipped to handle them end-to-end. This new project may have several components, but now is not the time to split those components into their own teams; keep a single team aligned to the user’s goals. After all, if a component isn’t aligned to some user’s goal, what is its value? Organize around components too early, and team cohesion is lost.
So we are off to a good start, but we need to adjust our strategy as a successful project matures. You may find parts of your system has value for other uses. This seems timeless: C first was a successful tool for building Unix, Ruby on Rails was an application before it was a framework, Amazon was an online book store before it was a cloud platform, and so on. In all of these cases a successful system was built, and then groups arise around the reusable pieces. Reusable technology will naturally arise from a successful project. Embrace that, but don’t force it.
Open source as a model for reuse
Later in a system’s life we find ourselves consuming many assets from a variety of teams. Now it’s easy to let coordination and communication friction kill our focused project. Fortunately, the open source world gives us the answer.
Quite simply, needed enhancements to common assets should be written and contributed by consuming teams. This offers several advantages over logging an enhancement request. For instance, we reduce deadline haggling and misaligned schedules. We also reduce the need for frequent status updates and opportunities for miscommunication between teams.
Of course, not all changes to open source projects come in the form of patches. There should still be a team around the asset in question, responsible for its architecture and fundamental direction. That team also operates as a gatekeeper, ensuring all patches are properly unit tested, documented, and don’t threaten the integrity of the system.
Changes must be handled on a case-by-case basis, but the primary mode of operation should follow the agile ideal: a single team with responsibility for a project end-to-end. This includes contributing to assets it consumes.
What if even the initial project is too big?
I’ll touch on one final question: what if even the initial scope of a project is beyond what a single team can accomplish? Find a smaller initial scope, get it working, and grow from there. Do this even if the initial scope means building placeholders that are discarded in the final result. The time saved and friction eliminated by creating a single, focused team will outweigh this. Think of it as scaffolding that eventually gets removed.
In the end, Agile in the Large only works if we make it more like Agile in the Small. Hopefully this article is a step in that direction.
Agile in the Large ranges from the Half-Arsed Agile Manifesto to Cargo Cult practices. Too often we simply bolt on a few agile practices without understanding how they provide value. Adding scrums and iteration reviews to your current process and expecting improvement is like building a runway in your back yard and expecting planes to land.
Whether it’s labelled agile or not, a cohesive development team works for a couple reasons. It creates a focused team fully aligned to a common goal. It avoids “Us and Them” mentalities, enabling everyone to adapt to meet the goal. It offers a self-correcting strategy for the thousands of micro-decisions when building software: It must be testable, “You Ain’t Gonna Need It”, and it creates quick feedback to make sure you’re solving the actual problem.
Agile in the Large is flawed because it often fails to achieve these basic ingredients. Take Scrum of Scrums, for example. It tries to coordinate and resolve technical dependencies between teams by having representatives from each team get together and talk through them. It’s better than nothing, but an emerging system has too much ambiguity: the interaction between components is still being defined, bugs can lead to finger pointing, and the important sense of ownership is lost. Everyone feels like cogs in a machine rather than someone solving an important problem for a real user. Our ability to adapt the microdecisions that build great software is lost.
In fact, Agile in the Large seems doomed from the beginning. Once you’re beyond some number of people in a single project, it’s impossible to create that sense of shared ownership and adaptability. Great software is created by small, dedicated teams.
So, what then? It seems the only way out is to make Agile in the Large more like Agile in the Small. A team should be working on one and only one project, and include everything necessary for that project to be successful. How we do that depends on where we are in a project’s maturity curve.
New vs. mature systems
New development is the most easily handled. Look at the problems at hand, and make sure the team is equipped to handle them end-to-end. This new project may have several components, but now is not the time to split those components into their own teams; keep a single team aligned to the user’s goals. After all, if a component isn’t aligned to some user’s goal, what is its value? Organize around components too early, and team cohesion is lost.
So we are off to a good start, but we need to adjust our strategy as a successful project matures. You may find parts of your system has value for other uses. This seems timeless: C first was a successful tool for building Unix, Ruby on Rails was an application before it was a framework, Amazon was an online book store before it was a cloud platform, and so on. In all of these cases a successful system was built, and then groups arise around the reusable pieces. Reusable technology will naturally arise from a successful project. Embrace that, but don’t force it.
Open source as a model for reuse
Later in a system’s life we find ourselves consuming many assets from a variety of teams. Now it’s easy to let coordination and communication friction kill our focused project. Fortunately, the open source world gives us the answer.
Quite simply, needed enhancements to common assets should be written and contributed by consuming teams. This offers several advantages over logging an enhancement request. For instance, we reduce deadline haggling and misaligned schedules. We also reduce the need for frequent status updates and opportunities for miscommunication between teams.
Of course, not all changes to open source projects come in the form of patches. There should still be a team around the asset in question, responsible for its architecture and fundamental direction. That team also operates as a gatekeeper, ensuring all patches are properly unit tested, documented, and don’t threaten the integrity of the system.
Changes must be handled on a case-by-case basis, but the primary mode of operation should follow the agile ideal: a single team with responsibility for a project end-to-end. This includes contributing to assets it consumes.
What if even the initial project is too big?
I’ll touch on one final question: what if even the initial scope of a project is beyond what a single team can accomplish? Find a smaller initial scope, get it working, and grow from there. Do this even if the initial scope means building placeholders that are discarded in the final result. The time saved and friction eliminated by creating a single, focused team will outweigh this. Think of it as scaffolding that eventually gets removed.
In the end, Agile in the Large only works if we make it more like Agile in the Small. Hopefully this article is a step in that direction.