As the Manifesto for Agile Software Development celebrates its 20th anniversary, I take a look at how agile practices have influenced and changed product management. I discuss the benefits that have been achieved and the challenges that still remain.
You can also listen to the audio version of this article and watch the video below:
Once Upon a Time in Waterfall Land
Before the advent of agile frameworks like Scrum, a product person — the product manager — would typically carry out the market research, compile a market requirements specification, create a business case, put together product roadmap, write a requirements specification, and then hand it off to a project manager. The latter would work with one or more development teams to get the specification implemented.
During the development phase, the product manager would be only loosely involved, typically attending a project steering meeting and possibly issuing change requests. Otherwise, the individual would hope that the requirements were implemented as specified. Only once the product was close to being finished would the product manager return to the project and prepare the release of the product.
This sequential, waterfall-based approach used to work when there was little change and innovation, when product managers could correctly predict what the users needed and describe the detailed product functionality upfront. But it is less suited to create complex digital products.
The Brave New Agile World
As agile practices have become more widely adopted, the processes used to develop products have significantly changed: Product people and development teams now tend to collaborate much more closely. Dev teams have become cross-functional consisting of UX designers, architects, programmers, testers, and other roles. Products are developed using iterative-incremental processes like Scrum. Requirements are no longer detailed and frozen before development starts but they emerge. An increasing number of organisations have moved away from organising around projects and have started to embrace a product-led approach.
Early user feedback, frequent solution validation: We now have the ability to collect early and frequent user and customer feedback, which helps us validate our ideas and update our plans accordingly. This has increased the chances of creating a product with the right UX and the right features.
Reduced time-to-market: We can now release new products and features more quickly. This is enabled by a closer, ongoing collaboration with cross-functional development teams, shifting from written documentation to face-to-face conversations, and using techniques such as user stories that reduce overhead — when applied correctly.
Better product quality and improved adaptability: The quality of our products has improved through the application of agile development practices like emergent design, test-driven development, and continuous integration. This has allowed us to adapt the product more quickly and to respond to user feedback more easily.
Better requirements: As product people, we are no longer solely responsible for coming up with the correct requirements. Instead, the dev team members actively participate in the product backlog refinement work and help us identify the necessary changes and capture new product backlog items. This leverages the team members’ creativity and expertise, creates a shared understanding, fosters collective ownership, improves the quality of the requirements, and ultimately results in better products.
Transparent development progress: We can see the development progress more clearly and make corrections early if required: The progress is now based on working software rather than a detailed, Gantt chart-based project plan. This mitigates the risk of discovering late that the product cannot be shipped on time or that some features were implemented incorrectly.
Improved alignment: Stakeholders and development teams are now better aligned through the use of regular collaborative workshops like sprint reviews. This creates a shared understanding and leads to greater commitment: Asking people about their perspectives and involving them in the process of making product decisions increases the likelihood that the individuals will support the decisions.
Motivated productive teams: Last but not least, self-organising development teams tend to be more motivated and productive compared to traditional ones, as they are able to determine themselves how much work can be done in a given period, decide who carries out a specific piece of work, and agree on how the team members collaborate.
Old and New Product Management Challenges
While agile methods have given us plenty of benefits, a number of product management challenges still persist. These were not caused by agile practices, as far as I can tell. Agile has made them more visible, though, and it has exacerbated some of them. Let’s look at the key challenges that remain.
Lack of empowerment: When coining the Scrum terms, Ken Schwaber and Jeff Sutherland chose the term product owner instead of product manager in order to emphasise the level of authority and empowerment a product person requires — especially in an agile context where collaboration is valued, and stakeholders and dev teams regularly contribute to product decisions.
Unfortunately, a lack of empowerment is still a common issue: Product people often don’t have the necessary authority to make strategic product decision, shape the strategy, and own the roadmap. Consequently, they are in danger of playing a tactical product role and being product backlog managers and user story writers rather than owning the product in its entirety and being able to maximise the value it creates.
Confusion about roles and responsibilities: Even 20 years after the Manifesto for Agile Software Development was conceived, agile roles like product owner and Scrum Master are not always clearly understood, let alone effectively applied. As indicated above, product owners are sometimes mistaken for product backlog managers and stakeholder pleasers instead of the individuals who are in charge of a product and whose decisions the entire organisation must respect, as the Scrum Guide puts it.
Lack of direct interaction with users and customers: Customer collaboration is one of the four values in the agile manifesto. Nevertheless, it’s not uncommon for me to meet product people who don’t have direct access to users and customers. Instead, they solely rely on quantitative data and feedback from sales, which makes it hard to empathise with the users and customers and to fully understand their needs. This, in turn, reduces the chances of offering a product that does a great job for its target audience and generates the desired business benefits.
Product discovery and product strategy poorly practiced: The most beautiful user stories are useless if it’s not clear who the users are and why they would want to use the product. Acquiring this information requires focused product discovery and strategy work. Unfortunately, product discovery and strategy aren’t always effectively practiced. This is not helped by the fact that agile approaches like Scrum do not offer any tools and techniques to help product owners understand if a product should be developed. Fortunately, a number of tools and techniques have emerged in recent years that help product people with their product discovery and strategy work including my product vision board and GO product roadmap.
Lack of sustainable pace: “The sponsors, developers, and users should be able to maintain a constant pace indefinitely,” states the Manifesto for Agile Software Development. Ironically, working at a healthy, sustainable pacecan be particularly challenging in an agile context: Product people have to regularly interact with development teams in order to update the product backlog, agree on sprint goals, answer questions, and provide feedback on product increments in addition to all the other product management duties. Being overworked, however, has a negative impact on our productivity and mental health. Luckily, there are a number of practical measures you can take to achieve sustainable pace. These include not covering other people jobs on a continued basis — like acting as a stand-in Scrum Master — and having the courage to decline stakeholder requests. Please see my article “Sustainable Pace in Product Management” for more guidance.
Looking into the Crystal Ball
So where do we go from here? How will product management change over the next 20 years, and which role will agile methods play — if any?
Without having a crystal ball at my disposal, I believe that the future of product management will be shaped by our ability to address the product management challenges mentioned above. I sincerely hope that in 20 years from now, empowerment will no longer be a significant issue, product roles will be better understood and more effectively applied, and product discovery and strategy will always be practiced by anybody who manages or owns a product. I also believe that agile methods will continue to enrich product management, alongside other influences that have emerged in the last few years including Lean Startup and business modelling.
While there is still plenty of work to be done, I am optimistic about the future of product management and agile.