In Scrum, the expectation is that the entire Scrum Team owns quality, but what does that really mean? Isn’t the Product Owner only worried about value? Doesn’t the team own all the testing? Don’t they really own quality? Like many other concepts in Agile the answer to who owns quality is subtler than that, and as my first Agile mentor liked to say, “It depends.”

Before we define ownership, let’s define what we mean by quality in Agile. In terms of Agile software development, Jim Highsmith in Agile Project Management: Creating Innovative Products identifies quality as two points of the Agile Triangle: Intrinsic quality and Extrinsic quality.

This suggests that the definition of quality comes from two different sources, externally from the customer and internally from the development teams.

Defining Quality

Joseph Kelada, author of Integrating Reengineering with Total Quality defines these two aspects of quality:

Intrinsic Quality is all of the qualities that are built into the product: suitability, durability, reliability, uniformity, and maintainability. This type of quality can be measured quantitatively such as test coverage, bugs per line of code, escaped defects, cohesion, etc.

Extrinsic Quality is the buyer’s perception of quality and the value to the customer. This is a more qualitative measurement based upon sales, usage and customer feedback.

When most people think of quality, they think if intrinsic quality, which is why we have team members who are Quality Assurance specialists. They aren’t evaluating the customer’s perception of the product, they are performing verification and validation (V&V), e.g. Are we building the system right? Are we building the right system? The goal of V&V is to test the product against the written business and technical requirements.

On Agile projects, we build products with the premise that every requirement ties back to a customer-facing vision and value proposition. If a requirement doesn’t align with that vision, then it doesn’t matter how reliable it is, for it is of no value. With each iteration and release, the goal is to provide the highest value features balanced against the cost to develop those features. This the goal of each Sprint in Scrum.

Given this, I would suggest that Jim Highsmith’s Agile triangle is visually inaccurate. It gives equal weight to intrinsic and extrinsic quality. The reality is that in order for organizations to compete, extrinsic quality, in most cases, is more important. Remember, the first principle in the Agile Manifesto, “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.”

The Role of Product Owner in Quality

I recently encountered a simple yet telling example of how a Product Owner addressed intrinsic versus extrinsic quality while facilitating a Sprint planning meeting. A developer submitted a backlog item to refactor some previously written code that was causing a large increase in storage each time a report was run. This was an unexpected outcome of a customer-facing requirement. The assumption from a technical perspective was that the storage issue was egregious and the impact was large, but the effort to refactor the code was large as well. In a traditional software development environment it would be assumed that a defect like this would automatically be addressed in the next release. On this high-performing Scrum Team, the Product Owner questioned the value of addressing the defect in this release.

This involved an in-depth discussion of the issue and the options, and ultimately in came down to a quantifiable measurement. The technical debt of that storage issue could be measured in dollars and could easily be compared against the business value of other requirements.  Once the issue was quantified, the Product Owner removed the requirement as a candidate for this release. The Product Owner had more “valuable” items slated for this release. Not all issues of extrinsic quality versus intrinsic quality can be that easily quantified, but in mature Agile teams, the correct dialogue will occur and the decision that is most often made will balance the short-term customer facing goals with the long-term viability of the product.

Quality Ownership

So, back to the original question, who owns quality? When speaking of Agile roles, it is easiest to use the Scrum roles. In the most recent annual VersionOne Agile survey, 75% of companies practicing Agile are using Scrum or at least the Scrum terminology. The roles of Product Owner, Scrum Master, and Development Team have become ubiquitous terms in our industry.

In the Scrum Guide, it states, “As Scrum Teams mature, it is expected that their definitions of “Done” will expand to include more stringent criteria for higher quality.” “Scrum Teams” includes all three Scrum roles. So this would presume that the entire Scrum Team owns the intrinsic quality. However, the Guide also states that the Product Owner is responsible for “Optimizing the value of the work the Development Team performs.”  So, ultimately, the Product Owner owns extrinsic quality, and as we just discussed, this may supersede intrinsic quality.

The Reality

Though organizations understand this theoretically, this is seldom put fully into practice. For the Product Owner to truly own quality the following best practices would need to be in place:

• Quality Assurance specialists would functionally report to the business
• The Product Owner sign-off on design documents
• Performance and non-functional requirements would be approved by the Product Owner
• All defects would be approved by the Product Owner before being added to the backlog.
• The scope of pre-production testing and readiness would be approved by the Product Owner

Those changes and many more would need to occur, both culturally and organizationally for an organization to truly align quality ownership with the Product Owner role.

At minimum, in order for the Product Owner to truly own value and quality, the Development Team needs to educate the Product Owner on the cost and effort of addressing intrinsic quality issues. The Product Owner needs to understand that well written code costs less to support and maintain. Technical debt eventually needs to be paid back and only an educated Product Owner can make the best decision as to when. This often creates conflicts within newly emerging Agile teams. In my experience, especially with large traditional organizations, this can shift the ownership of building the product away for the IT department and more to the business. This is intentional in Agile so that we ensure that delivering value to the customer is a primary goal.

Personally, I have seen too many well-built, highly reliable, stable products sit on a shelf because ultimately they missed customer’s expectations or were too late to market. Often a product that was not built half as well can dominate market share. There are exceptions to this of course, but for those companies at the forefront of their industries, they are adopting a Lean and Agile approach to software development that front-end loads value. They then add stability and reliability once the product achieves traction and validates the ROI for both the initial investment and continuing investment.

It is a team effort to understand how each requirement, whether technical or functional, will drive value to the customer. This type of understanding requires close collaboration between the Product Owner and the development team. It means truly embracing the “one team” concept and allowing full transparency into the decisions about which requirements make it into each release. Ultimately, it will be the Product Owner’s decision, but only in an environment that had truly embraced what it means to be Agile can this occur.

References:

Joseph Kelada, Integrating Reengineering with Total Quality

Jim Highsmith, Agile Project Management: Creating Innovative Products

10th Annual VersionOne Survey

Jeff Sutherland, Ken Schwaber 2016 Scrum Guide

About the Author: Dan Tousignant

Dan has been managing software development projects for 20 years. He was first introduced to Agile via a Scrum Implementation in 2000 and has since adopted Agile as the primary method for managing software development projects.

Dan holds a BS in Industrial Engineering from UMASS, Amherst and is a Professional Scrum Master, Certified Product Owner, PMI Agile Certified Practitioner and Certified Scrum Professional.

Dan Tousignant, PMP, CSP, PMI-ACP.

President Cape Project Management, Inc.

www.capeprojectmanagement.com

The Project Management Institute (PMI®) has proceeded down the path of embracing Agile. On March 26, 2018 both the PMP® and PMI-ACP® exams with be updated to reflect PMI's new Agile Practice Guide and the PMBOK® 6th edition. You can read about the changes here.

My Observations

PMI stated there were not any changes to the overall course outline for the PMI-ACP exam, but changes were made to "harmonize with terminology" used in the practice guide. After scouring the guide, I made a number of updates to my practice exam questions, training content and my self-study training.

Here are my takeaways after reviewing the guide in detail:

• Ultimately their goal is to become framework neutral so there is an attempt to create some common terminology and practices.
• There are now 3 generic Agile roles, taken primarily from Scrum:
  • Team Facilitator
  • Cross Functional Team Members
  • Product Owner
• There was recognition that there is no PM role in Agile but the PM can play the role of Team Facilitator
• They have stated the need for PMOs in an Agile environment
• There are now 2 generic Agile approaches:
  • Iteration-based Agile (essentially Scrum)
  • Flow-based Agile (essentially Kanban)
• Transparency, Inspection and Adaptation (from Scrum) are now considered to be generally Agile principles
• Stories (for Agile requirements) now seem to be accepted terminology. I don't think they were even referred to in the initial version of the exam
• I like how they now have a Venn diagram for Lean and Agile and how Kanban bridges both. That will become a popular training graphic
• Hybrid and scaling were now acknowledged, though minimally. I don't expect to see many more questions on this area yet- too much inconsistency in the marketplace.
• I think they have embraced the following pure Agile concepts even though they are the very opposite of traditional PM:
  • Servant leadership
  • Generalizing specialists
  • Burndown charts and Kanban boards instead of Gantt charts
• Even though there was a lot alignment with Scrum, there were still some contradictions:
  • "The Product Owner sees the demonstration and accepts or declines the stories" - that is a big faux-pas in formal Scrum. The Product Owner should be signing off throughout the iteration and showing the final product to stakeholders at the demonstration.
  • Iterations are usually 2 weeks - that is another false reality. I trained hundreds if not thousands of people over the years and the length of a Sprint is anywhere from a week to a month. If anything, three weeks seems to be as popular as two weeks.
  • The Product Owner asks a "triad"; a developer, tester and analyst, to get together to write a story as a way to refine the backlog. Hmmm that sounds dysfunctional to so many ways...

Summary

In addition to those terminology changes I mentioned, the guide provided a little more detail into some of the definitions of terms that were already in the exam outline. Though it is by no means perfect, it is a big step in developing a common terminology and set of practices called "Agile."  The exam is difficult to study for and difficult to train because there was, and still is, so many contradictory terms in the Agile community. Even within PMI's recommended reading list the authors contradict each other. The best part is that the Agile Practice Guide is a little more than 100 pages, as opposed to the nearly 1000 pages in the latest PMBOK. Let's hope it stays that way!

Remember, if you are studying for the exam I have the most popular FREE PMI-ACP study guide and other great resources for studying for the exam.