If you aren't looking at quality, then who is?
Bill Hoberecht -
An implicit responsibility of managers is to ensure the high quality of work performed by their team. In high maturity organizations, managers have direct visibility to the effectiveness of work process and quality of work products - they might even be required to report this information to their management or a centralized quality department. This article describes some common used methods for ensuring high quality of project artifacts and deliverables, and advocates the adoption of Peer Reviews as an effective quality assurance technique.
How does a manager know the quality of deliverables produced by their team?
Developers, project managers and other team members: How do you ensure that your work products are "good enough?"
- If your job involves producing a tangible artifact (e.g., software, memos, process descriptions, reports), you have an interesting situation: you are accountable for creating something, yet how do you ensure that your deliverable is suitable for its intended use?
- Suppose you are a manager of a team of people who produce tangible deliverables. You are probably responsible and accountable for the work of your team. How do you know that the work products of your team members are satisfactory?
Here's one aspect of the problem that points to a possibly much more serious difficulty: If a manager has little visibility to the quality of project artifacts, how does that manager know that the team is operating effectively and is producing an end product that has sufficient quality? In my responsibilities within a technical organization, I frequently review project documents (project plans, status reports, release plans, sprint plans). Not infrequently, I find some of these to be inadequate - they are incomplete, having some significant deficiency. For me, this naturally needs me to wonder "if those project artifacts that are visible to me are of low quality, then what do I know about the quality of the software that we produce?" I am almost fully reliant upon testing as the vehicle for assessing product quality. This is worrisome because an entire body of testing literature speaks to the difficulty of comprehensively and definitively assessing quality primarily through testing activities.
The answer is far more than simply hiring talented, qualified individuals who you feel will do a good job. Hoping isn't good enough either - assuming that your talented staff will naturally work with one another in the preparation & review of documents and software will likely result in disappointment. This article suggests that a team manager has an obligation to ensure the quality of artifacts and deliverables produced by the team they manage.
Various Approaches to Ensuring Quality of Artifacts and Work Products
A commonly held perspective is that Quality Assurance (QA) is synonymous with testing. A developer writes some code, tests it a bit, and passes it on to the QA team. This view, unfortunately, ignores the importance of building quality into the artifacts and work products that support project activities.
A broader view might be that QA must be inherent in most managerial and technical processes.
I've directed small, large and huge technology delivery teams. Looking back on how these teams (spanning over 10 companies in six countries) I've encountered some common approaches to handling quality assurance. Most have been ineffective in achieving the goal of producing quality artifacts and work products. One method has repeatedly proven to be a bona fide "best practice."
For the remainder of this article, I'm primarily focused on ensuring the quality of technical artifacts that are produced during the course of creating software. These artifacts include those you might associate with waterfall methods (e.g., requirements, project plans, status reports, design documents), as well as those associated with agile methods (e.g., user stories, acceptance criteria, release plans, burndown charts). All of this is surely applicable to software deliverables as well.
Here is the short list of the techniques that I've found in many different software development teams:
- Absence of quality assurance.
- Team members work independently, with no collaborative review of artifacts. The tacit assumption is that individuals will generally produce information that is good enough.
- Difficulties with this approach: Managers have no (or very little) visibility to the accuracy, completeness or usability of the documents and information used by the organization. Quite frequently, this lack of attention to quality assurance is accompanied by a steady stream of complaints about the low quality of the work products produced by other people - these complaints are generally directed to a member of management; rarely will a "complainer" approach the author with feedback or offers of assistance.
- Managers assume that QA is occurring.
- In this case, the manager assumes that team members are working together, with active, timely feedback provided by experts to authors. There are no formal processes, expectations or reporting on the quality of produced artifacts. As above, managers have absolutely no role in ensuring that the work of the team is of sufficient quality.
- The manager assures the quality of the team's work products.
- A team members creates their document or information and submits it to the manager for quality review and approval. Although the individual may work with others in creating the artifact, there is no quality review by team members of the completed work product.
- Difficulties with this approach: This technique puts all responsibility (for assuring the quality of project documents and information) onto the shoulders of the manager. This suggests that the manager is expected to have the knowledge and experience so they can assess if the information is complete and accurate - a tall order for any manager that has a team of individuals that all are submitting their work products for approval. This approach is flawed with the assumption that the manager "knows all" and clearly fails frequently when the manager just doesn't have sufficient time to perform their review tasks. This method does not scale well; as project teams grow in size, the existing managers are expected to perform an increasing volume of quality reviews.
- I've also found a particularly devious variant of this technique: a lazy or under-qualified team member creates a "draft" document/information that is clearly incomplete. Then, under the guise of requesting a "review" or "approval,", the burden of actually creating content is pushed to the manager. (An experience manager will return the document as "not ready for review/approval" while many other managers end up doing the work of their team member in creating the artifact).
- Distributed quality assurance - Peer Review.
- Authors fully create their work product and the organization has the expectation (and perhaps formal processes) for reviews by qualified peers for the purpose of assessing quality and identifying needed improvements. (a variant of this is "pair authoring," a la "pair programming" of eXtreme programming, which combines authoring and reviewing as a single integrated activity). Deliverables are submitted to managers for final approval only upon successful exit from the peer review process.
- Difficulties with this approach: This technique can be disruptive if it the team members are untrained in peer reviews. As well, if there is no clear expectation that peer reviews are conducted with participation by team members, it is easy for team members to claim "I'm too busy to participate in a peer review of your artifact."
I'm a proponent of Peer Reviews. This is perhaps the most effective method of utilizing the experience and expertise in an organization to build quality artifacts. It scales well, as they population of reviewers naturally expands as the size of the project team increases. It encourages individuals to produce their best and expose it to others for verification and suggestions. It creates an opportunity for the organization to measuring effectiveness (and cost) of quality assurance activities, and to understand the trends, over time, of quality indicators.
Managers: What is your method of Quality Assurance for Artifacts and Information?
If you are a manager of people here's a quick means of assessing your situation - how would you answer these questions:
- Am I responsible and accountable for the quality of the software, artifacts and information produced by the team members that report to me?
- Most managers are in place to hire qualified people, develop their skills to make them more valuable to the company, and utilize their skills to the best advantage of the company. That lays the foundation for being able to create quality work products, and it follows that the manager is responsible for ensuring ever-improving quality of work products produced by the team. Of course, if it isn't your responsibility, then who is responsible for the quality of work products of your team?
- What methods do I currently use to ensure that my team members are creating high quality software, artifacts and information?
- Glance at the four techniques listed in the prior section of this article - does one of them seem to best describe your situation? Is your team's method of assuring quality working well for you and ensuring high quality deliverables every time? If not, you probably should be looking into a method that accomplishes the goal of assuring quality, while also being a technique that is viable for your team - you'll want something that is compatible with your team's culture and has the necessary scalability for all of the artifacts produced by your team.
- What reports or other data is available to me today about the quality of my team's deliverables produced in the past few months?
- If you have this information available to you today, then congratulations! (you are a member of a select set of managers who are able to actively manage the quality of their team's work products). A manager who is accountable for work product quality probably needs to understand the measured quality of each deliverable, the trends over time of measured quality, and the cost of quality assurance activities. Absent this information it seems difficult, if not impossible, to know your performance in fulfilling your quality responsibilities.
If you are a manager, take a good look at your methods for ensuring the quality of work products produced by your team - the three questions (posed immediately above) are a good place to start this self-assessment. If your methods appear insufficient, assess your own willingness to introduce a method of assuring work product quality. If you are up for the challenge, I'd recommend starting your research by understanding the various flavors of peer reviews and the methods used to introduce peer reviews into an organization.