The Origins of Waterfall Project Management
In August of 1970, a computer scientist by the name of Winston Royce wrote an article for IEEE entitled, “Managing the Development of Large Software Systems” defining the following phases:
Waterfall Method c. 1970
Royce’s system would later be christened the ‘Waterfall Method’ by Bell & Thayer in their 1976 conference paper, “Software Requirements: Are they Really the Problem?” And thus a classic project management methodology was born. What is ironic is that Royce actually gives a number of warnings about this method in his original piece, stating that it is, “risky and invites failure”. He proceeds to recommend “iterative relationships” between phases and prototyping (what he terms a “pilot model” or “early simulation”) of products, as well as involving the customer early on – all concepts that sound much closer to an Agile approach than that of Waterfall (more on that in a minute).
Countless people and organizations have latched on to the Waterfall concept since its inception, many of which continue to use it today. Project Managers lean toward this approach because it provides control around a project (with the addition of phase gates, etc.) and ensures all phases are taken into account. It is a simple, step-by-step approach to project management; In fact, both PMI’s PMBOK Guide and PRINCE2, the leading framework and methodology for Project Management worldwide, reference the concept of project phases.
Despite the popularity of the Waterfall Method, its rigid approach to project management can create problems. The assumption that teams will know everything up front, for instance, or get everything right the first time, are both incorporated into the Waterfall structure, setting unrealistic expectations for any project group, and allow for very little flexibility to address these inevitable issues. It also assumes that nothing changes (stakeholder needs, technology requirements, etc.), which grows progressively unrealistic as technology continues to change with increasing speed. These weak points in the Waterfall methodology often have the potential to cause substantial rework and “scope creep”, among other problems. And yet, this methodology continues to guide how many organizations get work done – much to the disappointment of our teams, our customers, and ourselves.
The Origins of Agile Project Management
Agile concepts originated as early as the 1600s with the creation of the scientific method, wherein a scientist defines a hypothesis, tests that hypothesis, studies the results, and draws conclusions based those findings.
It was not until the 1930s, however, with Walter Sherwart’s “Plan-Do-Study-Act” (PDSA) iteration of the scientific method, that a product quality approach was applied. Sherwart’s PDSA approach was an iterative one, which promoted inspection, adaptation, and continual, incremental improvements over time. It was then W. Edwards Deming who introduced the PDSA cycle (later called Plan-Do-Check-Act) to Japan in the 1950s, a move that dramatically improved the quality of Japanese products and gave rise to the Total Quality Management (TQM) movement.
Copyright 2015, The W. Edwards Deming Institute®
Fast-forward to 1986 when Hirotaka Tekeuchi and Ikujiro Nonaka published an article in the Harvard Business Review entitled, “New New Product Development Game”. In it, they describe how organizations require a rapid and flexible development strategy to meet changing product demands from a highly competitive market. Tekeuchi and Nonaka clearly articulate having an integrated and iterative approach to new product development in their work, and were the first to use the word “Scrum” – a term originating from the sport of rugby, wherein the ball is passed within a team as it moves as a unit up the field – to describe this new approach. Read Tekeuchi & Nonaka’s original piece here.
Far from Japan, Scrum’s next major milestone was achieved in the Wasatch Mountains of Utah, nearly 15 years after Tekeuichi & Nonaka’s missive. In February of 2001, a group of software and project experts from a variety of disciplines (extreme programming, Scrum, etc.) gathered at a remote ski lodge to discuss what all of their successful projects had in common. This discussion, and new perspective of what had been working consistently across disciplines, resulted in the Agile Manifesto, a simple statement of their core values. The manifesto in its entirety reads as follows:
We are uncovering better ways of developing software by doing it and helping others do it.
Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.
While a host of organizations have incorporated the above values into their development strategies, many have a tendency to overemphasize those items on the right side of each of the above statements, while neglecting those on the left. Customers want, need, and expect the items on the left, and the organizations that get it right are those that maintain a healthy balance of both sides.
Having a plan, for example, is important for providing direction and anticipating risks, but a plan cannot be so rigid that it does not accommodate the inevitability of change. Or documentation, while absolutely critical for maintaining and improving products, is moot if the product does not work. And even though we are delivering products, it is the people – the cohesiveness and commitment of a team, their frequent communication and incorporation of customer feedback – matters tremendously and, many times, determines whether or not we are able to effectively deliver a great product.
To supplement the Agile Manifesto, its creators developed 12 principles meant to guide execution of the philosophy. Since that time, a number of methods that adhere to these core values and principles have cropped up under the umbrella of Agile – the most common, of course, being Scrum (more on that in my next blog post).
Agile methods have changed the role of the traditional Project Manager and the entire Project Management industry as a result of their overwhelming adoption and popularity across industries. A few years ago, for instance, PMI developed an Agile extension to the PMBOK Guide and a certification dedicated solely to Agile concepts, called the PMI-ACP credential. In kind, the creators of PRINCE2 in the UK have begun incorporating DSDM (Driving Strategy Delivering More) concepts into their methodology.
Major shifts toward the Agile framework by industry leaders such as these have opened the eyes of Project Managers worldwide. The industry is quickly learning that, with Agile tools and techniques, Project Managers are better equipped to lead teams that deliver business value quickly and provide customers with quality products that meet their needs. With this trend, the Agile community is sure to bring further change to our industry in the future – and I very much look forward to watching them take shape.