I often hear people talk about "waterfall" process software development. But a waterfall process doesn't really exist.
Well, there are software projects that meet the description of waterfall — the post-publication name give to the project structure described in Walter Royce's 1970 paper as the wrong way to develop software.
But there aren't any published methodologies, processes, books, tutorials, courses, tools, or certifications for waterfall. Because it isn't really a thing you should do.
But there are no such processes for waterfall. Take a look and search for yourself. If you do find one, let me know, because they don't appear to exist.
I am PRINCE2 Certified. Some non-software projects might be somewhat sequential, e.g. building a single house you need to build the walls first before the roof, right?
Well, that's not quite correct; you may need the frame, but not the actual walls. A usual project sequence might be something like: lay the foundation, then frame, then maybe the roof. Modern construction often then wraps the house, so that the internals fittings and exterior walls can be installed at the same time.
Each type of task activity (foundation, frame, roof, walls, etc) is only done once, due to the dependencies between them.
But even that is not always correct. On a recent house extension due to delays they couldn't get a certain part of the frame, a steel support for the portico corner. So they put in a temporary stand and continued building and finishing the roof, plumbing, electrical, and fit out. The frame was actually one of the last items completed!
Isn't that waterfall?
No, it isn't called waterfall. It is just called project management. Breaking down the project into a series of steps relevant to the particular project, whether it is constructing a building, organising a conference, or research a new medical drug.
These steps might be mostly sequential tasks of different types, like building a single villa, or be iterative, like building an olympic village with multiple buildings where you might repeat the same tasks multiple times. General project management frameworks (PRINCE2 or PMBOK) support both types, whichever is appropriate for the relevant project.
Sure, in some cases the steps might look vaguely like a waterfall, but then so does an iterative software project look exactly like a waterfall, where it goes from iteration 1 to 2 to 3, etc. Take a closer look at the first diagram, above — while it looks like a waterfall, it is actually a series of iterations.
Someone building a house, planning a conference, or developing a drug doesn't call what they are doing a waterfall process; they just call it project management. And it is the right type of project management, for the particular project (sometimes sequential, sometimes iterative).
There are also many myths and misconceptions, where I have had people claim particular methodologies are waterfall, but on further investigation they are not. For example Axelos, the owners of PRINCE2, specifically say PRINCE2 is not waterfall. Similar explanations have been made about the iterative nature of the Unified Process. I am still searching for an actual waterfall type process.
Non-software projects can also be iterative and incremental. e.g. if you were building an estate of multiple houses you might work in iterations, building one block of townhouses at a time. Or building a skyscraper you might build (at least parts of) one floor at a time, iterating up the building.
Building the estate with an iterative approach, one block at a time, allows you to start selling the first block early, earning value sooner, and providing the funding to complete the subsequent blocks.
It would be quite silly in these circumstance to require all of one type of activity (e.g. pouring foundations) to be completed for the entire estate before any of the next type of activity (frames) could be commenced.
It is likewise bad planning on a software project to require all implementation be completed before any testing is commenced. Some projects (whether software or not) simply make more sense to do in an iterative manner.
Even building a single house will have iterative parts, e.g. where the architect provides a drawing and maybe a 3D model, then you make changes, and they produce a new model, and so on, iterating until you are happy with the design. And sometimes people even make changes to the design after the build has started!
So then what is waterfall?
The term waterfall is only used in software development. It is used to name the incorrect arrangement of project steps as described by Royce as "risky and invites failure".
To use it for anything other than the incorrect way to manage software does not make sense. You can certainly say a particular software project plan has been structured with a waterfall approach (i.e. badly), but there is no such thing as a waterfall methodology or waterfall process.
Planning a single design step, then a single implementation step, then a single testing step at the end of a software project for anything but the simplest is "risky and invites failure". It is planning for a train wreck.
It would be like planning for the architect to present the plans for a house and have them accepted the first time, without any consideration for iterative revisions and changes. It would be bad project management.
You need to plan for multiple design-build-test cycles, preferably organised by product component (feature).
How to manage a software project
I am also a Certified Scrum Master (CSM), and Microsoft Certified Solutions Developer (MCSD) in their Application Lifecycle Management (ALM) products. The best practice way of applying project management to software development is to use an iterative and incremental processing, implementing a bit at a time, testing and deploying in each cycle (or PRINCE2 work package), to gain incremental benefit and allow functionality to be refined.
This has been the case since the 1960's early NASA space shuttle development, as documented by Craig Larman and Victor Basili in their 2003 article Iterative and Incremental Development: A Brief History, and continues so today.
Instead of having the planning steps as the different types of activities that are done (Requirements, Design, Implementation, Test, Deployment), planning and work packages should be based around the features and use cases required, from high level themes down to specific components. This, like all projects, will depend on the specific project being undertaken.
Using such a product based approach, where you decompose the product into sub-components to be delivered, using a product breakdown structure (PBS) is the recommended approach for modern project management methodologies such as PRINCE2.
Even if not iteration based, such a product based approach will still result in similar benefits, with task activities (Design, Build, Test, etc) repeated for each component of the product. Kanban is one such agile approach that is product breakdown focused, not iteration focused.
If you are using a formal iterative based approach (such as Scrum or Unified Process) then a generic project plan can consist of the iterations and (if using Unified Process) the phases.
A good diagram that shows the relationship between phases & iterations, and the different activity disciplines is this Unified Process overview.
This shows that even the first iteration, in the Initiation phase, includes all of the disciplines, from requirements through to test and deployment. Although the emphasis may vary (more design in earlier iterations; more deployment in later iterations), all iterations result in ready and tested product components.
Because iterative development also includes iterative refinement of requirements, you won't know all the specific detailed features at the start, although it would be a good idea to plan (and estimate) at a high level the themes and epics that will be covered.
As the project progresses, these high level components will be further broken down and refined, whether as part of Scrum backlog refinement sessions, or as a part of PRINCE2 detailed planning when executing a stage or work package.
Many iterative projects use time boxed iterations. To use a time box iteration in PRINCE2 , you would specify packages with 0% tolerance for time, but instead have an allowed tolerance for scope variance (if the scope variance is exceeded, the project management action is flagged).
Why would you do anything different?
I was quite lucky in my early career that I worked at a start up, just looked for what was known best practice in education and industry, and used iterative methodologies from the beginning.
The main processes we followed were based around the Microsoft Solutions Framework (MSF), with some influence from the Unified Process, and then eXtreme Programming and early Scrum.
It was only later that I came across other companies that had failures trying to do software projects without following best practice or any published methodology, and had ended up with projects like that described by Royce.
I've never really understood why, or found a source of where they got their process from, apart from vague references to Royce's diagram. I can only presume they never read the paragraph below ("risky and invites failure"), the diagrams on the next page, or the rest of the paper.
Non-software projects might have a sequential structure by task activity (but are not called "waterfall"); they may also have iterative structures (where tasks are repeated) if needed.
The nature of software projects is that they are best managed with an iterative process, based on a product (feature) breakdown; trying to break them down by sequential activity is just wrong and invites failure.
There is no published or documented software development process of the type "waterfall"; it is simply the post-publication name give to the incorrect way to organise a software project (by sequential task activity) in Royce's 1970 paper.
While individual projects maybe be structured this way, no actual published waterfall methodologies, processes, books, courses, tools, or certifications for software projects exist; and they never will, because who would ever publish "the wrong way to do things".
There are general project management methodologies, such as PRINCE2 and PMBOK, that support both iterative and waterfall style structures, whichever is best for the specific project. Using a waterfall approach when it should have been iterative (e.g. the housing estate example) is valid, but bad project management.
There are many agile and iterative project management methodologies (Scrum, UP, Kanban, etc), along with associated certifications and tools. They are usually software project focused, and require that an iterative approach (or product feature based) be used.
There are no waterfall project management methodologies.