AgileFall – When Waterfall Sneaks Back Into Agile

This article previously appeared in the Harvard Business Review

AgileFall is an ironic term for program management where you try to be agile and lean, but you keep using waterfall development techniques. It often produces a result that’s like combining a floor wax and dessert topping.

I just sat through my a project management meeting where I saw AgileFall happen first-hand. The good news is that a few tweaks in process got us back on track.

I just spent half a day with Henrich, the head of product of a Fortune 10 company. We’re helping them convert one of the critical product lines inside an existing division from a traditional waterfall project management process into Lean.

Henrich is smart, innovative and motivated. His company is facing disruption from new competitors. He realized that traditional Waterfall development wasn’t appropriate when the problem and solution had many unknowns.a

This product line has 15 project managers overseeing 60 projects. Over the last few months we’ve helped him inject the basic tenets of Lean into these projects. All are Horizon 1 or 2 projects – creating new features for existing products targeting existing customers or repurposing existing products, tools, or techniques to new customers. Teams are now creating minimum viable products (MVPs), getting out of the building to actually talk with users and stakeholders, and have permission to pivot, etc.  All good Lean basics.

AgileFall in Real Life
But in our latest meeting I realized Heinrich was still managing his project managers using a Waterfall process. Teams only checked in – wait for it – every three months in a formal schedule review. I listened as Henrich mentioned that the teams complained about the volume of  paperwork he makes them fill out for these quarterly reviews. And he was unhappy with the quality of the reports because he felt most teams wrote the reports the night before the review. How, he asked, could he get even more measures of performance and timely reporting from the project managers?

At first glance I thought, what could be bad about more data? Isn’t that what we want – evidence-based decisions? I was about to get sucked down the seductive path of suggesting even more measures of effectiveness when I realized Henrich still had a process where success was measured by reports, not outcomes. It was the same reporting process used to measure projects that used linear, step-by-step Waterfall.

(To be fair to Henrich, his product team is a Lean island in a company where Waterfall still dominates. While his groups has changed the mindset and cadence of the organization, the folks he reports up to don’t yet get Agile/Lean learning and outcomes. They just want to see the paperwork.)

In both managing down and up we needed a very different project management mindset.

Lean Project Management Philosophy
So our discussion was fun. As the conversation progressed, we agreed about the ways to manage projects using a few operating principles of Lean/Agile project management (without ever mentioning the words Lean or Agile.)

  1. It was the individuals who were creating the value (finding solution/mission ft) not the processes and reports
  2. However, the process and reports were still essential to management above him.
  3. Having the teams build incremental and iterative MVPs was more important than obsessing about early documentation/reporting
  4. Allow teams to pivot to what they learned in customer discovery rather than blindly follow a plan they sold you on day one
  5. Progress to outcomes (solution/mission ft) is non-linear and not all teams progress at the same rate

Pivoting the Process
With not too much convincing, Henrich agreed that rather than quarterly reviews, the leadership team would to talk to 4-5 project managers every week, looking at 16-20 projects. This meant the interaction cycle – while still long – would go from the current three months to at least once a month.

More importantly we decided that he would focus these conversations on outcomes rather than reports. There would be more verbal communication and a lot less paper. The reviews would be about frequent delivery, incremental development and how leadership could remove obstacles. And Henrich’s team would continue to share progress reporting across the teams so they could learn from each other.

In sum, the radical idea for Henrich was tha this role was not to push the paperwork down. It was to push an outcome orientation down, and then translate its progress back up the chain. While this was great for the teams, it put the onus on Henrich to report progress back up to his leadership in a way they wanted to see it.

I knew the lightbulb had fully gone on when near the end of the meeting Henrich asked his team, “Going forward, who are the right team members to manage a Lean process versus a Waterfall?” And I smiled when they concluded that Lean could be managed by fewer people who could operate in a chaotic learning environment, versus a process-driven, execution one.

I can’t wait for the next meeting.

Lessons Learned

  • AgileFall is a seductive trap – using some Lean processes but retaining the onerous parts of Waterfall
  • The goal of leadership in Lean product management is not to push the paperwork down. It’s to push an outcome orientation down and translate its progress back up the chain

7 Responses

  1. These oppressive documentation practices are very prevalent with a lot of companies that utilize the traditional phased-and-gated process (a.k.a. waterfall). There is a document required for every phase that ties up valuable team resources. I especially like the idea of recording outcomes, which may be a significant mind-shift for management that has been entrenched in these practices over the years.
    I propose management by exception for product development teams. Management determines clear parameters (resources, budget, time, etc.) of an experiment. The team is then able to work without interruptions unless the experiment goes beyond the set parameters. In this setting, outcomes are especially important.

  2. Waterfall as characterized by some amounts to mismanagement. In this light I agree with your article. Collaboration in an iterative development cycle or an sdlc project cycle includes coordination of efforts and regular check-in, as in weekly and daily as teams near their shared goals with testing and implementation And as with lean, scrum, iterative, and other development and project management methods communications are key to coordination.
    After a long career, waterfall has signified poor management, inadequate teamwork, and misinterpretation of methods.

  3. What if instead of meetings, they could go on a dashboard and get the outcomes in a centralized dashboard. And this data wouldn’t be just limited to Henrich/Executive Leadership/PMs, but everyone on the value delivery chain including PMM, Marketing, Documentation, Support, etc? Let me know if you’re curious.

  4. Meetings are good idea, same time evaluating the methods by doing a survey would help more and give them a great outcome. Just a thought would may be helpful..

  5. I hope here Agile approach does not get mistaken to producing less or no documentation. This would be too myopic IMO. These “documentation” are still required for maintenance of the system after it had gone “live”. And it could be 5-15 years, where this system might be continually iteractively improved. This is part of the quality of good delivery.

    In product development, there is not denying on the benefits to using Agile/Scrum/DevOps frameworks. But for commercial project implementations, AgileFall is not all bad. Bcos the full scope is often known upfront (though there will always be haggling for changes and scope creep, and this can be contained with good PM skills) & contractual binding so we are expected to deliver on the scope, time, costs and quality. By default the project already srarted with Waterfall approach, with the known scope, and likely the design will follow with the given scope.

    There is no stopping the team from bringing in stakeholder to participate closely in the project at any phase in the project. But for commercial ones, this should be employed with care.

    From development onwards, we can switch to Agile. But my personal experiences Agile team better be using automation as much as possible otherwise it had too much overhead to be handled manually, and it can end up worse than Waterfall.

    Teams using Agile (& esp DevOps) shld have a somewhat complete suite of ALM tools like tools for project mgt, team collaboration, requirements mgt,design, development, build, release, test, issues/defects, deployment.

    My point is that taking the best of different approaches is a good thing if the situation calls for it to be more efficient.

    A lot of software engineering frameworks are still not addressing commercial projects well.

  6. Thank you – great write up. Can you share a more readable version of the second graphic? It’s so blurry that I can’t quite make out a lot of even the headline text. It looks interesting.

  7. Not exactly agile, perhaps, but I thought this technique worth passing along:

Leave a Reply