Agile rebooted

Dave Waller charts the progression of agile from its birth 21 years ago in a Utah ski lodge to its current maturity as an iterative project life cycle. Where might it be headed next? And what will this mean for the world of projects?

Last year, when Duena Blomstrom went to get an ‘agile’ tattoo on her forearm, her London‑based tattooist wasn’t convinced. Blomstrom’s proposed design featured lots of tickets under the headings ‘to do’ and ‘done’, but only two under ‘doing’. The artist thought it looked unbalanced. “I had to spend an hour explaining the whole concept of agility and work in progress,” says Blomstrom, co‑founder of software start‑up PeopleNotTech and author of People Before Tech: The importance of psychological safety and teamwork in the digital age, who points out that a key principle of agile is to keep the number of active tasks to a minimum.

“In the end she agreed that just a few tickets under ‘doing’ is better than many. That was a lesson for me: even a tattoo artist can appreciate the principles of agile eventually,” she explains. What may be harder for the reader to appreciate is that someone would want to get a project management methodology tattooed onto their arm in the first place. Blomstrom calls it an “agile fetish”. She’s not alone.

The early days

It’s now over 21 years since a group of organisational architects met at a ski lodge in Utah, frustrated that top‑down, documentation‑heavy processes were impeding their software work, and came up with the Agile Manifesto (see box). At its core, agile is about maintaining a fixation on the customer and delivering them increments of value – fast. This is achieved by breaking project work down into defined sprints and allowing teams to self‑organise, harnessing a process of estimation, tracking and continuous improvement.

“There’s no process that can’t be done with agile at the core,” says Blomstrom. Indeed, the iterative life cycle of agile has since been adapted and applied as a project management methodology everywhere from marketing to aviation, often by organisations seeking a better response to a turbulent and uncertain global marketplace. Yet this widespread adoption has also led to dilution. Many organisations have taken on the concept, often at great expense, without truly understanding it: leading to clashes with project professionals’ need for bigger‑picture clarity and thoroughness, and passing off traditional linear life cycle ‘waterfall’ habits, such as asking for clearance before starting new tasks, as agile.

“Companies so badly want to be agile, but when you really look under the hood, many aren’t exhibiting those core principles,” says John Carter, a Silicon Valley‑based consultant whose past clients include Apple and Amazon. “Someone gets some agile training and adopts two‑week work schedules in their team, and leadership sees that and tells all their teams to do the same. But they’re not self‑organising teams. They’re not doing customer demos, estimation or so many things that really define agile. The fall‑out is they can’t deliver on time, and they don’t delight their customers.”

More than two pizzas

To those companies, it may simply seem like agile itself isn’t working. And in some respects, even when it’s applied as intended, it isn’t. For example, the leading players in Silicon Valley are still wrestling with the question of how to get agile to scale. Amazon CEO Jeff Bezos has described the ultimate agile unit as a “two pizza” team. Meaning every scrum team, one of the basic building blocks of agile, should be able to be fed with two pizzas. Carter, who has worked with Mozilla, the company behind the Firefox browser, points out that it takes 1,000 people to write code for a browser. By Bezos’s measure, that’s well over 100 scrum teams: a lot of pizzas, and a huge amount of coordination, which opens the door to some very un‑agile complexity.

“When scrum teams have to coordinate, they get hit with dependency,” explains Carter. “One team depends on another to get a certain task done. But agile is not a milestone‑driven technique; it’s interval‑driven – you ship whatever is done at the end of each interval. So if one team starts carrying things over to the next sprint, the dependent team may be dead in the water because it needs that other team’s product in order to work.”

Back in 2011, Spotify published a framework designed to tackle this very issue. Known since as the Spotify model, it added terms like Squads, Tribes, Chapters and Guilds to the agile lexicon, and put a greater emphasis on teams’ autonomy. And it was duly lauded by countless imitators around the world. Yet the Spotify model should serve as a cautionary chapter in this broader story of agile adoption: its authors have since revealed it was never intended as a published framework for other companies to copy. And it’s been criticised for assuming people are competent collaborators and failing to strike the right balance between autonomy and alignment.

In fact, according to Joakim Sunden, an agile coach who worked at Spotify from 2011 to 2017, Spotify didn’t even use its own model. “Even at the time we wrote it, we weren’t doing it,” he has said. “It was part ambition, part approximation. People have really struggled to copy something that didn’t really exist.”

How to make agile work

There have been other efforts. Carter describes the IBM‑originated Disciplined Agile framework as “the most clunky, heavyweight, process‑rich disaster I’ve ever seen”. But it helped spawn the industry’s current standard, the Scaled Agile Framework for the Enterprise (SAFe). And in 2019, Atlassian, the company behind agile project management tools like Jira and Trello, rolled out Jira Align, a cloud‑based platform designed to “connect business strategy to customer outcomes at enterprise scale”. Yet the task of figuring out how to make agile work on a bigger scale remains very much an active ticket.

Back in the land of project management, Adrian Pyne, author of Agile Beyond IT, has had a front‑row seat for agile’s journey beyond the tech world. In 2008, he was brought in to troubleshoot at an insurance company that had introduced scrum for project management and was struggling. “I looked at them and wondered why they were using a hammer to put a screw into a wall,” he says. “It’s the wrong tool. Scrum is software development. Project management is project management. And scrum does not contain about 80% of what project management does.”

Tipping points

Pyne’s initial observation sparked a period of deeper investigation. The more he talked to people, the more he realised that there was “this massive misunderstanding about adapting agile practices into project management”. Yet he acknowledges there’s plenty of reason why these companies should try. As everything from technology to regulation and consumer demand changes fast, companies have to be able to respond and adapt quickly, and innovate more quickly than their competition. Hence agile remains the de facto framework in Silicon Valley.

A 2021 report by PA Consulting, The Evolution of the Agile Organisation, found that the top 10% of financial performers were 30% more agile than the competition. Companies are more likely to be successful if they can remain close to their customers and adapt to changing conditions. “In many sectors, companies are going to reach certain tipping points where they either move towards being agile organisations or they simply won’t survive,” says Pyne.

Amazon is one Silicon Valley company that’s given a lot of thought to survival. In a famous letter to Amazon shareholders in 2017, CEO Bezos wrote: “Day 2 is stasis. Followed by irrelevance. Followed by excruciating, painful decline. Followed by death. And that is why it is always Day 1.”

Refining agile

Refining agile remains a key factor in maintaining a start‑up mentality. Organisational network analysis is used to audit Amazon’s team structures, examining how communications, information and decisions flow through its informal employee relationships, helping to avert bottlenecks in decision‑making and other issues. Amazon also employs a Working Backwards process, where teams write press releases, FAQs and product manuals for products that haven’t yet been approved, to test and improve their quality and customer fit.

Carter reports how his clients too are honing their use of agile by ensuring they retain a laser focus on their customer – to ensure products and services are helping them get their most important jobs done. “That’s a very powerful way to take agile principles that maybe aren’t fully working and put them in the right space, getting greater customer feedback throughout development,” he says. “That will improve your course.”

Another powerful development lies in DevOps, an extension of agile that takes its innovations in development and applies them to operations processes. Yet like agile, DevOps struggles with larger tasks, so its use in project management necessitates a combination with more traditional linear life cycle methods to achieve the necessary oversight and governance.

For Blomstrom, meanwhile, Silicon Valley’s approach to agile now sees organisations putting their own people at the centre of everything they do. This includes investing time in concepts like psychological safety, which, says Blomstrom, is “crucial for any type of agile enterprise”.

“The type of extreme collaboration seen today requires the team to feel safe to take interpersonal risks with each other,” says Blomstrom, “to tell each other when they think something’s wrong and change direction, and to go together as fast as possible as a unit, often not doing the work they were hired for. I’m a big believer in the fact that agile is the only way to go. But teams can’t be high performing without this healthy dynamic, based on psychological safety.”

Where next?

It’s unlikely that Blomstrom will be getting a cover‑up for her agile tattoo any time soon. Silicon Valley is still in thrall to its principles, and its advance beyond the tech sector is sure to continue, as change in everything from tech to supply chains and regulations keeps getting faster. The future of projects means adopting agile right: changing the organisational culture and mindset, keeping the customer front and centre, and ensuring teams are set up psychologically to handle the rapidly changing demands of work. And that’s plenty to keep project professionals’ minds focused.

“Agile is something that keeps evolving,” says Pyne. “So it’s pointless wondering what’s next after agile. You’d just as well ask what’s next after project management. Let’s do it properly first, and then take it where it goes.”

The Agile Manifesto (2001): Six key principles

1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
2. Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference for the shorter timescale.
4. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
5. Agile processes promote sustainable development. The sponsors, developers and users should be able to maintain a constant pace indefinitely.
6. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behaviour accordingly.

To agility and beyond: Next steps for future-proofing your projects

1. Know where you’re struggling. Get feedback from every team member on the problems caused by company culture, process and structure – whether that’s poor communication between teams, or leadership’s insistence on using slow waterfall techniques.
2. Take the parts that work for you. Most projects combine agile with other methodologies, so you don’t need to strip down your organisational chart and rebuild it completely to fit the agile model.
3. Adjusting your way of working isn’t enough. Agile is a mindset, so people’s mentality has to change to embrace the new working methods too.
4. Run an audit of your teams’ psychological safety. Are your people fearful? Are they empowered? As work gets faster and more unpredictable, psychological safety becomes a key condition of successful agile teams.
5. It’s about delighting your customer. Project management is all about delivering value. So is agile. And that value comes through ensuring your customer, and their needs, are guiding decisions.

THIS ARTICLE IS BROUGHT TO YOU FROM THE AUTUMN 2022 ISSUE OF PROJECT JOURNAL, WHICH IS FREE FOR APM MEMBERS.