Scrum fails in a lot of businesses. Most don’t realize it, and the rest accept it as an industry standard they can’t avoid. Here are three reasons why Scrum fails and how to fix it.

Scrum became an industry standard so long ago that I don’t even remember when it started. Still, I do remember thinking it was a response to a world of increasingly complicated IT projects.

Agile should be part of every education out there. It’s useful for more than just software development because it provides a framework for handling loosely defined projects, without having the overhead of having to map out everything you need to do immediately.

While I will advocate for being agile every chance I get, it’s essential to understand that agile is not for everyone. If it’s failing you, it might be a sign agile isn’t the right approach for your project or business.

An agile world

To ensure we’re on the same page, I want to go over the basic principles of what Agile is and when it’s a good idea to use it.

Agile started with the Agile Manifesto, which contains the twelve tenants of the Agile Doctrine. The purpose of the doctrine is to shift the way we think about software development away from fixed features, processes, and contracts, and to the idea of flexibility, interaction, and collaboration.

IT projects have become ever more complicated, and while we like to invent ways to make them simpler to code, the reality is that very few — if any — people on a project understand the project end to end. In an ever more specialized world of frontend, backend, cloud, architecture, and project management, it’s understandably hard to keep up with everything, so we hire specialists for each task.

With that comes an increased need for flexibility, interaction, and collaboration. Three words, if anything, have defined successful IT projects for the past two decades — if not more.

Scrum is a doctrine

“Scrum is an Agile doctrine for working with a loosely defined project without having to plan every step of that project out.”

Please re-read that quote one more time, and fix it in your mind.

Scrum consists of a number of ceremonies, which all have their place and purpose in the doctrine. Taking away any of them means the interaction between ceremonies becomes less stable and eventually breaks down entirely.

There is a simple way to know if this has happened. When taking on new projects, I always ask, “What agile method are you using?” And if the response goes something like “We use Scrum, but…” followed by an explanation of all the ceremonies or actions not taken. This is a red flag and clearly indicates that Scrum is failing a project or business.

Every ceremony has value, and taking one or more of them out will make Scrum fail. Take out the retrospective, and you won’t learn from your mistakes but be doomed to repeat them. Take out planning or backlog estimation sessions, and you will suffer from poorly defined tasks which lead to more tasks not finishing within a sprint or the backlog becoming full of entirely customer-facing tasks.

The fix to this problem is easy. Implement all ceremonies, and follow them like doctrine. Remember to include stakeholders — if possible — in as many of them as possible. This is an Agile process, and stakeholder feedback is paramount to success. After all, one of the central tenants of the agile manifesto is:

“Business people and developers must work together daily throughout the project.”

Fixed projects

The biggest reason Scrum is failing your business is this; Your projects are tightly defined, with a fixed delivery time and scope. The only reason you’re calling it agile is you don’t have to plan it all out in detail but can do it on the fly.

Anything fixed — such as scope, deadline, price, and team size — is deadly to any agile project. You lose the ability to adapt to changing requirements or needs of your stakeholders, and this is counter to another of the Agile manifesto tenants:

Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.

We can estimate only that one more sprint is needed or more than one sprint is needed. We cannot tell you that the project — or even a task — will be completed by a specific date. To do so would lock us into place and make us unable to respond to change.

Fixing this problem can be difficult. It requires a business to accept agile not only in the IT department but in the whole organization. Some businesses are unprepared for this and often try to do things in increments. But the reality is that if stakeholders are not agile, their projects won’t be either.

Being agile is not for everyone, and it’s OK. If parts of your business are not agile, sit down with them and discuss what being agile means. Start with the agile manifesto, and talk about the tenants individually. When everyone understands what agile means, you can start talking about what it means in a business context.

To me, the rewards outweigh agile’s uncertainty, but uncertainty can be deadly to many projects. If it is to your project, it can’t be agile, and you should choose a different approach that eliminates or minimizes uncertainty rather than creates it.

Unclear goal and purpose

This may seem counterintuitive to everything I’ve just told you, but having a clear goal or purpose with a project is very important. This doesn’t mean it can’t change or we need a fixed scope. These things are counter to being Agile.

A clear goal for a project could be to build a new BI tool for sales or perhaps a new CRM for marketing use. These may seem like project scopes, but you’ll notice that we have not defined a fixed scope but rather a flexible one that can be adapted to fit the needs and purposes of the business as the project goes on.

Bad examples of Scrum projects are ongoing maintenance projects or research and prototype projects. To avoid these projects running wild and emptying your budgets, they should be scoped for time. Maintenance projects specifically fit better into the Kanban method, another agile software development approach.

Research and prototyping can happen within a Scrum project. These are called spike tasks and are scoped for a specific amount of time to gather more information and create properly defined tasks for the backlog. Larger projects entirely focused on creating better-defined tasks — such as a series of workshops with stakeholders — can similarly be scoped for time and revisited at the end of the project. But these should not be treated like scrum projects.

Fixing a project with an unclear goal or scope can be done in two ways. Either you find a goal and purpose for the project and let that shape the sprints, or you don’t and convert the project into a Kanban or spike project. Changing your approach to an agile project midway can save the project, and not being afraid to adapt is part of being agile.

Cultists and Cults

I like to reread the short Wiki on Cargo Cults whenever I run into Scrum teams that either partly or entirely ignore vital elements of Scrum. I often refer to these Pacific cults of World War II, using them to illustrate the dangers of ignoring parts of Scrum.

“A cargo cult is a millenarian belief system in which adherents perform rituals which they believe will cause a more technologically advanced society to deliver goods.” — Cargo Cult

During WWII, the Japanese and Americans used Pacific islands as supply depots. On some islands, indigenous people had lived for centuries, largely without contact with the rest of the world. Upon seeing the large amounts of food, goods, and cargo dropping from the sky, the indigenous people believed these “new” people’s gods must be mighty to provide such a plentiful bounty.

In the aftermath of WWII, cults started appearing on the islands, worshipping entities with names like “John Frum” or “Tom Navy.” The cults believed these spiritual entities would also provide cargo to them if they followed the same “rituals” as the newcomers had.

They would light signal fires and build replica airplanes and airstrips, hoping that by pleasing their new gods, cargo would drop from the sky. The cults persist even into our times, with the Prince Phillip Movement being the most widely known.

When I tell the Cargo Cult Story, I usually get a few laughs before someone tries to explain why their process isn’t the same. Arguments like: “We don’t need a daily stand-up; our devs just talk to each other whenever.” Or: “We don’t do planning poker with the whole team because our lead estimates everything.”

When asked what effect they think it has on their process, many struggle to find words or can only come up with positive effects. When challenged on it, answers like: “This is how it was where I worked before, and it got results!” or: “It has always been this way, and it works!” often surface.

These arguments are the result of a failure of logic. Correlation is, after all, not causation.

Be truly agile

The Agile manifesto is still a valuable source of inspiration to those who aspire to its tenets and ideas. I am one of those people. To me, agile — and Scrum in particular — is about people and creating an environment where we can freely discuss, iterate, and produce solid software. To be agile is to understand how processes interact and how those interactions change when the process, the people, or the product changes.

Scrum is, at face value, a set of rituals and ceremonies. It’s a framework for improving your team and your product. How you use the framework is up to you, but understanding, what the rituals and ceremonies are there to do, is vital.

Otherwise, you’re like the native, lighting a signal fire on the beach, expecting the cargo to drop from the sky any moment now.