I used to play Texas Hold’em back when it was the popular thing to do. Like so many others I’d watched it on TV, and with the sudden availability of online poker, I was hooked. I quickly learned the difference between the gamblers and the players, and it was an obvious one. The application of math, strategy and knowledge. The winning players understood how to apply probability to the game, as well as how previous actions and games affected the players and the metagame. The gamblers often either didn’t understand or didn’t care. They made risky bets or calls, all in the hopes of a big payday. Once in a while it would happen, as the odds dictate, and they would take it as proof of their genius play.

I was part of a Scrum team - working on a particularly delayed project - when it occurred to me; Scrum teams who misapply planning poker - or maybe don’t even use it - are pretty much gambling. They’re gambling on developers being able to estimate and specify correctly, and on the requirements being fully covered. More accurately, usually, they’re gambling on a developer… singular. Often, Scrum teams rely on the Lead to make all the estimates. This is truly a risky bet.

How it usually works

In a few projects I’ve worked on, planning poker was used in some way. I’ve really only seen one instance of it used the way it was envisioned, but usually, there is a fixed deadline or a certain amount of hours that can be spent on the project. This is the reason most PMs fall back on estimating tasks based on hours, and this is how it usually works. Task X takes 3 hours, task Y takes 11 hours, for a total of 14 hours. On rare occasions, the estimate comes from developers who had a clearly defined specification, but more commonly it’s based on incomplete information. In the latter case, the developer will usually add a buffer, and the PM will usually do the same. It’s worth mentioning that fixed deadlines and scopes don’t mix well with Scrum, and usually leads to delays and unsatisfied customers. However, most clients I’ve worked for decide to try and contain these things within a buffer period, often at their own expense.

Estimated hours say nothing about complexity, and the complexity of a task often depends on the experience level of the developer. A task your lead says takes 3 hours, might be a complex task to a more junior developer, but involves known information to the lead and so can be done faster if he/she does it. A development team usually consists of developers of various experience levels, so what happens is: the task gets assigned to another developer, who struggles to finish the task within the allotted 3 hours + buffer (let’s call it 4 hours). This has two clear effects and a third less clear one. First, it clearly delays your project, and second, it means you will likely consistently have tasks you can’t finish inside the sprint, so they carry over to the next one. The third and less clear effect is stressing your developers. Most people like finishing things on time, and get stressed if they are delayed. Consider your own work for a moment. You are given a task, this task shouldn’t take you more than 3-4 hours to complete. 3-4 hours in, you’re still not done, and you see the approaching delivery deadline. Now consider having to tell whoever gave you the task you can’t make it for this deadline, and the possibility of having to explain why. It’s an uncomfortable situation, one I’d wager most developers experience on at least a weekly basis, and one that could have been avoided.

How planning poker works

The idea of planning poker - and story points - is to assign values to user stories based on the effort required to complete the task. The effort required is not the same as time, but rather a function of complexity, workload (amount), and any uncertainty or risk involved in the task. Just like Texas Hold’em, Agile is a game of incomplete information, and discussion of the user stories and tasks can make uncertainties clear, possibly lowering complexity. The process is pretty straightforward. The developers agree on a well-defined task with a certain complexity (I like using 5) beforehand, and this serves as a comparison for all future task estimations. The task can’t be trivial, but it also mustn’t be too complicated and, most importantly, the scope must be understood by everyone.

At planning meetings, all the developers are issued cards with numbers (usually 0.5 - 100, in an exponential sequence), and a user story is brought up and is read by everyone. Next, everyone plays a card face down (to avoid influencing each other), and when everyone has played a card, they’re flipped over. From here, the person with the highest number explains how they would solve it, and then the lowest number does the same. A discussion about the solutions and any possible alternatives might be needed. Then, everyone plays a new card face down, and when they are flipped over, the numbers should have averaged out. The team then decides on a story point value based on either the one most used, or a number they can agree on.

The whole point of abstracting tasks into story points is to get an idea of how much effort is needed to complete a sprint. Effort varies from person to person and sprint to sprint. What’s more, if you assign your most time-consuming tasks to your most senior developers, you’re not only bottlenecking your progress, you also might miss potentially complicated tasks, which could stall the rest of your team. This is why story points work. Not only does everyone participate in discussing and estimating stories - thereby sharing knowledge - you catch more complicated stories, because they suddenly become visible to the whole team.

Are we there yet?

So when are you done with a project? In agile, this question is answered with: “When there are no more story points in the project”, but few people understand what that actually means. Effective time and velocity are two terms we need to understand before we can move on. Effective time is the amount of time you can actually focus on working on the project, and this time is never the same as a full working day. As a rule of thumb, developers usually spend 25-40% of the day doing things that are not directly related to working on the project. This time is spent on administrative work, dependency wait time, task coordination, meetings and breaks. The rest of the time is considered effective time. Velocity is the speed at which a story point is being resolved. This can be calculated as the amount of effective time divided by story points. This number is a moving average though, and can only be closely approximated over multiple sprints.

Burndown chart by Atlassian

Let’s say you have 4 developers working 8 hours a day at 75% effective time. This gives the team 4x6x5 hours a week, making it 120 hours per week, or 240 hours for a 2-week sprint. If your team resolves 60 story points in that sprint, this means your team took 4 hours per story point on average for that sprint. This gives you a guide for the next sprint, but when you have the numbers for 3-4 sprints you should begin to see a pattern and get a better approximation. The burndown chart is a good way of keeping track of how a single sprint is going, and by overlaying multiple sprints you will likely see patterns emerge. Is your team finishing early? Are there big drops in the line? These are both symptoms of possible issues with the sprint. It could be your team is under-committing, or maybe the assigned story point estimates are simply off for some tasks. Bring this information into your retrospective, and get the team’s feedback.

Stop gambling with Scrum

Taking chances is something we all do, but most of us prefer taking calculated risks. Gambling is not a calculated risk, it’s taking a risk far too great for gain that’s far too small by comparison.

I don’t see much potential gain in skipping planning poker when using Scrum, but I see plenty of risks. Delayed projects, scope creep, angry or disappointed stakeholders/clients, and last - but not least - frustrated and stressed developers. The potential gain from taking that risk, you ask? A project delivered on time, possibly slightly faster. Measured against the risk, I would call that gambling. Wouldn’t you?