In this article we review how to measure team performance with a sprint score. The primary measure of progress for a scrum team is “working software”, however, by using some basic empirical data, you can get a quick overview on how your teams are doing in general. Armed with this data, you can decide as a team what to improve next.
You need to be careful what you measure, or you could cause issues in your teams as they try to improve the wrong things when chasing a higher sprint score. Read on to find out more.
What is a sprint score?
Put simply, a sprint score is a single number that you can compare over a period of time to get an idea if your team is generally improving or not. Having a high score doesn’t mean we have the perfect team though, but it’s one indicator of team health.
How do you calculate it?
There are two component parts to my sprint scoring mechanism:
1) % complete score
i.e. Did we complete all the points we originally forecast to complete? Take the number of points the team forecast to complete at the start of the sprint, and compare it against the number of points they actually completed; then score it out of 10 using the following scale:
2) Cycle Time Score
You can calculate on average, how quickly each piece of work flowed through our development process from “not started” to “done”, then score it out of 10 using the following scale:
|Cycle Time (Days)||Score|
For example, if your average cycle time is 9 days, you didn’t quite get 8 days or less (you want a faster cycle time, so less days is better); you get a score of 3 points. If your average cycle time is 7 days, you made it under 8, but not quite 5, so you get a score of 5 points.
Simply add these two numbers together to arrive at a sprint score out of 20:
% complete score (/10) + cycle time score (/10)
= sprint score (/20)
Over time, you can track how a team is doing by looking at the trend of the sprint scores. Here’s an example from one of my teams, you can see that we had a bad sprint 19, but improved significantly after that, particularly by committing to the right amount of work, which boosted the green part (% complete score):
By targeting a higher % complete score, you encourage the teams to commit only to the work they think they can complete. By targeting a higher cycle time score (lower average cycle time); you encourage the teams to move tickets through the process quickly, removing any blockers and keeping story size small enough to allow an efficient process. These things should be encouraged to help you succeed.
Here is a working example of how I do this every single sprint. It takes me about 5-10 mins before sprint retrospective.
At the start of each sprint, I note down how many points the team forecasts that they can complete in a simple spreadsheet:
Calculating completed points
The number of completed points is easy to calculate. At the end of each sprint, I add up the story points of all the “done” items of work in the sprint, then calculate the percentage completion of the original forecast. With the % complete rate and the table above, I figure out how many points I should score this sprint. e.g. we planned 20 points, but completed only 10 points = 50% completion rate = score 3 (as we didn’t make it to 60%).
✋ Don’t add up points for partially completed work!
Only count points for completed items. There are no “partial credits” for partially completed work. At the end of a sprint, work is either “not done” or “done”. You only count points for “done” items.
Manually calculating the Cycle Time score
Cycle time is a little bit more complicated to calculate. I use JIRA’s built in control chart to do most of the hard work for me. But before we go there, here’s the manual method so that you understand how it’s done.
At the start of each sprint day (ideally in stand up), record the position of each story in your team’s development process this sprint.
|Sprint Backlog Item||Date||Position in process||Cycle Time|
|Item 1||Day 1||In Progress|
|Item 2||Day 1||Backlog|
|Item 3||Day 1||Backlog|
|Item 1||Day 2||In Progress|
|Item 2||Day 2||Backlog|
|Item 3||Day 2||Backlog|
|Item 1||Day 3||Done||2 days|
|Item 2||Day 3||In Progress|
|Item 3||Day 3||Backlog|
|Item 2||Day 4||Done||1 day|
|Item 3||Day 4||In Progress|
|Item 3||Day 5||In Progress|
|Item 3||Day 6||In Progress|
|Item 3||Day 7||In Progress|
|Item 3||Day 8||In Progress|
|Item 3||Day 9||In Progress|
|Item 3||Day 10||Done||6 days|
The cycle time for a piece of work is calculated from when it was started to when it was finished (not from when the story entered the sprint). I’m only interested in how quickly a team can convert a piece of work from not started to done, so I don’t include the time the work sat in backlog as well. If a team doesn’t start/complete a piece of work in a sprint, that is represented in the teams % complete score anyway.
By the end of the sprint, you should have the day work started on each item, and the day it was completed for each item. Using this data set, you can calculate the median average (I use median to prevent outliers massively skewing the result).
With the median average calculated, you can figure out what the cycle time score is for this sprint out of 10.
Using the above example, we have 3 items with cycle times of 1, 2 and 6 days. The median would be calculated as 2 in this example, and this would generate a sprint score of 9 out of 10.
How I use JIRA to extract the median average
If you are using JIRA to track your software projects, then you will find it automatically tracks the status of each ticket in realtime, so you don’t need to manually record each day a ticket moves status. You do need to keep your JIRA tickets up to date though 😉.
You need to be using a JIRA classic project to access the control chart report. At the time of writing, this report wasn’t available for next-gen projects.
JIRA generates the median average for your cycle time automatically, you just need to do a little bit of tweaking to the control chart report:
- When you close your team sprint, you can access the reports section and find the “Control Chart” report (highlighted in red below):
- When you click the control chart report, you are presented with the following chart:
- From this chart, you can see the “median” figure (4d 18h) on the top row of figures. This is a good start, but you need to tweak a few things to get it reporting correctly first.
- Underneath the chart you’ll see the “timeframe” section; make sure that’s set to your sprint start and end dates (I use “custom” and set the dates manually).
- In the “refine report” section, I also have a quick filter set up called “control chart”. This filter excludes anything that may be erroneously affecting the median of this sprint. For example, if you are using epics to group stories, then you may want to exclude epics using a quick filter on your teams JIRA board. My team delivers stories within epics, we don’t need to track both.
- You will also notice that my “columns” setting doesn’t include the backlog status (as discussed earlier, I don’t want to track this)
When you have the median value you are looking for, look it up against the sprint scoring table above to generate the correct cycle time score. Add that value to your % complete score, and you have your sprint score.
What you need to be careful of
“What is measured, is managed”Peter Drucker
I love this quote, if you start measuring something, then people will start to try and game the system to improve that thing. Here’s a few examples of things I wouldn’t include in my sprint score:
Number of tickets completed
Ofter referred to as “throughput”, sounds great at first glance right? If we get more tickets done, then we’ve done more work right?
But, what if I have a 10 line code change, I could ship my fix in one go, and complete one ticket. We’d have one set of code reviews, one set of manual tests (and hopefully a suite of automated tests), and we’d deploy that change live.
Or, I could do 10 x one line code changes so that my throughput looks better. The end result? The same outcome. But we just did 10 times more paperwork for the tickets, 10 times more code reviews and 10 times more manual testing, 10 times more deployments.
I’d argue that throughput is important to track, but shouldn’t be something to aim for on its own, and for that reason, it is not part of my sprint score.
Story points completed
More points means we’re doing better right? Well not exactly. There are a number of ways to improve the number of points you complete as a team each sprint. One is to do more work (good); another is to just increase your estimates in backlog refinement. The same software gets delivered though, which is why we measure outcomes not output.
If someone asks you to improve the story points your team are delivering, just ask your team to increase their estimates and you’ll get there in no time. This is why I don’t like using story points to measure team performance and why it doesn’t form part of my sprint score. My team estimates are for our sprint planning and long term forecasts only.
Average points per person per day
Using the total points we completed (see above as to why I think this is a bad idea); build on that by also dividing by the number of people in the team, and the number of days in the sprint. Now you know exactly how well each person is doing by measuring how many points they are individually completing. Except that, we are a team, working together, practicing concurrent engineering where we overlap work wherever possible… Do you still want to track average points per person per day? Who gets the credit, the designer, the developer? The tester? What about your poor scrum master? Maybe they are the architect of all this and deserve it. Save yourself the pain, don’t track this metric. Ever. We own success and failure as a team; and we measure team performance.
I hope you find this article useful. As I’ve mentioned in some of my other posts, I look after a few different teams, and by using the sprint score, I can see at a high level which teams need my help the most, by reviewing their sprint scores.
If you have any questions or comments, please leave them below.