Chalk X

Summary: 

• Visualizing how work gets done, the work product, or support systems can help teams improve their work. 
• Taking visualizations used by teams and adding management incentives and goals around them can hurt the ability of those teams not improve them. 
• Managers can use visualizations as an opportunity to ask questions and support their teams. 

Visualize the Work: 

The Good: 

In the Agile community, there is an adage, “Visualize Work.” This means visualizing where work is within the development process to help improve workflow. This works; teams share understanding, they can remove ambiguity in the workflow, and they can see (literally) ways to optimize it. There are many ways to visualize development workflow, from simple Post-it notes on the wall to large software packages like Jira, Rally, or ClickUp.

The Bad:

The danger with all of these tools is that once the workflow is visualized, there is a tendency to want to manage the metrics in the visualization and not the intent of why you started to do the work in the first place. Have you ever had one of these goals:

• You are pushing a particular release or milestone date for a new product. 
• You review metrics or a dashboard of the number of tasks, stories, or points a team has completed.
• There is a new organizational goal of increasing all team velocity.

Why is this bad? It is part of a common mistake: managing what you can see rather than what is essential. 

When we have data, we confuse numbers as having intrinsic value. We could count the number of chairs on the floor in an office, and pretty soon, everyone would want to have more or fewer chairs, depending on which way is deemed ‘important.’ We may even start to count the rate of change at which chairs are brought in and taken out and set goals based on that percentage. This may sound insane, but it is no more insane than the examples I listed before. 

 Let’s re-examine the above and highlight and see how the above goals miss the mark. 

• You are pushing a particular release or milestone date for a new product.
  • The schedule isn’t the most important goal. The feedback on what we are creating and whether we think it will make us a profit is. We can be done early or late, but if customers won’t use what we make, it doesn’t matter. Every once in a while, you may have a legal date, but more often than not, the date gets focused on because that is easy to see. User feedback on early versions of your product can be hard to quantify. The release date become an easy substitute for value. What is a more impactful statement, “We released two months early.” or “We created an extra 5 million dollars in revenue.” One of those statements is why you are in business and the other statement can be printed on a banner celebrating false success. 
• You review metrics or a dashboard of the number of tasks, stories, or points a team has completed.
  • How big is a task? Are all the tasks on your list the same size? If someone else had those tasks, would they finish them faster or slower than you? A common mistake in managing tasks is assuming they are the same unit of measure. They are all relative unless a machine is doing them. At most, you can ask why this one team is doing more or fewer tasks. The team may have changed, or they started grouping work differently. Neither tells you if that team is generating value.
• There is a new organizational goal of increasing all team velocity.
  • The point of scrum velocity is predictability, not acceleration… that’s why it is called velocity. The point of having a known velocity of a team is predictability, not to make the team go faster.

Software development visualization

The good: 

We visualize our code when developing software by putting it in software source control. These tools are GitHub, Subversion, CVS, and TFS, just to name a few I’ve used. These tools enable a whole team or sometimes the whole world to work on shared files that make up the source code of a software product. This is key for collaboration and for others to see opportunities for improvement. Sometimes, because it is a visualization, there is a temptation to start to manage it. 

The bad:

• Increase or counting the number of lines of code per developer.
  • This is mistaking coding with piecework. The best developer will use the fewest lines of code to accomplish their task. They want the software to be fast and run efficiently. Also, if you make this a goal, developers are smart and will game the system. They may even create software to write code no one needs. On second thought, they don’t have to. AI will do that for them; it is very good at overproduction. 
• 0 bugs created by unit tests.
  • Unit tests are responsible for testing individual components of code. Many software solutions are so big that changing one part of the code breaks the solution someplace else. This is normal, and you work this out in development. A goal like this is meant to instill quality, but it usually results in fewer and more fail-proof unit tests. The developer’s attitude quickly becomes, I will make tests that don’t fail. 
• 100% code coverage of tests.
  • Remember, we are here for value, not building tests. You want to create reliable code, but like everything, there is a diminishing return the more you refine something. When designs change, you must change not only the code but also all of the tests. This makes changes expensive, which is another way of saying this makes learning expensive. If your goal is to find the preciously scarce commodity that is user value, then anything you do to make that harder to find needs to be removed. 
• The number of check-ins to source control per developer.
  • This is a classic “chair example” simply because you can count it doesn’t give it meaning. One of the worst side effects of managing this visualization is that developers stop putting code in source control. One of the features all source control provides is a backup system for all the software source files. If developers avoid it, then the whole solution is at risk.

In this example, those who do the work start to focus on how to meet the management goals on metrics and not use the tools they best serve them. Once you try to manage the visualization, you can lose its value.

Operations 

The Good:

Generally, in any support system, the system takes requests/issues that come in and route them to the people who can address problems. This visualization allows you to see what issues to address. There may be common issues that highlight a bigger problem. Also, how to improve a system as you see issues that relate to one another. 

The Bad:

• 50% fewer issue tickets than last year.
  • What if our user base grew? If you have a 10% reduction in issues but users/transactions/accounts/files grew by 400%, you improved the system. This can be an impossible goal to achieve.
• Shorten your time to resolve all issues.
  • This is a great way to upset customers. Have you ever had a person try to get you off the phone as fast as possible? Another version of this metric is the number of calls handled by an employee. The hidden goal here is trying to increase an employee’s utilization, which has been proven over and over again to be rarely a good idea. It is especially bad if customer satisfaction is also a goal.
• Increase customer satisfaction scores to 4.5 or greater.
  • But if you reduced your tickets to only one per year and that one person didn’t like that you tried to get them off the phone quickly, then you’ll have a bad score. How can I increase my satisfaction if I am supposed to shorten my time with each customer?

Imagine you plan on going on vacation to relax. You pick a destination for your trip, you need to go travel 840km, and you plan to drive at most 8 hours. This means your average speed should be 105 km/h or around 65 mph to achieve your 8 hours goal. This is a good elementary/primary school math equation, and you set the next goal: I must average no less than 105km/h. 

Once you get on the road, you hit traffic and have to go slow for 2 hours; you start to get anxious, seeing your average speed is only 25 km/h.  At some point, you need to stop entirely for biological input/output needs; you make as fast a rest stop as you can as your anxiety rises. You do a quick calculation for the past 5 hours of your road trip, and the average speed has only been 30 km/h. You must finish in 3 hours, and you’ll need to average 230 km/h or 142 mph to hit your speed target.

Remember, the original point was to go on a vacation to relax. The goal switched and it became hitting speed and time metrics—to the ultimate absurdity of driving off a cliff at a mind-numbing 230 km/h. 

This happens all the time with business systems. We do a single calculation and set forth what we think is a good set of metrics in the abstract. We then incentivize people to hit or focus on that metric, and they proverbially drive the business off a cliff. 

Metrics cannot be considered in isolation; rather, they are a system of metrics. Meeting multiple metrics in combination can work against the ultimate goal.

But don’t all these software products have manager visualizations?

One reason is that software products for work management sell to companies, where they get feedback from managers who want ways to visualize data. Product owners build those features into the tool, making the tool worse for users and driving value. However, it can increase sales of their product because it has “management dashboards.” 

This is a good reminder to all product owners not to build what stakeholders want blindly. Management is a stakeholder in source control, process management, and operations, but they are not the primary users. The primary users are developers, product/process coaches, and operations teams. 

What should Managers do with Metrics?

If managers and leaders aren’t supposed to manage visualization, what should they do with them? 

Managers and leaders are there to ensure the company maximizes value. The value creators need their help; leaders have authority, after all. An agile mentor once told me, “The most powerful thing a leader can say is ‘How can I help?’” Visualization may help a leader understand who they need to ask, “Help me understand what is going on, and do you need any help?”

My recommendation is, when granted access to visualization, to do the following: 

1. Figure out when to ask questions. For example, metrics that are too consistent are usually not useful because they are irrelevant (e.g., the average pull of gravity on the team) or gamed by people in the system (e.g., our uptime is always 100%). 
2. We need to avoid poisoning it. Avoid put any mandates, targets, or incentives around the visualization of the work. When we do, it loses value and impacts it usefulness.
3. Cultivate and build the metrics into products so that we can see value in all its forms, including product usage, conversion rates, and how they correlate to product changes. Then, share these learnings with the teams and ask questions of customers.