Dr. Patrick Desbrow, Ed.D.-CIO & VP of Engineering, CrownPeak
The Power of Measuring
As Technology managers, we have long been drawn to using metrics to measure the effectiveness of our projects and routine operational procedures. Metrics can also be one of the best ways to declare success for your major initiatives. Why should we always be looking for interesting metrics that can be measured? Because this can make technology management a bit easier.
However, not all metrics are the same. Some metrics are strategic (high level) while others are tactical (low level). The difference can be very important. Tactical metrics that identify speed, quality, and accuracy are important and necessary as they capture the details of projects and processes. Strategic metrics are used to translate sometimes ambiguous business objectives into tangible non subjective values and can be a powerful tool for decision making. A company’s leadership team is often tasked with translating the organization’s vision, mission statement, and business objectives into these strategic metrics. Each of the department leaders are then charged with connecting the most appropriate tactical metrics for their respective groups. The organization is said to be “in alignment” if this process is done correctly.
Measuring Technology Leadership
The Technology Leader usually is responsible for the majority of the IT systems in the organization. Most of these systems have numerous tactical metrics that the leader can use to measure success in their group. The Key Performance Indicators (KPI) are the most appropriate, relevant, and important of these many measurements. Finding these KPIs can be a real challenge for the team of managers in the IT department. Here are some suggestions to help the process:
1. Make sure the KPIs really lines up with the business objects
2. Select KPIs that create the desired behavior of the team members doing in the work
3. Capture KPIs continuously through the tools and not as a manual process
4. Understand what is a “good” versus “bad” values for each metric
5. Know the steps to move the KPIs in the appropriate direction
6. Record the current KPI levels before making any changes. This is your baseline
7. Create a dashboard that shows the KPIs side by side. This sometimes is called a Balanced Scorecard.
8. Look at all of the KPIs together to see what overall picture is being created
9. Confirm that the KPIs do not contradict each other or lead to other negative behaviors
10. Do not create too many KPIs, but watch out for blind spots which you are not currently measuring
Tactical metrics that identify speed, quality, and accuracy are important and necessary as they capture the details of projects and processes
Using SMART Goals
Let’s put these suggestions to the test with an example. We can use SMART goals to create a connection between the business objective, the strategic goal, and the tactile measurements. SMART goals was a term introduced by George Doran in 1981. SMART is an acronym of the words: Specific, Measurable, Assignable, Realistic, and Time-related. By applying these 5 methods to judge your goal you can increase the likelihood of success. Consider the following business objective: Add new features to the current product to close the competitive gap in the marketplace. The IT Leader needs to translate this business objective into a strategic goal for the Engineering team. The new SMART strategic goal could be: The Engineering team will deliver 10 new major features to the existing product over the next 12 months. A major feature is defined as an enhancement to the product described in detail on the product roadmap. The sum total of agile story point values for each major feature must be between 20-40 agile story points. This strategic goal is Specific because the definition of features is known. It is measureable since the number of features have been set to 10 and the size in story points has been set to a known range. The goal is clearly assignable to the Engineering team within the organization. The goal is likely realistic assuming some level of productivity analysis, such as the team’s sprint velocity, was performed by the IT Leader. Finally, it is time-related because the completion date has been set 12 months in the future.
The next step is to convert this SMART goal into a set of metrics that managers can track on a real-time basis. Look for the keywords in the goal statement that can be measured such as: deliver, features, agile story points, 12 months. Look at the number of features across the expected timeframe (from start date to completion date).This process could result in the following potential metrics:
1. Total number of features completed
2. Number of features by month (with the plan to complete 1 per month)
3. Size of each feature (ranging from 20 to 40 story points)
4. Number of story points completes per sprint (sprint velocity)
Testing for Alignment
Next, we can test this SMART goal for alignment using the suggestions listed above. The prospective SMART goal does contain keywords found in the business objective. It does contain sufficient detail to the description to create the metrics. The resulting KPIs do appear to lined up with the corresponding business object. The proposed metrics very likely connect with the team and should create the desired behavior, as well as a higher level of productivity. The manager can then use the existing sprint management tools to capture, measure, and track progress against the goal. The sprint management tools will also provide charts and dashboards. This is the key to the last 5 suggestions. Daily review of the dashboards with the KPIs side by side enables the manager to properly motivate, remove road blocks, and make adjustments to the plan which will keep the teams moving in the right direction. Finally, remember to make time for a bit of fun in the team. This can be measure too.