Are we measuring right?

In the following thread, I will try to list some of the challenges that we face in the process of KPI (key Performance Indicators) definition.

Customer interest

few questions to ask your management team:

1. If you stop or delay measurement, is your customer going to complain and raise a red flag?
2. Does your customer question and discuss your measurement accuracy?
3. Is your customer using your KPI in process control, quality control, website info or advertising material?
4. Is your customer using your KPI in key-strategic decisions?

If most of the answers come negative then you are probably picking the wrong KPIs. In my opinion, measurement should be driven by customer interest and focus. It is not that difficult to predict what a customer wants, we are playing a customer role in our daily/weekly/monthly shopping. Customer cares about quality, cost and time to deliver the needs. So when you are addressing your customers, make sure you pick the right KPIs that trigger their interest in quality, cost and time. For instance, sharing employee performance measurement with your customer is not really the focus they want. However, sharing KPIs that reflect the quality and stability of a product is straight to the point. In addition, a KPI should reflect the past, the present and help to manage the future. The following are some of the interesting KPIs that you might want to share with customers:

1. Quality measurement: (The present) that includes current known issues in the product, (The past) a trend that shows the decrease/increase of issues over time with key technical/management decisions annotations, etc...
2. Cost measurement: A cumulative report that shows the trend of project cost over time relatively to project full budget as well as project phase sub-budget. If you are in a service model, then a trend of an average cost per resource would be very helpful along with annotations that represent key technical/management decisions over time.

The 5 perspectives

Assuming that you have a family of 4 members and you are the main income driver. You have been offered shares in 2 types of companies for the same price (equivalent to all your savings!)

• A company with 60% chances to double its price by end of year #1 and it belongs to a very competitive domain.
• A company that guarantees to you a minimum of 20% yearly interest on your initial investment for 5 years.
  

Which one do you pick? In fact it depends on your risk tolerance. However, your family is an important factor here because it represents your responsibilities and duties that you cannot avoid. I did the exercise and asked 8 married persons. 6 out of them picked option #2 and only 2 persons picked option #1.

Now you are a company owner, what would be your smartest target?

• Increase your revenues by selling more!
• Increase your end of year net profit
• Ensure your business continuity and almost same yearly profit
  

As you may notice, target 3 goes along with option #2 of the previous question. As a company owner, it's in your interest to increase the share price of your company and that's only possible by increasing the demand thus satisfying option #2. That leads us to the target of every business: Business continuity.
Business continuity consists of 3 factors: The customer, The company and The employee. You can easily evaluate your strength of business continuity through answering the following questions that represent the 5 perspectives of every business:

1. How do customers see us!? - Customer satisfaction: that consists of quality and excellence of work
2. What measures did we take to optimize our processes? - Optimization: That consists of optimizing processes and reducing the overhead.
3. What measures did we take to reduce our operating cost?
   
4. What are our latest innovations?
5. How do employees see us!? - Employee satisfaction: that consists of rewarding distinguished employees, fairly evaluating their performance based on objective measurements and comparative to their previous accomplishments and provide a satisfying working environment.
   

KPI & Scorecard Benchmarking

I think that Benchmarking methods differ relatively to the production model of the outsourcing provider. A unit based production such as data entry and processing should be handled in a total different way than a software service or product model. Below I will explain my benchmarking

method for a quality performance indicator in 3 different production models.

Data Processing - Unit based production

Obviously it is a highly (if not the highest) competitive market in outsourcing. India, China are on top of the list of data entry providers in the world for many reasons but mainly due to the availability of a huge man power pool and at the same at the lowest rates in the world. That does not mean that they make less money than eastern European or middle eastern providers, we are talking about the power of economy of scale. That makes India and China hard to beat on the cost (if not impossible). However when I started my blog, I defined outsourcing as a scientific solution for the best trade-off on cost, quality and production. Cost and production are always in advantage of the largest scale then that makes the quality as a unique competitive element between all providers. The following site http://www.dataentryindia.co.uk/ is top #1 on Google when you search for data entry, if you check it then you will notice their focus on quality on their main page where they claimed a 99.95% accuracy. But how easily can we benchmark our accuracy? is it by comparing to the top #1 Google site? How do we know whether our benchmark is reachable or a dream in the sky?

I am going to start from my last point "Reachable benchmark". Indeed ratios of 70% and 75% are too low because we can simply reach those levels by automated data processing, OCR scanning and other methods with a lower cost than manual data entry. So we know that the benchmark is higher than 75%. My second argument consists of watching what similar industries are doing such as factory massive production. That leads us straight to 6-sigma that's based on the science of probabilities and aim to minimize the probability of anomalies in production. The following table shows a simple demostration of how 6-sigma might be used for benchmarking in data processing production model

As you notice, sigma level 6 represents the case where there are only 3 anomalies in a production of 1 million units. So my suggested approach is to first locate your team on sigma scale (identify the level that you currently produce) and set the next sigma level as a target. My next question is whether or not should we adapt 99.99966% as a benchmark?

I believe there are many constraints that should be taken into consideration. However if the production process is very similar to a factory production then yes, the benchmark should be 99.99966%. The most important similarity aspect s to see whether there is a pattern in production process or not. As much as we can identify patterns and cluster them as much as we converge to a factory model because simply that's how machines work! In other cases where it is not that obvious to identify a production process pattern then a more modes sigma level should be set as a benchmark according to key constraints.

Change management

Change management is a double-edged sword when applied in outsourcing. If an outsourcing company is willing to track its performance the first step would consist of the definition of critical success factors and related key performance indicators. Then most probably there are some changes that need to be implemented on the ground in order to be able to collect useful data. In a service model outsourcing, the production development process itself is subject to changes from a client to another. When an outsourcing provider fails to implement the process changes required by the clients then that would jeopardize his commitments, service level agreements as well as his ability to operate efficiently on low cost. On the other hand implementing those changes in a cost effective way would increase the credibility of the outsourcing vendor as well as his ability to adapt in a timely manner.

Similarity with OOP

Scalability is another factor that plays an important role in change management. In Software engineering (Object Oriented Programming/ Inheritance) The object itself does not contain the business implementation; a software engineer can change the behavior of many objects by simply modifying their class or its root. When you implement a change affecting 4 resources, make sure that the same change is easily applicable when it affects 40 resources. Otherwise it would be better to invest the time into finding the right method to apply your changes.

Starting point

It is very important to identify 3 things when implementing a change:

1. What are the areas that you cannot touch?
2. What is the affected area by your change?
3. What are the subsets, within your affected area, where you do not have control over it?

Points 1 and 3 cannot be included in your plan. Make sure that you list these areas with large fonts otherwise your chances to fail would be 10 times more than your chances to succeed. Once you identify your affected area and the subsets under your control, you should then isolate a case and apply your changes at project manager level not at a team level. Make sure that only your PM is concerned by the first change. Once your PM digests the changes provide him/her the tools that allow him/her to implement scalable changes to his team. Change the content and not the concept; if your team is used to send daily emails then do NOT stop emails and use other collaboration tool but simply standardize the content of emails so you or your PM can easily extract information. This change would be easily accepted by your team because it does not require them to ramp up on a new tool. You can compare it to asking a left-handed to write with his right hand while he surely prefers to write whatever you ask him with his left hand than writing his name with his right hand (I know that because I am a left-handed!)…Simply minimize the feeling of a change across your team. Once you succeed implementing your strategy in 1 team, you should take lessons, adjust your plans and your tools in order to be able to implement a scalable solution at the level of your company using the same strategy that starts from Project managers down to team members.

Decision making workshops

Core changes require a lot more effort than simple data collection process changes. An innovative way would consist of organizing decision making workshops where your teams members play the role of decision makers in your company, propose the problem and let them converge by themselves to the required change modification. Group discussions are extremely important in that case and they should be guided by a smart person who really knows how to be a lead collaborator than a guide or a teacher.

Performance Measurement applied to Outsourcing

Performance Measurement applied To Outsourcing - Introduction

This article is intended for both outsourcing vendors and buyers. However I am exploring it from a vendor’s perspective because that’s what my experience is based on.

Before we start - What is outsourcing?

I believe that outsourcing is a scientific solution for an optimization problem that consists of the optimal trade-off between cost, productivity and quality. Sometimes buyers focus on cost and aim to minimize it compared to their in-house cost; however it's hard to compromise on quality. In other cases buyers focus on quality especially when they lack highly skilled in-house resources or a specific expertise that represents a critical success factor for their businesses.
I have made a brief introduction that really summarizes what I am going to discuss in my post. I have clearly mentioned the problem "an optimization problem that consists of the optimal trade-off between cost, productivity and quality" and I am going to follow a scientific analysis in order to converge to the desired solution.

Observations

Since we're scientifically searching for a solution then our first step would consist of observing and collecting the data we currently have. I am sure many of you have read or heard things like
"What gets measured gets done" or "If you can't measure it, you can't manage it nor improve it" but that does not tell us what and how to measure? It is not possible to find a common answer or a methodology of measurement. It needs to be done on a case by case but should at least follow the same scientific pattern of analysis. In our case, I believe we should measure the cost, the productivity or utilization and the quality ratio, because simply that's what we care about. We should never jump to conclusions while we are collecting/observing data.

Measurements

Capacity utilization is a concept in economics which refers to the extent to which an enterprise or a nation actually uses its installed productive capacity. “ Thus, it refers to the relationship between actual output that 'is' produced with the installed equipment and the potential output which 'could' be produced with it, if capacity was fully used the other day”. A software outsourcing company produces software related services or products. Process management, process control, HR activities (including hiring and interviews), Infrastructure facilities management, etc… are all crucial operational activities in order to support the production; however they are not part of the core output production. Ideally a software outsourcing company has a potential of producing only software related services and products 100% of the time. That leads us to define the real output as the man hours spent ONLY on software related services and products excluding all non-core-output activities. However the capacity utilization should be carefully defined according to the development model of the outsourcing vendor, whether it is a service, product or manufacturing and data processing model.

Quality measurement is not as obvious as capacity utilization. Quality should be measured in respect to the software outsourcing model and the associated activity type. Quality should be always measured relatively to a predefined requirements reference, or in other words, what the client is expecting to have. Number of bugs in itself is not representative of quality. It should be always associated with a reference. i.e: 10 bugs found over 200 tested test cases is not as severe as 10 bugs found over 50 tested test cases.

Cost measurement

The easiest way of cost measurement consists of evaluating the cost of production that simply represents the office expenses including payroll, infrastructure, taxes, etc…In order to standardize the cost measurement and be able to compare, we will have to evaluate the “cost per man. Day” or “cost per man. Month”. However if we take a deep look at cost evaluation we can split it into 4 main categories: cost of hiring, cost due to quality issues, cost of ramp up and cost of infrastructure.

Measurement tools

There are many tools and application, out there on the internet, developed by large enterprises that mainly focus on project management, collaboration and reporting as well. That would be perfect for an outsourcing startup that does not have existing and running processes or even an outsourcing vendor that does not have any process constraint required by their client(s). Some of the outsourcing vendors go through the customization of these tools and spend an important amount of money relatively to their gross income and in most of the cases it costs them even more than developing their own application that only includes features required to control their cost, processes, productivity and quality. I went through that experience and ended up building my own tools with one important characteristic that increased my chances of success: Interoperability. That means my tools do not impose any constraints or requirements on my processes nor the way my team is currently managing the production, nor the way I am currently dealing with my customers and using their bug tracking system or any other third party application required by their business process. Any change in the process is an additional cost on outsourcing vendor, because simply change management is not a short term activity especially if we’re talking about a large scale outsourcing vendor. It would be much more cost and time efficient if money and time are invested in developing Interoperability tools that co-exist with the current processes before implementing the change. We are measuring to know what we should modify in the process and improve our performance; it is not scientific at all to change the process in order to be able to measure. Once we have the measurements then we can easily decide whether investing in a change management is worth it or not.

Scorecards definition and tracking

Once we are now at a stage that allows us to measure our cost, productivity and quality; It is important to start organizing the measurements and tracking the performance. A scorecard is a simple measurement and tracking sheet that allows managers and decision makers to log and track performance of teams and individuals. Whenever we speak about tracking that leads us to set future targets as well and monitor our ability to reach the targets. And this where we introduce the following factors:

• Critical Success Factors (CSF)
• Key Performance Indicators (KPI)

An easy way to differentiate between the 2 factors is to compare them to a Rally competition: what do you consider a critical success factors to win a Rally?

It is obviously to be Number #1 (CSF).

How do you measure it?

You can watch your speed, how many cars are behind you, how many cars are in front of you, what is your fastest lap? These are your KPIs.

However if your success consists of safely finishing the rally then your KPIs would be: Watching your RPM, monitoring your engine temperature, monitoring your break censors, monitoring your wheels status, etc…

CSFs are to be defined by company’s directors and define the appropriate KPIs to track them. Once defined, individual and team scorecard should be issued and tracked with a clear target for a specific timeframe.

Reaching your targets – Corporate perspective

Performance measurement directly serves the ultimate vision of every outsourcing company. Performance measurement is not a product, nor a tool, it’s an Environment. Reaching your targets involve different roles in your company. It is not enough to show a chart, what really matters is to teach people how to read it, don’t jump to conclusion, analyze it then decide. People skills and ability to analyze in a scientific approach is the most important factor for this environment otherwise it is impossible to align your vision across all your resources.

It starts with the human factor and spreading a scientific approach within your team to believe that performance measurement is not the target but a mean. Performance measurement should not be presented as a judgment or evaluation tool.

Process Improvements are also another aspect where managers can improve the company’s performance by simply calibrating process or workflow factors accordingly.

Innovation is the most important success factor, specifically all solutions that focus on automation. In most cases automation have a direct positive impact on cost, productivity and quality.

Customer satisfaction

Customers want transparency; they expect outsourcing vendors to be ahead and smarter than them. They want quality and ability to evaluate quality at anytime. They want productivity and ability to evaluate productivity at anytime. Improving your efficiency and performance means that you’re getting to the optimal point of maximum productivity, minimum quality issues and minimum cost. That’s exactly what your clients or potential clients are looking for!

Soon on my blog

How to collect and measure Capacity Utilization?
How to set scorecard targets?
Benchmarking and scorecards
Automation and Interoperability tools
Process improvements and change management