Choosing Evaluation Metrics Through A Concept Tree


Evaluation metrics determine success or failure within a program or project because they are the standards we use to measure. There are many methods to obtain these metrics; however, a method with a strong correlation to the root causes can be identified through a concept tree. Concept trees allow you to identify root causes and their associated metrics while giving a more clear path of implementing theory based approaches. By having theories help explain your predictions for success, you can better your odds through any program or project’s life cycle.

Evaluation Metrics

Evaluation metrics are the overall view of how the program is performing compared to our standards. After evaluating performance, we can diagnose issues or work to identify future problems and mitigate them. Evaluation metrics provide us powerful insights into the program we manage, and the people we lead. It is important to note that not all managers and leaders are created equal, so it is vital that managers and leaders work to develop their professional capacities.

When we work to diagnose potential problems, we start with a hypothesis that works to identify known causes from our professional experiences. We should not let that persuade or bias the exploration into the problem. We should keep that hypothesis in the back of our minds while we check our employee pool to help better understand the context of the problem. Remember, not all problems (hardly any) are straight forward as we need to adopt systems thinking to comprehend the true nature of the problem. In medicine we call this referential pain, as pain is felt on a different part of the body that the problem lies within.

We should be thinking of a diagnosis similarly to how referential pain works. Just because our key performance metric is suffering, does not mean that we need to avoid topics on the tertiary (third connecting) level. By working through the problem, context, and “upstream” problems will set the program up for true success. For example, we may have less revenue incoming (you can switch this out with your business domain KPI), so it is tempting to blame the economy or another external factor compared to internal problems.

Concept Tree

If we are to look within the internal factors, then we should seek guidance from the concept tree in our root cause analysis process. Many business majors are familiar with Fish Bone Diagrams, even root cause trees, or the 5 Ws; however, not all are familiar with the concept tree. With Subthesis, we take a researchers approach to problem solving because all problems are relatable to a research question. In this same use case, we are going to solve a research problem that has business implications.

A Concept Tree is a model that works to identify ideas underlying how we view the world. We then take those ideas, and work to improve them through theory based approaches. We then take the theory based approaches and mesh them together in a process called Transdisciplinary Thinking. The end result is a program fix for better KPIs and outcomes through the lifecycle that every program should live within.

For this article, we can look at a mental health program that serves the community (you can visit my other articles that use other business domains as examples). Healthcare programs are unique because they require more skilled and trained professionals to fully operate the program. This presents a unique challenge on where we can focus our attention as we can help overcome the challenge of a concept called “Fidelity”. “Fidelity” is the concept of operating a program as it was intended to be operated.

Our concept tree will focus on “Fidelity”, and lets agree that two ideas that fill fidelity are: available professionals, and provider performance. Our first metric will focus on available professionals, and we target this by focusing on alternative services or primary services. If we wanted to perform alternative services, than we could implement an internship component to help improve the amount of available professionals. The policy accompanying can be a triage system allowing the patients with less severe problems to be treated by the interns and vice-versa. In another attempt, we could develop employment pipelines to improve the amount of professionals available. The incentivization could be giving interns a higher standing in being hired, or better workplace benefits to established providers.

By having more available providers, we are able to adequately operate the program as it was intended. This is metaphorically like having a brand new phone that processes data really fast out of the box. When you start to download your mobile applications, it starts to slow down. If you download a lot or “Too much” the phone will be laggy and slow. This is the idea of fidelity where you want the program to operate smoothly and to capacity. In other words, you need remain unbias in determining the root causes to establish meaningful evaluation metrics to increase your KPIs.

Theoretical Constructs

Now that we determined our evaluation metrics that are the root causes of our KPIs (Key Performance Indicators / i.e - Key Performance Metrics), we can focus on solving the problem or program ailment. In our earlier example, we used the idea of referential pain to understand deficiencies through the concept tree lens. This lens specifically allows us to develop solution for the particular problems. In this use case, lets focus on the second idea under fidelity, which was “Provider performance”.

Continuing the concept tree, burnout is a great sub-idea of performance. Another sub-idea would be professional tools ranging from software to ergonomic chairs or massage chairs. The main idea here is that we can really clarify the situation with a concept tree and focus our attention on very particular ideas. We will then use those ideas to build and formulate plans as we answer our research question(s). Note: this is not for the average managers or leaders because this is for people that want to optimize and operate their programs on jet fuel.

The literature supports that providers should obtain a break in between appointments or at the very least stagger the intensity amongst patient flow. In this case we are using a policy addendum to improve our evaluation metrics that feed into the KPIs. We will now add into the mix a medical scribe. Lastly, we will now add software focused on coordinating care. The medical scribe can operate this system.

When we look out burnout, there are many methods to overcome provider burnout as burnout indirectly impacts the KPI of the clinic or hospital. The idea beneath provider burnout is bandwidth power. Again in our mobile phone reference, imagine how fast that phone can operate with less mobile applications downloaded. Our goal here is to focus the energy needed on clinical and patient care while delegating other tasks to the medical scribe. The sustainability here is that the medical scribe will ensure high level of medical reimbursement though the Electronc Health Records. Note: there are different ways to improve provider burnout; however, the concept tree allows you to pin point your target.

The Lifecycle

I always recommend to my clients to implement a lifecycle into their program management because progress is a iterative process. More than often (over 95%) have some sort of life cycle either built in or considered in the back of their mind. What is not often thought of is how to modernize the evaluation metrics for cloud computing. This enables you to take your data, and build up your database for machine learning. As we come closer to the future, more and more business are going to use machine learning for predictive analytics. Predictive analysis is not deterministic as it is more probabilistic similar to the weather report.

For the previous examples, we provided policy interventions on our internal factors to help guide our progress into our standards of success. Policies help determine how we can navigate life or perform within our business. There are at times people who deviate from policy, and then there are people who are more supported from the policies than others. The lifecycle helps us ensure that the policies created are equitable for everyone. For example, maybe the software is too difficult for the medical scribes so we work to share the medical scribe load by giving the easy cases to the clinicians. This leaves the moderate and above cases to be focused on by the scribe.

Lifecycles provide us very meaningful and insightful analytics that would never have otherwise came up. If you are unfamiliar with lifecycle management, then think of this as a process evaluation. As we come closer to the annual report date, we can then focus on the outcome evaluation. The outcome evaluation is what you report to your upper management while the process evaluation is often in-house pulse checks on the program. In other words, you work to influence the process evaluations as the outcome evaluations are the by-product of your process evaluations. In another way of speaking, you bake your process evaluation and the outcome is the outcome evaluation (pun intended).


We use the concept tree to understand the evaluation metrics that influence our KPIs. It is imperative that we use the metaphor of “Referential Pain” to understand root cause analysis for deficient KPIs. There are many ways we determine the root cause and their associated ideas; however, the concept tree is tried and true method for this specific use. Not every manager or leader needs to understand research because a lot can be done on the superficial level. Identify the problem, ask the associated research question, develop a plan based in theory to answer, and then repeat the process for a full life cycle.

This is a picture of Angel Reyes the owner of SubThesis.