About once a week I get a call from a colleague asking me if I could send them information and/or data as to how we measure the effectiveness of our integrated care project. First off, those of you who might have read previous posts on this site know what I think about the term “project” in relation to integrated care. For now, we’ll set that aside.
I’ll usually start by asking them what specifically are they looking for? A lot of times they aren’t really sure, but generally they want to see some comparative data as to the efficacy of the model versus those who have not received integrated care. In a vacuum where we can control for the dozens of variables involved, that might be possible. In practice, though, it’s like measuring the weight of a specific wave. How much of it exactly do you want to weigh? How do you define a wave? At what point in its development from just a surge to full-on surfable wave? Where does it start and stop? What about the debris and sealife that might be rolled up in the wave? Do we measure that too, or just the water? So the point is made – it’s a bit of a loaded question, and frankly one that in a truly integrated system is impossible to answer to the satisfaction of someone wanting that level of detail. Now, in theory, measuring a wave is really as simple as measuring its height times width and factoring it at a measure of one cubic meter of water weighing one ton. But we’re not talking about theory here, are we? We want to compare a specific patient versus another specific patient controlling for specific variables. Patient A received interventions from a behaviorist (BHC), Patient B didn’t. Who is better?
Clearly we see the problems with this. We have to control for a lot of variables, set some very clear definitions, and design some very specific measures and procedures to vet this with any degree of accuracy. This is why we choose to focus on population health measures. This is for a couple of reasons:
- This is a population-based practice
- Causality and correlation is impossible to identify
- It’s not really a choice at all. It’s the only way to ethically and accurately measure its effectiveness
In a truly team based integrated setting, there are a number of connections, relationships, and communications that may benefit a particular patient. For example, Patient A comes in, scores high on her PHQ-9, and gets to chat with a BHC, and leaves with a few self-interventions she can apply in her life to decrease the impact of her depression (leaving out, of course, the medical interventions which may also play a significant role in the patient’s mental functioning – but we’ll control for that in this example). Patient B comes in, scores the exact measure on his PHQ-9, but does not see the BHC. Could we figure out some measures to track their outcome from that point? Maybe. But the scientists quickly start raising alarms about reliability, validity, distortion, bias, correlation, causality, etc. In a truly integrated setting, we need to ask a whole lot of questions about Patient B. Did the provider consult with a BHC (a “curbside consult”)? Did the provider use skills and protocols he might have learned from a BHC to assist the patient and his depression? Is the patient receiving counseling from an external therapist? Is he already on an anti-depressant and has suffered with depression for a long time? Quickly we’re trying to measure waves again, right?
In a behavioral enhanced patient centered medical home practice measuring specific patient outcomes in comparisons to others is a fool’s chase. It is a rabbit hole the size of the Holland Tunnel. And in the end its meaningless. This practice helped that patient. Great. So what? Are we prepared to transform an entire healthcare system based on isolated cases in statistically insignificant sample sizes? Is a grantor or payer going to be dazzled by that and commit to collaborate?
We measure our impact on populations and the unique challenges facing each particular practice. I always start with this graph based on data from an external payer that compares our practice with other local providers serving safety net patients:
In this graph you see that our practice produces more primary care encounters, and significantly decreases ER utilization, specialty care, and hospitalizations to the tune of an overall cost savings of 22 percent. We can dive deeper into this by looking at the factors that keep our patients out of the ERs 68 percent less than our peers. Better care coordination, accessibility to care, more targeted services, etc. This data, by the way, covers a very large sample size and time period, for those asking (and rightly so) about its context.
When stakeholders or payers ask about specific outcome data, it should not be a point of stress, but rather an opportunity to educate them on what integrated care truly is. As described by Parinda Khatri, Ph.D., our chief clinical officer, we need to also consider the common factors that tend to impact patient outcomes: Culture, family dynamics, habits, and health literacy. In the very least, the model helps mitigate barriers to care and does not contribute to the problems of access and the delivery system. In the ideal, we have some impact on keeping patients out of ERs and hospitals, and help with personal resiliency.
I know that is often an unsatisfactory answer to people who want simple measures and results. It would be far easier if we were segregating one population from another (for example, adult patients with a diagnosis of depression that all receive identical protocols from a trained team of specific providers that is responsive to this population). Sure, that’s an effective approach that will likely yield terrific results – but it is not what we think about in terms of integrated, population-based care.
So, surf’s up…who’s ready!?!