Balanced Scorecard for Clinics

At long last I’ve finally finished our F2007 clinic annual report. I’ve previously posted on the basics of what a small clinic can analyze at year end. I think a clinic’s scorecard needs to contain actionable information and be of a size that a manager/doctor can complete it in a day or two. What do I include in my own report?

Core Metrics
My core metrics are split by each clinic and include the number of referrals, total provider days, gross revenue and average daily revenue (gross/provider days). I compare each metric to the previous year. Included in the core metrics is a recommendation on the number of days to work by each specialty and in each office in the coming year. In general each 10% increase in patient referrals requires 5% more days. I don't act on a change in a referral pattern in an office unless it's off by more than 10% in 2 consecutive quarters

Referrals
We are a referral based practice so I analyze how many (by quarter), to which office (4 sites) and from which referring practitioner. We also divide each by specialty type. I’ve found that 20% variation is normal in referring practitioners. For each specialty, variation as little as 5% variation per quarter is more common. For generalists, I would recommend total patient encounters or total active patients (patients seen in the last 2 years) as a core metric.

Wait Time
The number of minutes waited in the office calculated from the time of the appointment until check-out and the number of days waited for different appointment types.

Our office uses block booking. I calculated the mean/stdev of the time from appointment creation to actual appointment (we ensure each block of patients is a normal distribution). This measure is accurate retrospectively. For a prospective calculation, we also measure time to an opening for a block on two consecutive days.

Cost
Total cost to run the clinic (including depreciation and bad debt but excluding capital and taxes) as a percent of revenue. Variation of less than 5% is normal over an entire practice but individual groups can vary widely. For variable expenses I also analyze the budget as costs per provider days.

Catchment Area
We determine where patients are coming from calculated as a rate per 10,000 population. In primary catchment areas less than 10% variation is normal and in secondary catchment areas 20-30%.

Voluntary Employee Turnover
Our goal is less than 5% turnover per year. Each lost employee is calculated as a fraction of an FTE based on days worked in the previous year. Total employee bank is calculated as total days worked divided as a fraction of FTE’s. We consider 4.5 days per week one FTE. Last year our turnover was <1%.

No Show Rate
Our goal is less than 5%


Most of the data has been automated over the last 5 years so I can expand the analysis each year. It usually takes me about 8 hours to get the preliminary data together and another 8 hours to analyze and complete the report. For a mid-sized clinic it’s time well spent.

Mashup of Hospital Scorecards

Love scorecards or hate them being able to view them on a Google map is cool technology.

Check it out at Hospital Impact

Hospital Report Cards – Follow Up to Dr. Wes

Dr. Wes has a recurring post about hospital report cards and their dubious use. I decided to do some research for myself to find out if and why hospital report cards are inaccurate?

Think of a hospital report card like any diagnostic test. They have an accuracy, false positive value and false negative value (for an in depth look at the accuracy of diagnostics go to the Users’ Guide to the Medical Literature). Tests will also be affected by the pre-test probability and the likelihood of further ‘tests’ being done (verification bias). The classic example is the diagnosis of pulmonary embolus. When a patient develops shortness of breath after surgery there is a chance that it’s a blood clot in the lung. Each test that is run has a certain likelihood of either diagnosing or ruling out disease when it is truly present or absent but few are completely accurate. For that reason, clinicians learn to treat PE based on probabilities rather than a true diagnosis. A subject dearer to my heart is the accuracy of CT scans in diagnosing deep neck infections which has similar inaccuracies that effect treatment.

The hospital report cards act like diagnostic tests for the hospitals. They test for inadequate performance. Take for example the reported rate of mortality from acute myocardial infarction (AMI) between hospitals. The reported mortality rate from AMI needs to be corrected for patient factors such as age, gender, cardiac severity and comorbid status. Even if one could perfectly risk adjust the data there is still the chance of random error misclassifying a hospital. A hospital that has, in truth, an unacceptably high mortality rate from AMI may be classified as normal (false negative). Conversely, a hospital with an acceptable mortality rate could be classified as substandard (false positive). .

As has been pointed out by Dr. Wes these report cards effect funding. They change where patients go for care, where public funds come from and which hospitals are targeted for quality improvement. It stands to reason then that the impact on hospitals is going to effect how the report cards are created. In addition to the paper quoted by Wes, a another recent study out of ICES (coincidently authored by Peter Austin, a co-author of mine and consultant to my Master’s Thesis) explored the relationship between outcome and report card design.

According to correspondance with Dr. Austin, "one can conceive of hospitals as either having acceptable performance or unacceptable performance. False positives (incorrectly classifying as having unacceptable performance a hospital that truly has acceptable performance) results in penalties being born by the hospital (damage to reputation, decreased staff morale, loss of referrals and business) and potentially by the patient (decreased local access to services if services were to be moved to a regional centre). False negatives (incorrectly labeling as acceptable a hospital that truly has unacceptable performance) result in penalties (or costs) being born by patients (unnecessary exposure to increased risk).

Hospital report cards can result in false positives and false negatives. How one weights the relative costs incurred due to false positives and false negatives can influence the threshold that one uses for classifying a hospital as having unacceptable performance."

Based on my reading, scorecards might be useful for improvement projects within a single hospital and identifying outliers that fall below acceptable standards assuming that you can clinically justify the standards, adjust for co-morbidities and impact. Even then, the research on how to normalize report cards is still in its’ infancy. I think they should be used with extreme caution by the public and by public I mean anyone without a PhD in statistics. The use of hospital report cards by the community to identify subtle differences in care is difficult at best and misleading at worst.

*this post was ameneded on June 20th after correspondance with Dr. Autin.

Balanced Scorecard

Every clinic should have some sort of a balanced scorecard. A scorecard is just a way to track the key metrics of you're clinic. In hospitals, they include a variety of metrics ranging from how the hospital is treating it's employees (turnover rates, hours, lost time) to how it's treating it's patients (infection rates, satisfaction, etc..). For an excellent example of a performance measurement scorecard check out Toronto's University Health Network performance measurement and balanced scorecard.

From a clinic's point of view however, there are usually problems creating such an extensive scorecard. Few of us have the staffing or desire to create these kinds of metrics nor the computer integration to make it happen (how many of us can track clinic acquired infection rates). That being said, a methodical approach to building key performance indicators in medical offices is achievable. Our clinic has been slowly building our own balanced scorecard by adding one or two metrics related to the service of patients and employees each year for the past 5 years. We now have a yearly report of 20 pages on which to base decisions.

I'd offer the following suggestions if you'd like to start you're own scorecard:
1. Think big -- carefully research clinic scorecards and decide which metrics you want to include. Create an overall plan of where you'd like it to be in the next 5 years. Aim for the stars.
2. Start small -- you'll be amazed at the road blocks you hit along the way. Start with one metric
3. Find ways to automate -- if it requires more than 5 minutes to retrieve the data it will not be collected 5 years down the road. See if you can automate the collection somehow. Often you can pay you're EMR or accounting software vendor a small fee to program it into a spreadsheet or report.
4. Keep building -- each year/quarter add another metric to you're scorecard. It'll be rich with data before you know it.
5. Use the data at least once a year -- make a concerted effort to collect, analyze and act on the data once or twice a year. Build you're own yearly practice report.
6. Share the data -- it's easy to read into the data you're expectations. Share it with as many stakeholders as reasonable and get their interpretation.

I'd suggest that each practice collect the following basic data to start:
1. Number of Patients (New and returning)
2. Number of Doctor Days (or Provider Days depending on who is providing the service)
3. Number of Referrals (if a referral based practice)
4. Average Wait Time (watch how it changes with number of provider days)
5. Average Daily Revenue (if fee-for-service = revenue/days worked)

I'd also recommend that you list the changes from quarter to quarter (e.g. Q1-2007 compared to Q1-2008). To calculate the percent change use the formula (New-Old)/(Old) x 100%. If for instance a practice had a provider work 58 days in Q1-2007 and 48 days in Q1-2008 then the difference is (48-58)/(58) x 100% = -17%. If you work 17% fewer days you'd expect a 17% rise in wait time. If there is only a 5% rise in wait time is it because you saw fewer patients (look at the patient data) or because you were more efficient (look at the average daily revenue data)?

These are 5 basic metrics that will help you adjust how many days to work and where to spend time each year. Since wait time is a major factor in patient satisfaction it will also give you an objective measure of patient service level. The hospital scorecards are fantastic but difficult to achieve in smaller practices so if you're practice requires other suggestions for key metrics feel free to email me or leave a comment.