Patient Quality Assurance

Two weeks ago we instituted a quality assurance program at our clinic for patient/referring office complaints. Normally, when someone complained we had whoever received the call deal with it and if they couldn't the complaint escalated up the ladder. The piece of the puzzle that was missing was how often people complained. For all I knew the front desk was dealing with 5 or 6 major complaints week.

Well Rule #10; "If you don't take a temperature you can't find a fever" bit me right in the ass. We started tracking the complaints. So far this week, we've had 3 (one was about me) and counting. I'm quickly finding out that sometimes you can't improve the situation but recording, acknowledging and trying to find a solution to a complaint is good customer service in-and-of itself. Recording the complaints also lets us look for patterns.

When I read about Paul Levy discussing a wrong limb error and John Halamka reflecting on how to leverage IT to minimize intra-operative errors our clinic concerns seem like small potatoes. But when you consider that most medical care is conducted in small clinics throughout the world small changes in quality assurance can make a big difference. I often wonder what the minimum standard should be for clinics in terms of quality assurance to ensure that biopsies are not forgotten, blood work is checked and patients are recalled for critical appointments.

I know that when our clinic transitioned to an electronic recall/treatment tracking program our follow-through changed by roughly 20%. It stands to reason that other tasks which require human follow-up improve when automated. If we can't track it however, we'll never know what we're missing. I'd ask the following question of clinicians -- if you're front desk failed to follow-through on a biopsy, bloodwork or critical appointment is there any way that you could tell? If you can; do you know what the error rate is and are you working to minimize it?

No Good Can Come of This

No good came come of the statement “honey, I was reading your blog”. When creating a blog I think everyone struggles with the decision on whether to do it under a pseudo name. A pseudo name probably makes you more honest but less careful with what you say. Your real name makes you more accountable to your words.

Especially to your wife.

So even if readers didn’t leave comments I still had to face the error of my logic on the post “please arrive 15minutes before your appointment”. Her comment was, “if I arrive 15minutes before my appointment that’s just 15minutes more I have to wait”.

Which creates a catch 22. If your clinic is hopelessly late with patients it’s unlikely they’ll willing show up 15minutes early. If you require 22 +/- 7minutes to prepare the patient for the visit you’ll constantly run 22 +/- 7 min late. You can’t win. Or can you?

The answer is to control the variation before you control the arrival time. If one clinic operates 22 +/- 7 minutes late, arrival time can be gradually changed. If it runs 22 +/- 30minutes late a significant portion of your patients are waiting greater than 52 minutes. Why would they arrive early?

The answer to the dilemma is not in looking at the average and variation per say, rather, examining the number of outliers. The technique goes back to my posts on six sigma in the office to minimize office wait times. The suggestions to my wife’s doctor are to first control the variation in the arrival times, then control the actual arrival time.

An Example of Zeroing-In on a Workflow Issue

Today I’ll show a hard data example of how to measure workflow and visualize how it can analyzed. I’ve previously described how a new bigger office actually hurt workflow but what was the specific problem? Our office tries to keep the time someone waits during an appointment to 50minutes. But when we moved offices in June 2007 the number of people waiting longer than 50 minutes increased. The number of “errors” (waiting longer than 50min) went from an average of 9% of people to 12.5% (a 39% increase).

To find the problem a patient flow map during consultation was made.



Using data mining, the times between each stage of an appointment could be retrieved for 596 patients. Three different processes could be found in the appointments to analyze the patient flow. The time from when the patient arrived to when they had they’re medical history completed (includes registration and review of medical history – “Arr to MedHx”), the time between when the medical history is reviewed and the doctor completes the consultation (“MedHx to Doc”) and the time between when the doctor completes the consultation and the patient leaves (“Doc to Out”). Whenever doing a process flow map, start globally and if necessary a more detailed analysis can be added. In this process flow map we used four time stamps to create three time frames.


In 2007 Qtr 2 and Qtr 3 there is a spike in the “Average of "Arr to Out"” (total time). Just prior to the move (Qtr 2) it because of an increase in the Average time from arrival to medical history (registration & medical history review) but after the move it’s because of the doctor consultation time.

In the new office the doctors offices are located further from the consultation area (one-short stairwell away but a world apart) so doctors were constantly running down stairs for chart-work (and to surf the net). By adding dictation, phones and internet access in the consultation area the problem seems to be resolving. We could have done more studies to look at times with arrival to registration, registration to xray, xray to medical history, etc… but the possibilities are endless. Always start with a more global view to scan for the problem then drill down as necessary.

For those who are mathematically inclined keep reading. The number of people waiting greater than 50minutes is our error rate and is synonymous with the sigma value. The sigma value went from 3 to 2 which prompted the intervention. When we compare the time before the problem period (2007 Qtr 2 and 3) look at what changed:

Error Rate (% waiting > 50min): Up 39% (9% to 12% of patients)
Mean Wait: Up 10% (from 34 to 37min)
Stdev (standard deviation) of Overall Wait: Up 18% (from 10 to 12min)

But drill down on the two time frames, “Arr to MedHx” and “MedHx to Doc”:
Arrival to Medical History: Mean Wait Up 11% and StDev Up 1%
Medical History to Doc Consult: Mean Wait Up 25% and StDev Up 40%

The take home message is that it is not enough to know the average wait time; you have to know the amount of variation in the wait time as well. In this case, the increase in mean wait was a smaller part of the problem than the increase in variability (of how long it took the doctor). To put it into an ER wait times scenario, a hospital may claim that the average wait from arrival to initial assessment is only up 10% but the variation may be much different. High variability will lead to high error rates quickly whereas it moves the mean up slowly. In ER wait times, the more critical value is how many people are waiting too long for assessment/triage (see CTAS scale for Canadian standards). In our case, the mean was up modestly but the “error rate” was catastrophically elevated by high variability of one segment of the appointment. Luckily, the problem was easily rectified once we realized what was happening.

Closed Malpractice Claims related to Delayed Diagnosis

Here is an article from the Annals of Internal Medicine that systematically reviewed the causes of closed malpractice claims related to delayed diagnosis & delayed care. Of note, failure to have a follow-up plan was attributed as the breakdown in 45% of the 181 cases (and most of the cases were cancer related). Failure to follow-through is significantly improved with EMR and electronic treatment planning but it goes back to the missed appointment problem. A patient books a follow-up to discuss a problem then misses the appointment. What happens to the chart? Does it sit on a shelf and wait for the patient to call back or does the office actively pursue contracting the patient. Does the work flow plan to contract the patient have an error rate (an error being that you forgot to contact the patient) of six sigma (3.4 per million) or is it much lower.

Sometime take a look back in the schedule for patients that no-showed on follow-ups and see what the office did to contract them again. Is there a group of charts somewhere going back years with forgotten & missed patients?

Another interest point was that handoff's were considered a contributing factor in 20% of cases. Multiple handoffs means multiple (and compounding) six sigma rates. The more steps there are the higher the sigma value must be of each step to maintain the same overall error rate. Thanks to Running a Hospital for the link to the article.

Complexity Increases Sigma Values

If most of the industry operates at 3-sigma (66,000 defects per million) per process what is the effect of multiple processes. When a patient goes through a process there are almost always multiple steps that can have an error. Each step is going to have it’s own sigma value. Remember that in health care a defect is any error that causes the patients course of treatment to be altered. Only one step needs to be in error to cause the patient’s course to be changed. Remember the old punch line “the operation was a success but the patient died”.

The sigma values of steps, therefore, are cumulative not independent. The chart below shows how “accurate” or error free a multi-step process is. It’s obvious that the more steps the process has the harder it is to achieve an overall sigma of six-sigma.

So when reviewing a process for patients one the initial steps when mapping the process and looking at accuracy is to determine whether a step can be eliminated. Not only will it cut down on wasted time but it will improve the accuracy of the process exponentially.

How My New Big Office Screwed Things Up

Check out the picture of our new big office reception area (moved in June of 2007). I've already given away the ending in other posts but this is what happened. We were in a smaller office with two consult rooms and x-ray very near the front desk. We would never have enough room to put people in when post-op visit and consults came together so people would sit out front and wait.







With the new office we put in 7 consult and 2 post-op rooms assuming everyone would already be seated and things would flow alot better. Big error in judgement. Look at the floor map below.

Everyone has to walk a lot further. People walk in and register, sit down, get an xray, sit down, go into the consult room and have a medical history, wait for the doctor, have their consult then leave. The front desk can't just direct them to the proper room -- they have to be escorted there (or they'll get lost/ wander into someone elses room).

I decided to look at our sigma values for wait. Anything greater than 50 minutes is too long. In 2005 we had a sigma value of 53915 defects per million (almost 4-sigma, not great but respectable). In 2006 it increased a bit; I thought it was becaue of patient volume and the new office would cure that. In 2007 it turned into 120294 per million! Almost down to 2-sigma.

We're working at it but it goes to show you that the numbers don't lie and assumptions are just that. Everytime a patient has to sit in another area and wait or everytime they have to move it adds another step to the process. That introduces a greater chance of error and the person that is responsible for that step being unavailable. Complexity can greater rather than cure problems (as we found out). The solution to the problem (I think) is to improve the flow of patients. We've eliminated the wait when we can at the front (have them seated in the consult room right away) and thought hard about when to have people arrive so there is no back ups at the front desk. I'll post our 2008 number some time soon and hopefully they'll be better.

Six Sigma in Healthcare

In the last couple of posts I’ve mentioned a lot about six sigma. For those of you who are not familiar with it here’s a summary (from a non-expert point of view – sorry if I offend the six sigma black belts out there) and how it will decrease you're clinics waits.

Six Sigma terminology and methods were developed in 1986 at Motorola but came out of many quality engineering projects from the 1920’s on. The premise is that in any manufacturing process there will be a certain amount of variation. Say, for instance a part needs to be made to 100mm. In reality there will be a range of exact measurements all around 100 but the engineers may tolerate anything between 98.5-101.5mm. Any measurements that fall outside of that range are beyond the acceptable limits. The sigma value refers to the number of defects that fall outside of that range. Three sigma is 66,000 defects per million (where most of industry is) whereas six sigma is to have less than 3.4 defects per million (where are few are). There is a cost to acheiving six sigma, but it's usually offset by not having to repeat works, have items returned in warranty, etc...

To understand how sigma applies to healthcare it's easier to consider another service industry. The classic example is a hotel. When you call to book a hotel there is a series of steps you go through. If one of those steps is a "defect" (such as their website is down, leave you on hold for 15 minutes, haven't staffed enough to open rooms) you will not book the hotel room. This is considered a "defect" in service terms. How does sigma apply to healthcare? I consider a defect in health care an error that causes a patients course of treatment to be altered. For example:

-failing to a call a patient back about test results or treatment
-failing to file/find paperwork
-keeping someone waiting too long
-failing to do something that prevents complications
-failing to do something that improves outcome

The best example I've found (as it impacts on wait times) is not calling patients back about tests, results or treatments. For instance, a patient books an appointment for a procedure then cancels at the last minute. Often the chart ends up on a "call back rack" . When I look at the “call back" racks they typically have an excess of charts dating back months and years. No one is being irresponsible, but the system is created around patients arranging their own follow-up. Also, most of us are short of time and the immedite need is to deal with the patient in front of us. The problem is that after a period of time the entire process has to be repeated, or worse, the administrative staff has to go through extraordinary efforts to rearrange appointments. All of which chews up time and takes it away time from other patients.

In our computer system we have automated the way patients like this are followed up tomake it fool-proof. For instance, if someone calls and cancels the day of surgery or fails to show-up they are automatically put on a call back list.


I’d ask you to look at how you’re office organizes appointments, follow-ups, procedures and investigations. Do you have a call back rack? If so, how large and dated is it? Do you have an automated means for follow-up? If not take a look at the last 100 charts and see how many have been lost to poor organization. There are all kinds of area that can be made to six sigma in our offices but it is not easy. It will decrease on wait times, however, because time saved with one patient can be spent with another.

Waiting too Long – Intro to Six Sigma

In my blog of February 27th I wrote about keeping patients waiting too long. I described the CSAT scales and why the tangible parts of health care are what patient usually use to form their opinions of us and our clinics. Wait times (and in particular waiting while in the office) are of particular concern. They are easily controlled with good time management.

First, not every clinic is going to have a lot of wait in the office. A clinic needs to be greater than 80% utilized before it will see significant wait times (this is just an approximatation but I’ll get to that in a minute). Remember, that wait increases exponentially as utilization increases.

But as clinic utilization increases so do the chances of excessive wait. How do you judge an excessive wait? Assume that patients arrive at a health care clinic expecting a certain amount of wait. It will be different for every specialty but for a basic clinic we’ve found that from the time of the booked appointment to the time they leave a wait of greater than 50 minutes has an effect on the patients’ follow-through.

The way we determined this, was not by using the CSAT but by treatment follow-through using a group of homogenous patients that presented to the clinic for consultation prior to minor surgery (in this case removal of wisdom teeth). The group was then divided into 10 minute segments and their follow-up rate compared. For those patient that left within 20 minutes of their appointment start time just over 80% followed through, 30min 75% and so on. When the group that was less than 50 minutes was compared to the group greater than 50 minutes there was a significant difference in follow-through. In a quick-and-dirty study such as this I make no attempt to establish the cause of their unhappiness, only that it is related to the time spent in the clinic. I can say this data was found when comparing multiple variables including location of clinic, doctor, age and gender. Time waiting seems to be an independent variable. The biggest assumption in this is that the length of consultation, x-rays and administration is approximately the same.

Our clinic, therefore defines a failure in service to be keeping a patient waiting for consultation greater than 50 minutes. In industries, they measure failures as defects per million units. A high failure rate is 3 sigma which is 66,000 defects per million (where most industry operates) and a low failure rate is 3.4 defects per million. You do not, however, have to wait for 1 million patients to come through the door, an adequate sample can be taken (in our case we data-mined approximately a thousand but several hundred is usually more than enough) and created this curve of waiting.

In the curve you can see that our sigma value has actually gone up! Or in other words, as the years have progressed we’ve kept more people waiting longer than we like. Why? Our office increased utilization and moved to a bigger space. One would assume that a bigger office and more capacity would help with in-office wait but it seems to disrupt the patient flow (we’re working hard on decreasing the in-office wait with process analysis but that’s another blog).

I would suggest that each of you examine your different patient pools and make an estimation of how long a wait is too long for each group. Then track how many patients fall outside of that range and work to six-sigma levels of not keeping patients waiting.

Wait Time -- A Universal Yardstick?

How much health care is too much and how much is not enough. In the United States the cost is the problem whereas in Canada it's access to care. Are the two related?

The time from seeking care to treatment is a good indicator of the level of service. When the helath care wait time is low there can be an excess of resources and when it is high there are not enough (in a system that is full). In theory, health care wait times can be calculated (and even better predict them going foward) for each sector of the system and resources allocated appropriately. Currently many governments are calculating wait times in the emergency rooms, monitoring patient flow and time to surgery. But wait times can be manipulated. Surgeons can limit consultation time so the actual time on the wait list for surgery is low, emergency rooms can leave the patients with the ambulance crew prior to registration.

Instead, wait times need to be considered from the patient's perspective. Not only will it better reflect the actual wait involved but it might allow for more solutions. Imagine a patient that phones the doctors office. If the wait time for the front desk is only measured as the time on-hold with the office administrator we see only half the story and consequently half the solutions. In this instance, the only option is to either hang up or keep holding. If the wait time is measured from the time the decision to call is made other options such as email, fax, etc... also become apparent. Both email and fax would potentially decrease the wait time and relieve the burden on the administrative staff.

While patients may choose to accelerate or delay treatment and inadvertantly manipulte wait times, our data suggests that the mean wait is a good predictor of availability in the clinic. Most of the patient populations have well defined normal curves with predictable variations. Since representative survey populations are already common in politics; can the same solution not be extended to health care? We don't need to record every surgery and every wait time, just a representative sample. This would significantly simplify the burden on the information technology sector in health care (who is currently trying to connect many different clinics) and potentially provide accurate data over wider sectors. To determine how many patients need be surveyed and across which sectors, it would be simple enough to look at existing data for power and size calculations. From there a uniform, automated system could be implemented for all sorts of health problems not just those choosen as priority programs.

More and more institutions are rightfully focusing on wait times and so should smaller clinics. I think we'll find that as systems evolve a universal yardstick for health care delivery will not only let us see the problems, it will let us see the solutions.

The Biggest Bang for the Buck

What gives the best “bump” to efficiency in a clinic? If I could pick only one tool to decrease health care wait times what would it be?

Offices world wide have a broad spectrum of automation (computerization) and business practices. My old family doctor had his wife run the front desk, the waiting room had five chairs and there was one exam room. I rarely waited once I arrived at the office. On the other hand, I’ve seen emergency rooms with open beds, lots of staff, enough computerization to launch the space shuttle and still experienced 3-5 hour emergency waits.

I can think of many tools that should, in theory, improve the efficiency of a clinic and patient flow, thereby decreasing helath care wait times. There are certain tools that make processes work faster, such as computerized billing, email confirmation, digital lab results and the like. I consider tools like these “velocity tools”. They improve efficiency by leaning down the time of a process.

There are tools that make mistakes or errors on the part of the clinic less likely such as automatic recall schedules and fool-proof treatment tracking. In turn, less mistakes results in less repeated work or more time for clients. There are tools that allow better control of scheduling and patient flow such as computerized booking. These tools decrease the amount of variation in the system by allowing for control of booking and the creation of “pull” which improves efficiency.

Having been analyzing our own data for 7 years I believe that the tools that make mistakes less likely have the biggest benefit. In an already busy practice, velocity tools and variation tools can change individual processes greatly, but the overall effect is only 5-10% each. On the other hand, decreasing errors seems to change the burden 10-20% across the board.

In our case, it means digital treatment plans and recall schedules with strict algorithms to prevent errors. The system makes it impossible to forget about follow-up work to be conducted. Imagine a patient that comes in for a problem then neglects to follow-up. There is time spent in the office tracking down why the person hasn’t followed up and then repeating the work if too much time has elapsed. Not only do appointments and clinic work need to be repeated but so does the administrative burden. In healthcare, this lack of follow-up includes a significant number of people.

Looking back at the data, that single step of fool-proof treatment tracking changed our efficiency by 10-20%. I don’t think I can find another single tool with as large of an effect over the entire practice. So, the one tool I’d pick before all others is a computer system that tracks treatment and follow-up.

Talk about Wait Time and Delayed Care in Clinics

Welcome to the wait time and delayed care blog. This blog is about the techniques and tools of managing wait times in a professional setting. Doctors’ offices, lawyers, dentists, chiropractors, other health care professionals (as well as hairdressers) all run into the same problems because of quick appointments with a large number of people. My premise is that if the staff that manage scheduling and patient flow can make modest improvements, major changes can happen. Most of us can find ways of shortening a 20 minute appointment by 4 minutes. This is a 20% improvement. But if that same 20% improvements are achieved with the family doctors across Canada; that's an extra 3,400 (20% of 17,000) physicians worth of time.

If you are the person that is organizing or managing an office, business or clinic then this blog is intended for you. Our office works hard to run on time and get people in quickly, despite being in an under serviced area. Shortening wait times is not just a matter of increasing the number of people/providers that care for the patients it's a matter of better management.

There are four main concepts.
1. Time saved with one patient can be spent with another
2. Work that runs smoothly will run faster
2. Having to repeat work wastes time
3. To improve something; you have to be able to measure it.

Industry (e.g. Toyota, GE, etc...) long ago embraced these concepts through Lean, Six Sigma, DMAIC and other programs. But many businesses still don't use them to improve the bottom line. In the service sector we need to be even more diligent because wasted time affects our patients.

I hope you'll find the blogs helpful and I look forward to bouncing ideas off of anyone interested in the topic. Health care wait times, delayed care, waiting lists, block booking, priority booking, queuing theory, six sigma, lean, repeated work, critical time to treatment and many other topics are all up for discussion. My hope is many people will make small changes that can have a major impact.