PATIENT SATISFACTION SURVEYS: ACCURATE, AVAILABLE, AND TIMELY

Dr. J.P. Shipley and Sam W. Westbrook, III

You are determined to find out how your patients really feel about their experiences with you and your office. To that end, you’ve even gone to the trouble of creating a set of survey questions that you feel are well designed and suited to measuring satisfaction with your practice. Now you are faced with the question: how shall I administer this survey?

Sounds simple, but as the old saying goes, the devil is in the details. For example, how many patients do I need to take this survey? How often does it need to be given? Should I mail it to them, ask them to fill it out online, interview them, give it to them in the office after their visit, or what? How do I get at the results of the survey? Are results compiled by my office or collected by an outside agency and mailed to me—or can I use a handheld Internet device (iPhone, Blackberry, etc.) to get access to the results more quickly and more frequently? And is there information in the survey beyond overall trend data that I might find useful?

Let’s answer these questions one at a time.

How Many Patients Do I Need to Take My Survey?

The simple and glib answer is: however many you’ve got. Clearly, if you could get all of your patients to tell you truthfully how they feel, then you would know with certainty their level of satisfaction.

This is a nice idea but seldom practicable. For one thing, some patients will not respond to a survey no matter how hard you try. Sometimes, getting a few more responses just costs too much for the benefit returned. So, how do you determine how many you need?

First, decide how certain you want to be about the meaning of the survey results. In statistics, this means choosing how confident you want to be that patient satisfaction lies within a certain percentage band. The usual choice for confidence level is 95%, but there is nothing sacred about this number. It’s just the one most commonly used. If your survey doesn’t generate enough responses to achieve this level of confidence, then you may have to live with a lower level, say 90%.

Second, choose the percentage band, or confidence interval. This band is also called the “margin of error”, and is the +/- value you see, for instance, in media political polls. The smaller you set this band, the more survey responses you will need. A typical value is between 3% and 5%, although for some applications 10% or even 20% is used. Again, this is your choice.

To illustrate using these two choices of 95% confidence level and 5% margin of error, if you get enough survey responses, and say 60% of them choose a particular answer, then you can be 95% certain that 55% to 65% of all your patients, not just those that responded, would choose the same answer.

So, we’re back to the same question: how many? You might think that if you want to be 95% certain, then you need to get 95% of the patients to respond. For small numbers of patients, that would be true. But, as the patient population increases, the number of responses necessary to achieve a given confidence level and confidence interval increases at a slower and slower rate.

You can find tables of these values in many standard statistics texts, and you can find online calculators for them as well. Here is a link to one, with associated explanations of some common terms:

http://www.raosoft.com/samplesize.html

Let’s take a few simple cases. If your desired margin of error is 5% and your patient population is just 20, then you would need responses from 19 of them to be 95% certain that the other patient would choose similar answers. But, if you survey 100 patients, then only 80 need to respond for you to be as certain. And, if you survey 1000 patients, then only 278 need to respond.

Note that these are not the small numbers that you’ve sometimes heard suggested or even recommended. Survey results based on 10 or 20 or 30 responses, unless that’s just about all the patients you have, are worth little and can even be misleading. As you will learn below, there’s no good reason for you to be so constrained, and several reasons for you to feel optimistic that you can understand your patients better than ever before, and at reasonable cost.

How Often Do I Need to Survey?

The quick answer is, only once, if you always have the same patients, and their feelings never change. You might hope for the latter (at least as a worst case), but it surely would be nice to have some new patients as time goes by. The reality is neither your patient population nor your practice’s work with them stays the same for very long. Moreover, patient attitudes toward medical and health matters change over time, not to mention changes in other external factors, like regulations and costs, for example.

The proper answer: to maintain adequate situational awareness, at a minimum you need to survey whenever you’ve made significant changes in the way your practice handles patients, and whenever your patient population changes significantly, whether in numbers, demographics, or attitudes toward medical care. Often such changes, even though significant, occur in slow, almost stealthy, ways.

One of your goals should be to prevent the emergence of an outlier, a patient who feels far more dissatisfied than just about any of your other patients. But wait, someone says; it’s just one or a few, so it’s not important. Maybe, but you know from your own experience that such incidents have a way of multiplying and becoming known among your potential patients. Given enough incidents and time, your standing in the patient community (and make no mistake, they do talk) will be eroded. It is difficult to recover lost credibility; far better to head off its loss beforehand.

To avoid the risk of missing such changes, especially given our current fast-paced society, some associations and practices make surveys a routine part of their day-to-day operations. That is, they offer each patient the opportunity to complete a short patient satisfaction survey following each visit. This is relatively easy and cost-efficient using today’s technology, and it is a vast improvement over past methods, which called for surveys of a few patients a few times a year, generally limited to paper surveys.

To give a concrete example, suppose a practice sees 20 patients a day. Over the course of a quarter, that would total up to about 1,200 patient visits (assuming the practice operates 20 days/month, or 60 days per quarter), or about 4,800 patient visits each year. To reach 95% confidence in your survey results with a 10% margin of error over each quarter, you would need 89 survey responses during the three months, or between one and two survey responses each day of practice operation, on average. If you want to achieve the same confidence level and margin of error, but over the whole year, then you would need 95 survey responses, or two responses per practitioner per week, on average.

Of course, if you’re willing to live with a lower level of confidence, say 90%, and/or a higher margin of error, say 15%, then the required number of responses in any time period can be lower. In practical reality, the thing to do is to set a goal for the number of responses, and then at the end of a survey period calculate the confidence level and margin of error achieved, using the number of responses you actually received and the number of patient visits.

One thing to keep in mind, though: the statistics above depend on the survey responses being independent. That is, if you see a patient more than once during a survey period, then that patient’s responses are likely dependent to some degree. It does not mean you should ignore the repeated surveys, but they need to be discounted somewhat. In other words, you will need somewhat more responses to truly reach the desired confidence level and margin of error. Offsetting this, a shift over time in one patient’s survey responses could have stronger implications for your practice.

In the end, your choice of survey timing and frequency depends on you and your view of your practice’s situation. From the standpoint of truly understanding patient satisfaction levels and trends, surveying more often is almost always better.

But if you’re going to survey more, what about cost and data collection?  The next section outlines some answers to these questions, which are necessarily related.

How Can I Get Patients to Respond to My Survey, While Controlling Costs?

Methods of getting survey responses fall roughly into three main categories:

1. Mail-in

2. Personal interview, either by telephone or in person at either the office or in home

3. Electronic, for example, by web or email survey

Mail-in and personal interview have always been the staples of the surveying art, as witness the political and marketing polls you see and read about (and sometimes feel assaulted by) in the media. They are still the only ways to reach certain demographics, but they have disadvantages, including often lower-than-desirable response rates and personnel-intensive data collection. These factors typically mean higher costs for the same quality of results. Moreover, response rates nation-wide to such surveys have generally declined in the last 30 years. See, for example, the 2000 University of Wisconsin paper on comparative responses to several survey forms:

http://jcmc.indiana.edu/vol6/issue1/yun.html

In contrast, electronic surveys are becoming increasingly common, partly because of the potential for lower costs, faster turnaround, more honest answers to sensitive topics and better response rates. For more details on the relative merits of these various methods of data collection, see the series of related articles at SuperSurvey dealing with survey forms.

http://knowledge-base.supersurvey.com/mail-vs-web-surveys.htm

Unfortunately, no one data collection method is likely to be as effective as we would like. Multi-mode methods will probably be needed for some time. Yet, we can encourage movement in the direction of improved response rates and lower costs with technology that is now emerging.

Consider, for instance, the medical office where a patient, who has just completed a visit and before leaving, is offered the opportunity to complete a patient satisfaction survey by any of the following:

1. a link (perhaps listed on the office business card or paperwork) to an online electronic survey that the patient can access at home;

2. a terminal or handheld device (like a PDA, for instance) that the office staff makes available to the patient on the spot to access the survey; or

3. a paper survey they can either fill out there and then or take with them to return later through the mail (using an included return envelope). Your staff can also mail it to them and request that they mail it back.

Other methods could be added, such as a personal or telephone interview, albeit at added expense to the office, to fit more comfortably with the particular office/practice situation.

However you choose to administer the survey, the second step is compiling the data and getting it to the decision maker(s). If a patient survey is mailed out twice a year, then the surveys that are returned can be tabulated and the results compared to earlier efforts. Depending on your field, six months between surveys can either be perfectly adequate or a problem to be addressed. If you hire a new receptionist in the first month after a survey is completed it will be five or six months before your survey results indicate that he/she is having a negative impact on patients (should that be the case).

With a survey that is taken just after a patient has been in the clinic—particularly if it is done electronically—many agencies that compile survey data can provide feedback to you on a daily basis. Another function that online services are providing that is increasingly popular is an alert function: the manager using the survey data can decide on a “normal” range of responses (say 3 to 5 on a five point scale); any time there is a rating below 3 an alert is immediately sent to the manager so the source of the discontent can be identified and resolved.

In summary, surveys are a powerful tool for keeping you in touch with your patients’ feelings. Newer technologies have the potential of giving you rapid access to results and expanding the information available to you. Don’t be daunted by what seems at first glance to be a difficult and costly task. Improved technology and creative use of it will make it much easier than you think.