BJA Advance Access originally published online on September 17, 2007
British Journal of Anaesthesia 2007 99(5):666-672; doi:10.1093/bja/aem259
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Patient Experiences with the Preoperative Assessment Clinic (PEPAC): validation of an instrument to measure patient experiences
1 Department of Anaesthesiology
2 Department of Medical Psychology, Academic Medical Centre, University of Amsterdam, PO Box 22700, 1100 DE Amsterdam, The Netherlands
* Corresponding author. E-mail: b.preckel{at}amc.uva.nl
Accepted for publication July 8, 2007.
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionAppendixAcknowledgementsReferences |
|---|
Background: Presently, no comprehensive and validated questionnaire to measure patient experiences of the preoperative assessment clinic (PAC) is available. We developed and validated the Patient Experiences with the Preoperative Assessment Clinic (PEPAC) questionnaire, which can be used for quantitative measurements of patient experiences of the PAC.
Methods: We adapted the National Health Service outpatient questionnaire, incorporating questions specific for anaesthesiology. To make the PEPAC appropriate for quantitative measurements, dimensions and single items suitable for statistical analysis were constructed. Each dimension consists of multiple items measuring the same aspect of care. Reliability was established by computing Cronbach's alpha coefficients. Construct validity was assessed by correlating the dimensions with the patient's overall appraisal (Pearson's r). These dimensions should explain a substantial level of variance of the patients’ overall appraisal; therefore, regression analysis was performed.
Results: After a pilot phase, the questionnaire was sent to 700 consecutive patients (response 74%). Five scales measuring five dimensions of patient experiences were constructed. Cronbach's alpha ranged from 0.56 to 0.84, supporting reliability of the PEPAC. Correlations between the dimensions and patients' overall appraisal ranged from 0.22 to 0.56. Collectively, the five scales explained 51% of patients' overall appraisal.
Conclusions: The PEPAC is a comprehensive, reliable, and validated questionnaire to measure patient experiences with the PAC. It might be a useful tool to identify the service areas of the PAC that require improvement and to determine which actions can bring about improvement.
Keywords: anaesthesia, audit; assessment, preanaesthetic; research, anaesthesia; screening; surgery, preoperative period
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAppendixAcknowledgementsReferences |
|---|
Progress in surgical and perioperative anaesthetic care and the consequent introduction of day-case surgery and same-day admission have led to the development of the outpatient preoperative assessment clinic (PAC). Previous studies have shown that the PAC improves hospital cost-efficiency.1–5 Though PACs have now been implemented in the organizational structure of most major hospitals, patients’ experiences and satisfaction with the PAC have been little studied. Patient satisfaction can be used as an indicator for the quality of health care.6 Information on patients' views and priorities can be used to improve the quality of health care services: their feedback can help identify the areas that are performing well and areas that would benefit from improvement.
In the UK, it has become common practice to perform surveys to obtain patients' views on their health care experiences. The Commission for Healthcare Audit and Inspection carries out national surveys of the National Health Service (NHS), producing detailed patient feedback on standards of service and care at both an institutional and national level. Repeating a survey enables longitudinal comparisons. A national survey of outpatients was performed in 2003 and repeated in 2004/2005. This NHS outpatient questionnaire was developed by Picker Institute Europe and is composed of report style questions.
Although some validated questionnaires to measure perioperative satisfaction, i.e. satisfaction with anaesthesia care and pain management, have been published,7–10 no comprehensive and validated questionnaire to explicitly measure patient experience of and satisfaction with the PAC is available at present. Only Hepner and colleagues11 have measured patient satisfaction with preoperative assessment at the PAC. Their questionnaire consisted of 18 report style questions, and so was quite concise.
The aim of the present study was to construct and validate the Patient Experiences with the Preoperative Assessment Clinic (PEPAC) questionnaire.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAppendixAcknowledgementsReferences |
|---|
Development of the PEPAC questionnaire
According to Dutch regulations, no formal approval is required for research with non-invasive interventions, such as this questionnaire. The hospital's Medical Ethics Committee agreed that our work was covered by this dispensation. We chose objectively formulated report style questions, where patients are asked to report their experiences in detail rather than subjectively rate their satisfaction.
We took the NHS outpatient questionnaire12 as our starting point, selecting those questions applicable to the PAC. The items were translated into Dutch by three independent translators. The translators were Dutch native speakers with a good knowledge of the English language and experience in both languages and cultures. Two of the translators are physicians, one of them being an anaesthetist with the third translator having experience in communication research and test development. The translations were compared and then combined to produce a final translation. This was then verified by means of a back translation: a different translator, an experienced, native English person working in the medical field, and familiar with the Dutch culture, translated the questionnaire back into English after which a psychologist, with extensive knowledge of test development and a good knowledge of the English language, compared the original questionnaire with the back translation of the questionnaire. The questions on patient characteristics were adapted to the Dutch system. Some questions were made specific for the PAC, e.g. the word ‘doctor’ was replaced with ‘anaesthetist’. Where appropriate, questions were reformulated to ascertain that the reading level was the same as in English, while making sure the essential meaning of the questions did not change.
In addition, specific questions for anaesthesiology and the PAC were developed in cooperation with an anaesthetist. These questions were partly based on previous questionnaires.11 13 However, the questions were transformed from a Likert scale,14 i.e. asking patients to indicate the extent of their agreement or disagreement, to a statement, to report style.
Finally, a scale to measure patients' overall appraisal of the PAC was constructed. This served as a criterion measure such as is used in validation research. The scale consisted of seven items. Three questions were drawn from the Satisfaction with Hospital Care Questionnaire, derived from Visser15 16 and validated by Hendriks and colleagues.17 These questions established the willingness of a patient to come back to our PAC18 and whether they would recommend our PAC to friends or family19 in addition to a question on the patient's expectations. Four questions were derived from the NHS questionnaire on satisfaction: satisfaction with the preoperative assessment, organization of the PAC, respectful treatment by staff at the PAC, and rating the PAC.
The questions were numbered, with those relating to similar issues grouped together, and placed in chronological order based on the usual route a patient would take through the PAC. The questionnaire was printed as an A4-booklet.
A pilot study was performed. Face-to-face interviews were conducted with a sample of 11 patients. The sample included patients of different age, ethnic backgrounds, and levels of education, representative of the patient population of our PAC. The pilot study contained 12 questions relating to the questionnaire itself, including questions on the length and comprehensibility of the questionnaire, the ease of completion and questions to identify any missing or superfluous items (Table 1). As a result of the pilot study, some questions were rephrased to make them more comprehensible. The pilot study showed no missing or superfluous items; therefore, no questions were added or omitted. The patients of the pilot study were not included in the analysis. The final questionnaire, ‘PEPAC’, included 72 items with multiple response options. The response options were not uniform, but tailored to the report style of the questions. At the end of the PEPAC, there was room for comments.
|
Validation study
The validation study was performed at the Academic Medical Centre, Amsterdam, The Netherlands in May and June of 2006. Annually, 12 000 patients are assessed at the PAC of this tertiary care centre. The PEPAC was sent to consecutive patients who visited the PAC; patients from all surgical specialties were included. Children under the age of 16 were excluded from the study. A poster at the PAC informed patients that they would receive a questionnaire, which was sent to the patient within 2 days of their visit to the PAC together with a covering letter and freepost return envelope. The questionnaires were numbered to allow non-blinded follow up of the response, but no personal details were identifiable on the questionnaire. After 2 weeks, we contacted the non-respondents to remind them of the questionnaire and ask for their participation, and another reminder letter was sent after an additional week.
Sample size considerations were based on the intended reliability analysis. Assuming four items per dimension, analysis showed that 420 patients needed to be included in the study for a Cronbach's alpha with a 95% confidence interval with a lower bound of 0.65 if the point estimate turns out to be 0.70.
The scores were entered into a Microsoft Office-XP Access database. SPSS 12.0.1 for Windows was used for statistical analysis.
Patient characteristics and missing values are described using descriptive statistics.
After analysing the internal structure of the PEPAC, dimension scales were constructed, where each dimension scale consists of multiple items that measure the same care aspect (see Appendix). Dimensions were chosen on the basis of substantive consideration, using the original groups of items in the questionnaire and clustering some groups. Subsequently, the dimensions were tested using principal components analysis and reliability analysis.
The patients answered report style questions. To allow statistical analysis, we scored the response options from 0 to 100, depending on the extent to which the patients' experience could have been better. One hundred represented the most positive experiences and zero the most negative experiences. Patients' dimension scores were constructed by averaging the item scores. If less than one third of the items were missing, the missing items were substituted by the mean of the items that were answered.
The reliability is the extent to which the questionnaire gives consistent and reproducible results. Reliability of the dimension scores was established by computing internal consistency. All items of a multi-item scale should measure the same aspect of the patient's experience. This was determined using Cronbach's alpha coefficient20 with a 95% confidence interval (95% CI).
The validity is the extent to which the questionnaire measures the aspects of the patient's experience that it was designed to measure. First, content validity was established by showing that all items of the PEPAC are a sample of the subject of interest, i.e. the PAC, and that the PEPAC represents all facets of the subject of interest. Four anaesthetists, a psychologist experienced in quality of care research, and the 11 patients from the pilot study evaluated the questionnaire to ensure it contained all the factors important to the PAC and especially the preoperative assessment itself. A pilot study was performed to make sure that all items of importance to the patient were included and to identify questions that were ambiguous, difficult to answer, or unclear.
The internal structure was verified by (a) principal component analyses and (b) internal item consistency, by calculating correlations between items and their corresponding dimension scores.
To assess the construct validity of our questionnaire, we correlated the specific dimensions with the patient's overall appraisal of the PAC (Pearson's r). It is hypothesized that the different dimensions should be reflected in the patient's overall appraisal of the care given. Therefore, at least a moderate (> 0.20) correlation between dimensions and overall appraisal is expected. A P < 0.05 was considered statistically significant. We also performed regression analysis to assess how much of the variance of the patients' overall care appraisal is explained by all the dimensions of patients' experiences, that is to say we performed a regression analysis in which the patients' overall care appraisal was the dependant variable and the dimensions of the patients' experiences were independent variables. The standardized regression coefficient was used to allow comparison of the relative importance of the different dimensions.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAppendixAcknowledgementsReferences |
|---|
The questionnaire was sent to 700 patients, of whom 519 (74%) returned it duly completed. The male:female ratio was 44:56 and mean(range) of patient age was 51(17–87) years.
Educational level varied from none (3%), primary education (18%), secondary education (47%) to college or university (26%); 3% of the respondents did not state their educational level. Most respondents were Dutch (78%), 9% were Surinamese, 4% were either of Turkish, Moroccan, Antillean, or African extraction, 6% had another nationality and 3% did not state their ethnic background. Patients rated their own health as excellent (7%), very good (14%), good (54%), fair (20%) to poor (5%); 2% did not rate their health.
The male:female ratio of the non-respondents was 47:53 and mean of patient age was 44(16–92) years. A patient's American Society Anesthesiologists' (ASA) physical status had little influence on participation. Day case surgery patients were slightly less prepared to participate in comparison to inpatients. Mean time to complete the PEPAC was 15 (SD 8) min.
Five dimension scales were constructed: reception (3 items), waiting (6 items), the nurse (5 items), the anaesthetist (20 items), and other questions (15 items). Three out of five specific dimension scales and the overall appraisal scale had a good internal consistency (Cronbach's 
> 0.7) (Table 2). Internal consistency was moderate for the dimension ‘the nurse’ (5 items, = 0.66) and low for ‘other questions’ (15 items, = 0.56).
|
The five dimensions correlated positively with patient's overall appraisal; correlations ranged from 0.22 to 0.56 (P < 0.01) (Table 3). Table 4 shows both standardized and unstandardized regression coefficients, and the 95% confidence intervals for the five dimension scales and the patient's overall appraisal. All dimensions contributed to the patient's overall appraisal; reception contributed most and waiting contributed least. Together, the dimensions explained 51% of the variance of the patients' overall care appraisal (Table 4). The mean (SD) dimension scores ranged from 49 (19) for ‘waiting’ to 91 (10) for ‘the nurse’; the mean overall appraisal score was 78 (16) (Table 2). The number of patients excluded from the calculation of the dimension scores varied from 6 (1%) for ‘reception’ to 48 (10%) for ‘the nurse’. Not all patients were seen by a nurse, explaining the relatively high number of missing values for this dimension.
|
|
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAppendixAcknowledgementsReferences |
|---|
Patients' experiences of the PAC have been little studied. The aim of this study was to construct and validate the PEPAC: an in-depth questionnaire, with objectively formulated report style questions, to measure patient experiences of the PAC. We adapted the NHS outpatient questionnaire. Our study had a large sample size and a high response rate (74%). Our results show that the PEPAC is a reliable and valid instrument, which can be used to obtain detailed patient feedback on standards of service of the PAC.
Limitations of the study
The PEPAC is quite long (72 items), but this did not affect the response rate, which was high. In their analysis of 210 published patient satisfaction studies, Sitzia and Wood21 found that the response rate is not correlated to questionnaire length. The 18-item questionnaire Hepner and colleagues11 used consists of general questions (6 items), questions about the patient's visit with the anaesthesia care provider (3 items), nurse or nurse practitioner (4 items), and laboratory technician (2 items) and questions on patient's overall satisfaction (3 items). It gives an indication of the service areas than can be improved, but the questionnaire is too concise to indicate how improvement can be brought about. The PEPAC is an in-depth questionnaire: the specific questions produce a detailed and factual account of patients’ experiences with the PAC. It provides insight into the service areas that are not performing well and it gives an indication of how improvement can be achieved.
The PEPAC is largely based on the NHS outpatient questionnaire, which was developed by Picker Institute Europe. The questions to measure patients' experiences used in the PEPAC are similar to those used in Picker surveys, which have been performed in several European countries and the USA.22–24 The questions specific for anaesthesiology include aspects also addressed in the questionnaires by Hepner and colleagues11 and Fung and Cohen.13 Only a few questions on patient characteristics had to be adapted to the Dutch system. Therefore, though the PEPAC was validated within one institution, the usability of the PEPAC is not limited to the Dutch healthcare system.
Discussion of the results
We decided to deviate from the most commonly used type of patient satisfaction questionnaire with rating style questions (e.g. How satisfied were you with the PAC?) as evaluations tend to be mostly positive.25 26 This may be influenced by the patient's expectations27 or feelings of gratitude.28 When the service is rated poorly, rating style questions provide little insight on how to improve the service. General evaluations do not propose actions to improve the quality of care.29 Jenkinson and colleagues30 recommend a questionnaire with detailed questions about specific aspects of patients' experiences, as they give a more complete and meaningful account than measuring satisfaction. The results of report style questions (e.g. Were you told how long you would have to wait?) are more reliable and can be easily interpreted and acted upon. Therefore, we opted for a questionnaire consisting of report style questions. However, report style questions about experiences generally have lower internal consistency than Likert type items with questions about attitudes, because the latter reflect satisfaction as a unidimensional patient characteristic, whereas the former also reflect separate PAC characteristics. Nevertheless, for our questionnaire all dimensions were sufficiently reliable with Cronbach's varying from 0.56 to 0.84. The rather low internal consistency for the dimensions ‘other questions’ ( = 0.56) can be explained by the fact that these questions are specific and do not necessarily have a common denominator.
Non-response can result in some bias of the results. The demographic data available for the non-respondents were age and gender. In their meta-analysis, Hall and Dornan31 found that there is only a very small relationship between patient sociodemographic characteristics and satisfaction with medical care, with age being the strongest correlate of satisfaction. The respondents in our study were found to be slightly older than the non-respondents. As older patients tend to be somewhat more satisfied about care,31 non-response may have resulted in somewhat more positive scores in our study. In addition, non-response may have caused a restriction of range, negatively affecting Cronbach's alphas and correlation coefficients. However, given that our response rate is good (74%) we do not believe that such bias had a substantial effect on our conclusions.
To make the questionnaire appropriate for more broad comparisons of patient experiences it was necessary to form dimensions. Not all items of the questionnaire were included in these dimensions, as not all items were suitable for this purpose. The questions that only applied to a small group of patients, e.g. questions on additional testing and consulting another physician, were excluded (five items). However, these questions are useful to help determine the standard of quality of care on this topic and therefore are kept in the final questionnaire. Items addressing patients' preferences (four items) and acceptable waiting times (two items) cannot be given an item score as they do not involve positive or negative experiences. However, these items can help set targets and therefore were also kept in the final questionnaire.
The PEPAC is a comprehensive, reliable, and validated questionnaire to measure patient experiences with the PAC. It can be used to determine the service level of the PAC in a single institute, as well as to compare service levels across institutions. The PEPAC can also help determine specific service areas of the PAC that require improvement, and thus to choose priorities for improvement. By using report style questions, the PEPAC helps identify actions appropriate to bring about improvement.
|
Appendix |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAppendixAcknowledgementsReferences |
|---|
|
Acknowledgements |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAppendixAcknowledgementsReferences |
|---|
We are grateful to the Picker Institute Europe, Oxford, UK, for their willingness to share the NHS outpatient questionnaire. Support was provided solely from institutional and departmental sources.
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAppendixAcknowledgementsReferences |
|---|
1 Starsnic MA, Guarnieri DM, Norris MC. Efficacy and financial benefit of an anesthesiologist-directed university preadmission evaluation center. J Clin Anesth (1997) 9:299–305.[CrossRef][Web of Science][Medline]
2 Ferschl MB, Tung A, Sweitzer B, Huo D, Glick DB. Preoperative clinic visits reduce operating room cancellations and delays. Anesthesiology (2005) 103:855–9.[CrossRef][Web of Science][Medline]
3 Fischer SP. Development and effectiveness of an anesthesia preoperative evaluation clinic in a teaching hospital. Anesthesiology (1996) 85:196–206.[CrossRef][Web of Science][Medline]
4 Pollard JB, Zboray AL, Mazze RI. Economic benefits attributed to opening a preoperative evaluation clinic for outpatients. Anesth Analg (1996) 83:407–10.[CrossRef][Web of Science][Medline]
5 Van Klei WA, Moons KG, Rutten CL, et al. The effect of outpatient preoperative evaluation of hospital inpatients on cancellation of surgery and length of hospital stay. Anesth Analg (2002) 94:644–9.
6 Fung D, Cohen MM. Measuring patient satisfaction with anesthesia care: a review of current methodology. Anesth Analg (1998) 87:1089–98.
7 Moerman N, van Dam FS, Muller MJ, Oosting H. The Amsterdam Preoperative Anxiety and Information Scale (APAIS). Anesth Analg (1996) 82:445–51.[Abstract]
8 Dexter F, Aker J, Wright WA. Development of a measure of patient satisfaction with monitored anesthesia care: the Iowa Satisfaction with Anesthesia Scale. Anesthesiology (1997) 87:865–73.[CrossRef][Web of Science][Medline]
9 Auquier P, Pernoud N, Bruder N, et al. Development and validation of a perioperative satisfaction questionnaire. Anesthesiology (2005) 102:1116–23.[CrossRef][Web of Science][Medline]
10 Heidegger T, Husemann Y, Nuebling M, et al. Patient satisfaction with anaesthesia care: development of a psychometric questionnaire and benchmarking among six hospitals in Switzerland and Austria. Br J Anaesth (2002) 89:863–72.
11 Hepner DL, Bader AM, Hurwitz S, Gustafson M, Tsen LC. Patient satisfaction with preoperative assessment in a preoperative assessment testing clinic. Anesth Analg (2004) 98:1099–105.
12 Reeves R. Outpatients Core Questionnaire. (2004) Oxford: Picker Institute Europe. [cited 2007 Jun 8]. Available from http://www.nhssurveys.org/docs/Outpatient_2005_CoreQuestionnaire_V4.pdf.
13 Fung D, Cohen M. What do outpatients value most in their anesthesia care? Can J Anaesth (2001) 48:12–9.[Web of Science][Medline]
14 Likert R. A technique for the measurement of attitudes. Arch Psychol (1932) 140:1–55.
15 Visser AP. Research on Hospital Inpatients’ Satisfaction with Care (in Dutch) (1988) Lochum: De Tijdstroom.
16 Visser AP. The Experience of Staying in Hospital (in Dutch) (1984) Assen: Van Gorcum.
17 Hendriks AA, Oort FJ, Vrielink MR, Smets EM. Reliability and validity of the Satisfaction with Hospital Care Questionnaire. Int J Qual Health Care (2002) 14:471–82.
18 Doering ER. Factors influencing inpatient satisfaction with care. Qual Rev Bull (1983) 9:291–9.
19 Abramowitz S, Cote AA, Berry E. Analyzing patient satisfaction: a multianalytic approach. Qual Rev Bull (1987) 13:122–30.
20 Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika (1951) 16:297–334.[CrossRef][Web of Science]
21 Sitzia J, Wood N. Response rate in patient satisfaction research: an analysis of 210 published studies. Int J Qual Health Care (1998) 10:311–7.
22 Cleary PD, Edgman-Levitan S, Roberts M, et al. Patients evaluate their hospital care: a national survey. Health Aff (Millwood) (1991) 10:254–67.[Medline]
23 Cleary PD, Edgman-Levitan S, Walker JD, Gerteis M, Delbanco TL. Using patient reports to improve medical care: a preliminary report from 10 hospitals. Qual Manag Health Care (1993) 2:31–8.[Medline]
24 Coulter A, Cleary PD. Patients’ experiences with hospital care in five countries. Health Aff (Millwood) (2001) 20:244–52.
25 Fitzpatrick R, Hopkins A. Problems in the conceptual framework of patient satisfaction research: an empirical exploration. Sociol Health Illn (1983) 5:297–311.[CrossRef][Medline]
26 Hall JA, Dornan MC. Meta-analysis of satisfaction with medical care: description of research domain and analysis of overall satisfaction levels. Soc Sci Med (1988) 27:637–44.[CrossRef][Web of Science][Medline]
27 Thompson AG, Sunol R. Expectations as determinants of patient satisfaction: concepts, theory and evidence. Int J Qual Health Care (1995) 7:127–41.
28 Sitzia J, Wood N. Patient satisfaction: a review of issues concepts. Soc Sci Med (1997) 45:1829–43.[CrossRef][Web of Science][Medline]
29 Cleary PD, Edgman-Levitan S, McMullen W, Delbanco TL. The relationship between reported problems and patient summary evaluations of hospital care. Qual Rev Bull (1992) 18:53–9.
30 Jenkinson C, Coulter A, Bruster S, Richards N, Chandola T. Patients’ experiences and satisfaction with health care: results of a questionnaire study of specific aspects of care. Qual Saf Health Care (2002) 11:335–9.
31 Hall JA, Dornan MC. Patient sociodemographic characteristics as predictors of satisfaction with medical care: a meta-analysis. Soc Sci Med (1990) 30:811–8.[CrossRef][Web of Science][Medline]
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  G. M. Edward, J. C. J. M. de Haes, F. J. Oort, L. C. Lemaire, M. W. Hollmann, and B. Preckel Setting priorities for improving the preoperative assessment clinic: the patients' and the professionals' perspective Br. J. Anaesth., March 1, 2008; 100(3): 322 - 326. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. M. Edward, S. F. Das, S. G. Elkhuizen, P. J. M. Bakker, J. A. M. Hontelez, M. W. Hollmann, B. Preckel, and L. C. Lemaire Simulation to analyse planning difficulties at the preoperative assessment clinic Br. J. Anaesth., February 1, 2008; 100(2): 195 - 202. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
T. Heidegger, M. Nuebling, D. Saal, and G. Kreienbuhl Patient-centred outcomes in clinical research: does it really matter? Br. J. Anaesth., January 1, 2008; 100(1): 1 - 3. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||