Skip Navigation



BJA Advance Access published online on April 30, 2007
British Journal of Anaesthesia, doi:10.1093/bja/aem097
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow All Versions of this Article:
98/6/769    most recent
aem097v1
Right arrow E-Letters: Submit a response to the article
Right arrow E-letters: View responses
Right arrow Alert me when this article is cited
Right arrow Alert me when E-letters are posted
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Disclaimer
Google Scholar
Right arrow Articles by Duckitt, R. W.
Right arrow Articles by Forni, L. G.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Duckitt, R. W.
Right arrow Articles by Forni, L. G.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

© The Board of Management and Trustees of the British Journal of Anaesthesia 2007. All rights reserved. For Permissions, please e-mail: journals.permissions@oxfordjournals.org

Worthing physiological scoring system: derivation and validation of a physiological early-warning system for medical admissions. An observational, population-based single-centre study{dagger},{ddagger}

R. W. Duckitt1, R. Buxton-Thomas1, J. Walker1, E. Cheek3, V. Bewick3, R. Venn2,* and L. G. Forni2

1 Acute Medical Unit
2 Intensive Care Unit, Worthing Hospital, Lyndhurst Road, Worthing, West Sussex, BN112DH, UK
3 School of Computing, Mathematical and Information Sciences, University of Brighton, UK

* Corresponding author. E-mail: Richard.Venn{at}wash.nhs.uk

Accepted for publication March 7, 2007.


   Abstract
Top
 Abstract
Introduction
 Methods
 Results
 Discussion
 References
 
Background: Several physiological scoring systems (PSS) have been proposed for identifying those at risk of deterioration. However, the chosen specific physiological values chosen and the scores allocated have not been prospectively validated. In this study, we investigate the relative contributions of the ventilatory frequency, heart rate, arterial pressure, temperature, oxygen saturation, and conscious level to mortality in order to devise a robust scoring system. All data were collected on admission to the emergency unit. Precise ‘intervention-calling scores’ could then be derived to trigger interventions.

Methods: Our observational, population-based single-centred study took place in a 602-bedded district general hospital. Patients admitted to the emergency care unit at Worthing general hospital during an initial study period between July and November 2003 (n = 3184) and a further validation period between October and November 2005 (n = 1102) were included.

Results: Multivariate logistic regression analysis demonstrated that a ventilatory frequency ≥20 min–1, heart rate ≥102 min–1, systolic blood pressure ≤99 mm Hg, temperature <35.3°C, oxygen saturation ≤96%, and disturbed consciousness were associated with an increase in mortality. The Worthing PSS was developed from the regression coefficients associated with each variable. The model showed good discrimination with an area under the receiver operating characteristic curve, 0.74, excluding age as a variable. The discrimination of this system was significantly better than the early-warning scoring system.

Conclusions: A simple validated scoring system to predict mortality in medical patients with precise ‘intervention-calling scores’ has been developed.

Keywords: critical illness, mortality; health status indicators, scoring system; medical admission; outcome assessment, methods; prognosis


   Introduction
Top
 Abstract
 Introduction
Methods
 Results
 Discussion
 References
 
In 2000, the Department of Health1 (DoH) suggested the development of ‘outreach services’ to support, and physiological scoring systems (PSS) to identify, those patients requiring additional care to that presently available on normal wards in an acute hospital. Theoretically, PSS would identify the ‘sick’ hospital patient at the earliest opportunity. Outreach services would then be able to advise on additional care required to prevent deterioration and, therefore, avoid irreversible organ failure and mortality. This may require transferring the patient to a high dependency unit (HDU) or intensive care unit (ICU) facility, or it may only require simple measures to be implemented in the normal hospital ward with additional monitoring of the situation with the PSS.

Evidence already exists that outreach services and Medical Emergency Teams (MET) in conjunction with physiological-based scoring systems can improve patient's care, demonstrating a reduction in cardiac arrests, mortality, length of critical care stay, and ICU/HDU readmission.2–5 Recently, in the MERIT study,6 a large cluster randomized, controlled trial failed to demonstrate improved outcomes in Australian hospitals, which may reflect the low sensitivity of the MET scoring system used. In addition, only about 50% of patients had the MET criteria documented before an adverse event.

Several PSS have been developed and are in use in the UK, the majority based around the original early-warning scoring system (EWS) devised by Morgan and colleagues7 in 1997. Changes have included the monitoring of urine output and variation in the other physiological variables chosen.8 While these systems are useful in identifying those at risk of deterioration, the specific physiological values chosen and the scores allocated have not been prospectively validated.9 Moreover, the scores derived demonstrate poor specificity.1011 Furthermore, some of the variables used (e.g. temperature, heart rate, arterial pressure, and urine output) may not be of use in predicting deterioration and hospital mortality, as has been confirmed previously by ourselves.12 Additionally, other physiological data such as oxygen saturation, not currently used in some scoring systems, are better predictors of deterioration.12

Our aim was to investigate the relative contributions of the ventilatory frequency, heart rate, arterial pressure, temperature, oxygen saturation and conscious level, to mortality in order to devise a robust scoring system. Precise ‘intervention-calling scores’ could then be derived to trigger interventions such as increased frequency of physiological measurements, seeking senior help, the outreach team or both.


   Methods
Top
 Abstract
 Introduction
 Methods
Results
 Discussion
 References
 
This prospective, observational study was conducted in the emergency admissions unit (EAU) of Worthing General Hospital in two phases. Worthing hospital is a 602-bed district general hospital on the south coast of England with an annual A&E attendance of >65 000, and 30–35 acute medical admissions per 24-h period. Medical admissions are either directly from the local primary care physicians, outpatients or via the A&E department. Local research ethics committee approval was obtained.

Phase 1 of the study was carried out between July and November 2003 during which the initial data were collected. Phase 2 was carried out between October and November 2005, involving prospective validation of the Worthing PSS. In accordance with the Data Protection Act, posters were displayed in the EAU explaining the methods and that physiological data would be used anonymously for research purposes at a later date. A proforma was designed for Phase 1 and included in the admission package of all new general medical admissions to the EAU during the study period. Data were collected on all admissions to the EAU (at the time of the initial nursing assessment) using the proforma and included patient characteristics details and the initial routine physiological observations. The physiological variables recorded were:

Arterial pressure (measured with CAS Medical Systems 9301)
Heart rate
Oxygen saturation in air (SpO2) (measured with Nelcor N550)
Ventilatory frequency
Level of consciousness [defined as alert, responsive to verbal command, responsive to pain or unresponsive (AVPU score)].
There were no specific exclusion criteria, but incomplete proformas were not included in the statistical analysis. All patients were followed up to determine length of stay, survival to hospital discharge, and incidence of cardiac arrest, using the hospital's patient administration system and data from the resuscitation department. Completeness of the collected data was independently verified, and the data were anonymized. The data were entered onto a password-protected Excel© spreadsheet (Microsoft Corporation, Richmond, WA, USA) by the authors, and a first phase of statistical analysis was carried out at the School of Computing, Mathematical and Information Sciences, University of Brighton. These initial data were used to produce the Worthing PSS (Table 3).

Phase 2 involved a second period of data collection to validate prospectively the PSS derived from the Phase 1 data. As before, all admissions to the EAU were included and identical data collection procedures performed. Throughout the study, all therapeutic management was at the discretion of the attending physician.

Statistical analysis
Using each of the prognostic variables, ventilatory frequency, pulse, arterial pressure, temperature, oxygen saturation, and conscious level, a generalized additive model procedure13 was used to fit a logistic model with a non-parametric spline smoother to the binary outcome variable in-hospital mortality. Each prognostic variable was partitioned by identifying cut-off points using the method of O'Brien.14 This method minimizes a measure of distance between the true expected value of the outcome for each subject and the estimated average outcome among subjects in the same partition.

A multivariate logistic regression analysis was performed using the partitioned prognostic variables as predictors of mortality. The new scoring system was developed from the regression coefficients associated with each variable. The calibration of the scoring system was assessed by univariate logistic regression and the Hosmer–Lemeshow goodness-of-fit test.15 Discrimination was assessed using the area under the receiver operating characteristic curve (AUC). Validation was carried out by applying the scoring system to the second data set.

As both the initial and the second data sets were collected from the same centre, the original and validation sets were combined in order to compare post-hoc the new scoring system with the EWS,7 which has been recommended by the DoH.1 The AUCs were compared using the method of Hanley and McNeil.16

To identify possible ‘intervention-calling scores’, the percentage of deaths associated with each score was calculated. As mortality rates are usually related to age, the percentages of deaths by score for different age groups were compared. The age groups were determined using the same method that was used for the other prognostic variables.

Statistical analyses were performed using SPSS v.14. P < 0.05 was considered as significant.


   Results
Top
 Abstract
 Introduction
 Methods
 Results
Discussion
 References
 
In Phase 1, initial data were collected on 4384 patients and were complete for 3184 patients (2914 survived, 270 died). In Phase 2, the validation data set comprised data for 1261 patients, which were complete for 1102 (1017 survived, 85 died). Combining the two data sets gave scores for 4286 patients (3931 survived, 355 died) (Table 1).


View this table:
[in this window]
[in a new window]

  		Table 1 Baseline characteristics for the derivation and validation data sets. LOS, length of stay

 
The multivariate logistic regression analysis using the partitioned prognostic variables indicated that some of the categories were not significantly different so they were subsequently amalgamated. The final model (Table 2) showed good discrimination with an AUC of 0.74 (95% CI, 0.71–0.77). The Hosmer–Lemeshow goodness-of-fit test indicated satisfactory calibration (P = 0.119). The weightings for the resulting categories of each prognostic variable were obtained from the regression coefficients (Table 3). When the new scoring system was applied to the validation data set, the AUC was 0.72 (95% CI, 0.66–0.79) with a P-value of 0.565 for the goodness-of-fit test, indicating that it validated well. Incorporation of age into the final model, developed by multivariate logistic regression, improved the AUC to 0.81, but age was excluded from the Worthing PSS for simplicity, because it did not influence the derivation of the ‘intervention-calling scores’ described later.


View this table:
[in this window]
[in a new window]

  		Table 2 Results of the multiple logistic regressions. Odds ratios are calculated relative to a base category given in parentheses after each variable name

 


View this table:
[in this window]
[in a new window]

  		Table 3 The Worthing PSS

 
When the AUCs for the new scoring system and EWS were applied to the combined data set (Fig. 1), the discrimination of the new scoring system was significantly better than that of the EWS (P < 0.001). The cut-off point that gave the maximum combined sensitivity and specificity for both the new scoring system and the EWS was 3. The sensitivity and specificity of the new scoring system for this cut-off point were 0.63 and 0.72, respectively, which compare favourably with 0.60 and 0.67 for the EWS. In the combined data set, the scores ranged from 0 to 13, with a mean and SD of 1.8 and 2.0, respectively. Figure 2 shows the distribution of scores together with the percentage of deaths for each score. The median length of stay for the non-survivors was 7 days, with 31% of the deaths within 2 days and 88% within 30 days. Of those who died within 48 h of admission, 43% of these demonstrated a score of ≥6 on presentation to the EAU. In the younger age group (<55 yr), 44% of deaths occurred within 2 days of admission and 83% of deaths within 7 days. Cardiac arrest occurred in 41 patients within 7 days of admission, but complete data were available for only 26 of these patients. These cardiac arrest patients were older [mean (SD) 78.9 (14.3) vs 68.1 (18.9)] and had higher scores on admission (77% with score ≥5 vs 43%) in comparison with the remainder of the study population.


Figure 1
View larger version (23K):
[in this window]
[in a new window]
[Download PowerPoint slide]
  		Fig 1 Receiver operating characteristic curves for the new scoring system and EWS.

 


Figure 2
View larger version (11K):
[in this window]
[in a new window]
[Download PowerPoint slide]
  		Fig 2 Distribution of percentage of deaths (solid line, and left axis (with 95% CI)] and number of patients by new scoring system (blocks and right axis).

 
Analysis of the percentage deaths by score identified ‘intervention-calling scores’. A score of 2 was identified because, as shown in Figure 2, above that score the overall mortality increases >10%. This coincided with a sharp increase for the youngest (<55 yr) and oldest (≥85 yr) age groups (Table 4 and Fig. 3). The sensitivity and the specificity at this cut-off point were 0.78 and 0.57, respectively. The urgent calling score was identified as 5, because above that score the overall mortality increases >20% and thereafter increased rapidly. ‘Intervention-calling scores’ and interventions are illustrated in Table 5.


Figure 3
View larger version (12K):
[in this window]
[in a new window]
[Download PowerPoint slide]
  		Fig 3 Percentage deaths by score for each age group.

 


View this table:
[in this window]
[in a new window]

  		Table 4 Percentage (n) deaths by score for each age group. *Scores of 5 and above combined in these age groups because of low numbers of patients

 


View this table:
[in this window]
[in a new window]

  		Table 5 ‘Intervention-calling scores’ and interventions

 

   Discussion
Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
References
 
This study has identified the relative contribution of individual physiological markers to mortality and a simple scoring system for medical patients admitted to the EAU derived. As demonstrated, this scoring system performs better than the EWS and the age-modified EWS, recommended by the DoH, and reflects the accuracy of the modelling technique used in this study.1 Similarly, precise ‘intervention-calling scores’ were derived from analysis of data, and we would expect these to be more robust than those based upon expert opinion. The numbers of patients who would have been identified as a consequence of these derived ‘intervention-calling scores’ are shown in Figure 4. Our derived score has several major advantages over those currently used elsewhere. Firstly, it is much simpler than any other published scoring system to date. Secondly, it has been derived directly from the observed physiological variables from general medical patients. Thirdly, all three major demands necessary to construct a severity of illness scoring system were met: the study sample size (4286 patients) was appropriate, given the in-hospital mortality rate of 8.3%; the PSS was clinically validated using a second data set comprising subsequent patients from the same centre; thus, establishing that the system works satisfactorily for patients other than those from whose data it was derived;17 and discrimination was assessed using the AUC and the calibration of the PSS was assessed by the Hosmer–Lemeshow goodness-of-fit test.


Figure 4
View larger version (6K):
[in this window]
[in a new window]
[Download PowerPoint slide]
  		Fig 4 Number and percentage of patients based on our data highlighted by the ‘intervention-calling scores’.

 
When compared with other studies that have used more cumbersome scoring systems, some interesting observations become apparent. For example, the study by Subbe and colleagues18 using a modified form of EWS demonstrated that the systolic blood pressure was rarely associated with increased risk until <100 mm Hg, in keeping with our observations. Similarly, they report that the presence of pyrexia again rarely increased risk whereas relative hypothermia did, which again is consistent with our results. Although they allude to the greater risk associated with age, their weighting associated with age is unrealistic. Interestingly, application of our derived age weighting to their AUC curves improves the AUC considerably and it approaches the values we described. The study by Olsson and colleagues19 evaluated the predicted accuracy of the rapid emergency medicine score (REMS) in patients attending a non-selected accident and emergency department. As a consequence, the observed mortality is much lower than we described because of the difference in patient group. The REMS score has been developed by taking those elements of APACHE II that can easily be obtained in the emergency room. APACHE II in keeping with other systems such as SOFA is a validated scoring system derived from a select group of critically ill patients in intensive care and, as such, may not be applicable to such a diverse patient group. Despite this, they do demonstrate that both temperature and arterial pressure did not independently predict mortality in a multivariate analysis, in keeping with our findings. Again marked differences in age weighting were observed principally because of the use of the APACHE II age weighting, which differs considerably from our own. A more recent study examined the prognostic significance of the observed non-traumatic hypotension in emergency department patients.20 This demonstrated a significantly increased risk of in-hospital death in patients exposed to hypotension within the 24 h after admission. Exposure to hypotension was defined as a systolic blood pressure of <100 mm Hg that is in keeping with our observations. A further interesting observation from our data is the observed change in physiological variables demonstrated with age (Table 6). For example, the mean ventilatory frequency observed in those <55 yr was 18 min–1, whereas in those who are >85 it was approaching 20 min–1. In tandem with this, was an observed decrease in measured oxygen saturations from 97.2 to 94.8%. It can be postulated that this reflects a decrease in lung compliance in this patient population, but what it highlights is the paucity of data for ‘normal values’ when applied to a cross-section of patients including the elderly. This should be borne in mind when considering variance from oft-quoted physiological norms.


View this table:
[in this window]
[in a new window]

  		Table 6 Mean (SD) of prognostic variables for each age group

 
Although this is the first fully validated PSS described, our study is not without limitations. The relationship between the Worthing PSS and cardiac arrest remains uncertain since the numbers of complete admission data collected for this secondary outcome, were small. The scoring system is based on single, unvalidated measurements taken by ward staff on admission to the EAU, and as such may be prone to measurement and recording errors. However, this reflects routine medical practice on all UK medical assessment units and the data upon which much clinical decision-making are based, and therefore does not detract from the pragmatic conclusions of the study. Another limitation is that this is a single-centre study in an acute medical setting and before general application it needs to be validated in other emergency assessment units. Gardner-Thorpe and colleagues21 recently prospectively studied the Modified EWS (MEWS) in 334 surgical ward patients and derived a threshold score for HDU/ICU admission. Prospective validation of the Worthing PSS in surgical patients and using this outcome (HDU/ICU admission) is to be evaluated. However, the original aim of the study was to generate a locally applicable scoring system that reflects our acute medical patient population. Our previous work in A&E patients showed similar findings that imply a general applicability.12 The adoption of this scoring system throughout the medical wards may not be appropriate given that the data are generated at initial presentation and repeated measurements were not undertaken. Interestingly, our data demonstrate that a high percentage of deaths occur within 48 h of admission for those scoring 6 and above and that the majority of deaths in the younger age group occurred within 2 days. As such, this implies that the derived score is temporally related to mortality and therefore may indeed be more widely applicable. Any observational study of this type is limited in that it is not known whether intervention aimed specifically at normalizing these physiological variables will improve mortality. Both these latter limitations are currently being explored in an on-going interventional trial. A difference in mode and urgency of treatment in the EAU may be a confounder in the usefulness of any scoring system. Of course, we could not control for this, but given the large number in our study population we would not expect systematic bias as a consequence.

We have devised a simple, robust scoring system to predict mortality in medical patients admitted to the EAU, with precise ‘intervention-calling scores’. To our knowledge this is the first system derived from basic principles using established methodology for constructing a new severity of illness classification system, as opposed to expert medical opinion.


   Footnotes
 
{dagger} Janene Holway, Kathleen Durick and Pat Kerry (administrative support) participated in data entry. J.W. and the Nursing staff (Emergency Admissions Unit) collected all the patient data. R.B.T., R.D., R.V., and L.F. participated in data entry and collection. R.V., L.F., R.D., E.C., and V.B. contributed to the concept, design, and implementation of the study. All authors contributed to the manuscript and read and approved the final version. All authors will act as guarantors for the paper. Back

{ddagger} This article is accompanied by the Editorial. Back


   References
Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
1 Department of Health. Comprehensive Critical Care: a Review of Adult Critical Services (2000) London: Department of Health.

2 Ball C, Kirkby M, Williams S. Effect of the critical care outreach team on patient survival to discharge from hospital and readmission to critical care: non-randomised population based study. Br Med J (2003) 327:1014–7.[Abstract/Free Full Text]

3 Bristow PJ, Hillman KM, Chey T, et al. Rates of in-hospital arrests deaths and intensive care admissions: the effect of a medical emergency team. Med J Aust (2000) 173:236–40.[Web of Science][Medline]

4 Buist MD, Moore GE, Bernard SA, Waxman BP, Anderson JN, Nguyen TV. Effects of a medical emergency team on reduction of incidence and mortality from unexpected cardiac arrests in hospital: preliminary study. Br Med J (2002) 324:387–90.[Abstract/Free Full Text]

5 Bellomo R, Goldsmith D, Uchino S, et al. A prospective before-and-after trial of a medical emergency team. Med J Aust (2003) 179:283–7.[Web of Science][Medline]

6 MERIT STUDY. Introduction of the medical emergency team (MET) system: a cluster randomised control trial. Lancet (2005) 365:2091–7.[CrossRef][Web of Science][Medline]

7 Morgan RJM, Williams F, Wright MM. An Early Warning Scoring System for detecting developing critical illness. Clin Intens Care (1997) 8:100.

8 Department of Health, Modernisation Agency. Critical Care Outreach 2003: Progress in Developing Services (2003) London: Department of Health.

9 Goldhill DR. Preventing surgical deaths: critical care and intensive care outreach services in the postoperative period. Br J Anaesth (2005) 95:88–94.[Free Full Text]

10 de Pennington J, Laurenson J, Lebus C, Sihota S, Smith P. Evaluation of Early Warning Systems on a Medical Admissions Unit. JICS (2005) 6:19–21.

11 Cuthbertson BH. Outreach critical care-cash for no questions? Br J Anaesth (2003) 90:4–6.[Web of Science][Medline]

12 Hucker TR, Mitchell GP, Blake LD, et al. Identifying the sick: can biochemical measurements be used to aid decision making on presentation to the accident and emergency department. Br J Anaesth (2005) 94:735–41.[Abstract/Free Full Text]

13 Hastie T, Tibshirani R. Generalised Additive Models (1999) London: Chapman and Hall.

14 O'Brien S. Cutpoint selection for categorizing a continuous predictor. Biometrics (2004) 60:504–9.[CrossRef][Web of Science][Medline]

15 Hosmer D, Lemeshow S. Applied Logistic Regression (2000) 2nd Edn. New York: Wiley.

16 Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology (1983) 148:839–43.[Abstract/Free Full Text]

17 Altman DG, Royston P. What do we mean by validating a prognostic model? Statist Med (2000) 19a:453–73.[CrossRef]

18 Subbe CP, Kruger M, Rutherford P, Gemmel L. Validation of a modified Early Warning Score in medical admissions. QJ Med (2001) 94:521–6.[Abstract/Free Full Text]

19 Olsson T, Terent A, Lind L. Rapid Emergency Medicine score: a new prognostic tool for in-hospital mortality in non-surgical emergency department patients. J Intern Med (2004) 255:579–87.[CrossRef][Web of Science][Medline]

20 Jones AE, Yiannibas V, Johnson C, Kline-Jeffrey A. Emergency department hypotension predicts sudden unexpected in-hospital mortality: a prospective cohort study. Chest (2006) 130:941–6.[CrossRef][Web of Science][Medline]

21 Gardner-Thorpe J, Love N, Wrightson J, Walsh S, Keeling N. The value of Modified Early Warning Score (MEWS) in surgical in-patients: a prospective observational study. Ann R Coll Surg Engl (2006) 88:571–5.[CrossRef][Web of Science][Medline]


Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?


This article has been cited by other articles:


Home page
 Br J AnaesthHome page
M. Dosani, J. Lim, P. Yang, C. Brouse, J. Daniels, G. Dumont, and J. M. Ansermino
Clinical evaluation of algorithms for context-sensitive physiological monitoring in children
Br. J. Anaesth., May 1, 2009; 102(5): 686 - 691.
[Abstract] [Full Text] [PDF]

Home page
 Br J AnaesthHome page
R. J. M. Morgan and M. M. Wright
In defence of early warning scores
Br. J. Anaesth., November 1, 2007; 99(5): 747 - 748.
[Full Text] [PDF]

E-letters:

Read all E-letters

Worthing PSS: would removing the temperature variable help?
Pierre-Antoine Laloë
British Journal of Anaesthesia, 11 Jul 2007 [Full text]
Early Warning Scores
Stephen J Fletcher Guy W Glover
British Journal of Anaesthesia, 20 Jul 2007 [Full text]
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow All Versions of this Article:
98/6/769    most recent
aem097v1
Right arrow E-Letters: Submit a response to the article
Right arrow E-letters: View responses
Right arrow Alert me when this article is cited
Right arrow Alert me when E-letters are posted
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Disclaimer
Google Scholar
Right arrow Articles by Duckitt, R. W.
Right arrow Articles by Forni, L. G.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Duckitt, R. W.
Right arrow Articles by Forni, L. G.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?