BJA Advance Access originally published online on March 24, 2006
British Journal of Anaesthesia 2006 96(5):590-596; doi:10.1093/bja/ael064
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
A comparison of bispectral index and entropy monitoring, in patients undergoing embolization of cerebral artery aneurysms after subarachnoid haemorrhage
Department of Anaesthesia, Critical Care and Pain Medicine, Western General Hospital Crewe Road, Edinburgh EH4 2XU, Scotland, UK
*Corresponding author. E-mail: douglas.duncan{at}btinternet.com
Accepted for publication February 8, 2006.
 |
Abstract |
|---|
 Top Abstract IntroductionPatients and methodsResultsDiscussionReferences |
|---|
Background. Processed EEG monitoring of anaesthetic depth could be useful in patients receiving general anaesthesia following subarachnoid haemorrhage. We conducted an observational study comparing performance characteristics of bispectral index (BIS) and entropy monitoring systems in these patients.
Methods. Thirty-one patients of the World Federation of Neurosurgeons grades 1 and 2, undergoing embolization of cerebral artery aneurysms following acute subarachnoid haemorrhage, were recruited to have both BIS and entropy monitoring during general anaesthesia. BIS and entropy indices were matched to clinical indicators of anaesthetic depth. Anaesthetists were blinded to the anaesthetic depth monitoring indices. Analysis of data from monitoring devices allowed calculation of prediction probability (PK) constants, and receiver operating characteristic (ROC) analysis to be performed.
Results. BIS and entropy [response entropy (RE), state entropy (SE)] performed well in their ability to show concordance with clinically observed anaesthetic depth. PK values were generally high (BIS 0.966–0.784, RE 0.934–0.663, SE 0.857–0.701) for both forms of monitoring. ROC curve analysis shows a high sensitivity and specificity for all monitoring indices when used to detect the presence or absence of eyelash reflex. Area under curve for BIS, RE and SE to detect the absence or presence of eyelash reflex was 0.932, 0.888 and 0.887, respectively. RE provides earlier warning of return of eyelash reflex than BIS.
Conclusion. BIS and entropy monitoring perform well in patients who receive general anaesthesia after good grade subarachnoid haemorrhage.
Keywords: anaesthetics i.v., propofol; analgesics opioid, remifentanil; complications, haemorrhage, subarachnoid; monitoring, bispectral index; monitoring, entropy
|
Introduction |
|---|
|
TopAbstractIntroductionPatients and methodsResultsDiscussionReferences |
|---|
Anaesthetic depth monitoring using processed EEG variables is gaining popularity in mainstream anaesthetic practice. Bispectral index (BIS)1 offers the potential to reduce intraoperative awareness with explicit recall.2 3 As calibration of these monitors was performed on subjects with normal cerebral function,4 our objective was to assess how these monitors perform in patients with known cerebral insult.
Embolization of cerebral artery aneurysms using interventional radiological techniques has revolutionized the management of subarachnoid haemorrhage (SAH) in recent years.5 6 Most centres perform these procedures under general anaesthesia and in our centre, all are performed under general anaesthesia with muscle relaxation.
Issues specific to anaesthesia for neuroradiological procedures of this type include:7
- The patient must not move under any circumstances.
- Catastrophic intracerebral events related to embolization of aneurysms occasionally occur whilst underanaesthesia.
- Access to the patient for the anaesthetist is limited.
- The degree of surgical stimulus is probably very low.
- Early assessment of cerebral function may be desirable after the procedure.
These points make the possibility of a reliable depth of anaesthesia monitor in brain injured patients receiving general anaesthesia appealing.
|
Patients and methods |
|---|
|
TopAbstractIntroductionPatients and methodsResultsDiscussionReferences |
|---|
We obtained approval from our local ethics committee to recruit patients for inclusion in this study. Individual informed consent was obtained from patients after SAH. All patients had the procedure of embolization of cerebral artery aneurysm performed under general anaesthesia using muscle relaxation.
BIS is a commercially available system that uses several analytical methods to assess EEG changes related to anaesthetic agent administration.8 These include Burst Suppression Analysis (BSR and QUAZI suppression), Fast Fourier Transformation (Beta Ratio analysis within 11–47 Hz) and Bispectral Analysis (SynchFastSlow analysis within 0.5–47 Hz). BIS has undergone several software developments since its initial release.9 BIS uses a 100 point index score where values close to 100 indicate the awake state, and values close to 0 indicate very deep general anaesthesia. The manufacturer's recommended range on this scale for general anaesthesia is between 40 and 60.
Entropy monitoring is a method of processing EEG signals that relies on entropy theory as first described by Shannon.10 After signal analysis and artifact removal a spline function is used to produce entropy values in the range 0–91 for state entropy, or 0–100 for response entropy.11 The entropy indices use the frequency ranges that relate to EEG activity that is 0.8–32 Hz for SE, and EEG plus EMG 0.8–47 Hz for RE. Entropy indices have been shown to correlate with anaesthetic depth for a variety of anaesthetic techniques.12–15
The EEG signal was collected with two disposable sensors. An Entropy Sensor (Datex-Ohmeda, Helsinki, Finland) was used for collecting entropy, and a BIS-XP Sensor (Aspect Medical Systems, Newton, MA, USA) for BIS. Monitoring was with Datex-Ohmeda M modules (Datex-Ohmeda, Helsinki, Finland) for both M-Entropy (version 01, Automatic Sensor check ‘on’) and M-BIS (software version 3.12, Automatic Sensor Check ‘on’, Smoothing Rate 15 s). After preparation of the skin using alcohol solution the sensors were placed in close proximity. The temporal electrode for BIS was placed to the right, the temporal electrode for entropy was placed to the left. For the entropy sensor electrode number 1 was placed at the centre of the forehead approximately 4 cm above the nose. Electrode number 1 for BIS was placed in the centre of the forehead approximately 5 cm above the nose. The positions for electrode 1 were as per the respective manufacturers' instructions. Electrode impedances were considered acceptable if below 7.5 and 10 kOhm for entropy and BIS, respectively. Sensor cables were directed over the top of the head and cables routed to either the left or right side of the head. The anaesthetist was blinded to the entropy and BIS readings by removing them from the monitor screen (Datex-Ohmeda S/5 Anaesthesia Monitor, Datex-Ohmeda, Helsinki, Finland).
Patients had radial arterial monitoring sited before induction of anaesthesia. Other parameters recorded included ECG, pulse oximetry (
) and end tidal gas analysis including carbon dioxide (
). All values from the anaesthetic monitoring (including BIS and entropy scores) were recorded every 5 s to a personal computer using Datex-Ohmeda S/5 CollectTM software.
Anaesthesia was induced after pre-oxygenation using propofol target controlled infusion (TCI) and remifentanil by infusion. All patients required tracheal intubation. Muscle relaxation was achieved using Atracurium. Degree of muscle relaxation and return of muscle function was determined by monitoring train of four (TOF). A peripheral nerve stimulator (Rutter RS6, Nikomed Ltd, Hampshire, UK) was placed over the Ulnar nerve at the wrist for TOF monitoring. Muscle relaxation was antagonized after the end of the radiological procedure. Lightening of anaesthesia was attempted after reversal of muscle relaxation. Maintenance of anaesthesia was by propofol TCI infusion, remifentanil infusion and atracurium infusion. Anaesthetists were allowed to administer anaesthetic drugs at their own discretion.
Clinical endpoints were noted during anaesthesia, these points and their definitions are shown in Table 1. These endpoints were taken when the patient was awake before administration of any sedative drugs (AW), at the time of loss of eyelash reflex and response to verbal stimulus (LOC),16 17 60 s prior to intubation (INT), during stable anaesthesia (SA) as defined by the period immediately before femoral artery cannulation, at the return of eyelash reflex (RR) and at the time of extubation (EXT). The timing of the endpoints was noted manually by the investigator and subsequently related to data recorded from the monitoring devices. The eyelash reflex was repeatedly tested along with response to verbal stimulation until both were absent. At the end of the procedure during the lightening phase, response to verbal stimulation and eyelash reflex were repeatedly tested until present.
|
BIS and entropy monitoring was stopped 15 min after tracheal extubation.
Statistical analysis
PK is a measure of the degree of concordance between two sets of ordinal data, that is clinically observed anaesthetic depth and anaesthetic depth monitor output.18 PK has a possible range of 0–1. A PK value of 0.5 indicates the monitor does not distinguish between the two anaesthetic depths, showing neither concordance, nor discordance with observed depth. A PK value of 1.0 indicates ideal concordance with no overlap between one clinical state and another. PK values have been widely used to describe the performance of anaesthetic depth monitoring systems.19–21
PK values were calculated using PKMACRO, a Microsoft Excel spreadsheet, as described by Smith and colleagues.18 Comparison of PK was performed by ANOVA with Bonferroni correction for multiple comparisons.
Calculation of PK for BIS and entropy was performed for the transition from awake through loss of consciousness (LOC) to intubation that is ‘PK deepen’. We also calculated PK values for the transition from stable anaesthesia through return of eyelash reflex to extubation that is ‘PK lighten’. In addition PK values were calculated for the component transitions between each of these states. These were defined as; PK AW vs LOC—the transition from awake state to the point of LOC (eyelash reflex and verbal response), PK LOC vs INT—the transition between the point of LOC and tracheal intubation, PK SA vs RR—the transition from stable anaesthesia to the return of the eyelash reflex during emergence and PK RR vs EXT—the transition between the return of the eyelash reflex during emergence and tracheal extubation.
The BIS and entropy data used to calculate PK for each of these transitions was a mean value taken over a 15 s period starting 15 s after confirmation of reaching each clinical state.
Receiver operating characteristic (ROC) curve analysis is a technique used to assess the overall ability of a ‘test’ to differentiate between normal and abnormal populations of results.22 23 The ‘tests’ investigated were BIS, RE and SE. The difference between the two populations was the presence or absence of eyelash reflex. Data were sampled from time periods representing ‘eyelash reflex present’ that is awake and up to loss of eyelash reflex during induction, and after return of eyelash reflex during emergence. The ‘eyelash absent’ time period was sampled during emergence from anaesthesia after the reversal of muscle relaxation until the return of eyelash reflex. ROC analysis and comparison of ROC curves was performed using Medcalc® for Windows v8.1.0.0.
Opportunity was given to the radiologists involved to suggest movement or removal of sensors if the sensors or their cables affected angiographic views.
Severity of cerebral injury was defined by the World Federation of Neurosurgeons (WFNS) classification of SAH as described in Table 2.24
|
|
Results |
|---|
|
TopAbstractIntroductionPatients and methodsResultsDiscussionReferences |
|---|
We recruited 31 patients for this study. The median duration between SAH and aneurysm embolization was 3.5 days [interquartile range (IQR) 2–5.25 days]. The mean age of patients was 52.1 (range 27–68) yr and there was a female preponderance with a female to male ratio of 2.2 to 1. Twenty-five patients were graded at WFNS grade 1 and 6 at WFNS grade 2.
The placement of sensors in the temporal–frontal montage in patients undergoing this procedure did not cause any impairment of angiographic views in our study. There was one request from a radiologist to move the attaching cable to a BIS sensor, but there were no requests to remove sensors to improve angiographic views.
Median BIS/entropy monitor values with IQR obtained at clinical endpoints are shown in Figure 1.
|
During deepening anaesthesia, PK values for BIS are consistently higher than those of entropy indices (Table 3). The transition between the awake state and loss of eyelash reflex (PK AW vs LOC) shows higher concordance for BIS than for entropy indices (P<0.01). The PK for RE and SE are lower for this transition because of a slower response to LOC compared with BIS. The median values (and IQR) at the point of LOC for BIS, RE and SE were BIS 65 (51–81), RE 71 (51–95), SE 76 (48–86) (Fig. 2). Thus a number of patients show entropy variables above 90 (for RE) or above 80 (for SE) at the point of LOC.
|
|
Subsequent transition from LOC to intubation (PK LOC vs INT) again reveals higher PK values for BIS than RE or SE (P<0.01). However, all monitoring indices recorded lower PK values over this transition and as the transition between LOC and intubation is also more difficult to distinguish clinically, this may influence the PK for this transition.
Overall the PK calculation for deepening anaesthesia (PK deepen) returned a higher value for BIS than for either RE or SE (P<0.01).
The PK value for lightening of anaesthesia (PK lighten) also shows that the values for BIS are higher than that for entropy. These values are comparable with other published series comparing BIS and entropy.14 25 Both BIS and the entropy indices show very good discriminatory ability to predict the transition from stable anaesthesia (SA) to return of eyelash reflex (RR). BIS shows a significantly higher PK for this transition (PK SA vs RR) indicating higher levels of concordance with clinically observed anaesthetic depth (P<0.01).
During the transition between return of eyelash reflex and extubation (PK RR vs EXT), BIS and entropy PK values are at their lowest. The median values (and IQR) at the point of return of eyelash reflex for BIS, RE and SE are 77 (63–84), 95 (74–98) and 85 (53–89) (Fig. 3). A median value for RE of 95 at this point during the lightening phase of anaesthesia may give earlier warning of impending wakening than BIS.
|
The ROC curve for BIS, RE and SE is displayed in Figure 4. The area under curve (AUC) for BIS was significantly higher than for either entropy variable (P<0.01) (Table 4). The difference between the AUC for RE and SE was not statistically significant (P=0.70).
|
|
The threshold values at the highest combination of sensitivity plus specificity were BIS 76, RE 88 and SE 79. The highest threshold for each monitor at a sensitivity of 1.0 was BIS 27, RE 17 and SE 12.
|
Discussion |
|---|
|
TopAbstractIntroductionPatients and methodsResultsDiscussionReferences |
|---|
This study was designed to investigate the performance of BIS and entropy monitoring in patients after SAH. The study is unique in that we performed this investigation in a subset of patients who have cerebral injury undergoing general anaesthesia. Several authors have reported the decrease of BIS variables associated with decreased cardiac output, cardiac arrest or cerebral injury.26 To date, no authors have reported on the behaviour of entropy in brain injured patients.27–33
Angiographic view quality does not appear to be affected adversely by the addition of anaesthetic depth monitoring sensors in these patients. It seems likely that the thin plastic strips used are highly radio-lucent. Care must be taken however to ensure the joint between the sensor and the attached cable is well away from radiologically important areas as this connection does contain radio-opaque material.
Examination of PK values for individual transitions within the major transitions may reveal reasons for the difference in performance between monitors. For example, we found that the decreased PK value for entropy variables in the overall deepening of anaesthesia was attributable to a decrease in concordance between observed clinical depth and indicated depth across several transitions. However, the decrease in entropy PK for lightening of anaesthesia was primarily the result of a lower PK value for entropy at the PK EXT vs AW transition. The lower PK values at this transition are because of the large number of RE values recorded near awake values despite the patient having no eyelash reflex (Fig. 4). It has been reported previously15 that RE shows faster and more complete return to baseline values than BIS during the emergence of anaesthesia. This is backed by our findings.
Because RE values are significantly higher than BIS both at the point of LOC and at the point of return of eyelash reflex we postulate that this difference could be attributable to different handling of frequencies originating from EMG between the two monitor types.
It could be argued that high entropy values before the return of the eyelash reflex whilst lowering the PK value, could actually be beneficial in warning the practising anaesthetist of ‘light’ anaesthesia. Further awake/asleep studies may be necessary to investigate this phenomenon.
The data to calculate PK for transitions was taken during a 15 s period commencing 15 s after eyelash reflex loss. Altering the timing of data capture, to a later or earlier time period, could alter the resulting PK value, in that later data capture would allow the monitor to ‘catch up’ with the clinical picture. Furthermore, longer sampling periods will alter resultant PK values by a similar ‘catch up’ phenomenon. Later data capture could thus result in higher PK values. We chose this time period as it gives both BIS and entropy a response time that is equitable both to the maximal response time of entropy of 1.92 s and around 15–45 s for BIS.
ROC analysis revealed that both BIS and entropy are highly specific and highly sensitive to distinguish the presence of eyelash reflex. We used this test as a surrogate of ‘light anaesthesia’. AUC calculation was performed with a cut off that provided the highest mathematical combination of sensitivity plus specificity. A second cut off was also used to calculate sensitivity and specificity at the upper level of recommended range for general anaesthesia. AUC for BIS was significantly higher than for either entropy variables (P<0.01). Our results would suggest that using the manufacturer's cut off point of 60 for both monitoring types will result in acceptable levels of sensitivity and specificity. To ensure a sensitivity of 1.0 or 100%, very low cut off levels are required, which in this patient group may have unacceptable haemodynamic consequences with respect to cerebral perfusion.
It would appear from PK and ROC curve analysis that patients who have suffered SAH and fall into either WFNS grade 1 or grade 2 have not suffered significant enough injury to preclude the use of either BIS or entropy whilst anaesthetized.
In conclusion, monitoring anaesthetic depth using either BIS or entropy in patients after a good grade subarachnoid haemorrhage is a reliable technique. Both monitoring indices return high levels of sensitivity and specificity in patients after a good grade subarachnoid haemorrhage. Both devices show high levels of concordance with clinically observed anaesthetic depth. Angiographic views appear to be unaffected by the placement of EEG sensors. This type of monitoring could provide neuroanaesthetists with valuable information regarding guidance as to drug dosing in patients after subarachnoid haemorrhage. Further studies to investigate more severely brain injured (e.g. WFNS grade 3) patients are warranted.
|
Acknowledgments |
|---|
This work was supported by an educational grant from Datex-Ohmeda Ltd. We are grateful for the support received from our anaesthetic, ODP and nursing colleagues in the Department of Clinical Neurosciences, Western General Hospital, Edinburgh, UK.
|
References |
|---|
|
TopAbstractIntroductionPatients and methodsResultsDiscussionReferences |
|---|
1 Rosow C and Manberg PJ. Bispectral index monitoring. Anesthesiol Clin North America 2001; 19:947–66[Medline]
2 Myles PS, Leslie K, McNeil J, Forbes A, Chan MTV. Bispectral index monitoring to prevent awareness during anaesthesia: the B-Aware randomised controlled trial. Lancet 2004; 363:1757–63[CrossRef][Web of Science][Medline]
3 Ekman A, Lindholm ML, Lennmarken C, Sandin R. Reduction in the incidence of awareness using BIS monitoring. Acta Anaesthesiol Scand 2004; 48:20–6[CrossRef][Web of Science][Medline]
4 Glass PSM, Bloom MMP, Kearse LMP, Rosow CMP, Sebel PMBPF, Manberg PP. Bispectral analysis measures sedation and memory effects of propofol, midazolam, isoflurane, and alfentanil in healthy volunteers. Anesthesiology 1997; 86:836–47[CrossRef][Web of Science][Medline]
5 Molyneux A. International Subarachnoid Aneurysm Trial (ISAT) of neurosurgical clipping versus endovascular coiling in 2143 patients with ruptured intracranial aneurysms: a randomised trial. Lancet 2002; 360:1267–74[CrossRef][Web of Science][Medline]
6 Maurice-Williams RS and Lafuente J. Intracranial aneurysm surgery and its future. J R Soc Med 2003; 96:540–3
7 Osborn IP. Anesthetic considerations for interventional neuroradiology. Int Anesthesiol Clin 2003; 41:69–77[Medline]
8 Rampil IJ. A Primer for EEG signal processing in anesthesia. Anesthesiology 1998; 89:980–1002[CrossRef][Web of Science][Medline]
9 Johansen J and Sebel PS. Development and clinical application of electroencephalographic bispectrum monitoring. Anesthesiology 2000; 93:1336–44[CrossRef][Web of Science][Medline]
10 Shannon CE. A mathematical theory of communication. Bell System Technical J 1948; 27:379–423 and 623–56
11 Viertö-Oja H, Maja V, Särkelä M, et al. Description of the EntropyTM Algorithm as applied in the Datex-Ohmeda S/5TM Entropy Module. Acta Anaesthesiol Scand 2004; 48:154–61[CrossRef][Web of Science][Medline]
12 Bruhn J, Heiko R, Hoeft A. Approximate entropy as an electroencephalographic measure of anesthetic drug effect during desflurane anesthesia. Anesthesiology 2000; 92:715–26[CrossRef][Web of Science][Medline]
13 Ellerkmann RK, Liermann VM, Alves TM, et al. Spectral entropy and bispectral index as measures of the electroencephalographic effects of sevoflurane. Anesthesiology 2004; 101:1275–82[CrossRef][Web of Science][Medline]
14 Schmidt GN, Bischoff P, Standl T, Hellstern A, Teuber O, Schulte EJ. Comparative evaluation of the Datex-Ohmeda S/5 Entropy Module and the Bispectral Index monitor during propofol–remifentanil anesthesia. Anesthesiology 2004; 101:1283–90[CrossRef][Web of Science][Medline]
15 Vakkuri A, Yli-Hankala A, Talja P, et al. Time-frequency balanced spectral entropy as a measure of anesthetic drug effect in central nevous system during sevoflurane, propofol, and thiopental anesthesia. Acta Anaesthesiol Scand 2004; 48:145–53[CrossRef][Web of Science][Medline]
16 Schmidt GN, Bischoff P, Standl T, et al. Comparative evaluation of Narcotrend[TM], Bispectral Index[TM] and classical electroencephalographic variables during induction, maintenance and emergence from propofol/remifentanil anesthesia. Anesth Analg 2004; 98:1346–53
17 Struys MMRF. Ability of the Bispectral Index, autoregressive modelling with exogenous input-derived auditory evoked potentials, and predicted propofol concentrations to measure patient responsiveness during anesthesia with propofol and remifentanil. Anesthesiology 2003; 99:802–12[CrossRef][Web of Science][Medline]
18 Smith WD, Dutton RC, Smith NT. Measuring the performance of anesthetic depth indicators. Anesthesiology 1996; 84:38–51[CrossRef][Web of Science][Medline]
19 Schmidt GN, Bischoff P, Standl T, et al. Narcotrend® and Bispectral Index® monitor are superior to classic electroencephalographic parameters for the assessment of anesthetic states during propofol–remifentanil anesthesia. Anesthesiology 2003; 99:1072–7[CrossRef][Web of Science][Medline]
20 Kreuer S, Bruhn J, Larsen R, Hoepstein M, Wilhelm W. Comparison of Alaris AEP index and bispectral index during propofol–remifentanil anaesthesia. Br J Anaesth 2003; 91:336–40
21 Iselin-Chaves IA, El Moalem HE, Gan TJ, Ginsberg B, Glass P. Changes in the auditory evoked potentials and the bispectral index following propofol or propofol and alfentanil. Anesthesiology 2000; 92:1300–10[CrossRef][Web of Science][Medline]
22 Galley HF. Editorial II: solid as a ROC. Br J Anaesth 2004; 93:623–6
23 Metz CE. Basic principles of ROC analysis. Semin Nucl Med 1978; 8:283–98[Web of Science][Medline]
24 Report of World Federation of Neurological Surgeons Committee on a Universal Subarachnoid Hemorrhage Grading Scale. J Neurosurg 1988; 68:985–6[Web of Science][Medline]
25 Vanluchene AL, Struys MM, Heyse BE, Mortier EP. Spectral entropy measurement of patient responsiveness during propofol and remifentanil. A comparison with the bispectral index. Br J Anaesth 2004; 93:645–54
26 Azim N and Wang CY. The use of bispectral index during a cardiopulmonary arrest: a potential predictor of cerebral perfusion. Anaesthesia 2004; 59:610–12[Medline]
27 Billard V. Brain injury under general anesthesia: is monitoring of the EEG helpful? Can J Anaesth 2001; 48:1055–60[Web of Science][Medline]
28 Chakravarthy M, Patil T, Jayaparakash K, Shivananda N, Jawali V. Bispectral index is an indicator of adequate cerebral perfusion during cardiopulmonary resuscitation. J Cardiothorac Vasc Anesth 2004; 17:506–8
29 England MR. The changes in bispectral index during a hypovolemic cardiac arrest. Anesthesiology 1999; 91:1947[CrossRef][Web of Science][Medline]
30 Fodale V and Pratico C. Bispectral Index monitoring for assessment of level of consciousness in brain-injured patients: the journey continues. Crit Care Med 2004; 32:2545–6[CrossRef][Web of Science][Medline]
31 Hana AR, Inchoisa MA, Frost EAM. The bispectral index as a predictor of outcome after head injury. Anesth Analg 1999; 88:S56[CrossRef]
32 Morimoto Y, Monden Y, Ohtake K, Sakabe T, Hagihira S. The detection of cerebral hypoperfusion with bispectral index monitoring during general anesthesia. Anesth Analg 2005; 100:158–61
33 Mourisse J and Leo B. Bispectral index detects periods of cerebral hypoperfusion during cardiopulmonary bypass. J Cardiothorac Vasc Anesth 2003; 17:76–8[CrossRef][Web of Science][Medline]
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||