Home Print this page Email this page Small font sizeDefault font sizeIncrease font size
Users Online: 538

 

Home  | About Us | Editors | Search | Ahead Of Print | Current Issue | Archives | Submit Article | Instructions | Subscribe | Contacts | Login 
     


 
 
Table of Contents
ORIGINAL ARTICLE
Year : 2021  |  Volume : 11  |  Issue : 3  |  Page : 161-166

Utility of early warning scores to predict mortality in COVID-19 patients: A retrospective observational study


1 Department of Emergency Medicine, All India Institute of Medical Sciences, Rishikesh, Uttarakhand, India
2 Department of Microbiology, All India Institute of Medical Sciences, Rishikesh, Uttarakhand, India

Date of Submission10-Jul-2021
Date of Acceptance26-Aug-2021
Date of Web Publication25-Sep-2021

Correspondence Address:
Dr. Prakash Mahala
Department of Emergency Medicine, All India Institute of Medical Sciences, Rishikesh, Uttarakhand
India
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/ijciis.ijciis_64_21

Rights and Permissions
   Abstract 


Background: Coronavirus disease 2019 (COVID19) has evolved as a global pandemic. The patients with COVID-19 infection can present as mild, moderate, and severe disease forms. The reported mortality of severe acute respiratory syndrome coronavirus 2 (SARS-COV-2) infection is around 6.6%, which is lower than that of SARS-CoV and (middle east respiratory syndrome CoV). However, the fatality rate of COVID-19 infection is higher in the geriatric age group and in patients with multiple co-morbidities. The study aimed to evaluate the utility of early warning scores (EWS) to predict mortality in patients with moderate to severe COVID-19 infection.
Methods: This retrospective study was carried out in a tertiary care institute of Uttarakhand. Demographic and clinical data of the admitted patients with moderate-to-severe COVID-19 infection were collected from the hospital record section and utilized to calculate the EWS-National early warning score (NEWS), modified early warning score (MEWS), Rapid Acute Physiology Score (RAPS), rapid emergency medicine score (REMS), and worthing physiological scoring system (WPS).
Results: The area under the curve for NEWS, MEWS, RAPS, REMS, and WPS was 0.813 (95% confidence interval [CI]; 0.769–0.858), 0.770 (95% CI; 0.717–0.822), 0.755 (95% CI; 0.705–0.805), 0.892 (95% CI; 0.859–0.924), and 0.892 (95% CI; 0.86–0.924), respectively.
Conclusion: The EWS at triage can be used for early assessment of severity as well as predict mortality in patients with COVID-19 patients.

Keywords: Coronavirus disease 2019, early warning scores, intensive care unit mortality


How to cite this article:
Kaeley N, Mahala P, Kabi A, Choudhary S, Hazra AG, Vempalli S. Utility of early warning scores to predict mortality in COVID-19 patients: A retrospective observational study. Int J Crit Illn Inj Sci 2021;11:161-6

How to cite this URL:
Kaeley N, Mahala P, Kabi A, Choudhary S, Hazra AG, Vempalli S. Utility of early warning scores to predict mortality in COVID-19 patients: A retrospective observational study. Int J Crit Illn Inj Sci [serial online] 2021 [cited 2021 Dec 9];11:161-6. Available from: https://www.ijciis.org/text.asp?2021/11/3/161/326601




   Introduction Top


Coronavirus disease 2019 (COVID19) is caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) virus, first detected in Wuhan, China. It was recognized as an International Public Health Emergency in January 2020. It causes a wide spectrum if symptoms categorized as mild, moderate, and severe forms. The reported fatality rate ranges from 11% to 62%.[1],[2],[3] The mortality rate is higher in geriatric patients and vulnerable population with multiple comorbidities such as hypertension, Chronic obstructive pulmonary disease, chronic kidney, and liver disease.[4] Because of the contagious nature of the disease, the second wave of COVID-19 infection has posed a huge challenge on the health care system of the country.

The dynamic nature of the pandemic has imposed a robust challenge on the health care personnel worldwide. It includes pressures such as working constantly in personal protective equipment, longer shift duties, and overburdened health care facilities. Thus, it is the need of the hour to set up protocols and scores for easy identification of high-risk patients with moderate-to-severe COVID-19 infection, aiding in rapid triaging of sicker patients in the emergency department.

The use of early warning scores (EWS) in the emergency setup has been advocated in COVID-19 pandemic times for timely screening of high-risk patients.[5],[6],[7] The parameters incorporated in EWS are vital signs such as respiratory rate, blood pressure, oxygen saturation, heart rate, temperature, use of supplemental oxygen, less of consciousness, and age.[8] They are multiple EWS described suitable for screening patients visiting the emergency department. They are National early warning score (NEWS), NEWS-2, Modified early warning score (MEWS), Hamilton early warning score, standardized early warning score (SEWS), rapid acute physiological score (RAPS), rapid emergency medicine score (REMS), and worthing physiological scoring system (WPS).[7],[8]

Currently, the risk stratification of COVID-19 patients is based on oxygen saturation and inflammatory biomarkers such as IL-6, (C-reactive protein), lactate dehydrogenase, serum ferritin, and D-dimer.[9] However, these parameters are time-consuming and cannot be used for immediate risk assessment of patients with COVID-19 infection at triage. EWS uses simple physiological parameters, which can be measured at the triage and can be utilized for the rapid identification of sicker patients. They can be used in emergency, intensive care units (ICU), high-dependency units, wards as well as nursing homes.

Royal College of Physicians formulated NEWS score in 2012. It includes parameters such as heart rate, systolic blood pressure, temperature, respiratory rate, oxygen saturation, need for supplemental oxygen, and mental status.[9] It was updated as NEWS-2 after 5 years. MEWS score was validated in 2000. It included parameters such as heart rate, systolic blood pressure, respiratory rate, temperature, and mental status.[8] Thus, in this study, we aimed to evaluate the utility of five EWS as a predictor of mortality in patients with COVID-19 infection in the emergency department.


   Methods Top


This retrospective observational study was conducted at the Emergency medicine department, AIIMS, Rishikesh, over a period of 3 months from March 10, to June 11, 2021. Adult patients more than 18 years old with moderate-to-severe COVID-19 infection defined as per National guidelines who were admitted to the hospital with oxygen saturation ≤93% were included in the study.[10] The diagnosis of COVID-19 infection was confirmed using reverse transcription-polymerase chain reaction (RT-PCR) testing. The patients who were RT-PCR negative but had radiological evidence of COVID-19 infection (CORADS 4 and above) were also included in the study. Patients with mild COVID-19 infection with oxygen saturation more than 93% were excluded from the study. Serial patients with moderate-severe COVID-19 pneumonia admitted to the emergency department with oxygen saturation <94% were enrolled in the study. Detailed demographic, clinical, and biochemical data of all the enrolled patients were obtained from the hospital record section after seeking approval from the Institutional Ethical and Research Committee (AIIMS/IEC/21/109;04/03/2021).

NEWS,[9] MEWS,[8] RAPS, REMS, and WPS[11] was calculated for all the patients with moderate-severe COVID-19 pneumonia utilizing the clinical parameters at the triage. NEWS score includes six parameters such as respiratory rate, oxygen saturation, systolic blood pressure, pulse rate, level of consciousness (alert, verbal, pain, unresponsive, AVPU), and body temperature.

The MEWS score is calculated using physiological parameters such as respiratory rate, oxygen saturation, systolic blood pressure, and pulse rate, level of consciousness, and body temperature. RAPS includes age, pulse rate, respiratory rate, and Glasgow coma (GCS) score. REMS includes age, pulse rate, mean arterial pressure, respiratory rate, and GCS and oxygen saturation. WPS System encompasses respiratory rate, pulse, systolic blood pressure, temperature, oxygen saturation and neurological status. History related to mortality, length of stay, need for invasive, noninvasive ventilation, need for high flow nasal cannulation, and ICU admission and vasopressor support of all the patients was noted from the hospital record section. All the patients were subdivided into two subsets-survivors versus nonsurvivors. All the clinical, biochemical parameters, history related to comorbidities, and EWS were compared between these two subgroups.

Statistical analysis

The statistical analysis was done using SPSS 20.0 (IBM Corp., Armonk, USA). Continuous variables were calculated as mean ± standard deviation whereas categorical variables were presented as ratio. Student t-test or Mann–Whitney U-test was used for continuous variables. Chi-square test or Fisher's exact test was utilized for categorical variables. The area under the receiver operating characteristic curve (AUROC) was calculated for each score. In addition, sensitively, specificity, negative predictive value, positive predictive value was estimated to determine the cut off value of each score. P =0.05 was considered statistically significant.


   Results Top


Three fifty COVID-19 patients with moderate to severe pneumonia were enrolled in the study, out of whom 168 (48.0%) were nonsurvivors and 182 (52.0%) were survivors. The mean age of nonsurvivors (71.9 ± 13.6) was significantly more than the mean age of survivors (52.4 ± 15.6) [Table 1]. Among, nonsurvivors 123 (73.2%) patients were males. However, there was no significant difference in gender between survivors and nonsurvivors groups. Respiratory rate (28.62 ± 8.44) and oxygen saturation (80.86 ± 6.24) were significantly associated with mortality of moderate-to-severe category COVID-19 patients.
Table 1: Demographic and clinical parameters of the patients with moderate to severe coronavirus disease 2019 pneumonia

Click here to view


GCS score (13.84 ± 2.84) was significantly lower in patients who succumbed to death. Mean values of NEWS (6.85 ± 2.52), MEWS (2.44 ± 1.22), RAPS (1.96 ± 1.09), REMS (8.22 ± 2.68), and WPS (4.81 ± 1.75) were significantly higher in the groups of COVID-19 patients who expired [Table 2]. Thus, there was statistically significant association between EWS and mortality in patients with moderate to severe COVID-19 infection (P < 0.05). In addition, the length of stay was shorter (3.98 ± 336.78) in nonsurvivors as opposed to survivors (13.88 ± 7.86). As evident, significantly higher number of patients belonging to the nonsurvivor group as compared to the survivor group required noninvasive ventilation (42; 25%), invasive ventilation (145; 86.3%), high flow nasal cannulation (121; 72%), and vasopressor support (59; 35.1%). hypertension (99; 58.9%) was found to be significantly associated with mortality in patients with moderate-to-severe COVID-19 infections.
Table 2: Mean values of early warning scores of the patients with moderate-to-severe coronavirus disease 2019 pneumonia

Click here to view


The area under the curve values of NEWS, MEWS, RAPS, REMS, and WPS are 0.813, 0.770, 0.755, 0.892, and 0.892, respectively [Table 3]. [Table 4] shows cut off values of NEWS, MEWS, RAPS, REMS, and WPS scores. Clearly, the AUROC of NEWS, REMS, and WPS was found to be higher as compared to AUROC of MEWS score. The cut off value NEWS score was five with sensitivity of 86.9% and specificity of 66.5%. The negative predictive value of NEWS score was 84.6%. [Table 4] shows sensitivity, specificity positive predictive value as well as negative predictive value of each score. [Figure 1] shows ROC curve of the early warning score to predict mortality.
Table 3: Values of area under the curve of early warning scores of patients with moderate-to-severe coronavirus disease 2019 pneumonia

Click here to view
Table 4: Cut off values of early warning scores of patients with moderate-to-severe coronavirus disease 2019 pneumonia

Click here to view
Figure 1: ROC curve of early warning scores. NEWS: National Early Warning Score, MEWS: Modified Early Warning Score, RAPS: Rapid Acute Physiology Score, REMS: Rapid Emergency Medicine Score, WPS: Worthing Physiological Scoring System

Click here to view



   Discussions Top


COVID-19 infections caused by SARS-COV-2 have emerged as a global pandemic taking a heavy toll on human life. It has varied presentations and multisystem involvement. The patients with a milder form of the disease recover with supportive treatment. However, patients with moderate to severe COVID-19 pneumonia require oxygen therapy, noninvasive or invasive ventilation. Early identification of patients with severe form of disease can prevent mortality in these patients. There are not many prognostic scores for COVID-19 patients that can be used in emergency medicine department. Thus, the preexisting scores such as EWS can be utilized as predictors of mortality in these patients. EWS has been used in the past for patients with sepsis, trauma, and surgical emergencies.[7],[8],[9]

A study conducted by Yildiz et al. has validated a risk score to predict risk of developing severe form of COVID-19 disease. They assessed “COVIDGRAM” risk score to predict mortality, invasive ventilation, and ICU admissions in patients with COVID-19 pneumonia. It has been observed that around 15% of patients with COVID-19 pneumonia develop a severe form of disease. Thus, it is the need of the hour to formulate and validate risk scores for prompt identification of at-risk patients.[12]

Gidari et al. concluded NEWS-2 score is a good predictor for ICU admission in patients of COVID-19 infection.[12] In this study, the age was found to be significantly associated with mortality. Previous studies have also predicted that geriatric patients are more prone to severe form of COVID-19 disease. Cummings et al. studied 257 critically ill COVID-19 patients. The median age of the patients was 62 years (51–72). Thus, geriatric patients because of altered immunity and comorbidities, are at risk of higher mortality.[13]

Among the comorbidities, hypertension was found to be significantly associated with mortality. In a meta-analysis conducted by Lippi G et al., hypertension has been identified as a poor prognostic marker in patients with COVID-19 infection. Patients with hypertension are at increased risk of complications such as ischemic heart disease, chronic heart disease, chronic kidney disease, and thus more prone to severe critical COVID-19 illness.[14]

In our study, higher number of patients who succumbed to death required invasive and noninvasive ventilation, high-flow nasal cannulation, and vasopressor support as compared to the survivor group. Similar findings were described by a previous study conducted by Su et al.[15]

There are multiple studies in the past, which have highlighted the importance of individual EWS in COVID-19 patients. In this study, we have incorporated all the five important EWS, which can be utilized for effective triaging of COVID-19 patients. In our study, the area under the curve for NEWS score was 0.813 with the negative predictive value of 84.6%. Thus, we inferred that NEWS score could be utilized for risk stratification of critically sick COVID-19 patients. Gautret and Cao et al. have done similar studies. They highlighted the role of EWS in predicting mortality in critically ill COVID-19 patients.[16],[17]

Myrstad et al. have studied the utility of NEW-2 score as a predictor of 7-day mortality in COVID-19 patients.[18]

Jang et al. studied the comparison of NEWS score and q sofa as predictors of 28-day mortality in patients with COVID-19 infection and predicted that NEWS score was superior as a predictor of mortality.[19]

Galassi and Schena have studied the utility of MEWS as a predictor of mortality in patients with COVID-19 infection. The second wave of COVID-19 infection had hit strongly, leading to shrinking medical facilities of the country. Both medical equipment and staff were exhausted. COVID-19 pandemic has both short and long-term effects on health care system. The direct impact of COVID-19 pandemic on health care services was in the form of inadequacy, inappropriateness, inaccessibility of health care services, and discontinuity of medical care. It was associated with increase in anxiety levels and depression rates in health care personal. It posed a mammoth challenge to provide immediate care to all the sick patients of COVID-19 infection. The long-term impact was in the form of behavioral health exacerbations. The health care practitioners have witnessed the devastating psychological impact and burnt out due to long hours of work, difficult circumstances, and witnessing increased mortality. Thus, it is important to identify at-risk patients who need immediate attention, especially while dealing with a pandemic such as COVID-19. EWS such as NEWS, MEWS can help in better triaging of sicker patients with COVID-19 infection.[20],[21]

Olsson et al. studied RAPS and REMS score as a prognostic marker in patients presenting as nonsurgical medical emergencies. Rapid Acute Physiology Score includes parameters such as blood pressure, respiratory rate; pulse rate and Glasgow come scale). In addition, to these parameters REMS utilizes peripheral oxygen saturation and age as clinical parameters. They concluded that REMS was superior to RAPS in predicting in hospital mortality of medical patients.[22]

The specificity of RAPS and REMS in our study was 74.7%, 78.6%, respectively. However, REMS had a slightly higher negative predictive value. Özdemir et al. evaluated the efficacy of RAPS and REMS score in predicating mortality in COVID-19 patients. The cut off value for REMS in their study was 3.5 with 73.08% specificity and 2.5 for RAPS with 97.38% specificity.[23]

The cut off value of RAPS in our study was two, and that of REMS was six. This difference could be because we only included patients with moderate-to-severe COVID-19 pneumonia and excluded patients with a mild infection, as they were not admitted.

Similarly, a study done by Covino M et al. has evaluated the effectiveness of early warning score for predicting ICU admission and mortality in patients with COVID-19 pneumonia.[24] They concluded that NEWS and MEWS score could be incorporated in triaging COVID-19 patients by virtue of higher negative predictive value. In our study, NEWS, MEWS, and REMS had the highest negative predictive value; thus, this score could be utilized to differentiate between patients with poorer outcome and better outcome.

Ha et al. have studied WPS system as a prognostic marker of mortality in emergency patients.[25]

Hence, we need ways to triage patients visiting emergency department to provide immediate help to the sickest patients.


   Conclusion Top


EWS can be utilized as prognostic tools in the risk stratification of COVID-19 patients. NEWS, MEWS, and REMS scores are potential scores, which can be utilized for identifying at-risk COVID-19 patients. This can prevent mortality in these patients. Steps should be undertaken towards early detection and prompt evaluation of the moderate-to-severe categories of COVID-19 patients to prevent morbidity and mortality.

Research quality and ethics statement

This study was approved by the Institutional Review Board/Ethical and Research Committee at the All India Institute of Medical Sciences (Approval#AIIMS/IEC/21/109; Approval Date 04/03/2021). The authors followed the applicable EQUATOR Network (http://www.equatornetwork.org/) guidelines, specifically the STROBE guidelines, during the conduct of this research project.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.



 
   References Top

1.
Chen N, Zhou M, Dong X, Qu J, Gong F, Han Y, et al. Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: A descriptive study. Lancet 2020;395:507-13.  Back to cited text no. 1
    
2.
Weiss P, Murdoch DR. Clinical course and mortality risk of severe COVID-19. Lancet 2020;395:1014-5.  Back to cited text no. 2
    
3.
Huang C, Wang Y, Li X, Ren L, Zhao J, Hu Y, et al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. Lancet 2020;395:497-506.  Back to cited text no. 3
    
4.
Liu FY, Sun XL, Zhang Y, Ge L, Wang J, Liang X, et al. Evaluation of the risk prediction tools for patients with coronavirus disease 2019 in Wuhan, China: A single-centered, retrospective, observational study. Crit Care Med 2020;48:e1004-11.  Back to cited text no. 4
    
5.
Nannan Panday RS, Minderhoud TC, Alam N, Nanayakkara PW. Prognostic value of early warning scores in the emergency department (ED) and acute medical unit (AMU): A narrative review. Eur J Intern Med 2017;45:20-31.  Back to cited text no. 5
    
6.
Martín-Rodríguez F, López-Izquierdo R, Del Vegas CP, Benito JF, Rodríguez VC, Rasilla MN, et al. Accuracy of National Early Warning Score 2 (NEWS2) in prehospital triage on in-hospital early mortality: A multi-center observational prospective cohort study. Prehosp Disaster Med 2019;34:610-8.  Back to cited text no. 6
    
7.
Mitsunaga T, Hasegawa I, Uzura M, Okuno K, Otani K, Ohtaki Y, et al. Comparison of the National Early Warning Score (NEWS) and the Modified Early Warning Score (MEWS) for predicting admission and in-hospital mortality in elderly patients in the pre-hospital setting and in the emergency department. PeerJ 2019;7:e6947.  Back to cited text no. 7
    
8.
Smith ME, Chiovaro JC, O'Neil M, Kansagara D, Quiñones AR, Freeman M, et al. Early warning system scores for clinical deterioration in hospitalized patients: A systematic review. Ann Am Thorac Soc 2014;11:1454-65.  Back to cited text no. 8
    
9.
Fan J, Upadhye S, Worster A. Understanding receiver operating characteristic (ROC) curves. CJEM 2006;8:19-20.  Back to cited text no. 9
    
10.
11.
Gidari A, De Socio GV, Sabbatini S, Francisci D. Predictive value of National Early Warning Score 2 (NEWS2) for intensive care unit admission in patients with SARS-CoV-2 infection. Infect Dis (Lond) 2020;52:698-704.  Back to cited text no. 11
    
12.
Yildiz H, Castanares-Zapatero D, Hannesse C, Vandermeersch D, Pothen L, Yombi JC. Prospective validation and comparison of COVID-GRAM, NEWS2, 4C mortality score, CURB-65 for the prediction of critical illness in COVID-19 patients. Infect Dis (Lond) 2021;53:640-2.  Back to cited text no. 12
    
13.
Cummings MJ, Baldwin MR, Abrams D, Jacobson SD, Meyer BJ, Balough EM, et al. Epidemiology, clinical course, and outcomes of critically ill adults with COVID-19 in New York City: A prospective cohort study. Lancet 2020;395:1763-70.  Back to cited text no. 13
    
14.
Lippi G, Wong J, Henry BM. Hypertension in patients with coronavirus disease 2019 (COVID-19): A pooled analysis. Pol Arch Intern Med 2020;130:304-9.  Back to cited text no. 14
    
15.
Su Y, Ju MJ, Xie RC, Yu SJ, Zheng JL, Ma GG, et al. Prognostic accuracy of early warning scores for clinical deterioration in patients with COVID-19. Front Med (Lausanne) 2020;7:624255.  Back to cited text no. 15
    
16.
Gautret P, Lagier JC, Parola P, Hoang VT, Meddeb L, Sevestre J, et al. Clinical and microbiological effect of a combination of hydroxychloroquine and azithromycin in 80 COVID-19 patients with at least a six-day follow up: A pilot observational study. Travel Med Infect Dis 2020;34:101663.  Back to cited text no. 16
    
17.
Cao B, Wang Y, Wen D, Liu W, Wang J, Fan G, et al. A trial of lopinavir-ritonavir in adults hospitalized with severe Covid-19. N Engl J Med 2020;382:1787-99.  Back to cited text no. 17
    
18.
Myrstad M, Ihle-Hansen H, Tveita AA, Andersen EL, Nygård S, Tveit A, et al. National Early Warning Score 2 (NEWS2) on admission predicts severe disease and in-hospital mortality from Covid-19 – A prospective cohort study. Scand J Trauma Resusc Emerg Med 2020;28:66.  Back to cited text no. 18
    
19.
Jang JG, Hur J, Hong KS, Lee W, Ahn JH. Prognostic accuracy of the SIRS, qSOFA, and NEWS for early detection of clinical deterioration in SARS-CoV-2 infected patients. J Korean Med Sci 2020;35:e234.  Back to cited text no. 19
    
20.
Raman R, Rajalakshmi R, Surya J, Ramakrishnan R, Sivaprasad S, Conroy D, et al. Impact on health and provision of healthcare services during the COVID-19 lockdown in India: A multicentre cross-sectional study. BMJ Open 2021;11:e043590.  Back to cited text no. 20
    
21.
Galassi L, Schena D. The Modified National Early Warning Score (m-NEWS) for COVID-19-infected patient evaluation: A proof-of-concept. SN Compr Clin Med published online ahead of print, 2021;31:1-2.  Back to cited text no. 21
    
22.
Olsson T, Terent A, Lind L. Rapid emergency medicine score: A new prognostic tool for in-hospital mortality in nonsurgical emergency department patients. J Intern Med 2004;255:579-87.  Back to cited text no. 22
    
23.
Özdemir S, Akça HŞ, Algın A, Altunok İ, Eroğlu SE. Effectiveness of the rapid emergency medicine score and the rapid acute physiology score in prognosticating mortality in patients presenting to the emergency department with COVID-19 symptoms. Am J Emerg Med 2021;49:259-64.  Back to cited text no. 23
    
24.
Covino M, Sandroni C, Santoro M, Sabia L, Simeoni B, Bocci MG, et al. Predicting intensive care unit admission and death for COVID-19 patients in the emergency department using early warning scores. Resuscitation 2020;156:84-91.  Back to cited text no. 24
    
25.
Ha DT, Dang TQ, Tran NV, Vo NY, Nguyen ND, Nguyen TV. Prognostic performance of the Rapid Emergency Medicine Score (REMS) and Worthing Physiological Scoring system (WPS) in emergency department. Int J Emerg Med 2015;8:18.  Back to cited text no. 25
    


    Figures

  [Figure 1]
 
 
    Tables

  [Table 1], [Table 2], [Table 3], [Table 4]



 

Top
 
  Search
 
    Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
    Access Statistics
    Email Alert *
    Add to My List *
* Registration required (free)  

 
  In this article
    Abstract
   Introduction
   Methods
   Results
   Discussions
   Conclusion
    References
    Article Figures
    Article Tables

 Article Access Statistics
    Viewed765    
    Printed12    
    Emailed0    
    PDF Downloaded19    
    Comments [Add]    

Recommend this journal