Received 2018-04-24

Revised 2018-05-14

Accepted 2018-06-12

 Correspondence to: Masoumeh Emamghoreishi, Department of Pharmacology, School of medicine, Shiraz University of Medical Sciences, Zand, Fars province, Shiraz, Iran Telephone Number: +987132307591 Email Address: emamm@sums.ac.ir

GMJ.2019;8:e1188

www.gmj.ir

Introduction

National Institute of Health Stroke Scale (NIHSS) is a ubiquitous, valid, and reliable clinical questionnaire predicting the severity of stroke. This questionnaire is useful for making medical and nursing decisions and is routinely being used in clinical trials [1-5]. It has 15 items including the level of consciousness (LOC), LOC questions, LOC commands, best gaze, visual, facial palsy, motor arm, motor leg, limb ataxia, sensory, best language, dysarthria, extinction, and inattention. Higher scores on the scale is an indicator of more severe neurological deficits [2]. In spite of its reliability, validity and widespread use, the NIHSS has been scrutinized for some redundant items including the OLC, facial palsy, limb ataxia, and dysarthria. These items have caused complexity for NIHSS implementation, prolongation of the questionnaire response time, tiredness of the person filling out the questionnaire and also deviance in its ultimate results. These limitations of NIHSS questionnaire have led to a reduced tendency for implementing and reporting NIHSS in emergency and stroke wards [6, 7], while an initial estimation of stroke severity is essential for the appraisal of the treatment and rehabilitation programs.Modified National Institute of Health Stroke Scale (mNIHSS) is a succeeding questionnaire in which the redundant items have been removed. Such changes have brought several advantages for mNIHSS over NIHSS including simpler grading scale, more accessible to administration, more precise results, and shorter time to perform. These advantages have made the mNIHSS more valid, reliable and acceptable among clinicians and researchers [6, 8].

Stroke is the main cause for physical disability and the second cause of mortality worldwide [9]. Two-thirds of all strokes occur in developing countries including Iran [9, 10]. The incidence of stroke in Iran is about 750-800 individuals in 100000 population per year, which is much higher than worldwide [11, 12]. Thus, having a valid and reliable questionnaire for evaluating the severity of stroke in stroke and emergency wards and clinical trials is an essential need. English version of NIHSS has been routinely used as a reliable and valid questionnaire in Iran. However, since misinterpretation/misunderstanding of the English version might lead to a deviation in the ultimate results, developing a Persian version of NIHSS seems necessary. There is only one previous study assessing the reliability and validity of the Persian version of NIHSS [13]. However, there were some limitations in the preceding study such as evaluation of validity and reliability of the questionnaire in non-hospitalized subjects; inclusion of only ischemic stroke patients; and exclusion of patients who have sever consciousness deficit, severe hearing or vision impairments [13]. Since this questionnaire has been developed to assess the severity of stroke at the time of evaluation in all stroke patients (ischemic or hemorrhagic) regardless of patients’ conditions (such as vision and hearing abilities of patients). Therefore, for assessing the validity of Persian version of the questionnaire all stroke patients has to be included in the study in order to generalize its applicability. Also, the evaluation of the severity of the stroke is needed at the time of admission to hospital in order to follow the improvement of patients’ complications during drug therapy and rehabilitation programs. Thus, it is also necessary to validate this questionnaire in hospitalized stroke patients.Furthermore, considering the advantages of mNIHSS over NIHSS, it is valuable and worthwhile to validate a Persian version of mNIHSS as well. The availability of a Persian version of this easier and less time-consuming questionnaire will encourage medical staffs to use it more frequently at the emergency and stroke wards. Therefore, this study aimed to examine and compare the validity and reliability of Persian versions of NIHSS and mNIHSS in hospitalized stroke patients.

Materials and Methods

This study was part of a comprehensive research approved by the local Ethics Committee and research council of Shiraz University of Medical Sciences (approval code: 10400). Patients were only enrolled after giving written informed consent. In the case that the patients couldn’t fill the informed consent, their legal guardians signed the consent forms. All procedures were followed in accordance with the guidance and approval of the Institutional Medical Ethics Committee.

Instruments and Participants

After obtaining permission from the National Institute of Neurological Disorders and Stroke (https://www.ninds.nih.gov), both questionnaires were translated to Persian by a single professional translator (forward translation) and then the translated questionnaires were translated into English (backward translation) by another single native English translator. Three neurologists examined the face and content validity of both translations. The final translated versions were finalized after the agreement of translators and neurologists on items that were difficult to be easily perceived.

Seventy-five patients (39 men and 36 females) aged 27 to 65 years were randomly and sequentially entered the study in order of their admission time to the stroke ward of Namazi teaching hospital, Shiraz, Iran from April 2017 through July 2017. An inclusion criterion was clinical diagnosis of stroke (hemorrhagic or obstructive) by computed tomography scan or magnetic resonance imaging. Patients with previous neurological disorders such as Alzheimer disease, Parkinson or seizure were excluded from the study. Sample size was calculated according to rule of thumb stating that the required number of subjects for testing validity and reliability of a questionnaire equals to at least 5-10 times of number of questions (items) in a questionnaire (i.e., 15 questions in the NIHSS× 5 = 75 subjects is needed for testing validity and reliability of NIHSS questionnaire) [14, 15]. For each patient, both NIHSS and mNIHSS were performed by a single trained researcher. Both NIHSS and mNIHSS questionnaires have multiple Likert-type scale questions. The Likert-type scale questions in both NIHSS and mNIHSS may have three, four or five options that the implementer must choose an answer among the choices for each question. The Likert-type scale questions in both questionnaires determine the patient’s condition in several domains (Table-1). These questionnaires have some differences as well. NIHSS has 15 questions while mNIHSS has 11 questions. In mNIHSS, the LOC, facial palsy, limb ataxia, and dysarthria items have been deleted. Also, question 8 (sensory item), which was a three-option question in NIHSS has been changed to a two-option question in mNIHSS.

Data Analysis

The collected data from patients were inserted in the SPSS (version 16, IBM USA) and were analyzed for reliability and validity. Reliability and validity of the two questionnaires were assessed by calculating Cronbach’s alpha coefficient and convergent validity of each questionnaire; respectively. Cronbach’s alpha coefficient measures the internal consistency (reliability) and ranges from 0 to 1. The values >0.65 are acceptable [16]. The higher grades (>0.65) indicates more reliable the questionnaire. Convergent validity is a subtype of construct validity. For calculating the convergent validity, the Pearson correlation coefficient was used. A value of a correlation coefficient of greater than 0.40 between an item and total score is regarded as adequate evidence of convergent validity [17]. Also, the scaling success of convergent validity for each questionnaire was calculated by dividing the number of correlations above 0.40 by the total number of correlations ×100 (Table-2).

Results

Face and content validity are subjective measures. If specialists approve the face and content validity, the questionnaire was applicable for assessing the desired goals. In this study, the face and content validities of Persian versions of NIHSS and mNIHSS questionnaires were approved by three neurologists. Cronbach’s alpha coefficient is the most frequent measure for assessing reliability (internal consistency). It is mostly used when there are multiple Likert-type scale questions in a questionnaire (such as the NIHSS questionnaire) that form a scale, and one wants to specify if the scale is reliable. In this study, Cronbach’s alpha coefficients were 0.81 for NIHSS and 0.86 for mNIHSS (Table-2).

Convergent validity is a subtype of construct validity, which was used to determine whether the NIHSS and mNIHSS were really assessing the severity of stroke. A correlation coefficient of greater than 0.40 is an index of convergent validity. The numbers of questions that have correlation coefficients of greater than 0.4 were 12 in NIHSS and 11 in mNIHSS. This means that the scaling success was 80% and 100% in NIHSS and mNIHSS, respectively (Table-2).

Discussion

The results of the current study confirmed the reliability and validity of Persian versions of NIHSS and mNIHSS in hospitalized stroke patients. Also, the findings of this study indicated that the Persian version of mNIHSS was more valid and reliable than the Persian version of NIHSS. English version of NIHSS is a valid and reliable questionnaire, which is routinely used to assess the severity of the stroke. In the present study, the Persian version of NIHSS showed an excellent Cronbach’s alpha coefficient confirming its reliability to predict the severity of stroke in hospitalized stroke (ischemic and hemorrhagic) patients. There was only one previous study evaluating the reliability of the Persian version of NIHSS in non-hospitalized ischemic stroke patients [13]. In line with our finding, the Cronbach’s alpha coefficient of the previous study (0.89) also indicated the reliability of Persian version of NIHSS in non-hospitalized stroke patients. Also, the finding of a scaling success of 80% in the present study demonstrated the validity of Persian version NIHSS. This is inconsonant with the results of the previous study using a different method for validity assessment of Persian version NIHSS [13].

The agreements between the outcomes of the current and previous research indicate that the Persian version of NIHSS can be implemented in both hospitalized and non-hospitalized stroke patients. Furthermore, the Cronbach’s alpha coefficient obtained in this study (0.81) was similar to Cronbach’s alpha coefficient reported by Kasner et al., for the English version of NIHSS (0.82) [3]. It suggests that the reliability of the current Persian version of NIHSS is comparable to that of the English version. Despite various methods used, the validity and reliability of NIHSS have also been reported in other languages such as Chinese, Italian, Spanish, Estonian, German, Hindi, Hungarian, Marathi, Telugu, and Portuguese [18-22]. mNIHSS, a modified form of NIHSS, is more convenient for use in emergency and stroke wards in clinical settings due to its several advantages over NIHSS. The current study, for the first time, reports the validity and reliability of the Persian version of mNIHSS for the appraisal of stroke severity in hospitalized patients. No other study has yet evaluated the validity and reliability of the Persian version of mNIHSS to compare with our results. However, our findings are in agreement with studies evaluated the validity and reliability of English version of mNIHSS. In the present study, both Persian versions of NIHSS and mNIHSS showed excellent Cronbach’s alpha coefficients representing that both questionnaires have appropriate reliability (internal consistency). These results indicated that the questions in both Persian versions of NIHSS and mNIHSS were generally consistent as a group of items to assess the severity of stroke. Nevertheless, the higher value of Cronbach’s alpha coefficient for mNIHSS revealed that the mNIHSS questions were more consistent and reliable than those of NIHSS. Comparing the validity of two Persian questionnaire, it was demonstrated that mNIHSS was more valid than NIHSS. All questions in mNIHSS had the correlation coefficient of greater than 0.4 leading to a scaling success of 100%. Though, 3 out of 15 questions in NIHSS showed a correlation coefficient of less than 0.40 directing to a scaling success of 80% for Persian version of NIHSS. Interestingly, these three questions including facial palsy, limb ataxia, and dysarthria were those that have been removed from NIHSS to generate mNIHSS. These results are in agreement with previous studies proposing that these questions are redundant and should be removed from NIHSS. In addition, correlation range of convergent validity (0.49-0.79) in mNIHSS was not scattered as much as in NIHSS (0.29-0.83), which represents more coherence among questions of mNIHSS compare to NIHSS. These results demonstrated that mNIHSS was more valid than NIHSS Persian version. The findings of this study are in line with previous studies indicating that the English version of mNIHSS is more valid and reliable than NIHSS [6, 7, 23, 24]. Considering that mNIHSS is easier to implement, needs shorter time to execute and has fewer redundant items, the valid and reliable Persian version of mNIHSS can resolve the discontent with NIHSS and helps nurses, clinicians and researchers to have an easy, suitable and applicable questionnaire to assess the severity of stroke in stroke and emergency wards as well as clinical trials. Replacing NIHSS with mNIHSS in stroke and emergency wards may increase the tendency for reporting the severity of stroke, which is necessary for clinical and nursing decisions as well as in treatment and rehabilitation programs. The limitation of this study was not able to implement the questionnaires with more than one researcher that prevented the calculation of the kappa coefficient (inter-rater agreement). Hence, future study should address this issue.

Conclusion

Persian versions of NIHSS and mNIHSS were reliable and valid questionnaires. However, the mNIHSS was more valid and reliable questionnaire than NIHSS. With regards to superiorities of mNIHSS vs. NIHSS, replacing NIHSS with mNIHSS in stroke and emergency wards and clinical trials may be an appropriate move.

Acknowledgment

This study was based on work done by Reza Dehghani in partial fulfillment of the requirements for his Ph.D. thesis in Pharmacology. This study was supported by a grant from vice chancellor for research affair Shiraz University of Medical Sciences (grant No. 10400). The authors appreciate the kind and helpful assistance of nurses in the stroke ward of Namazi teaching hospital, Ms.F.Shojaei, Ms.Z.Otofat, Ms. F. Pakizekar, and Ms. E. Monem. The authors wish to thank Mr. H.Argasi at the Research Consultation Center (RCC) at Shiraz University of Medical Sciences for his invaluable assistance in editing this manuscript. Also the authors would like to thank Dr. Peyman Jafari at Department of Biostatistics, Shiraz University of Medical Sciences, Shiraz, and Fars, Iran for his precious help for data analysis.

Conflict of Interest

The authors declare no conflict of interest.

Table 2. Cronbach’s Alpha and Convergent Validity for Persian Version of NIHSS and mNIHSS

Questioner	Items	Cronbach’s alpha	Convergent validity
			Range of correlationa	scaling successb (%)
NIHSS	15	0.81	0.29-0.83	0.8 (80%)
mNIHSS	11	0.86	0.49-0.79	1(100%)

a Minimum and maximum of correlation between each item and total score.

b The number of correlations above 0.40/the total number of correlations.

References

1. Goldstein LB, Samsa GP. Reliability of the National Institutes of Health Stroke Scale. Extension to non-neurologists in the context of a clinical trial. Stroke. 1997;28(2):307-10.
2. Adams HP, Jr., Davis PH, Leira EC, Chang KC, Bendixen BH, Clarke WR et al. Baseline NIH Stroke Scale score strongly predicts outcome after stroke: A report of the Trial of Org 10172 in Acute Stroke Treatment (TOAST). Neurology. 1999;53(1):126-31.
3. Kasner SE, Chalela JA, Luciano JM, Cucchiara BL, Raps EC, McGarvey ML et al. Reliability and validity of estimating the NIH stroke scale score from medical records. Stroke. 1999;30(8):1534-7.
4. Goldstein LB, Bertels C, Davis JN. Interrater reliability of the NIH stroke scale. Arch neurol. 1989;46(6):660-2.
5. De Haan R, Horn J, Limburg M, Van Der Meulen J, Bossuyt P. A comparison of five stroke scales with measures of disability, handicap, and quality of life. Stroke.1993;24(8):1178-81.
6. Meyer BC, Hemmen TM, Jackson CM, Lyden PD. Modified National Institutes of Health Stroke Scale for use in stroke clinical trials: prospective reliability and validity. Stroke. 2002;33(5):1261-6.
7. Meyer BC, Lyden PD. The Modified National Institutes of Health Stroke Scale (mNIHSS): Its Time Has Come. Int J Sroke.2009;4(4):267-73.
8. Siniscalchi A, Sztajzel R, Malferrari G, Gallelli L. The National Institutes of Health Stroke Scale: Its Role in Patients with Posterior Circulation Stroke. Hosp top. 2017:1-3.
9. Borhani-Haghighi A, Safari R, Heydari ST, Soleimani F, Sharifian M, Kashkuli SY, Khayatghuchani MN, Azadi M, Shariat A, Safari A, Lankarani KB. Hospital mortality associated with stroke in southern Iran. Iran J Medical Sci. 2013 Dec;38(4):314.
10. Borhani-Haghighi A. Review of Stroke: The 2nd International and the 8th National Iranian Stroke Congress : Shiraz, Iran. GMJ.2016.5(1):1-2
11. Ghandehari K. Epidemiology of Stroke in Iran. GMJ. 2016;5(S1):3-9.
12. Azarpazhooh MR, Etemadi MM, Donnan GA, Mokhber N, Majdi MR, Ghayour-Mobarhan M et al. Excessive incidence of stroke in Iran: evidence from the Mashhad Stroke Incidence Study (MSIS), a population-based study of stroke in the Middle East. Stroke.2010;41(1):e3-e10.
13. Kazemnejad-Leili E, Rezaei S, Hosseini-Nejad M, Bakhshayesh-Eghbali B, Saberi A, Keshavarz P. The Applicability, Concurrent Validity and Internal Consistency Reliability of the Persian Version of the National Institutes of Health Stroke Scale (NIHSS): Evidences for Gender Differences. gums-cjns. 2016;2(4):18-28.
14. Fayers PM, Machin D. Sample Sizes. Quality of Life. John Wiley & Sons, Ltd; 2007. p. 247-69.
15. Parvizi MM, Amini M, Dehghani MR, Jafari P, Parvizi Z. Psychometric properties of the Persian version of the Ambulatory Care Learning Educational Environment Measure (ACLEEM) questionnaire, Shiraz, Iran. Adv Medical Educ Pract. 2016;7:559-66.
16. Tavakol M, Dennick R. Making sense of Cronbach's alpha. Int J Med Educ. 2011;2:53-5.
17. Sim J, Arnell P. Measurement validity in physical therapy research. Phys Ther. 1993;73(2):102-10; discussion 10-5.
18. Sun TK, Chiu SC, Yeh SH, Chang KC. Assessing reliability and validity of the Chinese version of the stroke scale: scale development. Intern J Nurs Stud. 2006;43(4):457-63.
19. Pezzella FR, Picconi O, De Luca A, Lyden PD, Fiorelli M. Development of the Italian version of the National Institutes of Health Stroke Scale: It-NIHSS. Stroke. 2009;40(7):2557-9.
20. Dominguez R, Vila JF, Augustovski F, Irazola V, Castillo PR, Rotta Escalante R et al. Spanish cross-cultural adaptation and validation of the National Institutes of Health Stroke Scale. Mayo Clin Pro. 2006;81(4):476-80.
21. Berger K, Weltermann B, Kolominsky-Rabas P, Meves S, Heuschmann P, Bohner J et al. [The reliability of stroke scales. The german version of NIHSS, ESS and Rankin scales]. Fortschr Neurol Psychiatr. 1999;67(2):81-93.
22. Kwah LK, Diong J. National Institutes of Health Stroke Scale (NIHSS). J Physiother. 2014;60(1):61.
23. Kasner SE, Cucchiara BL, McGarvey ML, Luciano JM, Liebeskind DS, Chalela JA. Modified National Institutes of Health Stroke Scale can be estimated from medical records. Stroke. 2003;34(2):568-70.
24. Lyden PD, Lu M, Levine SR, Brott TG, Broderick J, Group NrSS. A modified National Institutes of Health Stroke Scale for use in stroke clinical trials: preliminary reliability and validity. Stroke. 2001;32(6):1310-7.