|Year : 2019 | Volume
| Issue : 3 | Page : 170-176
Development and psychometric evaluation of data collection tools for Iranian integrated road traffic injury registry: Registrar-station data collection tool
Soudabeh Marin1, Faramarz Pourasghar2, Alireza Moghisi3, Bahram Samadirad4, Mashyaneh Haddadi3, Davoud Khorasani-Zavareh5, Homayoun Sadeghi-Bazargani6
1 Department of Biostatistics and Epidemiology, School of Health, Tabriz University of Medical Sciences, Tabriz, Iran
2 Department of Health Service Management, School of Management and Medical Informatics, Tabriz University of Medical Sciences, Tabriz, Iran
3 Injury Prevention and Safety Promotion Department, Disaster and Emergency Management Center, Ministry of Health and Medical Education, Tehran, Iran
4 Forensic Medicine Research Center, Legal Medicine Organization, Tehran, Iran
5 Workplace Health Promotion Research Centre (WHPRC), Shahid Beheshti University of Medical Sciences, Tehran, Iran
6 Road Traffic Injury Research Center, Tabriz University of Medical Sciences, Tehran, Iran
|Date of Submission||04-Jul-2018|
|Date of Decision||06-Sep-2019|
|Date of Acceptance||21-Sep-2019|
|Date of Web Publication||26-Nov-2019|
Dr. Homayoun Sadeghi-Bazargani
Road Traffic Injury Research Center, Tabriz University of Medical Sciences, Tabriz
Source of Support: None, Conflict of Interest: None
Background: Comprehensive and accurate data are fundamentally needed for effective management of road traffic injuries (RTIs). Existing sources of RTI reports have a huge underestimation and inaccuracy at some levels. The aim of this study was to develop and validate the registrar-station data collection tool as a part of the Iranian Integrated Road Traffic Injury Registry (IRTIR). Materials and Methods: This study was conducted in Tabriz University of Medical Sciences in 2018. A data collection tool was developed to be used by the registrar for inpatient section of IRTIR by information retrieved from the literature review and road traffic experts' need assessment. The content validity of the preliminary tool was assessed. The feasibility of the tool was tested in two regional referral injury hospitals. Intra- and inter-rater reliability of the tool was evaluated using the individual/absolute intra-class correlation coefficient (ICC) and Kappa. Validity was revisited after 1 year of the pilot study. Results: The registrar-station data collection tool of IRTIR included 53 items, in five categories. Content validity was approved (modified content validity index was 0.8–1 and content validity ratio was one for all items). ICC was >0.6 for all items, and kappa index ranged between 0.69 and 0.92. The nurse data collection tool of IRTIR was applicable in the pilot phase. Conclusions: The Registrar-Station data collection tool of IRTIR was confirmed as a valid and reliable tool for inpatient traffic injuries as a part of the Iranian IRTIR.
Keywords: Data collection tool, injury registry, road traffic, validation
|How to cite this article:|
Marin S, Pourasghar F, Moghisi A, Samadirad B, Haddadi M, Khorasani-Zavareh D, Sadeghi-Bazargani H. Development and psychometric evaluation of data collection tools for Iranian integrated road traffic injury registry: Registrar-station data collection tool. Arch Trauma Res 2019;8:170-6
|How to cite this URL:|
Marin S, Pourasghar F, Moghisi A, Samadirad B, Haddadi M, Khorasani-Zavareh D, Sadeghi-Bazargani H. Development and psychometric evaluation of data collection tools for Iranian integrated road traffic injury registry: Registrar-station data collection tool. Arch Trauma Res [serial online] 2019 [cited 2019 Dec 13];8:170-6. Available from: http://www.archtrauma.com/text.asp?2019/8/3/170/271690
| Introduction|| |
Road traffic injuries (RTIs) are growing concerns in both public health and development. RTIs in Iran moved from the fifth leading cause of death in 1990 to the fourth leading cause of death in 2016.
In most countries, road traffic crash data are gathered routinely by various traffic-related organizations. There are significant discrepancies between reports on RTIs by different organizations. Previous studies consistently demonstrated that police records, which are internationally the main source of RTI's statistics, underreported RTIs,,, and two-wheelers are more likely to be underreported. RTI data need to be accurate and reliable to be comparable and inform policy-makers about the magnitude of the problem. Of 178 countries, only 22% provided adequate RTI data, which was pronounced by the World Health Organization. The principal obstacle to productive road traffic management, particularly in Iran, is “the lack of a system approach.” A basic data collection system is the prerequisite for improving road safety situation as well as for other kinds of injury prevention.,, Development of a well-established surveillance system as well as system approach on road traffic management, which produces timely, accurate data, is highly recommended to reduce road traffic burden. This approach should provide information about all fatal RTIs and severity of injuries as well as comprehensive data on vehicle, road user, and the environment to risk factor identification and goal setting.,,
Currently, the Police and Ministry of Health and Medical Education (MoHME) together with some different subdivision systems provide main sources of road traffic data in Iran. Considering the magnitude and importance of road traffic toll in Iran, a more comprehensive and precise data are needed for effective road safety management. The Iranian Integrated Road Traffic Injury Registry (IRTIR) project is a specific RTI registry that collects and integrates data in various stations. The present study aimed at developing and psychometrically evaluating the registrar-station data collection tool for inpatient RTIs as a part of the IRTIR implementation.
| Materials and Methods|| |
The current study was part of a larger research project, IRTIR, which was implemented in 2017 supervised by Tabriz University of Medical Sciences. The pilot study of the IRTIR was conducted in East Azerbaijan Province, Iran. The first scale-up of the IRTIR study will be conducted in six North-West Provinces of Iran, including East Azerbaijan, West Azerbaijan, Ardabil, Kurdistan, Zanjan, and Qazvin. Several sectors were involved in the IRTIR project, including the MoHME, police (mostly providing crash-scene data), Forensic Medicine Organization, and Iranian Red Crescent. In the MoHME section, there were several parts: (1) physician station (mostly focusing on injury severity), (2) registrar-station (focused on safety behaviors/equipment), and (3) the third station providing other information extracted from patients' hospital files. A detailed protocol of the IRTIR is provided elsewhere.
This article describes the development and psychometric evaluation of registrar-station data collection tool as a pilot study of the IRTIR. [Figure 1] illustrates the details of seven stages of this study; starting from development to finalization of the tool.
|Figure 1: The process of development and evaluation of the registrar-station data collection tool of Integrated Road Traffic Injury Registry|
Click here to view
Data collection tool development
First, several databases were searched to identify publications on RTI registry systems and fundamental items were retrieved from the literature review. In the next phase, current national RTI data collection tools, including the police data collection tool (form 114 KAM), forensic medicine organization tool (form no. 2) and emergency medical services were reviewed. Then, a panel of experts from different scientific backgrounds was formed to determine necessary RTI registry elements (Stage 1).
Several session of experts' panel was held to identify potentially essential elements for national RTI registry. At the final step, the preliminary edition of the tool was developed (Stage 2).
In the third stage, the preliminary edition of the tool was sent to 13 external experts to examine the content validity of the tool. They were asked to comment on the relevance, necessity, clarity, and simplicity of the items. The experts were of different academic backgrounds: epidemiology, road traffic management, EMS, forensic medicine, orthopedic, medical informatics, trauma department nurses, and medical record experts were also present in this panel of expert. They were selected based on their professional expertise in the research field. The content validity ratio (CVR) and modified content validity index (CVI) were used to examine content validity quantitatively. The modified CVI (the modified Kappa) was preferred over traditional content validity I-CVI  as it also measures chance agreement. The CVR is also used to evaluate the necessity and CVI related to clarity, simplicity, and relevance of the items for the research purposes. Scores higher than 0.75 were approved by the researchers. Qualitative content validity was evaluated using the experts' comments.
Experts' comments assisted the researchers in making necessary reforms in the tool. The revised version of the tool was re-distributed among the experts to confirm the content validity (CVI and CVR) (Stage 4).
To evaluate the feasibility, the tool was used for all road traffic victims (RTVs) admitted to the Imam-Reza University Hospital in Tabriz, Northwest of Iran, for 2 weeks. Executive difficulties and time consumption were investigated within this period. To assess time consumption, the registrars were asked to record the time needed to fill-up the tool. Then, the tool was used for 1 year (2016–2017) in Imam Reza and Shohada hospitals (two main trauma referral hospitals in the northwest of Iran) to investigate mid-term feasibility of the tool, and necessary revisions were made (the pilot phase). The tool will be used for a further 6 months in Tabriz and six other provinces (the first scale-up) before finalization as the last scale-up (Stage 5).
The reliability of each item was assessed separately. In addition, inter- and intra-rater reliability was conducted to evaluate the general reliability of the tool.
[Figure 2] illustrates the inter- and intra-rater reliability assessment process. The individual/absolute intra-class correlation coefficient (ICC) and Kappa value were used to assess the reliability. Four registrars (two males and two females) were employed to independently interview RTVs using a fully crossed design for a sample of subjects. To do so, the research team used fully crossed design because it helped assessing systematic bias between different registrars and improved overall inter-rater reliability estimation. Test–retest reliability was used to assess intra-rater reliability and stability of the instrument. After a week of the time interval between the tests, fort RTVs were interviewed in test and re-test phases.
|Figure 2: The Integrated Road Traffic Injury Registry's nursing station data collection tool reliability assessment process diagram|
Click here to view
In Stage 6, every RTV was interviewed twice by the same registrar. The intra-rater reliability was also assessed to demonstrate consistency among the registrars. Inter-rater reliability was assessed using the administration and re-administration method. In this stage, two different registrars interviewed 40 RTVs twice. The time interval between administration and re-administration was only a few hours. The researchers designed a random sequence for the registrars to interview the patients (Stage 6).
The reliability for dichotomous items was assessed using Kappa statistics. Kappa values of 0.8 and >0.8 represented excellent agreement, between 0.61 and 0.8 acceptable agreement, between 0.41 and 0.6 moderate agreement, and <0.40 weak agreement. The ICC was used to assess the reliability of numeric items. ICC ≤0.4 was considered poor to fair, 0.41–0.6 moderate, 0.61–0.8 good, and >0.8 excellent. A value of P < 0.05 indicated statistical significance. Data were analyzed using the Stata software version 14 (StataCorp, Texas, US).
The study protocol received approval from the Ethical Committee of Tabriz University of Medical Sciences under the ethical registration code TBZMED.REC.1394.1182. To ensure the ethical issues of this study, the participants agreed to grant verbal informed consent to the project.
| Results|| |
The final registrar data collection tool consisted of 53 items divided into five categories: (1) Crash description, (2) victim's demographic data, (3) crash and environment data, (4) vehicle data and (5) victim's behavior data. Five items of the 57 preliminary items were excluded based on content validity and reliability assessment results. The excluded items were as follows: the history of driving offenses, cause of driving offense, make and model of vehicle, severity score, and medical care received. Some items such as severity score and outcome indicators were excluded from the registrar data collection and transferred to other IRTIR's data collection tools such as physician data collection tool. One item (history of medication) was added based on the experts' opinion. The items were either open questions or multiple-choice questions. [Table 1] shows categories and items in each category of the registrar data collection tool.
|Table 1: Information collected by the registrar data collection tool of Iranian Integrated Road Traffic Injury Registry|
Click here to view
The panel of experts verified the content validity of both quantitative and qualitative dimensions of the tool. Moreover, both qualitative and quantitative content validity of the final version was approved by the panel of experts. The content validity assessed through modified CVI and CVR showed a satisfactory result.
The CVR of the preliminary version was 0.66 for total variables in the first assessment of content validity. After revising the scale in accordance with the applying experts' opinion, the questionnaire was redistributed among experts. Regarding content validity assessment, CVI of all the items was between 0.80 and 1, which was satisfactory, and CVR for all the items was 1, indicating that all the items were essential for the study purposes. The modified CVI for all the items was also 1 (complete agreement between the experts). Moreover, qualitative content validity was confirmed through written feedback on the clarity, necessity, and relevance of questions by a panel of 13 experts. The inter-rater reliability assessment yielded the Kappa value between 0.62 and 0.92 for items with dichotomous measurements and the individual/absolute ICC ranged from 0.61 to 0.79. Further, the test-retest reliability evaluation yielded the Kappa value between 0.62 and 0.92 and the individual/absolute ICC ranged from 0.61 to 0.79. Completing each tool took 10–15 min.
The registrar data collection tool as a part of the IRTIR was confirmed to be feasible to collect data from all inpatient victims of RTIs in both Imam-Reza and Shohada Hospitals (the trauma referral hospitals in the region).
| Discussion|| |
This study was, to the best of the researchers' knowledge, the first research aimed at developing and psychometrically validating a tool for registrar-station of the IRTIR. The tool addresses inpatient RTVs, and it must be filled up by a nurse or a trained registrar. Containing five main categories to tap all essential aspects of registrar-station as a part of IRTIR, the scale is hope to provide comprehensive data. Previously, some agencies such as police, insurance companies, and forensic medicine body sought to collect data for RTIs. However, they did not appear to yield comprehensive and detailed information.,,, Collecting data in the registrar-station have some advantages as follows: first, it can be an indication of concurrent validity at the evaluation phase; second, some parallel collection of data improves the registry coverage through compensating partly missing information at parallel stations; third, some questions may receive more valid answers in registrar-stations than in other data collection stations. For example, road users under the influence of alcohol or drug abuse may be less likely to report the crash to the police  and physician or give inaccurate answers to them (alcohol consumption is intrinsically an unlawful activity in Iran, and hence, the police's report in this aspect may be less valid). However, recording alcohol consumption in the registrar-station of the IRTIR is not included in the hospital medical record, and hence provides more accurate information. Subsequent to the national implementation of the IRTIR, minimizing some of the items may be taken into consideration. Although collecting each one of these items is of value individually, their value increases in an integrated system. We included all stakeholders in the development of the tool, which made the project more practical, and assessed the feasibility of the tool at the first-scale-up phase. Developing a postdischarge follow-up tool for severe cases of RTIs as a part of the IRTIR is recommended for future.
Some of road traffic registries worldwide gather information through existing traffic data collection systems and then integrate the existing data. Given that, overlaps might emerge in some areas or some RTIs in terms of severity and vehicle involvement might be left underreported.,, For example, RTI surveillance in Australia is a combination of police reports, hospital admissions, and emergency department information. A study in Australia showed that although trauma registry data are useful to improve road traffic safety efforts, they have limitations and are inadequate in some aspects.
What sets this study apart is developing such a detailed data collection tool for the inpatient section of the IRTIR. Including 53 items in registries covering all types of crashes regardless of their severity may not be cost-effective or feasible. However, in the IRTIR developed to cover severe crashes, it seems more reasonable to include them, especially when some nurses are exclusively hired to do the job, as in our case.
In this study, a standard methodology was used for developing and psychometrically evaluating the registrar-station data collection tool. Earlier registries were developed and used without any report on standard psychometric validation, which may lead to massive accumulation of invalid and unreliable crash data. Although the evaluation process slowed down the implementation of the registry, it increased the reliability and validity of the data, and eventually resulted in improving the cost-effectiveness and efficiency of the registry. Moreover, the research team assessed the quantitative and qualitative content validity at two stages: (at the beginning of the study in the pilot phase and then a year after pilot study). To our best knowledge, no study has investigated the validity and reliability of the instrument through 1 year pilot phase; therefore, we regarded this as one of the strengths of the current study. Because of the national importance of the issue, assessing the validity and reliability of the items in long-term period (for example, 5 years after implementation of the tool at national level) is suggested. According to the findings of this study, reliability and validity for all the items tool were satisfactory; therefore, each item could be separately used and shortening of the tool in future will not affect the reliability and validity of the entire tool. In addition, robust statistical methods for assessing validity and reliability were also employed. We used modified content validity, which takes into account the consistency of agreement.
Despite the emphasis placed on the following issues, they were not focused upon in this paper. For example, the question as to the data collection is tool is qualified enough to be employed for all RTIs is open to investigation. In addition, the issue of whether fixed nursing staff should be employed to fill out the scale or other human resources are also qualified to complete the tool needs attention. Cost-effectiveness of the registrar data collection scale deserves further investigation.
| Conclusions|| |
The registrar data collection tool was confirmed to be a reliable and valid tool to be used for inpatient RTIs in the Iranian IRTIR and similar settings.
Financial support and sponsorship
The study was funded by the Iranian Ministry of Health through the contract number 700/1482. We committed that this manuscript has not been published elsewhere and it is not under consideration for publication anywhere else. Publication of this manuscript and order of authors has been approved by all co-authors.
Conflicts of interest
There are no conflicts of interest.
| References|| |
Violence World Health Organization, Injury Prevention, World Health Organization. Global Status Report on Road Safety 2013: Supporting a Decade of Action. World Health Organization; 2013.
Bazargan-Hejazi S, Ahmadi A, Shirazi A, Ainy E, Djalalinia S, Fereshtehnejad SM, et al.
The burden of road traffic injuries in iran and 15 surrounding countries: 1990-2016. Arch Iran Med 2018;21:556-65.
Hatamabadi HR, Vafaee R, Haddadi M, Abdalvand A, Soori H. Necessity of an integrated road traffic injuries surveillance system: A community-based study. Traffic Inj Prev 2011;12:358-62.
Dandona R, Kumar GA, Ameer MA, Reddy GB, Dandona L. Under-reporting of road traffic injuries to the police: Results from two data sources in urban India. Inj Prev 2008;14:360-5.
Razzak JA, Luby SP. Estimating deaths and injuries due to road traffic accidents in Karachi, Pakistan, through the capture-recapture method. Int J Epidemiol 1998;27:866-70.
Ward H, Lyons R, Thoreau R. Under-Reporting of Roadcasualties? Phase 1. Department for transport, London; 2006.
Jeffrey S, Stone DH, Blamey A, Clark D, Cooper C, Dickson K, et al.
An evaluation of police reporting of road casualties. Inj Prev 2009;15:13-8.
Khorasani Zavareh D, Mohammadi R, Laflamme L, Naghavi M, Zarei A, Haglund BJ, et al.
Estimating road traffic mortality more accurately: Use of the capture-recapture method in the West Azarbaijan province of Iran. Int J Inj Contr Saf Promot 2008;15:9-17.
World Health Organization. Data systems: A road Safety Manual for Decision-Makers and Practitioners. World Health Organization; 2010.
Khorasani-Zavareh D, Mohammadi R, Khankeh HR, Laflamme L, Bikmoradi A, Haglund BJ, et al.
The requirements and challenges in preventing of road traffic injury in Iran. A qualitative study. BMC Public Health 2009;9:486.
Hizal Hanis H, Sharifah Allyana S, editors. The construction of road accident analysis and database system in Malaysia. 14th
IRTAD Conference; 2009.
Khorasani-Zavareh D, Shoar S, Saadat S. Antilock braking system effectiveness in prevention of road traffic crashes in Iran. BMC Public Health 2013;13:439.
Bliss T, Breen J. Country Guidelines for the Conduct of Road Safety Management Capacity Reviews and the Specification of Lead Agency Reforms, Investment Strategies and Safe System Projects. World Bank Global Road Safety Facility; 2009.
Khorasani Zavareh D. Toward safety Promotion among Road Users: Epidemiology and Prevention of Road Traffic Injuries in Iran. 2009.
Khorasani-Zavareh D. System versus traditional approach in road traffic injury prevention: A call for action. J Inj Violence Res 2011;3:61.
Movahedi M, Soori H, Eyni E, Mehmandar M. Comparison of traffic accident nformation provided by the major sources in Iran with the pattern used in American and Asian countries. Payesh 1390;11:21-7.
Homayoun Sadeghi-Bazargani AS, Kolivand PH, Zakeri R, Haddadi M, Golestani M, Noori H. Developing a National Integrated Road Traffic Injury Registry System with Applicability to LMICs: A Conceptual Model; 2018.
Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health 2007;30:459-67.
Hallgren KA. Computing inter-rater reliability for observational data: An overview and tutorial. Tutor Quant Methods Psychol 2012;8:23-34.
Sim J, Wright CC. The kappa statistic in reliability studies: Use, interpretation, and sample size requirements. Phys Ther 2005;85:257-68.
Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159-74.
Bartko JJ. The intraclass correlation coefficient as a measure of reliability. Psychol Rep 1966;19:3-11.
Abobakri O, Sadeghi-Bazargani H, Asghari-Jafarabadi M, Alizadeh Aghdam MB, Imani A, Tabrizi J, et al.
Development and psychometric evaluation of a socioeconomic status questionnaire for urban households (SESIran): The preliminary version. Health Promot Perspect 2015;5:250-60.
Janstrup KH, Kaplan S, Hels T, Lauritsen J, Prato CG. Understanding traffic crash under-reporting: Linking police and medical records to individual and crash characteristics. Traffic Inj Prev 2016;17:580-4.
Watson A, Watson B, Vallmuur K. Estimating under-reporting of road crash injuries to police using multiple linked data collections. Accid Anal Prev 2015;83:18-25.
Rojas Medina Y, Espitia-Hardeman V, Dellinger AM, Loayza M, Leiva R, Cisneros G, et al.
A road traffic injury surveillance system using combined data sources in peru. Rev Panam Salud Publica 2011;29:191-7.
Boufous S, Williamson A. Work-related traffic crashes: A record linkage study. Accid Anal Prev 2006;38:14-21.
Mitchell R, Williamson A, Curtis K. What is the potential of trauma registry data to be used for road traffic injury surveillance and informing road safety policy? J Safety Res 2011;42:345-50.
[Figure 1], [Figure 2]