The absence of data on driving under the influence of alcohol in road traffic studies: a scoping review of non-randomized studies with vote counting based on the direction of effects of alcohol policies
Substance Abuse Treatment, Prevention, and Policy volume 18, Article number: 46 (2023)
Data on driving under the influence of alcohol (DUIA) are not always available, accurate, or reliable, making it difficult to study the effects of alcohol policies on road traffic outcomes. The objectives of our study were twofold: 1) to describe how road traffic outcomes of alcohol policies are assessed when DUIA data are missing, and 2) to explore the effects of alcohol policies when DUIA data are missing.
We conducted a scoping review of non-randomized studies that assessed the road traffic outcomes of alcohol policies when DUIA data are missing. Until November 2021, we searched studies published between 2000 and 2021, in English or French, via MEDLINE, APA PsycInfo, CINAHL, and SocINDEX. We assessed the risk of bias in the included studies with the Quality Assessment Tool for Before-After (Pre-Post) Studies With No Control Group. The selection process, data extraction, and the risk of bias assessment were conducted independently and in duplicate. We used vote counting based on the direction of the effects of alcohol policies as a synthesis method. The protocol for this review was published in PROSPERO under record number CRD42021266744.
Twenty-four eligible studies were included. Regarding objective 1, most studies used uncontrolled interrupted time series designs to assess road traffic fatalities resulting from night-time crashes. The reasons for missing DUIA data were generally not reported. Regarding objective 2, we found evidence for an association between alcohol policies and decreased road traffic fatalities. Subgroup analyses found no evidence for an association between methodological modifiers and positive effect directions for road traffic fatalities.
Caution is needed when interpreting road traffic outcomes associated with alcohol policies when DUIA data are missing. Greater efforts should be made to improve the reporting of outcomes assessments. Future studies must address several methodological issues (e.g., more granular data, well-defined intervention and implementation, and controlled designs). Our results should be compared to those from others reviews where DUIA data were available to confirm or recalibrate the associations found in studies where DUIA data were missing.
Road traffic crashes, injuries, and deaths have serious public health and economic consequences affecting sustainable development [1, 2]. According to the most recent analyses for the Global Burden of Diseases, Injuries, and Risk Factors Study (GBD), road traffic injuries were responsible for more than 900,000 deaths in 2019 and are currently among the top causes of disability-adjusted life years, mainly affecting the male population, adolescents (10 to 24 years) and adults (25 to 49 years) .
Acute alcohol consumption, even at small doses, has been consistently associated with poor driving performance  and increased risk for road traffic crashes, injuries, and deaths [5, 6]. Alcohol consumption is one of the main contributors to adverse road traffic outcomes. Borges et al. , using the GBD 2019 data, found that 6.6% of all road traffic injuries in 2019 were attributable to alcohol consumption. The World Health Organization (WHO) estimated that alcohol consumption accounted for 27% of road traffic deaths worldwide .
Given the severe consequences of driving under the influence of alcohol (DUIA), the WHO has consistently urged member states to advance and enforce countermeasures explicitly aimed at reducing DUIA, such as low blood alcohol concentration (BAC) limits and sobriety checkpoints [9, 10]. These measures have been comprehensively studied and are mainly focused on increasing the perceived risk of detection and punishment [11,12,13]. A recent overview of systematic reviews concluded that sobriety checkpoints and random breath testing were consistently found to reduce alcohol-related road traffic crashes . However, a broad range of strategies may reduce the problem of DUIA . There is evidence that general alcohol policies restricting access to alcohol (e.g., alcohol taxation and minimum legal drinking age laws) may indirectly reduce road traffic deaths by decreasing high-risk alcohol consumption and DUIA [11, 15,16,17,18].
Notwithstanding, the impact assessment of these strategies is severely limited by the availability and reliability of road safety data, being prone to errors and inconsistencies [19,20,21,22]. Likewise, the legal and technical capabilities for DUIA enforcement vary considerably between and within countries . The challenges around road safety data and DUIA enforcement particularly disadvantage low- and middle-income countries (LMIC), those most affected by the burden of road traffic injuries and fatalities [23,24,25]. In some high-income countries, objective measurement of DUIA occurs sporadically and is not always registered in police reports [12, 20]. Thus, studies had to rely on surrogate measures (e.g., night-time fatal crashes), assuming a higher probability of alcohol involvement for these traffic crashes .
The objectives of our review were twofold: 1) to describe how traffic crashes outcomes of alcohol policies are assessed when DUIA data are missing, and 2) to explore the effects of alcohol policies when DUIA data are missing. The potential contribution of our study is relevant and timely. The descriptive approach may suggest alternative research designs, and easily accessible and reliable surrogate measures in settings where DUIA data may be challenging to obtain – especially in LMIC. The exploratory evaluation of alcohol policies – which we have broadly categorized as general and specific – and their effects on traffic outcomes – i.e., traffic crashes, injuries, and fatalities – is a critical application for surrogate measures of DUIA data.
Material and methods
We conducted a scoping review with vote counting based on the direction of effect, combining a descriptive with an exploratory synthetic approach. We reported our study per the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement . The protocol for this study was registered prospectively in the International Prospective Register of Systematic Reviews (PROSPERO) under registration number CRD42021266744 . All protocol updates are detailed in Additional file 1: Appendix 1.
We followed the Population, Intervention, Comparator, Outcomes, and Study Design (PICOS) framework to define eligibility criteria at the study level . Studies had the following characteristics to be eligible:
Population: All road traffic users (i.e., passengers, pedestrians, and drivers of motorized or non-motorized vehicles). We did not exclude any population based on age, sex, ethnicity, or other characteristics.
Intervention: An intervention condition in which the population was exposed to an alcohol policy. Alcohol policies were included based on their expected direct and indirect effects on DUIA. We followed the classification implied by Shults et al.  in which general (e.g., banning alcohol production and sale without a license) and specific (e.g., sobriety checkpoints) alcohol policies can be found. We included studies testing a combination of different alcohol policies.
Comparator: A control condition in which the population has not been exposed to the alcohol policies studied or acted as its own control (i.e., pre-intervention period). Studies with an active control condition (i.e., other alcohol policy) were also included.
Outcomes: Traffic crashes, injuries, and fatalities. These outcomes could be expressed as counts or rates per population. We excluded studies that reported DUIA data.
Study design: We followed the definition by Reeves et al.  of non-randomized studies: "any quantitative study estimating the effectiveness of an intervention (benefit or harm) that does not use randomization to allocate units to comparison groups". We excluded randomized controlled trials. Although labeling of non-randomized studies varies widely, examples of studies considered for inclusion were interrupted time series (either controlled or not), before-and-after studies (either controlled or not), retrospective or prospective cohort studies, case–control studies, and cross-sectional studies . We excluded case studies and case series.
To be included, reports must have been published from 2000 to 2021. Searches were limited to reports published in English or French. Publication types not containing original results (e.g., editorials, systematic reviews, and meta-analyses) were excluded.
We searched the MEDLINE, APA PyscInfo, CINAHL, and SocINDEX databases via EBSCOhost from inception to the present. We also searched the reference lists of included articles for additional eligible publications. The date when each source was last searched or consulted was November 2021. We conducted an updated MEDLINE search of reports published between November 2021 and April 2023 (Additional file 1: Appendix 1).
We used a simple and broad search strategy based on free-text words to maximize search results related to the intersection between DUIA and traffic outcomes (i.e., "alcohol" and "traffic crashes"). No filters or limiters were used except for the language limiter. The detailed rationale for the search strategy and its comparison with a more complex search strategy is provided in Additional file 1: Appendix 1.
All the records identified through the search process were extracted and imported into an Excel spreadsheet. Duplicate records were manually removed, titles and abstracts were screened, and full-text reports were retrieved and assessed for eligibility. Reports were examined to avoid duplication of data from the same study. JJ and PM conducted the selection process through independent and duplicate reviews. Disagreements were solved through discussion and the involvement of a third reviewer (JINM) who supervised the reviewing process.
Data collection process
More than one reviewer collected data from each report. First, the reports were distributed between JJ and PM for non-duplicate data extraction. Then, PM and JINM jointly conducted a final quality assurance check. Disagreements in data collection were solved through discussion between PM and JINM. We did not confirm data from study investigators or use automation tools during this process, and we did not find multiple reports of a single study.
The data items extracted were:
Report characteristics: Leading author, year of publication, complete reference, funding source, and declared conflicts of interest.
Population: Type of road traffic user (i.e., passengers, pedestrians, and drivers of motorized or non-motorized vehicles), mean age (or age range), sex (proportion female), and geographical location. We assumed the data referred to all road users if road traffic users were not specified. When information on drivers' age was unavailable, we assumed it referred to drivers aged 15 or older.
Intervention: Content of the alcohol policy evaluated, including implementation dates and places (i.e., national or sub-national level). A categorization of alcohol policies into general (i.e., measures restricting access to alcohol that may indirectly reduce road traffic deaths), specific (i.e., measures explicitly aimed at reducing DUIA), or mixed was conducted per their expected impact on DUIA.
Comparator: Whether the comparator was a no-intervention control condition, the population acted as its own control (e.g., a pre-intervention period) or an active control condition (e.g., another alcohol policy). If another alcohol policy was considered a comparator, we extracted data as in the intervention.
Outcomes: Type of road traffic outcome (i.e., crashes, injuries, and fatalities), type of road traffic crash (e.g., single-vehicle collisions, intersection collisions, or collisions involving pedestrians and cyclists), and temporal information on road traffic crashes (e.g., time of day and day of the week). Additionally, we considered the time interval between data points (e.g., daily, monthly, annually), the data collection period, the outcome data source (e.g., the national or sub-national institutions providing access to data), and the rationale provided by the study authors for not using alcohol-related crash data.
Study design: We distinguished between before-and-after studies and interrupted time series. Before-and-after studies conducted pooled analyses with a small number of measurements for the difference between the pre-and post-intervention periods, whereas interrupted time series considered at least two pre- and post-intervention measurements to account for changes over time and pre-intervention trends . We added the "controlled" prefix when an external comparison group was used.
Study results: We anticipated various effect measures reported in the included studies. Counts or rates of the selected road traffic outcomes might be analyzed as rate ratios, differences in rates, or mean differences . These effect measures were included with their available measures of uncertainty (e.g., 95% confidence intervals, standard errors, or P-values). Outcomes were treated differently and analyzed using different techniques across studies. Therefore, we did not consider the effect measures suitable for synthesis, and no data transformation was conducted . Data were extracted on the study's overall conclusion in a synthesized statement on the association between alcohol policies and road traffic outcomes (i.e., the direction of effect).
Study risk of bias assessment
We assessed the risk of bias (RoB) in the included studies using the Quality Assessment Tool for Before-After (Pre-Post) Studies With No Control Group (abbreviated QAT-BA) developed by the National Heart, Lung, and Blood Institute . The QAT-BA is a simple, generic quality and RoB assessment for before-and-after studies (including time series). The QAT-BA explores twelve domains 1) study question; 2) eligibility criteria and study population; 3) representativeness of study participants; 4) participants' enrollment; 5) sample size; 6) description and implementation of the intervention; 7) outcomes measures; 8) blinding of outcome assessors; 9) follow-up rate; 10) statistical analysis; 11) multiple outcome measures; 12) group-level interventions and individual-level outcome efforts. The domains related to blinding of outcome assessors, follow-up rate, and use of individual-level data did not apply to the RoB assessment, as all of the included studies used administrative datasets and worked (mostly) with pooled time-series cross-sectional data. We used the rule of thumb of 50 units of time as the minimum sample size required for a well-powered time series analysis in road safety studies . The RoB assessment provided a binary score for each QAT-BA's domains: "positive" for low risk or "negative" for some concern for bias. The QAT-BA does not provide a cut-off score for a general classification of studies according to their RoB. The tool's authors warn against using specific rules or adding-up items for the critical appraisal of studies . The RoB assessment was conducted by PM and Lysiane Robidoux with the supervision of JINM. We did not obtain or confirm relevant information from study investigators or use automation tools during this process.
As we found substantial variation in the populations, the interventions, the comparators, the outcomes, and the study designs, we did not conduct a meta-analysis and reported our synthesis approach following the Synthesis Without Meta-analysis (SWiM) guideline . We synthesized study characteristics and RoB assessment in included studies using a tabular format. Study characteristics were reported regardless of overlap with other included primary studies (e.g., two studies in the same place with overlapping years).
Our synthesis method was vote counting based on the direction of effect, regardless of statistical significance. We did not consider the statistical significance of effects as attention to this feature may lead to the exclusion of underpowered studies . Owing to the highly heterogeneous study characteristics (such as the lack of a consistent effect measure), we relied on the direction of effects as a standardized binary metric . Each effect estimate was categorized as positive or negative based on the observed direction of effect for road traffic outcomes. Road traffic outcomes showing a decrease were considered "positive" effect estimates (e.g., decreases in road traffic fatalities were judged to be desirable health outcomes), whereas increases in these outcomes were considered "negative" effect estimates (i.e., they represented undesirable health outcomes).
To implement vote counting based on the direction of effect, we followed the study-level step-by-step procedures detailed by Boon and Thomson :
We grouped traffic outcomes into crashes, injuries, and fatalities.
We counted positive and negative effect estimates per each type of traffic outcome.
A positive or negative direction of effect was assumed if the vast majority of effect estimates (i.e., 70% or more) of a traffic outcome within a study followed a positive or negative direction of the effect, respectively.
An unclear direction of effect was assigned in those cases where a consistent direction of effect was not found within a study (i.e., less than 70% of effect estimates were in the same direction).
A summary of the study-level findings for traffic crashes, injuries, and fatalities was presented through effect direction plots . We implemented a binomial probability test (setting the significance level at a P-value < 0.05) in Stata 16, to synthesize effect directions across studies, formally evaluating whether there was sufficient evidence of an effect (i.e., alcohol policies being associated with a positive or negative effect direction for a particular traffic outcome) . The binomial probability test did not include studies with unclear effect directions .
We conducted subgroup analyses to test the effects of potential methodological modifiers. Firstly, we stratified studies by type of alcohol policy (i.e., general or specific), type of road traffic crash, and temporal information on road traffic crashes. Then, we implemented vote counting based on the direction of effect . Finally, Fisher's exact test was used to explore whether the proportion of studies reporting positive or negative directions of the effect differed between the strata of the methodological modifiers considered.
Given the substantial variation in the attributes of included studies, we did not conduct a non-reporting bias assessment in our review nor provided a formal assessment of the certainty of our synthesis findings.
We identified a total of 2,539 records from database searching. After duplicate removal, we screened 1,465 records and retrieved 118 full-text reports. The assessment of eligibility criteria led to the final inclusion of 24 studies [37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60]. The reasons for excluding 94 full-text reports were listed, with the main reason being the presence of alcohol-related data for the outcomes. We found no reviews addressing the same research question. The PRISMA 2020 flow diagram was used to report the selection process (Fig. 1).
The characteristics of the 24 included studies are summarized in Table 1. Population characteristics (i.e., type of road traffic user, age, and sex) were often not reported [37, 38, 41,42,43,44, 46, 47, 49,50,51,52, 55,56,57,58,59,60]. Studies that reported data typically considered drivers aged 15 years or older.
The included studies evaluated alcohol policies implemented in the United States (n = 7), Australia (n = 4), Canada (n = 4), and Japan (n = 2), with a single study each conducted in Botswana, Brazil, Mexico, New Zealand, Norway, the Russian Federation, and Uruguay. We identified a total of 33 alcohol policies evaluated, which we defined as general (n = 12) [43, 44, 46, 48, 53, 54, 56,57,58,59] and specific (n = 21) [37,38,39,40,41,42, 45, 47,48,49,50,51,52,53, 55, 60].
Based on the information obtained from the studies, we categorized general alcohol policies into:
Alcohol regulation: shrinkage of the market-driven production and sales of alcohol by tightening state regulations controlling alcohol availability and consumption (e.g., banning of alcohol production and sale without a license, prohibition of alcohol sales at public places, taxes on the production, sales, or consumption of alcohol products) [54, 55, 57].
Alcohol deregulation: expansion of the market-driven production and sales of alcohol by increasing alcohol availability and lowering restrictions (e.g., the extension of drinking hours or increase in off-sale alcohol outlets) [43, 44, 46, 56, 58, 59].
Adult responsibility: laws related to adult responsibility for underage drinking, such as social host laws .
We categorized specific alcohol policies as follows:
BAC limit: introducing or modifying a BAC limit above which motor vehicle driving is prohibited. We also included laws related to zero tolerance and specific BAC limits for young people [37,38,39, 42, 45, 50, 52, 53].
Alcohol in transport: laws punishing possession and use of alcohol in transport, for instance, in the case of passengers of non-commercial vehicles .
Community awareness: programs to raise community awareness on the risk of DUIA .
Subsidized ridesharing: programs promoting rideshare (i.e., an alternative transportation means) to reduce DUIA .
Five studies complementarily evaluated non-alcohol-related road safety measures. In the United States, French et al. , Mader and Zick , and Notrica et al.  assessed the specific effects of several alcohol policies (e.g., lowering the minimum legal drinking age, the reduction of the BAC limit, and introducing penalties for DUIA, among others) and road safety policies, such as graduated driver license programs for adolescent drivers, maximum legal speed limits, seat belt laws, and universal helmet laws. In Australia, Jiang et al. assessed the joint effects of the introduction of compulsory seat belt legislation and random sobriety checks ; and Nghiem et al., among other specific alcohol policies, assessed the Safe4Life program: a road safety strategy that gave priority to raising awareness on DUIA and drugs .
Most of the comparators in the included studies were based on data for the same jurisdiction (i.e., not external) before the alcohol policy went into effect (i.e., pre-intervention) (n = 19) [37,38,39,40,41, 43,44,45, 48,49,50,51,52,53,54,55, 57, 59, 60]. There were five instances in which researchers relied on an external comparator [42, 46, 47, 56, 58]. For example, Davenport et al. used data from Chile as a comparator to assess a reduction in the BAC limit in Uruguay . According to Davenport et al., Chile had a continuous BAC limit during the study period . Similarly, in Canada, Sen compared varying degrees of alcohol deregulation in the provinces of Alberta, British Columbia, and Quebec to largely regulated alcohol markets in the provinces of Manitoba, Ontario, and Saskatchewan .
Regarding the type of intervention in the comparison condition, most studies did not specify the type of alcohol policy during the pre-intervention period or in the external jurisdiction (n = 8) [45, 47, 49, 52,53,54, 58, 60]. Studies that reported an alcohol policy for the comparison condition referred to a BAC limit (n = 7) [37, 39, 41, 42, 47, 50, 51] or the presence of stricter alcohol regulation (n = 5) [43, 44, 56, 57, 59]. The existence of lower penalties for DUIA (n = 2) [40, 55], alcohol deregulation , and a higher minimum-legal drinking age were also included as comparators .
As for the type of road traffic outcome, studies considered fatal crashes (n = 19) [37,38,39, 41,42,43,44,45, 48,49,50,51,52,53,54,55,56,57,58], injury crashes (n = 9) [37, 38, 42, 44, 45, 47, 56, 57, 59], or crashes (n = 4) [40, 46, 55, 60]. The type, time of day, and day of the week of crashes were not specified for all outcomes. We assumed these outcomes referred to all types of crashes, regardless of the time or day of the week [37, 41,42,43,44,45,46,47,48,49,50, 52,53,54,55,56,57,58]. When this information was provided, studies reported analyzing night-time crashes (n = 12) [38,39,40, 42, 44, 46, 47, 51, 55, 57, 59, 60], single-vehicle crashes (n = 6) [38,39,40, 44, 46, 55], weekend crashes (n = 2) [38, 42], or day-time crashes (n = 2) [44, 47]. On the time interval between outcome data points, most of the studies relied on annual (n = 12) [38, 43,44,45, 47,48,49,50, 52, 53, 56, 58] or monthly data (n = 10) [37, 39,40,41,42, 46, 54, 55, 59, 60], with a few using quarterly (i.e., 3-month interval) data (n = 2) [51, 57]. The median number of time units was 49.5, with a minimum of 9 annual data points in Humphreys et al. , to a maximum of 228 monthly data points in Colchero et al. .
When exploring the reasons for not using DUIA data, most studies did not report why DUIA data were missing (n = 12) [45, 46, 48,49,50,51,52,53,54, 56, 58, 59]. The remainder claimed that data on DUIA were unavailable, difficult to obtain, or partially available (n = 8) [37, 38, 41, 42, 44, 55, 57, 60] or stated that data on DUIA were unreliable (n = 4) [39, 40, 43, 47].
In terms of study design, we identified three studies with an uncontrolled before-and-after design [38, 43, 44], 15 studies with an interrupted time series design [37, 39,40,41, 45, 48,49,50,51,52,53,54,55, 57, 59], and six with a controlled interrupted time-series design [42, 46, 47, 56, 58, 60]. The most used time-series method was the autoregressive integrated moving average (ARIMA) model [37, 46, 48, 49, 51, 54, 55, 57,58,59,60]. Further details on the characteristics of included studies are provided in Additional file 1: Appendix 2.
Risk of bias in studies
A summary of the RoB assessment is provided in Fig. 2. Almost half of the studies (n = 11) did not have the minimum sample size required for a well-powered time series analysis (i.e., 50 units of time). The median sample size for this subset of studies was 17 units of time [38, 43,44,45, 47, 50, 51, 53, 56, 58, 60]. Most of these studies used annual data [38, 43,44,45, 47, 50, 53, 56, 58]. Six studies had a low RoB when assessing the description and implementation of the intervention [37, 38, 42, 47, 54, 60]. Some studies assumed a homogeneous roll-out of alcohol policies over time, but the implementation process was poorly described to justify such an assumption adequately [41, 43, 44, 46]. There were five studies in which the insufficient detail of the interventions was coupled with the risk of co-intervention, being these other alcohol policies or road safety interventions [45, 50, 52, 53, 57]. A subgroup of these studies did not report the implementation dates of alcohol policies, the extension of the post-intervention period or the degree of co-interventions remained unclear [45, 50, 53]. Eight studies acknowledged that the evaluation of alcohol policies was not consistently applied over time or to the whole study population [39, 40, 48, 49, 51, 55, 56, 58]. Of these studies, Sebego et al.  assessed alcohol policies that were introduced along with road safety co-interventions, and Jiang et al.  could only report the combined effects of an alcohol policy and a road safety co-intervention. Finally, Vingilis et al.  acknowledged concerns over attributing study results to road safety co-interventions.
Results of studies and synthesis
We identified a total of 188 road traffic outcomes grouped into road traffic crashes (n = 11), injuries (n = 29), and fatalities (n = 148). The effect direction plot summarizes the direction of the effect estimates for road traffic outcomes at the study level (Fig. 3).
For road traffic crashes, all five studies reported a positive effect direction (i.e., a decrease in crashes) (P-value for the binomial probability test = 0.125). Regarding road traffic injuries, 7 of 9 studies had a positive effect direction, one with a negative effect direction, and one with an unclear effect direction (P-value for the binomial probability test = 0.070). Concerning road traffic fatalities, 15 studies had a positive effect direction, two with a negative effect direction, and two with an unclear effect direction (P-value for the binomial probability test = 0.002). According to the binomial probability tests, there was sufficient evidence of an association between alcohol policies and positive effect directions for traffic fatalities.
The effect direction plots summarizing the direction of road traffic outcomes at the study level, grouped by potential methodological modifiers, are displayed in Additional file 1: Appendix 3, Figs. 1a through 4a. Data and the P-value for the binomial probability tests synthesizing effect directions across studies per potential methodological modifiers are shown in Additional file 1: Appendix 3, Tables 1a through 4a. We did not have enough information on the type of road traffic crash to consider this variable as a possible cause of heterogeneity among study results (i.e., only six studies reported such information, all of which used single-vehicle crashes).
Subgroup analyses initially showed evidence for an association between potential methodological modifiers and positive effect directions for road traffic fatalities. In the subgroup of 12 studies that assessed road traffic fatalities following the implementation of specific alcohol policies, ten studies reported a positive effect direction, and two had a negative effect direction (P-value for the binomial probability test = 0.039) (Table 2a). Thirteen of seventeen studies assessing road traffic fatalities not occurring during the night-time (i.e., any other crashes) had a positive effect direction, one had a negative effect direction, and three had unclear effect directions (P-value for the binomial probability test = 0.002) (Table 4a). There was no evidence for an association between traffic fatalities following the implementation of general alcohol policies or those occurring during the night-time and positive effect directions (Tables 1a and 3a) Fisher's exact test results suggest no statistically significant differences in the proportion of studies reporting positive effect directions between the strata of the potential methodological modifiers (i.e., general vs. specific policies and night-time crashes vs. any other crashes). These results are detailed in Tables 5a and 6a of Additional file 1: Appendix 3.
General interpretation of the results
We included twenty-four eligible studies. Regarding objective 1, when DUIA data were missing, most studies assessed road traffic fatalities following the introduction of specific alcohol policies (e.g., the modification of a BAC limit). These studies generally relied on night-time crashes as a surrogate for an objective measure of DUIA. Notably, the rationale for the surrogate measure was frequently not reported. These studies were mainly conducted in high-income countries, particularly USA, and employed uncontrolled interrupted time series designs. Regarding objective 2, we found evidence for an association between alcohol policies and decreased traffic fatalities, with no associations found in the case of road traffic crashes or injuries. Subgroup analyses for studies having a positive direction of effect on road traffic fatalities by type of alcohol policy or temporal information on road traffic crashes found no statistically significant differences between groups.
Limitations of the evidence included in the review
The evidence included in the review had critical limitations on sample size and the description and implementation of the interventions. Half of the included studies did not meet the minimum requirement of 50 units of time for well-powered time series analyses. Only two acknowledged problems related to statistical power and the length of the study period [50, 58]. Generally, studies using annual data had such limitations. While we recognize that an adequate assessment of power should consider a set of factors (such as sample size per time period, anticipated effect size, and location of the intervention in the time series, among others) [61, 62], none of the reviewed studies discussed these elements.
Regarding the description and implementation of the intervention, two key questions were not satisfactorily answered in the studies: the specific components of the policies and their consistent implementation over time and space. We found insufficient information on the components of general alcohol policies. For instance, it was unclear if alcohol deregulation jointly considered a decrease in alcohol-specific taxes and an increase in off-sale alcohol outlets. The heterogeneous or interrupted implementations further complicated policy evaluations. The capacity needed for implementation and enforcement was often overlooked, which is needed to judge implementation consistency. Other studies have also highlighted the challenges in alcohol policy evaluation due to their complex implementation [13, 63]. Concomitant interventions that could provide alternative explanations for observed changes attributed to alcohol policies were identified. These complexities affect model construction and estimation in time series analyses. Caution is warranted in interpreting syntheses results, as three-quarters of the reviewed studies were assessed with concerns for bias in this critical domain.
Limitations of the review processes
To ensure transparency and quality in our research process, we registered the review protocol in PROSPERO, and adhered to the recommendations of the Cochrane Collaboration, the PRISMA statement, and the SWiM guidelines for systematic reviews without meta-analyses. However, our review process is subject to several potential biases. Our scope was limited to studies published in peer-reviewed journals in English or French from 2000 onwards, potentially excluding relevant literature from non-conventional sources, other languages, or before 2000, resulting in selection bias. As the searches for our systematic review may have been outdated, we conducted an updated MEDLINE search of studies published between November 2021 and April 2023 but found no eligible studies (Additional file 1: Appendix 1).
Additionally, changes to the protocol, detailed in Additional file 1: Appendix 1, may have introduced bias. We broadened the search strategy to compensate for lowering the information sources. We used the QAT-BA  instead of the Risk of Bias In Non-randomised Studies – of Interventions (ROBINS-I) as the tool for RoB . The ROBINS-I was not used due to its complexity, high assessor burden, assessor dependency, and requirements for intensive training and supervision that exceeded the resources allocated to the study [65,66,67].
The results of our scoping review should be considered exploratory and provisional, given the limitations of our synthesis method and the small sample of included studies. Vote counting based on the direction of effects lacks information on the magnitude of effects, does not account for differences in the relative sizes of the studies, and does not provide a formal assessment of the certainty of our findings . The binomial probability test for effect direction synthesis should be carefully interpreted due to methodological concerns (e.g., issues of statistical power, the well-known limitations of P-values and significance testing, and restrictive underlying assumptions) . The binomial probability test results aid in interpreting the overall pattern of effect direction . The small sample of included studies limited our ability to explore potential effect modifiers in detail.
Implications for practice, policy, and future research
The availability and reliability of DUIA data (e.g., BAC) pose challenges for road safety professionals, policymakers, and researchers [12, 20, 22]. Surrogate measures like night-time crashes might seem reasonable alternatives when DUIA data are missing. However, caution is needed when interpreting road traffic outcomes associated with alcohol policies in such cases. In our scoping review, a minority of non-randomized studies provided sufficient information for outcomes assessment, utilizing uncontrolled designs with monthly counts of night-time crashes. DUIA data were missing in these studies because of their unavailability or unreliability. Notably, half of the studies did not report the type, time of day, day of the week crashes occurred, or the reasons for using surrogate measures. This lack of transparency hinders the proper evaluation of alcohol policies. Greater efforts should be made to report outcomes assessed in detail to inform evidence-based interventions.
Our exploratory synthesis found an association between alcohol policies and reduced road traffic fatalities when DUIA data are missing. However, as highlighted in the previous paragraph, this finding must be interpreted cautiously. Rather than providing certainty, the quality of the reviewed evidence makes us reflect on the characteristics that a study with missing DUIA data should have. We believe such a study should have access to more granular data (monthly or quarterly) and pay attention to the number of events per unit of time to have sufficient statistical power to detect differences. The intervention and its level of implementation should be well-defined, and this information should be incorporated into data modeling. The potential for co-intervention and confounders should be accounted . Such a study should use a controlled design, comparing outcomes with those of other jurisdictions with similar characteristics where the intervention has not occurred.
Regarding future research directions, we found that studies from LMICs are urgently needed to provide more direct evidence on using surrogate measures for DUIA data. We know from Brazil, Botswana, Mexico, and Russia studies that access to more granular data is possible in these locations. Complementarily, we also found that the evidence coming from developed countries is particularly weighted towards the USA. On the other hand, studying single-vehicle or weekend crashes requires further investigation as surrogate measures. Finally, results from other reviews where DUIA data were available should be formally compared with the results of our review to compare the proportions of outcomes with a positive direction of effect. Such a comparison could provide an estimate to confirm or recalibrate the associations found in studies where DUIA data were missing.
Blood alcohol concentration
Driving under the influence of alcohol
Global Burden of Diseases, Injuries, and Risk Factors Study
José Ignacio Nazif-Munoz
Low- and middle-income countries
Population, Intervention, Comparator, Outcomes, and Study Design
Preferred Reporting Items for Systematic Reviews and Meta-Analyses
Prospective Register of Systematic Reviews
Quality Assessment Tool for Before-After (Pre-Post) Studies With No Control RoB: Group Risk of Bias
Synthesis Without Meta-analysis
World Health Organization
Inada H, Li Q, Bachani A, Hyder AA. Forecasting global road traffic injury mortality for 2030. Inj Prev J Int Soc Child Adolesc Inj Prev. 2020;26:339–43.
Chen S, Kuhn M, Prettner K, Bloom DE. The global macroeconomic burden of road injuries: estimates and projections for 166 countries. Lancet Planet Health. 2019;3:e390–8.
Vos T, Lim SS, Abbafati C, Abbas KM, Abbasi M, Abbasifard M, et al. Global burden of 369 diseases and injuries in 204 countries and territories, 1990–2019: a systematic analysis for the Global Burden of Disease Study 2019. The Lancet. 2020;396:1204–22.
Irwin C, Iudakhina E, Desbrow B, McCartney D. Effects of acute alcohol consumption on measures of simulated driving: A systematic review and meta-analysis. Accid Anal Prev. 2017;102:248–66.
Taylor B, Irving HM, Kanteres F, Room R, Borges G, Cherpitel C, et al. The more you drink, the harder you fall: a systematic review and meta-analysis of how acute alcohol consumption and injury or collision risk increase together. Drug Alcohol Depend. 2010;110:108–16.
Taylor B, Rehm J. The relationship between alcohol consumption and fatal motor vehicle injury: high risk at low alcohol levels. Alcohol Clin Exp Res. 2012;36:1827–34.
Borges G, García-Pacheco JÁ, Familiar-Lopez I. Global estimates of the attributable risk of alcohol consumption on road injuries. Alcohol Clin Exp Res. 2021;45:2080–9.
World Health Organization. Global status report on alcohol and health 2018. Health Organization; 2019.
Peden MM(ed.). World report on road traffic injury prevention. Geneva: World Health Organization; 2004.
World Health Organization. The SAFER technical package: five areas of intervention at national and subnational levels. Geneva: World Health Organization; 2019.
Fell JC. Approaches for reducing alcohol-impaired driving: Evidence-based legislation, law enforcement strategies, sanctions, and alcohol-control policies. Forensic Sci Rev. 2019;31:161–84.
Shults RA, Elder RW, Sleet DA, Nichols JL, Alao MO, Carande-Kulis VG, et al. Reviews of evidence regarding interventions to reduce alcohol-impaired driving. Am J Prev Med. 2001;21(4 Suppl):66–88.
Siegfried N, Parry C. Do alcohol control policies work? An umbrella review and quality assessment of systematic reviews of alcohol control interventions (2006–2017). PLoS ONE. 2019;14:e0214865.
Fisa R, Musukuma M, Sampa M, Musonda P, Young T. Effects of interventions for preventing road traffic crashes: an overview of systematic reviews. BMC Public Health. 2022;22:513.
Greenfield TK, Cook WK, Karriker-Jaffe KJ, Li L, Room R. Are Countries’ Drink-Driving Policies Associated with Harms Involving Another Driver’s Impairment? Alcohol Clin Exp Res. 2021;45:429–35.
Nazif-Munoz JI, Anakök GA, Joseph J, Uprajhiya SK, Ouimet MC. A new alcohol-related traffic law, a further reduction in traffic fatalities? Analyzing the case of Turkey. J Safety Res. 2022. https://doi.org/10.1016/j.jsr.2022.08.015.
Wagenaar AC, Salois MJ, Komro KA. Effects of beverage alcohol price and tax levels on drinking: a meta-analysis of 1003 estimates from 112 studies. Addict Abingdon Engl. 2009;104:179–90.
Wagenaar AC, Tobler AL, Komro KA. Effects of Alcohol Tax and Price Policies on Morbidity and Mortality: A Systematic Review. Am J Public Health. 2010;100:2270–8.
Dimitriou L, Antoniou C. Road safety data considerations. Accid Anal Prev. 2019;130:1–2.
Imprialou M, Quddus M. Crash data quality for road safety research: Current state and future directions. Accid Anal Prev. 2019;130:84–90.
Huang CY, Chou SE, Su WT, Liu HT, Hsieh TM, Hsu SY, et al. Effect of Lowering the Blood Alcohol Concentration Limit to 0.03 Among hospitalized trauma patients in southern taiwan: a cross-sectional analysis. Risk Manag Healthc Policy. 2020;13:571–81.
Ahmed A, Sadullah AFM, Yahya AS. Errors in accident data, its types, causes and methods of rectification-analysis of the literature. Accid Anal Prev. 2019;130:3–21.
Stewart K, Silcock D, Wegman F. Reducing Drink Driving in Low- and Middle-Income Countries: Challenges and Opportunities. Traffic Inj Prev. 2012;13:93–5.
James SL, Lucchesi LR, Bisignano C, Castle CD, Dingels ZV, Fox JT, et al. Morbidity and mortality from road injuries: results from the Global Burden of Disease Study 2017. Inj Prev J Int Soc Child Adolesc Inj Prev. 2020;26(Supp 1):i46-56.
He J-Y, Xiao W-X, Schwebel DC, Zhu M-T, Ning P-S, Li L, et al. Road traffic injury mortality and morbidity by country development status, 2011–2017. Chin J Traumatol Zhonghua Chuang Shang Za Zhi. 2021;24:88–93.
Page MJ, Moher D, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ. 2021;372: n160.
Nazif-Munoz JI, Joseph J, Martínez P. Best practices to navigating absence of alcohol-related outcomes in traffic studies. PROSPERO. 2021. https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021266744. Accessed 8 Mar 2022.
Reeves BC, Higgins JPT, Ramsay C, Shea B, Tugwell P, Wells GA. An introduction to methodological issues when including non-randomised studies in systematic reviews on the effects of interventions. Res Synth Methods. 2013;4:1–11.
Reeves BC, Wells GA, Waddington H. Quasi-experimental study designs series-paper 5: a checklist for classifying studies evaluating the effects on health interventions-a taxonomy without labels. J Clin Epidemiol. 2017;89:30–42.
Kontopantelis E, Doran T, Springate DA, Buchan I, Reeves D. Regression based quasi-experimental approach when randomisation is not an option: interrupted time series analysis. BMJ. 2015;350: h2750.
Higgins J, Li T, Deeks J. Chapter 6: Choosing effect measures and computing estimates of effect. In: Higgins J, Thomas J, Chandler J, Cumpston M, Li T, Page M, et al., editors. Cochrane Handbook for Systematic Reviews of Interventions. Version 6.3. Cochrane; 2022.
McKenzie JE, Brennan SE. Chapter 12: Synthesizing and presenting findings using other methods. In: McKenzie JE, Brennan SE, Chandler J, Cumpston M, Li T, Page M, et al., editors. Cochrane Handbook for Systematic Reviews of Interventions. Version 6.3. Cochrane; 2022.
Jensen M, Ryan D, Donato K, Apovian C, Ard J, Comuzzie A, et al. Guidelines (2013) for managing overweight and obesity in adults. Obesity. 2014;22:i–xvi.
Meidinger EE. Applied time series analysis for the social sciences. Sage Publications; 1980.
Campbell M, McKenzie JE, Sowden A, Katikireddi SV, Brennan SE, Ellis S, et al. Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline. BMJ. 2020;368:l6890.
Boon MH, Thomson H. The effect direction plot revisited: Application of the 2019 Cochrane Handbook guidance on alternative synthesis methods. Res Synth Methods. 2021;12:29–33.
Andreuccetti G, Carvalho HB, Cherpitel CJ, Ye Y, Ponce JC, Kahn T, et al. Reducing the legal blood alcohol concentration limit for driving in developing countries: a time for change? Results and implications derived from a time-series analysis (2001–10) conducted in Brazil. Addict Abingdon Engl. 2011;106:2124–31.
Assum T. Reduction of the blood alcohol concentration limit in Norway–effects on knowledge, behavior and accidents. Accid Anal Prev. 2010;42:1523–30.
Bernat DH, Dunsmuir WTM, Wagenaar AC. Effects of lowering the legal BAC to 0.08 on single-vehicle-nighttime fatal traffic crashes in 19 jurisdictions. Accid Anal Prev. 2004;36:1089–97.
Brubacher J, Chan H, Erdelyi S, Asbridge M, Schuurman N. Factors Predicting Local Effectiveness of Impaired Driving Laws, British Columbia. Canada J Stud Alcohol Drugs. 2017;78:899–909.
Colchero MA, Guerrero-López CM, Quiroz-Reyes JA, Bautista-Arredondo S. Did, “Conduce Sin Alcohol” a Program that Monitors Breath Alcohol Concentration Limits for Driving in Mexico City Have an Effect on Traffic-Related Deaths? Prev Sci Off J Soc Prev Res. 2020;21:979–84.
Davenport S, Robbins M, Cerdá M, Rivera-Aguirre A, Kilmer B. Assessment of the impact of implementation of a zero blood alcohol concentration law in Uruguay on moderate/severe injury and fatal crashes: a quasi-experimental study. Addict Abingdon Engl. 2021;116:1054–62.
Desapriya E, Fujiwara T, Scime G, Sasges D, Pike I, Shimizu S. Are 1994 alcohol production and the sales deregulation policy in Japan associated with increased road traffic fatalities among adult and teenage males and females in Japan? Nihon Arukoru Yakubutsu Igakkai Zasshi. 2009;44:569–78.
Desapriya E, Fujiwara T, Dutt N, Arason N, Pike I. Impact of the 1994 alcohol production and sales deregulation policy on traffic crashes and fatalities in Japan. Asia Pac J Public Health. 2012;24:776–85.
French MT, Gumus G, Homer JF. Public policies and motorcycle safety. J Health Econ. 2009;28:831–8.
Han D, Shipp EM, Gorman DM. Evaluating the effects of a large increase in off-sale alcohol outlets on motor vehicle crashes: a time-series analysis. Int J Inj Contr Saf Promot. 2015;22:320–7.
Humphreys DK, Degli Esposti M, Williams FM, Kondo MC, Morrison C. Assessing the impact of a local community subsidised rideshare programme on road traffic injuries: an evaluation of the Evesham Saving Lives programme. Inj Prev. 2021;27:32–237.
Jiang H, Livingston M, Manton E. The effects of random breath testing and lowering the minimum legal drinking age on traffic fatalities in Australian states. Inj Prev J Int Soc Child Adolesc Inj Prev. 2015;21:77–83.
Jiang H, Livingston M, Room R. Alcohol consumption and fatal injuries in Australia before and after major traffic safety initiatives: a time series analysis. Alcohol Clin Exp Res. 2015;39:175–83.
Mader EM, Zick CD. Active transportation: Do current traffic safety policies protect non-motorists? Accid Anal Prev. 2014;67:7–13.
Miller T, Blewden M, Zhang J. Cost savings from a sustained compulsory breath testing and media campaign in New Zealand. Accid Anal Prev. 2004;36:783–94.
Nghiem S, Commandeur JJF, Connelly LB. Determinants of road traffic safety: New evidence from Australia using state-space analysis. Accid Anal Prev. 2016;94:65–72.
Notrica DM, Sayrs LW, Krishna N, Rowe D, Jaroszewski DE, McMahon LE. The impact of state laws on motor vehicle fatality rates, 1999–2015. J Trauma Acute Care Surg. 2020;88:760–9.
Pridemore WA, Chamlin MB, Kaylen MT, Andreev E. The impact of a national alcohol policy on deaths due to transport accidents in Russia. Addict Abingdon Engl. 2013;108:2112–8.
Sebego M, Naumann RB, Rudd RA, Voetsch K, Dellinger AM, Ndlovu C. The impact of alcohol and road traffic policies on crash rates in Botswana, 2004–2011: a time-series analysis. Accid Anal Prev. 2014;70:33–9.
Sen A. Is Retail Alcohol Deregulation Correlated with More Crime and Traffic Injuries? Evidence from Canadian Provinces. Can J Criminol Crim Justice. 2016;58:251–86.
Stockwell T, Chikritzhs T, Hendrie D, Fordham R, Ying F, Phillips M, et al. The public health and safety benefits of the Northern Territory’s Living with Alcohol programme. Drug Alcohol Rev. 2001;20:167–80.
Trolldal B. An investigation of the effect of privatization of retail sales of alcohol on consumption and traffic accidents in Alberta. Canada Addict Abingdon Engl. 2005;100:662–71.
Vingilis E, McLeod AI, Stoduto G, Seeley J, Mann RE. Impact of extended drinking hours in Ontario on motor-vehicle collision and non-motor-vehicle collision injuries. J Stud Alcohol Drugs. 2007;68:905–11.
Voas RB. A new look at NHTSA’s evaluation of the 1984 Charlottesville Sobriety Checkpoint Program: implications for current checkpoint issues. Traffic Inj Prev. 2008;9:22–30.
Hawley S, Ali MS, Berencsi K, Judge A, Prieto-Alhambra D. Sample size and power considerations for ordinary least squares interrupted time series analysis: a simulation study. Clin Epidemiol. 2019;11:197–205.
Zhang F, Wagner AK, Ross-Degnan D. Simulation-based power calculation for designing interrupted time series analyses of health policy interventions. J Clin Epidemiol. 2011;64:1252–61.
Siegfried N, Pienaar DC, Ataguba JE, Volmink J, Kredo T, Jere M, et al. Restricting or banning alcohol advertising to reduce alcohol consumption in adults and adolescents. Cochrane Database Syst Rev. 2014;2014:CD010704.
Sterne JA, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355:i4919.
Igelström E, Campbell M, Craig P, Katikireddi SV. Cochrane’s risk of bias tool for non-randomized studies (ROBINS-I) is frequently misapplied: A methodological systematic review. J Clin Epidemiol. 2021;140:22–32.
Jeyaraman MM, Robson RC, Copstein L, Al-Yousif N, Pollock M, Xia J, et al. Customized guidance/training improved the psychometric properties of methodologically rigorous risk of bias instruments for non-randomized studies. J Clin Epidemiol. 2021;136:157–67.
Jeyaraman MM, Rabbani R, Copstein L, Robson RC, Al-Yousif N, Pollock M, et al. Methodologically rigorous risk of bias tools for nonrandomized studies had low reliability and high evaluator burden. J Clin Epidemiol. 2020;128:140–7.
Matthay EC, Gottlieb LM, Rehkopf D, Tan ML, Vlahov D, Glymour MM. What to Do When Everything Happens at Once: Analytic Approaches to Estimate the Health Effects of Co-Occurring Social Policies. Epidemiol Rev. 2022;43:33–47.
Martínez P. Replication Data for: Navigating the absence of data on drinking under the influence of alcohol in road traffic studies: a systematic review without meta-analysis of quasi-experimental studies of alcohol policies. Harvard Dataverse, V1. 2022. https://doi.org/10.7910/DVN/NNATZM.
We would like to thank Lysiane Robidoux for her assistance in conducting the risk of bias assessment.
This work was supported by the Social Sciences and Humanities Research Council [grant number 430–2020-00877] and the Fonds de recherche du Québec-Santé [grant number 280205], both awarded to José Ignacio Nazif-Munoz. Complementarily, Pablo Martínez received funding by the Centre de recherche Charles-Le Moyne [grant Concours 2020 de bourses] and the Institut universitaire sur les dépendances et l’équipe Recherche et intervention sur les substances psychoactives – Québec [grant Concours de la bourse postdoctorale conjointe].
Ethics approval and consent to participate
Consent for publication
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Martínez, P., Joseph, J. & Nazif-Munoz, J.I. The absence of data on driving under the influence of alcohol in road traffic studies: a scoping review of non-randomized studies with vote counting based on the direction of effects of alcohol policies. Subst Abuse Treat Prev Policy 18, 46 (2023). https://doi.org/10.1186/s13011-023-00553-y
- Driving under the influence
- Alcohol drinking, Legislation & jurisprudence
- Traffic accidents
- Systematic review
- Non-randomized controlled trials