Open Access

Development and process evaluation of a web-based responsible beverage service training program

  • Brian G Danaher1Email author,
  • Jack Dresser2,
  • Tracy Shaw2,
  • Herbert H Severson1, 2,
  • Milagra S Tyler1,
  • Elisabeth D Maxwell2 and
  • Steve M Christiansen3
Substance Abuse Treatment, Prevention, and Policy20127:41

https://doi.org/10.1186/1747-597X-7-41

Received: 14 August 2012

Accepted: 19 September 2012

Published: 22 September 2012

Abstract

Background

Responsible beverage service (RBS) training designed to improve the appropriate service of alcohol in commercial establishments is typically delivered in workshops. Recently, Web-based RBS training programs have emerged. This report describes the formative development and subsequent design of an innovative Web-delivered RBS program, and evaluation of the impact of the program on servers’ knowledge, attitudes, and self-efficacy.

Methods

Formative procedures using focus groups and usability testing were used to develop a Web-based RBS training program. Professional alcohol servers (N = 112) who worked as servers and/or mangers in alcohol service settings were recruited to participate. A pre-post assessment design was used to assess changes associated with using the program.

Results

Participants who used the program showed significant improvements in their RBS knowledge, attitudes, and self-efficacy.

Conclusions

Although the current study did not directly observe and determine impact of the intervention on server behaviors, it demonstrated that the development process incorporating input from a multidisciplinary team in conjunction with feedback from end-users resulted in creation of a Web-based RBS program that was well-received by servers and that changed relevant knowledge, attitudes, and self-efficacy. The results also help to establish a needed evidence base in support of the use of online RBS training, which has been afforded little research attention.

Keywords

InternetWeb program developmentResponsible beverage serviceAlcoholTraining

Background

Drunk driving crashes are responsible for nearly 11,000 fatalities and one-third of all car accident fatalities in the United States each year[1]. Approximately half of automobile drivers found to be legally impaired had consumed alcohol at licensed alcohol outlets[2]. Moreover, Naimi and colleagues[3] found that over half of binge drinkers drinking in bars, clubs and restaurants participated in drunk driving and that they consumed an average of 8.1 drinks and over 25% of them consumed more than 10 drinks. The extent of this important public health problem might therefore be reduced were licensed alcohol servers to more effectively manage and curtail the sale of alcohol to underage patrons (e.g., via ID check) and those patrons who are intoxicated or alcohol-impaired. Improved alcohol server behaviors can be addressed by a combination of community campaigns, legislation, and enforcement aimed at the consumer, and improved responsible beverage service (RBS) training of servers in commercial alcohol establishments. RBS describes the set of behavioral strategies that can be used by beverage servers to support responsible and moderate alcohol use as a way to reduce the chances that their patrons become intoxicated. RBS trainings can combine serving practices along with changes in policies for both servers and their managers[47].

As of 2011, 17 states in the U.S. had enacted mandatory RBS laws that required training and certification of RBS. A total of 15 of these states – including Oregon – mandate RBS training for managers and 13 states require RBS training for servers[8]. RBS certification is typically required for new servers and periodic (often annual) recertification is required. Another 19 states have voluntary laws that promote RBS training through incentives such as penalty abatement. RBS training has typically been delivered by in-person trainings or workshops by state-approved training services. An increasing number of states have supported the use of Web-based RBS training as an alternative to live trainings. As a result, Web-based RBS training programs have become more widely available.

This report describes the formative development, design, and preliminary test of a Web-based RBS training program designed to encourage responsible alcoholic beverage service by servers and managers of commercial alcohol service establishments.

Methods

Formative development process and program design

Interdisciplinary team

Initial project development meetings were convened for a multidisciplinary team composed of individuals from Deschutes Research, Inc. (a company focused on the dissemination of evidence-based programs), InterVision Media (a Oregon-based technology media and Web intervention development programs), and Oregon Research Institute (a non-profit independent behavioral sciences research center) to generate ideas regarding the design, development, and evaluation of a Web-based RBS training program. Team members had experience and expertise in the development and evaluation of alcohol treatment programs, health behavior change programs, Web-based behavioral interventions, and the programming of complex interactive websites. In addition, the team had been previously involved in a Phase I SBIR research program that developed and tested a portion of the full RBS Web-based program that is described in this report. The team followed an incremental and iterative formative development process[9] spanning the steps between initial design and eventual program testing (see Figure 1). This process allowed for multiple opportunities for feedback from representative RBS trained servers.
Figure 1

Iterative and Incremental Development Process.

Program specifications and aesthetic design

The first step in the process involved development of written program specifications that included the design of each program module, its functionality as a component within the website, the information architecture and instructional design that would best allow the content to engage the user and facilitate the learning and mastery of the material[10]. Program content was drawn from our experience with alcohol treatment programs as well as from the content required for RBS training programs in various states in the U.S., especially Oregon[11]. For example, the team determined how best to break the program content into chunks that worked best for Web delivery, ways that encouraged interaction and provided user with opportunities for self-evaluation.

Focus groups

Focus group procedures were informed by the team’s experience and by published guidelines[12]. These focus groups helped to identify the factors that would encourage participants to use the Web-based RBS program to achieve their work goals. Focus groups also provided early feedback about the type of content that needed to be included and how it would be best delivered.

Focus group sessions for the current project involved nine experienced alcohol servers, bartenders, and managers. Four of these participants were female and most were Caucasian, with one participant indicating more than one race. During the 1.5-hour sessions, focus group participants received a guided tour of the features of a preliminary or alpha version of the Web-RBS program. Participants were encouraged to discuss their feedback with the group regarding the overall "look and feel" of the program, its navigation, graphics, interactive features, and the amount, arrangement, and type of information presented. Participants were also asked about any information they felt was missing in the program and how the website compared to what they had experienced in the current workshop-based RBS training mandated by the Oregon Liquor Control Commission (OLCC). Sessions were audio taped and transcribed, which permitted us to identify themes and highlight possible changes to the program.

Usability testing

For optimal adoption and accurate use, Web-based tools like the RBS program should embody established usability standards[13, 14]. A relatively small number of usability testers can provide extremely valuable data that inform revisions to a program[15, 16]. The typical procedure calls for each user to meet individually with a research staff member who acts as a facilitator. During the session the usability tester is asked to interact with functional portions of the program using a hands-on approach as he/she accomplishes certain assigned tasks while receiving minimal direction from the facilitator. During this time each tester is asked to describe their thoughts using a think-aloud technique derived from cognitive science[17] that has proven effective in the study of human-computer interactions[18, 19]. As noted by Hughes[20], the think-aloud technique provides “…direct, real-time observations [by] the user rather than self-reports such as surveys” (p.493). Think-aloud methods assess cognition concurrently with its occurrence, and thus may be better at describing the thoughts and attitudes of users rather than asking them for their retrospective self-reports[21]. At the conclusion of the session, testers were asked for their overall feedback including whether they would recommend the program to other servers, how interesting the program was, how much of the program information was new to them, whether important topics were omitted, how easy the program was to use, and how the program compared to in-person RBS training. Each usability test session was recorded and transcribed for review and analysis of themes and relevant statements.

Two rounds of usability tests were conducted. The first round involved seven alcohol servers and managers (six female and 1 male) of whom six described themselves as Caucasian, one as African American, and one as Hispanic/Latino (multiple categories were allowed). The second usability test consisted of 3 female alcohol servers and managers who described themselves as Caucasian and non-Hispanic/Latino. Usability testers were observed as they completed a series of tasks on the website including user registration, reviewing program information, completing section quizzes, and completing the online assessments.

RBS program components

The formative research steps described in this report directly informed the design of the resulting RBS Web-based training program for alcohol servers, managers and licensees. Other sources that shaped program design included the field experience of team members, feedback received from expert consultants (including RBS training directors in several states), and the requirements of the OLCC[11]. As depicted in Table 1, the Web-RBS intervention provided six modules for servers and another module for managers (see Table 1 and Figure 2).
Table 1

Components of Web-RBS program

Introduction to RBS

Effects of Alcohol

Who Not to Serve

Problem Prevention

Skills and Resources

Knowledge Test

For Managers

Alcohol safety and the customer

Alcohol content of different drinks

Carding procedures

Oregon DUII laws and liability for intoxicated driving

Carding (video)

Alcohol metabolism and development

State Systems for Regulating Alcohol

Alcohol safety and business

What affects blood alcohol content

Refusing service to underage customers

Preventing alcohol impairment

Refusing service to intoxicated customers (video)

Alcohol and car crashes and fatalities

RBS research

Alcohol safety and community

Alcoholism and problem drinking

Risks and legal issues of visible intoxication

Interactions between alcohol and other drugs

Preventing DUII and aggressive behavior (video)

Oregon alcohol laws and liability

House policies

Alcohol and safety and society

Alcohol and risk to minors

Recognizing visible intoxication

Preventing intoxicated driving

Preventing fetal alcohol exposure (video)

Effects of fetal alcohol exposure

Staff training

Oregon liquor license

Damage caused by fetal alcohol exposure

Preventing intoxication

 

Legal issues for alcohol service

Confidence handling underage customers, intoxicated patrons, and aggressive customers

Staff supervision

Oregon RBS program

Factors influencing risk in fetal alcohol exposure

Refusing service to intoxicated customers

 

Problem prevention

Confidence handling drinking drivers and pregnant women ordering alcohol

 
 

Typical effects of fetal alcohol exposure

Health impacts and legal issues of serving alcohol to pregnant women

 

Other resources

Satisfaction with establishment policies

 
  

Avoiding alcohol service to pregnant women

  

Satisfaction with the web-based training program

 
Figure 2

Example Webpage from Web-RBS Program.

The six separate server modules included: (1) Introduction to RBS training, (2) Effects of Alcohol, (3) Who Not to Serve, (4) Alcohol Problem Prevention, (5) Skills and Resources, and (6) an online Knowledge (certification) Test. Effective alcohol server responses to challenging situations were modeled in Modules 3–5 assisted by online role playing video vignettes. For example, videos demonstrated how to card (check the identification) of customers, ways to refuse service to an intoxicated customer, and how to discourage an intoxicated customer from driving. Each server module concluded with an interactive quiz that asked questions derived from the information covered in that module. A successful score on the Knowledge Test earned participants an informal certificate of program completion. The module for managers focused on their specific knowledge, roles, and responsibilities, including the State of Oregon system regulations, the house policies for risk reduction, self-protection, customer cooperation, training methods and the server-customer relationship, support and supervision through communications skills, and encouraging teamwork.

Acceptability and feasibility study

We conducted a preliminary acceptability and feasibility study that assessed the extent to which participants using the RBS training program displayed improvements in their knowledge about alcohol service, attitudes about RBS service, self-efficacy, and intention to change their behavior in their establishments following the training experience. The Web-RBS program was made available to all staff who worked in an alcohol establishment, including servers, bartenders, and owners. Screening, informed consent, and study assessments were all completed online. Both Baseline and Post-test assessed alcohol server attitudes, knowledge, and self-efficacy. The Post-test also measured consumer satisfaction with the website.

Confidentiality of data management

All data associated with the study were maintained in a confidential manner. All transmission of data over the Internet was protected by our use of Secure Sockets Layer (SSL) – an industry standard encryption protocol. Study data were stored on secure servers located within a locked computer room that could only be accessed by approved study staff. In addition, each study participant was assigned a unique username and password which provided secure access to the program website. All study results were described only as aggregate data and never in a manner that could be used to identify any participant’s identity. Participants agreed to an online informed consent before they were able to interact with the Web-based intervention. The research protocol was approved by the Institutional Review Board of Oregon Research Institute.

Recruitment

Study participants were recruited using informational announcements sent to local alcohol service establishments listed in the OLCC licensee database[22]. Establishments were considered eligible if they: (a) were located within any of three local Oregon counties (Linn, Benton, or Lane); (b) were independent establishments (not based in hotel, club, or larger organization); (c) had 12 or fewer alcohol service staff; and (d) had separate alcohol-serving and food service areas. The resulting sample of prospective service establishments that could be included in the study included rural, small town and small urban establishments.

An introductory recruitment meeting was held at each establishment attended by servers and establishment management. During this meeting, a research staff member provided an overview of the project. Topics included the study design, details about logging onto the RBS website to complete the screening, and the importance of completing the online informed consent. Printed materials were also left for any prospective participants who could not attend the meeting. Each study participant was given a card with a website address (url) and their unique registration information. Study participants had unique usernames and passwords, and they used this information to complete an online informed consent. Establishment owners/managers received $150 for supporting their staff to participate in the program. Study participants who completed Baseline and Post-test assessments received $50.

Characteristics of establishments and participants

Twenty-two establishments with 112 participants were recruited for the feasibility and acceptability study of the Web-RBS intervention. More than 60% of participating establishments were tavern/bars. The typical participant was a 35 year-old female bartender or server with some college education, 10 years of hospitality service (7 years of alcohol service), and currently certified to sell alcoholic beverages in Oregon. More precise descriptions of participant characteristics are shown in Table 2.
Table 2

Participant characteristics at baseline

Characteristic

 

Web-RBS

  

(N = 112)

Age (Mean yrs.)

34.10

(SD = 12.47)

Female (No.)

78

(69.6%)

Education (No.)

  

 No high school degree

3

(2.7%)

 High School graduate

16

(14.4%)

 Some college

61

(55.0%)

 College graduate

30

(27.0%)

 Post graduate

1

(0.9%)

Hospitality experience (Mean yrs.)

9.84

(SD = 9.42)

Alcohol service experience (Mean yrs.)

6.74

(SD = 7.95)

Length of RBS certification (Mean yrs.)

1.99

(SD = 1.49)

Measures

RBS knowledge

A set of 15 multiple choice questions was included on both the Baseline and Post-test assessments to measure knowledge of RBS content required by the State of Oregon and covered in our online RBS training program. For example, one item asked: "The brain is vulnerable to alcohol damage through the approximate age of 15, 20, or 25." Another item asked: "A server under 21 with a minor service permit in Oregon may: a) not mix drinks but may serve alcohol in a bar or lounge, b) only serve alcohol in areas where minors are permitted such as a restaurant, or c) serve alcohol from a kitchen but may not enter a restaurant bar or lounge)."

RBS attitudes and self-efficacy

The Baseline and Post-test assessments used a set of 40 items to test RBS attitudes related to content covered in our online RBS training. For example, one item asked respondents to rate the severity of the problem of fetal alcohol spectrum disorders and another item asked for a rating of confidence (self-efficacy) in dealing with types of challenging situations.

Consumer satisfaction

The Post-test included six Likert-scaled items that asked about whether the program provided new information useful for work, whether the information was personally interesting, whether they would recommend the program to other alcohol servers or owners/manager, how well organized the program was, and how they would compare the online RBS training program with live RBS training.

Program exposure

Unobtrusive measures of participant exposure to the RBS training were collected unobtrusively by the program, including the frequency and duration of each participant session and more detailed data on the viewing of individual program web pages[23].

Statistical analyses

RBS knowledge

Each participant was assigned a score defined as the sum of knowledge items answered correctly (possible range = 0–15). Analysis of knowledge change from Baseline to Post-test was examined using paired t-tests. Individual item analysis was accomplished using classification table tests.

RBS attitudes and self-efficacy

The set of 40 attitude items was initially grouped into 7 subscales according to theme and similarity of rating scales. All items were scaled in a manner such that a higher rating reflected a more desirable or positive attitude. Subscales were further refined by examining their internal reliability (Cronbach's alpha; a) which resulted in dropping 2 scales thus yielding 5 subscales: Usefulness at Work (8 items; a = .65), Self-efficacy (8 items, a = .85), Risks (2 items, a = .66), Being Prepared (2 items, a = .87), and Establishment Policies (10 items, a = .92). A composite score for each subscale was assigned to each participant defined as the mean of ratings of subscale items. Baseline to Post-Test comparisons of attitudes used t-tests for independent samples for each subscale. Analysis of changes in attitudes from Baseline to Post-test was examined using paired t-tests.

Program exposure

Program exposure (use of the program) was the number of visits and overall time spent using the program[24].

Results

A total of 84.8% (95/112) of study participants completed both the Baseline and the Post-test. The mean elapsed time between completion of the Baseline and the Post-test was 5.4 days (SD = 8.1, Median = 0, Range = 0–29).

RBS knowledge, attitudes, and self-efficacy

RBS knowledge change (Baseline to Post-test)

Improvement in RBS knowledge was observed among 95 study participants by comparing the mean number of correct items out of the 15 item assessment at Baseline (Mean = 9.07, SD = 2.11) and the Post-test (Mean = 11.9, SD = 2.02). Paired t-test results revealed that this improvement was statistically significant (paired difference Mean = −2.82, SD = 2.44, t = −11.29, df = 94, p < .001). Analysis of individual items using classification tables tests revealed significant improvement on 73% (11/15) of knowledge items.

RBS attitude and self-efficacy (Baseline to Post-Test)

Scores of the five attitude subscales were examined using paired t-tests for change from Baseline and Post-test. In each instance, statistically significant improvements in mean scale ratings were observed (see Table 3).
Table 3

Attitudes, self-efficacy, and knowledge at Baseline and Post-test (N = 112)

Measure

Baseline

Post-Test

t*

P

d

 

Mean

(SD)

Mean

(SD)

   

Attitudes

 Usefulness †

0.45

(0.27)

0.72

(0.30)

−8.86

< .001

0.95

 Risks

3.30

(1.54)

5.20

(0.99)

−11.78

< .001

1.47

 Being prepared

5.08

(1.32)

5.73

(0.57)

−5.01

< .001

0.64

 Establishment Policies

5.40

(0.81)

5.54

(0.73)

−2.47

.015

0.18

Self-efficacy

4.97

(1.08)

5.49

(0.71)

−5.10

< .001

0.57

Knowledge Change ‡

9.07

(2.11)

11.9

(2.02)

−11.29

< .001

1.29

* Paired t-test: 94 degrees of freedom and 2-tailed significance.

† Dichotomous items scored Yes = 1 and No = 0.

‡ Score defined as the sum of knowledge items answered correctly (possible range = 0–15).

Consumer satisfaction

Almost all of the 93 study participants who provided consumer satisfaction data reported that they completed all sections of the program. Most reported that the program contained information that was new to them (31.2% indicated "a lot" and 59.1% indicated "some"), that it was useful for their work (41.9% indicated "a lot" and 48.4% indicated "some"), and that the information was personally interesting (32.3% indicated "a lot" and 53.8% indicated "some"). Similarly, 79.6% of participants reported that they found the program to be very well-organized and 82.8% reported that the program contained "about the right amount of information." They also reported that they would recommend the program to other alcohol servers or owners/managers (38.7% answering "very strongly recommend" and 39.8% "might recommend"). The majority of participants reported that they liked the online RBS program "much better" (32.3%) or "better" (26.9%) than live RBS trainings.

Similarly very positive results were obtained regarding participant ratings of usefulness of the 10 program sections (Table 4). Each of these sections was assigned a rating of “Very Useful” or “Useful” to around 80% of participants.
Table 4

Rating of the usefulness of Web-RBS program at Post-test (N = 93)*

Feature, N (%)

Very useful

Useful

Only slightly useful

Useless

Did not visit

Overview

35 (37.6%)

46 (49.5%)

12 (12.9%)

  

Alcohol and minors

41 (44.1%)

44 (47.3%)

8 (8.6%)

  

Intoxication

41 (44.1%)

42 (45.2%)

9 (9.7%)

1 (1.1%)

 

Fetal Alcohol Spectrum Disorder

59 (63.4%)

30 (32.3%)

4 (4.3%)

  

RBS research findings

39 (41.9%)

47 (49.5%)

7 (7.5%)

1 (1.1%)

 

Hospitality industry viewpoint

36 (38.7%)

42 (45.2%)

12 (12.9%)

3 (3.2%)

 

Legal information

50 (53.8%)

40 (24.0%)

3 (3.2%)

  

Problem prevention ideas

44 (47.3%)

40 (43.0%)

7 (7.5%)

2 (2.2%)

 

Section on Management

34 (36.6%)

41 (44.1%)

10 (10.8%)

2 (2.2%)

6 (6.5%)

Skills and Resources Section

34 (36.6%)

46 (49.5%)

9 (9.7%)

1 (1.1%)

3 (3.2%)

*Empty cells indicate no answers of that type were recorded.

Program exposure

Most study participants visited the Web-based program on the first day it was made available for their use and participants visited an average 2.9 times (SD = 2.2; Median = 2.0). They viewed an average of 133.4 program webpages (SD = 50.5; Median = 148.5), and they spent an average of 113.2 min viewing the program (SD = 76.5; Median = 105.6).

Discussion

The development of an innovative Web-based RBS training program was informed by formative research procedures and the contribution of a multidisciplinary design team. An acceptability and feasibility study of a completed version of this program was conducted with servers recruited from alcohol service establishments. Almost all participants completed the program, they improved their RBS knowledge, attitudes and self-efficacy, and they reported that they found the program to be useful, engaging, personally interesting and well-organized. The magnitude of these effects using Cohen’s d[25] was encouraging with three measures (attitude changes for usefulness and risks; knowledge change) showing a large effect size, two measures (attitude change on being prepared; self-efficacy change) showing a moderate effect size, and only one measure (attitude change on establishment policies) showing a small effect size. These overall strong positive findings were obtained despite the fact that participants had an average of 6.8 years (SD = 8.0) experience in alcohol service and an average of 9.8 years (SD = 9.4) experience in the hospitality industry. Greater changes in knowledge and attitudes would have been expected among novice servers. The present study is noteworthy because, to our knowledge, it represents the first reported empirical test of Web-based RBS training.

This study has some limitations. It is important to acknowledge that Baseline to Post-test changes may not indicate that participants had a deeper understanding of RBS content nor do they provide a test of long-term retention of relevant RBS content[26]. The present study did not assess possible changes in server on-the-job behaviors. A more controlled outcome study would include direct observation of pseudo patrons in establishment settings who would model the behaviors that a server would need to identify and respond to appropriately[2733]. For example, these behaviors could likely include (1) carding of possibly underage patrons, (2) refusal to serve visibly intoxicated patrons, (3) preventing a visibly intoxicated patrons from driving, and (4) not serving visibly pregnant patrons.

It is also important to acknowledge that the efficacy of any brief RBS training intervention is likely to be enhanced were it to be used within the broader context of a community effort to institutionalize programs to prevent alcohol problems[3436]. For example, Wallin and colleagues[36] have described how a concerted community-wide effort focusing on adoption, sustainability, key leader support, structural changes, and compliance resulted in the institutionalization of support for the prevention of alcohol problems in Stockholm, Sweden. This community support and compliance was associated with higher rates of refusing alcohol service to intoxicated and underage patrons and a significant reduction in violent crimes. Shults and colleagues[35] report similar positive results for multi-component programs (with RBS training) with community mobilization for reducing alcohol-impaired driving.

Finally, it remains for further research to examine other questions that could have important implications for RBS practices. For example, one research direction might examine the extent to which combining RBS classroom training with a Web-based program adjunct might prove more beneficial than providing training by either modality on its own. Additional research might also include direct behavioral interventions in alcohol service settings. Moreover, future research might examine the use of pseudo patrons and observational assessments with longer follow-up intervals to assess for meaningful longer-term improvements in RBS practices. It is clear from the significant number of people killed, injured, and affected by drunk driving in the United States that there is a public health need for ongoing research on evidence-based, effective RBS practices to reduce these staggering statistics. The current study has added to this body of research in an important way.

Abbreviations

OLCC: 

Oregon Liquor Control Commission

RBS: 

 Responsible Beverage Service: The appropriate sale and service of alcohol in commercial establishments.

Declarations

Acknowledgements

The authors acknowledge the key contribution of our Web technology developers at InterVision Media (Eugene, OR) and Katie Clawson (Oregon Research Institute). This project was funded by a Phase II Small Business Innovation Research (SBIR) grant #5R44AA014108-03 from the National Institute on Alcohol Abuse and Alcoholism awarded to the Deschutes Research, Inc. (Jack Dresser, PhD, Principal Investigator). The copyright for the program is held by Deschutes Research, Inc. (2008).

Authors’ Affiliations

(1)
Oregon Research Institute
(2)
Deschutes Research, Inc
(3)
InterVision Media

References

  1. MMWR: Vital signs: alcohol-impaired driving among adults—United States, 2010. Morb Mortal Wkly Rep. 2011, 60: 1351-1356.Google Scholar
  2. O'Donnell MA: Research on drinking locations of alcohol-impaired drivers: implications for prevention policies. J Public Health Policy. 1985, 6: 510-525. 10.2307/3342050.View ArticlePubMedGoogle Scholar
  3. Naimi TS, Nelson DE, Brewer RD: Driving after binge drinking. Am J Prev Med. 2009, 37: 314-320. 10.1016/j.amepre.2009.06.013.View ArticlePubMedGoogle Scholar
  4. Mosher JF, Toomey TL, Good C, Harwood E, Wagenaar AC: State laws mandating or promoting training programs for alcohol servers and establishment managers: an assessment of statutory and administrative procedures. J Public Health Policy. 2002, 23: 90-113. 10.2307/3343120.View ArticlePubMedGoogle Scholar
  5. Saltz RF: Where we are in the evoluation of responsible beverage service program evaluations. Alcohol Drugs Driving. 1994, 10: 277-285.Google Scholar
  6. Saltz RF, Stanghetta P: A community-wide Responsible Beverage Service program in three communities: early findings. Addiction. 1997, 92 (Suppl 2): S237-S249.View ArticlePubMedGoogle Scholar
  7. Stockwell T: Responsible alcohol service: lessons from evaluations of server training and policing initiatives. Drug Alcohol Rev. 2001, 20: 257-265. 10.1080/09595230120079567.View ArticleGoogle Scholar
  8. Alcohol Policy Information System: Retail sales: beverage service training and related practice: beverage service training. 2012, http://www.alcoholpolicy.niaaa.nih.gov/Beverage_Service_Training_and_Related_Practices.htmlGoogle Scholar
  9. Cockburn A: Using both incremental and iterative development. 2008,http://www.crosstalkonline.org/storage/issue-archives/2008/200805/200805-Cockburn.pdfGoogle Scholar
  10. Danaher BG, McKay HG, Seeley JR: The information architecture of behavior change websites. J Med Internet Res. 2005, 7: e12-10.2196/jmir.7.2.e12.PubMed CentralView ArticlePubMedGoogle Scholar
  11. OLCC: Oregon administrative rules (Oregon Liquor Control Commission). 2011,http://arcweb.sos.state.or.us/pages/rules/oars_800/oar_845/845_tofc.htmlGoogle Scholar
  12. Krueger RA, Casey MA: Focus groups: A practical guide for applied research. 2009, Thousand Oaks, CA: Sage, 4Google Scholar
  13. Lynch PJ, Horton S: Web style guide: Basic design principles for creating Web sites. 2008, New Haven, CT: Yale University PressGoogle Scholar
  14. Lynch PJ, Horton S: Web style guide. 2009,http://webstyleguide.comGoogle Scholar
  15. Nielsen J: Why you only need to test with 5 users. 2000http://www.useit.com/alertbox/20000319.htmlGoogle Scholar
  16. Nielsen J: How many test users in a usability study. 2012http://www.useit.com/alertbox/number-of-test-users.htmlGoogle Scholar
  17. Davison GC, Vogel RS, Coffman SG: Think-aloud approaches to cognitive assessment and the articulated thoughts in simulated situations paradigm. J Consult Clin Psychol. 1997, 65: 950-958.View ArticlePubMedGoogle Scholar
  18. Tognazzini B: TOG on interface. 1992, Reading, MA: Addison-WesleyGoogle Scholar
  19. Nielsen J: Thinking aloud: the #1 usability tool. 2012http://www.useit.com/alertbox/thinking-aloud-tests.htmlGoogle Scholar
  20. Hughes M: Rigor in usability testing. Technical Communication. 1999, 46: 488-494.http://www.faculty.english.ttu.edu/carter/5388/Resources/Rigor.pdf%5DGoogle Scholar
  21. Rosson MB, Caroll JM: Usability engineering: Scenario-based development of human-computer interaction. 2002, San Diego: AcademicGoogle Scholar
  22. OLCC: Liquor stores by city. 2012http://www.olcc.state.or.us/pdfs/liquor_store_list.pdfGoogle Scholar
  23. Danaher BG, Seeley JR: Methodological issues in research on Web-based behavioral interventions. Ann Behav Med. 2009, 38: 28-39. 10.1007/s12160-009-9129-0.View ArticlePubMedGoogle Scholar
  24. Danaher BG, Boles SB, Akers L, Gordon JS, Severson HH: Defining participant exposure measures in Web-based health behavior change programs. J Med Internet Res. 2006, 8: e15-10.2196/jmir.8.3.e15.PubMed CentralView ArticlePubMedGoogle Scholar
  25. Cohen J: Statistical power analysis for the behavioral sciences 2nd edn. 1988, Hillsdale, NJ: sLawrence Erlbaum AssociatesGoogle Scholar
  26. Saltz RF: Evaluating specific community structural changes. examples from the assessment of responsible beverage service. Eval Rev. 1997, 21: 246-267. 10.1177/0193841X9702100207.View ArticlePubMedGoogle Scholar
  27. Freisthler B, Gruenewald PJ, Treno AJ, Lee J: Evaluating alcohol access and the alcohol environment in neighborhood areas. Alcohol Clin Exp Res. 2003, 27: 477-484. 10.1097/01.ALC.0000057043.04199.B7.View ArticlePubMedGoogle Scholar
  28. Lang E, Stockwell T, Rydon P, Beel A: Use of pseudo-patrons to assess compliance with laws regarding under-age drinking. Aust N Z J Public Health. 1996, 20: 296-300. 10.1111/j.1467-842X.1996.tb01032.x.View ArticlePubMedGoogle Scholar
  29. Lang E, Stockwell T, Rydon P, Beel A: Can training bar staff in responsible serving practices reduce alcohol-related harm?. Drug Alcohol Rev. 1998, 17: 39-50. 10.1080/09595239800187581.View ArticlePubMedGoogle Scholar
  30. Russ NW, Geller ES: Training bar personnel to prevent drunken driving: a field evaluation. Am J Public Health. 1987, 77: 952-954. 10.2105/AJPH.77.8.952.PubMed CentralView ArticlePubMedGoogle Scholar
  31. Toomey TL, Wagenaar AC, Erickson DJ, Fletcher LA, Patrek W, Lenk KM: Illegal alcohol sales to obviously intoxicated patrons at licensed establishments. Alcohol Clin Exp Res. 2004, 28: 769-774. 10.1097/01.ALC.0000125350.73156.FF.View ArticlePubMedGoogle Scholar
  32. Toomey TL, Wagenaar AC, Gehan JP, Kilian G, Murray DM, Perry CL: Project ARM: alcohol risk management to prevent sales to underage and intoxicated patrons. Health Educ Behav. 2001, 28: 186-199. 10.1177/109019810102800205.View ArticlePubMedGoogle Scholar
  33. Warpenius K, Holmila M, Mustonen H: Effects of a community intervention to reduce the serving of alcohol to intoxicated patrons. Addiction. 2010, 105: 1032-1040. 10.1111/j.1360-0443.2009.02873.x.View ArticlePubMedGoogle Scholar
  34. Holder HD, Moore RS: Institutionalization of community action projects to reduce alcohol-use related problems: systematic facilitators. Subst Use Misuse. 2000, 35: 75-86. 10.3109/10826080009147687.View ArticlePubMedGoogle Scholar
  35. Shults RA, Elder RW, Nichols JL, Sleet DA, Compton R, Chattopadhyay SK: Effectiveness of multicomponent programs with community mobilization for reducing alcohol-impaired driving. Am J Prev Med. 2009, 37: 360-371. 10.1016/j.amepre.2009.07.005.View ArticlePubMedGoogle Scholar
  36. Wallin E, Lindewald B, Andreasson S: Institutionalization of a community action program targeting licensed premises in Stockholm, Sweden. Eval Rev. 2004, 28: 396-419. 10.1177/0193841X04264951.View ArticlePubMedGoogle Scholar

Copyright

© Danaher et al; licensee BioMed Central Ltd. 2012

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Advertisement