Sampling and recruitment
We conducted this small scale study using a mixed methods research design with a non-randomized controlled sampling strategy as a pilot for a larger national evaluation of MAPs in Canada. Quantitative data on outcomes of interest were collected from male and female participants in a series of short monthly surveys, with more in-depth surveys at baseline and 6-month follow-ups. Qualitative interviews were conducted with MAP staff and residents to gain a deeper understanding of their experiences within the program. A mixed methods design incorporates multiple methods of inquiry to enhance scope and breadth of understanding in evaluation research [29, 30]. Both triangulation (convergence of findings) and complementarity (examining different aspects of the same phenomena) were employed within this study. A mixed-gender control group was included to help determine whether observed benefits or harms experienced by participants were specifically due to participation in the MAP. Control participants were recruited by the study researchers at the shelter next door to the MAP, which was a separate facility from the MAP but run by the same agency. The shelter provides access to set meals three times a day, and weather permitting, participants must be outside the building during the day other than at staff discretion if there is a risk of harm from being outside. There are no organized recreational activities offered at the shelter, and rides may be available for health-care appointments after 2 pm each day, but access to transportation is not guaranteed. Some Indigenous-focused activities such as drumming circles are also available, but there are no regular visits by Elders.
In order to be eligible to participate in the study, control participants had to meet the criteria for potential entry into the MAP but due to program space or personal choice, not currently be participants in the MAP. The proportion of controls that had not entered the MAP due to program space versus personal choice is not known. Participant reasons for not entering MAP included perceptions of stigma related to being administered alcohol and also some potential discomfort with having to follow rules within a more structured program setting. It was not feasible to randomly assign eligible individuals to the MAP or shelter controls because the MAP operates independently of the research study. The shelter and the MAP are run by the same organization and provide services to the same population. However, because the MAP was restricted to 15 beds and there was a low program turnover, many of the individuals attending the adjacent shelter would have been eligible for MAP and were therefore similar to a waiting-list control group. In order to locate comparable controls, the program staff initially compiled a list of individuals from the shelter who fit the criteria and would be eligible for the MAP but were not currently in the program. The first 15 controls who met the criteria were selected. The screening tool used to establish eligibility of controls consisted of four questions: (1) whether they had been without their own place to stay in the past 6 months, (2) whether they had been without their own place to stay more than four times in the past 6 months, (3) whether they have had many difficulties caused by drinking in the past few years, and (4) whether they had been picked up by the police due to alcohol, been to the hospital ER due to alcohol, or been to detox at least four times in total in the past 6 months. If they met each criterion, they were included in the study.
Quantitative data were collected for a 6-month period between March and September 2013. At the beginning of the 6-month period, structured quantitative baseline surveys were conducted with 18 consenting MAP and 20 consenting control participants in March and April of 2013. Fifteen of the 18 MAP participants had been residents in the program for at least 1 month when the first survey was conducted. Between the second and fifth month of data collection, a subset of six of the newly admitted MAP participants and 10 matched control participants were selected to complete shorter monthly quantitative follow-up interviews. Control participants were selected at a ratio of two MAP participants to three control participants to account for higher anticipated attrition among controls. At the 6-month mark, an in-depth quantitative survey was conducted with as many as possible of the smaller subset of MAP and control participants who had been selected for follow-up. Monthly follow-up surveys were conducted not only in part to gather information but also as a strategy to maintain contact with this population, especially the control participants who were more likely than MAP participants to be moving from place to place because of homelessness. One-time, face-to-face qualitative surveys were conducted with four MAP staff and seven MAP residents within the 6-month period in 2013.
Participation in the study was voluntary and written informed consent was obtained from all participants prior to their taking part. MAP and control participants received $25 gift vouchers for the longer quantitative interviews and $10 gift vouchers for the shorter, monthly quantitative follow-up interviews. MAP participants who completed the qualitative interviews received a $25 gift voucher. Ethical approval for this study was obtained from the University of Victoria and Lakehead University Human Research Ethics Committees, Thunder Bay Regional Health Science Centre (TBRHSC), and St. Joseph’s Care Group (SJCG) research ethics committees.
Measures
Quantitative surveys
The quantitative surveys conducted at the beginning and end of the 6-month data collection period covered the following domains: sociodemographic characteristics; housing status over the past 12 months; alcohol and other substance use; severity of alcohol-related problems and degree of alcohol dependence; health and mental health; and housing quality. A range of questions on individual-level alcohol-related social harms was also included. Several standardized instruments were included in the survey such as the Alcohol Use Disorders Test (AUDIT) [31], the Severity of Alcohol Dependence Questionnaire (SADQ) [32], the Colorado Symptom Index [33] which measures psychological symptomatology, and the WHOQOL-BREF [34], which is an assessment of quality of life. The shorter quantitative surveys conducted monthly included questions about housing and alcohol consumption and also the WHOQOL-BREF. Housing quality and satisfaction as well as quality of life data are reported separately [28].
Results from the quantitative surveys presented in this paper primarily include data from the initial interviews conducted at the beginning of the study. As just six MAP participants and seven controls were successfully followed up and completed the longer interview at the 6-month mark, only limited descriptive data will be provided on their outcomes. Analysis of the survey data from the larger number of initial intake interviews included chi-square tests [35] to determine significant differences between the MAP participants and control participants on selected demographic variables, self-reported alcohol consumption, and a selection of relevant individual-level alcohol-related social harms such as home life, housing status, legal issues, and experience of withdrawal seizures. A two-sample t test was used to test any significant differences in reported AUDIT scores and days of NBA use.
MAP alcohol consumption records
Alcohol administration data routinely collected by MAP staff were accessed for the study, and these included the number of drinks administered per serve as well as the time of day that the serve occurred. A question was also included about consumption outside the MAP in the previous 24 hours and was asked of residents as an open-ended question at the time of their first drink of the day.
Liver function tests
Blood samples for liver functions tests (LFTs) were collected by a nurse practitioner from a nearby health clinic at intervals throughout the program. In some cases, LFT results from MAP and control participants’ health records were also available. LFT results accessed included aspartate transaminase (AST), a liver enzyme sensitive to acute liver damage with a normal range between five and 40; alanine transaminase (ALT), a liver enzyme with a normal range between seven and 56; and gamma glutamyl transpeptidase (GGT), which measures liver dysfunction and has a normal range of zero to 65 for males and zero to 45 in females. Factors other than alcohol consumption can affect liver functioning on these tests, e.g., hepatitis, nutrition, and body weight.
Police and health-care records
Ethical approval and written consent from 13 MAP participants and 10 controls was obtained to access archival police and health-care records for the 5 years prior to the initiation of the study and 12 months afterwards for both MAP and control participants. The date range for these records was August 2008 to August 2013. Police records from the Thunder Bay Police included the number of police contacts and in cases when police contacts resulted in custody or jail time, the length of custody and jail time. Individual level health-care records from the Thunder Bay Regional Health Science Centre included information on frequency and duration of hospital visits, and data from St. Joseph’s Care Group included in-patient detoxification episodes.
MAP participants averaged 357.5 days (SD = 321.47, Min/Max 398/1728) on the program compared to 1220.9 days (SD = 143.98, Min/Max 91/529) off the program during that 5-year time period. The numbers of police contacts, hospital admissions, ER presentations, and detoxifications per 100 observed days were estimated for participants on MAP and off MAP and for controls. We used one-sided paired t tests and two-sample t tests to test the hypotheses that MAP participation was associated with improved outcomes, compared with periods prior to MAP entry and compared with similar controls who were not on a MAP [35]. Paired t tests were used to investigate any significant difference in the numbers of police contacts, hospital admissions, ER presentations, and detoxifications per 100 observed days between participants on MAP and off MAP. Two-sample t tests were used to investigate observed differences in rates of police contacts, hospital admissions, ER presentations, and detoxifications per 100 observed days between participants on MAP and controls. In each case, one-sided significance tests were employed to test explicit hypotheses that MAP participation would be associated with reductions in these areas. Chi-square (x
2) tests were used to compare the proportions of police contacts leading to custody time for participants while on the MAP compared with periods off the MAP as well as compared with controls.
Qualitative interviews
Experienced qualitative researchers from the study team conducted one-time face-to-face interviews with seven MAP residents, three females, and four males, all of whom identified as Indigenous, and who had been in the program at least 1 month. For MAP residents, the focus of the qualitative interview was on their experiences before entering the program, their experiences within the program, and the impact of the program on patterns of drinking, health, housing, quality of life, and social relationships. Interviews were conducted with four of the MAP staff with a focus on their experiences working in the program, including their thoughts on program goals and structures, changes in the program, program impacts, and community responses to the program. Qualitative findings related to health, quality of life, and housing are reported in the sister manuscript in this issue [28]. All interviews were audio-taped and transcribed. Qualitative interviews were not conducted with control participants.
Analysis of the qualitative data was conducted using constant comparative analysis, in which each transcript was read and re-read by two members of the research team and coded inductively by both research team members for key ideas and themes that described the experience of being in a MAP related to health, housing, quality of life, and harms of alcohol use and drinking patterns. An inductive coding framework was developed, and NVivo (NVivo qualitative data analysis software; QSR International Pty Ltd. Version 10, 2012) was used to organize and manage the data [36–39]. Qualitative results presented in this paper relate specifically to life in the MAP, use of NBA and alcohol consumption patterns, contacts with police, and access to and use of health-care services.