Study rationale:
Canada legalized the production, sale, and use of cannabis for non-medical (recreational) purposes in October 2018. Several other countries are currently considering similar policy changes. Measurement of cannabis use and related behaviours is fundamental in evaluating the impacts of cannabis legalization and regulation. Further, comparing cannabis data across regions (e.g., countries) can be important in assessing the effectiveness of a particular policy over another. A recent scoping review identified 187 unique instruments that measure some aspect of cannabis use, covering seven content domains and 35 subdomains (Lazor et al., 2022). Likewise, an ongoing systematic review identified almost 300 population surveys being used globally that assess at least some cannabis outcomes (Shield et al., in preparation). This systematic review involving population and drug use monitoring surveys found considerable heterogeneity among cannabis use indicators and measurement instruments. This has resulted in significant challenges in comparing and synthesizing cannabis data across jurisdictions and demographic populations.
Objectives:
This research is meant to facilitate the standardization of cannabis measurement in population surveys and to support the development of new population surveys that can evaluate the impacts of changing cannabis policies, namely cannabis legalization.
The objectives of this study are to:
I) identify the most important cannabis use indicators for evaluating the impacts of cannabis legalization and
II) determine how best to measure cannabis use indicators on a population level.
Significance:
The significance of this study entails advancing the consistency of cannabis measurement, both nationally and internationally, so that better comparisons can be made across jurisdictions and studies. This research will inform the development of policy guidelines for measuring cannabis use globally and consequently improve understanding of the impacts of cannabis legalization.
Dr. Robert Gabrys, Canadian Centre of Substance Misuse
Dr. Tyler Marshall, University of Alberta
Dr. Andrew J Greenshaw, University of Alberta
Disease Category: Tobacco, drugs, & alcohol dependence
Disease Name: Cannabis use disorder/problematic cannabis use
Age Range: Unknown
Sex: Either
Nature of Intervention: N/A
- Epidemiologists
- Governmental agencies
- Policy makers
- Researchers
- COS (Other)
- Consensus meeting
- Interview
- Survey
- Systematic review
An online, modified Delphi study will be conducted based on the Core Outcome Measures in Effectiveness Trials (COMET) framework, which aims to achieve consensus on (1) what types of outcomes should be measured and reported, and (2) what outcome measurement instruments are most applicable/appropriate. (Kirkham et al., 2017) This Delphi study will comprise a panel of international experts from the cannabis field to develop a core outcome set and recommendations for measurement. A three-step methodological approach will be employed: (1) conduct a systematic review to determine the scope of outcome heterogeneity and to inform the development of the Delphi questionnaire with an international advisory committee, (2) a three-round Delphi survey and (3) a consensus meeting to verify the findings and inform knowledge translation. The Delphi method was chosen because it is an efficient scientific method to develop expert consensus for developing core outcome sets in health-related research.
Delphi reporting checklists by Humphrey-Murto et al., 2019 and Sinha et al., 2011 were used to guide the development of this study and the Outcome Set–STAndards for Reporting (COS-STAR) Statement will be used to guide reporting of the results (Kirkham et al., 2016).
Ethics:
The protocol was developed a priori and obtained ethics approval from the University of Alberta Research Ethics Board (PRO 00128739)
Participant Eligibility Criteria:
Participants must have expertise in developing, informing (e.g., working groups), and reporting on population-level surveys or questionnaires that assess cannabis-related outcomes such as cannabis use and related behaviours. Individuals that have experience developing population surveys that focus on cannabis-only and those that include some cannabis outcomes will be included. Participants must have at least one publication (e.g., peer-reviewed, government report) on cannabis use and related behaviours.
Delphi Survey Procedures
A modified, three-round, online Delphi survey will be administered electronically via Qualtrics software (Provo, UT), spanning approximately six months (Jan 2023 – July 2023). The survey will be fully anonymous, and anonymous feedback on the results will be provided to each participant via email before the subsequent round. Only participants who complete at least 50% of the survey will be invited to the subsequent round. Attrition will be minimized by including a thank you text for participating in the study explaining the importance of participating in all Delphi rounds. Members of the expert advisory committee may participate in the Delphi rounds. Participants who complete all three surveys will be invited to a final consensus meeting to verify the core outcome set and determine the most suitable measurements.
Collection of Demographic Data:
The number of individuals contacted, registered, and demographic information such as age, gender identity, ethnicity, employment experience, country of residence, etc. (The complete demographic questionnaire can be found in Appendix C). The number of participants contacted, completing each round, and completing each survey will also be recorded.
Round 1:
Participants will be invited to participate in the first round of the survey via email. Participants will be provided with an information letter, asked to participate in all rounds of the study, and complete a brief demographic questionnaire. The Delphi round one questionnaire will be comprised of a series of free text and open-ended responses that were developed in consultation with the steering committee to identify a list of potential cannabis outcomes, which will create a comprehensive list of potentially relevant outcomes which will be ranked during each round of the questionnaire. Free-text questions will be asked to ensure all potentially relevant/important outcomes are identified with minimal selection or researcher bias (Sinha et al., 2011) (The round one questionnaire can be found in Appendix C). Participants will have two weeks to complete the survey, and a reminder e-mail will be sent 48 hours before the deadline. Non-responders will be excluded from subsequent Delphi rounds.
Rounds 2 and 3:
A series of outcomes/statements will be rated for importance using a Likert scale to assess consensus around potential core outcome measures across various domains. An “unsure” or “unable to rate” option will be added in addition to a free-text question to clarify each response. Non-responders will be excluded from subsequent Delphi rounds. Participants will have two weeks to complete the next survey (a total of 4 weeks). A list of outcomes will be generated from the results of round one. The items on this questionnaire will be developed after round one has been completed.
Data Analysis
Qualitative thematic analysis will be done using QSR International Pty Ltd. (2018) NVivo (Version 12) software by two independent coders using an iterative and open-coding framework for free text responses. The resulting themes (and minority themes) will be shared with the participants and used to inform the survey for Delphi Round 2. Subsequent surveys after round one will comprise a series of statements and be measured using a Likert scale (e.g., strongly disagree to strongly agree). Criteria for consensus will be met once 70% of the participants agree that an outcome is important (e.g., critically important or strongly agree). The survey will be revised and repeated until consensus is obtained on enough outcomes or a maximum of three Delphi rounds. Outcomes that do not meet consensus will be reported and discussed.
The number of participants invited to each round, the number of participants who completed each round, and the results for each outcome will be reported and a measure of group response per each stakeholder group (if applicable). Each group response's means and standard deviation will be reported according to each outcome from each round, and anonymized results will be provided to each study participant after each round. The summary statistics from the final round will be reported in the final manuscript. The entire list of outcomes from the start of the study will be reported, and new outcomes that are dropped or combined will be reported with reasons. Graphs displaying the distribution of ratings for each outcome by geographic region, stakeholder type and other demographic variables may be produced. SPSS 24 (SPSS Inc., Chicago, IL, USA) will be used to analyze all quantitative data.
Once consensus on a list of core outcome measures is determined, further research may be conducted, including a quality assessment of the outcome instruments (if required). A consensus meeting will select the best measurement approach (e.g., instrument, question) per outcome.
Dissemination
Following analysis of the round three results, an online stakeholder consensus meeting with all participants, authors, and advisory committee members will be held to discuss/verify the core outcome set and to develop consensus around determining the most suitable measurements, knowledge translation and dissemination methods. The results of this study will be written for a CCSA lay publication, and additional scientific manuscript (s) will be submitted to a relevant high-impact journal(s) in the field. Further, this study has a broad knowledge mobilization impact in cannabis research and for survey organizations looking to promote the use of agreed-upon indicators and work towards international standardization.