Working Procedures

B. SCIENTIFIC REVIEW AND EVALUATION

4. Efficacy and effectiveness of a screening procedure

For the evaluation of both efficacy and effectiveness, the Working Group considers the following general principles in making judgements about the available studies:

  • Relevance of the study;
  • Appropriateness of the study design and analysis to the question being asked;
  • Adequacy and completeness of the presentation of the results;
  • Degree to which chance, bias, and confounding may have affected the results.

  • 4.1 Efficacy

    In this section, evidence from randomized controlled trial (RCT) studies is reviewed. All aspects of study design and analysis are critically discussed. Indicators of the efficacy of the procedure in terms of mortality or incidence, as well as other relevant indicators, such as the detectable phases of the natural history of the disease, are presented.

    Aspects that are particularly important in evaluating RCTs are: the selection of participants, the nature and adequacy of the randomization procedure, evidence that randomization achieved an adequate balance between the groups, exclusion criteria used before and after randomization, compliance with the intervention in the screened group, and “contamination” of the control. Other considerations include the means by which the outcome (preneoplastic lesions or cancer) was determined and validated (either by screening or by other means of detection of the disease), the length and completeness of follow-up of the groups, and the adequacy of the analysis.

    When RCTs are lacking, efficacy cannot be directly evaluated, but only indirectly inferred from observational studies (see below).

    4.2 Effectiveness of population-based screening

    The impact of the screening procedure when implemented in defined populations is examined in this section.

    In this section, mostly observational studies are reviewed, conducted in settings with organized screening programmes or with opportunistic screening. In cohort studies, particular attention is paid to the length and completeness of follow-up; in case–control studies, particular attention is paid to the definition of cases and controls, and the screening method. In all observational studies, the potential for chance, bias, and confounding is carefully examined.

      (a) Beneficial effects
      Benefits include a decrease in the incidence of invasive cancer or in cancer-related mortality. In addition, indicators used to monitor effectiveness, such as detection rate, rates of interval cancers, and the number of tests performed, may also be considered. Studies of time trends before and after implementation of screening, as well as comparisons, including geographical comparisons, of the occurrence of the disease and death from the disease in populations exposed and not exposed to screening may also be reviewed and interpreted when relevant. In doing this, the Working Group takes into account differences in screening procedures (e.g. frequency and the age of the target population) and of participation rates.

      When appropriate, the extent to which improved treatment has been responsible for any observed changes in mortality should be considered in assessing the evidence for effectiveness.

      Compliance with participation in screening by a given procedure will also be considered as part of the evaluation of the effectiveness.
      (b) Adverse effects
      Adverse effects to individuals that are linked to the screening procedure are also reviewed. Evaluation of harms includes estimates of rates of false-positive and false-negative findings and their consequences in screened individuals, overdiagnosis, and interval cancers. Harms may also include screening-related medical complications or discomfort, or psychological effects such as anxiety induced by undergoing screening. The rates of short- and long-term adverse effects of the screening procedure and the likelihood of unnecessary treatment are discussed. Evidence on adverse effects may come from any type of epidemiological study design, including RCTs, observational studies, or other studies as relevant.
      (c) Harm-benefit ratio and cost-effectiveness
      Evidence for the harm-benefit ratio and the cost-effectiveness of the screening procedure in various settings is considered, mostly from modelling studies. The discussion takes into account the costs per case detected and the benefits per death prevented. Modelling studies will be reviewed similarly to other studies, with particular attention paid to assumptions.

     

    Updated 14 November 2017