Using Data for Continuous Quality Improvement

Researchers at the Vanderbilt University Center for Evaluation and Program Improvement have developed a system for collecting, tracking, and analyzing program and outcome data for continuous quality improvement (Bickman et al., 2006). It is called Contextualized Feedback Interventions and Training (CFIT), and has four major components, including:

  • Methods for assessing organizational needs, readiness for change, and implementation processes
  • Methods for measuring treatment process and outcomes
  • Methods for formative feedback via on-line reports of aggregated data (by clinic, treatment type, etc.)
  • Methods for training administrators and clinicians in the CFIT approach, effective therapeutic techniques, and evidence-based interventions

A power point presentation on the system with more information on the system can be found at: http://peabody.vanderbilt.edu/documents/pdf/cepi/heart_and_soul_of_change_bickman_june_2006.pdf

In an article aptly titled, From Data to Wisdom: Quality Improvement Strategies Supporting Large-scale Implementation of Evidence-Based Practices, Daleidan and Chorpita (2005) described strategies used by the state mental health agency in Hawaii to increase the utilization of three types of data that come from:  a) services and intervention research, b) case histories, and 3) aggregated practice-based evidence. Below are some of the strategies for using these types of evidence:

Strategies for Using Services Research Information

  • Use of an information tool called the “Blue Menu”, a one page matrix of target problem, description of treatment packages, and efficacy level
  • Incorporate into interagency performance standards and practice guidelines that are attached to service contracts
  • Use of a practice development office that is responsible for providing interagency training, mentoring, and consultation to promote ongoing skill development and dissemination of service research information
  • Funding structures in place to support EBPs
  • Utilization management procedures to monitor whether relevant populations are receiving evidence-based levels of care and whether service use is consistent with practice guidelines

Strategies for Using Data from Individual Case Histories

  • Use of diagnoses to consider complexity of symptomatology
  • Service plans to organize treatment targets and intervention data
  • Quarterly standardized assessments to monitor symptoms, functioning, and service needs
  • Treatment providers complete monthly summary of interventions provided
  • Case-based and administrative reviews with small random samples from each region and provider

Strategies for Using Aggregated Data (Practice-based Evidence)

  • Dissemination of information via user friendly graphic reports
  • System performance measures

Hawaii’s approach exemplifies the process of valuing many sources of data and institutionalizing or "acculturating" data-based approaches into everyday decision-making, which is considered to be one of the main features of an evidence-based culture. Their quality improvement framework, databases, and processes are being replicated in other states.

References

Bickman, L., Riemer, M. Breda, C., & Kelley, S.D. (2006). CFIT: A system to provide a continuous quality improvement infrastructure through organizational responsiveness, measurement training, and feedback. Emotional and Behavioral Disorders in Youth, 6 (4), 86-94.

Daleidan, E.L., & Chorpita, B.F. (2005). From data to wisdom: Quality improvement strategies supporting large-scale implementation of evidence-based services. Child and Adolescent Psychiatric Clinics of North America , 14 , 329-349.