Implementation of a “real-world” learning health care system: Washington state's Comparative Effectiveness Research Translation Network (CERTAIN)


      Learning health care systems apply the experiences of prior patients to inform care and help to guide decision making for current patients. These systems should help to deliver more effective, efficient, and appropriate care. Most examples of learning systems derive from integrated care delivery systems and examples of such systems in the community at large have been lacking.


      The comparative effectiveness research translation network (CERTAIN) is a learning system bringing together hospitals and outpatient clinics across Washington State. CERTAIN leverages existing medical record-based data collection taking place at nearly all statewide hospitals and links this data collection with patient-reported information about function and quality of life.


      We have described the components of the CERTAIN infrastructure, the elements of a pilot project evaluating treatments of claudication, and the opportunities and challenges of developing and implementing a “real world” learning system. Examples in the areas of vascular disease, spine care, gastrointestinal disease, and urology.


      Learning health care systems face many operational challenges but hold great promise for discovery and implementation of more effective clinical practices.
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Surgery
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Olsen L.
        • Saunders R.S.
        • McGinnis J.M.
        Patients charting the course: Citizen engagement in the learning health system: Workshop summary.
        National Academies Press, Washington (DC)2011
        • Greene S.M.
        • Reid R.J.
        • Larson E.B.
        Implementing the learning health system: from concept to action.
        Ann Intern Med. 2012; 157: 207-210
        • Flum D.R.
        • Fisher N.
        • Thompson J.
        • Marcus-Smith M.
        • Florence M.
        • Pellegrini C.A.
        Washington state's approach to variability in surgical processes/outcomes: Surgical Clinical Outcomes Assessment Program (SCOAP).
        Surgery. 2005; 138: 821-828
        • Kwon S.
        • Florence M.
        • Grigas P.
        • Horton M.
        • Horvath K.
        • Johnson M.
        • et al.
        Creating a learning healthcare system in surgery: Washington state's surgical care and outcomes assessment program (SCOAP) at 5 years.
        Surgery. 2012; 151: 146-152
        • Selvin E.
        • Erlinger T.P.
        Prevalence of and risk factors for peripheral arterial disease in the United States results from the national health and nutrition examination survey, 1999–2000.
        Circulation. 2004; 110: 738-743
      1. Hirsch AT, Haskal ZJ, Hertzer NR, Bakal CW, Creager MA, Halperin JL, et al. ACC/AHA 2005 guidelines for the management of patients with peripheral arterial disease (lower extremity, renal, mesenteric, and abdominal aortic). J Am Coll Cardiol 2006;47:e1-192.

        • Devine E.
        • Alfonso R.
        • Devlin A.
        • Edwards T.
        • Farrokhi E.
        • Kessler L.
        • et al.
        A model for incorporating patient and stakeholder voices in a learning healthcare network: Washington State's Comparative Effectiveness Research Translation Network (CERTAIN).
        J Clin Epidemiol. 2013; 66: S122-S129
        • Weinstein J.N.
        • Tosteson T.D.
        • Lurie J.D.
        • Tosteson A.N.
        • Blood E.
        • Hanscom B.
        • et al.
        Surgical versus nonsurgical therapy for lumbar spinal stenosis.
        N Engl J Med. 2008; 358: 794-810
        • Weinstein J.N.
        • Tosteson T.D.
        • Lurie J.D.
        • Tosteson A.N.
        • Hanscom B.
        • Skinner J.S.
        • et al.
        Surgical vs nonoperative treatment for lumbar disk herniation.
        JAMA. 2006; 296: 2441-2450
        • Drake F.T.
        • Alfonso R.
        • Cuevas C.
        • Dighe M.
        • Florence M.
        • Johnson M.
        • et al.
        Enteral contrast in the computed tomography diagnosis of appendicitis: comparative effectiveness in a prospective surgical cohort.
        Ann Surg. 2013; (Oct; Epub ahead of print)
        • Drake F.T.
        • Flum D.R.
        Short- and long-term mortality after appendectomy in Sweden 1987-2006: Influence of appendectomy diagnosis, sex, age, co-morbidity, surgical method, hospital volume, and time period–a national population based cohort study.
        World J Surg. 2013; 37: 982-983
        • Arterburn D.
        • Flum D.R.
        • Westbrook E.
        • Fuller S.
        • Shea M.
        • Bock S.
        • et al.
        A population-based, shared decision making approach to recruit for a randomized trial of bariatric surgery versus lifestyle for type 2 diabetes.
        Surg Obes Relat Dis. 2013; 9: 837-844
        • Kwon S.
        • Thompson R.
        • Dellinger P.
        • Yanez D.
        • Farrohki E.
        • Flum D.R.
        Importance of perioperative glycemic control in general surgery: a report from the surgical care and outcomes assessment program.
        Ann Surg. 2013; 257: 8-14
        • Anaya D.
        • Flum D.R.
        Risk of emergency colectomy and colostomy in patients with diverticular disease.
        Arch Surg. 2005; 140: 681
        • White K.L.
        Organization and delivery of personal health services: Public policy issues.
        Milbank Mem Fund Q. 1968; 46: 225-258
      2. Holve E. Ensuring support for research and quality improvement (QI) networks: Four pillars of sustainability - an emerging framework. EGEMS (Generating Evidence & Methods to Improve Patient Outcomes) 2013;1(1):Article 5, Available from: