Open Access Open Access  Restricted Access Subscription or Fee Access

Using a deliberative forum for engaging health system and health plan leaders to prioritize research topics

Kathleen M McTigue, E Johanna Hartelius, Timothey S Anderson, Andrew P Allsup, Treva Alston, Cynthia H Chuang, Stacey Dillon, Daniel E Ford, Nivedita Gunturi, Rachel Hess, H Lester Kirchner, Sharon L Larson, Anita B Leon-Jhong, David R McCoy, Anuradha Paranjape, James R Uhrig, Anam A Waheed, Gordon R Mitchell


Background: Including stakeholders in the process and outcomes of comparative effectiveness research (CER) can help ensure that research questions are relevant and findings are communicated to individuals who need them for decision-making. Yet limited strategies are available to assist researchers with stakeholder engagement.While health system leaders’ perspectives are increasingly recognized as valuable for CER planning, their inclusion in the stakeholder pool raises challenges due to differences in culture, training, incentives, priorities and language norms.

Objective: To convene and evaluate a deliberative forum for engaging health system leaders and other stakeholders in order to shape health system research priorities for the PaTH Clinical Data Research Network, a member of the National Patient-Centered Clinical Research Network (PCORnet).

Methods: Breakout sessions and large group deliberation solicited diverse perspectives and explored benefits and challenges of different research questions. Topic reframing, narrative integration and dynamic updating techniques facilitated communication across diverse backgrounds. Participants included 29 health system and health plan leaders, clinicians, clinical researchers and patients from the network’s 6 participating health systems. Main measures were audience response system (ARS) polling on general topic preferences and survey data on measures of engagement and deliberation success.

Results: A slate of 10 specific research topics was vetted; after deliberation, the group converged to favor the characterization of high utilizers of healthcare. Audience response polling revealed opinion shifts. Participants reported high levels of satisfaction with the experience and rated it highly for markers of deliberative quality (e.g., opportunity for active participation and adequate discussion, respect for others’ opinions and awareness of different perspectives). Fifty-four percent noted their views on the issues changed. Most participants learned from the experience (93%) and agreed that the process helped them to empathize with the challenges of others (85%).

Conclusions: A deliberation forum can incorporate diverse stakeholders into CER, enabling participants to inform and learn from each other’s perspectives while shaping a person-centered research trajectory.


Comparative effectiveness research, communication, health priorities, health services research, patient-centered care, patient engagement, person-centered healthcare, stakeholder engagement

Full Text:



Slutsky, J., Sheridan, S. & Selby, J. (2014). "Getting engaged". Journal of General Internal Medicine 29 (12) 1582-1583.

Pignone, M. (2012). Challenges to implementing patient-centered research. Annals of Internal Medicine 157 (6) 450-451.

Sparud-Lundin, C., Josefsson, U., Berg, M., Hellstrom, A.L., Koinberg, I., Nolbris, M.J., Ranerup, A. & Skärsäter, I. (2013). Use of participatory design in the development of person-centred web-based support for persons with long-term illness. European Journal for Person Centered Healthcare 1 (2) 369-380.

Ammerman, A., Corbie-Smith, G., St George, D.M., Washington, C., Weathers, B. & Jackson-Christian, B. (2003). Research expectations among African American church leaders in the PRAISE! project: a randomized trial guided by community-based participatory research. American Journal of Public Health 93 (10) 1720-1727.

Jones, L. & Wells, K. (2007). Strategies for academic and clinician engagement in community-participatory partnered research. Journal of the American Medical Association 297 (4) 407-410.

Phillipson, J., Lowe, P., Proctor, A. & Ruto, E. (2012). Stakeholder engagement and knowledge exchange in environmental research. Journal of Environmental Management 95 (1) 56-65.

Longstaff, H. & Secko, D.M. (2016). Assessing the quality of a deliberative democracy mini-public event about advanced biofuel production and development in Canada. Public Understanding of Science 25 (2) 252-261.

Hall, T.E., Wilson, P. & Newman, J. (2011). Evaluating the short- and long-term effects of a modified deliberative poll on Idahoans’ attitudes and civic engagement related to energy options. Journal of Public Deliberation 7 (1) 1-30.

Haywood, B.K. & Besley, J.C. (2014). Education, outreach, and inclusive engagement: Towards integrated indicators of successful program outcomes in participatory science. Public Understanding of Science 23 (1) 92-106.

Schmittdiel, J.A., Desai, J., Schroeder, E.B., Paolino, A.R., Nichols, G.A., Lawrence, J.M., O’Connor, P.J., Ohnsorg, K.A., Newton, K.M. & Steiner, J.F. (2015). Methods for engaging stakeholders in comparative effectiveness research: a patient-centered approach to improving diabetes care. Healthcare 3 (2) 80-88.

Workman, T., Maurer, M. & Carman, K. (2013). Unresolved tensions in consumer engagement in CER: a US research perspective. Journal of Comparative Effectiveness Research 2 (2) 127-134.

Concannon, E., McHugh, S., Healy, D.A., Kavanagh, E., Burke, P., Clarke Moloney, M. & Walsh, S.R. (2014). Diagnostic accuracy of non-radiologist performed ultrasound for abdominal aortic aneurysm: systematic review and meta-analysis. International Journal of Clinical Practice 68 (9) 1122-1129.

Selby, J.V. & Slutsky, J.R. (2014). Practicing partnered research. Journal of General Internal Medicine 29 (Supplement 4) 814-816.

Deverka, P.A., Lavallee, D.C., Desai, P.J., Esmail, L.C., Ramsey, S.D., Veenstra, D.L. & Tunis, S.R. (2012). Stakeholder participation in comparative effectiveness research: defining a framework for effective engagement. Journal of Comparative Effectiveness Research 1 (2) 181-194.

Miles, A. (2017). From evidence-based to evidence-informed, from patient-focussed to person-centered - The ongoing "energetics" of health and social care discourse as we approach the Third Era of Medicine. Journal of Evaluation in Clinical Practice 23 (1) 3-4.

Reid, C. (2012). Developing a research framework to inform an evidence base for person-centered medicine: keeping the person at the centre. European Journal for Person Centered Healthcare 1 (2) 336-342.

Miles, A. & Asbridge, J.E. (2016). The chronic illness problem. The person-centered solution. European Journal for Person Centered Healthcare 4 (1) 1-5.

Sibbald, S.L., Singer, P.A., Upshur, R. & Martin, D.K. (2009). Priority setting: what constitutes success? A conceptual framework for successful priority setting. BMC Health Services Research 9, 43.

Burgess, J., Stirling, A., Clark, J., Davies, G., Eames, M., Staley, K. & Williamson, S. (2007). Deliberative mapping: a novel analytic-deliberative methodology to support contested science-policy decisions. Public Understanding of Science 16 (3) 299-322.

Lavallee, D.C., Williams,. C.J., Tambor, E.S. & Deverka, P.A. (2012). Stakeholder engagement in comparative effectiveness research: how will we measure success? Journal of Comparative Effectiveness Research 1 (5) 397-407.

Hoffman, A., Montgomery, R., Aubry, W. & Tunis, S.R. (2010). How best to engage patients, doctors, and other stakeholders in designing comparative effectiveness studies. Health Affairs 29 (10) 1834-1841.

Carman, K.L., Maurer, M., Mallery, C., Wang, G., Garfinkel, S., Richmond, J. et al. (2013). Community Forum Deliberative Methods Demonstration: Evaluating Effectiveness and Eliciting Public Views on Use of Evidence. Executive Summary. (Prepared by the American Institutes for Research Under Contract No. 290-2010-00005.). Rockville, MD: Agency for Healthcare Research and Quality.

Abelson, J., Forest, P.G., Eyles, J., Smith, P., Martin, E. & Gauvin, F.P. (2003). Deliberations about deliberative methods: issues in the design and evaluation of public participation processes. Social Science & Medicine 57 (2) 239-251.

Califf, R.M. (2014). The Patient-Centered Outcomes Research Network: a national infrastructure for comparative effectiveness research. North Carolina Medical Journal 75 (3) 204-210.

Karpowitz, C.F. & Bansbridge, J. (2005). Disagreement and concensus: The need for dynamic updating in public deliberation. Journal of Public Deliberation 1(2) Article 2.

Gutmann, A. & Thompson, D.F. (2004). Why deliberative democracy? Princeton, NJ: Princeton University Press.

Fisher, W.R. (1984). Narration as Human Communication Paradigm: The Case of Public Moral Argument. Communication Monographs 51, 1-22.

Warren, M., Beauvais, E., Yaylaci, Ş., Mackenzie, M. & Spada P. (2015). Participedia core participants' survey. Unpublished instrument. Copy in possession of authors.

Farrell, T. (1976). Knowledge, consensus and rhetorical theory. Quarterly Journal of Speech 62, 1-14.

Fishkin, J.S. (1997). The voice of the people: Public opinion and democracy. New Haven, CT: Yale University Press.

Johnson, C. & Gastil, J. (2015). Variations of institutional design for empowered deliberation. Journal of Public Deliberation 11 (1) Article 2.



  • There are currently no refbacks.