DEVELOPMENT AND VALIDATION OF AN INDEX SCORE TO ADJUST FOR HEALTHY USER BIAS IN OBSERVATIONAL STUDIES
Main Article Content
Keywords
Administrative Data Uses, Bias, Biostatistical Methods, Observational Data
Abstract
Objectives
To develop a healthy user index to serve as a method of confounding adjustment in future observational studies of preventive therapies.
Methods
A large administrative database of patients with type 2 diabetes was split in half randomly, yielding derivation and validation cohorts. Influenza vaccination was used as a ‘prototypical marker’ of a healthy user. In our derivation cohort, we fitted a mixed effects logistic regression model, and a points-based system was used to construct the index. The healthy user index was then evaluated in the validation cohort.
Results
Overall, 13% had received the influenza vaccination. In the derivation cohort ( n = 914 732), the healthy user index ranged from 0 to 91 with a mean of 41.6 (SD 12.9). When applied to the validation cohort ( n = 913 231), the index ranged from 0 to 96 (mean 41.6, SD 12.9) and significantly predicted influenza vaccination with a c-statistic of 0.649 (95% CI = 0.647-0.650).
Conclusion
Our healthy user index combined age, sex, and healthy behaviours to predict healthy users within administrative datasets. This index score may allow for better adjustment of healthy user bias in health services research; however, external validation is further required.
To develop a healthy user index to serve as a method of confounding adjustment in future observational studies of preventive therapies.
Methods
A large administrative database of patients with type 2 diabetes was split in half randomly, yielding derivation and validation cohorts. Influenza vaccination was used as a ‘prototypical marker’ of a healthy user. In our derivation cohort, we fitted a mixed effects logistic regression model, and a points-based system was used to construct the index. The healthy user index was then evaluated in the validation cohort.
Results
Overall, 13% had received the influenza vaccination. In the derivation cohort ( n = 914 732), the healthy user index ranged from 0 to 91 with a mean of 41.6 (SD 12.9). When applied to the validation cohort ( n = 913 231), the index ranged from 0 to 96 (mean 41.6, SD 12.9) and significantly predicted influenza vaccination with a c-statistic of 0.649 (95% CI = 0.647-0.650).
Conclusion
Our healthy user index combined age, sex, and healthy behaviours to predict healthy users within administrative datasets. This index score may allow for better adjustment of healthy user bias in health services research; however, external validation is further required.
References
1. Eurich DT, Majumdar SR. Statins and sepsis - Scientifically interesting but clinically inconsequential. J Gen Intern Med 2012;27:268–69.
2. Shrank WH, Patrick AR, Alan Brookhart M. Healthy user and related biases in observational studies of preventive interventions: A primer for physicians. J Gen Int Med 2011;26:546–50.
3. Brookhart MA, Patrick AR, Dormuth C, et al. Adherence to lipid-lowering therapy and the use of preventive health services: An investigation of the healthy user effect. Am J Epidemiol 2007;166:348–54.
4. Dormuth CR, Patrick AR, Shrank WH, et al. Statin adherence and risk of accidents: A cautionary tale. Circulation 2009;119:2051–57.
5. Nelson JC, Jackson ML, Weiss NS, Jackson LA. New strategies are needed to improve the accuracy of influenza vaccine effectiveness estimates among seniors. J Clin Epidemiol 2009:62:687–94.
6. Lau D, Eurich DT, Majumdar SR, Katz A, Johnson JA. Effectiveness of influenza vaccination in working-age adults with diabetes: A population-based cohort study. Thorax 2013;68:658–63.
7. Eurich DT, Marrie TJ, Johnstone J, Majumdar SR. Mortality reduction with influenza vaccine in patients with pneumonia outside “flu” season: Pleiotropic benefits or residual confounding? Am J Respir Crit Care Med 2008;178:527–33.
8. Canadian Communicable Disease Report. Statement on seasonal influenza vaccine for 2013–2014. 2013;39(ACS-4). [Public Health Agency of Canada website]. October 2013. Available at: http://www.phac-aspc.gc.ca/publicat/ccdr-rmtc/13vol39/acs-dcc-4/assets/pdf/13vol39-acs-dcc4-eng.pdf. Accessed December 9, 2016.
9. Jackson LA, Nelson JC, Benson P, et al. Functional status is a confounder of the association of influenza vaccine and risk of all cause mortality in seniors. Int J Epidemiol 2006;35:345–52.
10. McGrath LJ, Cole SR, Kshirsagar AV, Weber DJ, Stürmer T, Brookhart MA. Hospitalization and skilled nursing care are predictors of influenza vaccination among patients on hemodialysis: Evidence of confounding by frailty. Med Care 2013;51:1106–13.
11. Gleason CE, Dowling NM, Friedman E, Wharton W, Asthana S. Using predictors of hormone therapy use to model the healthy user bias: How does healthy user status influence cognitive effects of hormone therapy? Menopause 2012;19:524–33.
12. Eurich DT, Simpson S, Senthilselvan A, et al. Comparative safety and effectiveness of sitagliptin in patients with type 2 diabetes: retrospective population based cohort study. BMJ 2013;346:f2267.
13. Padwal R, Lin M, Etminan M, et al. Comparative effectiveness of olmesartan and other Angiotensin receptor blockers in diabetes mellitus: retrospective cohort study. Hypertension 2014;63(5):977–83.
14. Weir DL, McAlister FA, Senthilselvan A, et al. Sitagliptin use in patients with diabetes and heart failure: a population-based retrospective cohort study. JACC Heart Fail 2014;2(6):573–82.
15. McAlister FA, Youngson E, Eurich DT. Treatment Deintensification Is Uncommon in Adults With Type 2 Diabetes Mellitus: A Retrospective Cohort Study. Circ Cardiovasc Qual Outcome 2017;10(4) doi: 10.1161/CIRCOUTCOMES.116.003514.
16. Husein N, Woo V. Canadian diabetes association 2013 clinical practice guidelines for the prevention and management of diabetes in Canada: Pharmacologic management of type 2 diabetes. Can J Diabetes [serial online] 2013;37(suppl 1): S93.
17. American Diabetes Association. Standards of medical care in Diabetes—2014. Diabet Care 2014;37(Supplement 1): S14–S80.
18. Remschmidt C, Wichmann O, Harder T. Frequency and impact of confounding by indication and healthy vaccinee bias in observational studies assessing influenza vaccine effectiveness: a systematic review. BMC Infect Dis 2015;15:429. doi: 10.1186/s12879-015-1154-y.
19. Remschmidt C, Wichmann O, Harder T. Vaccines for the prevention of seasonal influenza in patients with diabetes: Systematic review and meta-analysis. BMC Med 2015;13:53.
20. Jackson ML, Yu O, Nelson JC, et al. Further evidence for bias in observational studies of influenza vaccine effectiveness: the 2009 influenza A(H1N1) pandemic. Am J Epidemiol 2013;178(8):1327–36. doi: 10.1093/aje/kwt124.
21. Hottes TS, Skowronski DM, Hiebert B, et al. Influenza vaccine effectiveness in the elderly based on administrative databases: change in immunization habit as a marker for bias. PLoS One 2011;6(7):e22618. doi: 10.1371/journal.pone.0022618.
22. Baxter R, Lee J, Fireman B. Evidence of bias in studies of influenza vaccine effectiveness in elderly patients. J Infect Dis 2010;201(2):186–9.
23. Jackson LA, Jackson ML, Nelson JC, et al. Evidence of bias in estimates of influenza vaccine effectiveness in seniors. Int J Epidemiol 2006;35(2):337–44.
24. Crowcroft NS, Rosella LC. The potential effect of temporary immunity as a result of bias associated with healthy users and social determinants on observations of influenza vaccine effectiveness; could unmeasured confounding explain observed links between seasonal influenza vaccine and pandemic H1N1 infection? BMC Pub Health 2012;12:458. doi: 10.1186/1471-2458-12-458.
25. Achtymichuk KA, Johnson JA, Al Sayah F, et al. Characteristics and health behaviors of diabetic patients receiving influenza vaccination. Vaccine 2015;33(30):3549–55.
26. Hak E, Hoes AW, Nordin J, et al. Benefits of influenza vaccine in US elderly--appreciating issues of confounding bias and precision. Int J Epidemiol 2006;35(3):800–2; author reply 799-800. doi: 10.1093/ije/dyl068.
27. Nichol KL. Challenges in evaluating influenza vaccine effectiveness and the mortality benefits controversy. Vaccine 2009;27(45):6305–11.
28. Schwartz KL, Jembere N, Campitelli MA, et al. Using physician billing claims from the Ontario Health Insurance Plan to determine individual influenza vaccination status: an updated validation study. CMAJ Open 2016;4(3):E463-E70. doi: 10.9778/cmajo.20160009.
29. Kwong JC, Manuel DG. Using OHIP physician billing claims to ascertain individual influenza vaccination status. Vaccine 2007;25(7):1270–4.
30. Neuzil KM, Reed GW, Mitchel EF, Jr., et al. Influenza-associated morbidity and mortality in young and middle-aged women. JAMA 1999;281(10):901–7.
31. Centers for Disease Control and Prevention. Past weekly surveillance reports. [Centers for Disease Control and Prevention web site]. Available at: http://www.cdc.gov/flu/weekly/pastreports.htm. Accessed July 9, 2016.
32. Department of Health and Human Services. Preventive services. [Centers for Medicare and Medicaid Services web site]. Available at: http://www.cms.gov/Medicare/Prevention/PrevntionGenInfo/Downloads/MPS-QuickReferenceChart-1TextOnly.pdf. Accessed October 2016 accessed October 5, 2016.
33. Canadian Chronic Disease Surveillance System Osteoporosis Working Group. Use of administrative data for national surveillance of osteoporosis and related fractures in Canada: results from a feasibility study. Arch Osteoporos 2013;8:143. doi: 10.1007/s11657-013-0143-2.
34. Quan H, Li B, Saunders LD, et al. Assessing validity of ICD-9-CM and ICD-10 administrative data in recording clinical conditions in a unique dually coded database. Health Serv Res 2008;43(4):1424–41.
35. Quan H, Parsons GA, Ghali WA. Validity of procedure codes in International Classification of Diseases, 9th revision, clinical modification administrative data. Medical Care 2004;42(8):801–9.
36. Lofters A, Vahabi M, Glazier RH. The validity of self-reported cancer screening history and the role of social disadvantage in Ontario, Canada. BMC Pub Health 2015;15:28. doi: 10.1186/s12889-015-1441-y.
37. Ho PM, Bryson CL, Rumsfeld JS. Medication adherence: its importance in cardiovascular outcomes. Circ 2009;119(23):3028–35.
38. Ismaila A, Corriveau D, Vaillancourt J, et al. Impact of adherence to treatment with tiotropium and fluticasone propionate/salmeterol in chronic obstructive pulmonary diseases patients. Curr Med ResOpin 2014;30(7):1427–36.
39. Meijer WM, Penning-van Beest FJ, Olson M, et al. Relationship between duration of compliant bisphosphonate use and the risk of osteoporotic fractures. Curr Med Res Opin 2008;24(11):3217–22.
40. Hollestein L, Baser Ö, Stricker B, et al. The healthy user and healthy adherer bias: a nested case-control study among statin users in the Rotterdam Study. Arch Pub Health 2015;73(Suppl 1): O6–O6.
41. Silverman SL, Gold DT. Healthy users, healthy adherers, and healthy behaviors? J Bone Miner Res 2011;26(4):681–2.
42. Sullivan LM, Massaro JM, D’Agostino RB, Sr. Presentation of multivariate data for clinical use: The Framingham Study risk score functions. Stat Med 2004;23(10):1631–60.
43. Austin PC, Walraven C. The mortality risk score and the ADG score: two points-based scoring systems for the Johns Hopkins aggregated diagnosis groups to predict mortality in a general adult population cohort in Ontario, Canada. Med Care 2011;49(10):940–7.
44. Charlson ME, Pompei P, Ales KL, et al. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 1987;40(5):373–83.
45. Canadian Communicable Disease Report. Canadian Community Health Survey Results Influenza Immunization Coverage. Available from: http://www.bccdc.ca/resource-gallery/Documents/Statistics and Research/Statistics and Reports/Immunization/Coverage/CCHS20070820110601.pdf. Accessed July 30, 2017.
46. Jackevicius CA, Tu JV, Ross JS, et al. Use of ezetimibe in the United States and Canada. N Engle J Med 2008;358(17):1819–28.
2. Shrank WH, Patrick AR, Alan Brookhart M. Healthy user and related biases in observational studies of preventive interventions: A primer for physicians. J Gen Int Med 2011;26:546–50.
3. Brookhart MA, Patrick AR, Dormuth C, et al. Adherence to lipid-lowering therapy and the use of preventive health services: An investigation of the healthy user effect. Am J Epidemiol 2007;166:348–54.
4. Dormuth CR, Patrick AR, Shrank WH, et al. Statin adherence and risk of accidents: A cautionary tale. Circulation 2009;119:2051–57.
5. Nelson JC, Jackson ML, Weiss NS, Jackson LA. New strategies are needed to improve the accuracy of influenza vaccine effectiveness estimates among seniors. J Clin Epidemiol 2009:62:687–94.
6. Lau D, Eurich DT, Majumdar SR, Katz A, Johnson JA. Effectiveness of influenza vaccination in working-age adults with diabetes: A population-based cohort study. Thorax 2013;68:658–63.
7. Eurich DT, Marrie TJ, Johnstone J, Majumdar SR. Mortality reduction with influenza vaccine in patients with pneumonia outside “flu” season: Pleiotropic benefits or residual confounding? Am J Respir Crit Care Med 2008;178:527–33.
8. Canadian Communicable Disease Report. Statement on seasonal influenza vaccine for 2013–2014. 2013;39(ACS-4). [Public Health Agency of Canada website]. October 2013. Available at: http://www.phac-aspc.gc.ca/publicat/ccdr-rmtc/13vol39/acs-dcc-4/assets/pdf/13vol39-acs-dcc4-eng.pdf. Accessed December 9, 2016.
9. Jackson LA, Nelson JC, Benson P, et al. Functional status is a confounder of the association of influenza vaccine and risk of all cause mortality in seniors. Int J Epidemiol 2006;35:345–52.
10. McGrath LJ, Cole SR, Kshirsagar AV, Weber DJ, Stürmer T, Brookhart MA. Hospitalization and skilled nursing care are predictors of influenza vaccination among patients on hemodialysis: Evidence of confounding by frailty. Med Care 2013;51:1106–13.
11. Gleason CE, Dowling NM, Friedman E, Wharton W, Asthana S. Using predictors of hormone therapy use to model the healthy user bias: How does healthy user status influence cognitive effects of hormone therapy? Menopause 2012;19:524–33.
12. Eurich DT, Simpson S, Senthilselvan A, et al. Comparative safety and effectiveness of sitagliptin in patients with type 2 diabetes: retrospective population based cohort study. BMJ 2013;346:f2267.
13. Padwal R, Lin M, Etminan M, et al. Comparative effectiveness of olmesartan and other Angiotensin receptor blockers in diabetes mellitus: retrospective cohort study. Hypertension 2014;63(5):977–83.
14. Weir DL, McAlister FA, Senthilselvan A, et al. Sitagliptin use in patients with diabetes and heart failure: a population-based retrospective cohort study. JACC Heart Fail 2014;2(6):573–82.
15. McAlister FA, Youngson E, Eurich DT. Treatment Deintensification Is Uncommon in Adults With Type 2 Diabetes Mellitus: A Retrospective Cohort Study. Circ Cardiovasc Qual Outcome 2017;10(4) doi: 10.1161/CIRCOUTCOMES.116.003514.
16. Husein N, Woo V. Canadian diabetes association 2013 clinical practice guidelines for the prevention and management of diabetes in Canada: Pharmacologic management of type 2 diabetes. Can J Diabetes [serial online] 2013;37(suppl 1): S93.
17. American Diabetes Association. Standards of medical care in Diabetes—2014. Diabet Care 2014;37(Supplement 1): S14–S80.
18. Remschmidt C, Wichmann O, Harder T. Frequency and impact of confounding by indication and healthy vaccinee bias in observational studies assessing influenza vaccine effectiveness: a systematic review. BMC Infect Dis 2015;15:429. doi: 10.1186/s12879-015-1154-y.
19. Remschmidt C, Wichmann O, Harder T. Vaccines for the prevention of seasonal influenza in patients with diabetes: Systematic review and meta-analysis. BMC Med 2015;13:53.
20. Jackson ML, Yu O, Nelson JC, et al. Further evidence for bias in observational studies of influenza vaccine effectiveness: the 2009 influenza A(H1N1) pandemic. Am J Epidemiol 2013;178(8):1327–36. doi: 10.1093/aje/kwt124.
21. Hottes TS, Skowronski DM, Hiebert B, et al. Influenza vaccine effectiveness in the elderly based on administrative databases: change in immunization habit as a marker for bias. PLoS One 2011;6(7):e22618. doi: 10.1371/journal.pone.0022618.
22. Baxter R, Lee J, Fireman B. Evidence of bias in studies of influenza vaccine effectiveness in elderly patients. J Infect Dis 2010;201(2):186–9.
23. Jackson LA, Jackson ML, Nelson JC, et al. Evidence of bias in estimates of influenza vaccine effectiveness in seniors. Int J Epidemiol 2006;35(2):337–44.
24. Crowcroft NS, Rosella LC. The potential effect of temporary immunity as a result of bias associated with healthy users and social determinants on observations of influenza vaccine effectiveness; could unmeasured confounding explain observed links between seasonal influenza vaccine and pandemic H1N1 infection? BMC Pub Health 2012;12:458. doi: 10.1186/1471-2458-12-458.
25. Achtymichuk KA, Johnson JA, Al Sayah F, et al. Characteristics and health behaviors of diabetic patients receiving influenza vaccination. Vaccine 2015;33(30):3549–55.
26. Hak E, Hoes AW, Nordin J, et al. Benefits of influenza vaccine in US elderly--appreciating issues of confounding bias and precision. Int J Epidemiol 2006;35(3):800–2; author reply 799-800. doi: 10.1093/ije/dyl068.
27. Nichol KL. Challenges in evaluating influenza vaccine effectiveness and the mortality benefits controversy. Vaccine 2009;27(45):6305–11.
28. Schwartz KL, Jembere N, Campitelli MA, et al. Using physician billing claims from the Ontario Health Insurance Plan to determine individual influenza vaccination status: an updated validation study. CMAJ Open 2016;4(3):E463-E70. doi: 10.9778/cmajo.20160009.
29. Kwong JC, Manuel DG. Using OHIP physician billing claims to ascertain individual influenza vaccination status. Vaccine 2007;25(7):1270–4.
30. Neuzil KM, Reed GW, Mitchel EF, Jr., et al. Influenza-associated morbidity and mortality in young and middle-aged women. JAMA 1999;281(10):901–7.
31. Centers for Disease Control and Prevention. Past weekly surveillance reports. [Centers for Disease Control and Prevention web site]. Available at: http://www.cdc.gov/flu/weekly/pastreports.htm. Accessed July 9, 2016.
32. Department of Health and Human Services. Preventive services. [Centers for Medicare and Medicaid Services web site]. Available at: http://www.cms.gov/Medicare/Prevention/PrevntionGenInfo/Downloads/MPS-QuickReferenceChart-1TextOnly.pdf. Accessed October 2016 accessed October 5, 2016.
33. Canadian Chronic Disease Surveillance System Osteoporosis Working Group. Use of administrative data for national surveillance of osteoporosis and related fractures in Canada: results from a feasibility study. Arch Osteoporos 2013;8:143. doi: 10.1007/s11657-013-0143-2.
34. Quan H, Li B, Saunders LD, et al. Assessing validity of ICD-9-CM and ICD-10 administrative data in recording clinical conditions in a unique dually coded database. Health Serv Res 2008;43(4):1424–41.
35. Quan H, Parsons GA, Ghali WA. Validity of procedure codes in International Classification of Diseases, 9th revision, clinical modification administrative data. Medical Care 2004;42(8):801–9.
36. Lofters A, Vahabi M, Glazier RH. The validity of self-reported cancer screening history and the role of social disadvantage in Ontario, Canada. BMC Pub Health 2015;15:28. doi: 10.1186/s12889-015-1441-y.
37. Ho PM, Bryson CL, Rumsfeld JS. Medication adherence: its importance in cardiovascular outcomes. Circ 2009;119(23):3028–35.
38. Ismaila A, Corriveau D, Vaillancourt J, et al. Impact of adherence to treatment with tiotropium and fluticasone propionate/salmeterol in chronic obstructive pulmonary diseases patients. Curr Med ResOpin 2014;30(7):1427–36.
39. Meijer WM, Penning-van Beest FJ, Olson M, et al. Relationship between duration of compliant bisphosphonate use and the risk of osteoporotic fractures. Curr Med Res Opin 2008;24(11):3217–22.
40. Hollestein L, Baser Ö, Stricker B, et al. The healthy user and healthy adherer bias: a nested case-control study among statin users in the Rotterdam Study. Arch Pub Health 2015;73(Suppl 1): O6–O6.
41. Silverman SL, Gold DT. Healthy users, healthy adherers, and healthy behaviors? J Bone Miner Res 2011;26(4):681–2.
42. Sullivan LM, Massaro JM, D’Agostino RB, Sr. Presentation of multivariate data for clinical use: The Framingham Study risk score functions. Stat Med 2004;23(10):1631–60.
43. Austin PC, Walraven C. The mortality risk score and the ADG score: two points-based scoring systems for the Johns Hopkins aggregated diagnosis groups to predict mortality in a general adult population cohort in Ontario, Canada. Med Care 2011;49(10):940–7.
44. Charlson ME, Pompei P, Ales KL, et al. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 1987;40(5):373–83.
45. Canadian Communicable Disease Report. Canadian Community Health Survey Results Influenza Immunization Coverage. Available from: http://www.bccdc.ca/resource-gallery/Documents/Statistics and Research/Statistics and Reports/Immunization/Coverage/CCHS20070820110601.pdf. Accessed July 30, 2017.
46. Jackevicius CA, Tu JV, Ross JS, et al. Use of ezetimibe in the United States and Canada. N Engle J Med 2008;358(17):1819–28.
Article Sidebar
Published
Nov 22, 2017
Article Details
How to Cite
Achtymichuk, K. A., Johnson, J. A., Minhas-Sandhu, J. K., Lin, M., & Eurich, D. T. (2017). DEVELOPMENT AND VALIDATION OF AN INDEX SCORE TO ADJUST FOR HEALTHY USER BIAS IN OBSERVATIONAL STUDIES. Journal of Population Therapeutics and Clinical Pharmacology, 24(3), e79-e89. https://doi.org/10.22374/1710-6222.24.3.6
Section
Articles
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
All articles published in JPTCP are licensed under Copyright Creative Commons Attribution-NonCommercial 4.0 International License.