A Socio-technical Approach to Evaluating an Electronic Medical Record System implemented in the Public Health Services of Aguascalientes
DOI:
https://doi.org/10.17488/RMIB.41.2.3Keywords:
Information science, Information technology management, Medical informatics, Implementation science, Primary health careAbstract
The objective of this study was to explore a strategy for evaluating an Electronic Medical Record (EMR) system implemented in the public health services of Aguascalientes, Mexico. A questionnaire based on DeLone and McLean’s Model of Information Systems Success (MISS) was adapted to Spanish and applied with 62 primary care physicians working in health centers of the Instituto de Servicios de Salud del Estado de Aguascalientes (ISSEA or the State of Aguascalientes Institute for Health Services). Opportunities for improving EMR systems were also explored from the informants’ perspectives. Additionally, the relationships between MISS components were analyzed using Structural Equations Modeling (SEM). Some MISS components and particular items (service quality and overall satisfaction) presented low averages, reflecting opportunities for improving the development and implementation of EMR, such as the need to continuously update information pertaining to diagnostic and medicine catalogs and develop systems that are interoperable between the second and third levels of care. In conclusion, the present study contributes generating evidence on the use of the MISS to evaluating EMR systems in public health services of Mexico. More evidence should be generated in this field in order to promote the continuous improvement of these information systems.
Downloads
References
World Health Organization. e-Health (WHA58.28). In: Fifty-eighth World Health Assembly (WHA58/2005/REC/1) [Internet]. Geneva: WHO; 2005. p. 108–10. Available from: http://www.who.int/iris/handle/10665/20398
World Health Organization. mHealth, use of appropriate digital technologies for public health. Report by the Director-General. In: Seventy-first World Health Assembly - Provisional agenda item 124 (A71/20) [Internet]. Geneva: WHO; 2018. p. 26. Available from: http://apps.who.int/gb/e/e_wha71.html
World Health Organization. Digital Health. In: Seventy-first World Health Assembly - Agenda item 124 (A71/A/CONF/1) [Internet]. Geneva: WHO; 2018. p. 21. Available from: http://apps.who.int/gb/e/e_wha71.html
World Health Organization. WHO guideline: recommendations on digital interventions for health system strengthening [Internet]. Geneva:WHO; 2019. Available from: https://www.who.int/reproductivehealth/publications/digital-interventions-health-system-strengthening/en/
Aceto G, Persico V, Pescapé A. The role of Information and Communication Technologies in healthcare: taxonomies, perspectives, and challenges. J Netw Comput Appl [Internet]. 2018;107:125–54. Available from: https://doi.org/10.1016/j.jnca.2018.02.008
Mettler T, Raptis DA. What constitutes the field of health information systems? Fostering a systematic framework and research agenda. Health Informatics J [Internet]. 2012;18(2):147–56. Available from: https://doi.org/10.1177/1460458212452496
Barrett D, Liaw S-T, De Lusignan S. Unravelling the tangled taxonomies of health informatics. J Innov Heal Informatics [Internet]. 2014;21(3):152–5. Available from: http://dx.doi.org/10.14236/jhi.v21i3.78
Boogerd EA, Arts T, Engelen LJ, van de Belt TH. “What is eHealth”: time for an update? JMIR Res Protoc [Internet]. 2015;4(1):e29. Available from: http://dx.doi.org/10.2196/resprot.4065
Shaw T, McGregor D, Brunner M, Keep M, Janssen A, Barnet S. What is eHealth (6)? Development of a Conceptual Model for eHealth: Qualitative Study with Key Informants. J Med Internet Res [Internet]. 2017;19(10):e324. Available from: http://dx.doi.org/10.2196/jmir.8106
Elbert NJ, Van Os-Medendorp H, Van Renselaar W, Ekeland AG, Hakkaart-Van Roijen L, Raat H, et al. Effectiveness and cost-effectiveness of ehealth interventions in somatic diseases: A systematic review of systematic reviews and meta-analyses. J Med Internet Res [Internet]. 2014;16(4):e110. Available from: http://dx.doi.org/10.2196/jmir.2790
Flodgren G, Rachas A, Farmer AJ, Inzitari M, Shepperd S. Interactive telemedicine: effects on professional practice and health care outcomes. Cochrane Database Syst Rev [Internet]. 2015;(9):CD002098. Available from: http://dx.doi.org/10.1002/14651858.CD002098.pub2
Iribarren SJ, Cato K, Falzon L, Stone PW. What is the economic evidence for mHealth? A systematic review of economic evaluations of mHealth solutions. PLoS One [Internet]. 2017;12(2):e0170581. Available from: http://dx.doi.org/10.1371/journal.pone.0170581
Totten AM, Womack DM, Eden KB, McDonagh MS, Griffin JC, Grusing S, et al. Telehealth: mapping the evidence for patient outcomes from systematic reviews [Internet]. Rockville, MD: Agency for Healthcare Research and Quality; 2016. Available from: www.effectivehealthcare.ahrq.gov/reports/final.cfm
World Health Organization. Atlas of eHealth country profiles: the use of eHealth in support of universal health coverage, based on the findings of the third global survey on eHealth 2015 [Internet]. Geneva: WHO; 2016. Available from: http://www.who.int/goe/publications/atlas_2015/en/
World Health Organization, International Telecommunication Union. National eHealth Strategy Toolkit [Internet]. Geneva: WHO; 2012. Available from: https://apps.who.int/iris/handle/10665/75211
Organización Panamericana de la Salud. Estrategia y plan de acción sobre eSalud (2012-2017) [Strategy and plan of action on eHealth 2012-2017] [Internet]. 2011. Available from: http://www.paho.org/ict4health/index.php?option=com_content&view=article&id=54:estrategia-y-plan-de-accion-sobre-esalud-2012-2017&Itemid=&lang=es
Organización Panamericana de la Salud. La eSalud en la región de las américas : derribando las barreras a la implementación. Resultados de la Tercera Encuesta Global de eSalud de la Organización Mundial de la Salud [eHealth in the Americas region: breaking down the barriers to implementation. Results of the Third Global Survey on eHealth of the World Health Organization] [Internet]. Washington, D.C.: OPS; 2016. Available from: http://iris.paho.org/xmlui/handle/123456789/31287
Organization for Economic Cooperation and Development (OECD). Improving Health Sector efficiency: The Role of Information and Communication Technologies. Paris: OECD; 2010. https://doi.org/10.1787/9789264084612-en
Comisión Económica para América Latina y el Caribe (CEPAL). Salud electrónica en América Latina y el Caribe: avances y desafíos [Electronic health in Latin America and the Caribbean: progress and challenges] [Internet]. Fernández A, Oviedo E, editors. Santiago de Chile: Comisión Económica para América Latina y el Caribe (CEPAL), Naciones Unidas; 2010. Available from: https://www.cepal.org/es/publicaciones/salud-electronica-en-america-latina-y-el-caribe-avances-y-desafios
Palacios J, Flores-Roux E, García Zaballos A. Diagnóstico del sector TIC en México, Conectividad e inclusión social para la mejora de la productividad y el crecimiento económico [Diagnosis of the ICT sector in Mexico, Connectivity and social inclusion for the improvement of productivity and economic growth] [Internet]. Documento de debate IDB-DP-235. Banco Interamericano de Desarrollo (BID); 2013. Available from: https://publications.iadb.org/es/diagnostico-del-sector-tic-en-mexico-conectividad-e-inclusion-social-para-la-mejora-de-la
World Health Organization. Classification of Digital Health Interventions v 1.0, a shared language to describe the uses of digital technology for health [Internet]. Geneva: WHO; 2018. Available from: http://www.who.int/iris/handle/10665/260480
Díaz de León-Castañeda C. Electronic health (e-Health): a conceptual framework for its implementation in health services facilities. Gac Med Mex [Internet]. 2019;155(2):161–8. Available from: http://dx.doi.org/10.24875/GMM.M19000251
Glasgow RE, Phillips SM, Sanchez MA. Implementation science approaches for integrating eHealth research into practice and policy. Int J Med Inform [Internet]. 2014 Jul;83(7):e1–11. Available from: http://dx.doi.org/10.1016/j.ijmedinf.2013.07.002
Bergmo T. How to measure costs and benefits of eHealth interventions: An overview of methods and frameworks. J Med Internet Res [Internet]. 2015;17(11):e254. Available from: http://dx.doi.org/10.2196/jmir.4521
McNamee P, Murray E, Kelly MP, Bojke L, Chilcott J, Fischer A, et al. Designing and Undertaking a Health Economics Study of Digital Health Interventions. Am J Prev Med [Internet]. 2016;51(5):852–60. Available from: http://dx.doi.org/10.1016/j.amepre.2016.05.007
Yusof MM, Papazafeiropoulou A, Paul RJ, Stergioulas LK. Investigating evaluation frameworks for health information systems. Int J Med Inform [Internet]. 2008;77(6):377–85. Available from: https://doi.org/10.1016/j.ijmedinf.2007.08.004
Delone W, Mclean E. The DeLone and McLean Model of Information Systems Success: A Ten-Year Update. J Manag Inf Syst [Internet]. 2003;19(4):9–30. Available from: https://doi.org/10.1080/07421222.2003.11045748
Gómez-Dantés O, Sesma S, Becerril VM, Knaul FM, Arreola H FJ. Sistema de Salud de México [Mexico Health System]. Salud Publica Mex [Internet]. 2011;53(supl 2):S220–32. Available from: http://saludpublica.mx/index.php/spm/article/view/5043/10023
Instituto Nacional de Estadística y Geografía (INEGI). Encuesta Intercensal 2015 [Intercensal Survey 2015] [Internet]. 2015. Available from: http://www.beta.inegi.org.mx/proyectos/enchogares/especiales/intercensal/
Diario Oficial de la Federación. DECRETO por el que se reforman, adicionan y derogan diversas disposiciones de la Ley General de Salud y de la Ley de los Institutos Nacionales de Salud [DECREE by which various directions of the General Health Law and the Law of the National Institutes of Health are reformed, added and repealed] [Internet]. 2019 Nov 29; Available from: http://www.dof.gob.mx/nota_detalle.php?codigo=5580430&fecha=29/11/2019
Gobierno Federal. Estrategia Digital Nacional [National Digital Strategy] [Internet]. 2013. Available from: https://www.gob.mx/cms/uploads/attachment/file/17083/Estrategia_Digital_Nacional.pdf
Mariscal Avilés J, Gil-García JR, Ramírez-Hernández F. E-Salud en México: antecedentes, objetivos, logros y retos [E-Health in Mexico: background, objectives, achievements and challenges]. Espac Públicos [Internet]. 2012;15(34):65–94. Available from: http://www.redalyc.org/articulo.oa?id=67623463015
Gertrudiz N. e-Health: the case of Mexico [e-Health: the case of Mexico]. Lat Am J Telehealth [Internet]. 2010;2(2):127–67. Available from: http://cetes.medicina.ufmg.br/revista/index.php/rlat/article/view/71
Organización Panamericana de la Salud. Conversaciones sobre eSalud. Gestión de información, diálogos e intercambio de conocimientos para acercarnos al acceso universal a la salud [Conversations about eHealth. Information management, dialogues and knowledge exchange to get closer to universal access to health] [Internet]. Washington, DC: OPS; 2014. Available from: http://iris.paho.org/xmlui/handle/123456789/28391
González-Rétiz ML, Pacheco-López A. Desarrollo de la telesalud en México [Development of telehealth in Mexico] [Internet]. Santiago de Chile: Comisión Económica para América Latina y el Caribe (CEPAL), Naciones Unidas; 2013. Available from: http://hdl.handle.net/11362/4055
Vélez-Andrade C, Rodríguez-Hernández J. Salud Móvil: Nuevos Horizontes para la Promoción de la Salud. Proyecto m-Salud para la prevención, diagnóstico oportuno y control del sobrepeso/obesidad y diabetes tipo 2 [Mobile Health: New Horizons for Health Promotion. m-Health Project for the prevention, timely diagnosis and control of overweight / obesity and type 2 diabetes]. México City: Fundación Mexicana para la Salud (Funsalud); 2018. Available from: https://funsalud.org.mx/wp-content/uploads/2019/11/Salud-Movil_-Nuevos-Horizontes-para-la-Promocion-de-la-Salud.pdf
Cabrera NI, Castro P, Demeneghi VP, Fernández L, Morales J, Sainz L, et al. mSalUV: un nuevo sistema de mensajería móvil para el control de la diabetes en México [mSalUV: a new mobile messaging system for diabetes control in Mexico]. Rev Panam Salud Publica [Internet]. 2014;35(5–6):371–7. Available from: https://www.scielosp.org/pdf/rpsp/2014.v35n5-6/371-377
Tapia-Conyer R, Gallardo-Rincón H, Saucedo-Martinez R. CASALUD: An innovative health-care system to control and prevent non-communicable diseases in Mexico. Perspect Public Health [Internet]. 2015;135(4):180–90. Available from: http://dx.doi.org/10.1177/1757913913511423
Tapia-Conyer R, Saucedo-Martinez R, Mujica-Rosales R, Gallardo-Rincon H, Campos-Rivera PA, Lee E, et al. Enablers and inhibitors of the implementation of the Casalud Model, a Mexican innovative healthcare model for non-communicable disease prevention and control. Heal Res Policy Syst [Internet]. 2016;14(1):1–12. Available from: http://dx.doi.org/10.1186/s12961-016-0125-0
Gallardo-Rincón H, Saucedo-Martínez R, Mujica-Rosales R, Lee EM, Israel A, Torres-Beltran B, et al. Online continuing medical education as a key link for successful noncommunicable disease self‑management: The CASALUDTM Model. Diabetes, Metab Syndr Obes Targets Ther [Internet]. 2017;10:443–55. Available from: https://doi.org/10.2147/DMSO.S137891
Hernández-Ávila JE, Palacio-Mejía LS, Lara-Esqueda A, Silvestre E, Agudelo-Botero M, Diana ML, et al. Assessing the process of designing and implementing electronic health records in a statewide public health system: The case of Colima, Mexico. J Am Med Informatics Assoc [Internet]. 2013;20(2):238–44. Available from: http://dx.doi.org/10.1136/amiajnl-2012-000907
González-Block M, Lopez-Santibañez C, Aguilera Aburto N, Aldama A, Barajas M, Becerril Montekio V, et al. Evaluación y estrategias de portabilidad y convergencia hacia la integracion del Sistema Nacional de Salud, resumen ejecutivo [Evaluation and strategies of portability and convergence towards the integration of the National Health System, Executive summary] [Internet]. Cuernavaca: Instituto Nacional de Salud Pública; 2011. Available from: http://pliopencms05.salud.gob.mx:8080/BasesCDI/Archivos/Informes/portabilidad_convergencia.pdf
Fernández Morales K, McAnally Salas L, Vallejo Casarín A. Apropiación tecnológica: una visión desde los modelos y las teorías que la explican [Technological appropriation: a view from the models and theories that explain it]. Perspect Educ [Internet]. 2015;54(2):109–25. Available from: http://dx.doi.org/10.4151/07189729-Vol.54-Iss.2-Art.331
Korpelainen K. Theories of ICT system implementation and adoption - a critical review [Internet]. Helsinki: School of Science and Technology, Aalto University; 2011. Available from: https://aaltodoc.aalto.fi/handle/123456789/5019
Yu P, Qian S. Developing a theoretical model and questionnaire survey instrument to measure the success of electronic health records in residential aged care. PLoS One [Internet]. 2018;13(1):e0190749. Available from: https://doi.org/10.1371/journal.pone.0190749
Tharmalingam S, Hagens S, Zelmer J. The value of connected health information: perceptions of electronic health record users in Canada. BMC Med Inform Decis Mak [Internet]. 2016;16(1):93. Available from: http://dx.doi.org/10.1186/s12911-016-0330-3
Raymond L, Paré G, Ortiz de Guinea A, Poba-Nzaou P, Trudel M-C, Marsan J, et al. Improving performance in medical practices through the extended use of electronic medical record systems: a survey of Canadian family physicians. BMC Med Inform Decis Mak [Internet]. 2015;15(1):27. Available from: http://dx.doi.org/10.1186/s12911-015-0152-8
Bossen C, Jensen LG, Udsen FW. Evaluation of a comprehensive EHR based on the DeLone and McLean model for IS success: Approach, results, and success factors. Int J Med Inform [Internet]. 2013;82(10):940–53. Available from: http://dx.doi.org/10.1016/j.ijmedinf.2013.05.010
Messeri P, Khan S, Millery M, Campbell A, Merrill J, Shih S, et al. An Information Systems Model of the Determinants of Electronic Health Record Use. Appl Clin Inform [Internet]. 2013;4(2):185–200. Available from: http://dx.doi.org/10.4338/ACI-2013-01-RA-0005
Alharthi H, Youssef A, Radwan S, Al-Muallim S, Zainab A-T. Physician satisfaction with electronic medical records in a major Saudi Government hospital. J Taibah Univ Med Sci [Internet]. 2014;9(3):213–8. Available from: http://dx.doi.org/10.1016/j.jtumed.2014.01.004
Tzeng WS, Kuo KM, Lin HW, Chen TY. A Socio-technical assessment of the success of picture archiving and communication systems: The radiology technologist’s perspective. BMC Med Inform Decis Mak [Internet]. 2013;13(1):1. Available from: http://dx.doi.org/10.1186/1472-6947-13-109
Sicotte C, Paré G, Bini KK, Moreault M-P, Laverdure G. Virtual Organization of Hospital Medical Imaging: A User Satisfaction Survey. J Digit Imaging [Internet]. 2010;23(6):689–700. Available from: https://doi.org/10.1007/s10278-009-9220-x
Paré G, Lepanto L, Aubry D, Sicotte C. Toward a multidimensional assessment of picture archiving and communication system success. Int J Technol Assess Health Care [Internet]. 2005;21(4):471–9. Available from: https://doi.org/10.1017/S0266462305050658
Yin RK. Case Study Research: Design and Methods. 5th ed. Thousand Oaks: SAGE Publications; 2014. 282 p.
Consejo Nacional de Evaluación de la Política de Desarrollo Social (CONEVAL). Información de Pobreza y Evaluación de las Entidades Federativas [Poverty Information and Evaluation of the Federal Entities] [Internet]. 2019. Available from: https://www.coneval.org.mx/coordinacion/entidades/Paginas/inicioent.aspx
Secretaría de Salud. Portal de datos abiertos de Secretaría de Salud [Open data portal of the Ministry of Health] [Internet]. 2018. Available from: https://datos.gob.mx/busca/organization/salud
Díaz de León Castañeda C, Góngora Ortega J. eSalud en servicios de salud públicos en México: estudio de caso [E-Health in Public Health Services in Mexico: Case Study]. Región y Soc [Internet]. 2020;32:e1256. Available from: https://doi.org/10.22198/rys2020/32/1256
Canada Health Infoway. System and use assessment survey (refresh february 2015) [Internet]. 2015. Available from: https://www.infoway-inforoute.ca/en/component/edocman/resources/toolkits/change-management/national-framework/monitoring-and-evaluation/resources-and-tools/991-benefits-evaluation-survey-process-system-use-assessment-survey?Itemid=101
Lau F, Hagens S, Muttitt S. A proposed benefits evaluation framework for health information systems in Canada. Healthc Q [Internet]. 2007;10(1):113–8. Available from: https://www.longwoods.com/content/18657
Stata 25 Software. Structural Equations Modelling Reference Manual [Internet]. 2019. Available from: https://www.stata.com/bookstore/structural-equation-modeling-reference-manual/
Fundación Carlos Slim. Programas de Innovación [Innovation programs] [Internet]. 2018. Available from: https://www.salud.carlosslim.org/innovacion/
American Educational Research Association (AERA), American Psychological Association (APA), National Council on Measurement in Education (NCME). Standards for Educational and Psychological Testing. Washington, DC: AERA, APA, NCME; 2014. 230 p.
Buabbas AJ, Al-Shamali DA, Sharma P, Haidar S, Al-Shawaf H. Users’ Perspectives on a Picture Archiving and Communication System (PACS): An In-Depth Study in a Teaching Hospital in Kuwait. JMIR Med Informatics [Internet]. 2016;4(2):e21. Available from: https://doi.org/10.2196/medinform.5703
Otieno GO, Hinako T, Motohiro A, Daisuke K, Keiko N. Measuring effectiveness of electronic medical records systems: Towards building a composite index for benchmarking hospitals. Int J Med Inform [Internet]. 2008;77(10):657–69. Available from: https://doi.org/10.1017/S0266462305050658
Kuo K, Liu C, Talley PC, Pan S. Strategic Improvement for Quality and Satisfaction of Hospital Information Systems. J Healthc Eng [Internet]. 2018:1–14. Available from: https://doi.org/10.1155/2018/3689618
Cohen JF, Coleman E, Kangethe MJ. An importance-performance analysis of hospital information system attributes: A nurses’ perspective. Int J Med Inform [Internet]. 2016;86:82–90. Available from: http://dx.doi.org/10.1016/j.ijmedinf.2015.10.010
Bauer MS, Damschroder L, Hagedorn H, Smith J, Kilbourne AM. An introduction to implementation science for the non-specialist. BMC Psychol [Internet]. 2015;3(1):32. Available from: https://doi.org/10.1186/s40359-015-0089-9
Lau F, Kuziemsky C, Price M, Gardner J. A review on systematic reviews of health information system studies. J Am Med Informatics Assoc [Internet]. 2010;17(5):637–45. Available from: https://doi.org/10.1136/jamia.2010.004838
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2020 Revista Mexicana de Ingeniería Biomédica
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Upon acceptance of an article in the RMIB, corresponding authors will be asked to fulfill and sign the copyright and the journal publishing agreement, which will allow the RMIB authorization to publish this document in any media without limitations and without any cost. Authors may reuse parts of the paper in other documents and reproduce part or all of it for their personal use as long as a bibliographic reference is made to the RMIB. However written permission of the Publisher is required for resale or distribution outside the corresponding author institution and for all other derivative works, including compilations and translations.
