Strategies for automating pharmacovigilance adverse event case processing

Mythily Easwar a, *, Kenneth Gossett b, Manish Shashi c

a School of Management, Walden University, Minneapolis, MN, USA

b Contributing Faculty Member at Walden University, Minneapolis, MN, USA

c School of Management, Walden University, Minneapolis, MN, USA

 

A R T I C L E  I N F O  

A B S T R A C T  

Received 23 September 2021;

Revised 16 October 2021;

Accepted 17 November 2021.

Introduction: In 2020, the U.S. Food & Drug Administration received reports of 2.2 million adverse events, which is a 350% increase since 2010 with an associated increase in pharmaceutical companies' pharmacovigilance (PV) operational cost. Guided by the task technology fit model as the conceptual framework, the purpose of this qualitative single case study was to explore strategies used by PV systems leaders to implement innovative technology solutions. The objective of the study was to develop a system model to explore strategies to implement innovative technology to automate AE detection, processing, and reporting.

Methods: Data were collected using semistructured interviews and company data to address the research question on strategies to implement innovative technology solutions to automate adverse events case processing. The collected data was analyzed using 5-step data analysis, compiling, disassembling, reassembling, interpreting data, and concluding the findings.

Results: The implications for positive social change include the potential to identify strategies in making pharmaceutical medicines safer for human consumption.

Discussion: PV leaders can redirect the savings from PV operations in terms of cost and workforce tasks towards investing in the actual PV tasks like benefit-risk assessments of products; thereby, improving patient outcomes and making the products more efficacious and safer for human use.

Keywords:

Adverse Events, Negative Feedback System, Qualitative Case Study, Pharmacovigilance, Task Technology Fit Model, Preventable Medical Errors, and System’s Theory.

An official publication of Global Pharmacovigilance Society This is an open-access

article under the CC BY-NC-ND license. COPYRIGHT ©2021 Author(s)

 


Introduction

The goal of pharmacovigilance (PV) is to protect patients from unnecessary harm. When medicine is marketed, the identification and monitoring of adverse events (AEs) and reactions is a part of a post-marketing safety surveillance program as required by federal regulations (Kothari, Shah, & Patel, 2018; Tan et al., 2016). Health regulatory agencies have regulations to ensure that drug products are safe for human consumption. Companies that are licensed marketing authorization holders are required by regulation to perform active drug safety surveillance and monitor the safety profiles of the products during the entire marketing lifecycle (Basile, Yahi, & Tatonetti, 2019). Companies use safety databases and reporting software to process and report AEs to health authorities around the world (Lewis & McCullum, 2019). Companies can use innovative information technology (IT) solutions based on artificial intelligence and robotics to detect, process, and report AEs, thereby adopting automation (Beninger, 2018; Donzanti, 2018). Companies can adopt an automation-enabled digital PV path for drug safety surveillance and redirect the savings from PV operations towards investing in the actual PV tasks like benefit-risk assessments of products, thereby making the products more efficacious and safer for human use.

Problem Statement

In 2020, the U.S. Food & Drug Administration (2021) received reports of 2.2 million adverse events (AEs), which was a 350 % increase since 2010, with an associated increase in pharmaceutical companies' AE-related operational costs.

AEs are one of the top three reasons for drug recalls and can cost pharmaceutical companies $900 million for a recall (Hall, Stewart, Chang, & Freeman, 2016). The general business problem is that PV systems managers may experience high operational costs for AE detection and reporting leading to poor financial performance. The specific business problem is that some PV systems managers in the pharmaceutical industry lack the strategies to implement innovative technology solutions to automate AE case processing (Easwar, 2020).

Methods

The purpose of this qualitative single case study was to explore the strategies that PV systems managers in the pharmaceutical industry use to implement innovative technology solutions to automate AE case processing (Easwar, 2020). Since not many company managers have implemented innovative technology like artificial intelligence (AI) and robotics-based solutions for PV (Beninger & Ibara, 2016), we have focused on a select group of managers who have successfully used strategies to implement innovative technology solutions to automate AEs case processing. Therefore, because of the need to use qualitative data collection procedures like interviews and documentation reviews, a qualitative method was more appropriate than quantitative or mixed methods.

We used a purposive sampling technique to select four PV systems managers working in a pharmaceutical company located in the Boston area of Massachusetts, United States, who have used successful strategies to implement innovative automation technology solutions for AEs case processing.

We collected data from semistructured interviews, conducted using an interview protocol until data saturation was reached, and triangulated the data with documents collected from the participants. We used the TTF model as the conceptual framework to guide our research on identifying the strategies PV systems managers use for automating PV case processing.

We used ATLAS.ti software to organize and analyze the data collected from the interviews. We manually analyzed the documents collected from the participants for document review. For initial analysis, we used word cloud for a visual representation of word frequency to get the first look and summarize the interview transcripts. The words were then mapped to the themes and strategies that were identified during the detailed analysis.

Research Question

The primary research question for this study was: What strategies do PV systems managers use to implement innovative technology solutions to automate AEs case processing?

Results

Three themes with nine strategies emerged during data analysis. The themes were (a) automation solution selection and implementation strategies, (b) business operational model changes, and (c) communication and training strategies. On initial analysis, the word cloud showed system, project, vendor, automation, implementation, communication, change, issues, and users as the key and most frequently used words by participants in the interviews as shown in Figure 1.

Fig. 1: Word cloud

Word cloud showing the most frequently used words in the interview data. Created from ATLAS.ti.

The organization had contracted an external vendor to build a strategic long-term partnership for their PV database, which was their foremost selection and implementation strategy. On detailed analysis, three main themes and nine key strategies emerged as shown in Table 1 below. The references coded refers to the number of times the data references were coded to the strategies.


 

 

Table 1: Themes and Strategies

Themes

 

Strategies

 

References coded

Selection and implementation strategies

Internal drivers like vision, objectives, timelines, cost, and compliance

18

Ensure solution is fit for purpose

18

Strong project management with a defined plan, dedicated resources, and stakeholders

16

Strategic partnership with an external vendor

14

Defined metrics to measure the effectiveness of the solution and project

7

Business operational model changes

Simplified, standardized, and efficient business processes by supplementing and reducing the operational workforce

15

 

Changes to organizational structure and job profiles

8

Communication and training strategies

Open communication, collaboration, and engagement with stakeholders incl. end-users and vendor

17

 

Training

5

 


Theme 1: Selection and Implementation Strategies

Automation solution selection and implementation strategies were some of the key themes that emerged from the interview data. Initial analysis of the word cloud created by ATLAS.ti included vendor, system, project, time, implementation, automation, and communication as the keywords. The word cloud reflected the strategies identified in the analysis of data. The leaders of the PV organization decided to strategically engage with an external vendor to supply the automation solution and implement the same. This organization adopted the leader strategy for automation. The following five key strategies emerged for selection and implementation strategies from the interview data:

Internal drivers like vision, objectives, timelines, cost, and compliance. All participants identified internal drivers as one of the important strategies to guide the selection and implementation of automation solutions for AEs case processing. The participants identified multiple internal drivers or factors like organization vision and objectives for automation, tight timelines, cost, and competing priorities while ensuring regulatory compliance. The PV organization created a long-term vision to leverage automation as much as possible to build efficiencies within PV business operations.

Ensure the solution is fit for purpose. All participants unanimously identified the fit of the automation solution to effectively perform as expected and match the tasks and constraints as a key strategy. Certain tasks which were performed manually were now replaced by a tactical automated solution. Therefore, the solution must be a fit for the tasks that needed to be performed, only more efficiently.

Strong project management with a defined plan, dedicated resources, and stakeholders. The project team consisted of internal employees and external vendor team members. To successfully execute the project, a strong project management function was led by an internal temporarily contracted project manager to track the project and manage stakeholder engagement. To ensure the milestones and deliverables were met as defined in the project plan, a structured project plan was built, and dedicated resources and stakeholders were identified. All participants indicated strong project management and leadership as a key strategy.

Strategic partnership with an external vendor. Since the automation solution was sourced and implemented by an external vendor, the PV systems managers decided to strike a strategic partnership with the vendor selected after a rigorous vetting process. Participants shared the realization that being a small pharmaceutical company, they did not have the relevant expertise in-house. Therefore, PV management decided to build a strategic partnership with an external vendor who can source the automation solution and implement it. In addition, the participants stressed partnering with a vendor with a vision, who was invested in the technology and the solution for the long term. Participants summarized that a good partnership ensured the vendor was engaged and acted on any issue escalations.

Defined metrics to measure the effectiveness of the solution and the project. The PV leaders built multiple metrics to measure the effectiveness of both the automation solution and the execution of the project. To ensure the solution was fit for purpose, before and after metrics measuring the time and resources were built. All participants indicated that the metrics measured the amount of time taken to create and process AEs before and after the automation solution was implemented. To ensure the project matched the internal drivers, the project was closely monitored to ensure all the milestones were achieved.

Documents provided by the participants showed automation, business process improvements, elimination of nonvalue-added activities, and metrics were the key internal strategies of focus. The documents also indicated strong leadership, open communication, and collaboration as key strategies for successfully implementing automation solutions for AEs case processing. Technical limitations like multi-tenant cloud implementation and adopting ISP configuration were included as internal drivers. The document review revealed the timeline as an internal driver, as mentioned by all the participants in their interviews.

The findings matched the academic literature review. PV managers must create a vision and project new technology as a product designed to create, attract, and satisfy demand (Gherasim, 2011). Padgett, Gossett, Mayer, Chien, and Turner (2017) found that as an organization developed a culture of safety, the staff gained skills required to handle dynamic patient conditions within the facility, reducing their preventable medical errors and dependence on external healthcare and emergency medical services. Though many companies automate many steps in the PV process, plenty of opportunities exist to make PV agile and efficient (Ghosh et al., 2020). Organization leaders have realized that proactive external collaboration is beneficial for the creation of new products and solutions (Di Fiore et al., 2017). Therefore, the PV leadership decided to partner with an external vendor.

PV managers must ensure the technology solutions are aligned with the business needs and led by business operations (Mishra et al., 2019). Managers must improve employees’ job satisfaction and solution adoption by implementing IT solutions that fit the tasks and employees’ long-term professional needs (Wang et al., 2020). Internal drivers of a company, specifically, organizational culture and environmental issues must be considered to leverage the advantages of data analytics (Hawley, 2016). Automation can support the human workforce in managing the increasing AE volume, complexity, reporting timelines, and increasing efficiency, consistency, and compliance (Routray et al., 2020). Companies typically measure the benefits of automation in terms of quality, compliance, and efficiency (Ghosh et al., 2020).

The TTF model links the task requirements, individual abilities, and the functionality of technology (Goodhue & Thompson, 1995). TTF theory focuses on the application and the general reliance on technologies and does not recommend any task or technology pairings that could produce a strong effect (Howard & Rose, 2019). The participants explained that the manual redundant tasks were identified first and then the automation solution that matched the tasks and the constraints were selected to ensure that the automation initiative was a success. Therefore, the TTF conceptual framework provided the correct lens for exploring strategies used by the PV systems managers to successfully automate AEs case processing.

Theme 2: Business Operational Model Changes

The second key theme that emerged from the data was business operational model changes and referred to the outcome of implementing the automation solution. Initial analysis of the word cloud created by ATLAS.ti included AEs also called cases, system, automation, change, implementation, and automatically as the keywords. Business operational model changes theme included the following two strategies:

Simplified, standardized, and efficient business processes by supplementing and reducing the operational workforce. All participants stated that the introduction of the automation solution simplified and standardized the business processes and assisted in building efficiencies, as intended. Although reduction of the workforce was not the primary intention, the managers were able to successfully supplement the manual, redundant tasks like double data entry to free up and repurpose the workforce to focus on more value-added tasks. The PV operations team could reduce the number of full-time employees.

Changes to organizational structure and job profiles. The introduction of automation solutions led to changes in the way the AEs case processing employees performed their jobs. Participants indicated that the AE case processor now only has to read the data on the screen as opposed to having to type or enter it manually. The mailbox that had to be monitored for new AEs need not be monitored because the AEs were automatically created in the database and the task was eliminated.

A review of the documents provided by the participants showed process improvements and elimination of nonvalue-added activities as key objectives for automation. One of the challenges that the PV organization targeted to resolve was manual, time-consuming, and inefficient processes. Ever-increasing AE case volume, derivative operational cost, and resources were the other key challenges that the PV leadership decided to mitigate.

The findings for business operational model changes matched previous studies in this regard. Stergiopoulos et al. (2019) noted that all PV leaders believed implementing AI-based solutions will decrease costs for PV operations including simplifying AE detection and processing. Mishra et al. (2019) summarized that business process simplification, standardization, and re-engineering are the critical factors for a successful automation strategy. Routray et al. (2020) confirmed that human intervention and review of AE seriousness will be required and that automation can augment human review by increasing efficiencies and consistency, thereby enabling reporting compliance. Beninger (2018) emphasized the new skillsets, which will define the roles of the next generation of professionals who perform PV activities. Tomita (2019) added that organizations must expect to supplement the human workforce with expert systems.

The TTF model matches the tasks as the events occur with the ability of a technology to address these events. TTF focuses on the application of technology (Howard & Rose, 2019). The results of this study were in line with those of Wang et al. (2020) and indicated that the automation technology implementation resulted in business process re-engineering. The results also indicated that if the automation solution fits the business processes to complete the tasks and enables employees' fulfillment, then the managers can expect enhanced adoption of the technologies from the employees and a favorable attitude towards their jobs.

Theme 3: Communication and Training Strategies

The third key theme that emerged from the data was communication and training strategies also included change management-related strategies. Initial analysis of the word cloud created by ATLAS.ti included communication, vendor, training, project, team, and management as the keywords. Effective communication and training strategies played a key role in successful change management and adoption of the innovative technology solution within the PV organization. Communication and training strategies theme included the following two strategies:

Open communication, collaboration, and engagement with stakeholders including end-users and the vendor. Considering the high priority and ranking of the AE management automation project, the project team maintained open communication and collaboration channels to ensure all the relevant stakeholders stayed engaged throughout the implementation. All participants indicated that communication was key and that introducing automation was viewed as a welcome change for the AE processing operational activities. The end-users were made an integral part of the project, which made change management effortless. Managing the vendor using open communication and collaboration was critical. Multiple touchpoints like steering committee and core team meetings were built to ensure the information is flowing through the organization and timely escalation and resolution of issues. Collaborating externally with other companies to explore the opportunities for automation, which can be practically implemented.

Training. Since the introduction of automation changed the way certain business processes were managed, training was a critical aspect. The participants noted that multiple cycles of training were provided to the end-users and the impacted third-party call center vendor. The end-users were engaged in the implementation project and were trained multiple times.

A review of the documents shared by the participants revealed that open communication and collaboration were key to setting up an optimum governance structure during implementation. Lack of efficient collaboration tools was a key challenge in the initial stages of the project. Considering varied stakeholders, both internal and external, and the vendor implementation team located remotely added to the communication challenges. The team used online collaboration tools like Google and Smart Sheets to help track tasks and keep the teams aligned.

Findings regarding communication and training strategies matched previous studies. Training and re-learning initiatives assist in preparing and guiding the in-house staff for digital transformation (Mishra et al., 2019). The human and technology relationship should also include user training, interaction, and collaboration sub-goals to be effective (Oyekan et al., 2017). PV systems managers realized the importance of communication and collaboration early during the implementation project and ensured adequate measures were included to create open communication channels assisting in timely escalations and change management. (Ghosh et al. 2020) also recommended collaboration across the pharmaceutical industry between regulators and companies to build a generic process flow to fuel third-party innovations.

In the TTF model, (Goodhue and Thompson, 1995) linked the technology solution with individual performance. (Wang et al., 2020) used the user task technology fit model to summarize that the technology solutions alter business processes and existing work patterns leading to re-training and re-learning, which could increase employees’ work-related stress. The study results showed that engaging the users early during the technology implementation led to easier change management and enhanced adoption of the solution and possibly job satisfaction since the solution was fit for purpose.

Significance of the Study

Applications to Professional Practice

Processing of an AE is an expensive resource-intensive process, which includes a high risk of errors and operational inefficiencies (Ghosh et al., 2020). A detailed review of the literature concluded that innovative technology such as data mining, machine learning, AI, and robotics could result in new operating models for PV (Beninger & Ibara, 2016; Lai, 2017), improve efficiencies and reduce PV operations costs (Karimi et al., 2015; Lu, 2009; Schmider et al., 2018), better allocation of resources, and improve patient outcomes (Lewis & McCallum, 2019). In this doctoral study, we explored the strategies that PV systems managers in the pharmaceutical industry use to implement innovative technology solutions to automate AEs case processing (see System’s model in Fig 2)


 

Timeline

Description automatically generated with low confidence

Fig 2:  System’s Model


The single case that we selected to study was the implementation of an innovative technology solution to automate AE processing. The organization had a larger vision to automate PV business operations and this project was a part of that vision. Leaders and managers in the PV function decided to partner with a vendor to source and implement the solution, having realized that the organization lacked the necessary expertise on the technology. Multiple internal drivers such as objectives, timelines, cost, and compliance shaped the strategies the PV systems managers used to execute the project. Considering this being their first automation solution, the team ensured that the solution is fit for purpose, both task (functionality) and technology (architecture) perspectives. Strong project leadership and steering committee were established for robust governance and monitoring.

The managers established an open communication, collaboration, and training model to engage with the relevant stakeholders across multiple functions within the organization and the vendor. Measures were established to capture metrics in terms of volume, time, and quality of the business AE processing tasks before and after automation. Regular project governance and monitoring measures ensured the automation solution was implemented on time. The metrics defined the successful outcome of this implementation.

The results showed that implementing the automation solution resulted in favorable changes to the PV business operational model. Business processes are now more simplified, standardized, and efficient. The metrics revealed major savings in terms of time and effort, which led to a reduction in the operational workforce and better allocation of resources. The organization has started reallocating the savings from manual labor towards more value-added tasks. By successfully implementing the automation solution, the PV organization managers have started defining a sustainable model for PV business operations. Other pharmaceutical companies can use the strategies identified in this study to realize the benefits of implementing innovative automation technology solutions to build a resilient and sustainable PV business operations organization.  The themes and strategies and internal processes are described in the following model as an outcome of this study.

Implications for Social Change

The successful strategies identified in this study can be used by organizations to adopt a digitized PV future. The study findings show a positive outcome and realization of savings from PV business operations because of strategically implementing and embedding automation. PV leaders can redirect the savings from PV operations towards investing in the actual PV tasks like benefit-risk assessments of products, thereby improving patient outcomes and making the products more efficacious and safer for human use.

The challenges from the pharmaceutical industry's perspective include the management of the high volume of AEs, globally diverse and changing regulations, complex and evolving systems, and sparse PV expertise and talent (Price, 2018). Automation technologies can detect, database, and report AEs quicker, and manage the high volume of AEs, which can help in protecting patients from unnecessary harm. The implications for positive social change include the identification of strategies to assist in making pharmaceutical medicines safer for human consumption. The savings from PV operations could also be used to help or assist with community projects that could bring about social change. Companies can also use the savings to offer scholarships to students pursuing courses and majoring in PV, thereby contributing to increasing PV expertise and talent.

Processing of an AE is an expensive resource-intensive process, which includes a high risk of errors and operational inefficiencies (Ghosh et al., 2020). As deducted in multiple previous research, technology can accelerate efficiencies and can be used to identify and process AEs faster and contribute to a reduction of costs (Beninger, 2018; Donzanti, 2018; Karimi et al., 2015; Kusch et al., 2020; Lewis & McCullum, 2019; Schmider et al., 2018; Stergiopoulos et al., 2019). Organizations must internalize digitalization and incorporate it in the operations methodology and structure to select the most suitable technological solutions and reap the maximum benefits (Buyukozkan & Gocer, 2018). Many pharmaceutical organizations are experimenting, and some are implementing automation technology solutions for AEs case processing.

Conclusions

This study focused on one such company whose PV organization managers have implemented an automation solution for AEs case processing. The results of the study are in line with previous research. By prudently executing automation solution selection and implementation strategies and establishing communication and training strategies, the organization is now benefitting from simplified and efficient AEs case processing business operations. Multiple factors such as internal drivers like cost and timelines, the functionality of the automation solution, strong project leadership, stakeholder partnerships, and engagement, including the size of the organization play an important role in shaping the outcome of the digitalization initiative. With careful planning and execution, intelligent AEs case processing automation initiatives can be a success and be sustained over the long term.

Acknowledgment

We want to acknowledge the contribution that Dr. M. S. Hammoud (Walden University) made in the development of the model for this study.

Conflict of Interest

No conflict of interest.

References

Basile, A. O., Yahi, A., & Tatonetti, N. P. (2019). Artificial intelligence for drug toxicity and safety. Trends in Pharmacological Sciences, 40, 624-635.

Beninger, P. (2018). Pharmacovigilance: An overview. Clinical Therapeutics, 40, 1991-2004.

Beninger, P., & Ibara, M. A. (2016). Pharmacovigilance and biomedical informatics: A model for future development. Clinical Therapeutics, 38, 2514-2525.

Buyukozkan, G., & Gocer, F. (2018). Digital supply chain: Literature review and a proposed framework for future research. Computers in Industry, 97, 157-177.

Di Fiore, A., Vetter, J., & Capur, D. R. (2017, Winter). Collaborative innovation: How to avoid the four traps. Rotman Management, 41-45. Retrieved from https://hbsp.harvard.edu/tu/b280eb86

Donzanti, B. A. (2018). Pharmacovigilance is everyone’s concern: Let’s work it out together. Clinical Therapeutics, 40, 1967-1972.

Easwar, M. (2020). Strategies for automating pharmacovigilance adverse event case processing (Unpublished doctoral dissertation). Retrieved from ProQuest Dissertations & Thesis Global (Order No. 28087464)

Gherasim, A. (2011). The creation of partnerships - essential objective in industrial marketing. Economy Transdisciplinarity Cognition, 14(1), 391-401. Retrieved from http://etc.ugb.ro/home

Ghosh, R., Kempf, D., Pufko, A., Barrios Martinez, L. F., Davis, C. M., & Sethi, S. (2020). Automation opportunities in pharmacovigilance: An industry survey. Pharmaceutical Medicine, 34, 7-18.

Goodhue, D. L., & Thompson, R. L. (1995). Task-technology fit and individual performance. MIS Quarterly, 19, 213-236.

Hall, K., Stewart, T., Chang, J., & Freeman, M. K. (2016). Characteristics of FDA drug recalls: A 30-month analysis. American Journal of Health-System Pharmacy, 73, 235-240.

Hawley, D. (2016). Implementing business analytics within the supply chain: Success and fault factors. The Electronic Journal of Information Systems Evaluation, 19, 112-121. Retrieved from www.ejise.com

Howard, M. C., & Rose, J. C. (2019). Refining and extending task–technology fit theory: Creation of two task–technology fit scales and empirical clarification of the construct. Information & Management, 56(6), 103134.

Karimi, S., Wang, C., Metke-Jimenez, A., Gaire, R., & Paris, C. (2015). Text and data mining techniques in adverse drug reaction detection. ACM Computing Surveys, 47(4), 1-39.

Kothari, C. S., Shah, M. P., & Patel, R. M. (2018). Introductory chapter: Pharmacovigilance. In C. S. Kothari, M. Shah, & R. M. Patel (Eds.), Pharmacovigilance [E-book].

Lai, P. C. (2017). The literature review of technology adoption models and theories for the novelty technology. Journal of Information Systems and Technology Management, 14, 21-38.

Lewis, D.J., & McCallum, J.F. (2019). Utilizing advanced technologies to augment pharmacovigilance systems: Challenges and opportunities. Therapeutic Innovation & Regulatory Science. Advance online publication.

Lu, Z. (2009). Information technology in pharmacovigilance: Benefits, challenges, and future directions from industry perspectives. Drug, Healthcare and Patient Safety, 1, 35-45.

Mishra, S., Devi, S. K. K., & Narayanan, B. M. K. (2019). People & process dimensions of automation in business process management industry. (2019). International Journal of Engineering and Advanced Technology, 8(6), 2465-2472.

Oyekan, J., Prabhu, V., Tiwari, A., Baskaran, V., Burgess, M., & Mcnally, R. (2017). Remote real-time collaboration through synchronous exchange of digitised human–workpiece interactions. Future Generation Computer Systems, 67, 83-93.

Padgett, J., Gossett, K., Mayer, R., Chien, W., & Turner (2017). Improving patient safety through high reliability organizations. The Qualitative Report, 22, 410-425. Retrieved from https://nsuworks.nova.edu/tqr/

Price, J. (2018). Pharmacovigilance in crisis: Drug safety at a crossroads. Clinical Therapeutics, 40, 790-797.

Routray, R., Tetarenko, N., Abu-Assal, C., Mockute, R., Assuncao, B., Chen, H., … Mingle, E. (2020). Application of augmented intelligence for pharmacovigilance case seriousness determination. Drug Safety, 43, 57-66.

Schmider, J., Kumar, K., LaForest, C., Swankoski, B., Naim, K., & Caubel, P. M. (2018). Innovation in pharmacovigilance: Use of artificial intelligence in adverse event case processing. Clinical Pharmacology & Therapeutics, 105, 954-961.

Stergiopoulos, S., Fehrle, M., Caubel, P., Tan, L., & Jebson, L. (2019). Adverse drug reaction case safety practices in large biopharmaceutical organizations from 2007 to 2017: An industry survey. Pharmaceutical Medicine, 33, 499-510.

Tan, Y., Hu, Y., Liu, X., Yin, Z., Chen, X., & Liu, M. (2016). Improving drug safety: From adverse drug reaction knowledge discovery to clinical implementation. Methods, 110, 14–25.

Tomita, A. (2019) A managing framework for artificial intelligence to supplement human workforce in a digital economy. 2019 Portland International Conference on Management of Engineering and Technology (PICMET), 1-7.

U.S. Food & Drug Administration. (2021). FDA adverse event reporting system (FAERS) public dashboard. Retrieved from https://www.fda.gov/drugs/questions-and-answers-fdas-adverse-event-reporting-system-faers/fda-adverse-event-reporting-system-faers-public-dashboard

Wang, W., Wang, Y., Zhang, Y., & Ma, J. (2020). Spillover of workplace IT satisfaction onto job satisfaction: The roles of job fit and professional fit. International Journal of Information Management, 50, 341-352.