How the FDA guidance on the use of AI will impact drug development

Artificial intelligence (AI) is increasingly being applied to different parts of the drug development and regulatory decision-making process with the objective of improving efficiency, accuracy, and speed. More specifically, AI is playing an integral role in personalized medicine to identify genetic markers, optimize drug dosages, and minimize adverse reactions[1].

The increasing application of AI in drug development has prompted regulatory authorities to evaluate how AI-enabled drugs should be regulated. In January, the US Food and Drug Administration (FDA) issued draft guidance on the use of AI to support regulatory decisions about the development of drugs and biologics. The European Medicines Agency released a Reflection Paper on the use of AI in the medicinal product lifecycle that address similar issues as the new FDA draft guidance[2].

In announcing the first AI guidance from the agency, FDA Commissioner Robert M. Califf, M.D., emphasized the importance of a risk-based framework that would both promote innovation while ensuring robust regulatory and scientific standards are maintained. “With the appropriate safeguards in place, artificial intelligence has transformative potential to advance clinical research and accelerate medical product development to improve patient care,” he said[3].

The draft FDA guidance, Considerations for the Use of Artificial Intelligence to Support Regulatory Decision-Making for Drug and Biological Products[4], pulls together some broad concepts for industry, with the goal of assisting in the communication with industry to facilitate FDA review and interpretation of the data.

The guidance offers a risk-based credibility assessment framework for determining the credibility of an AI model within a context of use (COU) to help sponsors and other relevant parties to “plan, gather, organize, and document information to establish the credibility of AI model outputs when the model is used to produce information or data intended to support regulatory decision-making.”

To better understand the objectives and implications behind the guidance, PharmaLex sat down with Thomas (Tom) Stover, Principal Consultant, Regulatory Affairs, and Daniel Galla, Associate Director, Regulatory Affairs, to gather their insights.

Can you share any initial thoughts on what the guidance seeks to address and the FDA’s objectives behind it?

Daniel: This is the first time that FDA has shared its expectations on how AI should be constructed or the type of data you would need to present to use AI as part of your application. Key takeaway from the guidance was that companies are expected to share the source data that were given to the algorithm to learn, which may be very specific or unique for each use-case. The challenge for small and mid-sized companies is to have adequate source data to train the AI well enough to be reasonably predictable in the outcome. Data in the public domain is limited, and it will be difficult to retrieve some source data to prove to the FDA that the data used to train the AI model is reliable.

So, the agency is trying to put some framework around this and is encouraging sponsors to engage with them as soon as possible to guide them along the way, because this is new territory for the FDA too.

Can you offer some context for the guidance that might shed light on the FDA’s thinking and objectives as they work with industry to apply the framework?

Tom: If I look at it in the context of how AI is being used in drug development today, there has been some anxiety within the FDA over how to review some AI-supported personalized vaccines programs, as our colleagues highlighted in their article on the subject[5]. As an example, Moderna and Merck are developing an individualized neoantigen cancer vaccine, mRNA-4157, which involves the identification of tumor-specific neoantigens from the analysis of both healthy and tumor tissue samples from a patient. A proprietary AI algorithm is used to decide which patient-specific neoantigen to target with an mRNA vaccine. Since each patient’s mRNA vaccine may be unique to that patient, it poses challenges for the FDA when assessing the safety and efficacy of these varying mRNA vaccine sequences, as well as with ensuring quality control since there is no standardized mRNA sequence from product to product. In order to address some of these challenges, the FDA asked Moderna to lock the algorithm before the study to prevent any bias from being introduced with an evolving AI algorithm[6].

Are there examples from the guidance that stand out for you?

Daniel: The guidance does go into assessing the risk matrix and provides a couple of examples. This will help sponsors to understand at what step in the risk-based credibility assessment framework they could apply an AI model. However, it clearly outlines the need for some form of control mechanism. Whether a human control is necessarily may vary on the COU of the AI.

Tom: I agree, it provides a framework to develop such a model, with the key advice being early engagement with the FDA as soon as feasible, particularly since a significant amount of effort, time, and money is necessary to train these models.

The draft guidance establishes a framework for industry to address and implement COU in a risk-based manner. One example is given in the guidance is using AI to perform an assessment of the fill volume in vials, which seems to be a fairly simple automated process. As we get into more complex areas, such as non-clinical or clinical development, how the framework is applied becomes a lot more complex. Consequently, the FDA is expected to release more specific guidance that touch on more complex COU for AI in the future.

What makes the review process so challenging for FDA?

Tom: If you think about traditional recombinant proteins or monoclonal antibodies, the manufacturing process is often considered the “drug product”. If the manufacturing process is changed, a manufacturer is expected to complete a robust comparability assessment, which could potentially include clinical testing, in order to implement the change. With AI-enabled drug products, the proprietary AI algorithm may be considered a critical component of what the manufacturer defines as the drug product, since the algorithm determines the mRNA sequence. This is a new way of thinking because there is no standardized manufacturing process that can be inspected by the FDA.

Daniel: Personally, the challenge for the FDA is that they have a plethora of data from various product either approved or currently under development. Can they use this without compromising data exclusivity? When we’re talking about clinical models, there’s a lot of work to do in terms of gathering sufficient and reliable data for an AI model that will result in a good outcome. Further considering the FDA is part of the US government and relies on annual budget approval, do they have the funds to develop the capabilities and solutions to use AI for decision-making and approving applications in the near future?

Tom: I agree. In addition to the large volume of clinical data to review in any marketing application, the FDA now has to prepare itself to review the algorithm. The challenge is how quickly can the FDA add the necessary resources and expertise to review and scrutinize these proprietary AI algorithms? For instance, it will be interesting to see how manufacturing inspections of AI-enabled technologies will be implemented, considering the physical requirements of the computing power involved.

What would your advice be for applicants looking to apply the guidance to any AI model within their drug development program?

Tom: Step four in the FDA’s seven-step process is to develop a plan to establish the credibility of the AI model. As recommended in the guidance, this is the ideal time to engage the FDA prior to implementation of the model. Aligning with the FDA prior to implementation of the model (step five) will be critical, particularly if clinical development is involved, given the expense and time commitment. Of course, the FDA will have a learning curve, and some offices and divisions within FDA are further ahead than others. One question is: will the FDA establish a new division focused on AI algorithm assessments that can consult with all divisions, as they have done with the Division of Clinical Outcomes Assessment, within the Office of Drug Evaluation Science, to support the integration of the patient voice in drug development[7].

Conclusion

AI is revolutionizing drug development, but its application continues to pose challenges both for drug developers and regulators. Regulatory guidance and collaboration with industry, for example through continued workshops and consortia, will help to advance understanding of how AI-enabled technologies can be applied in regulatory decision-making. There will be a learning curve for industry and the health authorities as both parties apply the draft guidance and gain practical experience with AI-enabled drug development. However, the FDA’s draft guidance reflects a commitment to adapting regulatory frameworks to accommodate the transformative potential of AI in healthcare.

Have you considered how the FDA guidance can be applied to your own innovative medicines initiatives? We would be interested in hearing your thoughts and exchanging ideas. Contact us