Risk and reward: tools and metrics for identifying and monitoring risk in clinical trials

When conducting a clinical trial, the stakes are notoriously high. Because you're dealing with unapproved drugs and vulnerable human subjects, your natural response as a sponsor may be a state of hypervigilance. You want to do all you can to ensure that subjects are safe and that you've stamped out any source of risk.

In fact, a degree of 'red alert' is specified by FDA, which requires sponsors to monitor the conduct and progress of their clinical investigations. This is intended to protect their human subjects and ensure the quality of the data they ultimately submit.
Historically speaking, many sponsors have responded to the specification with a huge injection of money, time and resources. They would typically conduct 100% source-document verification (SDV) at every data point collected, complementing their efforts with frequent on-site monitoring.

Unfortunately, today's sponsors are required to do more with less. As trials grow in scale and complexity, they must contend with elaborate infrastructure (wide geographic dispersion and site variability, for example) and tightened budgets, leading to dwindling resources and fewer personnel. Complete data verification is no longer a feasible option.

As Lori Convy, monitoring subject matter expert at Sanofi, explains: "There are increasing amounts of data and potentially increasing numbers of subjects. Complexity risk increases because there is a greater potential for protocol violations or inadvertent errors that can result in data integrity or safety issues."

It may seem like an insurmountable challenge, but today's new systems and technology present opportunities too, enabling sponsors to achieve their aims even with budget constraints. Using electronic systems and records, and availing themselves of improvements in statistical assessments, sponsors are finding new ways to surmount their growing list of hurdles.

In particular, there is a trend towards risk-based monitoring - essentially, targeting your monitoring activities to wherever they're they are going to deliver the most benefit. This might entail conducting SDV only in response to triggers, or setting caps on the amount of SDV at the site or subject level.
While sponsors have been thinking this way for some time, the trend has really gathered pace in recent years as regulatory agencies have taken note. In the years since the financial crisis, the advice available has changed, with sponsors and CROs encouraged to adopt a more nuanced approach.

For instance, in August 2013, FDA finalised its guidance on clinical trial oversight, claiming that "risk-based approaches to monitoring... are more likely than routine visits to all clinical sites and 100% data verification to ensure subject protection and overall study quality".

As the paper describes: "A risk-based approach to monitoring does not suggest any less vigilance in oversight of clinical investigations. Rather, it focuses sponsor oversight activities on preventing or mitigating important and likely risks to data quality and processes critical to human subject protection and trial integrity."

Quality control
The Medicines and Healthcare products Regulatory Agency (MRHA) and European Medicines Agency (EMEA) have advocated for a similar culture change, determining that more-efficient strategies are needed. According to EMEA: "The current approach to clinical quality management is in need of review and reorientation... There is a need to find better ways to make sure that limited resources are best targeted to address the most important issues and priorities, especially those associated with predictable or identifiable risks to the well-being of trial subjects and the quality of trial data."

So what does that mean for sponsors and CROs in practice? While 100% data verification may have burned through resources, it was at least relatively straightforward to implement. Sponsors are now required to be far more tactical. As Convy explains, they should begin by conducting a robust risk assessment and implementing a plan.

"Although we typically focus on increasing risk, there are also some scenarios where risk decreases over the life of a trial," she says. "So when risk decreases, the monitoring needs to decrease. The most important aspect of monitoring the risk is to ensure there are clear directions about what type of response to the increasing risk is required. This should be clearly documented in the associated functional plan, such as the monitoring plan."

This is not always an easy task. Since the regulatory requirements are relatively open, it has been left to the pharmaceutical companies to work out their own methodology. This means there are no uniform guidelines for reporting various levels of risk and no standard frameworks for risk-based monitoring.

Along with several other biopharmaceutical companies, Sanofi is a member of TransCelerate BioPharma, a non-profit organisation focused on identifying and solving common R&D challenges and further improving patient safety. One of its key initiatives is risk-based monitoring - specifically, the development of an industry-wide standard and approach.

"Sanofi advocates for public-private partnerships, and we realise that no one company has all the answers - that's why we became an active member of TransCelerate," says Convy. "It works to improve patient safety with the goal of developing more high-quality medicines, and it evolved from discussions at various forums for R&D leadership, looking at relevant issues that were facing the industry and solutions for addressing those common challenges."

Its risk-based monitoring position paper, released in 2013, outlines the proposed methodology and tools. Four steps are listed: the cross-functional risk assessment of the programme and protocol; the identification of critical data and processes within a risk management plan; the assessment of potential key risk indicators (KRIs); and the monitoring execution of the study. It includes details of a Risk Assessment Categorisation Tool (RACT) that could help sponsors determine which risks matter most.

"Many of the vendor solutions that are being developed are trying to incorporate the output of the RACT, and others are focused on key risk indicators," says Convy. "So there's some interesting discussion going on now about which of those KRIs is most important, which one should be standard and how we can establish the appropriate thresholds so that we ensure action is taken at the appropriate time."

Defining these risks is critical. Sanofi has found that its main challenges in this space are related to data sourcing and integration - specifically, accessing the correct data within data warehouses. If you're not sure exactly what risks you're looking for, you will wrestle with information overload and struggle to refine your search. Convy believes the solution is simply to become more experienced at dealing with KRIs.
Sanofi itself is looking into the TransCelerate methodology, and considering how the steps can be integrated into its own internal processes and standard operating procedures (SOPs).

The company expects to implement a commercially available risk-based-monitoring dashboard system and has assessed numerous solutions to work out which one aligns best with the TransCelerate recommendations.This will mean staying abreast of the key technological developments as they arise.

"Hopefully, new approaches will continue to drive new technology," says Convy. "The monitoring practices themselves really haven't changed much over the years, but the regulators have said that the industry needs to fully use any electronic capabilities that are available to continue to drive the development of these new-technology approaches."

Holding pattern
For the time being, there are various systems available for remote data capture and clinical trial management, both of which can support a risk-based monitoring approach. The more advanced versions of these systems can be set up to flag outlying data, ensuring that the monitoring team are aware of any issues as and when they arise.

"Our industry will incorporate technology such as that used in the credit card industry, where outliers are the areas that really need to be investigated further rather than increasing time and on-site activity," continues Convy. "With big data, we can see those outliers - there's electronically sourced data that will allow us to view some of the medical records from afar.

"And there are new barcodes being developed that will help us keep track of our investigational products without having to actually go on site and do physical counts. We need to work towards the ability to do real-time monitoring, so we can log what's happening with our clinical trials at individual sites at any given point in time," she adds.

For the time being, Sanofi is working to ensure that its external stakeholders truly understand its wider aims and the associated methodology.
It also works with sites to ensure maximum transparency and understanding. After all, risk-based monitoring is not simple but, equally, it is not going anywhere. As the practice gathers momentum it is vital that all parties know where they stand.
"We're looking forward to incorporating the systems data that we have, and moving forward in step with the rest of our TransCelerate partners and the rest of the industry," says Convy. "We're very excited about risk-based monitoring."