Risk and reward

The same year the International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH) launched its E6 Guideline for good clinical practice (GCP E6 (R1)), two Stanford University students began to develop a new type of search engine they jokingly called BackRub.

It was 1996, the first year Americans sent more mail through the internet than postal service, and the year Pfizer patented Viagra. Information technology was enabling globalisation like never before, and data-driven processes like clinical trials were right at the forefront of the change. As its name might suggest, the ICH implemented its guidelines to standardise the process for researching and testing pharmaceuticals, ensuring there was no need for procedures to be repeated in different markets. Similarly, BackRub was beginning a mission to organise the world’s information and make it universally accessible and useful, although it’s hard to imagine ‘BackRubbing’ some conversational trivia.

No matter. BackRub is now known as Google. The rebrand was one of tens of thousands of updates it went through over the next 20 years, a process that made it one of the most important companies in a world it was indelibly shaping. Similarly, whereas clinical research projects were once conducted with stacks of paper in one or two highly controlled settings, they are now spectacularly complex digital procedures that take place over multiple sites and types of sites spread across the planet. While Google adapted, however, the GCP guidelines stayed exactly the same.

To quote from the More Trials campaign’s open letter to the ICH, which was signed by five CROs and 119 health researchers across 22 countries, by 2016 there22 countries, by 2016 there was a consensus that a lack of flexibility in the ICHGCP guideline and its interpretation “has resulted in clinical trial processes that are unnecessarily complex and expensive, as well as seriously hindering the adoption of innovative approaches to their conduct”.

For its part, the ICH maintains that “E6 gave sponsors flexibility to implement innovative approaches, but has been misinterpreted and implemented in ways that impede innovation.” Even so, after the European Medicine Agency (EMA) published a report summarising the results of almost 400 GCP inspections in 2014, finding scope for improvement in monitoring, data management, and reports and documentation, the ICH convened a working group of international experts to draft an update. In late 2016, two decades since the original GCP E6 guidance, the council adopted its first addendum. The most significant additions focus on increasing flexibility, without compromising patient rights or welfare, by taking a risk-based approach to designing and managing studies, as well as specifying sponsor oversight responsibilities, and buttressing it all with improvements to technology use and data integrity.

It’s an update for a post-BackRub world. Indeed, one could argue that the search engine is itself a model for the changes, and not simply because it reflects the inexorable rise of electronic records. ‘Google’ displaced the verb ‘search’ in an online context because of the approach it takes to determining the relevance of web pages. ICH E6 (R2) applies similar insights to monitoring risk and ensuring quality in clinical trials.

Show your workings

It’s the great cliché of contemporary business – if you’re not on Google, you don’t exist. It’s not always so brand-specific, but everyone in clinical trials knows a similar feeling. As Elizabeth Bodi, senior consultant at Halloran Consulting Group, says, “If it isn’t written down, it didn’t occur.” In her experience, a lot of the changes mandated by ICH E6 (R2) actually formalise shifts that were already taking place on the level of individual clinical trials. In one form or another, risk was always on the minds of sponsors, CROs and investigators, but, from a regulatory perspective, there’s no thought process without the definitions and documents to record and evaluate it.

“Now sponsors are trying to think, ‘How do we evaluate risk, and how do we calculate the likelihood of problems or errors to impact on the reliability of our trials?’” explains Bodi. The FDA’s advice on monitoring asserts that “sponsors should prospectively identify critical data and processes, then perform a risk assessment to identify and understand the risks that could affect them”. Bodi gives the example of trials involving special populations, unusual treatment approaches or seasonal diseases, where enrolment is a defining challenge. “They need to ask, ‘What’s our riskbased plan towards handling that?’ and take it all the way through to, ‘How do we report, review and communicate it with a risk-management plan?’”

Still, the Google comparison goes further. Before 1996, search engines delivered results primarily by counting keyword matches. Intuitive as that might sound, it wasn’t a particularly efficient way to serve users with the information they wanted. Using a word a lot isn’t equivalent to understanding it. Or, as Bodi puts it for trials, “Collecting 1,000 data points is not necessarily the best thing unless you’re going to do something with them.”

She continues, “In the past, we just said we’ll review 100% of everything, no matter what it is. But for some studies, the level of rigour of the review of that data is not as critical. To use an exaggerated example, an outpatient study for the elimination of acne probably has a lower risk versus an inpatient gastrointestinal tumour study.”

In the ICH’s own words, sponsors were “emphasising less important aspects of trials (such as focusing on the completeness and accuracy of every piece of data) at the expense of critical aspects (such as carefully managing risks to the integrity of key outcome data)”.

By contrast, Google perfected the science of searching thanks to its PageRank technology, which organises results by taking into account each page’s place in the wider internet. It works by tracing links that direct to a particular web page back to their sources (hence BackRub), recording that page’s connections to other trusted, well-linked sites to determine the quality and value of its content. PageRank treats links like votes in assessing and ranking search results in the context of a larger network of information. To use Bodi’s terminology, search results are ordered not so much by the density of data points as each data point’s impact.

Similarly, in requiring sponsors to take a risk-based approach to their studies, GCP E6 (R2) shifts the focus into identifying and monitoring the data and processes that can directly impact efficacy, subject safety, rights and welfare – the variables that can put the whole trial into question. Under the new guidelines, each study requires a specific monitoring plan and rationale that reflects this. As the FDA specifies, “A risk-based approach to monitoring does not suggest any less vigilance in oversight of clinical investigations. Rather, it focuses sponsor oversight activities on preventing or mitigating important and likely risks to data quality, and to processes critical to human subject protection and trial integrity.”

In a much more succinct manner, Bodi says, “It forces sponsors to think about what they are going to do with the data points they’re collecting.”

61%
Clinical trial starts in 2018 employed riskbased management, up from 18% in 2016 .
Association of Clinical Research Organizations

The central argument

Helpfully, the technological advances that have taken place in the past 20 years enable ongoing, near-realtime analysis of the relevant data points. According to the results of the Risk ADApted MONitoring (ADAMON) study published in 2017, “Central statistical monitoring has the potential to detect a considerable proportion of findings without on-site monitoring.” Issues with missing or inconsistent data across sites thousands of miles apart can be addressed as they arise, and the right opportunities for targeted on-site monitoring identified. The report concluded that risk-based approaches are “noninferior” to extensive monitoring, although they require less than 50% of the resources.

However, centralisation is far from a replacement for traditional on-site visits. The way forward in a trial environment, where less than 20% of clinical leaders at sponsors believe they have productive relationships with their CROs, is not to push stakeholders further apart. What’s more, while sponsors and CROs can cut costs with risk-based monitoring approaches, centralised statistical evaluation of data can actually be more time-consuming for sites. For efficiency to become more than a buzzword, the industry needs to walk a very particular tightrope.

According to the FDA, a risk-based approach is appropriate for such high-wire acts. Done properly, it should be “dynamic, more readily facilitating continual improvement in trial conduct and oversight. For example, monitoring findings should be evaluated to determine whether additional actions (such as training of clinical investigator and site staff, or clarification of protocol requirements) are necessary to ensure human subject protection and data quality across sites.”

The word ‘oversight’ appears three times in the text of the R2 addendum, up from zero mentions in the original GCP E6 document. Despite this, the council frames the additions as, “a clarification of expected trial conduct to reduce misinterpretation of oversight responsibilities”. Without clear guidelines on how sponsors, CROs and other third-party vendors should work together, says Bodi, the industry focus on “dollars and efficiency” has led to issues where sponsors are unable to provide documentation explaining vendor processes and actions. In such a scenario, sponsors and regulators are limited in their ability to manage, assess or improve trial quality. As such, the new guidelines use oversight to make clear that even when sponsors delegate aspects of monitoring and management to a third-party vendor, they must ensure these responsibilities are carried out properly – even if vendors also go on to subcontract some of the work.

Put your project management hat on

And it may well take a lot of work. “Just from a demographic perspective, our industry is mostly very bright people with biology, nursing, medical or pharmacy backgrounds, but throughout our schooling, we’re not always taught about project management,” points out Bodi. “Risk management is part of that; some people just can’t get their heads around describing risk and defining an impact. There might be a scientific reason behind why they collect a data point, but they haven’t necessarily thought through what that means in their clinical development strategy for the entire product line.” Even though the study documentation required by E6 (R1) already specifies some risk-based considerations, Bodi and her fellow consultants at Halloran are still supporting sponsors that find the idea of supplying direct evidence of how they manage risk quite “nebulous”. Similarly, some sponsors and CROs are lagging behind sites in training their staff on the new requirements.

The opposite is also the case. At the moment, Bodi explains, a lot of sponsors are “not stopping to think” how the changes can affect existing processes. Some are so keen to adapt that they want to implement the new guidelines into ongoing trials, where they could actually affect the integrity of data and its collection.

In more ways than one, risk is a difficult concept to build behaviour around. Though the additions make clear that oversight and monitoring procedures should be responsive to the level of risk involved in a trial, as Beth Harper, president of research consultants Clinical Performance Partners, wrote in 2017, “For many in the industry, when trust starts to erode, more oversight is needed. The application of this oversight is often translated into more micromanagement of the CRO, which is counterproductive and costly, not only from a financial perspective but also from a relationship perspective.”