Reducing Protocol Deviations With Training-Based Metrics

Protocol deviations can have a detrimental effect on new drug development, adding to costs and time to clinical trials and even putting the acceptability of data generated during a study at risk. Despite this, protocol deviations seem to be viewed as just part of doing business in the clinical research arena. And one question that arises is whether training in individual protocols is adequate, given the frequency with which deviations occur.

In its recent analysis of inspectional findings during 2021, the FDA said in its annual BIMO metrics report that protocol deviations or failure to follow the investigational plan was the most common observation listed on Form 483s after a BIMO inspection. This was also the most-frequent Form 483 observation during 2020, and has been at the top of the list for the last several years, the FDA noted.

And in its January/February 2022 Impact Report, Tufts University Center for the Study of Drug Development (CSDD) reported that protocol deviations rose in the 2018-2020 period compared to the 2013-2015 period, as have substantial amendments to protocols. According to the CSDD report, Phase 3 trials had the highest number of deviations per protocol, with an average of 118.5, which affected 32.8% of patients enrolled. Phase 2 studies saw an average of 75.3 deviations affecting 30% of patients, and Phase 1 trials had an average of just 8.7 deviations affecting 15.3% of patients.

Oncology trials saw a higher number of deviations, on average, compared to other areas of research, with Phase 2 and 3 oncology protocols having 30% more deviations compared to non-oncology protocols. Additionally, protocols outsourced to CROs had more deviations and substantial amendments per protocol, CSDD reported.

But despite the potential for negative repercussions, protocol deviations remain an ongoing challenge for researchers. As a matter of fact, in a June 2020 blog post, clinical trial platform provider Castor referred to protocol deviations as “the new normal.”

Precisely quantifying the impact of all of these protocol deviations on clinical research is difficult because of the variation among different areas of study and even among individual clinical trials. However, it is clear that deviations do affect trial progress and outcomes.


Industry seeking to change

Despite these figures, however, researchers and sponsors appear dedicated to reducing or even eliminating protocol deviations. In fact, some research organizations use protocol compliance as one of the metrics by which site performance is measured. For instance, Diane Whitham et al., on behalf of the Site Performance Metrics for Multicenter Randomized Trials Collaboration, listed several metrics related to protocol compliance in an October 2018 Trials article, including:

  • Percentage of randomized participants with at least one protocol violation;
  • Percentage of randomized participants receiving allocated intervention as intended per protocol;
  • Number of missed visits per number of randomized participants;
  • Number of late visits per number of randomized participants; and
  • Number of critical or major audit findings per number of randomized participants.
  • And the entirety of a seasoned clinical research audience attending a 2021 webinar agreed that one key metric should define success for site training: zero important protocol deviations.

The message seems to be that the goal of zero deviations—or at least zero important deviations—can be achieved with the right strategy. The industry is clearly keen to avoid the potential blowback from protocol deviations, which can include additional time to complete a trial, the costs associated with extending a trial.

Protocol deviations can increase costs due to the study having to be extended, additional patients enrolled, replicated work and prolonged timeline for product development. Additional monitoring and retraining costs may be incurred, as well. Deviations also may have an impact on data quality that could ultimately have long-term implications on product approval.

In fact, the ICH E9 guidance, Statistical Principles for Clinical Trials, stated clearly that protocol deviations “always bear the potential of invalidating the trial results.”

And all protocol deviations require documentation and reporting, Laurie Halloran, president and CEO of Halloran Consulting Group, noted in the May 2, 2018 Life Science Leader, activities that eat up staff resources and can add to costs in that way. If an unreported deviation is discovered during a regulatory inspection, it is likely to result in penalties from the FDA or other regulators.

In some cases, deviations may lead to protocol amendments, which can delay trial completion, potentially costing $35,000 per day or more, according to a 2015 survey by Clinverse, a statistic still used for cost estimates. Amendments can add an average of 30 days—and associated costs—to the length of a study, according to a 2021 report from CSDD.

Protocol deviations—whether enrolling an unqualified patient, incorrectly performing a visit or procedure or failing to collect data necessary to interpret primary endpoints—can result in unusable data that requires additional time and resources to replicate. In its March/April 2021 Impact Report, CSDD noted that mid-study amendments typically require 30 days to complete before a study can resume, adding significantly to the time required to complete a clinical trial.


Addressing deviation risks with training

There are myriad reasons behind protocol deviations. Ever-increasing protocol complexity could be one factor that contributes to protocol deviations. CSDD noted that protocols have been steadily increasing in complexity since 2009, with increasing numbers of endpoints and distinct protocol procedures. Those increases provide more opportunities for investigators or research staff to make mistakes.

Sometimes a change in procedure may be made to ensure patient safety, as in the case of emergency treatment, Halloran said.

And social distancing requirements during the COVID-19 pandemic spurred some protocol deviations, such as replacing in-person site visits for patients with remote or telehealth visits. The FDA in August 2021 updated its guidance, Conduct of Clinical Trials of Medical Products During the COVID-19 Public Health Emergency, recognizing these necessities and clarifying what protocol deviations due to COVID would be acceptable with appropriate reporting.

But these relatively innocuous reasons are not the only causes of protocol deviations. For instance, deviations may occur because the protocol is hard to understand or allows different interpretations at different sites. And they may occur because study teams have not been trained adequately or because knowledge isn’t transferred well during staff turnover. In addition, Insufficient performance-based tracking throughout site training makes it difficult for proactive actions to be taken to address high-risk areas for protocol deviations that may result from site error.

All of these causes can be addressed with the right type of training. Thorough, protocol-specific training can help ensure that investigators, coordinators and other research staff fully understand how to perform all tasks included in the protocol. And properly designed training will develop mental muscle memory so that researchers can perform correctly under pressure or when substantial time has passed since training. Besides developing participants’ comprehension, proper training should track performance and allow sponsors to identify and correct potential problems before they impact a study’s budget or timeline.

For instance, protocol simulation-based training can mimic realistic scenarios and guide research staff through each step, including making the right decision when problems occur, like:

  • A patient takes a restricted medication just before the randomization visit;
  • A temperature excursion occurs for the investigational product; or
  • A patient misses a dose or a visit where a procedure was to be completed.

Ideally, the training allows the learner to critically think through preventive actions. The simulation scenarios should not only model what can go wrong and assess potential corrective actions, but more importantly, how the issues can be prevented. When users make decisions in a simulation experience it can be tracked, collected, and presented in performance and compliance reports. These reports provide invaluable insight that can identify and address individual or site-wide weak points before a protocol deviation occurs. In contrast to traditional training, simulation-based training aims to ensure study training is consistent, intense, and proactive.

Simulation-based training in particular allows research staff to practice a protocol and all its nuances in advance of applying it to real patients and real data, with the associated real potential risks. And in addition to identifying and addressing protocol deviation risks, the designed scenarios can allow research staff to learn how to prevent the issues that lead to deviations before they arise.

And this practical approach to training can help sponsors and researchers reduce the incidence of protocol deviations and avoid the additional costs—in both time and money—associated with fixing those deviations.


Contact us to discover how our simulation-based training approach can reduce protocol deviations across studies.

32 replies
  1. Hokicoy Daftar
    Hokicoy Daftar says:

    Link exchange is nothing else however it is only placing the other person’s
    web site link on your page at suitable place and other person will also do similar
    in support of you.

  2. warung makan
    warung makan says:

    Greetings! I know this is kinda off topic but I was wondering which blog platform are you using for
    this website? I’m getting fed up of WordPress because I’ve had problems with hackers
    and I’m looking at alternatives for another platform.

    I would be great if you could point me in the direction of a good platform.

  3. Effie
    Effie says:

    I have read a few excellent stuff here. Definitely price bookmarking for revisiting.

    I wonder how so much attempt you set to make one of these wonderful informative web site.

  4. Kalon
    Kalon says:

    What’s up mates, nice paragraph and pleasant arguments commented at this place, I am actually enjoying by these.

  5. Margarito
    Margarito says:

    Amazing issues here. I’m very happy to peer your post.
    Thanks so much and I am having a look ahead to touch you.
    Will you please drop me a e-mail?

  6. Titus
    Titus says:

    Wow, fantastic blog layout! How long have you been blogging for?
    you made blogging look easy. The overall look of your website is
    fantastic, let alone the content!

  7. Takina
    Takina says:

    We’re a group of volunteers and opening a new scheme in our community.
    Your site offered us with valuable information to work on. You’ve done an impressive job and our entire community will be grateful
    to you.

  8. Shamyra
    Shamyra says:

    Great blog right here! Also your web site lots up fast!
    What web host are you using? Can I get your associate link on your host?
    I wish my site loaded up as fast as yours lol

  9. Othello
    Othello says:

    Excellent article. Keep writing such kind of info on your page.
    Im really impressed by it.
    Hey there, You have done a fantastic job.
    I will certainly digg it and in my opinion recommend to my friends.
    I am confident they’ll be benefited from this website.

  10. Savvas
    Savvas says:

    Do you mind if I quote a few of your articles as long as
    I provide credit and sources back to your website?
    My website is in the exact same niche as yours and my visitors would definitely benefit from a lot of the information you
    provide here. Please let me know if this okay with you.

  11. Tandria
    Tandria says:

    When I initially left a comment I seem to have clicked on the
    -Notify me when new comments are added- checkbox and from now on whenever
    a comment is added I get 4 emails with the exact same comment.
    Is there a way you can remove me from that service?

    Appreciate it!

  12. says:

    I know this if off topic but I’m looking into starting mmy own blog and was
    curious what all is needed to get set up? I’m assuming having a blog like
    yours would cost a pretty penny? I’m not vey internet savvy so I’m not 100% positive.

    Any tips or advice would be greatly appreciated. Thanks

  13. says:

    I rarely comment, however i did a few searching and wound up here Reducing Protocol Deviations With Training-Based Metrics
    – Pro-ficiency. Annd I actually do have a few questionns for
    you if you don’t mind. Is it only mme or ddoes it
    look like some of the responses look as if they
    are written by brain dead visitors? 😛 And, if you aare posting on other
    sites, I would like to keep up with anything new youu
    have to post. Would you list of every one of all
    your communal sites like your linkedin profile, Facebook page or twitter feed?

  14. Nancy
    Nancy says:

    What’s Taking place i’m new to this, I stumbled upon this I have discovered It
    absolutely helpful and it has aided me out loads.
    I hope to contribute & aid different customers like its helped
    me. Great job.

  15. Reda
    Reda says:

    I loved as much as you will receive carried out right here.
    The sketch is tasteful, your authored material
    stylish. nonetheless, you command get got an shakiness over that you wish be delivering
    the following. unwell unquestionably come further formerly again since exactly the same nearly very often inside case you shield this increase.

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *