Using Objectives and Key Results (OKRs) To Ensure Success

 

“If you don’t know where you are going, you might not get there.”  

-Yogi Berra

 

 

At Pro-ficiency, we are scaling the business to meet and exceed the expectations of our customers.  A key part of that is having great execution across our organization aligned to True North for our customers and our company.

To operate Pro-ficiency, we utilize a variation of the Toyota Lean Management System.   John Doerr, one of the leading venture capital investors in Silicon Valley, wrote about this approach in his book Measure What Matters.  From an early stage, Google built their business around this model.

“OKR’s have helped lead us to 10X growth, many times over. They helped make our crazily bold mission of “organizing the world’s information” achievable.  They’ve kept me and the rest of the company on time and on track when it mattered most”.

– Larry Page, Google Co-Founder

This management system can apply to any industry as witnessed by the successful deployment by Intel, Bono (the entertainer), Zume Pizza and the Gates Foundation just to name a few.  

As John Doerr relates in his book, the definition of “OKRs” is “Objectives and Key Results.” It is a collaborative goal-setting tool used by teams and individuals to set challenging, ambitious goals with measurable results. OKRs are how you track progress, create alignment, and encourage engagement around measurable goals.

Our capital partner, NovaQuest Capital, uses a similar model with each of their portfolio companies.

At Pro-ficiency, Wes Stewart (our CFO) and I have worked with the management team to implement a series of Tracking Sheets for each functional area (currently 12 tracking sheets).  Each functional area has 5-8 measurable goals that are aligned to the other functional areas across the company.   Each month we measure each result as either red or green.  There is no yellow.

We had our first Operating Committee Meeting with the new management team this past month.  We were able to accomplish a deep review of the business in less than 2 hours and focused the team on the critical performance issues.

If a measure on a Tracking Sheet is red, we create a detailed corrective action plan on a single sheet of paper.  This allows us to quickly align on a cross functional plan for getting us back on track.

We are scaling rapidly as a business to meet the market demands for our products.  The Management System allows us to quickly assimilate new talent into the organization and quickly align around a shared vision for our future.

We know where we are going.  At Pro-ficiency, we want to enable every clinical investigator, in every clinical trial, to operate at their peak performance through our immersive simulation experience.   Join us on this noble mission.

Opening Back Up: When Will Investigator Meetings Resume?

The announcement by the CDC on May 13th indicated that vaccinated consumers could fully resume activities that you did prior to the pandemic.  Personally, I was able to secure my Pfizer vaccines in March from a daily overflow of excess doses at the Kaiser Health System here in Colorado.  I had little reaction to the vaccine and would encourage others to get vaccinated.  

It is gratifying to be a part of an industry that is making such a difference in consumers lives around the globe.  The expedited development and clinical trial process for COVID vaccines illustrated that there is an opportunity to speed the timeline for new drug discoveries.  Pro-ficiency is poised to help sponsors efficiently and effectively conduct clinical trials.

With travel slowly opening back up, the question is when will investigator meetings resume?   We recently asked over 1350 investigators in one of our trials whether investigator meetings were preferred over our simulation training.  Almost 90% of the investigators indicated that our simulation training approach was on par or better than investigator meetings.  That could have a profound impact on the on-going cost and speed of clinical trials.

The sponsor for the above clinical trial was thrilled by this feedback from investigators.  It provided them new flexibility in launching the rollout process for their next clinical trial.   

One new option available to sponsors might be the hybrid approach.  The simulation training could be leveraged as a tool to assess where the investigators might struggle in the execution of a clinical trial.  The in-person investigator meeting could be shortened to only those high-risk areas.  The simulation training could also make the in-person investigator meetings optional for those where the time away from clinical practice is a challenge.

Pro-ficiency has also started examining the 10% that prefer investigator meetings.  Is there a way to create more connections with the sponsor via Podcasts?  Live Stream events?

As travel starts to open back up, Pro-ficiency provides sponsors the opportunity to leverage our simulation training platform to better meet the needs of investigators. 

Building a Focus Differentiated Company: Success Through Innovation

Pro-ficiency is a Focus Differentiated Company

 

Michael Porter, a Harvard Business Professor, analyzed 1000’s of successful companies and determined that the most successful companies were either a focus-differentiated or cost-differentiated company.  A good example of a cost-differentiated company would be Costco.  They have built their brand on price.  Costco’s mission is to to continually provide members with quality goods and services at the lowest possible prices.   Whole Foods would be an example of a focus-differentiated company.  They found a sizable niche of the consumer market interested in organic food products. Pro-ficiency is a focus-differentiated company.  We win today based upon the uniqueness of our offering and our performance.   We have defined and will continue to lead the clinical trial simulation-based training market segment.   We “help” create differentiation by positioning the creation of our simulation training experiences as analogous to making a movie.  Rather than Project Managers, a generic term from the clinical trial industry, we have Producers.  Apple did not name their support operations in their stores as the help desk, but rather the Genius Bar.  The Apple Store is now the world’s most lucrative per-foot-retailer based upon the anchor of the Genius Bar in their stores. Here is the back story on the Genius Bar at Apple.

Arguably the centerpiece of what became the Apple Store is the Genius Bar, one of Johnson’s ideas. Customers can take private lessons in how to use their new Apple products, or take existing products in for tech support and repairs.

Jobs hated the idea.

“I remember the day I came in and told Steve about the Genius Bar idea and he says, ‘That’s so idiotic! It’ll never work!’” Johnson said. “He said, ‘Ron, you might have the right idea, but here’s the big gap: I’ve never met someone who knows technology who knows how to connect with people. They’re all geeks! You can call it the Geek Bar.’”

“And I said, ‘Steve, kids who are in their 20s today grew up in a very different world. They all know technology, and that’s who’s going to work in the store.’”

The next day, Steve called Apple’s general counsel to trademark the phrase “Genius Bar.”

Pro-ficiency creates simulation experiences that clinical trial investigators rank as best in the world.  In a recent survey of 1352 site investigators, 85.5% rated Pro-ficiency as a superior on-line training experience with 87.2% indicating that the Pro-ficiency simulation experience helped them avoid a protocol deviation.  Think of us as your “Simulation Safety Net”.  While it may not be as clever as “Genius Bar”, we’re confident that our simulation approach is a genius way to approach investigative site training. Don’t settle for inferior education solutions.  Partner with the best.  Pro-ficiency.

The Art and Science of Our Solution

Pro-ficiency is known for creating the category of simulation training for clinical trials by leveraging our film production skills.  Many do not realize that computer science is also an integral part of our product offering.

The “Art” at Pro-ficiency is led by the Simulation Producer for each project.  They facilitate the simulation design through the Pre-Production, Production and Post-Production stages by working with our Emmy-Award winning executive producer.  Our goal is to create a magical, immersive experience for investigators with the goal of elevating their performance for each clinical trial.

Our solution is not complete, however, without our software platform.  The simulation experience is controlled by our simulation composition platform.  This portion of our intellectual property is our secret sauce.  It allows us to link the simulation vignettes to the workflow that supports the simulation journey for each protocol.   

Our secure cloud-based platform gives access to clinical trial investigators once they are invited onto the network for their specific clinical trial.  All that is required for the investigator is an internet connection and a laptop or tablet.   Once logged in, they can see all clinical trials that they are supporting on their MyAgenda page.

Each sponsor is able to monitor progress on a real-time basis across the globe for their clinical trial.  They can determine which sites are ready for enrolling patients and which might struggle with protocol execution.

The beauty of software is that it can be changed in reaction to customer suggestions, technology advances and market demands.  

Take advantage of the Art and Science of the Pro-ficiency solution for your next clinical trial.

How to Recover From Protocol Deviations in 6 Steps

Honesty.  

We must acknowledge as an industry that we have an unacceptable waste in the operations of clinical trials.

The Critical Role of Learning.

The root cause of many clinical trial deviations can be traced directly back to ineffective training.

Embrace the Opportunity to Innovate

COVID has permanently changed business practices across every industry.  Great leaders have embraced the opportunity to innovate!

Research the Power of Simulation

Go online.  Explore how simulation training has transformed other industries.  According to Arnold Barnett, a professor of statistics at M.I.T., flying is now so safe that a traveler could fly every day for an average of 123,000 years before experiencing a fatal crash. 

Schedule a Demonstration of Pro-ficiency

Go to www.proficiency3.wpengine.com and click on the button to schedule a demonstration.  See how Pro-ficiency combines the power of Science and Film Production to create simulations that engage clinical investigative teams and elevate their performance.

Experience the Difference

Experience the ability to monitor in real-time the global investigator engagement in your clinical trial new education experience.  Have a fact-based approach with the Pro-ficiency analytics engine to assess the readiness to start enrolling patients in your clinical trial.  

Post-COVID Clinical Trial Performance Innovations

COVID has presented many challenges to the clinical trial industry over the past year.  These challenges have included:

  • Moving from in-person to virtual events
  • Difficulty in accessing clinical trial enrollees
  • Ability to monitor the readiness of clinical trial investigators
  • Reducing the variability of clinical trial execution

These challenges are not easily addressed by the same approaches leveraged in a pre-COVID operational framework.  The industry must adapt to the global “new normal”. 

The Boston Consulting Group studies the performance of companies relative to their commitment to innovation.  They identified five key attributes to driving innovation for a company.   Those include:

  1. Setting a clear ambition
  2. Establishing the right playing fields
  3. Prioritizing performance management
  4. Pushing project management
  5. Developing talent and culture

Pro-ficiency can be a key partner in improving clinical trial performance on all five dimensions.

Setting a clear ambition

Our customers can improve clinical trial performance while simultaneously saving money by moving to virtual simulation training.  There is a reason why all of the critical job functions around the world learn via simulation rather than lectures.  Would any of us want our airline pilot trained via PowerPoint trapped on a Zoom call or in a hotel ballroom?  COVID has been a tipping point for sponsors to elevate their clinical trial performance by moving to simulation training through a partnership with Pro-ficiency.

Establishing the right playing field

Sponsors need to adapt to the changing needs of their investigators in a post COVID world.  Restrictive travel, shifting staff assignments and new on-site clinical demands are impacting the availability of investigators. Innovative sponsors are adapting to this new playing field by leveraging Pro-ficiency.  Sponsors entice more clinical investigators by offering the convenience and engagement of virtual simulation training.  New staff can be quickly trained on the protocol without being a drag on the productivity of the existing staff.  Pro-ficiency offers an on-demand training platform by only requiring a computer and an internet connection.

Prioritizing performance management

Leveraging clear, consistent, global KPIs (Key Performance Indicators) are the key to better execution of clinical trials.  Pro-ficiency enables sponsors to gain insights on the readiness investigators and sites by providing real-time insight to their retention and understanding of protocol.  Empirically, this has been shown to reduce variation of practice and to reduce the potential for protocol amendments.

Pushing project management

Rather than jetting around the country and the globe, the clinical trial operations team can focus their energies on managing the launch of the trial.  This not only focuses the activities of the operations team but also minimizes the expense and downtime of travel.  The real-time tools from Pro-ficiency allow the team to quantitatively assessment assess the readiness of the global clinical trial investigator community.

Developing talent and culture

Our customers benefit from the talent assembled by Pro-ficiency in the disciplines of producing, creative direction, script writing, studio production, animation, video editing and the building of the simulation training to support their trial.  Pro-ficiency becomes the creative extension of your clinical team as the simulation training is created.  Once we develop the first simulation training, our sponsors consistently return to our team to partner with them on new clinical trial launches.

COVID has been a drag on parts of society while simultaneously unleashing innovations.  Let Pro-ficiency help you innovate your clinical trial performance.

Learning Is an Experience

 

“Learning is an experience.  Everything else is just information”

 

 

Albert Einstein

Pro-ficiency is poised to play a key role in improving the execution of clinical trials through our immersive learning platform.   Simulation is proven as a learning approach by engaging clinical investigators in real-world situations.  As Einstein said, learning is an experience.  

Fatimah Lateef, from Singapore General Hospital, has proven that simulation is the most effective way to develop health professionals’ knowledge, skills, and attitudes, while protecting patients from unnecessary risks.  He also points out in his article, Simulation-based learning: Just like the real thing, that healthcare should embrace simulation training tools just like the fields of aviation and military.

We all watched in amazement as Captain Chelsey Sullenburger flawlessly landed the Airbus A320-214 on the Hudson River after experiencing total engine failure on takeoff from LaGuardia Airport.  Captain Sullenburger attributed his ability to execute in this life-or-death situation to having performed this maneuver many times in a flight simulator.  He had the ability to fail and learn before needing to execute flawlessly in real-time.

The following month, there was a more tragic end to the same scenario in a flight approach to the Buffalo Niagara International Airport.   Captain Marvin Renslow reacted to an impending stall by pulling up the nose of the airplane.   The result was a pivot toward the ground, crashing into a house, killing an occupant and everyone on board.  Captain Renslow received his training in a classroom session and did not have access to a flight simulator.

A recognized specialist in instructional design and technical training, Dr. Ruth Colvin Clark, suggested (Clark 2010) that although Captain Marvin Renslow’s didactic training may have resulted in knowledge transfer, it did not result in an improvement in skill. Although information was learned, appropriate procedures and actions were not retrievable in a crisis situation. Clark states, “Simulator training would have more effectively implanted performance relevant retrieval cues and enough simulator training would have led to automaticity of response”. (Clark 2010) Appropriate simulation training may have allowed the pilot to respond automatically with the correct, albeit, counterintuitive response of nosing down when the plane was about to stall rather than pulling up. 

Just as Einstein noted years before, learning is an experience.  Everything else is just information.

Let’s empower clinical trial investigators to operate at their peak efficiency via the simulation experience with Pro-ficiency in your next trial.

Enrolled Subject Training

Clinical trials are becoming increasingly patient-centric. Pro-ficiency has trained tens of thousand of patients in multiple therapeutic areas using it’s ground breaking Pro Patient online training system. Now Pro Patient is available for Enrolled Subject Training.

Subjects are expensive to identify and enroll. It only makes sense to support them throughout the study process to keep them informed and engaged.

Recent studies suggest that only 50% of information provided by Health Care Providers (HCP) is retained by patients, and depending on the medical condition, 40-80% can be forgotten immediately after speaking to the HCP. Supplementing verbal information with written, graphical, and video materials significantly improves knowledge retention, treatment compliance, and patient satisfaction, which, in turn, has a compelling positive impact on patient health behaviors and health status.1

Additional studies “have identified both knowledge (information) and attitudes (motivation) as major factors that influence behavior related to therapy adherence. For instance, being afraid and feeling asymptomatic were prominent reasons for inadequate follow-up care and medication adherence.”2

Issues such as miscommunication, protocol confusion, and medication adherence have the potential to significantly increase dropouts and lost-to-follow-ups. In the most extreme cases they may derail a trial altogether.
es such as miscommunication, protocol confusion, and medication adherence have the potential to significantly increase dropouts and lost-to-follow-ups. In the most extreme cases they may derail a trial altogether.

ProPatient is a simulation-based training product for study subjects and their caregivers. It improves patient engagement by training patients and their caregivers about the goals of the study, its risks, their role in the study, and adherence strategies. This also meets emerging regulations requiring that evidence is collected that patients understand the study considerations and informed consent in addition to just agreeing to it.

Pro-ficiency has developed a deep bench of patient advocacy group partners and will engage with such partners where and when appropriate to help with protocol feasibility from the patient perspective and recruitment.

Make Your Studies More Proficient

Pro-ficiency Simulation Training

Pro-ficiency is a simulation-based training platform and site-performance management system. Pro-ficiency mirrors site challenges through simulations of potential subjects to ensure sites are fully trained on the mission-critical aspects of protocols. Throughout the training, Pro-ficiency measures site knowledge, skill and engagement. When skill gaps are detected, Pro-ficiency launches a corrective action.

What is Pro-ficiency?

  • Turnkey online platform for all site training
  • Simulations of potential subjects mirrors real life site challenges
  • Captures real-time performance metrics on site readiness, engagement and capability
  • Choose your own adventure-style experience consisting of 100+ 10 to 30 second videos.
  • Skill gaps identified and corrective actions implemented automatically

Pro-ficiency Data dashboards provide real-time reporting on site readiness, strengths and weaknesses, and identifies your site heroes.

The Pro-ficiency Online Platform will transform your training from a burdensome cost center into a powerful human performance management system.

Turnkey Solution:

  • Simulation-based training promotes activated and “deep” learning
  • FDA CFR 21 820/11 and ICH/GCP compliant system and content
  • Can replace or augment traditional Investigator Meetings
  • Training available at any time for any site user (and new site staff)
  • “Real-world,” simulated clinical challenges keep sites engaged and demonstrates their understanding
  • Easily identify engaged sites and Site Heroes
  • Analytics predict errors, engagement, and capability
  • Training audit trail and smart-certificates (auto expiration and renewal)
  • GCP, PV, EDC, Disease State, IP, SOP, Policy, Placebo Effect modules
  • Makes complex amendment management simple
  • Manage site staff turnover through simple dashboards
  • Subtitles in multiple languages available as needed

 

Is Your Trial Training Compliant With FDA and ICH Regulations?

Are you outsourcing any of your protocol, GCP, or Safety training for investigations and site staff?

Training is important for study quality and patient safety. There is no point in investing tens of millions in a clinical trial and not training the sites appropriately.

Many training outsourcing opportunities exist from CROs and training companies to assist you with training your site staff which are based on old and ineffective technology, and are not validated according to FDA and ICH standards.

Training can impact people’s decisions which can impact subjects in turn. In if you are providing training in support of a GxP activity, you must provide evidence that the person creating the content was qualified or provide evidence that the person noted actually completed the training. This can only be verified through an audit trail of a validated system.

The Pro-ficiency online platform and the simulation-based training content housed there is fully validated and audit-ready according to CFR 21 parts 820 and 11 and applicable sections of ICH GCP E6(R2).

  • Use the best training for your sites
  • Play it safe by using validated systems and content that meets CFR 21 820/11 and ICH E6(R2).
  • Protect patient safety by validating site training content before it is delivered

Predictive Analytics (real-time access)

The Pro-ficiency dashboard shows investigator performance in the training simulations across dozens of critical protocol and GCP metrics (the vertical columns). A blue checkmark indicates that a specific decision was made appropriately. If the learner makes a mistake, then a brief video-based corrective action is launched and they get a yellow icon. If the learner makes that mistake again, they get red icon. Lots of yellow icons indicate that, while knowledge gaps were identified, they were also easily corrected. Lots of red icons means those investigators are either not trying or they are having real problems understanding the protocol.

Based on the performance of the 2 sites above, which one would you trust more with your protocol and expect better performance from?