From Competition to Collaboration: precisionFDA Challenges

By: Taha Kass-Hout, M.D., M.S., Zivana Tezak, Ph.D., and Elaine Johanson

Next week, we will announce the winners of the first precisionFDA challenge.

Taha Kass-Hout

Taha A. Kass-Hout, M.D., M.S., FDA’s Chief Health Informatics Officer (CHIO) and Director of FDA’s Office of Health Informatics

We set up precisionFDA as an online, cloud-based, virtual research space to allow scientists from academia, industry, health care organizations, and government to work together on creating tools to evaluate a method of “reading” DNA known as next generation sequencing (or NGS). The ultimate goal of precisionFDA is to foster innovation and develop regulatory science around NGS tests, which are essential to achieving the promise of President Obama’s Precision Medicine Initiative. So far, the response to precisionFDA has been gratifying, with participants calling precisionFDA “refreshing”, “innovative,” and “a new paradigm in regulatory science.” The community currently boasts more than 1,500 users from 600 organizations, with more than 10 terabytes of genetic data stored.

To engage users and encourage sharing of data and ideas, precisionFDA has offered two competitions to date. These competitions are motivating community members to demonstrate the effectiveness of their tools, test the capabilities of the precisionFDA platform, and engage the community in discussions and data analysis that will provide new insights and serve as a comprehensive source of information about reference data and software pipelines used to analyze sequencing results.

Zevana Tezak

Zivana Tezak, Ph.D., Associate Director for Science and Technology at FDA’s Office of In Vitro Diagnostics and Radiological Health, Center for Devices and Radiological Health

The first precisionFDA challenge, the Consistency Challenge, closed in April 2015, with 21 entries from 17 submitters. Participants were given two datasets of whole genome sequences from a known human sample, sequenced at two different sites and generously donated to precisionFDA by Garvan Institute of Medical Research and Human Longevity, Inc. Challenge participants were instructed to use the informatics pipeline (software) of their choice to identify genetic variants and check for consistency between results in the provided datasets. The top performers will be announced in person by FDA Commissioner Robert Califf on May 25, 2016. At the time of his announcement, we will post detailed information here on all the recognitions and how the top performers were selected. So stay tuned!

The second challenge, the Truth Challenge, closes on May 26, 2016, and was designed in collaboration with the Genome in a Bottle consortium and the Global Alliance for Genomics and Health. Participants are expected to identify genetic variants in one known and one unknown sample dataset. The goal is to see how close they come to the truth when analyzing data from a human sample with variant results unknown to them, which we will reveal at the end of the challenge. An exciting characteristic of this challenge is that the Genome in a Bottle consortium will release for the first time new high confidence variant calls for the unknown sample dataset (we refer to in this challenge as “truth dataset”) for the human sample at the end of the contest.

Elaine Johanson

Elaine Johanson, precisionFDA Project Manager and Deputy Director of FDA’s Office of Health Informatics.

The knowledge generation potential for precisionFDA is immense, and leverages the collaboration of the global community to help solve challenges that will ultimately provide insight to regulation to ensure the safety and efficacy of genetic tests. The platform offers users the ability to assemble and run apps, learn from other experts, share lessons learned, participate in competitions, and help to develop standards for assessing the effectiveness of genetic tests.

With new challenges and opportunities for ongoing discussions and collaborations between FDA and the global community, we look forward to the precisionFDA community facilitating and advancing development and assessment of new tools and tests in this fast growing field of genetic testing.

 

Taha A. Kass-Hout, M.D., M.S., is FDA’s Chief Health Informatics Officer (CHIO) and Director of FDA’s Office of Health Informatics

Zivana Tezak, Ph.D., is Associate Director for Science and Technology at FDA’s Office of In Vitro Diagnostics and Radiological Health, Center for Devices and Radiological Health

Elaine Johanson, is precisionFDA Project Manager and Deputy Director of FDA’s Office of Health Informatics

FDA Launches precisionFDA to Harness the Power of Scientific Collaboration

By: Taha A. Kass-Hout, M.D., M.S. and Elaine Johanson

Imagine a world where doctors have at their fingertips the information that allows them to individualize a diagnosis, treatment or even a cure for a person based on their genes. That’s what President Obama envisioned when he announced his Precision Medicine Initiative earlier this year. Today, with the launch of FDA’s precisionFDA web platform, we’re a step closer to achieving that vision.

Taha Kass-Hout

Taha A. Kass-Hout, M.D., M.S., Chief Health Informatics Officer and Director of FDA’s Office of Health Informatics.

precisionFDA is an online, cloud-based, portal that will allow scientists from industry, academia, government and other partners to come together to foster innovation and develop the science behind a method of “reading” DNA known as next-generation sequencing (or NGS). Next Generation Sequencing allows scientists to compile a vast amount of data on a person’s exact order or sequence of DNA. Recognizing that each person’s DNA is slightly different, scientists can look for meaningful differences in DNA that can be used to suggest a person’s risk of disease, possible response to treatment and assess their current state of health. Ultimately, what we learn about these differences could be used to design a treatment tailored to a specific individual.

The precisionFDA platform is a part of this larger effort and through its use we want to help scientists work toward the most accurate and meaningful discoveries. precisionFDA users will have access to a number of important tools to help them do this. These tools include reference genomes, such as “Genome in the Bottle,” a reference sample of DNA for validating human genome sequences developed by the National Institute of Standards and Technology. Users will also be able to compare their results to previously validated reference results as well as share their results with other users, track changes and obtain feedback.

Elaine Johanson

Elaine Johanson, precisionFDA Project Manager.

Through such collaboration we hope to improve the quality and accuracy of genomic tests – work that will ultimately benefit patients.

Over the coming months we will engage users in improving the usability, openness and transparency of precisionFDA. One way we’ll achieve that is by placing the code for the precisionFDA portal on the world’s largest open source software repository, GitHub, so the community can further enhance precisionFDA’s features.

precisionFDA leverages our experience establishing openFDA, an online community that provides easy access to our public datasets. Since its launch in 2014, openFDA has already resulted in many novel ways to use, integrate and analyze FDA safety information. We’re confident that employing such a collaborative approach to DNA data will yield important advances in our understanding of this fast-growing scientific field, information that will ultimately be used to develop new diagnostics, treatments and even cures for patients.

Taha A. Kass-Hout, M.D., M.S., is FDA’s Chief Health Informatics Officer and Director of FDA’s Office of Health Informatics. Elaine Johanson is the precisionFDA Project Manager.

FDA Taking Genomic Testing to the Next Level

By: Adam Berger and Zivana Tezak

President Obama’s Precision Medicine Initiative (PMI) envisions a day when the specific differences between people – genetic, environmental, lifestyle – will be used to customize the healthcare that we receive. Many of the current efforts toward achieving this goal have focused on analyzing and interpreting a person’s unique genetic makeup, including the identification of genetic alterations that may impact his or her health.

Adam Berger

Adam C. Berger, Ph.D., is Senior Staff Fellow on the Personalized Medicine Staff at FDA’s Office of In Vitro Diagnostics and Radiological Health, Center for Devices and Radiological Health

Next generation sequencing (NGS) technologies have significantly advanced the ability to derive more comprehensive genetic information on individuals in a relatively inexpensive and fast manner. In order to help achieve the goals of the PMI, FDA is developing new regulatory strategies for NGS-based clinical tests. We aim to ensure that these tests provide accurate, reproducible, and meaningful results relevant to a person’s medical condition while continuing to foster innovation so that people have access to the best available results generated by the most cutting-edge medical technologies.

In December 2014, we issued a preliminary discussion paper describing how FDA might go about creating a modern, flexible and dynamic regulatory system for NGS, which could potentially be applied to many other types of genomic tests. We expanded on this discussion by holding a workshop in February 2015 with a variety of stakeholders and received many helpful comments. To further advance this stakeholder conversation, we are holding two back-to-back public workshops on November 12th and 13th, 2015. The first will focus on analytical performance evaluation standards, including potential ways to develop these standards, which can be used by test developers to ensure their tests produce accurate and reliable results. We believe that stakeholder input about ongoing community standardization efforts is essential to construct flexible analytical evaluation approaches for a wide variety of NGS tests and modifications.

Zevana Tezak

Zivana Tezak, Ph.D., is Associate Director for Science and Technology at FDA’s Office of In Vitro Diagnostics and Radiological Health, Center for Devices and Radiological Health

The second workshop will address current challenges in clinical validation of NGS tests. A single company, lab, or institution is unlikely to have enough information to definitively determine the clinical importance of test results. The aggregation of clinical information in curated databases will create a “data commons” that could serve as a reliable source of scientific evidence that test developers could use to demonstrate that NGS test results are relevant to a person’s disease or outcome. The workshop will highlight how scientists, patient groups, and private industry can work together to develop high-quality, curated clinical databases of genomic information that associate specific genetic changes with various diseases, such as cardiovascular disease or diabetes.

In advance of these workshops, FDA will be releasing additional discussion papers informed by public input we have received, which will also include some general questions for stakeholder consideration. These documents will provide a high level overview of regulatory considerations for the development of analytical standards and the use of curated clinical databases to support NGS test submissions.

We look forward to engaging with the public on these important issues to ensure that advances in precision medicine rapidly turn into treatments that benefit everyone.

For more information:

President’s Precision Medicine Initiative

Sign up for the workshops or related webinars:

“Standards-Based Approach to Analytical Performance Evaluation of Next Generation Sequencing In Vitro Diagnostic Tests”

“Use of Databases for Establishing the Clinical Relevance of Human Genetic Variants”

Adam C. Berger, Ph.D., is Senior Staff Fellow on the Personalized Medicine Staff at FDA’s Office of In Vitro Diagnostics and Radiological Health, Center for Devices and Radiological Health

Zivana Tezak, Ph.D., is Associate Director for Science and Technology at FDA’s Office of In Vitro Diagnostics and Radiological Health, Center for Devices and Radiological Health

Advancing precision medicine by enabling a collaborative informatics community

By: Taha A. Kass-Hout, M.D., M.S., and David Litwack, Ph.D.

FDA plays an integral role in President Obama’s Precision Medicine Initiative, which foresees the day when an individual’s medical care will be tailored in part based on their unique characteristics and genetic make-up. Yet while more than 80 million genetic variants have been found in the human genome, we don’t understand the role that most of these variants play in health or disease. Achieving the President’s vision requires working collaboratively to ensure the accuracy of genetic tests in detecting and interpreting genetic variants. We are working towards that goal by developing an informatics community and supporting platform we call precisionFDA.

Taha Kass-Hout

Taha A. Kass-Hout, M.D., M.S., FDA’s Chief Health Informatics Officer and Director of FDA’s Office of Health Informatics.

Sophisticated, relatively inexpensive technology known as next generation sequencing (NGS) already exists to sequence a person’s genome quickly. Developers and users of NGS tests must then comb these sequences to look for segments that suggest potentially meaningful differences and determine whether those differences provide useful and actionable information about the state of a person’s health, and their future risk of disease, behavior, or treatment choices.

Special features of this technology pose novel regulatory issues for FDA. Most diagnostic tests follow a one test-one disease paradigm that readily fits FDA’s current device review approaches for evaluating a test’s accuracy and clinical interpretation. Because NGS tests may be used in many ways in the clinic and can produce an unprecedented amount of data about a patient, we are working to evaluate whether a better option might simply be requiring each NGS test developer to show that the test meets certain standards for quality. Similarly, to demonstrate a test’s clinical value, we are assessing whether it may be more efficient for developers to refer to evidence in well-curated, validated, and shared databases of mutations instead of independently generating data to support a mutation-disease association.

David Litwack

David Litwack, Ph.D., Policy Advisor, Office of In Vitro Diagnostics and Radiological Health, at FDA’s Center for Devices and Radiological Health.

To begin to realize this new vision, precisionFDA is designed as a crowd-sourced, cloud-based platform to advance the science needed to develop the necessary standards. PrecisionFDA will supply an environment where the community can test, pilot, and validate new approaches. For example, NGS test developers, researchers, and other members of the community can share and cross-validate their tests or results against crowd-sourced reference material in precisionFDA.

Planned for beta release (work in progress) in December 2015, precisionFDA will offer community members access to secure and independent work areas where, at their discretion, their software code or data can either be kept private, or shared with the owner’s choice of collaborators, FDA, or the public. Initially, precisionFDA’s public space will offer a wiki and a set of open source or open access reference genomic data models and analysis tools developed and vetted by standards bodies, such as the National Institute of Standards and Technology (e.g., Genome in a Bottle). We believe precisionFDA will help us advance the science around the accuracy and reproducibility of NGS-based tests, and in doing so, will advance consumer safety. We look forward to continuing to update the community on the development of these new tools.

Taha A. Kass-Hout, M.D., M.S., is FDA’s Chief Health Informatics Officer and Director of FDA’s Office of Health Informatics.

David Litwack, Ph.D., is Policy Advisor, Office of In Vitro Diagnostics and Radiological Health, at FDA’s Center for Devices and Radiological Health.

Developing new tools to support regulatory use of “Next Gen Sequencing” data

By: Carolyn A. Wilson, Ph.D.

When you’re thirsty, you don’t want to take a drink from a fire hose. And when scientists are looking for data they don’t want to be knocked over with a flood of information that overwhelms their ability to analyze and make sense of it.

Carolyn WilsonThat’s especially true of data generated by some types of both human and non-human genome research called Next Generation Sequencing (NGS). This technology produces sets of data that are so large and complex that they overwhelm the ability of most computer systems to store, search, and analyze it, or transfer it to other computer systems.

The human genome comprises about 3 billion building blocks called nucleic acids; much medical research involves analyzing this huge storehouse of data by a process called sequencing—determining the order in which the nucleic acids occur, either in the entire genome or a specific part of it. The goal is often to find changes in the sequence that might be mutations that cause specific disease. Such information could be the basis of diagnostic tests, new treatments, or ways to track the quality of certain products, such as vaccines made from viruses.

NGS is a complicated technique, but basically it involves cutting the genome into millions of small pieces so you can use sophisticated chemical tricks and technologies to ignore the “junk” you don’t need, and then make up to hundreds of copies of each of the pieces you want to study. This enables additional techniques to identify changes in the sequence of nucleic acids that might be mutations. NSG enables scientists to fast-track this process by analyzing millions of pieces of the genome at the same time. For comparison, the famous human genome sequencing and analysis program that took 13 years to complete and cost $3 billion could now be completed in days for a few thousand dollars.

Man with HIVE Computer

The Center for Biologics Evaluation and Research (CBER) supported the development of High-Performance Integrated Virtual Environment (HIVE) technology, a private, cloud-based environment that comprises both a storage library of data and a powerful computing capacity being used to support Next Generation Sequencing of genomes.

In order to prepare FDA to review and understand the interpretation and significance of data in regulatory submissions that include NGS, the Center for Biologics Evaluation and Research (CBER) supported the development of a powerful, data-hungry computer technology called High-Performance Integrated Virtual Environment (HIVE), which can consume, digest, analyze, manage, and share all this data. HIVE is a private cloud-based environment that comprises both a storage library of data and a powerful computing capacity. One specific algorithm (set of instructions for handling data) of HIVE that enables CBER scientists to manage the NGS fire hose is called HIVE-hexagon aligner. CBER scientists have used HIVE-hexagon in a variety of ways; for example, it helped scientists in the Office of Vaccines Research and Review study the genetic stability of influenza A viruses used to make vaccines. The scientists showed that this powerful tool might be very useful for determining if influenza viruses being grown for use in vaccines were accumulating mutations that could either reduce their effectiveness in preventing infections, or even worse, cause infections.

There’s another exciting potential to HIVE-hexagon research: the more scientists can learn about variations in genes that alter the way they work—or make them stop working–the more they can help doctors modify patient care to reflect those very personal differences. These differences can affect health, disease, and how individuals respond to treatments, such as chemotherapy and influenza vaccines. Such knowledge will contribute to advances in personalized medicine.

Team members at work in FDA's HIVE server room.

CBER scientists showed that HIVE might help scientists determine if influenza viruses being grown for use in vaccines were accumulating mutations that could either reduce their effectiveness in preventing infections or cause infections. Genome studies supported by HIVE will also contribute to advances in personalized medicine.

Because CBER’s HIVE installation has been so successful we are now collaborating with FDA’s Center for Devices and Radiological Health (CDRH) to provide a second installation with greater capacity and computer power that takes advantage of the high-performance computing capacity there. When ready and approved by FDA for use, we will use this powerful, CBER-managed, inter-center resource to handle regulatory submissions.

HIVE-hexagon and its innovative NGS algorithms are just one major step CBER has taken recently as it continues its pioneering work in regulatory research to ensure that products for consumers are safe and effective. I’ll tell you about other exciting breakthroughs in my next update on CBER research.

Carolyn A. Wilson, Ph.D., is Associate Director for Research at FDA’s Center for Biologics Evaluation and Research.

For more HIVE photos go to Flickr