February 25, 2005
To:Jeffrey Murray, MD & Kimberly Struble, MD
U.S. Department of Health and Human Services
Food and Drug Administration
Center for Drug Evaluation and Research (CDER)
From:Robert W. Shafer, M.D.
Division of Infectious Diseases and Geographic Medicine
Stanford University
Re:Comments and suggestions regarding the draft guidance document entitled "Guidance for Industry Role of HIV Drug Resistance Testing in Antiretroviral Drug Development" (Docket number 2004D-0484)

Antiretroviral drug development has been a qualified pharmaceutical success story. The FDA has approved eight nucleoside reverse transcriptase (RT) inhibitors (NRTIs), seven protease inhibitors (PIs), three nonnucleoside RT inhibitors (NNRTIs), and one fusion inhibitor. Four fixed-dose NRTI combination formulations and several modified versions of existing drugs with improved bioavailability have also been approved. Additional NRTIs, NNRTIs, and PIs are in advanced clinical development and several entry inhibitors are in the early stages of clinical development.

Success in treating HIV-1-infected persons is due not just to the large numbers of approved drugs but also to the many clinical trials that have identified optimal drug combinations for the initial treatment of HIV-1 infection. But this success has come at a cost: an epidemic of drug resistance has been brought on in part by the years it took to develop highly active combinations as well as by repeated underestimations of the extent of cross resistance to newly approved drugs. These underestimations led to false expectations of drug activity when treating persons who had failed previous regimens, resulting in the use of insufficiently potent salvage therapy regimens, treatment failure, and increasing drug resistance when new but highly cross-resistant drugs were used for salvage therapy.

It is therefore critical that drug resistant data be considered when new drugs are developed. An FDA guidance document on the role of HIV drug resistance testing in antiretroviral drug development is appropriate - and overdue. The draft of the guidance document contains recommendations addressing two major types of studies: pre-clinical (section IV) and clinical (section V). There is an appendix with suggestions on how to format drug-resistance data when submitting it to the FDA, and the draft also mentions each of the resistance studies that should be done during drug development. However, there is one pivotal recommendation that should be added to the document, which would increase the document's utility and render immediate benefits for patients and their physicians and long-term benefits for drug developers: The drug resistance data submitted to the FDA as part of a new drug application should be made publicly available!

In order to optimally use a new antiretroviral drug it is essential to have sufficient knowledge of how genetic changes in the therapeutic target (genotypic data) affect in vitro drug susceptibility (phenotypic data) and how the virus responds to the inclusion of the drug in a new treatment regimen (clinical data). Although correlations between phenotype and the patient's clinical outcome are also important, most clinicians and turn to genotypic resistance tests to assess the spectrum and level of drug resistance when a patient fails therapy or is infected primarily with a drug-resistant strain.

However, these essential correlations between genotype and phenotype and genotype and virological response, despite being presented to the FDA, are rarely made publicly available. Indeed, the absence of such data has led to a situation in which there are comprehensive, specific guidelines for the initial treatment of HIV-1 but only vague and sometimes contradictory recommendations for treating persons who have failed an initial regimen or have been primarily infected with a drug-resistant HIV-1 strain (Table 1). Treatment of such persons is often a process of trial and error, which increases a patient's risk of treatment failure, drug toxicity, and development of increased drug resistance.

This astounding incongruity - between the guidelines for treating drug-susceptible as opposed to drug-resistant HIV-1 - developed because the standard drug-development paradigm does not generate results robust enough for developing guidelines for treating patients with drug-resistant virus. Because HIV-1 develops drug resistance through a variety of genetic mechanisms, the development of drug resistance segregates patients into a large number of different strata and makes it impossible for any one study to identify optimal treatments for all or even most subsets of patients. Therefore, HIV-1 drug resistance - like immunological escape from vaccine-induced immunity - looms as a problem that cannot be solved by any single study or clinical trial but only by the effective synthesis of data from multiple studies and clinical trials.

It is a basic principle of scientific publication that the data supporting the results of a paper be made publicly available. This allows others to confirm the original analysis, to conduct additional analyses, and, most importantly, to combine the results with those of other studies. Unfortunately, much of the data presented to the FDA is either never published or, when published, consist of aggregate data that severely curtails the number and type of possible analyses. There is no unsurmountable obstacle or justifiable excuse for this practice. The correlations between genotype and phenotype and between genotype and clinical outcome provide part of the basis for drug approval and are essential for knowing when a new drug should be used. These data do not constitute trade secrets and do not contain confidential patient information. As Rennie and others have argued about clinical trials data: "they are essential information for patients and their physicians - gleaned from studying patients who might have been less willing to participate had they known their results would be treated as trade secrets and often never made public1."

The Stanford HIV RT and Protease Sequence Database (HIVRT&PrDB, http://hivdb.stanford.edu)2,3 is a publicly available online database with the following goals: (1) To represent, store, and analyze the diverse forms of data underlying drug resistance knowledge and to make these data available to the broad community of researchers studying HIV drug resistance and clinicians using HIV drug resistance tests. (2) To provide a publicly available online resource to help those developing guidelines for the interpretation of HIV drug resistance testing. (3) To provide a publicly available online resource to help those developing new antiretroviral drugs that are less likely to trigger resistance or are that are effective against drug-resistant HIV strains. (4) To identify gaps in HIV drug resistance knowledge that could be filled by retrospective or prospective studies.

The two main types of data in HIVRT&PrDB are (i) correlations between HIV-1 protease and RT sequences and the treatments received by persons from whom the sequences were obtained; and (ii) correlations between HIV-1 protease and RT sequences and drug susceptibility data. Correlations between genotype, treatment, and virological outcome have been collected from several well-characterized patient populations in California, published papers, and two ACTG studies as part of an ACTG Data Analysis Concept Sheet. A section has been created on the database website to allow users to download the complete set of resistance data associated with a clinical trial or published study (Appendix 2). This site will become publicly available pending review and approval by the researchers who have submitted these data. Our group and at least one other group have created standard approaches to representing the types of data described in this letter.

Appendix 1 contains comments about specific types of data that are discussed in the guidance document. Appendix 2 contains brief descriptions of the data that should be made publicly available. The overriding principle of both appendices is that the raw data must be available in a format that allows others to reproduce submitted analyses as well as to pool the data in an analysis with data obtained from other non-pharmaceutical studies.

In accordance with the best interests of public health and the principles of scientific publication, the drug-resistance data described in the guidance document and outlined in detail in the appendices should be made publicly available upon FDA approval of a drug. This data can be stored on the website of the pharmaceutical sponsor, the FDA, or on a specialized site designed to represent these types of data such as the HIVRT&PrDB.

Table 1. DHHS Panel Recommendations for Changing an Antiretroviral Regimen for Virological Failure*

  • For the patient with virologic failure, perform resistance testing while the patient still is taking the drug regimen or within 4 weeks after regimen discontinuation (AII).
  • Use the treatment history and past and current resistance test results to identify active agents (preferably 3 or more) to design a new regimen (AII).
  • If three active agents cannot be identified, consider pharmacokinetic enhancement of protease inhibitors (with the exception of nelfinavir) with ritonavir (BII) and/or re-using other prior antiretroviral agents (CIII).
  • Adding a drug with a new mechanism of action (e.g. HIV entry inhibitor) to an optimized background antiretroviral regimen can add significant antiretroviral activity (BII).
  • In general, one active drug should not be added to a failing regimen because drug resistance is likely to develop quickly (DII). However, in patients with advanced HIV disease (e.g. CD4 <100) and higher risk of clinical progression, adding one active agent (with an optimized background regimen) may provide clinical benefits and should be considered (CIII).
*Page 25 of the DHHS Guidelines (October 29, 2004). Evidence basis is provided in parenthesis with detailed explanation available on page 40 of the Guidelines.

  1. Rennie D. Trial registration. A great idea switches from ignored to irresistible. JAMA 2004;292:1359-62.

  2. Kuiken C, Korber B, Shafer RW. HIV Sequence Databases. AIDS Reviews 2003;5:56-65.

  3. Rhee SY, Gonzales MJ, Kantor R, Betts B, Ravela J, Shafer RW. HIV reverse transcriptase and protease sequence database, 2003. Nucleic Acid Res 2003;31:298-303.

  4. Jie Zhang, Soo-Yon Rhee, Jonathan Taylor and Robert W. Shafer (2005). Comparison of the Precision and Sensitivity of the Antivirogram and PhenoSense HIV Drug Susceptibility Assays. JAIDS 38(4):439-444.

  5. Johnston E, Dupnik KM, Gonzales MJ, Winters MA, Rhee SY, Imamichi T and Shafer RW (2005). Panel of prototypical infectious molecular HIV-1 clones containing multiple nucleoside reverse transcriptase inhibitor resistance mutations. AIDS 19(7):731-3.

Appendix 1. Comments on Types of Drug-Resistance Data Obtained During Antiretroviral Drug Development

PreclinicalRationale and Comments
In vitro passage experimentsRationale: (i) Development of virus mutations confirms that the drug specifically inhibits HIV-1 rather than the growth of cells that HIV-1 requires for replication.
(ii) Identifies one or more of the genetic mechanisms of resistance (i.e. mutations), making it possible to compare these mechanisms to those associated with resistance to other drugs of the same class.
(iii) Determines the time it takes for resistance to emerge in vitro.
(iv) Determines if the drug-resistant variants replicate less well than wild-type variants.

Comment on the clinical significance of in vitro passage experiments: The development of mutations that differ from those associated with other drugs does not guarantee the absence of cross-resistance. Mutations that emerge during in vitro passage are clearly sufficient to cause resistance but are rarely the only mechanism by which resistance can develop. Mutations that emerge in vitro usually represent only a subset of the mutations that emerge in vivo.

Drug susceptibility testingRationale: (i) Drug susceptibility testing of wild-type isolates belonging to different HIV-1 subtypes defines the biological variability of a drug's anti-HIV-1 activity.
(ii) Drug susceptibility testing of isolates emerging during in vitro passage experiments quantifies the relative effects of the mutations required for virological escape in vitro and whether isolates with these mutations can be suppressed by other drugs.
(iii) Drug susceptibility testing of isolates with mutations conferring resistance to other drugs determines whether viruses resistant to currently approved drugs will also decrease susceptibility to the new drug.

Comment on choice of susceptibility assay: The biological and clinical significance of the IC50 and fold decrease in susceptibility are assay dependent. To interpret these results it is necessary to know (i) the technical variability of an assay; (ii) the range in susceptibilities observed for wild type isolates ("biologic" variability); (iii) the dynamic range in susceptibility for a drug (highest levels of fold resistance in HIV-1 isolates exposed to selective drug pressure); and (iv) level of resistance that is clinically meaningful. The technical and biologic variability of the PhenoSense and Antivirogram assays have been described but similar data are not available for most other assays often making it difficult to interpret results obtained with these less well-characterized assays. We have also recently reported that for NRTIs with narrow dynamic ranges, the PhenoSense assay is more precise than the Antivirogram assay and as a consequence is more sensitive at detecting resistance to the NRTIs with low dynamic susceptibility ranges4.

Comment of isolates chosen for cross-resistance testing: New compounds that demonstrate in vitro antiretroviral activity are usually tested on a range of drug-resistant clinical isolates. However, because no standard sets of drug-resistant isolates are routinely tested, it is usually not possible to determine the activity of a new drug relative to other experimental and approved drugs. To address this need, we have created a panel of 12 recombinant infectious molecular clones containing combinations of mutations that confer resistance to multiple NRTIs and made these available to the NIH AIDS Research and Reference Reagent Program without restrictions5. We are currently developing a similar panel of multidrug-resistant protease clones.

ClinicalRationale and Comments
Genotypic mechanisms of resistance in patients Rationale: The size and heterogeneity of the HIV-1 population in a laboratory isolate is much lower than that within a person making it impossible to identify the most relevant drug-resistance mutations by performing in vitro passage experiments.

Comment: Most recently approved drugs are used either for salvage therapy or as part of highly effective initial HAART regimens. Virological failure during salvage therapy results from mutations that confer cross-resistance to a new drug but these mutations are not necessarily those that would occur had the new drug been the first drug of its class administered to a patient. Because initial HAART has become increasingly effective, most virological failures on initial regimens result from nonadherence or drug toxicity. In addition, resistance during initial HAART usually emerges only to those drugs with the lowest genetic barriers to resistance. Therefore, data of this type will accrue slowly and post-approval surveillance will be necessary to fully obtain a complete understanding of the mutations selected in vivo by a new drug or drug combination.

Genotypic and phenotypic predictors of virological response Rationale: For a drug to be used in previously treated patients or in patients who have been primarily infected with a drug-resistant HIV-1 strain, it is essential to correlate the pre-therapy genotype (and/or phenotype) with the virological response to the addition of a new drug or to the inclusion of the new drug in a new treatment regimen. Because salvage therapy fails much more commonly than initial therapy, it is possible to obtain much more data in this type of study.

Comment: However, it is difficult to obtain robust conclusions from these studies because the virological response to a new treatment regimen is complicated by the complexity of baseline genotypes (and to a lesser extent phenotypes) and confounded by other parameters that influence response such as (i) baseline treatment history, (ii) other drugs used in combination with a new drug, (iii) baseline plasma HIV-1 RNA level, and (iv) baseline CD4 count. Adherence is also a confounder but data on adherence are usually not available.

Comment: Published studies of this type have used different methods and endpoints. For example, virological endpoints have included both the decrease in virus load and/or the proportion of persons with undetectable viremia by week 4, 12, 24, or 48. A draft summary of these studies can be found in the "Genotype-Clinical Outcome" section of the Stanford HIV Drug Resistance Database. Despite their limitations, these studies are the only clinical data available to clinicians for using genotypic and phenotypic resistance tests to select new drug treatment regimens for patients harboring drug-resistant viruses. Only by making such data publicly available will it be possible to combine the results from a sufficient numbers of patients to learn how to optimally use a new drug for salvage therapy.

Appendix 2: Description of the Data Elements that Should be Made Publicly Available

Type of dataDescriptionRationale
Genotypic dataThe complete nucleic acid sequence in fasta or some other standard text formatThe complete nucleic acid sequence is required to validate the quality of the sequencing, rule out cross-contamination or sample mix-up in a set of sequences, identify the HIV-1 subtype, and unambiguously identify amino acid mutations, including mixtures.
Drug susceptibility (phenotypic) dataThe IC50 and level of fold-resistance for each drug.For correlations between genotype and phenotype, it is only necessary to provide the IC50 and level of fold-resistance for the experimental drug and each approved drug. For correlations between phenotype and clinical outcome, the date should also be provided to be able to correlate the phenotypic result with the subsequent response to therapy.
Treatment historyA complete list of antiretroviral drugs and their dates of administration.The treatment history should consist of the antiretroviral drugs received prior to the experimental drug as well as the antiretroviral drugs received in combination with the antiretroviral drug.
More precision is required for the more recent treatment regimens. For distantly received drugs, it may not always be possible to provide detailed information as to the dates that drugs were received.
Virus loadPlasma HIV-1 RNA level (copies/ml) and date.*Self explanatory.
CD4 countCD4+ lymphocyte (cells/ul) and date.*Self-explanatory.
*Exact dates or relative dates can be used. For example, the date an experimental drug is administered could be labeled day 0 and all previous and subsequent dates should be labeled by the number of weeks preceding or following day 0.