One of the major challenges to the successful development of therapies for Alzheimer’s disease (AD) is the poor translation of preclinical efficacy from animal models to the clinic. A number of key factors have been identified as contributors to the unsuccessful translation of therapeutic efficacy, these include: the failure of the models to fully recapitulate human AD, poor rigor, study design and data analysis, insufficient attention given to using a standard set of “best practices”, failure to match outcome measures used in preclinical animal studies and clinical studies, poor reproducibility of published data and publication bias in favor of reporting positive findings.
To address this challenge and ameliorate some of the factors contributing to the preclinical to clinical gap in the development of AD therapies the National Institute on Aging (NIA) and the National Institutes of Health (NIH) Library have created a publicly available data repository- the Alzheimer’s Preclinical Efficacy Database, or AlzPED. AlzPED is designed as a web-based portal for housing, sharing and mining of preclinical efficacy data. The data are submitted to AlzPED through a curator and gleaned from at least two sources; 1) the scientific literature; 2) directly from researchers. These data include information on AD animal models, therapeutic agents, therapeutic targets, outcome measures, related clinical trials, patents and study design. Most importantly, AlzPED is designed to help identify critical experimental design elements and methodology missing from studies that make them susceptible to misinterpretation and reduce their reproducibility and translational value. Through this capability, AlzPED is intended to guide the development and implementation of strategies and recommendations for standardized best practices for the rigorous preclinical testing of AD candidate therapeutics.
This growing knowledge platform currently houses 720 preclinical efficacy studies published between 1996 and the present (Table 1), collected from databases like PubMed and EMBASE using key word search strings specific to AD. Each study is carefully curated by 2 experts in AD research prior to publication in the database. Efforts are underway to expand the database further and balance the number of studies curated based on the year of publication.
AlzPED identifies 24 experimental design elements that should be included in any preclinical efficacy study to improve its rigor, reproducibility and translational value (Figure 1). Comprehensive analysis of the 720 published preclinical efficacy studies compiled in AlzPED demonstrate considerable variation in the frequency of including and reporting these elements of experimental design. For example, experimental design elements like dose and formulation of the therapeutic agent being examined and treatment paradigms are included and reported with consistency (at 95% or greater) whereas others like power/sample size calculation, blinding, inclusion and exclusion criteria are less frequently reported (less than 10%).
AlzPED further defines 9 core experimental design elements that are critical for ensuring scientific rigor and reproducibility of a preclinical efficacy study, derived from Shineman et al., 2011, Landis et al., 2012, Snyder et al., 2016 and ARRIVE guidelines (Figure 2). These include power/sample size calculation, randomization, blinding for treatment and outcomes, sex as a biological variable and balancing for sex, animal genetic background, conflict of interest statement and inclusion and exclusion criteria. Of these 9 core design elements, there has been an upward trend over the past decade in reporting the sex as a biological variable and genetic background of the animals used in the study as well as a financial conflict of interest statement from authors, though this is reflective of changes in data reporting and publication policies and requirements recommended by federal funding agencies, private foundations and peer-reviewed scientific journals.
Further evaluation of the reporting trends in the 9 core experimental design elements demonstrates that of the 720 preclinical studies in the database 5% report none of the core design elements, 14% report at least 1 core design element, 25% report at least 2, 26% report at least 3, 18% report 4, 8% report 5, 2% report 6 and 1% report 7 core design elements. There are no studies in the database that report 8-9 core design elements currently (Figure 3).
Comparisons of the reporting trends in the 9 core experimental design elements between NIH-funded studies and those funded by non-NIH agencies demonstrate statistically significant differences. NIH-funded studies show a significantly higher frequency of reporting blinding for outcome measures and whether the study is balanced for sex as a biological variable (Figure 4).
Further comparisons of the reporting trends in the 9 core experimental design elements between NIH-funded studies and those funded by pharmaceutical companies also demonstrate statistically significant differences. NIH-funded studies show a significantly higher frequency of reporting blinding for outcome measures, whether the study is balanced for sex as a biological variable and whether inclusion or exclusion criteria for animals used in the study are included (Figure 5).
Within the 720 curated studies compiled in AlzPED, 6 different animal species have been utilized, a majority of which are mouse models of AD (Figure 6). Other animal species include rat, guinea pig, rabbit, dog and non-human primate models of AD. Preclinical efficacy data from 149 different AD animal models are currently available in AlzPED, with new models set to be included as they become available.
Transgenic animal models display considerable variability in the extent and time course of disease phenotype development and expression and consequently demonstrate variable responses to the candidate therapeutic agents being tested. Reporting the genetic background of animal models is yet another important experimental design element that can improve the rigor and reproducibility of preclinical efficacy studies. Detailed analysis of the curated studies compiled in AlzPED exhibit a moderate frequency of mouse model genetic background reporting (Figure 7).
A diverse array of therapeutic agents and targets are reported in the 720 studies curated to AlzPED. The database catalogues 642 novel therapeutic agents into 13 distinct categories (Figure 8) based on agent source (natural product or synthetic), molecular structure (biologic or small molecule), chemical nature (peptide, nucleic acid, or hormone) and mechanism of action (immunotherapy – active or passive).
Currently, AlzPED stores information on 145 therapeutic targets that aim to reduce beta amyloid and tau-related pathology and address disease-associated inflammation, oxidative stress, metabolic, synaptic and behavioral dysfunction. These assorted targets are categorized into amyloidogenic proteins, tau protein, non-amyloid proteins, enzymes, receptors and transporters, metal ions, free radicals and multi target (Figure 9).
Within this diverse group, the most frequently targeted are beta amyloid peptides, beta and gamma secretases, tau protein, cholesterol metabolism regulator HMG CoA reductase, inflammatory response regulating enzyme cyclooxygenase (1 and 2), glucose metabolism regulator peroxisome proliferator-activated receptor gamma (PPAR gamma), and critical neurotransmission and synaptic signaling molecules like NMDA receptors and acetylcholinesterase. Notably, numerous therapeutic agents demonstrate varying extents of anti-inflammatory, anti-oxidant, beta amyloid-reducing, neuroprotective and cognition enhancing properties and are categorized as multi-target therapeutics (Figure 10).
The most frequently targeted amyloidogenic proteins include beta amyloid peptides, oligomers and fibrils (Figure 11).
Currently information on 25 non-amyloid protein targets is available in AlzPED, within which, the most frequently targeted proteins include tubulin, tumor necrosis factor, APOE, synaptic vesicle protein and complement factors (Figure 12).
AlzPED also houses information on 46 enzyme and 60 receptor targets, within which, the most frequently targeted enzymes include beta and gamma secretases, acetylcholinesterase, HMG CoA reductase and cyclooxygenase (Figure 13). The most frequently targeted receptors include NMDA receptors, peroxisome proliferator-activated receptor gamma, nicotinic acetylcholine receptors, retinoid receptors and glucagon-like peptide 1 receptors (Figure 14).
Each curated study provides an individual snapshot of the measures tested and outcomes achieved in response to the therapeutic agent tested. AlzPED defines 20 different outcome measures that are categorized as either functional or descriptive. Functional measures include behavioral, motor, electrophysiological and imaging outcomes (Figure 15). Of these functional measures, behavioral outcomes are most commonly tested.
Within the 68 different behavioral outcomes measured, the Morris water maze, novel object recognition and open field tests are the most frequently studied (Figure 16).
Within the 16 different motor function outcomes measured, locomotor activity, swimming speed and the rotarod test are the most frequently studied (Figure 17).
Within the 50 diverse electrophysiological outcomes measured, long term potentiation (LTP), field excitatory postsynaptic potentials (fEPSP) and input/output ratio are the most frequently studied (Figure 18).
Within the 28 diverse imaging outcomes measured, cerebral blood flow, structural MRI and in vivo two-photon amyloid imaging are the most frequently studied (Figure 19).
Descriptive measures include ADME, biochemical, biomarker, cell biology, electron microscopy, histopathological, immunochemical, immunological, microscopy, omics (proteomics, lipidomics, metabolomics, transcriptomics and others), pharmacodynamic, pharmacokinetic, pharmacological, physiological, spectroscopy and toxicology outcomes (Figure 20).
Within the descriptive measures tested, beta amyloid pathology-related biochemical, histopathological and immunochemical outcomes are a major focus in the studies curated to AlzPED. These measures analyze several species of beta amyloid including soluble, insoluble, monomers, oligomers, fibrils and plaques. Other measures in these categories include evaluation of several species of tau (soluble, insoluble, aggregated, hyperphosphorylated and others), and astrocytic and microglial markers (Figures 21-23).
Notably, even though beta amyloid and tau species, and glial markers are a major focus, an extraordinary range of factors and molecules are investigated within these 3 descriptive measures. In total, information from 910 biochemical, 39 histopathological and 280 immunochemical measures are currently available in AlzPED.
Other frequently studied descriptive measures include biomarkers (Figure 24), toxicology outcomes (Figure 25) and ADME measures (Figure 26). As many as 34 different biomarkers have been analyzed, and beta amyloid markers in plasma, serum or CSF constitute a large proportion. A comprehensive listing of at least 93 toxicology measures such as Ames tests, enzyme profiles, organ histology and others are available in the database as well. Of these, the most frequently evaluated are body weight, general behavior and food intake. Of the 24 ADME measures studied, the most commonly tested are biodistribution, metabolic stability and cytochrome p450 inhibition capability of therapeutic agent.
Similarly, 90 different pharmacodynamic measures are examined with key focus on reducing beta amyloid species (Figure 27). As many as 54 pharmacokinetic measures have been analyzed, and drug concentration in brain and plasma are most frequently evaluated (Figure 28). Of the 7 pharmacological measures studied, the most commonly tested are binding affinity and target selectivity of the therapeutic agent (Figure 29).
AlzPED reports on 12 different physiological measures from which blood pressure and cerebral blood flow are most frequently evaluated (Figure 30). As many as 71 cell biology outcomes are measured, and cell viability and cytotoxicity are the most common measures (Figure 31). Of the 32 immunological measures reported, antibody titers and target specificity are most frequently evaluated (Figure 32). AlzPED also informs on 9 OMICS-related measures such as metabolomics and gene expression profiles (Figure 33).
Finally, AlzPED also reports on 25 electron microscopy outcomes (Figure 34), 46 microscopy outcomes (Figure 35) and 11 spectroscopy outcomes (Figure 36).
In summary, AlzPED provides the AD research community free access to a treasure trove of information pertaining to experimental designs, Alzheimer’s animal models, therapeutic agents and targets, outcome measures, and principal findings from preclinical studies, along with related PubMed and PubChem literature, clinical trials, patents, funding sources and financial conflict of interest. AlzPED is also designed to serve as a platform for reporting unpublished negative findings to mitigate publication bias favoring reporting of positive findings. Researchers can use this resource to survey existing preclinical therapy developments, understand the requirements for rigorous study design and transparent reporting and plan preclinical intervention studies.
- Accelerating drug discovery for Alzheimer’s disease: best practices for preclinical animal studies. Shineman et al., Alzheimers Res Ther. 2011, Sep 8;3(5):28.
- A call for transparent reporting to optimize the predictive value of preclinical research. Landis et al., Nature. 2012, Oct 11;490(7419):187-91.
- Guidelines to improve animal study design and reproducibility for Alzheimer’s disease and related dementias: for funders and researchers. Snyder et al., Alzheimers Dement. 2016, Nov;12(11):1177-1185.
- ARRIVE Guidelines - https://www.nc3rs.org.uk/arrive-guidelines