Understanding the limits of animal models as predictors of human biology: lessons learned from the sbv IMPROVER Species Translation Challenge

Motivation: Inferring how humans respond to external cues such as drugs, chemicals, viruses or hormones, is an essential question in biomedicine. Very often, however, this question cannot be addressed since it is not possible to perform experiments in humans. A reasonable alternative consists of generating responses in animal models and “translating” those results to humans. The limitations of such translation, however, are far from clear, and systematic assessments of its actual potential are urgently needed. sbv IMPROVER (systems biology verification for Industrial Methodology for PROcess VErification in Research) was designed as a series of challenges to address translatability between humans and rodents. This collaborative crowd-sourcing initiative invited scientists from around the world to apply their own computational methodologies on a multi-layer systems biology dataset comprised of phosphoproteomics, transcriptomics, and cytokine data derived from normal human and rat bronchial epithelial cells exposed in parallel to 52 different stimuli under identical conditions. Our aim was to understand the limits of species-to-species translatability at different levels of biological organization: signaling, transcriptional, and release of secreted factors (such as cytokines). Participating teams submitted 49 different solutions across the sub-challenges; two-thirds of which were statistically significantly better than random. Additionally, similar computational methods were found to range widely in their performance within the same challenge, and no single method emerged as a clear winner across all sub-challenges. Finally, computational methods were able to effectively translate some specific stimuli and biological processes in the lung epithelial system, such as DNA synthesis, cytoskeleton and extracellular matrix (ECM), translation, immune/inflammation, and growth factor/proliferation pathways, better than the expected response similarity between species.