ForestPMPlot: A Flexible Tool for Visualizing Heterogeneity Between Studies in Meta-analysis

Our group recently published a paper in G3 that presents a new method for interpreting meta-analysis of genomic studies. Our software, called ForestPMPlot, is a free, open-source, python-interfaced R package tool available for download from ZarLab Software. In our article, we demonstrate how ForestPMPlot facilitates interpretation of meta-analysis results by producing a plot that visualizes the heterogeneous genetic effects on the phenotype in different study conditions. We show an example analysis where our visualization framework leads to plausible interpretations of gene-by-environment interaction and multiple tissue eQTL, which would not have been straightforward with the traditional framework.

Meta-analysis has become a popular tool for increasing power in genetic association studies, yet it remains a methodological challenge. Genetic association studies can differ from each other in terms of environmental conditions, study design, population types and sizes, statistical noise, and analytical use of covariates. These factors produce different effect sizes between studies, a phenomenon called between-study heterogeneity. Correctly interpreting and accounting for heterogeneity in genetic association studies would give us a more accurate model of the true effects genetic variants have on traits under specific conditions.

Compared to traditional forest plotting techniques, ForestPMPlot visualizes a broader depth of information useful to interpretation of meta-analysis results. Specifically, our tool helps visualize differences in the effect sizes of genetic association studies and clarify why such studies exhibit heterogeneity for a particular phenotype and locus pair under different conditions. To distinguish studies with an effect from studies without an effect, we use the m-value framework. The m-value (Han and Eskin 2012; Kang et al. 2014) is the posterior probability that the effect exists in each study. In our paper, we explain how to compute an m-value and propose using the PM-plot framework (Han and Eskin 2012) to plot the P-values and m-values of each study together. The PM-Plot visualizes the relationship between m-values and P-values in a two-dimensional space, allowing a researcher to easily distinguish which study is predicted to have an effect, and which study is predicted not to have an effect.

We applied ForestPMPlot to a GWAS meta-analysis of 17 HDL mouse studies that have different environmental conditions, such as diet (e.g., high fat/low fat), and genetic knockouts, including homozygous deficiency in leptin receptor (db/db), LDL receptor knockouts, and Apoe gene knockouts. Here, we observe that two confidence intervals of effect estimates overlap each other when only considering the effect size estimates in forest plot format. This result is ambiguous if the observed heterogeneity is a result of stochastic errors. However, in the PM-Plot, we observe that the posterior probabilities are well segregated for these two studies (m-value: 0.93 vs. 0.03), allowing us to hypothesize that the SNP effects on HDL in these strains under the Western diet condition can be interacting with sex.

ForestPMPlot

Seventeen mouse HDL studies with various environmental/genetic conditions are combined in this meta-analysis. (A) Forest plot and (B) PM-plot for rs32595861 locus (Fabp3 gene) analyzing data from the Kang et al. (2014) study.

 

We continue to develop new applications for ForestPMPlot, and we hope that our tool will facilitate more accurate interpretations of meta-analysis in future genetic association research.

ForestPMPlot was developed by Eun Yong Kang and Yurang Park. The article is available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4938634/.

Visit the following page to download ForestPMPlot: http://genetics.cs.ucla.edu/meta_jemdoc/

The full citation to our paper is: 

Kang, Eun Yong; Park, Yurang; Li, Xiao; Segrè, Ayellet V; Han, Buhm; Eskin, Eleazar

ForestPMPlot: A Flexible Tool for Visualizing Heterogeneity between Studies in Meta-analysis. Journal Article

In: G3 (Bethesda), 6 (7), pp. 1793-8, 2016, ISSN: 2160-1836.

Abstract | Links | BibTeX


This paper describes methods implemented based on research originally published by this group: 

Han, Buhm; Eskin, Eleazar

Interpreting meta-analyses of genome-wide association studies. Journal Article

In: PLoS Genet, 8 (3), pp. e1002555, 2012, ISSN: 1553-7404.

Abstract | Links | BibTeX

Han, Buhm; Eskin, Eleazar

Random-Effects Model Aimed at Discovering Associations in Meta-Analysis of Genome-wide Association Studies. Journal Article

In: Am J Hum Genet, 88 (5), pp. 586-98, 2011, ISSN: 1537-6605.

Abstract | Links | BibTeX

We discussed these methods and papers in a 2013 blog post: http://www.zarlab.xyz/heterogeneity-and-meta-analysis/

Mixed Models and Confounding Factors Talk @ Simons Institute

mouse-phylogeny-slideI recently gave a talk on mixed models and confounding factors which is a long time interest of our research group at a workshop which is part of the Evolutionary Biology and the Theory of Computing program which was held at the Simons Institute on the UC Berkeley Campus. The talk was held on February 21st. This talk spans many years of work in our group including work by Hyun Min Kang (now at Michigan), Noah Zaitlen (now at UCSF), and Jimmie Ye (now at Harvard) as well as a sneak peak at very recent work by Joanne Joo, Jae-Hoon Sul and Buhm Han.

The video of the talk is available here and is also on our YouTube Channel ZarlabUCLA.

The papers which are covered in the talk include the EMMA, EMMAX and ICE papers published in 2008 as well as a very new paper that should be coming out soon. The key papers from the talk are:

Kang, Hyun Min; Sul, Jae Hoon ; Service, Susan K; Zaitlen, Noah A; Kong, Sit-Yee Y; Freimer, Nelson B; Sabatti, Chiara ; Eskin, Eleazar

Variance component model to account for sample structure in genome-wide association studies. Journal Article

In: Nat Genet, 42 (4), pp. 348-54, 2010, ISSN: 1546-1718.

Abstract | Links | BibTeX

Kang, Hyun Min; Zaitlen, Noah A; Wade, Claire M; Kirby, Andrew ; Heckerman, David ; Daly, Mark J; Eskin, Eleazar

Efficient control of population structure in model organism association mapping. Journal Article

In: Genetics, 178 (3), pp. 1709-23, 2008, ISSN: 0016-6731.

Abstract | Links | BibTeX

Kang, Hyun Min; Ye, Chun ; Eskin, Eleazar

Accurate discovery of expression quantitative trait loci under confounding from spurious and genuine regulatory hotspots. Journal Article

In: Genetics, 180 (4), pp. 1909-25, 2008, ISSN: 0016-6731.

Abstract | Links | BibTeX

Genes, Environments and Meta-Analysis

Figure 1. Application of Meta-GxE to Apoa2 locus. The forest plot (A) shows heterogeneity in the effect sizes across different studies. The PM- plot (B) predicts that 7 studies have an effect at this locus, even though only 1 study (HMDP-chow(M)) is genome-wide significant with P-value. doi:10.1371/journal.pgen.1004022.g001

It is well known that both genetic factors and environmental factors contribute to traits and specifically disease risk. In addition, an area of great interest in the research community is the interaction between genetic factors and environmental factors and their contribution to disease risk and other traits. Genetic variants that are involved in gene by environment interactions (denoted GxE) have a different effect on the trait spending on the environment. For example, some variants can have an effect on cholesterol levels only in the presence of a high fat diet. Discovering variants involved in GxE has been tremendously difficult and even though thousands of variants have been implicated in disease related traits using genome wide association studies, very few variants have been implicated in GxEs. Part of the difficulty in detecting GxEs is that the traditional approach requires analyzing studies which contain individuals with multiple environments.

We have recently published a paper with the A. Jake Lusis group in PLoS Genetics which presents a novel approach to discovering GxEs. In our approach, many different studies, each which was performed in different environments, are combined to identify GxEs. The key idea is that if variants have a different genetic effect in different environments, then these variants are candidates for being involved in GxEs. Combining studies together is a statistical technique called meta-analysis which has been a major focus of our lab the past few years. We show in the paper, the mathematically, searching for GxEs using the traditional approach and a type of meta-analysis framework called the random effects model(21565292) are very closely related.

We applied our approach to identify GxEs affected mouse HDL cholesterol by combining 17 mouse studies collected by A. Jake Lusis’ group containing almost 5,000 animals. Our approach discovered 26 loci involved in HDL, many of which appear to be involved in GxE. Virtually all of these loci were not previously discovered in any of the individual studies, but many of them map to genes known to affect HDL. Our approach also includes a visualization framework called a PM-plot which helps interpret the associated loci to help identify GxE interactions(22396665).

From the paper:

Discovering environmentally-specific loci using meta-analysis
The Meta-GxE strategy uses a meta-analytic approach to identify gene-by-environment inter- actions by combining studies that collect the same phenotype under different conditions. Our method consists of four steps. First, we apply a random effects model meta-analysis (RE) to identify loci associated with a trait considering all of the studies together. The RE method explicitly models the fact that loci may have different effects in different studies due to gene-by- environment interactions. Second, we apply a heterogeneity test to identify loci with significant gene-by-environment interactions. Third, we compute the m-value of each study to identify in which studies a given variant has an effect and in which it does not. Forth, we visualize the result through a forest plot and PM-plot to understand the underlying nature of gene-by-environment interactions.
We illustrate our methodology by examining a well-known region on mouse chromosome 1 harboring the Apoa2 gene, which is known to be strongly associated with HDL cholesterol (8332912). Figure 1 shows the results of applying our method to this locus. We first compute the effect size and its standard deviation for each of the 17 studies. These results are shown as a forest plot in Figure 1 (a). Second we compute the P-value for each individual study also shown in Figure 1 (a). If we were to follow traditional methodology and evaluate each study separately, we would declare an effect present in a study if the P-value exceeds a predefined genome-wide significance threshold (P < 1.0×10−6). In this case, we would only identify the locus as associated in a single study, HMDP-chow(M) (P = 6.84×10−9). On the other hand, in our approach, we combine all studies to compute a single P-value for each locus taking into account heterogeneity between studies. This approach leads to increased power over the simple approach considering each study separately. The combined meta P-value for the Apoa2 locus is very significant (4.41 × 10−22), which is consistent with the fact that the largest individual study only has 749 animals compared to 4,965 in our combined study.
We visualize the results through a PM-plot, in which P-values are simultaneously visualized with the m-values, which estimates the posterior probability of an effect being present in a study given the observations from all other studies, at each tested locus. These plots allow us to identify in which studies a given variant has an effect and in which it does not. M-values for a given variant have the following interpretation: a study with a small m-value(≤ 0.1) is predicted not to be affected by the variant, while a study with a large m-value(≥ 0.9) is predicted to be affected by the variant.
The PM-plot for the Apoa2 locus is shown in Figure 1 (b). If we only look at the separate study P-values (y-axis), we can conclude that this locus only has an effect in HMDP-chow(M). However, if we look at m-value (x-axis), then we find 8 studies (HMDPxB-ath(M), HMDPxB- ath(F), HMDP-chow(M), HMDP-fat(M), HMDP-fat(F), BxD-db-5(M), BxH-apoe(M), BxH- apoe(F)), where we predict that the variation has an effect, while in 3 studies (BxD-db-12(F), BxD-db-5(F), BxH-wt(M)) we predict there is no effect. The predictions for the remaining 6 studies are ambiguous.
From Figure 1, we observe that differences in effect sizes among the studies are remarkably consistent when considering the environmental factors of each study as described in Table 1. For example, when comparing study 1 – 4, the effect size of the locus decreases in both the male and female HMDPxB studies in the chow diet (chow study) relative to the fat diet (ath study). Thus we can see that when the mice have Leiden/CETP transgene, which cause high total cholesterol level and high LDL cholesterol level, effect size of this locus on HDL cholesterol level in blood is affected by the fat level of diet. Similarly, when comparing study 12 – 15, the knockout of the Apoe gene affects the effect sizes for both male and female BxH crosses. However, in the BxD cross (study 8 – 11), where each animal is homozygous for a mutation causing a deficiency of the leptin receptor, the effect of the locus is very strong in the young male animals, while as animals get older and become fatter, the effect becomes weaker. However in the case of female mice, the effect of the locus is nearly absent at both 5 and 12 weeks of age. Thus we can see that sex plays an important role in affecting HDL when the leptin receptor activity is deficient .

The full citation of our paper is:

Kang, Eun Yong; Han, Buhm; Furlotte, Nicholas; Joo, Jong Wha J; Shih, Diana; Davis, Richard C; Lusis, Aldons J; Eskin, Eleazar

Meta-Analysis Identifies Gene-by-Environment Interactions as Demonstrated in a Study of 4,965 Mice Journal Article

In: PLoS Genet, 10 (1), pp. e1004022, 2014, ISSN: 1553-7404.

Abstract | Links | BibTeX

Bibliography