pingouin.multicomp

pingouin.multicomp(pvals, alpha=0.05, method='holm')[source]

P-values correction for multiple comparisons.

Parameters
pvalsarray_like

uncorrected p-values.

alphafloat

Significance level.

methodstring

Method used for testing and adjustment of pvalues. Can be either the full name or initial letters. Available methods are

'bonf' : one-step Bonferroni correction
'holm' : step-down method using Bonferroni adjustments
'fdr_bh' : Benjamini/Hochberg FDR correction
'fdr_by' : Benjamini/Yekutieli FDR correction
'none' : pass-through option (no correction applied)
Returns
rejectarray, boolean

True for hypothesis that can be rejected for given alpha.

pvals_correctedarray

P-values corrected for multiple testing.

See also

bonf

Bonferroni correction

holm

Holm-Bonferroni correction

fdr

Benjamini/Hochberg and Benjamini/Yekutieli FDR correction

pairwise_ttests

Pairwise post-hocs T-tests

Notes

This function is similar to the p.adjust R function.

The correction methods include the Bonferroni correction (“bonf”) in which the p-values are multiplied by the number of comparisons. Less conservative methods are also included such as Holm (1979) (“holm”), Benjamini & Hochberg (1995) (“fdr_bh”), and Benjamini & Yekutieli (2001) (“fdr_by”), respectively.

The first two methods are designed to give strong control of the family-wise error rate. Note that the Holm’s method is usually preferred over the Bonferroni correction. The “fdr_bh” and “fdr_by” methods control the false discovery rate, i.e. the expected proportion of false discoveries amongst the rejected hypotheses. The false discovery rate is a less stringent condition than the family-wise error rate, so these methods are more powerful than the others.

References

  • Bonferroni, C. E. (1935). Il calcolo delle assicurazioni su gruppi di teste. Studi in onore del professore salvatore ortu carboni, 13-60.

  • Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6, 65–70.

  • Benjamini, Y., and Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society Series B, 57, 289–300.

  • Benjamini, Y., and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. Annals of Statistics, 29, 1165–1188.

Examples

FDR correction of an array of p-values

>>> from pingouin import multicomp
>>> pvals = [.50, .003, .32, .054, .0003]
>>> reject, pvals_corr = multicomp(pvals, method='fdr_bh')
>>> print(reject, pvals_corr)
[False  True False False  True] [0.5    0.0075 0.4    0.09   0.0015]