# What’s new

Table of Contents

## v0.3.2 (January 2020)

Hotfix release to fix a critical issue with `pingouin.pairwise_ttests()`

(see below). We strongly recommend that you update to the newest version of Pingouin and double-check your previous results if you’ve ever used the pairwise T-tests with more than one factor (e.g. mixed, factorial or 2-way repeated measures design).

**Bugfixes**

MAJOR: Fixed a bug in

`pingouin.pairwise_ttests()`

when using mixed or two-way repeated measures design. Specifically, the T-tests were performed without averaging over repeated measurements first (i.e. without calculating the marginal means). Note that for mixed design, this only impacts the between-subject T-test(s). Practically speaking, this led to higher degrees of freedom (because they were conflated with the number of repeated measurements) and ultimately incorrect T and p-values because the assumption of independence was violated. Pingouin now averages over repeated measurements in mixed and two-way repeated measures design, which is the same behavior as JASP or JAMOVI. As a consequence, and when the data has only two groups, the between-subject p-value of the pairwise T-test should be (almost) equal to the p-value of the same factor in the`pingouin.mixed_anova()`

function. The old behavior of Pingouin can still be obtained using the`marginal=False`

argument.Minor: Added a check in

`pingouin.mixed_anova()`

to ensure that the`subject`

variable has a unique set of values for each between-subject group defined in the`between`

variable. For instance, the subject IDs for group1 are [1, 2, 3, 4, 5] and for group2 [6, 7, 8, 9, 10]. The function will throw an error if there are one or more overlapping subject IDs between groups (e.g. the subject IDs for group1 AND group2 are both [1, 2, 3, 4, 5]).Minor: Fixed a bug which caused the

`pingouin.plot_rm_corr()`

and`pingouin.ancova()`

(with >1 covariates) to throw an error if any of the input variables started with a number (because of statsmodels / Patsy formula formatting).

**Enhancements**

Upon loading, Pingouin will now use the outdated package to check and warn the user if a newer stable version is available.

Globally removed the

`export_filename`

parameter, which allowed to export the output table to a .csv file. This helps simplify the API and testing. As an alternative, one can simply use pandas.to_csv() to export the output dataframe generated by Pingouin.Added the

`correction`

argument to`pingouin.pairwise_ttests()`

to enable or disable Welch’s correction for independent T-tests.

## v0.3.1 (December 2019)

**Bugfixes**

Fixed a bug in which missing values were removed from all columns in the dataframe in

`pingouin.kruskal()`

, even columns that were unrelated. See https://github.com/raphaelvallat/pingouin/issues/74.The

`pingouin.power_corr()`

function now throws a warning and return a np.nan when the sample size is too low (and not an error like in previous version). This is to improve compatibility with the`pingouin.pairwise_corr()`

function.Fixed quantile direction in the

`pingouin.plot_shift()`

function. In v0.3.0, the quantile subplot was incorrectly labelled as Y - X, but it was in fact calculating X - Y. See https://github.com/raphaelvallat/pingouin/issues/73

## v0.3.0 (November 2019)

**New functions**

Added

`pingouin.plot_rm_corr()`

to plot a repeated measures correlation

**Enhancements**

Added the

`relimp`

argument to`pingouin.linear_regression()`

to return the relative importance (= contribution) of each individual predictor to the \(R^2\) of the full model.Complete refactoring of

`pingouin.intraclass_corr()`

to closely match the R implementation in the psych package. Pingouin now returns the 6 types of ICC, together with F values, p-values, degrees of freedom and confidence intervals.The

`pingouin.plot_shift()`

now 1) uses the Harrel-Davis robust quantile estimator in conjunction with a bias-corrected bootstrap confidence intervals, and 2) support paired samples.Added the

`axis`

argument to`pingouin.harrelldavis()`

to support 2D arrays.

## v0.2.9 (September 2019)

**Bugfixes**

Disabled default l2 regularization of coefficients in

`pingouin.logistic_regression()`

. As pointed out by Eshin Jolly in PR54, scikit-learn automatically applies a penalization of coefficients, which in turn makes the estimation of standard errors and p-values not totally correct/interpretable. This regularization behavior is now disabled, resulting in the same behavior as R`glm(..., family=binomial)`

.

**Code and dependencies**

Pandas methods are now internally defined using the pandas_flavor package package.

Internal code refactoring of the

`pingouin.pairwise_ttests()`

(to slightly speed up computation and improve memory usage).The first argument of the

`pingouin.anova()`

,`pingouin.ancova()`

,`pingouin.welch_anova()`

,`pingouin.pairwise_ttests()`

,`pingouin.pairwise_tukey()`

,`pingouin.pairwise_gameshowell()`

,`pingouin.welch_anova()`

,`pingouin.kruskal()`

,`pingouin.friedman()`

,`pingouin.cochran()`

,`pingouin.remove_rm_na()`

functions is now`data`

instead of`dv`

(to be consistent with other Pingouin functions). This will cause error if the user runs previous Pingouin code with positional-only arguments. As a general rule,**you should always pass keywords arguments**(read more here).For clarity,

`pingouin.fdr()`

,`pingouin.bonf()`

,`pingouin.holm()`

have been deprecated from the API and must be called via`pingouin.multicomp()`

.`pingouin.pairwise_ttests()`

output does not include the`CLES`

column by default anymore. Users must explicitly pass`effsize='CLES'`

.The

`remove_na`

argument of`pingouin.cronbach_alpha()`

has been replaced with`nan_policy`

(‘pairwise’, or ‘listwise’).Disabled Travis / AppVeyor testing for Python 3.5 While most functions should work just fine, please note that only Python >3.6 is supported now.

**New functions**

Added

`pingouin.harrelldavis()`

, a robust quantile estimation method (to be used in a future version of the`pingouin.plot_shift()`

function). See PR63 by Nicolas Legrand.The

`pingouin.ancova()`

can now directly be used a Pandas method, e.g.`data.ancova(...)`

.The

`pingouin.pairwise_tukey()`

can now directly be used a Pandas method, e.g.`data.pairwise_tukey(...)`

.Added Sidak one-step correction to

`pingouin.multicomp()`

(`method='sidak'`

).

**Enhancements**

Added support for pairwise deletion in

`pingouin.pairwise_ttests()`

(default is listwise deletion), using the`nan_policy`

argument.Added support for listwise deletion in

`pingouin.pairwise_corr()`

(default is pairwise deletion), using the`nan_policy`

argument.Added the

`interaction`

boolean argument to`pingouin.pairwise_ttests()`

, useful if one is only interested in the main effects.Added

`correction_uniform`

boolean argument to`pingouin.circ_corrcc()`

. See PR64 by Dominik Straub.

**Contributors**

Nicolas Legrand

Dominik Straub

## v0.2.8 (July 2019)

**Dependencies**

Pingouin now requires SciPy >= 1.3.0 (better handling of tails in

`pingouin.wilcoxon()`

function) and Pandas >= 0.24 (fixes a minor bug with 2-way within factor interaction in`pingouin.epsilon()`

with previous version)

**New functions**

Added

`pingouin.rcorr()`

Pandas method to calculate a correlation matrix with r-values on the lower triangle and p-values (or sample size) on the upper triangle.Added

`pingouin.tost()`

function to calculate the two one-sided test (TOST) for equivalence. See PR51 by Antoine Weill–Duflos.

**Enhancements**

`pingouin.anova()`

now works with three or more between factors (requiring statsmodels). One-way ANOVA and balanced two-way ANOVA are computed in pure Pingouin (Python + Pandas) style, while ANOVA with three or more factors, or unbalanced two-way ANOVA are computed using statsmodels.`pingouin.anova()`

now accepts different sums of squares calculation method for unbalanced N-way design (type 1, 2, or 3).`pingouin.linear_regression()`

now includes several safety checks to remove duplicate predictors, predictors with only zeros, and predictors with only one unique value (excluding the intercept). This comes at the cost, however, of longer computation time, which is evident when using the`pingouin.mediation_analysis()`

function.`pingouin.mad()`

now automatically removes missing values and can calculate the mad over the entire array using`axis=None`

if array is multidimensional.Better handling of alternative hypotheses in

`pingouin.wilcoxon()`

.Better handling of alternative hypotheses in

`pingouin.bayesfactor_ttest()`

(support for ‘greater’ and ‘less’).Better handling of alternative hypotheses in

`pingouin.ttest()`

(support for ‘greater’ and ‘less’). This is also taken into account when calculating the Bayes Factor and power of the test.Better handling of alternative hypotheses in

`pingouin.power_ttest()`

and`pingouin.power_ttest2n()`

(support for ‘greater’ and ‘less’, and removed ‘one-sided’).Implemented a new method to calculate the matched pair rank biserial correlation effect size for

`pingouin.wilcoxon()`

, which gives results almost identical to JASP.

## v0.2.7 (June 2019)

**Dependencies**

Pingouin now requires statsmodels>=0.10.0 (latest release June 2019) and is compatible with SciPy 1.3.0.

**Enhancements**

Added support for long-format dataframe in

`pingouin.sphericity()`

and`pingouin.epsilon()`

.Added support for two within-factors interaction in

`pingouin.sphericity()`

and`pingouin.epsilon()`

(for the former, granted that at least one of them has no more than two levels.)

**New functions**

Added

`pingouin.power_rm_anova()`

function.

## v0.2.6 (June 2019)

**Bugfixes**

Fixed

**major error in two-sided p-value for Wilcoxon test**(`pingouin.wilcoxon()`

), the p-values were accidentally squared, and therefore smaller. Make sure to always use the latest release of Pingouin.`pingouin.wilcoxon()`

now uses the continuity correction by default (the documentation was saying that the correction was applied but it was not applied in the code.)The

`show_median`

argument of the`pingouin.plot_shift()`

function was not working properly when the percentiles were different that the default parameters.

**Dependencies**

The current release of statsmodels (0.9.0) is not compatible with the newest release of Scipy (1.3.0). In order to avoid compatibility issues in the

`pingouin.ancova()`

and`pingouin.anova()`

functions (which rely on statsmodels for certain cases), Pingouin will require SciPy < 1.3.0 until a new stable version of statsmodels is released.

**New functions**

Added

`pingouin.chi2_independence()`

tests.Added

`pingouin.chi2_mcnemar()`

tests.Added

`pingouin.power_chi2()`

function.Added

`pingouin.bayesfactor_binom()`

function.

**Enhancements**

`pingouin.linear_regression()`

now returns the residuals.Completely rewrote

`pingouin.normality()`

function, which now support pandas DataFrame (wide & long format), multiple normality tests (`scipy.stats.shapiro()`

,`scipy.stats.normaltest()`

), and an automatic casewise removal of missing values.Completely rewrote

`pingouin.homoscedasticity()`

function, which now support pandas DataFrame (wide & long format).Faster and more accurate algorithm in

`pingouin.bayesfactor_pearson()`

(same algorithm as JASP).Support for one-sided Bayes Factors in

`pingouin.bayesfactor_pearson()`

.Better handling of required parameters in

`pingouin.qqplot()`

.The epsilon value for the interaction term in

`pingouin.rm_anova()`

are now computed using the Greenhouse-Geisser method instead of the lower bound. A warning message has been added to the documentation to alert the user that the value might slightly differ than from R or JASP.

Note that d. and e. also affect the behavior of the `pingouin.corr()`

and `pingouin.pairwise_corr()`

functions.

**Contributors**

## v0.2.5 (May 2019)

**MAJOR BUG FIXES**

Fixed error in p-values for

**one-sample one-sided T-test**(`pingouin.ttest()`

), the two-sided p-value was divided by 4 and not by 2, resulting in inaccurate (smaller) one-sided p-values.Fixed global error for

**unbalanced two-way ANOVA**(`pingouin.anova()`

), the sums of squares were wrong, and as a consequence so were the F and p-values. In case of unbalanced design, Pingouin now computes a type II sums of squares via a call to the statsmodels package.The epsilon factor for the interaction term in two-way repeated measures ANOVA (

`pingouin.rm_anova()`

) is now computed using the lower bound approach. This is more conservative than the Greenhouse-Geisser approach and therefore give (slightly) higher p-values. The reason for choosing this is that the Greenhouse-Geisser values for the interaction term differ than the ones returned by R and JASP. This will be hopefully fixed in future releases.

**New functions**

Added

`pingouin.multivariate_ttest()`

(Hotelling T-squared) test.Added

`pingouin.cronbach_alpha()`

function.Added

`pingouin.plot_shift()`

function.Several functions of pandas can now be directly used as

`pandas.DataFrame`

methods.Added

`pingouin.pcorr()`

method to compute the partial Pearson correlation matrix of a`pandas.DataFrame`

(similar to the pcor function in the ppcor package).The

`pingouin.partial_corr()`

now supports semi-partial correlation.

**Enhancements**

The

`pingouin.rm_corr()`

function now returns a`pandas.DataFrame`

with the r-value, degrees of freedom, p-value, confidence intervals and power.`pingouin.compute_esci()`

now works for paired and one-sample Cohen d.`pingouin.bayesfactor_ttest()`

and`pingouin.bayesfactor_pearson()`

now return a formatted str and not a float.`pingouin.pairwise_ttests()`

now returns the degrees of freedom (dof).Better rounding of float in

`pingouin.pairwise_ttests()`

.Support for wide-format data in

`pingouin.rm_anova()`

`pingouin.ttest()`

now returns the confidence intervals around the difference in means.

**Missing values**

`pingouin.remove_na()`

and`pingouin.remove_rm_na()`

are now external function documented in the API.`pingouin.remove_rm_na()`

now works with multiple within-factors.`pingouin.remove_na()`

now works with 2D arrays.Removed the remove_na argument in

`pingouin.rm_anova()`

and`pingouin.mixed_anova()`

, an automatic listwise deletion of missing values is applied (same behavior as JASP). Note that this was also the default behavior of Pingouin, but the user could also specify not to remove the missing values, which most likely returned inaccurate results.The

`pingouin.ancova()`

function now applies an automatic listwise deletion of missing values.Added remove_na argument (default = False) in

`pingouin.linear_regression()`

and`pingouin.logistic_regression()`

functionsMissing values are automatically removed in the

`pingouin.anova()`

function.

**Contributors**

Raphael Vallat

Nicolas Legrand

## v0.2.4 (April 2019)

**Correlation**

Added

`pingouin.distance_corr()`

(distance correlation) function.`pingouin.rm_corr()`

now requires at least 3 unique subjects (same behavior as the original R package).The

`pingouin.pairwise_corr()`

is faster and returns the number of outlier if a robust correlation is used.Added support for 2D level in the

`pingouin.pairwise_corr()`

. See Jupyter notebooks for examples.Added support for partial correlation in the

`pingouin.pairwise_corr()`

function.Greatly improved execution speed of

`pingouin.correlation.skipped()`

function.Added default random state to compute the Min Covariance Determinant in the

`pingouin.correlation.skipped()`

function.The default number of bootstrap samples for the

`pingouin.correlation.shepherd()`

function is now set to 200 (previously 2000) to increase computation speed.`pingouin.partial_corr()`

now automatically drops rows with missing values.

**Datasets**

Renamed

`pingouin.read_dataset()`

and`pingouin.list_dataset()`

(before one needed to call these functions by calling pingouin.datasets)

**Pairwise T-tests and multi-comparisons**

Added support for non-parametric pairwise tests in

`pingouin.pairwise_ttests()`

function.Common language effect size (CLES) is now reported by default in

`pingouin.pairwise_ttests()`

function.CLES is now implemented in the

`pingouin.compute_effsize()`

function.Better code, doc and testing for the functions in multicomp.py.

P-values adjustment methods now do not take into account NaN values (same behavior as the R function p.adjust)

**Plotting**

Added

`pingouin.plot_paired()`

function.

**Regression**

NaN are now automatically removed in

`pingouin.mediation_analysis()`

.The

`pingouin.linear_regression()`

and`pingouin.logistic_regression()`

now fail if NaN / Inf are present in the target or predictors variables. The user must remove then before running these functions.Added support for multiple parallel mediator in

`pingouin.mediation_analysis()`

.Added support for covariates in

`pingouin.mediation_analysis()`

.Added seed argument to

`pingouin.mediation_analysis()`

for reproducible results.`pingouin.mediation_analysis()`

now returns two-sided p-values computed with a permutation test.Added

`pingouin.utils._perm_pval()`

to compute p-value from a permutation test.

**Bugs and tests**

Travis and AppVeyor test for Python 3.5, 3.6 and 3.7.

Better doctest & improved examples for many functions.

Fixed bug with

`pingouin.mad()`

when axis was not 0.

## v0.2.3 (February 2019)

**Correlation**

shepherd now also returns the outlier vector (same behavior as skipped).

The corr function returns the number of outliers for shepherd and skipped.

Removed mahal function.

**Licensing**

Pingouin is now released under the GNU General Public Licence 3.

Added licenses files of external modules (qsturng and tabulate).

**Plotting**

NaN are automatically removed in qqplot function

## v0.2.2 (December 2018)

**Plotting**

Started working on Pingouin’s plotting module

Added Seaborn and Matplotlib to dependencies

Added plot_skipped_corr function (PR from Nicolas Legrand)

Added qqplot function (Quantile-Quantile plot)

Added plot_blandaltman function (Bland-Altman plot)

**Power**

Added power_corr, based on the R pwr package.

Renamed anova_power and ttest_power to power_anova and power_ttest.

Added power column to corr() and pairwise_corr()

power_ttest function can now solve for sample size, alpha and d

power_ttest2n for two-sample T-test with unequal n.

power_anova can now solve for sample size, number of groups, alpha and eta

## v0.2.1 (November 2018)

**Effect size**

Separated compute_esci and compute_bootci

Added corrected percentile method and normal approximation to bootstrap

Fixed bootstrapping method

## v0.2.0 (November 2018)

**ANOVA**

Added Welch ANOVA

Added Games-Howell post-hoc test for one-way ANOVA with unequal variances

Pairwise T-tests now accepts two within or two between factors

Fixed error in padjust correction in the pairwise_ttests function: correction was applied on all p-values at the same time.

**Correlation/Regression**

Added linear_regression function.

Added logistic_regression function.

Added mediation_analysis function.

Support for advanced indexing (product / combination) in pairwise_corr function.

**Documentation**

Added Guidelines section with flow charts

Renamed API section to Functions

Major improvements to the documentation of several functions

Added Gitter channel

## v0.1.10 (October 2018)

**Bug**

Fixed dataset names in MANIFEST.in (.csv files were not copy-pasted with pip)

**Circular**

Added circ_vtest function

**Distribution**

Added multivariate_normality function (Henze-Zirkler’s Multivariate Normality Test)

Renamed functions test_normality, test_sphericity and test_homoscedasticity to normality, sphericity and homoscedasticity to avoid bugs with pytest.

Moved distribution tests from parametric.py to distribution.py

## v0.1.9 (October 2018)

**Correlation**

Added partial_corr function (partial correlation)

**Doc**

Minor improvements in docs and binder notebooks

## v0.1.8 (October 2018)

**ANOVA**

Added support for multiple covariates in ANCOVA function (requires statsmodels).

**Documentation**

Major re-organization in API category

Added equations and references for effect sizes and Bayesian functions.

**Non-parametric**

Added cochran function (Cochran Q test)

## v0.1.7 (September 2018)

**ANOVA**

Added rm_anova2 function (two-way repeated measures ANOVA).

Added ancova function (Analysis of covariance)

**Correlations**

Added intraclass_corr function (intraclass correlation).

The rm_corr function uses the new ancova function instead of statsmodels.

**Datasets**

Added ancova and icc datasets

**Effect size**

Fixed bug in Cohen d: now use unbiased standard deviation (np.std(ddof=1)) for paired and one-sample Cohen d. Please make sure to use pingouin >= 0.1.7 to avoid any mistakes on the paired effect sizes.

## v0.1.6 (September 2018)

**ANOVA**

Added JNS method to compute sphericity.

**Bug**

Added .csv datasets files to python site-packages folder

Fixed error in test_sphericity when ddof == 0.

## v0.1.5 (August 2018)

**ANOVA**

rm_anova, friedman and mixed_anova now require a subject identifier. This avoids improper collapsing when multiple repeated measures factors are present in the dataset.

rm_anova, friedman and mixed_anova now support the presence of other repeated measures factors in the dataset.

Fixed error in test_sphericity

Better output of ANOVA summary

Added epsilon function

**Code**

Added AppVeyor CI (Windows)

Cleaned some old functions

**Correlation**

Added repeated measures correlation (Bakdash and Marusich 2017).

Added robust skipped correlation (Rousselet and Pernet 2012).

Pairwise_corr function now automatically delete non-numeric columns.

**Dataset**

Added pingouin.datasets module (read_dataset & list_dataset functions)

Added datasets: bland1995, berens2009, dolan2009, mcclave1991

**Doc**

Examples are now Jupyter Notebooks.

Binder integration

**Misc**

Added median absolute deviation (mad)

Added mad median rule (Wilcox 2012)

Added mahal function (equivalent of Matlab mahal function)

**Parametric**

Added two-way ANOVA.

Added pairwise_tukey function

## v0.1.4 (July 2018)

**Installation**

Fix bug with pip install caused by pingouin.external

**Circular statistics**

Added circ_corrcc, circ_corrcl, circ_r, circ_rayleigh

## v0.1.3 (June 2018)

**Documentation**

Added several tutorials

Improved doc of several functions

**Bayesian**

T-test now reports the Bayes factor of the alternative hypothesis (BF10)

Pearson correlation now reports the Bayes factor of the alternative hypothesis (BF10)

**Non-parametric**

Kruskal-Wallis test

Friedman test

**Correlations**

Added Shepherd’s pi correlation (Schwarzkopf et al. 2012)

Fixed bug in confidence intervals of correlation coefficients

Parametric 95% CI are returned by default when calling corr

## v0.1.2 (June 2018)

**Correlation**

Pearson

Spearman

Kendall

Percentage bend (robust)

Pairwise correlations between all columns of a pandas dataframe

**Non-parametric**

Mann-Whitney U

Wilcoxon signed-rank

Rank-biserial correlation effect size

Common language effect size

## v0.1.1 (April 2018)

**ANOVA**

One-way

One-way repeated measures

Two-way split-plot (one between factor and one within factor)

**Miscellaneous statistical functions**

T-tests

Power of T-tests and one-way ANOVA

## v0.1.0 (April 2018)

Initial release.

**Pairwise comparisons**

FDR correction (BH / BY)

Bonferroni

Holm

**Effect sizes**:

Cohen’s d (independent and repeated measures)

Hedges g

Glass delta

Eta-square

Odds-ratio

Area Under the Curve

**Miscellaneous statistical functions**

Geometric Z-score

Normality, sphericity homoscedasticity and distributions tests

**Code**

PEP8 and Flake8

Tests and code coverage