EffectSizes.jl

EffectSizes.jl is a Julia package for effect size measures. Confidence intervals are assigned to effect sizes using either the normal distribution or by bootstrap resampling. The package implements types for the following measures:

Measure	Type
Cohen's d	`CohenD`
Hedge's g	`HedgeG`
Glass's Δ	`GlassΔ`

Installation

] add https://github.com/harryscholes/EffectSizes.jl

Examples

julia> using Random, EffectSizes; Random.seed!(1);

julia> xs = randn(10^3);

julia> ys = randn(10^3) .+ 0.5;

julia> es = CohenD(xs, ys, quantile=0.95); # normal CI (idealised distribution)

julia> typeof(es)
CohenD{Float64,ConfidenceInterval{Float64}}

julia> effectsize(es)
-0.507…

julia> quantile(es)
0.95

julia> ci = confint(es);

julia> typeof(ci)
ConfidenceInterval{Float64}

julia> confint(ci)
(-0.924…, -0.0889…)

julia> es = CohenD(xs, ys, 10^4, quantile=0.95); # bootstrap CI (empirical distribution)

julia> effectsize(es) # effect size is the same
-0.507…

julia> typeof(es)
CohenD{Float64,BootstrapConfidenceInterval{Float64}}

julia> ci = confint(es); # confidence interval is different

julia> lower(ci)
-0.597…

julia> upper(ci)
-0.418…

Index

EffectSizes.AbstractConfidenceInterval
EffectSizes.AbstractEffectSize
EffectSizes.BootstrapConfidenceInterval
EffectSizes.CohenD
EffectSizes.ConfidenceInterval
EffectSizes.EffectSize
EffectSizes.GlassΔ
EffectSizes.HedgeG
EffectSizes.effectsize
EffectSizes.lower
EffectSizes.upper
Statistics.quantile
Statistics.quantile
StatsBase.confint
StatsBase.confint

API

EffectSizes.AbstractEffectSize — Type

AbstractEffectSize

An abstract type to represent an effect size.

Effect	Effect size
Small	0.2
Medium	0.5
Large	0.8

Subtypes implement:

Method	Description
`effectsize`	returns the effect size index
`confint`	returns the confidence interval

source

EffectSizes.EffectSize — Type

const EffectSize = CohenD

See CohenD.

source

EffectSizes.CohenD — Type

CohenD(xs, ys[, bootstrap]; [quantile=0.95])

Calculate Cohen's $d$ effect size index between two vectors xs and ys.

A confidence interval for the effect size is calculated at the quantile quantile. If bootstrap is provided, the confidence interval is calculated by resampling from xs and ys bootstrap times.

\[ d = \frac{m_A - m_B}{s}\]

where $m$ is the mean and $s$ is the pooled standard deviation:

\[ s = \sqrt{\frac{(n_A - 1) s_A^2 + (n_B - 1) s_B^2}{n_A + n_B - 2}}\]

If $m_A$ > $m_B$, $d$ will be positive and if $m_A$ < $m_B$, $d$ will be negative.

Note

HedgeG outperforms CohenD when sample sizes are < 20.

Examples

xs = randn(100000)
ys = randn(100000) .+ 0.01

using EffectSizes
CohenD(xs, ys)

using HypothesisTests
EqualVarianceTTest(xs, ys)

source

EffectSizes.HedgeG — Type

HedgeG(xs, ys[, bootstrap]; [quantile=0.95])

Calculate Hedge's $g$ effect size index between two vectors xs and ys.

\[ g = \frac{m_A - m_B}{s}\]

where $m$ is the mean and $s$ is the pooled standard deviation:

\[ s = \sqrt{\frac{(n_A - 1) s_A^2 + (n_B - 1) s_B^2}{n_A + n_B}}\]

If $m_A$ > $m_B$, $g$ will be positive and if $m_A$ < $m_B$, $g$ will be negative.

Note

HedgeG outperforms CohenD when sample sizes are < 20.

source

EffectSizes.GlassΔ — Type

GlassΔ(treatment, control[, bootstrap]; [quantile=0.95])

Calculate Glass's $Δ$ effect size index between two vectors treatment and control.

\[ Δ = \frac{m_T - m_C}{s_C}\]

where $m$ is the mean, $s$ is the standard deviation, $T$ is the treatment group and $C$ is the control group.

If $m_T$ > $m_C$, $Δ$ will be positive and if $m_T$ < $m_C$, $Δ$ will be negative.

Note

GlassΔ should be used when the standard deviations between the two groups are very different.

source

EffectSizes.effectsize — Function

effectsize(es::AbstractEffectSize)

Return the effect size index.

source

StatsBase.confint — Method

confint(es::AbstractEffectSize) -> ConfidenceInterval

Return the confidence interval of an effect size as a ConfidenceInterval object.

source

Statistics.quantile — Method

quantile(es::AbstractEffectSize) -> Float64

Returns the quantile of a confidence interval.

source

EffectSizes.AbstractConfidenceInterval — Type

AbstractConfidenceInterval{T<:Real}

A type representing a confidence interval.

Subtypes implement:

Method	Description
`confint`	returns the lower and upper bounds
`quantile`	returns the quantile

source

EffectSizes.ConfidenceInterval — Type

ConfidenceInterval(lower, upper, quantile)

A type representing the lower and upper bounds of an effect size confidence interval at a specified quantile.

ConfidenceInterval(xs, ys, es; quantile)

Calculate a confidence interval for the effect size es between two vectors xs and ys at a specified quantile.

source

EffectSizes.BootstrapConfidenceInterval — Type

BootstrapConfidenceInterval(lower, upper, quantile, bootstrap)

A type representing the lower and upper bounds of an effect size confidence interval at a specified quantile with bootstrap resamples.

BootstrapConfidenceInterval(f, xs, ys, bootstrap; quantile)

Calculate a bootstrap confidence interval between two vectors xs and ys at a specified quantile by applying f to bootstrap resamples of xs and ys.

source

StatsBase.confint — Method

confint(ci::AbstractConfidenceInterval{T}) -> Tuple{T,T}

Return the lower and upper bounds of a confidence interval.

source

EffectSizes.lower — Function

lower(ci::AbstractConfidenceInterval{T}) -> T

Return the lower bound of a confidence interval.

source

EffectSizes.upper — Function

upper(ci::AbstractConfidenceInterval{T}) -> T

Return the upper bound of a confidence interval.

source

Statistics.quantile — Method

quantile(ci::AbstractConfidenceInterval) -> Float64

Returns the quantile of a confidence interval.

source