shape

A-priori Sample Size Calculator for Hierarchical Multiple Regression

free online calculators > calculators > A-priori Sample Size Calculator for Hierarchical Multiple Regression

This calculator will tell you the minimum sample size required for a hierarchical multiple regression analysis; i.e., the minimum sample size required for a significance test of the addition of a set of independent variables B to the model, over and above another set of independent variables A. The value returned by the calculator is the minimum sample size required to detect an effect of the specified size, probability level, and power level for the addition of set B to the model.

A Priori Sample Size Calculator for Hierarchical Multiple Regression

A Priori Sample Size Calculator

For hierarchical multiple regression testing the added value of a block of predictors, the noncentrality parameter is given by:

$$ \lambda = f^2 (N – k – 1) $$

and the achieved power is computed as:

$$ \text{Power} = 1 – F_{nc}\Bigl(F_{1-\alpha}(u, N-k-1);\, u,\, N-k-1,\, \lambda\Bigr) $$

Step 1: Enter Parameters

Enter the total number of predictors in the full model.

Enter the number of predictors added in the second block.

A typical value might be 0.02 (small), 0.15 (medium), or 0.35 (large).

Typical value is 0.05.

Enter a value between 0 and 1 (e.g., 0.80).

Hierarchical Multiple Regression: Testing the added value of a block of predictors uses the noncentral F–test with

$$ \lambda = f^2 (N – k – 1) $$

and

$$ \text{Power} = 1 – F_{nc}\Bigl(F_{1-\alpha}(u, N-k-1);\, u,\, N-k-1,\, \lambda\Bigr). $$

A Priori Sample Size Calculator for Hierarchical Multiple Regression

A Priori Sample Size Calculator for Hierarchical Multiple Regression

In hierarchical multiple regression, predictors (independent variables) are entered into the regression model in blocks or stages. This approach allows researchers to see how much additional variance is explained by a new set of predictors after accounting for the variance explained by previously entered predictors.

A frequent question is: “How large does my sample need to be so I can detect a meaningful increase in $R^2$ when new predictors are introduced?” This is essentially an a priori power analysis for detecting an $R^2$ change (denoted $\Delta R^2$ or $R_{\mathrm{change}}^2$) at a certain significance level ($\alpha$) and power ($1 – \beta$).

1. Key Concepts for Hierarchical Regression

In a hierarchical design, you usually:

  1. Enter a set of predictors (Block 1). Perhaps these are control variables or core covariates.
  2. Add a new block (Block 2) with additional predictor(s). You test whether the new block significantly increases $R^2$ (i.e., the proportion of variance explained in the outcome) above and beyond Block 1.
  3. Continue with more blocks, if needed, each time testing the significance of the increment in $R^2$.

The sample size needed depends primarily on:

  • The expected $R^2$ (or partial $R^2$) increase contributed by the block of interest.
  • The total number of predictors (and how many are in each block).
  • The desired significance level $\alpha$ for the partial F-test of $\Delta R^2$.
  • The desired power $1 – \beta$ to detect that $R^2$ increase if it truly exists.

2. Partial F-Test for $\Delta R^2$

In hierarchical regression, when you add one or more predictors to an existing model, you can test whether the extra predictors significantly improve the model fit. This can be done with an F-test on $\Delta R^2$. One can define an effect size for the increment as:

$$f^2_{\mathrm{inc}} = \frac{\Delta R^2}{1 – R^2_{\text{full}}},$$

where:

  • $\Delta R^2$ is the additional proportion of variance explained by the new block of predictors (compared to the model without these predictors).
  • $R^2_{\text{full}}$ is the final model’s total $R^2$ (i.e., after the new block is added). Sometimes people approximate $R^2_{\text{full}} \approx (R^2_{\text{old}} + \Delta R^2)$.
  • $f^2_{\mathrm{inc}}$ is the effect size measure for the increment. Larger $\Delta R^2$ (compared to the overall variance left unexplained) yields a bigger effect size.

The sample size formula or approach for a partial F-test is similar to general multiple regression power calculations, but focuses on the difference in $R^2$ and the degrees of freedom for the new block of predictors. A commonly cited approach (Cohen, 1988) for an incremental F-test is:

$$N \;\ge\; \bigl(\nu_1 + \nu_2 + 1\bigr) \;\text{where}\; \nu_1 = p_{\text{new}}, \;\; \nu_2 = N – p_{\text{total}} – 1,$$

subject to reaching the needed power with an $f^2_{\mathrm{inc}}$ effect size. In practice, software like G*Power provides a “F-test: Linear multiple regression, fixed model, R^2 increase” module, which automates these calculations.

3. A Conceptual Formula

If you want a more direct formula, we can approximate something akin to:

$$n \;=\; \frac{\bigl(z_{1-\alpha} + z_{1-\beta}\bigr)^2}{f^2_{\mathrm{inc}}} \;+\; p_{\text{total}} + 1,$$

where:

  • $p_{\text{total}}$ is the total number of predictors (old + new) in the final regression model. We need to ensure we have enough degrees of freedom to estimate all parameters plus detect the incremental effect.
  • $f^2_{\mathrm{inc}}$ is as defined above. You can get it from an expected $\Delta R^2$ relative to the final $R^2$.
  • $z_{1-\alpha}$ is the z-score matching your significance level (one-sided or two-sided if you adjust $\alpha$ accordingly). Some prefer $z_{1-\alpha/2}$ for two-tailed testing of the increment in $R^2$. If your test is specifically directional or the software uses an F-distribution approach, it can differ.
  • $z_{1-\beta}$ is the z-score for your desired power. For 80% power, $z_{0.80}\approx 0.84$; for 90% power, $z_{0.90}\approx 1.28$; and so forth.

This style of formula is an approximation. Many prefer direct F-distribution solutions (like G*Power or specialized formulas) because the distribution is more exact for regression increments.

Example: Adding 1 New Predictor in Block 2

Let’s suppose we already have 3 predictors in Block 1 (control variables). We plan to add a 4th predictor in Block 2. We expect an additional $\Delta R^2 = 0.05$ (5% more variance explained) on top of an existing $R^2_{\text{old}}$ of 0.25, for a final $R^2_{\text{full}} = 0.30$. We want $\alpha=0.05$ (two-sided) and $80\%$ power ($\beta=0.20$).

Step 1) Calculate $f^2_{\mathrm{inc}}$:

  • $\Delta R^2 = 0.05$
  • $R^2_{\mathrm{full}} \approx 0.30$
  • $f^2_{\mathrm{inc}} = \frac{\Delta R^2}{1 – R^2_{\mathrm{full}}} = \frac{0.05}{1 – 0.30} = \frac{0.05}{0.70} = 0.0714.$

Step 2) Approximate sample size with a z-based formula (rough approach):

$$n \approx \frac{(z_{1-\alpha/2} + z_{1-\beta})^2}{f^2_{\mathrm{inc}}} + p_{\text{total}} + 1.$$

For $\alpha=0.05$ (two-sided), $z_{0.975}\approx 1.96$. For $1-\beta=0.80$, $z_{0.80}\approx 0.84$. The total final number of predictors, $p_{\text{total}}=4$ (3 old + 1 new).

  • $z_{0.975} + z_{0.80} = 1.96 + 0.84 = 2.80$
  • $(2.80)^2 = 7.84$
  • Divide by $f^2_{\mathrm{inc}}=0.0714$: $7.84 / 0.0714 \approx 109.8$
  • Add $p_{\text{total}} + 1 = 4 + 1 = 5$
  • $n \approx 109.8 + 5 \approx 115$

So, we need roughly $115$ participants total (not per group, since this is a single regression sample). This is a ballpark figure using the z-based approximation. More exact methods using an F-distribution or software might give a slightly different number.

Important Considerations:
  • F-Distribution vs. z-Approx: Many tools (like G*Power) solve for $N$ using noncentral F distributions, which can be more accurate than a z-based approximation, especially for smaller effect sizes or smaller $\alpha$.
  • Blocks with Multiple New Predictors: If you add 2 or 3 new predictors in Block 2, the partial degrees of freedom is $p_{\text{new}}$. The test is an $F$ with $p_{\text{new}}$ and $N – p_{\text{total}} – 1$ degrees of freedom in many references.
  • Collinearity & Overfitting: Real-world data complexities (like correlated predictors) can affect how $\Delta R^2$ translates into actual model performance, meaning practical $N$ might need to be bigger than purely theoretical calculations suggest.
  • Estimating $\Delta R^2$: You typically guess $\Delta R^2$ from pilot data, prior research, or theoretical considerations (how big an incremental contribution you expect from new predictors).

In summary, hierarchical multiple regression sample size planning involves estimating the extra variance ($\Delta R^2$) you expect from a new block, then determining how large $N$ must be so your partial F-test has enough power ($1-\beta$) at your chosen $\alpha$-level. While formulas exist for approximate guidance, many researchers rely on software (e.g., G*Power’s “Linear multiple regression, fixed model, R^2 increase”) for more precise calculations.