Choosing the wrong statistical test is one of the most common reasons reviewers ask for major revisions. The good news is that the choice follows a simple logic. Answer three questions and you are most of the way there:
- What is my question? Comparing groups, or looking at a relationship?
- What type of data do I have?
- Do my data meet the test's assumptions?
Step 1: Know your type of data
- Nominal: categories with no order, such as gender, state or department
- Ordinal: ordered categories, such as education level or a single Likert item (strongly disagree to strongly agree)
- Interval and ratio (continuous): measurements on a scale, such as age, income, marks or a total score from many Likert items
Whether a scale built from several Likert items can be treated as continuous is debated; many researchers do so when the scale has several items and the scores are reasonably spread. Say what you did and why.
Step 2: Match the question to a test
| Your question | Data meet assumptions | If not (non-parametric) |
|---|---|---|
| Compare 2 independent groups (e.g. male and female) | Independent samples t-test | Mann-Whitney U test |
| Compare the same people twice (before and after) | Paired samples t-test | Wilcoxon signed-rank test |
| Compare 3 or more independent groups | One-way ANOVA | Kruskal-Wallis test |
| Compare the same people 3 or more times | Repeated measures ANOVA | Friedman test |
| Association between two categorical variables | Chi-square test of independence (Fisher's exact test for small samples) | |
| Relationship between two continuous variables | Pearson correlation | Spearman's rank correlation |
| Predict a continuous outcome from several variables | Multiple linear regression | |
| Predict a yes/no outcome | Binary logistic regression | |
| Test a model with latent constructs (e.g. satisfaction, trust) | Confirmatory factor analysis and structural equation modelling (SEM) | |
Step 3: Check the assumptions
Parametric tests such as the t-test and ANOVA assume, among other things:
- Approximately normal distribution of the outcome within groups (check with histograms, Q-Q plots, and tests such as Shapiro-Wilk; remember that with large samples these tests flag tiny departures that do not matter)
- Similar variances across groups (Levene's test); when they differ, Welch's t-test or Welch's ANOVA is a safe choice
- Independent observations: each person counted once, unless the design is paired
Regression adds further checks, such as linear relationships, no extreme multicollinearity between predictors (look at VIF values) and residuals that behave well. When assumptions clearly fail, use the non-parametric alternative or a transformation, and report it.
After ANOVA: post-hoc tests
A significant ANOVA tells you that at least one group differs, not which one. Follow it with post-hoc comparisons such as Tukey's HSD (equal variances) or Games-Howell (unequal variances).
Report more than the p-value
- Effect sizes: Cohen's d, eta squared or r tell readers how big a difference is, which a p-value cannot
- Confidence intervals show the precision of your estimates
- Exact p-values (p = .032) rather than only "p < .05"
- Descriptive statistics for every group: means and standard deviations, or medians and ranges
A typical report reads: "Students in the online group scored higher (M = 72.4, SD = 8.1) than those in the classroom group (M = 68.9, SD = 9.0), t(118) = 2.24, p = .027, d = 0.41."
Which software?
- SPSS: menu-driven and widely taught in Indian universities
- R and Python: free, powerful and reproducible, with a steeper start
- Stata and EViews: popular in economics and econometrics
- AMOS, SmartPLS and the R package lavaan: for structural equation modelling
- NVivo and ATLAS.ti: for qualitative data
Whatever you use, keep your data file, syntax or script and output together, so you can answer reviewers' questions quickly.
Common mistakes
- Running many t-tests instead of one ANOVA, which inflates false positives
- Treating a correlation as proof of cause
- Ignoring missing data, or deleting cases without saying so
- Testing assumptions and then ignoring the results
- Reporting only significant results
Need a second pair of eyes? Our data analysis service runs and explains the analysis with you, and helps you write the results section in the journal's style. For models with latent constructs, read our beginner's guide to SEM.
