Add Maple
← All help articles

Significance Testing

AddMaple can highlight where a segment is performing higher or lower than expected in any pivot table that compares two categorical groupings (including multi-select tags). We run z-score based significance tests for every cell so you can see which intersections are worth a second look without exporting to a separate stats package.

Turn on significance shading

  1. Build the table you want to analyze. Significance is available for stacked tables with at least two segment columns and more than one category per axis.
  2. Open the chart actions menu (the three-dot More menu) and toggle Significance Testing.
  3. We automatically switch the table into percentage view (% of column) so you can compare segments on the same base size. Count view is disabled while significance is on.
  4. A legend appears underneath the table explaining the colors that are now applied to the cells.

Read the colors

  • Warmer shading means the observed share for that segment is higher than expected; cooler shading means it is lower.
  • The deeper the color, the stronger the effect size (Cohen's h). Neutral (white/grey) cells show no statistical signal.
  • Hover any colored cell to see a tooltip with the z-score, Holm-adjusted p-value, effect size (Cohen's h), and expected baseline percentage.
  • We use the same color settings as set for Opinion Scales. Learn how to customize these colors here.

How the coloring works

AddMaple uses two visual signals:

  • No color (white/grey): Holm-adjusted p-value > 0.10 — no clear signal
  • Circle marker (faint color): Holm p-value between 0.05 and 0.10 — marginal signal (interesting hint, but not statistically strong)
  • Background shading (solid color): Holm p-value ≤ 0.05 — statistically significant. The color intensity reflects the effect size (Cohen's h): stronger effects get darker shading

For multi-select columns, the same thresholds apply; significance is calculated at the overlap level.

Why colors don't change when you switch "% of column / % of row / % of all"

Significance is computed from the raw contingency counts, which don't change when you change how the table displays percentages. Switching the displayed denominator only changes the formatting of the numbers, not the underlying counts or the chi-square residuals — so the z-scores (and therefore colors) stay the same.

What the tiers mean

AddMaple places each cell into one of three p-value brackets:

  • No clear signal (p > 0.10): Holm-adjusted p-value above 0.10. These cells remain uncolored. They may be real patterns but don't meet the confidence threshold.
  • Marginal signal (0.05 < p ≤ 0.10): Cells marked with a faint circle. These are interesting hints—worth a second look—but not yet statistically convincing after correction for multiple comparisons.
  • Statistically significant (p ≤ 0.05): Cells with background shading. These pass the Holm-adjusted significance threshold. The depth of shading reflects effect size (Cohen's h): larger differences get darker colors.

The practical effect size (≥ 5 percentage points or h ≥ 0.20 for multi-select) is baked into the backend calculations and helps determine which cells are worth investigating, but all cells that meet the p ≤ 0.05 threshold will be shaded.

How we calculate it

  • We build a contingency table of the two pivoted columns and run a chi-square test.
  • Each cell's color is based on its Holm-adjusted p-value and effect size (standardized residual / Cohen's h).
  • We correct p-values within each column using Holm-Bonferroni so repeated comparisons stay statistically conservative.
  • Color intensity (darkness) is proportional to effect size: larger Cohen's h values produce darker shading for p ≤ 0.05 cells.
  • Numeric vs categorical pivots (e.g. scalar scores against segments) still generate the same chi-square contingency table behind the scenes, so the z-scores remain comparable.

Tips & limitations

  • Significance only appears when every column has more than one category and the table is not filtered down to a single row or column.
  • Charts keep significance on when you swap between pivoted table and chart views, but we only color the table itself.
  • Turning the toggle off restores your previous count/percentage settings.
  • Combine with filters to focus on segments with enough responses before trusting the directional hints.
  • If you apply respondent weights in the pivot builder, those weights flow into the contingency counts and z-scores, so the shading and tiers reflect the weighted contribution of each response.

With significance testing enabled you can scan for reliable lifts or drop-offs in seconds, then dive into the supporting rows or excerpts to understand why those differences exist.