The 6% Improvement That Wasn't: Nutrition Survey Metric Skepticism

A district nutrition survey report claims "average household calorie intake improved 6% this year." But when the analyst breaks the data down by income quartile, every single quartile shows a decline. Something is wrong with the headline number — and the solver must find the hidden trap before the report goes to the District Magistrate.

# Aplly.xyz Case Study Submission

## Title
The 6% Improvement That Wasn't: Nutrition Survey Metric Skepticism

## Type
Data Analytics

## Difficulty
Advanced

## Estimated Time
75 minutes

## Overview
A district nutrition survey report claims "average household calorie intake improved 6% this year." But when the analyst breaks the data down by income quartile, every single quartile shows a decline. Something is wrong with the headline number — and the solver must find the hidden trap before the report goes to the District Magistrate.

## Case Details

Function Focus: Simpson's Paradox detection, stratified analysis, metric integrity evaluation, data-driven decision-making under compositional bias

Scenario:
The District Nutrition Office's annual survey report is ready for the District Magistrate's signature. The headline: "Average daily calorie intake per household improved 6% — from 2,090 kcal to 2,215 kcal." The ICDS program is being celebrated as a success. But the data analyst notices something odd when she asks for the raw data: every single income quartile actually ate less than last year. The overall average increased only because the survey sampled more high-income households this year. The report needs to be rewritten — but first, the analyst must prove the paradox exists and quantify the real per-quartile trend.

Dataset Structure:
- Year 1 and Year 2 household survey data
- Respondents grouped by income quartile (Q1 = poorest, Q4 = wealthiest)
- Number of households surveyed per quartile and average daily calorie intake per quartile
- Overall average (weighted) and simple average for comparison

Tasks:
1. Compute the overall average calorie intake for Year 1 and Year 2 using the per-quartile data — confirm the reported 6% improvement
2. Now compute the per-quartile change from Year 1 to Year 2 — calculate the percentage change for each quartile individually
3. Explain why the overall average went up while every quartile went down — identify which data element caused the reversal
4. Produce a corrected report: what would the Year 2 overall average be if the survey had sampled the same income mix as Year 1? Compute this adjusted number and state the real trend
5. Only after submitting your manual analysis, re-run the calculations using a spreadsheet or AI tool and verify your adjusted average — report how the headline would need to change

Expected Output:
A two-page analytical memo containing: (a) overall average calculation for both years confirming the headline number, (b) per-quartile trend table showing the hidden decline, (c) explanation of Simpson's Paradox with specific reference to the sample composition shift, (d) composition-adjusted average for Year 2 and the corrected trend, and (e) a recommended revised headline for the District Magistrate's report.

Evaluation Criteria:
Correct overall and per-quartile average calculations, clear identification of the compositional bias mechanism, correct computation of the composition-adjusted average using Year 1 weights, and a clear, honest restatement of the real trend for the decision-maker.

## Data Sources

Year 1 Survey:
| Income Quartile | Households Surveyed | Avg Daily Calorie Intake (kcal) |
|---|---|---|
| Q1 (Poorest) | 300 | 1,800 |
| Q2 | 250 | 2,000 |
| Q3 | 250 | 2,200 |
| Q4 (Wealthiest) | 200 | 2,500 |
| Total | 1,000 | |

Year 2 Survey:
| Income Quartile | Households Surveyed | Avg Daily Calorie Intake (kcal) |
|---|---|---|
| Q1 (Poorest) | 100 | 1,750 |
| Q2 | 150 | 1,950 |
| Q3 | 300 | 2,150 |
| Q4 (Wealthiest) | 450 | 2,450 |
| Total | 1,000 | |

Context:
- Year 2 survey faced access challenges in low-income wards due to monsoon flooding — field teams substituted easier-to-reach higher-income households to meet their sample size targets
- No other methodology changes between years
- RDA recommended minimum: 2,100 kcal per adult equivalent per day

## Solution Frameworks
Simpson's Paradox detection, stratified analysis, composition-adjusted weighting, metric integrity auditing

## Solver Guidance & Tutorials
Link to: "When Averages Lie — Detecting Simpson's Paradox in Survey Data" tutorial

## What You'll Learn
- Detecting when an overall average hides a per-group decline
- Adjusting for sample composition bias using base-year weights
- Communicating a nuanced data story to a non-technical decision-maker
- Resisting the pressure to celebrate a headline that masks a real problem

## Tags
Simpson's paradox, nutrition survey, data integrity, stratified analysis, metric skepticism

## Registration Links
- Register as Solver
- Register as Evaluator

Year 1 Survey:
| Income Quartile | Households Surveyed | Avg Daily Calorie Intake (kcal) |
|---|---|---|
| Q1 (Poorest) | 300 | 1,800 |
| Q2 | 250 | 2,000 |
| Q3 | 250 | 2,200 |
| Q4 (Wealthiest) | 200 | 2,500 |
| Total | 1,000 | |

Year 2 Survey:
| Income Quartile | Households Surveyed | Avg Daily Calorie Intake (kcal) |
|---|---|---|
| Q1 (Poorest) | 100 | 1,750 |
| Q2 | 150 | 1,950 |
| Q3 | 300 | 2,150 |
| Q4 (Wealthiest) | 450 | 2,450 |
| Total | 1,000 | |

Context:
- Year 2 survey faced access challenges in low-income wards due to monsoon flooding — field teams substituted easier-to-reach higher-income households to meet their sample size targets
- No other methodology changes between years
- RDA recommended minimum: 2,100 kcal per adult equivalent per day

Simpson's Paradox detection, stratified analysis, composition-adjusted weighting, metric integrity auditing