Glaze Works fires ceramic tiles in batches. Each batch gets a defect score (percentage of tiles with glaze flaws); lower is better. After one quarter the plant manager singled out 10 batches and assigned them a new slow-cool firing tweak for their next run. She reports: those 10 batches averaged a 22 percent defect rate before the tweak and only 13 percent after, so the slow-cool tweak cut defects by nine points. We should roll it out plant-wide.
Two comparison groups of batches were also tracked across the same two runs. Their mean defect rates are below. (The identity of each group and how it was formed are given in the Methodology note.)
| Batch group | Mean defect rate first run | Mean defect rate second run |
|---|---|---|
| Comparison group 1 | 12 percent | 12 percent |
| Comparison group 2 | 12 percent | 11 percent |
The 10 batches the manager singled out were flagged because they scored worst on their first run; selection was on the first-run score itself. The slow-cool tweak was applied only to those 10 batches. In the Comparison data, Comparison group 1 is the quarter-wide average across all batches in the quarter, and Comparison group 2 is 10 batches drawn at random that received no tweak. Defect rate varies run to run for the same batch even with no change in method, partly from kiln-loading and humidity fluctuations.
Which of the following, drawn from the sources, most strongly undermines the manager's claim that the slow-cool tweak caused the flagged batches' improvement?
Five fresh questions every day, your progress tracked, every miss explained. Free with an account.