Two processes, same average
Two machines make the same shaft, 48.00 ± 0.10 mm. Both average 48.00. On the first machine almost every value lies between 47.97 and 48.03. On the second, values range from 47.92 to 48.08. Both are producing good parts today, but only the first has real room for a small shift or a worn tool. The difference between them is the spread, and the standard deviation is how you put a number on it.
What sigma is
The standard deviation is, loosely, the typical distance between a value and the mean. It is expressed in the same unit as the measurement, so a sigma of 0.02 mm means that values typically lie a couple of hundredths from the mean.
Strictly, σ is the standard deviation of the whole population: every part the process makes or could make. In practice it is always estimated from a sample, and the estimate is written s or σ̂.
Calculating it from a sample
For n values with mean X̄, the sample standard deviation is
s = √( Σ (Xi - X̄)² / (n - 1) )
Worked example (illustrative values). Five shafts measure 48.02, 47.98, 48.01, 48.04 and 47.99 mm, with mean 48.008. The deviations from the mean are 0.012, -0.028, 0.002, 0.032 and -0.018. Their squares add up to 0.00228. Divided by n - 1 = 4 this gives 0.00057, and the square root is s ≈ 0.024 mm.
Sigma and the normal distribution
Many process characteristics follow, approximately, a normal distribution: a symmetric bell curve around the mean. If they do, sigma tells you how values are distributed:
- about 68% of values lie within ±1σ of the mean
- about 95% within ±2σ
- about 99.7% within ±3σ
This is where “three sigma” comes from. Control limits are placed three standard deviations from the centre line because, in a stable process, very few points fall outside them by chance. The same rule underlies capability: a process spread of ±3σ, six sigma in total, is compared with the width of the tolerance.
Short-term and overall sigma
In process work there are two common estimates of sigma, and mixing them up is a frequent source of confusion.
- Short-term (within-subgroup) sigma is estimated from the spread inside subgroups, usually as σ̂ = R̄ / d2, where R̄ is the average range and d2 a constant (2.326 for subgroups of five). With R̄ = 0.050 mm, σ̂ ≈ 0.0215 mm. It describes what the process can do when only everyday variation is present, and it is used for control limits and for Cp and Cpk.
- Overall (long-term) sigma is the sample standard deviation s of all individual values, regardless of subgroup. It also includes shifts between subgroups, such as set-ups, material batches or temperature over the day. It is used for Pp and Ppk.
In a perfectly stable process, the two estimates are close. When overall sigma is clearly larger, the process moves between subgroups, and process capability figures based on short-term sigma will flatter it.
Limitations
- The percentages need a normal distribution. Many characteristics are not normal: flatness, runout, roughness and other one-sided characteristics are often skewed. The 68-95-99.7 rule then does not hold, and capability must be calculated with a suitable distribution.
- Sigma from few values is uncertain. An estimate from five values can easily be off by a third. Use plenty of data before drawing conclusions.
- Measurement variation is included. The spread you see includes the spread of the measurement system. If it is large, sigma overstates the process spread, which is why measurement system analysis comes first.
- “Six Sigma” is a different thing. The improvement methodology with that name borrows the unit but is not the same as the six-sigma spread used in capability calculations.
In short
Sigma turns “how much does it vary?” into a number. Know which sigma you are using, short-term or overall, check that the distribution is reasonable, and remember that the measurement system is part of what you are measuring.
