The phrase Type III error carries two distinct meanings.
In 1968, Howard Raiffa (a statistician) coined the phrase Type III error, though for a short while some researchers also referred to it as Type 0 error. Raiffe’s idea was that Type III error occurred when one correctly rejected the null hypothesis, but incorrectly attributed the cause. In other words, the researcher misinterpreted what was happening. Or to put it yet another way, the researcher thought something was happening, and in a sense they are right, something was happening, just not what they thought.
The classic example of this is novelty. Let us say you train or treat an individual or group who subsequently improve. For obvious reasons you think it is your intervention that has made the difference, but who’s to say that it was not just the novelty of the exercise they were responding to, and not the particular training or treatment technique itself? Perhaps not, but the point is you do not know for sure.
How does the placebo effect fit into all this? The placebo effect could explain some Type III error, but not all Type III error is necessarily caused by the placebo effect. That is to say, if the placebo effect can be described as a change simply due to believing it will happen, mind over matter, then that is not the only possible source of Type III error.
By this definition, Type III error can only be ruled out in the same way as the placebo effect. No statistical method exists to overcome the problem. The only way around the misattribution is to design around it in methodological terms – i.e. experimental design.
The second definition is similar to the first in some respects, but is different in important ways. Type III error can also refer to one correctly concluding that there is an effect, but wrongly interpreting the direction of that effect. To use the same example as above, one assumes that if the group you train or treat changes in any significant way, that it must be an improvement. Clearly, despite the best effort of researchers, some effects are negative.
Unlike the first definition of Type III error, this directional error can be controlled to some extent by mathematics. Most popular statistical packages include controls for Type III error, and it is of course this latter concept to which they are referring. In this case, the symbol for the probability of a Type III error is (gamma).
Type III error is also used facetiously to refer to the mistake of switching the definitions of Type I error and Type II error.