Image credit: from a talk of Mike West

Figure legend (by me):

The Homo apriorius establishes the probability of an hypothesis, no matter what data tell.

The Homo pragamiticus establishes that it is interested by the data only.

The Homo frequentistus measures the probability of the data given the hypothesis.

The Homo sapients measures the probability of the data and of the hypothesis.

The Homo bayesianis measures the probability of the hypothesis, given the data.

Give a look to my Bayesian primer for astronomers

The frequentist approach (i.e. the non-Bayesian one) prohibits to talk about the error on the true value of a quantity because the true value is a constant, and cannot fluctuate. Luis Lyon 2002 used the word "Anathema" for frequentist statiscians talking about error of a true value. Nevertheless, H

If you question the validity of the above confidence interval, let now consider the standard confidence interval and the following case: the observed background plus signal is smaller than the expected background alone (e.g. Kraft et al. 1991, ApJ, 374, 344 for an astronomical worked out example, and ... for an high energy particle example). For example, you expected to observe 5 background events, but only 3 (signal+background) have been recorded. There is nothing unusual in that, because the background can fluctuate. In such a case, the 68 % confidence interval has zero lenght. Does this mean that the measure of the signal is extreamly accurate (what is shorter than a zero lengh interval?), and cannot be improved, for example in a much expensive experiment with a lower background?

Let now consider any other confidence interval, at the reader choice. The frequentist paradigm provides an interval with the stated coverage. For example, in the long run 68 % of the 68 % confidence intervals includes the true value. Does this means that the true value has a 68 % probability to be inside the 68 % confidence interval ? No, it doesn't, and it is not a word game. Instead of start to be tedious with the Bayes theorem (that states precisely that), let consider the much simpler example: there is a 3 % probability that a woman is pregnant. Does it means pregnant peoples have a 3 % probability to be woman? (example taken from Luis Lyon, 2002, where you also find explained the other famous example of the dog and the hunter, due to D'Agostini). If

If you think that, most of the times, pregnant peoples are women, you agree with I.J. Good: "

To conclude, confidence contours are not measures of the uncertainty of the measured quantity, unless they (numerically) coincide with credible (Bayesian) intervals.

D'Agostini pages: and his lanl talk

Bayesian primer di Luis Lyon, 2002

CDF

Asymetric errors

look at this list too

Durham 2002 workshop

Jim Linnemann links