The Bayesian approach is a wonderful way to uncover your hidden prior assumptions. Here's one kind of assumption that perhaps even Bayesians wouldn't pick up on.
Consider this description:
We expect a Gaussian distributed source, having mean between -5 and 5, and standard deviation between 1 and 3
What is the prior? Well, there are two variables, the mean and standard deviation. For the mean our expectation is a uniform distribution between -5 and 5, and for the standard deviation a uniform distribution between 1 and 3.
Did you spot what I missed?
That's right, that we expect a Gaussian distribution is also part of our prior beliefs. Distributions other than Gaussian are assigned zero probability.
The effect is obvious here, but it can be subtler:
The distribution can be described by a mixture of "classes", each class will have the same shape, Gaussian, but the a center point, spread, and rate of occurrence will differ between classes. The liklihood of there being one class is 0.5, the likelihood of two classes is 0.25, of three is 0.125, etc. The center points will be betweem blah and blah, the spread between blah and blah.
Consider this not in terms of model internal structure, but in terms of the overall distributions predicted by such models. In this sense, this example is generic: it can specify, to arbitrary accuracy, any proability distribution.
Therefore, and unlike in the first example, the choice of class shape does not mean that some distributions are given zero probability. But it does affect how likely each distribution is thought to be.