So I'll sketch out my reasons why I don't think this has a pat answer.

BTW, I was mistaken earlier in the expression I gave for finding the MLE. The expression I gave for the conditional probability is actually the CDF, not the PDF of that r.v.

If the r.v. X is the max of the 5 numbers chosen, and the r.v. C is the upper bound chosen by the folks running the lottery, then:

(F_{X|C}(x) = P(X < x | C = c) = \frac{x^5}{c^5}, 0 < x < c)

(f_{X|C}(x)=F'_{X|C}(x) = \frac{5x^4}{c^5}, 0 < x < c)

This is the correct expression. Its sup, however, also occurs at c.

What we want is the conditional probability of C given X, so the natural way to get from one to the other is the joint PDF.

(f_{X|C}(x) = \frac{f_{X,C}(x,c)}{f_C(c)})

(f_{X,C}(x,c) = f_{X|C}(x)f_C(c))

Where:

(f_{X,C}(x,c)=P(X = x, C = c))

(f_C(c) = P(C = c))

Of course we know the conditional probability of X given C, but in order to get the joint PDF, we also have to know the distribution the folks running the lottery are drawing their C from. This makes sense intuitively, since if they're choosing a number at random between 10 and 100 uniformly, we'll want to guess rather differently than we would if they're choosing a random number from an exponential distribution, say, with mean 100.

If you happen to discover the distribution of C, then the rest of the calculation is easy in concept.

(f_{C|X}(c) = \frac{f_{X,C}(x,c)}{f_X(x)}= \frac{f_{X|C}(x) f_C(c)}{f_X(x)})

With:

(f_X(x) = \int_x^{+\infty}f_{X,C}(x,c) dc)

...although you have to be careful with the limits on this integral, and as a practical matter it might be rather nasty to perform.

Assuming this can all be done, all that remains is to find c* which minimizes the conditional expectation:

(E_X[|C - c*|])

Overall, the point is that without the information of C's distribution, we're flailing around. Information on past winning C's might help give an inkling of what distribution is being used if the lottery organizers are too lazy to change their distribution.

...But all this is just a way to determine some ideal guess, which is only really half the problem. The real problem is to make our guess capture as much of the conditional distribution as possible so that we maximize our chances of winning, which may mean stepping some distance away from the "optimal" C if the other entrants' bets tend to cluster all around our c*...assuming one can get this information, that is. It seems quite likely, for example, that if there are many thousands of entrants in the lottery, the optimal strategy overall will be to make a guess far above our estimate of c* where the other entrants' guesses are more widely spaced.