When Is Skepticism Justified?

Consider the following scenario: There is a school of thought that makes a theoretical prediction based on what appear to be good reasons. For some reason, the evidence to back up said prediction does not seem to be forthcoming, which is cited by people disagreeing with it. Time passes … and something resembling evidence at long last shows up. On the other hand, it's much less than the people who originally issued the prediction had in mind.

How skeptical should we be about the prediction? In a related question, what is the track record of earlier predictions that fit the pattern?

I can think of several predictions that fit the above pattern. One of them is believed by the Left. Another is believed by the Right. I am disinclined to take either that seriously. On the other hand, there are other predictions that I am inclined to take seriously that also fit the pattern.