Predictions are overrated?

Sabine Hossenfelder has a blog post (via Not Even Wrong) asserting that Predictions are overrated. Obviously, in PW's words, the naive view that you can evaluate a physical theory simply by the criterion “Does it make predictions?” is wrong; but SH goes too far in dissing predictions.

Happily, after all the physics, which they definitely know more about than me, SH comes on to climate models:

Another example where this misunderstanding matters are climate models. Climate models have correctly predicted many observed trends, from surface temperature increase, to stratospheric cooling, to sea ice melting. That’s an argument commonly used against climate change deniers. But the deniers then go and dig up some papers that made wrong predictions. This, so their claim, demonstrates that really anything is possible and you can’t trust predictions.

This is naive to the point of being wrong. Most importantly, most climate modelling - well, especially the IPCC - has been careful to talk of projections rather than predictions. The outside world hasn't been great at picking up that nuance, but it is there, and needs to be considered if you're attempting some scientific evaluation rather than a political one. That we don't know the value of climate sensitivity is well up front, and clearly century-scale predictions of climate evolution aren't possible without that. Secondly, some of the predictions she mentions - sea ice melting is the most obvious - are ones where the models have done a fairly poor job, other than getting the sign right. And thirdly, there's a variety of models predicting different things, and they can't all be right.

How you should evaluate the credibility of the climate modelling community / scientists / effort, based on past "predictions"? Clearly, picking any one "prediction" at simply verifying that is wrong. You need to look at more of an amalgam, like the IPCC. As to the kinda question "should we trust them now, based on what they said in the past?" you need to look at how the predictions were presented. If you could find multiple frequent cases where people confidently published clear predictions which were subsequently proved wrong, then you would indeed mark them down. Since that isn't actually the case, you don't.

And then there's the point, which SH notes that we have moved on to arguing about the integrity of scientists and the policies of their journals instead about science. If you're talking about denialists, then yes, you're talking about "integrity" and sociology of science, not about actual science. What if you're actually interested in evaluating the science? Then I still think the ability of the models to make predictions matters. In this case it's hard to say quite what we mean by "the science" - all the subcomponents like radiative transfer theory are effectively "unit tested" to borrow from software engineering, but those aren't the bits we're evaluating; in terms of GW, "the science" means that integrating them all together within a GCM (a) works and (b) captures enough of the physical world to say something useful. Without something in the way of prediction, I don't see how that's possible. Where generally you allow "prediction" to include predicting the past; i.e. temperature evolution over the 20th century.

SH proposes instead What, then, is the scientific answer for the climate change deniers? It’s that climate models explain loads of data with few assumptions. Which is nice, but never convincing, unless you believe it anyway. It isn't convincing because it isn't clearly true from the outside: you can't tell that they have "few" assumptions (hence, it is a useless answer for denialists themselves, and not much use when in front of an audience of the general public). Worse, the statement isn't even meaningful; GCMs have lots of "assumptions" in them, measuring "few" or "lots" in any meaningful way would be a difficult task in itself. This is why predictions are good: they don't require looking inside the black box. Perhaps it is just her rhetorical question that's wrong: if she'd asked, What, then, is the scientific answer for the climate change scientists? it would make some sense, but still be hard to evaluate.

Just to be clear, I don't think there's an absolute answer. Trying to evaluate a scientific theory and coming up with an opinion as to whether it is likely true or not isn't a science. There's a question of how much weight you give to predictions versus other factors. But making good predictions is generally so hard, anything that can predict correctly gets a lot of weight.