Using multiple catastrophe models doesn’t necessarily lead to more accurate predictions

Using multiple models to predict catastrophe losses does not always enhance the accuracy of the models’ predictions, the Association of British Insurers (ABI) has said in a good practices guide for catastrophe models.
Also, insurers must be careful that, in an effort to reduce uncertainty inherent in modelling catastrophe losses, they do not simplify the results of their models’ output to such a degree that their damage estimates go wildly astray.
ABI includes these and numerous other observations in its Industry Good Practice for Catastrophe Modelling, a guide to managing catastrophe models as part of an internal model under Solvency II. The document was a result of discussions between P&C insurers and reinsurers operating in the United Kingdom and European states and the Financial Services Authority (FSA). The document does not amount to FSA guidance, nor does the FSA necessarily endorse the paper’s views.
The 68-page document has nine chapters relating to the use of selecting, setting and validating models. In a section on multi-modelling approaches, the paper cautions that using multiple models does not necessarily lead to more accurate results.
“The use of multiple models is sometimes seen as one way to reduce uncertainty by providing several informed estimates of loss, though it can be better seen as a way to reduce the risk of model incompleteness or bias,” the paper notes. “However, if the models represent risk poorly, then the use of multiple models can compound this risk or lead to a lesser understanding of uncertainty.”
Furthermore, the paper cautions, some uncertainty inherent in modeling might actually be a good thing.
“While catastrophe model vendors try to reduce uncertainty in their models, these models themselves are simplifications of complex physical phenomena,” the paper says. “This simplification, the sparsity of data and incomplete understanding may introduce material sources of uncertainty into the models.
“It is important to understand that although some of the uncertainty in the modelled results are characterized in current catastrophe models, many sources of uncertainty are not fully represented or understood. As such, relying on the results without reference to the uncertainties, can lead to a material misrepresentation of risk.”
The full report can be found at:
http://www.abi.org.uk/Publications/59999.pdf