Insurers placing too much emphasis on cat model PML numbers, not enough on uncertainty models

***CORRECTION*** Jacqueline Friedland was incorrectly identified as a “senior manager” at KPMG. She is in fact the  actuarial practice leader in KPMG’s Canadian insurance practice.

Insurers are over-relying on catastrophe modeling, focusing too much on specific Probable Maximum Loss (PML) numbers, and not enough on the uncertainty inherent in all models.
A KPMG study undertaken on behalf of Insurance Bureau of Canada (IBC) is due out sometime next month. Jacqueline Friedland, the actuarial practice leader in KPMG’s Canadian insurance practice, discussed some of the general findings of the study at KPMG’s 19th Annual Insurance Issues Conference in Toronto on Nov. 30.
“Models are approximations, subject to significant uncertainty, and provide estimates, not answers,” Friedland said. “There’s too much reliance on a number as an answer.”
Friedland cited many leading authorities on catastrophe modeling based on discussions she had with them while attending the 2010 Reinsurance Association of America (RAA) Conference in Florida. Among them, she cited Karen Clark, the founder of AIR Worldwide, who is now with the consulting firm Karen Clark & Company.
“Confusion occurs when the industry relies on and emphasizes the Probable Maximum Loss, the PML,” Friedland recalled Clark saying at the RAA conference. “And this leads to inaccurate assessments.”
Friedland said Clark talked about how PML estimates can be represented to the fifth decimal point, often expressed in precise 1-in-3 or 1-in-500 probability ratios.
“But precision is confused with accuracy,” Friedland quoted Clark as saying. “Accuracy should not even be in our vocabulary when we talk about these models.”
Uncertainty arises out of every aspect of catastrophe models, Friedland continued, again citing Clark. “From the scientific estimates of frequency and severity of large-magnitude events in specific geographical areas, the lack of knowledge with respect to ground motion and the dynamics of wind speed, the lack of knowledge about how structures respond to ground motion intensity and wind, model error and data quality.”
Over-reliance on models is also partly a product of people feeling company CEOs and directors want easily digestible, quantifiable answers.
“Regulators and rating agencies require an annual filing with a number,” Friedland observed. “The need for answers and numbers, the scientific seduction of the models and a false sense of precision are contributing factors to the over-reliance on catastrophe models.”
But the answer is not to stop using models, Friedland said.
She quoted 2010 RAA Conference participant Paul VanderMarck, chief products officer at RMS, as saying there is a new approach to models that should be incorporated into a company’s best practices.
“We are in the early stages of a paradigm shift in catastrophe modeling,” Friedland said, citing VanderMarck. “Increasingly the focus is turning away from the ability to quantify the risk, to the ability to understand more robustly the uncertainty in that quantification and to use that uncertainty explicitly in making decisions.”