Please use this identifier to cite or link to this item:
Appears in Collections:Law and Philosophy Journal Articles
Peer Review Status: Refereed
Title: Methodological and conceptual challenges in rare and severe event forecast verification
Author(s): Ebert, Philip A
Milne, Peter
Keywords: General Earth and Planetary Sciences
Issue Date: 2022
Date Deposited: 15-Mar-2022
Citation: Ebert PA & Milne P (2022) Methodological and conceptual challenges in rare and severe event forecast verification. Natural Hazards and Earth System Sciences, 22 (2), pp. 539-557.
Abstract: There are distinctive methodological and conceptual challenges in rare and severe event (RSE) forecast verification, that is, in the assessment of the quality of forecasts of rare but severe natural hazards such as avalanches, landslides or tornadoes. While some of these challenges have been discussed since the inception of the discipline in the 1880s, there is no consensus about how to assess RSE forecasts. This article offers a comprehensive and critical overview of the many different measures used to capture the quality of categorical, binary RSE forecasts – forecasts of occurrence and non-occurrence – and argues that of skill scores in the literature there is only one adequate for RSE forecasting. We do so by first focusing on the relationship between accuracy and skill and showing why skill is more important than accuracy in the case of RSE forecast verification. We then motivate three adequacy constraints for a measure of skill in RSE forecasting. We argue that of skill scores in the literature only the Peirce skill score meets all three constraints. We then outline how our theoretical investigation has important practical implications for avalanche forecasting, basing our discussion on a study in avalanche forecast verification using the nearest-neighbour method (Heierli et al., 2004). Lastly, we raise what we call the “scope challenge”; this affects all forms of RSE forecasting and highlights how and why working with the right measure of skill is important not only for local binary RSE forecasts but also for the assessment of different diagnostic tests widely used in avalanche risk management and related operations, including the design of methods to assess the quality of regional multi-categorical avalanche forecasts.
DOI Link: 10.5194/nhess-22-539-2022
Rights: © Author(s) 2022. This work is distributed under the Creative Commons Attribution 4.0 License (
Licence URL(s):

Files in This Item:
File Description SizeFormat 
nhess-22-539-2022.pdfFulltext - Published Version365.47 kBAdobe PDFView/Open

This item is protected by original copyright

A file in this item is licensed under a Creative Commons License Creative Commons

Items in the Repository are protected by copyright, with all rights reserved, unless otherwise indicated.

The metadata of the records in the Repository are available under the CC0 public domain dedication: No Rights Reserved

If you believe that any material held in STORRE infringes copyright, please contact providing details and we will remove the Work from public display in STORRE and investigate your claim.