Continued development of the New Zealand earthquake forecast testing centre

Technical Abstract

Recently the New Zealand Earthquake Forecast Testing Centre (NZTC) has been implemented within GNS Science. The NZTC is based on worked with the global Collaboratory for the Study of Earthquake Predictability (CSEP) and is: 1) a rigorous and community accepted rule-set for how earthquake forecasts, such as the National Seismic Hazard Model, should be evaluated; and 2) a computation environment that allows for transparent and reproducible tests of any forecast model implemented within the NZTC.

The NZTC allows for prospective and retrospective testing of forecast models and in this report we focus on the latter which test a forecast against historical earthquake data, including earthquake location and magnitude. We present the results of a suite of statistical tests that, collectively, can be used to understand the performance of a model against observed earthquakes. We have tested thirteen different models that are broken down into four classes which are based on the expected forecast length of each model: 1) one-day models; 2) three-month models; 3) six-month models; and 4) five-year models. The one-day model class is tested using observed earthquakes of magnitude 4.0 to 9.0 for all of New Zealand; all other models are tested using earthquakes of magnitudes 5.0 to 9.0.

The tests of the one-day models were hampered by an error in the CSEP testing procedure that was discovered during the testing, and were inconclusive. Five models were tested in the three-month category using data from 1996 to 2007, and all but one were shown to be consistent with the data. In a relative comparison test of the five models, the EEPAS-0F model was shown to provide a statistically significant improvement over the other four models. Only one model, M8 was tested in the six-month category and it was shown to be inconsistent with the earthquake data between 1996 to 2007. The five-year models were tested using observed earthquake data from 1984 to 2009 and all three models, a uniform Poisson model, a smoothed seismicity model, and the National Seismic Hazard Model, were shown to be consistent with the data. In a comparison test the smoothed seismicity model was able to significantly reject the other two models.

Also reported are two alternative testing routines to those used in the standard CSEP implementation. One improves the efficiency of an existing CSEP test to reduce unnecessary computation time and the other aims to provide a more powerful test of longterm forecasting models that aim to forecast for decades or more by using recorded ground motions and felt intensity reports.

Finally the M8 model was adapted to the CSEP requirements and appropriate computer code was written so that it could be evaluated in the testing centre.