I could also use the mean square error (MSE) - which is typically done with probability verifications, 
but that gives the impression forecasts are more accurate than they actually are. For example,
suppose no precipitation occurred and the following probabilities were forecast:

.8 .7 .5 .3 .3 .2 .2 .1 0 0

Sum of errors = 3.1
Average error = .31
RMS error = .406
MSE = .165

The root mean square error (RMSE) more closely resembles the average (linear) error - which though 
not providing a proper scoring system does give a better indication of how far off the forecasts 
actually were (quantitatively). The errors are squared because they must be to ensure a forecast 
cannot be biased to verify better in some cases, and either the MSE or the RMSE can be used. Taking 
the square root does not change the rank among several sets of forecasts, but the RMSE in most cases 
will give a number more similar to the average errors as shown above - a little higher, with the MSE 
being quite a bit lower - that more so being the case the better the forecasts are and with more of 
them. For example, forecast errors:

.5 .3 .3 .3 .3 .2 .2 .2 .2 .1 .1 .1 .1 .1 0 0 0 0 0 0

Sum of errors = 3.0
Average error = .15
RMS error = .202
MSE = .041

So if Brier can get his name on the MSE, put mine on the RMSE because it is better - j/k.



Home Page