Jump to content

Sign in to follow this  
Rainshadow

2018-19 Cold Season Model QPF/Snow Modeling Performance/Verification. Nov 13th Event. SREF Wins & OP EC Goes Into This Wintry Mix Event With A Last Place Finish.

Recommended Posts

Hello, with this weekend system Tom & I will start the new cold season qpf verification.  Like last season, we will wait until there is a clear end within 84 hours (to include the nam & sref) before starting any evaluations.

Last season it was all about the EPS: out of 48 events, it came in first place 20 times.   While the GEFS gave it a run for its money, the NAM came in last place in 16 events.   If all models had equal skill, they would have come in first/last place 8 times each.

This is the average error & bias (in inches) for all events:

Capture.JPG.9d1df4ce11a119e90d4b3323950315bd.JPG

This was the accumulative stats for predicted/observed snow events at PHL:

Capture.JPG.0d0eb3328a8827990574764a8af6ecb3.JPG

 

Share this post


Link to post
Share on other sites

When did the NAM come in first place last season:

December 5th, a cold frontal passage

December 12th, a cold frontal passage

December 24th, a double barrel Ohio Valley and North Carolina coastal low

February 2nd, a warm frontal event with inland low

April 3rd, Great Lakes low track

April 19th, Ohio Valley low

April 28th, a cold frontal passage

 

All four of the EPS's fifth or sixth place finishes occurred in April.

 

Share this post


Link to post
Share on other sites

eps

10/25 0z: 1

10/25 12z: 1.1

10/26 0z: 1./05

10/26 12z: 1.17

euro

10/25 0z: 1.18

10/25 12z:  .8

10/26 0z: 1.3

10/26 12z: .8

  • Like 2

Share this post


Link to post
Share on other sites
19 hours ago, tombo82685 said:

eps

10/25 0z: 1

10/25 12z: .8

10/26 0z: 1.3

euro

10/25 0z: 1.18

10/25 12z: 1.1

10/26 0z: 1.05

Tom,

12z run today (Friday) last one for this one.  Then can we start the Sunday night event at 12z also?

 

Share this post


Link to post
Share on other sites

euro

10/26 12z: .08

10/27 0z: .1

10/27 12z: .03

10/28 0z: .05

10/28 12z: .06

eps

10/26 12z: .08

10/27 0z: .15

10/27 12z: .1

10/28 0z: .12

10/28 12z: .08

Share this post


Link to post
Share on other sites
On 10/25/2018 at 12:21 PM, tombo82685 said:

eps

10/25 0z: 1

10/25 12z: .8

10/26 0z: 1.3

10/26 12z: .8

euro

10/25 0z: 1.18

10/25 12z: 1.1

10/26 0z: 1.05

10/26 12z: 1.17

The numbers for the Euro look pretty solid with about 90% of this event done.

Share this post


Link to post
Share on other sites
1 hour ago, Rainshadow said:

The numbers for the Euro look pretty solid with about 90% of this event done.

I got the numbers mixed up when I went back and looked. Had the eps numbers for the euro and euro numbers for eps. So it was the eps that did the best. I adjusted them

Share this post


Link to post
Share on other sites

the eps numbers for the 2nd event from yesterday and last night are a total guess and might be way off. I can't tell with the maps I have since the first event skews the table so widely. This is what I mean. 

KPHL_2018102700_eps_precip_240.png

Share this post


Link to post
Share on other sites
16 minutes ago, tombo82685 said:

I got the numbers mixed up when I went back and looked. Had the eps numbers for the euro and euro numbers for eps. So it was the eps that did the best. I adjusted them

Ok.  I didn't do the table yet anyway.  BTW the new GFS looked like it was closer than the current GFS for 3 of the 4 runs.  I will eyeball it when I can, but I am not going to include it since it becomes operational I am guessing in January.

Just do the best you can.  I can tell it is not 2.34" like one of the GFS runs had.

  • Like 1

Share this post


Link to post
Share on other sites

First Event in the books and it starts out as a repeat of last season. The EPS owned the other models/ensembles.  The NAM took its usual and customary last place finish, first starting too dry and ending way too wet.   There is old data from last season in this spread sheet, I am saving the formulas and graphs for now.

NEW2018-9qpf.xlsx.

 

Share this post


Link to post
Share on other sites

This is how the NAM fails with bogus (convective?) triple point heavy rain offshore which later (not shown) became bogus too far west heavy rain.

nam_mslp_pcpn_frzn_us_19.png.a597dae247de97ce7aa30dcd6e40dd0c.png

nam_mslp_pcpn_frzn_us_14.png.33d9bfad7ac34c5fad689933dd4eba58.png

Share this post


Link to post
Share on other sites
On 10/26/2018 at 6:06 PM, tombo82685 said:

euro

10/26 12z: .08

10/27 0z: .1

10/27 12z: .03

10/28 0z: .05

eps

10/26 12z: .08

10/27 0z: .15

10/27 12z: .1

10/28 0z: .12

Make the day run for this one (10/28/12z) the last one needed.  The NAM might have a fighting chance, no coastal low.

 

Share this post


Link to post
Share on other sites

The NAM might have a fighting chance, no coastal low.

Then again I didn't look at the NAM until this morning.  It gagged again with way too much precip and came in last place.  All the models were too wet. The EC swapped with the EPS and came in first place with this one.  The GFS came in 5th place.  The New GFS wasn't much better with this one with .59" vs .62" for the current GFS accumulative amount over the 5 evaluated runs.  If it was perfect, that value would have been .05".

NEW2018-9qpf.xlsx

Share this post


Link to post
Share on other sites

What's the over/under for rain on 11/1 - 11/2?  This a non-event?

Share this post


Link to post
Share on other sites
2 hours ago, cbelke said:

What's the over/under for rain on 11/1 - 11/2?  This a non-event?

Depends on what model you're hoping for. Euro was over an inch, eps are like .7

Share this post


Link to post
Share on other sites
14 minutes ago, tombo82685 said:

Depends on what model you're hoping for. Euro was over an inch, eps are like .7

But looking like it is coming in Thursday evening right? Friday being the wettest? 

Share this post


Link to post
Share on other sites
3 minutes ago, cbelke said:

But looking like it is coming in Thursday evening right? Friday being the wettest? 

euro is fiday afternoon into early sat morning. 

Share this post


Link to post
Share on other sites
On 10/29/2018 at 6:55 PM, tombo82685 said:

euro is fiday afternoon into early sat morning. 

We can start this event with the 12z run today (the 31st).  The 2,412 consecutive hours of measurable pcpn on the GFS finally has an end to it.

 

Share this post


Link to post
Share on other sites

eps

10/31 12z: 1.25

11/1 0z:  1.45

11/1 12z: .95

11/2 0z:  1.18

euro

10/31 12z: 1.15

11/1 0z: 1.83

11/1 12z: .8

11/2 0z: 1.3

  • Like 1

Share this post


Link to post
Share on other sites

this one is going to be a tough one for the models, convection embedded with this tomorrow night. 

Share this post


Link to post
Share on other sites
12 hours ago, tombo82685 said:

this one is going to be a tough one for the models, convection embedded with this tomorrow night. 

If PHL rainfall observation is totally unrepresentative, I may bag this one.  Mark used to never use convective events for his regression equations.  Just in case it is a rumble of thunder, let's include the 02/00z run also and stop there.

 

Share this post


Link to post
Share on other sites

While there was convection, it didn't occur at PHL and their rainfall was not out of line with neighboring places.

Anyway the NAM went from the Outhouse to the Penthouse and won. The forever wet GEFS came in last place.  For the NAM this is a pattern from both the summer and last cool season. It does better when it is not dealing with coastal lows and also does better with qpf when direct convection is involved.  Or if you prefer other models do worse. 

NEW2018-9qpf.xlsx

 

BTW only 4 runs because the GFS was forecasting 30-36 hours of consecutive measurable rain at Philly when in reality it was two five hour periods.

Share this post


Link to post
Share on other sites

eps

11/3 0z: .55

11/3 12z: .55

11/4 0z: .53

11/4 12z: .45

11/5 0z: .61

euro

11/3 0z: .15

11/3 12z: 1.02

11/4 0z:.3

11/4 12z: .43

11/5 0z: .65

  • Like 1

Share this post


Link to post
Share on other sites
On 11/3/2018 at 12:32 PM, tombo82685 said:

eps

11/3 0z: .55

11/3 12z: .55

euro

11/3 0z: .15

11/3 12z: 1.02

It looks like we can start Phase II with the 00z run this morning (Sunday).   This Phase I we can go through the 00z run on the 5th.  This is just practice for February.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Recently Browsing   0 members

    No registered users viewing this page.

×