Jump to content

Postmortem - Modeling/Snow Maps/Communication


BeerandWXTogether
 Share

Recommended Posts

I'm going to throw a hot take out there and start the discussion, based on a comment that I made in the obs thread.

I have a few opinions on meteorology in general right now.

1) While modeling in general is much better than 25 years ago, we are still having problems with exacts and specifics in a small geographic area. I'm not sure that will ever 100% improve given chaos theory and resolution unless you shrink resolution down to 1km or some ridiculous number like that...even then, I think we're going to have problems in model world with ptype because of temperature profiles in the atmosphere, etc. I don't think this storm was a bust in the sense that the storm developed pretty much as advertised. Were forecasts wrong? Absolutely. We could throw more money at modeling and it may improve a few aspects of things but by and large I think rain/snow/sleet line misfires are going to be a part of the problem with forecasting forever.

2) Snow maps are stupid. We (collective community) need to stop taking them as gospel, especially in marginal events. And I think meteorologists or vendors who peddle them out simply to get weenies to sign up at $13/month or to get follows, etc. are probably the worst for meteorological discourse right now.

3) Communication of impacts matters much more than "how much for?" discussion but I don't think the meteorological community and public have it in them to switch gears from numbers and colors on a map to reading a list of how fooked they are or aren't. I'm not sure how this can change but I think it needs to. It's like Saffir-Simpson and hurricanes - it's a garbage scale and most smart weather people know that but meteorologists who know better still use it and talk about "upgrades" and "downgrades" because of a wind speed change.  How do we (weather community) do a better job of getting people out of that habit (how much for) and into "how bad is it gonna be"

  • Like 4
Link to comment
Share on other sites

phlwx, 1) Yep 2) Yep 3) Yep.  Pretty much nails it on the head.  Models are never going to be right on at r/s line.  And really, they don't need to be.  My weenie-ness is not a financial justification for better snowline on models lol.  Really, they just need to parse out the broad details of strong systems for public safety and government and utility preparedness.  Which actually the models and mets do a very fine job at. 

The communication aspect is the part that can use the most help, for exactly the reasons you mentioned.  It's about clicks, not assessments of reality.  I can't see a way to change that for the internet, but I continue advocate our fine NWS folks and all our mets to continue the excellent work of informing and warning in the bigger picture. 

Thinking back on it, there've been a number of busts on r/s line amounts, and, if I may say, bfd.   Lol.  The only event I can think of that was regrettable in the last 10 years or so was the surprise ice storm a few years back the roads weren't prepped for, and there were a lot of accidents.  Nothing to do with the mets, really just an example of the incredible trickiness of ice forecasting, but more just to show the not infrequent event misfires on pretty color snow maps miss the big picture, that public safety problems due to forecasting misses is a rare issue. 

 

Anyway, NAM did well with this.  (And I duck....) :tongue:

Link to comment
Share on other sites

Grades for this storm the professor is quite cranky after 5 very long days and two big snow events. 

UKMET- A+

NAM- A- Tombo will go crazy over that. 

GFS- F

CMC-C-

ECMWF - D

HRRR- D

RAP- D

Others- who the heck knows

  • Like 3
Link to comment
Share on other sites

Just now, Mitchg said:

Grades for this storm the professor is quite cranky after 5 very long days and two big snow events. 

UKMET- A+

NAM- A- Tombo will go crazy over that. 

GFS- F

CMC-C-

ECMWF - D

HRRR- D

RAP- D

Others- who the heck knows

Tombo wont over the NAM, he chose it over the GFS yesterday.

  • Like 1
Link to comment
Share on other sites

6 hours ago, Mitchg said:

Grades for this storm the professor is quite cranky after 5 very long days and two big snow events. 

UKMET- A+

NAM- A- Tombo will go crazy over that. 

GFS- F

CMC-C-

ECMWF - D

HRRR- D

RAP- D

Others- who the heck knows

The NAM was solid inside 24hrs I thought. Really picked out the increasing thermal concerns. Thats the only time I will look at it. 

Link to comment
Share on other sites

7 hours ago, phlwx said:

I'm going to throw a hot take out there and start the discussion, based on a comment that I made in the obs thread.

I have a few opinions on meteorology in general right now.

1) While modeling in general is much better than 25 years ago, we are still having problems with exacts and specifics in a small geographic area. I'm not sure that will ever 100% improve given chaos theory and resolution unless you shrink resolution down to 1km or some ridiculous number like that...even then, I think we're going to have problems in model world with ptype because of temperature profiles in the atmosphere, etc. I don't think this storm was a bust in the sense that the storm developed pretty much as advertised. Were forecasts wrong? Absolutely. We could throw more money at modeling and it may improve a few aspects of things but by and large I think rain/snow/sleet line misfires are going to be a part of the problem with forecasting forever.

2) Snow maps are stupid. We (collective community) need to stop taking them as gospel, especially in marginal events. And I think meteorologists or vendors who peddle them out simply to get weenies to sign up at $13/month or to get follows, etc. are probably the worst for meteorological discourse right now.

3) Communication of impacts matters much more than "how much for?" discussion but I don't think the meteorological community and public have it in them to switch gears from numbers and colors on a map to reading a list of how fooked they are or aren't. I'm not sure how this can change but I think it needs to. It's like Saffir-Simpson and hurricanes - it's a garbage scale and most smart weather people know that but meteorologists who know better still use it and talk about "upgrades" and "downgrades" because of a wind speed change.  How do we (weather community) do a better job of getting people out of that habit (how much for) and into "how bad is it gonna be"

1) Yes, greater resolution and improved parameterization will help somewhat.  Hopefully improved satellite data could help as well, but I'm not sure if there's a plateau where more obs won't help because of the resolution threshold of satellites.  Maybe incorporating more non-standard observations will help (iot, etc)?  Of course QC would be an issue with that.  More REAL upper-level obs would help, but how much more data could we get without soundings?  Me <--- not an expert on this, but would love to hear your thoughts.

2) Agree, but I still love the hell out of them.

3) Unsure on this one

Link to comment
Share on other sites

I've been thing about this and have a couple of thoughts, and may chime in again tomorrow.

(1) I am old enough that I remember the state of weather forecasting 25 and more years ago.

(2) Snow maps? For me, a mixed bag. On the one hand, I could just as well do without most of the snow maps. On the other hand, I don't like being surprised by big (snow) storms, or other major weather events. At least with the snow maps, I have an idea of what the worse case scenario might be. And I know that such maps are not gospel, that forecasts can change.

(3) I appreciate and rely on communications of impacts, one example being some of the impacts statements that WFO Mt Holly did in their products in advance of today's storm. I was reminded to get more heating oil, and that I was out of things in my emergency food stockpile. I appreciated the reminders of how to be safe in a storm. And I put my old wooly blanket back in my car.

People say so and so got the snow forecast wrong, or another forecast wrong so I'm not going to heed their warnings? How to get beyond that kind of thinking and have more people take the impacts seriously? I for one think this an important discussion. But I am not in the position to have any suggestions.

But it is getting late for me, and it is time for me to sign off. I am looking forward to your thoughts on this discussion.

 

On 3/14/2017 at 11:17 AM, phlwx said:

I'm going to throw a hot take out there and start the discussion, based on a comment that I made in the obs thread.

I have a few opinions on meteorology in general right now.

1) While modeling in general is much better than 25 years ago, we are still having problems with exacts and specifics in a small geographic area. I'm not sure that will ever 100% improve given chaos theory and resolution unless you shrink resolution down to 1km or some ridiculous number like that...even then, I think we're going to have problems in model world with ptype because of temperature profiles in the atmosphere, etc. I don't think this storm was a bust in the sense that the storm developed pretty much as advertised. Were forecasts wrong? Absolutely. We could throw more money at modeling and it may improve a few aspects of things but by and large I think rain/snow/sleet line misfires are going to be a part of the problem with forecasting forever.

2) Snow maps are stupid. We (collective community) need to stop taking them as gospel, especially in marginal events. And I think meteorologists or vendors who peddle them out simply to get weenies to sign up at $13/month or to get follows, etc. are probably the worst for meteorological discourse right now.

3) Communication of impacts matters much more than "how much for?" discussion but I don't think the meteorological community and public have it in them to switch gears from numbers and colors on a map to reading a list of how fooked they are or aren't. I'm not sure how this can change but I think it needs to. It's like Saffir-Simpson and hurricanes - it's a garbage scale and most smart weather people know that but meteorologists who know better still use it and talk about "upgrades" and "downgrades" because of a wind speed change.  How do we (weather community) do a better job of getting people out of that habit (how much for) and into "how bad is it gonna be"

 

Link to comment
Share on other sites

I'll say again what I mentioned in the OBS thread.  Hoping to change the narrative of the populace is a fool's errand.  The "HOW MUCH SNOW FOR EAST BUMBLE****?" thing will never go away, just like the "Them damn weathermen have the easiest job ever"!  They can be wrong 99% of the time and still make money!" rhetoric.  The public at large doesn't care about the actually interesting manner in which this storm "busted", because there seems to be this perverse sense of glee whenever the forecast is wrong.

It's always the same people leaving the same inane comments on every article about upcoming weather.  The NWS has their "least much" and "most possible" maps, but those aren't interesting to the public.  I know this is a pessimistic view, but I think "it is what it is", so to speak.  Following this forum has made me respect the hell out of the responsible and disciplined meteorologists out there, which are 99% of them IMO.  You guys and gals have a thankless job.

  • Like 5
Link to comment
Share on other sites

A couple of comments:

The public impact of sleet is very similar to snow. So for most people the forecasts worked well. There was plenty of warning that a disruptive storm was coming and schools, business, airlines, families, etc. prepared/adjusted accordingly to minimize disruption. However a story on how well the area prepared has limited appeal.

I personally like snow maps. Just like any other weather map they are a handy way of communicating and visualizing patterns. The problem is bad snow maps. However if the error is due to poor modeling removing the snow map won't solve the problem.

With a major snowstorm coming, social media/internet is going to have its pluses and minuses with or without snow maps. In the long run I think most people are smart enough to figure out who is giving them good information.

In a sleet storm a map that includes sleet as snow is useless. Not sure why these maps persist since it should be a relatively easy fix.

Regarding #2 from the top post. Even us weather board amateurs know enough not to rely on a snow map to make a forecast. I checked soundings before the storm and they were generally consistent with the maps. Models that were forecasting sleet had sleet soundings and vice-versa. Snow maps were not the cause of bad forecasts. Which brings me to the last point.

As humans we all have confirmation bias that is very difficult to shake. If you like snow you will look for reasons why it will snow. If you hate snow you will look for reasons why it won't snow. Most weather board hobbyists like big storms and snow so we tend to look for ways it could snow. Heavy snow forecasts once made tend to be sticky. To me as the storm approached sleet appeared more and more likely in our area.  Yet I kept looking more aggressively for reasons to support a heavy snow forecast than for reasons to support a sleet forecast. I knew it was going to sleet but couldn't quite pull the trigger on a big sleet forecast, even though I had enough information to do so. Of course hindsight is 20/20 and if the models were more aligned on a sleet forecast I would have done better, but I can only blame myself, not the the snow maps, for any pro-snow bias. The funny thing is. This isn't the first time this has happened and probably won't be the last.

  • Like 1
Link to comment
Share on other sites

20 minutes ago, Chubbs said:

A couple of comments:

The public impact of sleet is very similar to snow. So for most people the forecasts worked well. There was plenty of warning that a disruptive storm was coming and schools, business, airlines, families, etc. prepared/adjusted accordingly to minimize disruption. However a story on how well the area prepared has limited appeal.

I personally like snow maps. Just like any other weather map they are a handy way of communicating and visualizing patterns. The problem is bad snow maps. However if the error is due to poor modeling removing the snow map won't solve the problem.

With a major snowstorm coming, social media/internet is going to have its pluses and minuses with or without snow maps. In the long run I think most people are smart enough to figure out who is giving them good information.

In a sleet storm a map that includes sleet as snow is useless. Not sure why these maps persist since it should be a relatively easy fix.

Regarding #2 from the top post. Even us weather board amateurs know enough not to rely on a snow map to make a forecast. I checked soundings before the storm and they were generally consistent with the maps. Models that were forecasting sleet had sleet soundings and vice-versa. Snow maps were not the cause of bad forecasts. Which brings me to the last point.

As humans we all have confirmation bias that is very difficult to shake. If you like snow you will look for reasons why it will snow. If you hate snow you will look for reasons why it won't snow. Most weather board hobbyists like big storms and snow so we tend to look for ways it could snow. Heavy snow forecasts once made tend to be sticky. To me as the storm approached sleet appeared more and more likely in our area.  Yet I kept looking more aggressively for reasons to support a heavy snow forecast than for reasons to support a sleet forecast. I knew it was going to sleet but couldn't quite pull the trigger on a big sleet forecast, even though I had enough information to do so. Of course hindsight is 20/20 and if the models were more aligned on a sleet forecast I would have done better, but I can only blame myself, not the the snow maps, for any pro-snow bias. The funny thing is. This isn't the first time this has happened and probably won't be the last.

Charlie, I guess for point #2 the increasing problem I saw before I retired all the Tom, Dick & Harrys & Authorities/Experts just ripped and posted those maps (the one with greatest snowfall of course) on social media.  Then we would have to spend valuable resources on "rumor control".  So I must admit my bias is not great about these maps because of the amount of wasted time I remember answering questions about whether or not e.g. that 30" Ggem map for NJ is right/possible.  This is even exacerbated further that Mount Holly is working 4 people short (out of 19 operational support positions) now.  There is the better ptype snowfall Cobb Method  which doesnt seem to get much map generation, not sure if its a resource hog and why it isnt used. Ultimately even Cobb is only as accurate as the model.  Last GFS run prior to the snow Cobb gave PHL 13", which was still twice too high, but better than those other maps/methods that gave PHL 20" and Kuchera with its 23.5".  No matter what my opinion of this is, they are not going away, I just hope the algorithms and model used improve so that the op community has to spend less time debunking fake snow.

 

 

  • Like 1
Link to comment
Share on other sites

BTW, I am waiting for the Cocorahs to come in to corroborate the PHL water equivalent,  the SREF ensemble mean was the most consistent/accurate for snowfall in PHL (range was 5.2 to 9.2 inches last 6 sounding runs prior to the snow starting). Its not as if it was just low across the board, I posted a couple of graphs that included AVP and it had around 20" for them.

Link to comment
Share on other sites

14 minutes ago, Chubbs said:

A couple of comments:

The public impact of sleet is very similar to snow. So for most people the forecasts worked well. There was plenty of warning that a disruptive storm was coming and schools, business, airlines, families, etc. prepared/adjusted accordingly to minimize disruption. However a story on how well the area prepared has limited appeal.

I personally like snow maps. Just like any other weather map they are a handy way of communicating and visualizing patterns. The problem is bad snow maps. However if the error is due to poor modeling removing the snow map won't solve the problem.

With a major snowstorm coming, social media/internet is going to have its pluses and minuses with or without snow maps. In the long run I think most people are smart enough to figure out who is giving them good information.

In a sleet storm a map that includes sleet as snow is useless. Not sure why these maps persist since it should be a relatively easy fix.

Regarding #2 from the top post. Even us weather board amateurs know enough not to rely on a snow map to make a forecast. I checked soundings before the storm and they were generally consistent with the maps. Models that were forecasting sleet had sleet soundings and vice-versa. Snow maps were not the cause of bad forecasts. Which brings me to the last point.

As humans we all have confirmation bias that is very difficult to shake. If you like snow you will look for reasons why it will snow. If you hate snow you will look for reasons why it won't snow. Most weather board hobbyists like big storms and snow so we tend to look for ways it could snow. Heavy snow forecasts once made tend to be sticky. To me as the storm approached sleet appeared more and more likely in our area.  Yet I kept looking more aggressively for reasons to support a heavy snow forecast than for reasons to support a sleet forecast. I knew it was going to sleet but couldn't quite pull the trigger on a big sleet forecast, even though I had enough information to do so. Of course hindsight is 20/20 and if the models were more aligned on a sleet forecast I would have done better, but I can only blame myself, not the the snow maps, for any pro-snow bias. The funny thing is. This isn't the first time this has happened and probably won't be the last.

No doubt my gut sensed sleet was a strong possibility and yes I agree a confirmation bias exists for the amateur weather community in general, it's how we get our jollies.  I would think some in the professional community have a snow weenie bias in them too if they are lovers of a good snowy noreaster.  It's events like this that spark the interest of a young wide eyed kid that realizes a passion for the weather exists at a young age.  That doesn't ever get shaken off.  The mystique and excitement keeps many of us engaged no matter what color our tags are.

When watching Glenn Hurricane Schwartz (side note I don't even have cable so this is the only time I watch the news, for old times sake) had a noticeable uncertainty with the rain/snow line and his forecast.  He trimmed numbers back but he lacked the confidence and enthusiasm he typically has over those numbers.  I really think he saw the writing on the wall but was too deep into the process to make drastic changes.  There must be incredible pressure in that highly visible community to risk busting LOW on forecasts given hype sells, and in some core sense I'd rather see them over prepare the public for worst case than the contrary.

On the flip side, there isn't much more that brings an entire community together than the forecast of a large storm.  It's amazing.  Grocery stores, beer and liquor, home improvement stores are buzzing with the fervor.  Strangers can share a brief conversation on the impending storm, neighbors who never speak chat about the common topic.  There is a physiological element that kicks in (and/or cultural) and gets us into a strange but intoxicating survival mode.   I mean seriously, the odds of a healthy person dying in a storm like this, outside of poor decision making (driving, hiking, skydiving) are extremely low.  Shoveling sucks as does losing power, but refer back to neighbors helping each other out to solve most of those problems.

Ultimately, it was an incredibly difficult storm.  The kind of CLASSIC Philly noreaster with an I95 battle zone and Poconos jackpot.   It's been years since we've had one.  We all might be a little rusty given the southern leaning massive snow totals of storms over the past 8 or so.  Forecasters have to decide on a specific forecast but could be crystal clear on a contingency forecast (IF sleet mixes in earlier than expected totals could be THIS) but in reality that's just another reason for the public to bash the merit of weather forecasting.  It's almost lose-lose.

The blizzard warning was questionable and bold, even if it had remained snow.  There was limited wind and marginal model output for justiflying interior counties to have that.  Snowfall rates would have been next level but if the storm had pushed more off the coast we may not have seen those extreme precip rates.  Those 2-4" rates I suspect are what justified the blizzard warning more than anything.

It was an amazing storm.  Seeing it come together on satellite and water vapor was incredible.   One of the best ever, despite the outcome.  It made my winter.  8 days of tracking, about as good as it gets.  

Weenie satisfaction level - A+.  Thanks for coming along for the ride and thanks to the pros for everything they do.  

  • Like 3
Link to comment
Share on other sites

But since snow maps are not going to disappear, I will talk out of both sides of my mouth and say that these mesoscale ensemble model mean maps that NCAR generates do a pretty good job storm in and storm out.  This was the last available run (there is a big lag time for display) before the snow started.  They dont go past 48 hours, which is very smart. They also hit the ice very hard right along the Delaware River (separate map below).

snowacc_mean_f033_MATL.png.08cea8c2a80d0f3dc43937cfb43c603b.png

fzraacc_mean_f033_MATL.thumb.png.bb8b2e50700036269ec6b677b0646119.png

  • Like 1
Link to comment
Share on other sites

56 minutes ago, Rainshadow said:

Charlie, I guess for point #2 the increasing problem I saw before I retired all the Tom, Dick & Harrys & Authorities/Experts just ripped and posted those maps (the one with greatest snowfall of course) on social media.  Then we would have to spend valuable resources on "rumor control".  So I must admit my bias is not great about these maps because of the amount of wasted time I remember answering questions about whether or not e.g. that 30" Ggem map for NJ is right/possible.  This is even exacerbated further that Mount Holly is working 4 people short (out of 19 operational support positions) now.  There is the better ptype snowfall Cobb Method  which doesnt seem to get much map generation, not sure if its a resource hog and why it isnt used. Ultimately even Cobb is only as accurate as the model.  Last GFS run prior to the snow Cobb gave PHL 13", which was still twice too high, but better than those other maps/methods that gave PHL 20" and Kuchera with its 23.5".  No matter what my opinion of this is, they are not going away, I just hope the algorithms and model used improve so that the op community has to spend less time debunking fake snow.

 

 

Tony - I guess I am a little out of touch since I don't do social media. What you describe is certainly is a poor use of limited NWS resources, however, I'm guessing that social media is the main problem not snow maps. You are still going to have every Tom, Dick and Harry with a potentially large audience. As soon as someone figures out you can put 30" on a map and get hits the game is over. It would be interesting to try an experiment for a couple potential snowstorms without any snow maps to see what would happen. If indeed snow maps are wasting NWS resources perhaps a blanket NWS policy to disavow and ignore all non-NWS maps is needed.

Link to comment
Share on other sites

One question I had is Chester County was lumped into the 18-24 warning but on the snow maps they were never projected above 15". What was the thinking behind this? Just curious, mostly bc my sister is complaining she didn't get 20+ lol

Link to comment
Share on other sites

53 minutes ago, Rainshadow said:

But since snow maps are not going to disappear, I will talk out of both sides of my mouth and say that these mesoscale ensemble model mean maps that NCAR generates do a pretty good job storm in and storm out.  This was the last available run (there is a big lag time for display) before the snow started.  They dont go past 48 hours, which is very smart. They also hit the ice very hard right along the Delaware River (separate map below).

I agree they have been solid this winter. Do you have a link.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...