Forecast Verification Report for April 15, 2018 (Preliminary)
Figure 1. Official (filtered) storm reports from the National Weather Service Storm Prediction Center (SPC) from April 15, 2018.
On Sunday, April 15, 2018, a severe weather outbreak occurred across the Southeast, spawning hundreds of damaging wind reports and over a dozen tornadoes during the afternoon and evening hours on Sunday. The storm system responsible for the outbreak also produced widespread severe weather over the Central Plains, Midwest, and the Deep South on April 13-14, resulting in a combined total of nearly seventy tornadoes, along with myriad damaging wind reports. For the Carolinas, April 15 marked the first severe weather outbreak the region has seen since last May.
Western Piedmont Weather Authority (WPWA) began issuing official forecasts for the April 15 outbreak on Friday, April 13, although WPWA had been following the system since Tuesday, April 10.
The initial forecast released at 6:34 PM EDT on Friday, April 13 introduced a marginal risk of severe thunderstorms across the majority of the WPWA coverage area, with the exception of the eastern foothills.
Figure 2. WPWA Severe Weather Potential Outlook (SWPO) issued at 6:34 PM EDT, Friday, April 13, 2018.
According to the WPWA risk category descriptions, a marginal risk is defined as a severe thunderstorm environment characterized by: "scattered severe thunderstorms with a moderate risk of severe wind and hail. An isolated tornado cannot be ruled out."
At the time, this was the best forecast assumption. This assumption was based on careful analysis of the European (ECMWF) ensemble and Global Forecast System (GFS) for the large-scale weather patterns and the North American Mesoscale Model (NAM) with a blend of the Short Range Ensemble Forecast (SREF) for small-scale weather patterns.
In reality, this forecast was below average in terms of the impacts associated with the event on Sunday, April 15, 2018. Based on the amount of damaging wind reports alone, there should have been an elevated risk across the majority of the Piedmont. The marginal risk should have been extended further west into the eastern Foothills, with an isolated risk over the southeast mountains, extending into the western foothills. A general risk should have been placed west of the isolated risk.
On the other hand, the placement of the marginal risk was considerably accurate for a day three forecast, so the forecast was not entirely a bust.
So, now for the ESGVA (Estimated Severe to General Verification Assessment).
Placement/Location: A (93%)
Coverage/Intensity: C- (70%)
Data Analysis/Forecast Integrity: A+ (95%)
Final "Grade": 86% (B+)
On Saturday, April 14, WPWA upgraded southeastern portions of the NC Piedmont to an elevated risk at 06:42 AM EDT.
Figure 3. WPWA STPO issued at 06:42 AM EDT Saturday, April 14, 2018.
According to the WPWA risk category descriptions, an elevated risk is defined as a severe thunderstorm environment characterized by: "a severe weather outbreak is possible with an upper moderate risk of severe wind and hail. A few isolated tornadoes are possible."
Although the intensity forecast was correctly upgraded since the last forecast on Friday afternoon, this was the worst forecast in regards to placement of highest impacts. This was largely due to the fact that the forecast was based off the Short Range Ensemble Forecast (SREF) for small-scale details, as the NAM was slowly becoming an outlier in the guidance. The GFS, deterministic ECMWF, and the ECMWF ensemble were used to examine large-scale patterns.
The placement of the elevated risk was largely inaccurate due to poor analysis of all available model data. However, the placement of the marginal risk was slightly more accurate in terms of the concentration of severe weather on April 15.
Now, for the ESGVA.
Placement/Location: D+ (68%)
Coverage/Intensity: C- (73%)
Data Analysis/Forecast Integrity: C- (70%)
Final "Grade": 70% (C-)
By Saturday afternoon, most of the western and central Piedmonts were upgraded to an elevated risk of severe weather. Also on Saturday afternoon, WPWA began issuing specialized Dynamicast products for the event.
Figure 4. WPWA STPO issued at 4:15 pm EDT Saturday, April 14, 2018.
This forecast was based off several different models and also maintained some degree of continuity from the past two forecast issuances. The forecast was primarily based on the SREF and the High Resolution Ensemble Forecast (HREF), although more weight was placed on the SREF and to a degree, the NAM (this model was excluded for intensity). For large-scale details, the GFS was used along with the European ensemble.
The placement and location of impacts was the best in this forecast. The forecast verification report (pictured in Figure 5) shows that this forecast excelled in pinpointing the greatest coverage of severe weather. However, the forecast was not perfect as the marginal and isolated risks were shunted too far to the east. Otherwise, this forecast was nearly perfect for Sunday's severe weather outbreak.
Now for the ESGVA:
Placement/Location: B+ (88%)
Coverage/Intensity: A (93%)
Data Analysis/Forecast Integrity: A+ (98%)
Final "Grade": 93% (A)
Figure 5. WPWA STPO issued at 4:15 PM EDT with SPC storm reports overlaid on the map.
Legend:
Blue circles (damaging wind report)
Red stars (tornado report)
White diamond (hail report)
Because Dynamicast products were issued, there is a separate assessment that is combined with the outlook grade to yield a final forecast "grade".
Figure 6. WPWA Dynamicast Impact/Timing Meter issued at 4:00 PM EDT on Saturday, April 14, 2018.
The timing was based primarily on the HREF, and the impacts were based on a blend of the SREF and the NAM. The timing was only an hour off, and the tornado threat was a little on the high end. Also, the heavy rain threat was overestimated.
Timing: A- (90%)
Impacts: B-(80%)
Final Dynamicast "Grade": 85% (B)
Now, we will combine the forecast grade with the Dynamicast grade to yield a final value.
Overall Grade: 89% (B+)
On the morning of the event, WPWA downgraded the central NC Piedmont to a marginal risk and retained the elevated risk across the Charlotte Metro area and most of the Central SC Piedmont.
Figure 7. WPWA STPO issued at 06:45 AM EDT Sunday, April 15, 2018.
This forecast was largely influenced by satellite trends from GOES-16 and regional surface observations, which indicated considerably less instability across the Piedmont as the models had indicated the day earlier. This prompted the downgrade to a marginal risk for more areas north and east of the Charlotte Metro area.
Additionally, the HREF, SREF, RAP, and the WRF showed that the greatest overlap of severe weather parameters was along and directly east of the I-77 corridor. However, the HRRR was showing severe weather potential increasing across Central North Carolina later in the afternoon. This trend was not incorporated into the official forecast above, as it was a fairly new trend in the model data. However, what I failed to realize was that my previous forecast had supported the HRRR's views of increasing severe weather potential across Central NC. Had I maintained continuity with my previous forecast, I would have had the elevated risk almost perfectly placed in accordance with the actual storm reports on Sunday, April 15.
The main forecast reasoning behind the decision to downgrade most of Central NC was simply the satellite trends and the decreasing instability values on nearly all the convective allowing-models, including the HRRR. Additionally, the SREF supported this trend, which is what prompted me to downgrade areas north and east of the Charlotte Metro to a marginal risk.
Despite my rigorous analysis of the satellite and model trends, this reasoning would prove incorrect as depicted in the verification graphic shown below:
Figure 8. WPWA STPO issued at 6:45 AM EDT on Sunday, April 15, with SPC storm reports overlaid on the map.
Legend:
Blue circles (damaging wind report)
Red stars (tornado report)
White diamond (hail report)
Now, for the ESGVA:
Placement/Location: C (75%)
Coverage/Intensity: B (85%)
Data Analysis/Forecast Integrity: A- (90%)
Final "Grade": 83% B-
Because Dynamicast products were issued, there is a separate assessment that is combined with the outlook grade to yield a final forecast "grade".
Figure 9. WPWA Dynamicast Impact/Timing Meter issued at 6:30 AM EDT on Sunday, April 15, 2018.
HREF/HRRR trends had picked up on a possible faster storm motion than previously expected, so the arrival of highest impacts was pushed forward an hour to 4 pm. This was still about 90 minutes off from the actual timing, but was still better than our previous forecast. Also, heavy rainfall potential was over exaggerated to a high risk, but a medium risk could have sufficed.
Timing: A- (90%)
Impacts: A (95%)
Final Dynamicast "Grade": 93% A-
Now, we will combine the forecast grade with the Dynamicast grade to yield a final value.
Overall Grade: 88% B+
Final ESGVA:
All three forecast grades are average to yield the final forecast "grade."
Overall forecast grade: 81% B
Here is the rating scale for my forecasts:
A: Well above average forecast
B: Above average forecast
C: Average forecast
D: Below average forecast
F: Busted forecast
Summary and Reflections
I believe my largest mishap with this event was not maintaining enough continuity between forecasts. I relied on new model trends to refine my thinking on the weather event, instead of blending these new changes with previous forecasts. Even on the day of the event, I should have blended my previous forecast with current trends to place the elevated risk better than the official forecast. Below is what the outlook should have looked like had I payed attention to trends in my forecasts and maintained continuity:
Figure 10. WPWA unofficial, corrected STPO for Sunday, April 15, 2018 based on previous WPWA official forecasts and SPC storm reports/NWS radar imagery.
