Every year after NXN, I run an analysis to see how the various NXR courses compare based on times ran at NXN (including removing outliers, so this isn't influenced by how well specific individuals ran but rather the groups of individuals ran after discounting outlier performances). This accomplishes two goals: identifying what regions I might have over/undervalued going into NXN, and estimating just how fast the course at NXN should be rated.
The results from last year's NXR and NXN analysis gave the following results:
200.0 mark for NXN Nationals Girls race should be about 15:02
200.0 mark for NXN Nationals Boys race should be about 15:11
(Side note: I feel good about those ratings, I think they are very fitting. For anyone reading these that want to compare to Meylan's speed ratings: they are derived differently, and it's not uncommon that I'm a little more generous in my ratings than Meylan is. That isn't always true, and for many meets we are close, but for whatever reason there are some circumstances where my ratings are consistently higher than his -- I have my theories as to why that difference exists and is as notable as it exists, but it would only be guesses and not worth spending time talking about ... so just realize that Meylan's speed ratings and my own form of ratings, beyond the difference of how they are calculated e.g. Meylan's +/- system vs. my multiplier system, are not identical and can't really be compared directly)
200.0 mark for each regional race:
14:43 NXR Southwest
14:44 California State
14:45 NXR Heartland
14:53 NXR Southeast
14:53 NXR Midwest
15:12 NXR Northwest
15:20 NXR South
15:32 NXR Northeast/New York
Taking those differences in the course, I thought it would be interesting to see how the differences might look BEYOND my usual methods, just as a check to see if I'm significantly over/under estimating any regional race this year.
Milesplit has a pretty neat tool that allows you to compare how the same races compare from year to year. Using that tool, and limiting my scope to the runners that ran at both races and taking the average time, average of the middle 80% of times, average difference for specific runners, and average difference of the middle 80% for specific runners (those are the measurements that seemed most consistent from region to region), I ran some numbers on how much faster/slower the courses seemed to run this year vs. last (if we assume that every region progresses year-to-year at roughly the same rate).
[EDIT - I found the reason why the Northwest runners improved by comparably less ... for some reason, Milesplit didn't bother to log the boys races in 2018! Serves me right for not double-checking Milesplit's data I guess ... as a result, I'll be removing the Northwest from this comparison, and revised the rest of the regions to look at data from the boys races only only ... which didn't change the results too much as would be expected unless a region had significantly more girls returning than boys, but aside from New York things did become slightly more in-line with what I was seeing while rating the different regional races]
For all of those measurements, the median differences throughout the regions were around 35 seconds improvement from last year to this year. I considered that the standard, and any region that improved by less than 35 seconds is an estimate that the course ran that much SLOWER (e.g., the same Midwest athletes average times were 22.9 seconds faster in 2019, so that would suggest the course ran 12.1 seconds slower this year compared to last: 35.0 - 22.9 = 12.1).
This isn't a perfect analysis for several reasons (it relies on the accuracy of milesplit's compiled data which I'm not double checking, different training programs will in general progress at different rates, environmental situation may influence how much a these runners progress from year to year e.g. less likely to train year-round in hot/cold regions or more likely to progress more significantly if living and training at altitude, it goes beyond the championship races for both years which is the focus of the ratings and the racing situation can influence times between different races, etc. etc.), which is why I'm not changing my regional ratings due to the findings, but it provides an idea of if I'm maybe on the right track.
The results, region-by-region:
NXR Heartland was rated at 14:45 last year in the final review. The analysis (702 boys) shows this year it should be rated anywhere from 14:41-14:46. I rated it a 14:46, so that's a pretty good fit!
NXR Midwest was rated at 14:53 last year in the final review. The analysis (676 boys) shows this year it should be rated anywhere from 15:02-15:05. I rated it at 14:58, so maybe a slight underrating of the region, but 4-7 seconds is within the margin of error IMO especially if it is on that lower end.
NXR Southwest was rated 14:43 last year in the final review. The analysis (683 boys) shows this year it should be rated anywhere from 14:55-14:59. I rated it at 14:50, so this could be a notable underrating by 5-9 seconds!
NXR South was rated at 15:20 last year in the final review. The analysis (513 boys) shows this year it should be rated anywhere from 15:14-15:15. I rated it at 15:14, so that's a pretty good fit!
NXR Southeast was rated at 14:53 last year in the final review. The analysis (252 boys) shows this year it should be rated anywhere from 14:49-14:50. I rated it at 14:53, so that's a pretty good fit!
California State was rated at 14:44 last year in the final review. The analysis (319 boys) shows this year it should be rated anywhere from 14:43-14:44. I rated it at 14:40, so that's a very good fit!
I don't rate the Northeast and New York races separately, but rather combine them, UNLESS there is a situation where the course conditions change notably between the races (e.g., a really muddy course that got worse as the day wore on). Unfortunately, Milesplit's tool looks at NXR-NE and NXR-NY separately, but these stay mostly in the same ballpark so it's not too big of a deal.
NXR Northeast was rated at 15:32 last year in the final review. The analysis (218 boys) shows this year it should be rated anywhere from 15:33-15:34. For NXR New York, the analysis (148 boys) shows anywhere from 15:27-15:30. Combined, that suggests anywhere from 15:27-15:34 but probably closer to 15:30-15:33. I rated it at 15:37, so that's a pretty decent fit but maybe slightly overestimating by as 5 seconds or so.
We will find out on Saturday how much I over/under estimated the regional races. If the Southwest teams and individuals are notably more dominant than I already have them and Northeast/New York less competitive, I guess I need to be more aggressive with my year-to-year adjustments and that the regions are even less balanced that I thought!
BTW, here's a summary table of the analysis for those that might want to dig a little deeper.
2018 Rating - Region - Avg. Time - Middle 80% Time - Avg. Difference - Middle 80% Difference - Region - Averaged from Analysis - 2019 Rating
14:45 - HL - 14:46 - 14:42 - 14:46 - 14:41 - HL - 14:44 - 14:46
14:53 - MW - 15:05 - 15:03 - 15:05 - 15:02 - MW - 15:04 - 14:58
14:43 - SW - 14:59 - 14:55 - 14:59 - 14:55 - SW - 14:57 - 14:50
15:20 - SO - 15:15 - 15:14 - 15:15 - 15:14 - SO - 15:14 - 15:14
14:53 - SE - 14:50 - 14:50 - 14:50 - 14:49 - SE - 14:50 - 14:53
15:32 - NE - 15:34 - 15:34 - 15:34 - 15:33 - NE - 15:34 - 15:37
15:32 - NY - 15:27 - 15:29 - 15:27 - 15:30 - NY - 15:28 - 15:37
14:44 - CA - 14:43 - 14:43 - 14:43 - 14:44 - CA - 14:43 - 14:40
Since I consider 5-10 seconds to be a reasonable margin of error for comparing across different regions and different races (human performance is variable, and a runner being 5-10 seconds off wouldn't be considered an outlier in a statistical comparison), all of the regions would be within at least the larger side of that margin. The Southwest (+7), Midwest (+6) and New York (-9) are beyond 5 seconds but all less than 10, meaning I might be underrating the Southwest and Midwest while overrating New York -- but since New York and the Northeast run on the same course and the Northeast fits, I don't think the idea that New York is potentially being slightly overrated will end up playing out. The other 5 regions (sans Northwest, which Milesplit doesn't have the data for) are within 0-3 seconds of how I had them rated, which is a very good fit.
Post-NXN Nationals EDIT: After the conclusion of the national race, the post-race analysis had these as the suggested regional ratings:
(Boys Nationals = 15:24, Girls Nationals = 15:46)
NXR Heartland: 14:46 vs. the 14:46 I had them at, and 14:44 from the analysis above
NXR Midwest: 15:04 vs. the 14:58 I had them at, and 15:04 from the analysis above
NXR Northwest: 15:10 vs. the 15:10 I had them at (and didn't include above because Milesplit didn't have the right data to use)
NXR Southwest: 14:52 vs. the 14:50 I had them at, and 14:57 from the analysis above
NXR South: 15:16 vs. the 15:14 I had them at, and 15:14 from the analysis above
NXR Southeast: 14:49 vs. the 14:53 I had them at, and 14:50 from the analysis above
NXR Northeast/New York: 15:39 vs. the 15:37 I had them at, and 15:32ish from the analysis above
California State: 14:46 vs. the 14:40 I had them at, and 14:43 from the analysis above
In review, I was slightly underrating the Midwest and California (though not by a large margin, only 6 seconds), to an even lesser degree I slightly overrated Southeast (4 seconds), and every other region was within 2 seconds.
Post-NXN Nationals EDIT: After the conclusion of the national race, the post-race analysis had these as the suggested regional ratings:
(Boys Nationals = 15:24, Girls Nationals = 15:46)
NXR Heartland: 14:46 vs. the 14:46 I had them at, and 14:44 from the analysis above
NXR Midwest: 15:04 vs. the 14:58 I had them at, and 15:04 from the analysis above
NXR Northwest: 15:10 vs. the 15:10 I had them at (and didn't include above because Milesplit didn't have the right data to use)
NXR Southwest: 14:52 vs. the 14:50 I had them at, and 14:57 from the analysis above
NXR South: 15:16 vs. the 15:14 I had them at, and 15:14 from the analysis above
NXR Southeast: 14:49 vs. the 14:53 I had them at, and 14:50 from the analysis above
NXR Northeast/New York: 15:39 vs. the 15:37 I had them at, and 15:32ish from the analysis above
California State: 14:46 vs. the 14:40 I had them at, and 14:43 from the analysis above
In review, I was slightly underrating the Midwest and California (though not by a large margin, only 6 seconds), to an even lesser degree I slightly overrated Southeast (4 seconds), and every other region was within 2 seconds.
No comments:
Post a Comment