Thursday, June 26, 2014

Ratings, Difficulty and the World's First 9c/15d

Antonio on No Skill (12c) Owl Tor, Santa Maria, CA

I guess 12c is a lot
harder than 12b.
 Antonio Labaro             

I've planned on writing something about ratings and difficulty for a while, but Antonio's comment finally motivated me to do the work. Antonio has been progressing through the grades at the Owl Tor and in April he redpointed The Hell of the Upside-Down Sinners, his first 5.12b. Full of psyche and energy, he moved on to No Skill, a 5.12c variation of two other routes. He's doing well, but he did notice: yes, in fact, 12c is a lot harder than 12b.
   But how much harder? And aren't ratings subjective? Most climbers talk about them as if they are. I haven't seen this done before, and I wish I had more data, but we need to start somewhere and try to have an objective discussion about ratings.

Why Ratings Exist
Ratings exist for a reason: to give someone an indication of how hard something is to do. What does that mean? Well, if 100 people can do task #1 and 50 of them can do task #2, and 25 can do task #3, then a logical rating system might rate task #1 a 1 and task #2 a 2, and task #3 a 3. Clearly, the symbol — 1, 2, 13a, 8c+, IV — is irrelevant. The point is that the relative likelihood of doing a task is represented by certain, agreed upon, symbols. If something changes regarding task 2 which now enables 150 people to do it, it would be illogical to still rate it a 2, it might, perhaps, be rated a 0. All of this should be obvious.
   Back to climbing ratings... I'm sure you all know that climbers (in America) have agreed that the scale starts at 5.0 and increases until 5.10 where the grading symbol gains an a,b,c,d. Currently, the scale tops out at 5.15c. In context of the above example, it is quite apparent that ratings — as a whole — are not arbitrary or subjective. We can all agree that fewer people can climb 5.15c than 5.9. I know, this is boring, but I have to start somewhere.

An Objective Scale
Let's add another "rating" system to the analysis: weights. Clearly, people typically don't think of weights as a rating system, but that's exactly what they are — or can be used for. That way, when you walk into a gym to do curls, you might grab the 25 pound dumbbells, but probably not the 100 pound dumbbells. (That would be rad.) Weights highlight a key element of how I'm going to look at difficulty. Let's suppose someone asks the question, "How much more difficult is it to bench press 600 pounds than 300 pounds?" Should the answer be "twice" as hard? And, using the same logic, is 900 pounds "50%" harder than 600 pounds?
   Although you could do that, I don't see how it is in any way productive. Here's why: tens of thousands of people can bench 300, but only hundreds of people can bench 600. At a minimum, 100 times as many people can lift 300 as 600 and around 5 people can bench 900. So it is much more helpful and informative to call 600 pounds 100 times harder than 300 pounds, and 900 pounds about 100 times harder than 600 pounds. This is how I will use the term "harder" when describing the increased difficulty between climbing ratings.

What Are Climbers Doing?
How hard is "hard?" How much harder is one grade than another? I know many readers are convinced that answers to questions like these are meaningless and subjective. But just because the rating on one route, for one person, can be subjective does not mean that the aggregate is subjective; and I do not see why we don't try to put some objective analysis behind it. So, sticking with the approach described above whereby something is harder if fewer people are doing it and how much harder is determined by the relative decline in people doing it, we need data on what climbers are actually doing. I looked at this in three different ways...
Rating Climbers "Harder" 3.61
9b+/15c 1 2.0 1
9b / 15b 2 8.5 3.6
9a+ / 15a 17 4.9 13
9a / 14d 84 2.0 47
8c+ / 14c 170 170
   First, I went to and downloaded the data for all climbers that have redpointed 8c+/5.14c or harder; 170 climbers have done this. (Here is the raw data; the "climbers" tab.) Then I looked at the rate at which these climbers drop off as the difficulty increases. 84, or 49.4%, of the 170 climbers have redpointed 9a/14d. This would indicate that 9a/14d is twice as hard as 8c+/14c — because twice as many people are doing 8c+/14c. Next, 17 climbers redpointed 9a+/15a, this is a 79.8% decline; meaning almost five times as many people have redpointed 9a/14d as 9a+/15a. Of those 17 9a+/15a climbers, only 2 have redpointed 9b/15b, translating to it being 8.5 times harder. Finally, only one of those two has redpointed 9b+/15c making it twice as hard. So over four grades (8c+/14c to 9b+/15c) the numbers of climbers drops from 170 to 1; this means that averaged over these four grade increases, each grade is 3.61 times harder than the previous (3.61^4 = 170.)
   Second, I looked at how many redpoints the 84 9a/14d climbers have done. (Downloading redpoint data for all 170 in the set above was too epic.) There were 4360 redpoints graded 8b+/14a or harder for this group. This data is summarized in the chart below; see the "redpoints" sheet in the raw data. The "Harder" column again indicates how many times more the previous grade was climbed; i.e., 9a/14d is 2.4 times "harder" because 8c+/14c has been climbed 2.4 times more often. The three columns on the right average how hard it is to move between different grades. As we would predict, as the grades get higher/harder, moving between them increases in difficulty; that is, it is easier to go from 14a to 15a than from 14c to 15c. Here, the data would indicate that moving from 8c+/14c to 9b+/15c is, averaged over the four grade increases, 3.95 times harder — quite close to the finding above of 3.61. For the low end of the difficult range (14a to 15a) the data shows an average of each grade being 2.4 times harder than the previous; this is also close to the result above. Below is an attempt to combine the two approaches and see what the tail end of the difficulty curve looks like.

Rating Redpoints "Harder" 14a to 15a 14b to 15b 14c to 15c
9b+ / 15c 3 3.3
9b / 15b 10 5.2
1.0 3.3
9a+ / 15a 52 5.9 1.0 5.2 17.3
9a / 14d 305 2.4 5.9 30.5 101.7
8c+ / 14c 729 1.9 14.0 72.9 243.0
8c / 14b 1420 1.3 27.3 142.0
8b+ / 14a 1841 35.4
Avg. "harder" over 4 increments: 2.44 3.45 3.95

My intent is not to achieve a scientific "proof" around the difficulty question, but I do think that facts will help us approach a reasonable estimation of the relative difficulties involved. At the very top of the scale, both approaches indicate that each grade is four times harder than the previous. (There are 17 15a climbers for 1 15c climber and 52 15a redpoints for 3 15c redpoints,also a 17 times multiple. Then we take the square root of 17 (4.123) since this is over two grade increments. The raw data for this chart is visible on the "aggregate" tab.) So if you're ever wondering how hard 9b+/15c is, I'd say, "about 500 times harder than 8b+/14a."
   Of course, there are some problems, and many over-simplifications with this approach, but as I said at the beginning, you have to start someplace. Overall, the data indicates that grades are large ranges, 14b is definitely not 10% harder than 14a — it's closer to 60% harder. At lower grades, the difference between grades will lessen. My guess would be that (if we had the data) it would show the 5.13a-d range having a 30-50% drop-off between each grade; and the 5.12a-d range having a 20-30% drop-off.
   If anyone wants to write a basic data-scraping bot to download all the data we could probably learn a lot more, and maybe look at the bouldering data as well.

Adam Ondra
Adam Ondra's Data
Rating Date Months Redpoints
7b+ / 12c 7/28/2001 13
7c / 12d 5/10/2002 10 15
7c+ / 13a 7/20/2002 2 14
8a / 13b 8/20/2002 1 241
8a+ / 13c 9/14/2003 13 238
8b / 13d 4/10/2004 7 210
8b+ / 14a 6/4/2004 2 119
8c / 14b 12/31/2004 7 127
8c+ / 14c 12/29/2005 12 108
9a / 14d 11/11/2006 11 65
9a+ / 15a 2/10/2008 15 19
9b / 15b 3/13/2010 25 9
9b+ / 15c 10/4/2012 31 3
9c / 15d 1/7/2016 40
9c+ / 16a 2/14/2020 50
10a / 16b 12/24/2025 71
Finally, it's always interesting to put a name to the data; and here, we're in luck... because Adam Ondra has entered a lot of redpoints into — he's redpointed 450 routes rated 8b+/5.14a and higher. In looking at his data, I also wanted to get a picture of how long it has taken him to progress between grades (since that was the question that started us on this journey.)
   At the lowest end of the rating spectrum (that we have data for) we can see that it took Mr. Ondra 25 months to go from 12c to 13c, or basically 6 months per grade. This rate continues, generally, up to 8c/14b where it begins to slow noticeably; as would be expected.
   Finally, for fun, I forecast when he might climb grades beyond what anyone has done — 9c/15d (January 7th, 2016) and higher. So that's about 18 months away from now for 15d, seems like a pretty good over/under date for the gamblers out there. (If pressed, I'd take the over.)
   Antonio, I think you'll get No Skill around November, plus or minus a month, based on data. So how much harder is it? Well, it's 6-9 months of hard work harder.

I'd like to welcome Elijah to the local 1-5-8 club. I believe he is the fourth member (with Dean, Thomas, and myself.) (1" rungs, 22cm spacing, clearly.) Hopefully there will be more soon, now that SBRG has a campus board. Hans and the Touchstone team better start working if they ever want to win the team title. Maybe I'll do a write-up similar to this one on campus board move difficulty someday.

Hard Things
Finally, it's always good to mention Neil deGrasse Tyson, here's his input on hard things...