Summary: lays out a skill leveling system based on time. I don’t endorse it, but maybe you’ll have fun thinking about it.

Back in 201X[1], I was thinking about how to convey mastery of a skill.

For example, at some point I was talking to a young kid that also played the violin. The kid was playing through the Suzuki instruction program, and since he knew I also played the violin, they asked me what Suzuki book (1-10) I was working from at the moment. For context, I was in late high school/college, and had barely touched any of the Suzuki books (since I didn’t go through the program). And, the last book just contains Mozart’s Concerto 4, which is not difficult in the grand scheme of things[2].

I told the kid that there was an entire other universe outside of the Suzuki books, and at some point you have to leave them behind. The kid wandered away looking confused and unconvinced, but it got me thinking: how do I convey just how much distance they have to go? Or, turn it around: what would I need to hear in order to make the call “yes, this person is way better than I am at $TASK“?

Some fields have it easy, with rankings built in: chess and Elo come to mind, as does Go and Dan ranks. Or if you’re an insider, you can use your knowledge to make judgments: for example, maybe you just happen to know that Mendelssohn’s violin concerto is harder to play than Bach’s Concerto for Two Violins, or that one youth symphony is generally known to be better than another.

But if you don’t happen to be looking at a field that self ranks, or a field that you deeply know, is there a way to get a sense of someone’s mastery?

What if we used time spent practicing?

Time Leveling

The general idea is that the longer someone has spent practicing a skill is correlated[3] with how good the person is at that skill.

While thinking about this around 4 or 5 years ago, I randomly came up with the following way to map hours of practice to a “skill level”:

minimum hours of practice = level · 2^level

So, for the first 10 levels:

• Level 1 → 2 hours
• Level 2 → 8 hours
• Level 3 → 24 hours
• Level 4 → 64 hours
• Level 5 → 160 hours
• Level 6 → 384 hours
• Level 7 → 896 hours
• Level 8 → 2048 hours
• Level 9 → 4608 hours
• Level 10 → 10240 hours

It’s appealing because the levels work out to a logarithmic relation with hours, and people love logarithmic magnitude progressions. Plus, it seems to intuitively fit the range from not knowing anything about about the skill, up to the point that one should be more than competent at the skill. I’ll give examples in the next section to help show that the intuition seems correct. And, the system reaches level 10 around 10000 hours (from the 10000 hours to mastery meme), which makes it easy to re-derive the exact hour calculation.

Why hours? Minutes are overly fine grained, and days are too coarse grained: practicing 1 minute and 24 hours are vastly different outcomes, both of which might count as “I practiced today”. While it’s possible to use fractions of a day instead, hours are easier to think about as a sub-day increment, instead of a unit that is conflated between 24 hours and “around 16 hours of being able to do things”[4].

Examples

To demonstrate, I’ll run some Fermi estimates[5] of hours of practice I have done for different skills. I’ll be conservative with most of my estimates, because much of my practice was done before I had really rigorous time tracking[6].

Level 9 Programmer

I started programming around the end of middle school, but in a fantastically undisciplined and sporadic way, where sometimes I would hack feverishly through the night, and other times would do nothing for long periods of time. Surprisingly, this continued into actually joining a CS program, so I extended the range of really uncertain estimates[7] over 3 years of HS and 5 years of college.

8 years · 12 months/year · 1-10 hour / month = 72-720 hours

I graduated and went off to full time work, which is really where I started putting in lots of time to coding, minus weekends and some number of vacation days[8]. I’m uncertain whether I should discount from 8 hours/day quite so much, but I think it’s right for a target between deliberate practice and ass-in-seat time.

6 years · 250 weekdays/year · 3-4 hours / weekday = 4500-6000 hours

Total: 4572-6720 hours, just meeting[9] or well over the level 9 limit, 4608 hours.

Level 7 Violinist

I started playing in the 5th grade, and wasn’t especially serious about it until the 8th grade. The summer was especially bad, so I’ve heavily discounted a large portion of the year.

3 years · 2/3year · 52 weeks/year · 0.5 hour/week = 51.5 hours

Then I got a private teacher, joined a youth symphony, and I started putting in more time during high school[10].

4 years · 50 weeks/year · 2-5 hour/week = 400-1000 hours

I continued into college, and was a bit more studious. And then I moved to New York, and basically stopped playing. I do vaguely remember having my shit more together during college, but I figure that weekly figure is better on the somewhat conservative end.

3 years · 50 weeks/year · 4-8 hour/week = 600-1200

Total: 1051.5-2251.5 hours, solidly in level 7 (896 hours) but not really solidly in level 8 (2048), so I rounded down.

Level 4 Statistician

I know I’ve picked up things like the normal distribution by osmosis throughout life, but have no idea how to price that in, so I’m leaving it out for now as “not even trying to learn statistics”.

0 hours

However, I do know that high school was basically worthless.

0-4 hours

I took a statistics course during college, which was at best mildly better, and required actual thinking.

8-16 hours

I did pick up some probabilistic thinking from LessWrong, but I question whether anything beyond “An Intuitive Explanation of Bayes’ Theorem” really made a difference. However, it was great at making me care about probability/statistics.

1-8 hours

More recently, I spent some time doing statistical work while reanalyzing a game development postmortem analysis. I time tracked the entire thing, so I have a more precise figure without uncertainty[11].

13 hours

I somewhat recently read Jaynes’ textbook Probability Theory. Also time tracked.

101 hours

And, I spent 10 hours investigating how to use Stan, a statistical tool.

10 hours

And I’m currently reading/working through problems in the textbook Statistical Rethinking, which compared to “Probability Theory” is a billion times more practical and billion times less theoretical[12].

9 hours (as of mid-August)

Total: 142-161 hours, solidly above level 4 (64 hours), but not well into level 5 (160 hours) yet[13].

Level 4 Drawer

Much like many children, I spent a fair amount of time in school and at home scribbling at random. However, I didn’t do that much of it, and instead spent a lot of my post-primary school days reading instead of scribbling[14], so I’m heavily discounting the time I would have spent. Plus, I don’t have a good way to estimate my time doing art in primary school.

8-16 hours

More recently, after the point I had actually grown into a Real Person™, a friend gave out drawing lessons, and afterwards I spent a fair amount of time practicing sketching[15].

12 weeks (~3 months) · 2-4h/week = 24-48 hours

Since then, I’ve done some more miscellaneous doodling, for example in support of my current avatar image.

4-8 hours

Finally, I spent some time working on art for a puzzle. It didn’t go anywhere, but there’s still sketches saved to my hard drive, and I time tracked this work.

20 hours

Total: 56-92 hours, ranging from just under or well into level 4 (64 hours).

(Point in favor of the leveling system: I’m not a good artist, but even the short amount of time I’ve dedicated to actual practice has gotten me visibly better at drawing, compared to a random sampling of software engineering coworkers who could barely put out stick faces during a recent brainstorming exercise.)

Level 1 Potter

We’re going to discount all the time I spent playing with playdoh as a kid. Indeed, the only time I’ve worked with ceramics is in the last month, when I took a 2 hour wheel workshop.

2 hours

Total: 2 hours, exactly the threshold for level 1 (2 hours), going from “completely clueless” to “can slap clay on a wheel and intellectually understand how things have gone so wrong“.

Downsides

All that said, I don’t think this system is useful as anything other than a curiosity.

Practice is heterogeneous

Not all practice is equal. Deliberate practice yields greater dividends than forced feet-dragging practice. Practicing in bad conditions (say, before getting coffee) is not as useful as practicing when awake and alert[16]. People themselves are quick witted or slow to learn[17].

And this is all before considering what is being practiced. Practicing tic-tac-toe has a much lower skill ceiling than playing Go, and playing the kazoo has a much lower skill ceiling than playing anything else[18].

That is, someone at some level could be worlds better/more impressive than another person at the same level, and the system makes no effort to distinguish between them.

10000 hours is bullshit

The system maps level 10 to a number of hours close to 10000, which Malcolm Gladwell talks about in Outliers as the amount of time before you get to mastery. This is a blessing, because the 10000 hours meme[19] is everywhere and thus easy to remember. It is also a curse, because it’s a bad meme.

There’s a kernel of truth to it: you need to put in lots of time before becoming a master. But, as explained in the book Peak, a sort of response to Gladwell, Anders Ericsson explained that his research, which served as the source for Gladwell’s 10000 hour limit, is at best approximate. Deliberate practice matters, and if you don’t do that sort of focused practice, you can plink away at the hours and barely make any progress.

So the 10000 hour link isn’t one that should be reinforced. Or, maybe the sort of disclaimer I just gave would arrive intact to whoever hears about this system, but it’s more likely that some radical simplification happens in transmission and nobody hears the more nuanced take.

(On the other hand, this system is a radical simplification of practice and mastery already; maybe it deserves to be tied to Gladwell.)

Equivalence with plain hours

Once you have an estimate of hours practiced, one can convert hours to levels. But, the two are roughly equivalent, so why go through an extra step?

You could argue the wide buckets are helpful:

• since the buckets are so wide, people won’t be tempted to spend lots of effort getting a precise estimate that doesn’t matter.
• again, it maps to a logarithmic understanding of the world, which seems correct: practicing for twice as long usually does not mean someone becomes twice as good.

However, what if given an hour count, we simply threw away all digits aside from the most significant one? So 4356 hours would map to 4000 hours, and 356 hours would map to 300. This has many of the same benefits as the system (aside from the logarithmic understanding), while avoiding an arbitrary division of hours into levels. The main downside would be that it’s only a step away from just reporting a precise hour count, and then we’re back at doing detailed accounting, just to show up your rivals by practicing 10 hours more than them even at 4000 aggregate hours.

Adoption could have bad effects

Remember Goodhart’s law? “When a measure becomes a target, it ceases to be a good measure.” It would be easy to make this system a target (contrast this with IQ, which is not easily movable[20]). If we apply optimization pressure towards leveling, the obvious point of failure is that the system cannot capture how deliberate any practice is, so mindless practice is rewarded.

And then I do wonder if calling out the lifetime amount of time practiced would damage intrinsic motivation. Again, applying optimization pressure to a measure that only pays attention to time practiced would drive out other motivational factors.

Network effects

The system is primarily a social technology: it’s most useful when exchanging information quickly. If you need to explain what the hell “level 6” is each time you invoke it, then what’s the point?

And then on the other hand, you could run into the same network effects as lojban: “your communication would be unambiguous and logical, but with the kind of people who learn lojban.” In this case, you can exchange information quickly, but among people willing to ignore the caveats that ought to encrust the system.

What’s the point?

The most obvious use of the system is in an informal contest of skill among friends, acquaintances and enemies, or what one might possibly call a “dick measuring contest”. Given the flaws above, and the subjectivity of use that Fermi estimation introduces, it’s not especially interesting to judge the system by this usage.

On the other hand, one could construct cases in which it is difficult for laymen to identify masters. In such a case, the system might actually give the best estimate of skill available. However, it’s hard to see how this might be generally useful: sports teams have playoffs, Go has Kyu/Dan rankings, chess has Elo, the violin has competitions and visible positions for professionals. Hell, for the violin, even listening to someone should be enough to at least pick someone out of the “dying cat” category[21]. If nothing else, you could do the same thing as everyone else and outsource your judgment to college[22].

Somehow, the scenario where the system becomes useful requires profound ignorance on part of the arbiter, while also caring a lot about picking winners and losers. In this specific case, such a metric as the system could work. However, if mastery actually matters, profound ignorance in the thing you are trying to measure is an “oh shit” state of affairs, not something you want to keep long-term.

So, to close: I lightly endorse the (sparse) thinking that lead up to it, but with all the downsides I don’t recommend actually trying to use/distribute this system.

[1] ↑ For those confused, the X is deliberate: I don’t have a hard date to nail down exactly when I was doing this thinking, just that it was within the last 8 years.

[2] ↑ For example, if you leave high school without being able to breeze through that concerto, and you want to play at a professional level, then something has gone terribly wrong.

[3] ↑ The alarm bells should be going off right about now: everything is correlated with everything else (in fiction form), so just claiming any level of correlation is Not Helpful.

[4] ↑ And don’t forget to knock off another 8 hours for work, and another couple hours for making sure you do things to stay alive.

[5] ↑ Fermi estimates are non-ideal because it’s usually thought that they will only get you within an order of magnitude of the actual number. Unfortunately, that might mean getting us just within the ±1 level range.

[6] ↑ If you want more on time tracking, and enjoy podcasts, you can check out episode 44 of Cortex.

[7] ↑ The low end estimate seems really low: I feel like I must have spent half that time just physically punching programs into a TI-83.

[8] ↑ It’s true that I do some coding on the side, but it’s a drop in the bucket compared to my day job.

[9] ↑ If it bothers you I didn’t strictly hit the minimum, imagine I added in the couple extra months of full time work I dropped when I rounded down the time range from “6 years and change” to “6 years”.

[10] ↑ It was clutch discovering that I could go practice when a “spirit” assembly was happening.

[11] ↑ It’s true that this figure isn’t super-precise: ideally we would only count really deliberate practice, and I’m sure I must have spent some time staring off into space wondering what’s for lunch while I was on the clock. I’m sure, because I just did it today. However, we don’t have that, and time tracking means I’m reasonably on task.

[12] ↑ NEWS FLASH: textbook with the word “Theory” in the title contains theoretical content! More at 11!

[13] ↑ By the time this is published, it’s likely that I’ll be almost done with Statistical Rethinking, and solidly in level 5.

[14] ↑ Plus, I had arty friends that I couldn’t compete with. For example, there was a marvelous sketch of a cat-me that I am sad to say probably no longer exists.

[15] ↑ I recorded these sketch times, but it’s currently tied up in the sketchy books, and not easily accessible otherwise.

[16] ↑ This isn’t universally true: if you’re special ops, then part of your job description is being able to function when not fully awake.

[17] ↑ Only taking into account practice time could be seen as a systematic under weighting of the talented.

[18] ↑ I will never turn down a chance to bash on the kazoo, because then I get to link to people doing insane things with kazoos, like making actual passable music.

[19] ↑ This is the tiny hill I will have part of my soul die on: memes are the Dawkins concept, not whatever the EU just banned.

[20] ↑ Otherwise, the system shares more than a few features with IQ: it’s a single ordinal number which nosily measures (all else equal, does it matter if someone has an IQ of 115 vs 112? : spent 400 or 500 hours practicing bonsai trimming?) some attribute (intelligence : mastery), and is heavily modulated by all sorts of things when you consider what you actually want done (conscientiousness : deliberateness of practice).

[21] ↑ I hesitate to make any stronger of a claim, since I remember Joshua Bell playing in DC public transit and not getting any looks.

[22] ↑ Elephant in the Brain (which I’ve read) borrows from The Case Against Education (which I haven’t read) to make the claim that higher education is primarily useful for credentialing. Or: education is not about learning.