Critiquing Wages: a Comprehensive Index

A few months back, when this blog first launched, I wrote a piece that was meant to be less a discussion and more a diatribe starring the work done by the Wages of Wins blog. It was the opening piece of our Juwan a Blog? series, and it currently stands as by far the most negative portrayal of any blog I've reviewed. Aaron and I both have strong opinions about what Wages does, and both its strengths and weaknesses. One strength, which we really could've done a better job of highlighting, is the sheer volume of intelligent people the WoW network has accumulated -- while we can disagree with their orthodoxy and strict adherence to their way of thought (which, obviously, we do), you can't knock the hustle, nor can you knock their intelligence. There are a lot of smart, smart people at Wages, and in my takedown of their methods, I didn't necessarily articulate that. So, please consider this articulated.

Here at the Gothic, what we hope to do more than anything is start conversations. When Aaron wrote his piece this weekend examining Kobe via Stavrogin, he didn't intend it to be the be-all or end-all of his writing on Kobe, or the discussion of his comparison -- he intended it to start the conversation. How valid is the comparison? How well does it fit Kobe, and can one stretch it further? Those are the questions we like to ask. When we created the STEVE NASH model, we didn't do so intending it to be a be-all and end-all of our statistical meandering -- we merely want to add another model to the discussion atop the various standard prediction models, and see if we can't get a few more ideas on the table. Like my pantheon, it's only the stepping stone to -- hopefully -- a some-day valuable index of the absolute best sportswriting the NBA has produced. More than anything, that's what we like doing here.

I say this all because Wages of Wins recently addressed diminishing returns on defensive rebounds in an update to their main metric, Wins Produced. Such a tweak might sound fairly standard, but they've previously exhibited stubbornness and an almost impossibly high standard for making even minor changes. By their standards, it's a huge deal. They also published a link to this very post, summarizing more well-written (and my own) critiques with links to the pieces. Again, that might not seem like a big deal that they posted it, but they didn't used to post comments like that. And the fact that they seem at least marginally earnest about starting a dialogue is fantastic. It would be intellectually dishonest to ignore progress to suit my existing narrative. So good on them. I thought it would be more fair to them as a self-contained blog if I could stop cluttering their comment pages and repost this as an well-linked, oft-updated summary of their primary critiques here on Gothic Ginobili -- ripe for their own responses, when they get a chance, and ourselves isolating things we feel should be addressed. This is that ostensible entry, if you haven't gathered already. Let's get to it.

• • •

Before I get into the critiques, let me note that apparent critiques of "style" are inextricable from those of "substance". This isn't because everyone that disagrees is trying to invent a hole in substance by calling Dr. Berri curmudgeonly. It's because just as Berri values the peer review process in and of itself, much of the Internet (especially among the stats/blogger niche) finds the closed-access, privileged nature of this process to be anathema to the open spirit of inquiry of the Internet. They feel that Berri's measured, stubbornly-academic approach has negative effects on the substance, context, fluidity, and ultimate ceiling of his research that has allowed others to leapfrog WP48. That said, while this is meant to be a compendium of critiques against WP, it's hardly for attention (we're kind of burying this between two pieces more important to us, to be honest) and it's not meant to be a purely negative. It's just meant to be a conversation-starter, and a positive force for the understanding of basketball through statistics and discussion. If we can raise the tenor of debate, we've done our job and that's the bottom line.

• In the spirit of this disclaimer, shall we start with my own criticism? (In their collective defenses, this was written before I knew about the team dreb% adjustment, which does begin to challenge my narrative to a real extent.) I argue that (and this will be a theme in all of these) the Wins Produced metric is perfectly reasonable but is not so reasonable that we have to throw out all our other reasonable concepts and metrics, and given Dr. Berri's stubborn and ideological approach to the metric, it's unlikely that WP48 will ever get to the point that justifies the arrogant, often lazy attitude of this blog towards its metric. It's not absolutely substantive, I admit, but I don't think it's fallacious, either. I read the books and I read this blog far too often, and I feel I've diagnosed the key "problem" that many individuals smarter than I have with their stat and their approach.

• Here are a couple of links that - in the first one - fantastically detail the state of basketball statistics from a well-reasoned, overarching point of view. In__ the second link__, the blogger EvanZ (a friend of the blog and of STEVE NASH; hell, dude even helped us find the last two links) posits and computes a strong, substantive +/- analogue to WP that (as far as I understand it) uses play-by-play data to award what is captured by the box score credit in a similar way to WP. Evan does change how rebounds, assists, shot attempts, and defense are weighted and it is a completely different metric, but his ezPM starts with (and is most apt to be apprecated by) people that get WP and agree with it to large extent but find it has troublesome components.

• I don't know anything about Phil Birnbaum, but this response to the rebounding section of WoW's FAQ is very well-argued. Accounting for diminishing returns on rebounds as Dr. Berri et al. did recently is a step in the right direction, but as far as I can tell, these critiques of WP and rebounding are still absolutely valid as conversation-starters, at least, and best of all they actually start with the words of Dr. Berri, decreasing the amount of abstraction into which fallacies and sentiment can enter the conversation.

• Here are two disgruntled amazon.com reviewers (at least one of which runs a stats blog that I know of) sketch out their frustrations with the WP model.

• Nathan Walker argues (and, to be fair, partially rants) that there is existing, solid empirical evidence against the empirical value of offensive rebounds and the assignment of team statistics to player statistics is extremely shortsighted, looking instead for more sophisticated +/- models. This captures perhaps the most visceral statistical critique of WP: "We've already thought about this along the same lines. Who are you to call us more irrational?"

• D. Blum "eclectic reader" puts together a constellation of objections whose home planet is the loaded terms "Wins Produced" and "productivity" as used in the WoW books and blog. A lot of us can accept (because of its reliability) that WP measures something, and that something is often pretty close to basketball productivity. But using the term (with an associated absolutist approach) forces the metric into an uncomfortable Platonic standard which it doesn't seem to live up to for critics. Probably the strongest critique here is #3, which persuasively argues that WP's fixation on retrodicted correlation with Win% doesn't (by itself) indicate a robust statistic by creating an absurd - but illuminating - parody.

• The Problem with Wins Produced by dhackett1565 of Raptors HQ is a wonderful (and - as far as I can tell - mathematically sound) examination of the rebounding question by showing that WP is a degenerate (in the good, mathematical way) special case of a slightly more complicated formula. It illustrates that WP does make choices in its allocation of individual statistics not necessarily based on strict logic and correlation, and even purports to show a contradiction in how WP is awarded over the course of a single possession. (Personally, I don't see the contradiction yet. I'll update if I do.) I think this is an important step which shows both the elegance and the problematic simplifications of WoW. For me as a math major? Well, this one gave me a lot to think about. The rules and structure of basketball (esp. the demand for transition defense and the 3-second rule) make problematic the idea (seen in WP) of offensive rebounding as a regained possession without further context. Also, this link is a bit more respectful than the Amazon links, and really does try to get at both sides of the argument. Hat tip to EvanZ (@thecity).

• An Incredibly Long And Educated (though probably partially outdated) Look from APBRMetrics is, well, uh... an incredibly long and educated (though probably partially outdated) look from APBRMetrics. Unfortunately it's from 2004-5, but it features a lot of good content if you're willing to pore through it. Hat tip to EvanZ (@thecity). Again.

• Dave Berri's Dismal Science - An alternative conclusion to the WoW network's typical conclusion of the bounded rationality of NBA decision-makers, SilverBird5000 of Freedarko.com deconstructs the economics of the scorer's market in a world run by Berri's metric, reversing the causal chain between pay and quality to an extent, to argue (somewhat convincingly) that scorers are "overpaid" partially for the risk-taking activity of scoring as opposed to crashing the boards. I don't know that I agree totally, but it is at least reasonable and a solid interpretation.

All of us care (way more than we should) about getting it right, WoW and above critics included. That might sound trivial, but I think it's a good first step. Thanks for reading.