This led to a FB discussion of selection bias. We can discuss the merits of these five and ponder why the Simpsons did so poorly (perhaps we need a consistent plot progression?), or why the Wire is under-valued yet again. That the 2nd and 3rd Matrix movies sucked so much that they sucked all the air out of the studying the Matrix enterprise? But what is most obvious is that this "study" is that it ignored the big, enduring elements of pop culture that we have been obsessing about for years/generations: Star Wars, Star Trek (which is tossed off as an side), Lord of the Rings, and, more recently, Harry Potter. Using just the Berkeley source, Star Wars appears to be ahead of Buffy. Using scholar.google:
- Buffy produces 6k hits;
- Star Trek: 30k;
- Star Wars: 51k (affected by Reagan's naming of the Strategic Defense Initiative);
- Lord of the Rings: 22k;
- Harry Potter: 32k.
|Papers||Cites||Cites per paper||Cites per year||H index*||Most cited piece|
|Lord of the Rings||100||8339||83||83||33||1961|
|The Matrix Trilogy||100||973||10||22||14||100|
Star Wars has the most citations but probably the most error given the aforementioned SDI bias. What this does show is that the classics (old) double more or less Buffy while the new (Harry) has 50% more citations. In terms of which papers have the most impact on average, again Star Wars prevails but Lord of the Rings and Harry Potter lead the rest (and probably have fewer accidental cites along the way). Harry Potter has the highest rate of cites per year since all of JK's stuff came out since 1997. This produces the highest H-index. Buffy performs quite well, certainly outclassing the Matrix because, well, quality does matter after all. The Simpsons do pretty well. Aliens? Scary but not worthy of citations. Star Trek is a steady performer, among the leaders across the border, and with fewer false positives than Star Wars. The Wire, well, hard to measure since even restricting P&P searches to social sciences gives way too many non-The Wire hits (same goes for Breaking Bad and Mad Men). I guess folks will have to settle for The Wire remaining the best TV program in most folks' minds and perhaps the best application of social science in a TV show.
Aside from the lesson that Slate does apparently poor pop social science, what can we learn from this exercise (other than my priorities are lousy--I should be doing something other than this--including my summer project of finally watching season one of Buffy)? That we live in a golden age of pop culture and its over-analysis? That Harry beats Buffy? That Star Wars and Star Trek fans have something else over which to fight? That the Matrix really did suck? What conclusions do the readers have?
Of course, one could argue that I have some selection bias myself--that this was hardly a systematic study of pop culture. There might be other books/tv programs/movies that get more analysis than these. But I did do due diligence, web 2.0 style by crowdsourcing first. Certainly, one can do this better if one is writing a dissertation. And surely someone is.
* from P&P: H-index aims to measure the cumulative impact of a researcher's output by looking at the amount of citation his/her work has received. Publish or Perish calculates and displays the h index proper, its associated proportionality constant a (from Nc,tot = ah2), and the rate parameter m (from h ~ mn, where n is the number of years since the first publication).