Aaron Schatz is the founder and President of Football Outsiders, Inc., a writer for ESPN.com and ESPN the Magazine, and the Editor of Football Outsiders Almanac, which will be released in July (pending ownership and players getting along.) Aaron and I exchanged e-mails on a variety of topics, including Mark Sanchez, his business model, the quarterback projections, and how the lockout may affect his workload.

JL: Our site’s editor-in-chief and founder is a huge Jets fan, and he is very protective of Mark Sanchez. So I must ask on his behalf before we talk about other things, why does Football Outsiders hate Mark Sanchez with the intensity of a thousand solar powered calculators?

Aaron Schatz: We don’t hate Mark Sanchez. We just hate Mark Sanchez hype. It’s tied in with the whole concept of “quarterback wins,” the idea that because the quarterback is the most important position on the team, wins must equal a good quarterback and losses must equal a bad quarterback. Mark Sanchez is not the reason why the Jets have gone to two straight AFC Championship games, and all the attention paid to Mark Sanchez takes away accolades that belong to other players like David Harris, Shaun Ellis, and Nick Mangold. (Darrelle Revis has somehow managed to escape Sanchez’s shadow and get proper appreciation.)

We want to make it clear that Mark Sanchez is not a great quarterback right now – but that doesn’t mean that we don’t think he can become a great quarterback. People just need to stop hyping him as if he is fully formed. He’s still raw because he only started one year of college ball. Philip Rivers had more starts at N.C. State than Sanchez has had at the college and NFL levels combined. So Sanchez sucked in his rookie season, but he was actually an average quarterback in 2010 (DVOA: 1.6%). If he can continue this growth, then he’ll be a good quarterback next year, and by 2012, he might actually become what Jets fans think he already is.

JL: Part of what you do at Football Outsiders is try to strip away the effects of luck when you evaluate how good teams are. So, how much of where you are today is based on luck, how much of where you are today was based on establishing a vision when you started the website and wrote that first piece about ”establishing the run” back in 2003, and how much has come from making adjustments over time?

Aaron Schatz: A lot of it has been luck, some of it has been skill, but really very little of it was vision. There really was no vision when I launched FO in 2003 — no business vision, anyway. I had been goofing around with some football stats, and I needed a place to get them out to the public. I figured there was an audience for advanced stats in football because I knew it was something *I* was looking for. FO exists in part because there were a couple of guys who came out with a Pro Football Prospectus book in 2001, and I thought it was really disappointing, so I tried to do something myself instead. But I never thought this would become a big deal. I certainly never thought it would become my full-time job. The fact that I was doing FO full-time within a year of launching the site was a bit of a shocker. A shocker helped along by losing my job at Lycos in February of 2004, of course.
Anyway, luck played a large role. I just happened to start the website right after Michael Lewis came out with Moneyball, so suddenly everyone was going onto the web looking for “the moneyball of other sports,” and there we were. I took that hook and became an unrepentant media whore in our first couple seasons, getting our name out everywhere, because I knew the interest was there if people just knew where to look. Once we established ourselves as the first big website to do football analysis, good writers came out of the woodwork, people who had been waiting for years for a website like FO to emerge so that they would have somewhere to write smart football analysis — Michael David Smith, Mike Tanier, and all the other guys who have written for FO over the last seven years. I’ve been privileged to have some really good analysts write for me.

JL: Your site began with all material being freely available (and I’m assuming any income you derived was based on advertising), and over time you have gone to more of a mixed model where you still put many articles such as your weekly DVOA ratings on the website, but also have some content behind a pay wall with ESPN. How has the partnership with ESPN impacted your website, and do you feel the tradeoff of having some material behind a pay wall has been a net positive?

Aaron Schatz: Actually, from almost the beginning, the business model of FO was to make a large portion of income from writing for larger websites. There was very little money coming in from advertising, and at that point we weren’t doing a book or selling fantasy projections in the preseason. I was able to go off unemployment in September 2004 because I was writing once a week for ESPN and twice a week for the New York Sun.
People who write need to make a living. That’s why subscription Internet sites exist. I don’t think it’s a terrible thing. I also think that our deal with ESPN has been great for both us and our readers. If all the content we do for ESPN was instead put behind a subscription wall on FO, we honestly wouldn’t make as much money. At the same time, people don’t have to pay $50 to get FO content … instead, they can pay less money to ESPN and get not just Football Outsiders but also K.C. Joyner, Scouts Inc. Mel Kiper, plus John Hollinger and folks in other sports.

JL: I have been a big fan of the site and breaking down the play by play and finding out which teams are weak against tight ends and all that. But I am skeptical of the QB projection stuff from college, and I was skeptical when the first version came out. I feel like it is over-fitting data with a small sample size, and I’m worried that when we call something a Projection System, and put very specific projection numbers on it, we give the appearance of more certainty than there is. Your response?

These are certainly reasonable complaints. We’re sort of stuck with the small sample size, so we give up on finding something perfect or near-perfect. Instead, we look to see if there are indicators and what we can learn from them. I think that the Lewin Career Forecast makes some interesting points. Guys with bad college accuracy rarely succeed in the NFL. Guys who come out with 3+ years as a starter generally do better than guys who come out with 1-2 years as a starter. Taking lots of sacks in college is bad. It’s not a good sign when a quarterback’s growth stagnates in his senior year. Those are the basics, and the numerical projections come from those. And the specific numerical projections are meant to be seen as numbers on a scale from good to bad, not as exact predictions.

There’s a general problem with any attempt to create a system to project the success of college quarterbacks: from 1990 through at least 2005, college games started was BY FAR the most important indicator forecasting the success of highly-drafted quarterbacks. No matter what statistical analysis you do, no matter how much you want to avoid overfitting, you are going to be stuck with games started as your most important factor. In the data set I used for LCF v2.0 (Rounds 1-3, 1998-2008), the p-value for games started is .0000014. And yet, suddenly in the last couple years, guys who have had long, successful college careers are getting identified as top prospects by scouts — and then flopping. So no matter what, you’re stuck with the most important factor being the factor that doesn’t look good in recent drafts. It’s unavoidable.

One other thing I’ll note: I’ve often said that Football Outsiders “leads the league in couching our opinions.” We always try to point out that our stats are not meant to describe the world in black and white. We try to point out when we have small sample sizes, and we try to point out when we’re not sure if our own results really make sense or not. I went out of my way in the LCF v2.0 article to make sure it was clear that this forecast system was meant to be a cross-check on scouting, not a replacement for scouting. And I’ll be honest — it’s possible that the games started thing represents a change in how scouts evaluate college quarterbacks which renders moot the whole concept of predicting future success of these players based on past success of other players. I don’t think that’s the way it truly is, but it is certainly possible.

JL: Your projections this year are almost completely opposite of where we are hearing guys projected. You have Andy Dalton with the highest projection, and Cam Newton at the bottom presumably because he only played 1 year of major college ball. Which projections do you personally agree with the most and the least?

Can I say the ones in the middle? I mostly agree that Jake Locker, Christian Ponder, and Ryan Mallett will be average NFL quarterbacks –below-average starters but above-average backups.

JL: Your site is entirely focused on football. If the lockout continues into the regular season, how does Football Outsiders plan to deal with the delay in terms of content?

Aaron Schatz: We’ll write more college football stuff on FO and for ESPN Insider. I am not sure how we will schedule the writing of the Football Outsiders Almanac if free agency gets postponed any later than mid-May. At that point we’ll have to write sort-of skeleton chapters with the knowledge that a lot of teams won’t be fully fleshed out until player movement finally takes place. There’s really no way to write about Cincinnati or Arizona until the lockout ends.

JL: What new regular weekly features will we see at Football Outsiders once football begins in 2011?

Aaron Schatz: I’m not sure. We’re going to bring on a couple of new people now that Bill Barnwell is leaving us, and we’ll see if they have any particularly good new column ideas.

JL: A few years ago, Football Outsiders went away from print publishing under the name “Pro Football Prospectus”, and has gone to distributing a .pdf available version as Football Outsiders Almanac. How has that decision worked, and how do you see the future of Football Outsiders Almanac?

Aaron Schatz: Oh, it’s worked out great, and the funny thing is that the self-distribution model was our second choice. It was Plume’s decision, not ours. It was important for us to do the Pro Football Prospectus as a regular book the first couple years — we needed the experience putting books together, and we needed the promotion from Baseball Prospectus and from the publishers, Workman and then Plume. It’s important to let folks know that at no point did we have any kind of “falling out” with Prospectus. Their publisher at the time, Plume, decided that they didn’t want to do any of the books from sports other than baseball, and that’s when we went to the self-distribution model. And if we were going to do the book ourselves instead of through a contract with Prospectus, it didn’t make sense to pay for the name Prospectus. But we’re still friends with those guys.

Anyway, with the self-distribution model we’re selling one-third as many books but we’re making three times the money. There’s literally no overhead on the PDF versions of the Almanac, and the profit from the physical books sold through Createspace is a lot higher than it would be selling through a conventional publisher. Most importantly, since we don’t need to go through the usual edit-and-print process, we have a lot more lead time. The PFPs needed to be written by mid-May and fully edited by June 1 in order to get out by mid-July. With FOA, we’re working right up until we release the book in July, because Vince Verhei can make changes on the PDF up until the day we start selling it on our website. That’s important for my sanity, and particularly for my wife’s sanity, because the book crunch is a lot easier now. And there would simply be no FOA 2011 if it wasn’t for self-publishing, because a book company would not have been able to sit around waiting to see when the lockout ended. We may get the thing out late this year, and it may have more errors than usual because we won’t catch everyone changing teams, but even if the lockout doesn’t end until August we should be able to put out a book before the season starts. It just won’t be *two months* before the season starts, the way it usually is.

JL: Favorite player growing up? My longshot money is on Stanley Morgan.

Aaron Schatz: Actually, my favorite player growing up was Eric Dickerson. I lived in Mission Viejo, California until I was 13 years old — ironically, Mark Sanchez’s hometown! — so when I was a little kid I rooted for the Rams. At one point, I was very proud of my Eric Dickerson rookie card, which is probably still in a box in my mom’s basement. The Rams traded Dickerson a couple weeks after my family moved to Massachusetts, and for a while I sort of half-heartedly rooted for both the Patriots and the Rams. Honestly, I wasn’t even a very big football fan for most of my youth, it was baseball first until Parcells and Bledsoe showed up in 1993 and made the NFL relevant in Boston for the first time in years. That’s when I started really setting aside time on Sundays to watch games, usually while I was doing my laundry in college. I didn’t become a die-hard NFL fan until I spent a year on the radio in Daytona Beach. I used to go hang out with other New England expatriates on Sundays at the Ocean Deck, that’s how I stayed connected to home, and of course the Pats made the Super Bowl run that year. When I came back to Boston a year later, I was a hardcore fan.

JL: Jason McIntyre (TBL) recently became a father, so feel free to impart any words of wisdom on being a work from home dad operating a successful website while dealing with the randomness of parenthood.

Aaron Schatz: Go out to a Starbucks or something whenever your wife is home at the same time, in order to try to fully separate between work time and daddy time. My wife is a teacher, and during school vacations or summertime it can be tough for her to resist knocking on that door to get me to help out with my daughter. But I’ve got stuff to do, you know, like putting out a book every summer.

[Aaron Schatz photo courtesy of Aaron Schatz; other photos courtesy Getty images]