Thursday, September 15, 2016

What Renewology Will Look Like This Fall

Check out yesterday's introduction to Renewology, this site's new attempt to predict renewal and cancellation outcomes. This post will continue the explanation with a look at how you can expect to see these numbers shared and used on a daily and weekly basis.

Daily Ratings Table

For years, I have posted a full table with the day's ratings each day after finals come out. This fall, those tables will begin appearing based on prelims. They won't be up with the initial submission of the post, because I need estimates of viewing levels before I can generate True ratings, and True ratings are vital to the process. But I would say the average will be somewhere in the vicinity of 11:30 to 12:30 ET (or 30-90 minutes after the prelims first come out). Some days may be later or after finals, especially once the dust settles on fall ratings. But in the early part of the season, I'll try to be pretty quick about it.

These tables will still contain the A18-49 rating, and week-to-week and year-to-year for the show and timeslot. They'll also still contain True and the week-to-week change for that. But we're dropping a couple of the other things to add Renewology percentage, and week-to-week change for that.

Fresh Off the Boat1.1
The Real O'Neals0.9
Agents of SHIELD0.9
Beyond the Tank0.6

Note that the R% comes from rolling averages of recent ratings results, not just what happened this week. Sometimes, the new point that goes into the formula might be less important than the one that got kicked out. You can see that here with Fresh Off the Boat, which gained in R% despite having a down week-to-week point in True.

I also anticipate writing a brief "Renewology update" looking at the big estimates of the day. I'm not sure it will be sustainable to do both my usual snap reaction write-up, plus a Renewology write-up every single day, but I'll definitely do it that way at the start of the fall and see how it goes.

Renewology in the SpotVault

You can see the most recent Renewology percentages on the daily tables, but if you want a little more detail than that, head for that show's SpotVault page. The Renewology table (replacing the Timeslot Stats/Projections table) will list every episode's True score, as well as how much viewing, competition, lead-in and (new this year!) lead-out each affect the rating. And it also has some new Renewology stats:

RenewologyThis Week's True

"Proj" is the mean prediction for where the show's True ratings will end up settling in the late part of its order, while "Target" is the network-based average that the formula expects the show will need to be exactly a 50% renewal. It's our attempt to quantify the "bubble."

"Collapse" and "Resilient" are both derived from the formula's thousands of simulations of future ratings, as a sort of reasonable "best case" and "worst case" scenarios for where the True scores will go from this point forward. More concretely, they are the 5th and 95th percentiles out of all the simulations. So you can look at them as a sort of 90% confidence interval. The "Proj" at the end of the season should only finish outside of the Collapse to Resilient range about 10% of the time. Whether they actually will is another story...

In this case, I kinda cherry-picked a show (Criminal Minds: Beyond Borders) where the initial True projection was almost exactly right. I did this so you can see the formula becoming a bit more confident at the end of the order, even with the proj/target staying pretty steady. That's because the ratings uncertainty is gone. We're more certain that it's settled a little above the "bubble," and the "Collapse" scenarios that show it dropping below are gone. More on that in the summary of the process.

There's also a chart, tracking each show's evolution in Renewology % over the course of the season.

The chart uses the same dark red to dark green color coding as the tables, so you can visualize a show's movement through the various zones. If I cherry-picked a strong example of projections for the table, I cherry-picked an awful one (season one of Secrets and Lies) for this chart, just to give you a view of a show that navigated a bunch of different zones. S&L was a case where the show premiered barely above toss-up levels. The formula said that's a bad thing, because normally it would drop well below the bubble, but this was a rare case where the pundits would've been correct to treat it like a normal rating! It was way more resilient than the formula expected and then grew big at the end, allowing it to make an incredible post-premiere gain in Renewology %.

Daily Renewology

What I used to call the "Weekly True Power Rankings" will now be called "Daily Renewology," with one post for each network so you can compare the most recent R% for shows on the same network. These posts are where you should go for the very freshest forecasts, because they will use the most recent projection of the "Target" ratings average for every show. (In the Vault, the most recent episode will use the network projections as of the time that episode aired.) This may be important if a network has an awesome spring or a complete meltdown, and the targets shift a lot more than expected.

Renewology Roundup

Most weeks, I plan on doing a column of sorts recapping the week's Renewology results, along the lines of the "True Power Rankings" writeups that I used to do several times per year.

It will be a process to figure out how these will work, but there are a few things I expect will be staples in this column:

1. Swings of the Week. One thing that I anticipate being a lot of fun is looking at which data points each week had the biggest impact on their shows' renewal forecast.

2. Buy/Sell on Renewology Percentages. This thing is a hard-numbers equation with no room for subjective adjustments after the fact. It's the best attempt I have come up with so far, but it will be far from perfect, and there will probably be some obvious examples of those imperfections. (Hopefully not too many!) So I'll see this as my opportunity to offer my own take on the percentages. Perhaps that just means it's a weekly opportunity to say, "Madam Secretary is not actually 25% to be renewed......"

3. The Bubble.  Not sure whether every show will be on the one post every week, but we may have some consistent way of just focusing on the shows with really uncertain outcomes.

First Two Weeks

I thought the advent of Renewology might make this long-running game obsolete, but I've actually decided it's kind of a perfect way of putting it to the test. So this game will continue just like before, but the Renewology model will be a contestant just like you. Check out the standings from last season.

Renewology's submissions will try to maximize its point haul in every game, knowing that you have to be at least two-thirds confident to make Confident a better choice than Not Confident. It will strictly follow the Renewology percentages after week two as follows:

0-33%: Cancel (Confident);
34-50%: Cancel (Not Confident);
51-66%: Renew (Not Confident);
67-100%: Renew (Confident).

Best Case/Worst Case

The preseason predictions in Best Case/Worst Case will be scaled back this year, due to my utter absorption in Renewology over the last couple months, and desire to do more Renewology-related writing as the season is ongoing. If Renewology continues beyond this season, it's possible that Best Case/Worst Case could fold into Renewology with the addition of a preseason prediction model. But that's not ready to go this year. So this year, Best Case/Worst Case will continue, but for a reduced volume of series. Check that out starting on Monday.

Daily Year-to-Date Rankings

"True Power Rankings" are technically no more, and that can also be said of "Plus Power Rankings." The five Weekly Plus Power Rankings are now being combined into one mega-post called the Daily Year-to-Date Rankings. While Renewology will dwell heavily on recent results, these rankings are your way of keeping up with full season averages. These will include season-to-date averages in total viewers, A18-49, A18-49+, and True, plus a show's age skew and gender skew. Like the Plus Power Rankings, we will have a projection of the show's full season raw and Plus average. These projections use the same kind of equation that the Renewology projection step does, but it will use raw numbers instead of True, which may lead to some particularly egregious mistakes.  It's all in one mega-post because there will be the same filters you can see in the 2015-16 Yearly Index. This will allow you to boil this long list of shows down to one network, one category of shows, or new vs. returning.

No comments:

Post a Comment

Newer Post Older Post Home
© 2009-2016. All Rights Reserved.