Welcome to another week of Telemetrics updates. In this blog we’ll take a quick look back at the last set of ‘predictions’, explore the Twitter Excitement Index and finally take a look at this Monday’s Australian TV premieres, The Block & My Kitchen Rules, both alone and in the context of other recent Australian Reality TV ‘events’.
A Quick Look Back
So, as we left things, we put out a few predictions for the week beginning January 20, as follows:
So, some hits and misses here.. The most concerning of these are Pretty Little Liars & Ravenswood, so let’s have a closer look at those, starting with Ravenswood. Essentially, I think this was an over-prediction influenced by the second half premiere, which rated at 181,200 tweets. We have now added an adjustment for these type of shows, that will highlight second half premieres in our predictor, with the option to manually exclude them where they rate significantly different. More concerning was “Pretty Little Liars”, which wasn’t a second half premiere but a regular show. Here, I think we had the reverse problem. Because there weren’t enough shows in the past two months (only 1 after the second half premiere), our prediction algorithm defaulted to include episodes from the previous series, where 300-400k tweets was the norm. Combining this with the 488,000 tweets from last week, this seemed a reasonable estimate, but the show actually nearly doubled that performance — it will be interesting to see how Pretty Little Liars fairs over the coming weeks.
The other errors of over 10% were both reality shows, and here I’m tempted to put the predictions on hold until we’ve established a measure of season context. Just as Big Brother follows a weekly cycle of Daily Shows, Nomination Shows and Eviction Shows, shows like American Idol, The Voice etc follow a somewhat standard season format between auditions, performances, eliminations and so forth. Modelling this is on my to-do list, but it’s after the establishment of the Twitter Excitement Index and indeed the modelling of regular season shows with their premieres, finales and other formats. Time and other priorities didn’t allow us to get predictions out for this week, and I won’t do so retrospectively, but we’ll get some out either late this week or early next.
Twitter Excitement Index
One thing we did finish last week was the establishment of our Twitter Excitement Index, which is a measure of the volatility of conversation on Twitter based on the principles of Brian Burke’s Excitement Index at Advanced NFL Stats. Over a few weeks Katie Prowd and I went through a few variations on this approach, looking at different measures to see which best captured the patterns in Twitter conversation, and I also owe thanks to Patrik Wikstrom for his help in tweaking our statistical approach. Essentially though, the theory here is that if you have two shows, they may both average 100 tweets per minute, but have very different engagement levels, as seen in this dummy data:
In the first graph, the activity is ‘spiky’, that is, every other minute people are prompted to tweet, while in the second there is an underlying level of 100 tweets, with minimal variation around that average, suggesting a constant stream of conversation but no particular moments which provoke users to tweet. These should be at opposite ends of the spectrum, and with our Twitter Excitement Index, the top graph would see a TEI of 9.9, while the second gets a TEI of 0.5. The scale here has been calculated to vary between 0 and 10 for presentation purposes, although in practice from our test data it would seem very unusual for a show to achieve a TEI of over 5.
The Australian Premieres
We are currently working on adjusting our metrics to work with Australian television, and this weeks premieres of The Block and My Kitchen Rules gave a good first test for this approach. But first, let’s cover the basics, and look at how these shows performed on Twitter:
As you can see here, My Kitchen Rules clearly won the night, both in terms of total tweets and audience peaks, from The Block in second place. The third line here represents The Biggest Loser, which wasn’t premiering but did air an episode in competition with the other two reality shows. The twitter audience for The Biggest Loser has fallen off a bit since its premiere, but this performance must be considered low by any standards. A similar picture was seen in the TV ratings, with My Kitchen Rules reaching 2.4m viewers, The Block 1.1m, and The Biggest Loser just 560k. However, it’s interesting to compare these to other recent reality ‘events’:
What is evident here is that Big Brother still owns the crown for Australian reality television, at least on Twitter, with both the Premiere and Finale easily outperforming the launch episode of My Kitchen Rules. For Channel 9, the performance of The Block suggests that the reality show success they achieved with Big Brother is not easily transferable to other shows, and for Channel 10 both the performance of Masterchef and Biggest Loser must be a cause of some concern… A similar pattern can be seen in the unique audience for Monday’s shows (n.b. the percentages add up to over 100% because of viewers who tweeted about multiple shows):
Finally, the launch of these shows also gave us an opportunity to test some of our new metrics, and after a slight hiccup caused by time zone confusion (the TEI saw the drop-off to QLD discussion as a good thing for the show, viewing NSW levels as large spikes), we ended up giving a slight win to MKR, although both ranked before the season launch of The Biggest Loser. The TEI does not take into account tweet volume, only measuring the volatility of conversation among the audience that *did* watch the show, and so should generally be read in combination with the audience size to properly understand the conversation around a show.
Mumbrella also covered some of this, and you can view their article here.
And so wraps another week in Telemetrics.. until next time!