Measuring Interaction on the Bottom Line (Views)
As we start to see more interaction on our more recent episodes of The Hayley Project, I noticed that this interactivity really does impact the bottom line of TOTAL VIEWS for online video players with auto-play such as YouTube.
This post is really about two things. What are the right interaction metrics to look at on a standalone basis, and how can we estimate what the incremental lift might be on total views due to this interaction. My methodologies are not going to be perfect, but it’s fair to say that they’re directional correct.
1) Interaction Metrics
As a standalone set of metrics, you could analyze the following to assess how engaged your audience is with your video. When it comes to any sort of performance measurement, you really should be looking at both a volume and an efficiency/ratio metric. This is the only way you can get a clearer picture of what is going on.
- Number of total comments per episode – this volume metric helps indicate the pure volume of discussion going on for any given episode
- Unique number of commenters – this helps paint a picture of whether your volume is coming from just one comment-happy poster or the larger community. It’s good to use this in conjunction with the total number of comments to assess the validity of volume
- Comments/Views – in order to normalize for the different view totals which might be a function of time, divide your total comments by number of views to come up with a ratio metric. It’s possible that this metric will tell you something different than the first metric. E.g., one episode might have the most total number of comments due to it’s longest release time, but another might be the “hottest” interactive video, as illustrated by a high comments to view ratio. To make this example real, THP Episode 1 is close to 4,000 views and has an average comment/view ratio of 1.7%. THP Episode 27 was only released a little over a week ago and has close to 1,500 views with an average comment/view ratio of 5.6%. To be fair, the longer your video is out, the more likely that this ratio metric will decrease over time as the “hotness” factor diminishes. But you get the point as to why this is important.
- Number of Ratings – This is a lesser engagement metric but it also illustrates the number of people who are motivated to rate your video
- Ratings/Views – Same concept as the Comments/View ratio. You need both a volume and efficiency metric to assess the situation
2) Interaction Impact on Total Views
A slightly more complicated “bottom line” measure is to analyze what the incremental lift impact of interactivity is on total view numbers. Every time someone leaves a comment or replays the video looking for a clue, you’ll receive a credit for a view, especially on auto-start players like YouTube. So all things being equal, more interactive elements should result in more views, right?
To test this theory, I looked at all of The Hayley Project episodes that were released once we came back from break on 1/20. TubeMogul makes it easy for me to see the total amount of views per day. We haven’t done much marketing that might impact or skew these numbers, but I recognize that my methodology isn’t perfect. If for nothing else, over time you should expect your total views to creep up, because you have more inventory. Regardless, the point is more illustrative than anything.
By comparing the average number of views per day for the time period 1/20-2/8 (representing releases of the first 6 episodes from break) against the time period of 2/9-2/21 (representing the last 4 episodes where we start to see an uptick in comments), we see the following results:
- +16% in average total comments for the most recent 4 videos
- Resulting in +23.6% more views on average for the most recent 4 videos
Does correlation mean causation? Not necessarily. But the point is, there is a way to link interactive metrics with total views to measure what benefit interactivity may have on your series.