Movie Reception and Follow-Up Success: A Correlational Analysis

There has been a great deal of conversation among the superhero fan community about the reasons behind Justice League's underperformance at the box office. It is currently on its way to $650+ Million worldwide, which will place it as the 12th biggest movie of the year -- nothing to scoff at, but everyone agrees this is a disappointment.

What we don't agree about is the root cause(s) of the disappointment. There are two main contentions that I want to address in this post.

Contention A: Justice League is underperforming because Warner Brothers executives inserted themselves into the creative process, enacting reshoots (and a new score) that changed the tone of the film and altered characters in ways that betrayed the loyal fans of prior DCEU movies. Thus the fanbase is disappointed with the film (i.e., not giving it as much business) and the rushed final production resulted in an inferior film, which explains the relatively low box office totals. People in this camp are quick to point out that prior DCEU movies, even those that were critically maligned, earned more money at the box office than Justice League.

Contention B: Justice League is underperforming because the mixed reception of Man of Steel and the generally negative reaction to Batman v Superman have finally caught up to WB. A substantial portion of the general audience stayed home from Justice League because they either watched BvS once and didn't care for it or they never saw BvS and just accepted the pervasive notion that it was an overblown, poor-quality movie. (If you read this blog or listen to my podcast, then you know I wholeheartedly disagree with this assessment. I think BvS is a brilliant and well-executed film, but I've heard countless times at work, in stores, on TV, and even in political discourse people stating as an accepted fact that BvS is a bad film.) What about the higher performance of past DCEU films? Well, these were benefiting from the good will of prior films, particularly The Dark Knight trilogy (Batman and Joker boosted BvS and Suicide Squad both) and also Man of Steel to an extent (MoS was better received across the board than BvS, although not as universally praised as TDK).

Of course, the above are simplifications and many complex factors are at play, but the descriptions will suffice for the purposes here. What I decided to do for this post was actually test these two contentions statistically. This is just a rough-and-dirty correlational analysis, but I think it is revealing. And one of the contentions certainly does come out ahead.

What I decided to do was collect 22 film pairs (22 originals and 22 follow-ups, though the two sets are not mutually exclusive) in the modern superhero-movie era and investigate which of the following factors was most strongly correlated with the follow-up movie's box office performance:

• Predictive Variables
• Original film total box office
• Original film Cinemascore
• Original film user rating
• Follow-up film Cinemascore
• Follow-up film user rating
• Outcome Variables
• Follow-up film opening weekend
• Follow-up film total box office

For consistent comparisons, all box office numbers are in the U.S. User ratings were taken from Metacritic. I didn't include critic scores because I'm tired of critics and I think they have too much power as it is, so I don't want to give them further attention. Full disclosure: I came into this analysis suspecting that Contention B has more merit than Contention A, primarily because Justice League's opening days were already substantially lower than expected, before any of its perceived flaws or its own word-of-mouth could take off. Moreover, millions of people seeing BvS and a year-and-a-half of nonstop online coverage and BvS-word-of-mouth seems like it would be more potent than a Justice League marketing campaign. But even though Contention B was my hypothesis going in, I chose the list of 22 film pairs a priori and did not add or remove any films to attempt to shift the results. I just ran the correlations and am reporting them now.

Note: I am using BvS, rather than Wonder Woman, as the lead-in for Justice League because the continuity of Superman, Batman, Zack Snyder, and the crew all indicate a close link between those films. The coverage of Wonder Woman over the summer also made it seem like its own entry in the superhero film landscape, and the announcement of a separate Wonder Woman sequel made it clear, and I think it was clear in the mind of the general audience, that fans of Wonder Woman should stay tuned for that direct sequel, while Justice League is a continuation of BvS.

Correlational Results

First of all, a follow-up's total box office is positively correlated with the original's box office. This makes sense, of course, and is why studios love producing sequels to successful films. The correlation coefficient in this case is 0.485, so that gives us somewhat of a benchmark for comparison. (The outlier visible at the top, by the way, is The Dark Knight, which performed much higher than expected given the modest box office of Batman Begins. Though, as a preview of results below, Batman Begins was highly regarded by audiences.)

Correlation: 0.485

Now, if Contention A is true, then a follow-up film's own Cinemascore and user ratings should be highly correlated with its box office performance. That would explain at least a substantial portion of Justice League's underperformance because it has only a B+ Cinemscore and a 6.9 user rating -- not bad but not especially good either.

However, looking across the 22 follow-up films in my sample, a follow-up's Cinemascore is weakly correlated with its final box office (0.263) and almost not related to its opening weekend at all (0.161). The user ratings tell a similar story, though they are slightly more correlated. Follow-up user ratings to final box office have a correlation of 0.415 and user ratings to opening weekend have a correlation of 0.373.

Correlation: 0.161

So overall we can say that a follow-up's opening weekend performance has very little to do with its own audience reception or word-of-mouth. This makes sense because, as I said before, it has just been released and so hasn't had any time to build up its word of mouth. Yet even the final box office total of a follow-up film is only moderately correlated to its Cinemascore and user rating.

On the other hand, if Contention B is true, then we would expect a follow-up's film to be more strongly correlated with the original's Cinemascore and user ratings. In other words, if people were left with a positive feeling about a movie, then they're going to be more likely to head to the theater to see the follow up. And indeed, that's exactly what the data show. For these 22 superhero film follow-ups, the original film's Cinemascore and user rating was more strongly correlated with their success. For original film Cinemascore, the correlation to follow-up opening weekend was 0.531 and to total box office was 0.524. For original film user ratings, the correlation to follow-up opening weekend is 0.528 and to total box office was 0.483. Each of these correlations is higher than its counterpart from the follow-up itself; that is, by all measures, the original is more predictive of follow-up performance than the follow-up is for itself.

Correlation: 0.528

Conclusion

As I wrote earlier, this is a quick-and-dirty analysis so obviously it doesn't prove anything, but the evidence here clearly supports Contention B over Contention A. Justice League seems to be suffering more from BvS's B Cinemascore (tied for the lowest in my entire set) and BvS's 7.0 user rating (bottom third) more than from its own merits or faults. Now, these correlations between the original film and the follow-up performance still leave a lot of variability to be explained by other factors, so this is certainly not the entire story, and one could still argue that if Justice League had been a better film itself, it could've overcome the drag of BvS. But the data indicates that that probably wouldn't have happened opening weekend anyway, because a follow-up film's own merits don't really seem to kick in until the second weekend and beyond.

But when told that BvS dampened Justice League's box office performance, some people might say, "That doesn't make sense, because BvS made quite a bit more money than Justice League is making." To this I would say notice that the correlation between original reception and follow-up success is actually stronger than the correlation between original box office and follow-up success. In other words, even if tons of people saw the original movie, if they weren't that thrilled with it the follow-up can underperform. This happened with TMNT: Out of the Shadows and Transformers: The Last Knight, and it is happening with Justice League too.

To close, I just want to make clear that although I like Justice League quite well, this post is not meant to be taken as an endorsement of WB's interference. I personally would've much preferred Zack Snyder and Chris Terrio's original vision for the film. But I also think -- and the data supports -- that WB was facing an uphill climb at the box office regardless of whether they instituted reshoots or not, because of the divisive/derided films leading into Justice League. Because that challenge was going to be there regardless, WB probably would've been wise to save the money they spent on the reshoots because the changes failed to turn it into a universally acclaimed film. But superhero production budgets are a topic for another post.