Visibility data on VOD services, from top to bottom.

Get the full picture

Content availability and audience metrics do not tell the whole story.

Tackling streaming’s analytical blind spot: content prominence

We took a look for the CNC at the visibility of French titles on VOD.

Arvester data included in the CNC’s annual VOD observatory

We give each title a score that corresponds to its degree of visibility. Here's how that works.

Our visibility score explained

Arvester was selected to be part of the French delegation to MTL Connect in Montreal.

Notes from Montreal

Want to know the month's top 10 most visible titles? Or how prominent an actor is on the homepage? How about the visibility of Japanese anime to a user who likes Korean dramas? We've got you covered.

The right data for your needs

, former Sorbonne University researchers. We founded Arvester with the aim of taking analytics in the audiovisual industry to the next level.

Our expertise sets us apart

Imagine knowing a film’s box office receipts but not how wide of a release it got, or a TV program’s audience but not on what channel or at what time it aired. That’s where we are at right now when it comes to video-on-demand analytics. Sure, 

 was a hit. But how prominent was it? And can we really say that 

 was a flop when we don’t know how visible it was on Prime Video's homepage?

Just like a TV program’s audience depends on where and when it is scheduled (network position, primetime slot), a title on a VOD service will succeed only if it is 

 It seems obvious, but a user is more likely to pick a title that is featured on the first row of the homepage than a title on the bottom 

 Yet discoverability on VOD services has until now remained beyond the scope of analysis in the industry. Rights holders, studios, sales agents and creatives frequently denounce the opacity of VOD services, which causes them to make decisions based on incomplete information. We strongly believe that by introducing visibility as a new metric, we can help all parties get access to the information they need.

Addressing widespread information asymmetry

As a rights holder, licensing to a VOD service is a risky affair. Throughout the industry, VOD rights are being sold “blind”, using woefully inadequate metrics like box-office performance to estimate the on-demand value of titles. The fact is that you know far less about your content’s worth than the buyer:

They know how similar titles have performed on their service; you don’t. Or maybe you’ve bought vague data from a “panel of consumers” that’s supposed to tell you, through considerable interpolation, what millions of users like to watch.

They know how much prominence they will give that title; you don’t. You also don’t know how prominent similar titles are or have been on that service.

Not to say that all rights holders are getting ripped off, but information asymmetry certainly doesn’t help. In fact, one of the conclusions of the Berlinale EFM's 2024 Think Tank was this: "The lack of data transparency puts producers at a disadvantage during negotiations, reinforcing their lack of bargaining 

We provide data that helps rights holders (studios, producers, distributors, sales agents) negotiate deals on equal terms with VOD services. Access to the right information allows them to better estimate the value of their content and, crucially, stops them from selling themselves short. Over the past few years, we have seen multiple instances of films being sold cheap to a streamer because of poor box office performance, only for those films to be given massive prominence on that service and enter the Top 10 most viewed titles of the week.

It is not uncommon, when selling theatrical or TV rights, to demand a minimum marketing spend on the part of the buyer, or a minimum number of copies on which the film should screen. Perhaps the time has come to apply the same logic to VOD by demanding a minimum level of visibility on the interface.

Visibility is a potent indicator of performance. Streaming has taught us that measuring performance in a non-linear context isn’t quite as straightforward as it used to be in the linear world. Netflix can dump six months of viewership data and there is very little that we learn from it. Measurement firms (Nielsen, Barb, Médiamétrie and others) have to contend with panels and extrapolation leading to inaccuracy. Then there is that whole “what counts as a view” debate…

And even when we do know viewership, it seldom tells the whole story. For example, do low audience numbers point to a lack of attractiveness or a lack of visibility? Is a sharp increase in a title’s viewership the result of extraneous factors or is it due to improved visibility on the interface?

The right approach would be to combine different metrics. By complementing viewership with visibility and availability data, we obtain a more detailed picture of a title’s performance on VOD.

Visibility data can help analysts break free of what Amanda Lotz calls the 

 like scheduling, prime time, audience measurement... These norms, which are often awkwardly transposed to the on-demand ecosystem, can only offer a partial understanding of the actual mechanisms at play. Only by making use of a VOD-specific analytical tool set can industry professionals better understand content strategies in a non-linear context. Measuring content prominence on a user interface may be a new approach, but it is long overdue.

When applied on a per-title basis, visibility metrics allow us to establish rankings of the most visible titles on a service at any given time, which often have only little overlap with the Top 10 charts on those services. Our 

 also makes it possible to look at the overall visibility of any variable, such as genres, nationalities, languages, even actors and directors. Here are some of the questions that we can answer:

What genre of content (reality TV, animation, documentaries...) is most visible on the homepage?

How prominent is local content compared to international productions?

What are the most prominent actors/directors on the service and how does their visibility evolve over time?

Taking algorithmic personalisation into account

But what of algorithmic personalisation, you might ask. Isn’t each profile unique, based on past behaviour? Well, that’s where looking at prominence gets even more critical.

The most popular VOD services satisfy niches, and personalisation implies that the more a user veers toward a certain niche, the less other niches will be visible on the interface. This is what viewership data fails to take into consideration. As it stands, measuring viewership on VOD is a bit like measuring broadcasters’ audience shares, only each viewer is in a totally different market. 

At any given time, my Netflix homepage, with all its rows and thumbnails, is showing me about 800 unique titles out of a library of 5000. But these 800 titles are curated according to my past behaviour, and my next door neighbour probably has 800 completely different titles on their homepage. For example, the fact that my neighbour has watched the latest season of 

 and I haven't isn't necessarily due to the show's appeal or lack thereof. It may simply be that it was never suggested to me in the first place.

Our methodology allows us to overcome the limitations of viewership data. By looking at content prominence on the homepages of users who have expressed interest in specific genres, languages or formats, we can start to unravel the process through which a user comes to watch certain shows or films over others. Analytics in streaming have tended to focus on the demand side of the market, but it is really by looking at the interplay between supply (content prominence) and demand (viewership) that one can arrive at an understanding of the VOD ecosystem.

Here are a few examples of the kind of questions we provide answers to:

Are K-drama fans being suggested documentaries?

What titles are prominent no matter what the user has watched in the past?

Is auteur fare only being shown to users who have previously watched indie movies?

Are WWII documentaries more prominent when the user has watched 

Analytics in the VOD industry have to keep up with the times. Library data (what's available on Netflix, Prime Video and others at a given moment) is essential. But it is also very basic information that, all things considered, doesn't tell us a great deal about those services' content strategies. 

Many professionals still approach the on-demand ecosystem with a mindset inherited from the world of cinema and TV. The shift from linear to non-linear consumption of audiovisual content has mostly been interpreted as an evolution within an established industry, and analytical tools have stayed unchanged. But what we should be doing is deploying new tools to reflect changing practices in the industry and keep up to date with the new ways that audiences discover content.

Visibility data is the missing piece of the puzzle when it comes to understanding how supply and demand interact in an on-demand world. Content is no longer broadcast or scheduled, it is suggested in a thumbnail, and where that thumbnail appears on the user's screen is of the utmost importance. This is where the core of the industry now lies, and it is also where we should focus our analytical gaze.

Interview by Davide Abbatescianni. Read the article 

We spoke to Grégoire Bideau and Steven Tallec, researchers at 

. The two scholars recently published a study titled 

Algorithmic Panic: European Contents on the Netflix Homepage

, during which 20 bots watched Netflix every day for eight days and tried to gain insights into European works’ prominence on the platform.

Cineuropa: Why did you start studying Netflix’s algorithm?

For the past couple of years, we’ve had the opportunity to speak to a number of stakeholders in the European audiovisual industry, from regulators to producers, government officials and analysts. During our conversations, the question of the impact of Netflix’s recommender system on the visibility of European works came up almost every time, but there did not seem to be any reliable data from which to determine whether that impact was positive or negative, although the latter was often assumed.

 Prominence is an important aspect of the EU’s 

, which states that services “shall ensure prominence of [European] works”, and not simply set aside a 30% share of their catalogues for them. Even though “prominence” has never been properly defined, let alone regulated, professionals do seem aware of its importance in the circulation of European works. However, it is a slippery notion in an algorithmically personalised context, where prominence is tied to past viewing behaviour. As such, it is very difficult to study quantitatively, even though it plays an important mediating role between a vast catalogue of contents and the end user [read 

]. This is why, at the Chaire PcEn, we placed content prominence at the heart of our research programme two years ago. We chose Netflix as our first test subject because of its popularity and its famed recommender system, and we hoped that our research would shed light on an otherwise opaque mechanism.

What was your methodology when conducting this experiment?

 First, we gathered metadata for each title available on Netflix in France by finding its corresponding page on IMDb (not by hand, but with a specially designed script), which allowed us to collect its producing countries, languages, actors and so on.

 In parallel, we set up a cluster of 20 tiny computers (Raspberry Pi’s), each tasked with creating its own Netflix profile on a dedicated account [read more]. Then, each computer was fed a schedule of when to watch Netflix (in this case, three hours each day for eight days) and a list of titles to watch. Ten computers were given a list made up solely of European titles, and the remaining ten were given a list of non-European works. All lists were arranged in a random order so as to prevent any selection bias and to ensure that all computers were watching different titles at the same time. We relied on automated web browsing so that there was no human intervention during the whole of the experiment. We collected the contents of each bot’s homepage once a day, including the exact placement of each title.

Could you talk us through the results? What kind of insights did you gain in terms of the circulation of European works?

 Contrary to what many feared, European works certainly do not disappear from the Netflix homepage, even when the user chooses not to watch any European titles. There is a slight downward trend in the number of thumbnails promoting European works, but nothing that would make European content absent from the homepage. As for the bots that watched only European content, the number of European thumbnails on the homepage increases, but not so much as to create a “filter bubble” that would suggest only European titles to the user. In both cases, it seems that the extent to which the recommender system influences the visibility of European works has been greatly overestimated.

 Of course, this also has to do with using “European” as the sole differentiating variable between our two sets of bots. European works do not constitute a genre, nor are they perceived as particularly similar to all of Netflix’s users. In fact, we could go as far as to say that the diversity of European works partly ensures their prominence on Netflix, as they are likely to be included in a wide variety of taste clusters, and thus recommended to users with very different tastes.

This eight-day experiment is certainly interesting, but perhaps a longer run would even provide more solid, additional takeaways. Are you willing to prolong it, or if not, why do you believe such a length is appropriate?

Sure, a longer experiment would be interesting, and we hope to be able to carry out other experiments that will last longer in the future (technical issues notwithstanding!). We felt that eight days, or just over a week, was an appropriate period for looking at European works because any longer would have meant some extremely unusual behaviour from our bots, which could have skewed the results.

What will your next steps be? Would you be interested in conducting a similar study on other VoD platforms?

 First, we will continue to put this methodology to use in future experiments, where we will explore other variables, such as languages and genres. We will also strive to look more closely at the results, and provide more in-depth analyses of the movements on the homepage. We have already started a partnership with Professor 

 at the Seoul National University to understand how Netflix’s recommender system promotes Korean content in France. The results will be presented at a conference in Seoul on 9 April. It would also be interesting to measure the prominence of European works in English-speaking countries, such as the UK and the USA, but that would mean either moving our computers to another country to access its Netflix catalogue or using a VPN for the whole process.

Of course, it is technically feasible to conduct a similar experiment on other VoD platforms that use algorithmic personalisation. Some elements of the methodology would have to be adapted, of course, requiring quite a bit of effort to automate browsing and data collection on those platforms, but the crux of the process would remain the same. Amazon Prime Video, Disney+ or even Twitch would be interesting to look at, for example.

The two scholars have developed a method for studying the recommendations of the Netflix algorithm

Interview Cineuropa

Arvester was part of the French delegation to the 

 event in October 2023, made up of a dozen entrepreneurs and researchers each working on a project to promote the discoverability of cultural content online.

I was invited by the French and Canadian Ministries of Culture to speak on the subject of measuring the visibility of cultural goods online. I took this opportunity to present our innovative methodology to a large number of film industry professionals.

Other speakers relayed their fears at not knowing how prominent content is online, notably French-language titles in an English-language dominated industry.

The event brought to the fore the necessity of introducing new measurement tools to the VOD ecosystem, and of making them accessible to creatives, rights holders and institutions so that they may gain access to information that has until now remained beyond their reach.

I am grateful to the organisers of MTL Connect and to Marie Béraud and Sébastien Thévenet at the French Ministry of Culture for selecting us.

The CNC's VOD market observatory is getting an update this year. On top of presenting data on the demographics of VOD subscribers and the place of French films and series in VOD catalogues, it now includes our data on the visibility of French titles on VOD interfaces.

Every week from early December 2023 to early January 2024, we looked at the homepages of Netflix, Prime Video and Disney+ with the goal of analysing how visible French 

 were. The CNC has known the share of French titles in VOD catalogues for years, but they remained frustrated at not knowing which of those titles were given prominence on those services, and how French productions fared against other countries. For us, working for the CNC was an opportunity to apply our methodology on a macro level, meaning not only operating on a per-title basis but also looking at the entirety of French-produced titles. Here is what we learned.

It should first be noted that the design of the interface is somewhat different from one service to another. This is particularly obvious when you compare the homepages of Netflix and Prime Video. Netflix's homepage contains about 1400 thumbnails, while Prime Video shows over 4000.

Our analysis shows quite clearly the differing strategies adopted by each streamer when it comes to the prominence of films and series. While Netflix tends to award greater prominence to series, Prime Video does the opposite. Overall, series were 3.5 times more prominent than movies on the Netflix homepage. It's the other way round on Prime Video, where movies were 2.3 more prominent than series. Disney+ sits in the middle, with a more balanced approach to the visibility of films versus series.

Also of note is the ranking of the most visible countries of production. American content is given greater prominence on all three platforms, as was expected given the sheer volume that US productions occupy in their catalogues. France is in close second on all three services, followed by the UK. This shows a strong commitment to the promotion of local content on the part of Netflix, Prime Video and Disney+. Of the three streamers, Netflix is the one where the gap in visibility between US-produced and French-produced titles is the narrowest.

An analysis of the 10 most visible French works over the period reveals major differences in prominence strategies. Prime Video highlights French comedians, while Netflix focuses on series. Kamel Guemra's French series 

 is in fact so prominent that it is also in the top 10 for all nationalities 

, available on Prime Video, is also in the top 10 for all nationalities, in second place.

We would like to thank Cécile Lacoue, Director of Studies at the CNC, and her team for their confidence. We look forward to continuing this work next year.

In order to make sense of the intricate mechanisms at play regarding content prominence on VOD services, we've created a visibility score, which helps us to carry out better analyses and facilitates understanding for our clients. This score allows us to rank all thumbnails on a homepage —and by extension all titles, genres, nationalities, actors etc.— based on how visible they are to the user.

A score from 0 to 100 is automatically assigned to each thumbnail on the homepage. It takes into account the position of the thumbnail in relation to the top and the left edges of the page, as well as its size on the screen. A score of 100 means that the thumbnail appeared at the very top of the page and filled the user's whole screen. The lower the score, the less visible the thumbnail was on the page. Titles often appear multiple times on the same homepage, so the total visibility score for a single title can be above 100. The same goes for genres, actors/directors, languages etc.

This simple operation makes it possible to establish rankings of the most visible titles, countries of production, genres and so on at any given time or over a period of time.

Get the full picture

Find out just how
visible
prominent
discoverable

titles are on VOD.

Blog

Clients

The right data for your needs

Use cases

Rights holder

Studio

Content strategist

Collective management organisation

Our expertise sets us apart

2020

2022

2023

2024

Get the full picture

Find out just how visibleprominentdiscoverabletitles are on VOD.

Blog

Clients

The right data for your needs

Use cases

Rights holder

Studio

Content strategist

Collective management organisation

Our expertise sets us apart

2020

2022

2023

2024

Find out just how
visible
prominent
discoverable

titles are on VOD.