Analytics and private videos

mlapis · January 9, 2021, 10:48pm

Hi Doug,

I am trying to understand and create some strategy on how private videos are used. Using the latest SDK for Node and returned data, I cannot differentiate between the viewing of two separate accesses to the same private video from the same environment (the same end-user).

If I play the same private video using two different tokens, the returned analytics for that video contains only one element in the “data” array, and the “session” property indicates “loadedAt” and “endedAt” as a combination of times from those two separate accesses. So it’s not possible to differentiate and understand when and how long each of those accesses takes. Also, analytics data for session events (because all is included under the same session) mixes all events together.

The result means that it’s very hard to get, how many times that private video was accessed, and other details. Maybe detail parsing of session events and not trivial logical analysis can lead to some results, but it’s really not an easy way.

I guess that the actual state is because the private “token” param is now ignored as a differentiation flag, and because all other information is the same (the same client in the same environment).

Can you comment on my observations?

anon89371750 · January 10, 2021, 7:32pm

Hi @mlapis

This is an interesting one

This is a case where the session analytics are set by a cookie in the browser. What this means: I watch your private video with token “a”, and then (in the same browser session) watch the same video, but I had a new token “b” - it will appear in the same session. (I’ll verify this is the case with the devs on Monday).

Even if the sessions were hours apart (or, if you’re like me and you never close Chrome, these could be days apart )

Technically - since it is the same session - you could have re-used token “a” on the second visit. If you do visit 1 in Chrome, and visit 2 in another browser - you’ll see 2 sessions for the 2 visits.

I think that we might be able to add some definition around this, but it might add complexity to other parts of the puzzle… Just off the top of my head:

We could force the private token as dynamic metadata:
PRO: Would force exactly one session per token.
CON: One user could create many “views” (and sessions) by accessing the video multiple times.
(for example - to see how much of the video this user watched - you’d have to access each session to add up the “watched time”)
Leave it as is: Know that you might have users that consume several tokens, and watch the video several times - but the viewer is really just one user.

What do you think?

Doug

mlapis · January 10, 2021, 8:40pm

Hi Doug,

This is the case, as you described with the tokens “a” and “b”, exactly. In reality, you can imagine that there is an application that serves private videos for registered users under some business conditions. Each user can select from available subscriptions and run individual videos (where each run requests a new token = to be identified uniquely).

To be correct and deeply understand how those users use videos (how frequently are videos viewed, what parts, and so …), it’s necessary to have separate analytics details. So for our case, only point 1) has a sense > force exactly one session per a token. I understand that the complexity would increase slightly, but I don’t see it as a problem, especially if there is a simple way to get a list of all sessions related to one [videoId].

Btw, that application has to use some relation between each [token] and an individual registered user to assign concrete analytics to a correct user later. Each new view video invoking requires an appropriate request against API.VIDEO from a server-side to get a new token, and each user id can be also added as dynamic metadata to analytics, because each time the URL is constructed dynamically from the ground.

Milos

anon89371750 · January 11, 2021, 1:36pm

Milos,

I’m going to add this to the roadmap. I think it is worth closing this loophole.

In the meantime, if you’re already creating dynamic metadata for each video: If you add a “token” metadata field (and make it dynamic), you can insert the token as a parameter:

https://embed.api.video/vod/vitkytkieTnEvUjKE0VQpd2?token=523019d8-87f8-4a02-a096-79c6995d61f6&metadata[token]=523019d8-87f8-4a02-a096-79c6995d61f6

This will force every view (even in the same window session) into a new analytics session.

Doug

mlapis · January 11, 2021, 6:19pm

Yep, I had hoped so, when I wrote the previous reply. I will try this way and see. When done, I’ll let you a summary.

The URL would be a bit longer and redundant but as a temporary solution still useful.

mlapis · January 12, 2021, 3:23pm

Hi Doug,

I tried to do it, but something is wrong because dynamic metadata doesn’t work as expected. I think that I managed all things correctly, for example:

the called URL: https://embed.api.video/vod/vi4FENtkNOwuxtq0JCNzIts3?token=71f5c5d3-6885-4dbb-aed8-d2bd5de6ac54&metadata[token]=71f5c5d3-6885-4dbb-aed8-d2bd5de6ac54
metadata defined on the uploaded video: token > __token__

The result is that the analytics metadata property is still an empty array (both on the session level and the top-level), and the “data” property has only one element that is updated on each new video viewing with a different token, so no elements are added for new tokens, but the already existed element is updated. It looks that there is a problem even with the basic functionality of dynamic metadata because it still contains an empty array in analytics data.

Not sure, if it can be important, but videos (each time with a new token) are opened inside the same browser window, as modal dialogs.

PS: Another explanation would be that I made a mistake somewhere, but I can’t see it.

UPDATE: If I open a video with a new token inside a separate browser window, then a new element is added to the “data” property of analytics, so it’s the behavior as expected, but “metadata” property is still an empty array, so “token” value is not recorded.

UPDATE: If I added “userId” to do dynamic metadata, as in URL:

https://embed.api.video/vod/vi3DmzZc2VAjmVzERT6bkQkQ?token=a36d628f-8162-4b61-9865-cd1539296c6c&metadata[token]=a36d628f-8162-4b61-9865-cd1539296c6c&metadata[userId]=4

the “metadata” analytics property inside a newly added “data” element is still an empty array if the video is opened inside a new browser window.

Milos

anon89371750 · January 13, 2021, 10:51am

Milos,

In my testing, I must have been opening new tabs for each test.

Supporting your testing: if you open two copies of the same video (with different metadata) in the same tab - only one analytics session, and the data in this session is updated from both videos.

If you open the same video with 2 sets of metadata in 2 tabs - there will be 2 sessions in the analytics.

Regarding your updates:

I am seeing that you are getting the user/token data in your metadata (when you open a new tab):

{
			"session": {
				"sessionId": "ps6S6tTb9SZppOAT3xdCCaWN",
				"loadedAt": "2021-01-12T16:46:26+00:00",
				"endedAt": "2021-01-12T16:46:41+00:00",
				"metadata": [
					{
						"key": "token",
						"value": "3a84b2a2-bf81-4c05-ad71-5f4796bab916"
					},
					{
						"key": "userId",
						"value": "4"
					}
				]
			},
			"location": {
				"country": "Czechia",
				"city": "Brno"
			},
			"referrer": {
				"url": "http://localhost:4200/",
				"medium": "unknown",
				"source": "unknown",
				"searchTerm": "unknown"
			},
			"device": {
				"type": "computer",
				"vendor": "unknown",
				"model": "Other"
			},
			"os": {
				"name": "Windows",
				"shortname": "Windows",
				"version": "10"
			},

Can you please clarify where you are seeing empty data elements in your analytics?

anon89371750 · January 13, 2021, 11:05am

On to the issue you are facing - if user “doug” watches the same video >2x in the same tab (so it is recorded in the same session). How can we break apart these into two views?

What if you went into the session data? for example, here is a video I watched a few days ago:

sessions [
  { type: 'ready', emittedAt: '2021-01-11T14:22:43+00:00', at: 0 },
  { type: 'play', emittedAt: '2021-01-11T14:22:43+00:00', at: 0 },
  { type: 'pause', emittedAt: '2021-01-11T14:22:54+00:00', at: 9 },
  { type: 'ready', emittedAt: '2021-01-11T14:22:59+00:00', at: 0 },
  { type: 'play', emittedAt: '2021-01-11T14:22:59+00:00', at: 0 },
  { type: 'ready', emittedAt: '2021-01-11T14:23:14+00:00', at: 0 },
  { type: 'play', emittedAt: '2021-01-11T14:23:14+00:00', at: 0 },
  { type: 'pause', emittedAt: '2021-01-11T14:23:17+00:00', at: 2 },
  {
    type: 'seek.forward',
    emittedAt: '2021-01-11T14:23:18+00:00',
    from: 2,
    to: 102
  },
  { type: 'resume', emittedAt: '2021-01-11T14:23:20+00:00', at: 102 },
  { type: 'pause', emittedAt: '2021-01-11T14:23:22+00:00', at: 104 },


<snip>

  { type: 'play', emittedAt: '2021-01-11T14:34:23+00:00', at: 0 },
  { type: 'end', emittedAt: '2021-01-11T14:34:27+00:00', at: 215 },

]

I cut some of the session events for clarity, but here we can see this session for a video had a “jump” in the emmitedAt - that is larger than the video length (or the timings shown in the “at” parameter.

This is not as easy as simply counting the views of a private video. But you could use these events to see if the are actually watching the video (by adding up the timings) on each view.

For example on the first view, my start time may have been 0, and the end time 104 - but I also skipped ahead from 2->102, bit watching a large section of the video).

Doug

mlapis · January 13, 2021, 11:05am

Hi Doug,

but I am seeing this in the same returned analytics data:

The related call is simple as: .analyticsVideo.get(videoId, '2021-01')

So I don’t get why I haven’t been receiving that metadata property.

PS: Our web application works as SPA application and we don’t want to open other browser windows, that’s why those modal dialogs.

Milos

mlapis · January 13, 2021, 11:38am

Yep, parsing, and analyzing events data could help, how to understand what happened.

If I understand, the principal problem is that there is only one permanent “sessionId” for one browser tab.

So if we would like to solve it, the “sessionId” would have two parts. The new second one could be parametrized by some flag to activate it compared to the current functionality, right?

Milos

anon89371750 · January 13, 2021, 11:55am

Correct: Today there is one sessionId per video per tab.

Something like multiple parameters would have to occur. A way to differentiate between those session Cookies, and for each one to be treated differently (and also in a way that it will not break any current functionality )

Doug

mlapis · January 13, 2021, 11:59am

So, the actual main problem is, why I receive empty “metadata” for the same session when you can see the correct expected data (the above example, where we both look at the same “sessionId” analytics data, already saved in the system: "sessionId": "ps6S6tTb9SZppOAT3xdCCaWN"). Without a solution to this problem, we can’t even understand, who is seeing videos. I don’t have any ideas, why it happens, but I suppose that the called code is so simple, that the problem can’t be on our side.

Can you try to explain, what is happening?

Thanks a lot.

Milos

anon89371750 · January 13, 2021, 1:10pm

Milos,

When I try that same video in CURL, I get the metadata array:

In your screenshot, I see a little carat pointing right - can you click the carat and see if it “opens” the array?

Doug

mlapis · January 13, 2021, 1:56pm

Doug,

it’s a screenshot from the browser console and even an empty array has a carat mark, because of other JS object properties, like the length.

It’s really a mystery. Could it be related that I am accessing data through NodeJS SDK and you probably by some other way? Maybe NodeJS SDK server-side has a problem somewhere.

Milos

anon89371750 · January 13, 2021, 2:41pm

Looks like a bug in the Node JS SDK.
Filing a ticket now.
In the meantime:

//first we must authenticate
	var authOptions = {
		method: 'POST',
		url: 'https://ws.api.video/auth/api-key',
		headers: {
			accept: 'application/json'
			
		},
		json: {"apiKey":apiVideoKey}

	}

	request(authOptions, function (error, response, body) {
		if (error) throw new Error(error);
		//this will give me the api key
		//console.log("body", body);
		var authToken = body.access_token;
		console.log(authToken);
		
		//now lets look at the video in question
		//with the Username metadata

		
		var firstSessionQuery = {
			method: 'GET',
			url: 'https://ws.api.video/analytics/videos/'+videoId+'?currentPage=1&pageSize=1&metadata[userName]='+userName,
			headers: {
				accept: 'application/json',
				authorization: 'Bearer ' +authToken
			}
		}
		request(firstSessionQuery, function (error, response, body) {
			if (error) throw new Error(error);
			body = JSON.parse(body);
			console.log("body", body);
			
                        //do you thing

		});

I have this code up and running at resume.a.video right now. It works

Thanks for working through this with us.

Doug

mlapis · January 13, 2021, 3:22pm

Doug,

great. So, it’s clear now where the problem is. Do you approximately know how long it takes to fix it?

anon89371750 · January 15, 2021, 11:17am

NodeJS metadata fix is pushing live now.

mlapis · January 15, 2021, 11:52am

Doug,

super. I tested it already, and it works as it should. Good job and fast. Thanks.

Milos

Topic		Replies	Views
How to use react-player with private video Player & Analytics	1	624	January 26, 2023
API functionality to have a custom private video control VOD	4	822	October 5, 2021
How are set sessionIds VOD	2	742	November 11, 2020
Cache != single use token on private video VOD	2	722	December 11, 2020
What’s the difference between Private and Public vídeos? VOD	3	1307	March 17, 2020

Analytics and private videos

Related Topics