Re: Beacon API

[Reitbauer, Alois]

Some comments on the post:

First I do not think we need special markup for this. Analytics tools will
trigger this programatically. I am however not sure whether the update
handler would really work. It does not solve the problem of the cancelled
XHR requests. It will also not be enough to just send the request when the
page is unloaded as some code might need to be executed first. In some
case a single XHR request is also not enough. We have rare situations when
we need to send more than one request.

// Alois

On 2/13/13 5:45 PM, "Tab Atkins Jr."  wrote:

>On Wed, Feb 13, 2013 at 8:03 AM, Jatinder Mann 
>wrote:
>> The Web Performance working group has been tracking a known poor
>>performance
>> pattern involving XHRs.
>>
>> We have seen cases where analytics code will block the unloading of the
>> document in order to send data. To guarantee that the data is sent to
>>their
>> servers, analytics will typically register a handler on the unload
>>event,
>> which will make a synchronous XHR call to submit the data. The
>>synchronous
>> XHR forces the browser to delay unloading the document, and makes the
>>next
>> navigation appear to be slower. There is little the next page can do to
>> avoid this perception of poor page load performance.
>>
>> Frankly, analytics don¹t have many good options. Browsers will typically
>> just ignore asynchronous XHR in an unload handler. Sending the data too
>>soon
>> may mean that they miss out on some data gathering opportunities. To
>>solve
>> this problem, the Web Performance WG has included writing a Beacon API
>>in
>> its charter [1]. This API would be an interoperable means for site
>> developers to asynchronously transfer data from the user agent to a web
>> server, with a guarantee from the user agent that the data will be
>> eventually sent.
>>
>> However, instead of inventing a new way to send data, it may make sense
>>to
>> first explore whether we can update XHR to help in this scenario. This
>> change could be as simple as adding an optional parameter to XHR, a new
>>type
>> of XHR (e.g., BeaconXHLHttpRequest), or just normative text on the
>>expected
>> user agent behavior when a synchronous XHR call is made in the unload
>>event
>> handler.
>>
>> How interested is this working group in taking on such work in the XHR
>>Level
>> 2 [2] specification?
>
>I started a thread last year in WHATWG about this subject, though from
>a slightly different angle:
>
>.
> The analytics use-case is a good one.
>
>~TJ
>

The contents of this e-mail are intended for the named addressee only. It 
contains information that may be confidential. Unless you are the named 
addressee or an authorized designee, you may not copy or use it, or disclose it 
to anyone else. If you received it in error please notify us immediately and 
then destroy it. Compuware Austria GmbH (registration number FN 91482h) is a 
company registered in Vienna whose registered office is at 1120 Wien, Austria, 
Am Euro Platz 2 / Gebäude G.

Re: Beacon API

[Reitbauer, Alois]

Anne,

Some more details that should help to clarify:

You are right we would not need any progress events and you are also
typically not interested in the response. The server should normally reply
only with a no content reply. Conceptually this reply would not be needed,
however, HTTP requires it. You can think of this type of interaction like
sending an UDP packet. You want it to be send and delivered at best
effort.

// Alois

On 2/13/13 5:46 PM, "Anne van Kesteren"  wrote:

>On Wed, Feb 13, 2013 at 4:03 PM, Jatinder Mann 
>wrote:
>> How interested is this working group in taking on such work in the XHR
>>Level
>> 2 specification?
>
>There's plans to develop a better API for XMLHttpRequest, that would
>be somewhat more object-oriented, use DOMFuture, etc., but reuse the
>same underlying architecture.
>
>I have started drafting the plan to define the underlying
>architecture: http://wiki.whatwg.org/wiki/Fetch
>
>I'd prefer waiting with adding new features in networking land until
>that is sorted out, but I'm open to suggestions on how to proceed
>otherwise. Also, some more details with respect to expected
>functionality would be necessary. I assume for this object you would
>not need progress events or return values, but should the browser
>await a full HTTP response from the server anyway? Etc.
>
>Thanks!
>
>
>--
>http://annevankesteren.nl/
>

The contents of this e-mail are intended for the named addressee only. It 
contains information that may be confidential. Unless you are the named 
addressee or an authorized designee, you may not copy or use it, or disclose it 
to anyone else. If you received it in error please notify us immediately and 
then destroy it. Compuware Austria GmbH (registration number FN 91482h) is a 
company registered in Vienna whose registered office is at 1120 Wien, Austria, 
Am Euro Platz 2 / Gebäude G.

Re: Beacon API

[Reitbauer, Alois]

I think CORS might still be needed as the data is not necessarily posted to the 
origin server. The name unloadRequest might be a bit misleading as this 
functionality might be used in a non unload situation as well. I also agree on 
the script-only capability.

// Alois

From: Dave Methvin mailto:dave.meth...@gmail.com>>
Date: Wednesday, February 13, 2013 7:18 PM
To: "Tab Atkins Jr." mailto:jackalm...@gmail.com>>
Cc: Jatinder Mann mailto:jm...@microsoft.com>>, 
"ann...@opera.com" 
mailto:ann...@opera.com>>, 
"public-webapps@w3.org" 
mailto:public-webapps@w3.org>>, Alois Reitbauer 
mailto:alois.reitba...@compuware.com>>
Subject: Re: Beacon API

On Wed, Feb 13, 2013 at 11:45 AM, Tab Atkins Jr.
I started a thread last year in WHATWG about this subject, though from
a slightly different angle:
.

A new simple API sounds like the best solution. Adding a sufficiently limited 
beacon into XHR would seem to involve a lot of special cases, since you don't 
want the response, callbacks, CORS, etc. but the requests themselves would need 
to stay around after the page is done.

It might be handy to have a method property to specify get/post and I'd prefer 
a name like unloadRequest since it's making a request and not setting a 
handler. Finally, is script-only capability sufficient or does it make sense to 
also have some form of markup specifying a link to be visited when the page 
unloads? 


The contents of this e-mail are intended for the named addressee only. It 
contains information that may be confidential. Unless you are the named 
addressee or an authorized designee, you may not copy or use it, or disclose it 
to anyone else. If you received it in error please notify us immediately and 
then destroy it. Compuware Austria GmbH (registration number FN 91482h) is a 
company registered in Vienna whose registered office is at 1120 Wien, Austria, 
Am Euro Platz 2 / Geb?ude G.

Re: Beacon API

[Reitbauer, Alois]

In the cases I see we are never interested in the response. The question
would also be how to handle it as the page that initiated it might - or
will - no longer be there.

I do not see how this relates to the ping. If I understand the Ping
correctly it send back the ping when a ling was clicked. The scenario here
is totally different. An analytics tool - whether RUM or other analytics -
collects a set of data like timing values etc. and then wants to beacon
this data back to the server for further processing. I am not sure how to
achieve this using the ping attribute.

This could be achieved by a method like this:

beacon (method, URL, postBody);

Today people are also using img tags that are dynamically created to
achieve this. This has the downside that a) it does not always work and b)
is limited in the amount of data that can be sent back due to URL length
restrictions.

// Alois

On 2/14/13 11:04 AM, "Anne van Kesteren"  wrote:

>On Thu, Feb 14, 2013 at 9:55 AM, Reitbauer, Alois
> wrote:
>> You are right we would not need any progress events and you are also
>> typically not interested in the response. The server should normally
>>reply
>> only with a no content reply. Conceptually this reply would not be
>>needed,
>> however, HTTP requires it. You can think of this type of interaction
>>like
>> sending an UDP packet. You want it to be send and delivered at best
>> effort.
>
>Are you sometimes interested in the response? Kind of makes a difference.
>
>Also, how is this different from
>http://www.whatwg.org/specs/web-apps/current-work/#ping Especially if
>we make this a dedicated feature in some way it seems all the same
>considerations apply and that feature has thus far not been hugely
>successful.
>
>
>--
>http://annevankesteren.nl/
>

The contents of this e-mail are intended for the named addressee only. It 
contains information that may be confidential. Unless you are the named 
addressee or an authorized designee, you may not copy or use it, or disclose it 
to anyone else. If you received it in error please notify us immediately and 
then destroy it. Compuware Austria GmbH (registration number FN 91482h) is a 
company registered in Vienna whose registered office is at 1120 Wien, Austria, 
Am Euro Platz 2 / Gebäude G.

Re: Beacon API

[Reitbauer, Alois]

What exactly do you mean by failed. Was nobody interested in it?

On 2/14/13 1:34 PM, "Anne van Kesteren"  wrote:

>On Thu, Feb 14, 2013 at 12:28 PM, Reitbauer, Alois
> wrote:
>> I do not see how this relates to the ping. If I understand the Ping
>> correctly it send back the ping when a ling was clicked. The scenario
>>here
>> is totally different. An analytics tool - whether RUM or other
>>analytics -
>> collects a set of data like timing values etc. and then wants to beacon
>> this data back to the server for further processing. I am not sure how
>>to
>> achieve this using the ping attribute.
>
>Sure, you can achieve even more with an API as you propose, but the
>idea is the same. The user navigates and the user agent is responsible
>to do some phone-home thingie. My point is that ping="" failed thus
>far, even though it's simpler conceptually.
>
>
>--
>http://annevankesteren.nl/
>

The contents of this e-mail are intended for the named addressee only. It 
contains information that may be confidential. Unless you are the named 
addressee or an authorized designee, you may not copy or use it, or disclose it 
to anyone else. If you received it in error please notify us immediately and 
then destroy it. Compuware Austria GmbH (registration number FN 91482h) is a 
company registered in Vienna whose registered office is at 1120 Wien, Austria, 
Am Euro Platz 2 / Gebäude G.

RE: Beacon API

[Reitbauer, Alois]

I agree with Jatinder that overloading technologies to do one more thing always 
created confusion and also makes APIs brittle.

The conversations this week were very helpful in deciding how to move forward. 
I second Jatinder's idea that we come up with a specification that describes in 
details what we need. We should also treat it as a separate specification. If 
we then see that it has enough commonalities with XHR and does not introduce 
unnecessary complexity we can still merge it.  I also think that this is the 
most efficient way to move forward here.

// Alois

From: Jatinder Mann [jm...@microsoft.com]
Sent: Saturday, February 16, 2013 12:50 AM
To: Anne van Kesteren; Reitbauer, Alois
Cc: public-webapps@w3.org
Subject: RE: Beacon API

I worry that overloading the ping attribute here may cause confusion and may 
not be well adopted for the use case we have presented.

Let me describe some potential requirements for such an interface. We want an 
asynchronous method of sending data. The interface shouldn't return a HTTP 
response, as the expectation is that the user agent would be responsible for 
sending this data when it could. The user agent must be able to send this data 
even after the page had unloaded, potentially even attempting to re-send it if 
the first attempt fails. The interface must support CORS, as one may want to 
send this data to a different origin. The interface wouldn't be limited to the 
unload and could be used at any time to reliably send data.

What we wanted to understand was whether it makes more sense to create a XHR 
variant that does this or if it would just be less confusing to create a new 
beacon API, as Alois had suggested.

Considering the requirements, especially if it is only designed to send data 
and not receive, it may just make more sense to create a specific beacon API 
here rather than creating a XHR invariant. I think Alois and I should come up 
with a more concrete proposal and then we can better weigh the pros and cons of 
the different approaches.

Thanks,
Jatinder

-Original Message-
From: annevankeste...@gmail.com [mailto:annevankeste...@gmail.com] On Behalf Of 
Anne van Kesteren
Sent: Thursday, February 14, 2013 5:54 AM
To: Reitbauer, Alois
Cc: Jatinder Mann; public-webapps@w3.org
Subject: Re: Beacon API

On Thu, Feb 14, 2013 at 12:38 PM, Reitbauer, Alois 
 wrote:
> What exactly do you mean by failed. Was nobody interested in it?

There was some misguided controversy about the feature and I think that pretty 
much did it in. It has all the same characteristics as this new proposal, but 
maybe this one will not get the misguided controversy?

(The controversy was that ping was designed for tracking. That it would improve 
the situation for the end user over invisible tracking (as this could be 
disabled) was not taken into account obviously.)


--
http://annevankesteren.nl/


The contents of this e-mail are intended for the named addressee only. It 
contains information that may be confidential. Unless you are the named 
addressee or an authorized designee, you may not copy or use it, or disclose it 
to anyone else. If you received it in error please notify us immediately and 
then destroy it. Compuware Austria GmbH (registration number FN 91482h) is a 
company registered in Vienna whose registered office is at 1120 Wien, Austria, 
Am Euro Platz 2 / Gebäude G.

Re: Beacon API

[Reitbauer, Alois]

From my perspective the point is that we should rather have a clear(er)
definition of what we need, rather than starting to see how it fits into
existing specs. Having this initial spec it will be also easier to decide
about the actual fit into XHR or PING

// Alois

On 2/18/13 10:19 AM, "Anne van Kesteren"  wrote:

>On Sat, Feb 16, 2013 at 11:36 AM, Reitbauer, Alois
> wrote:
>> The conversations this week were very helpful in deciding how to move
>>forward. I second Jatinder's idea that we come up with a specification
>>that describes in details what we need. We should also treat it as a
>>separate specification. If we then see that it has enough commonalities
>>with XHR and does not introduce unnecessary complexity we can still
>>merge it.  I also think that this is the most efficient way to move
>>forward here.
>
>It's not entirely clear to me how we moved from "we need a simple flag
>on XHR" to lets create a whole new API...
>
>
>--
>http://annevankesteren.nl/
>

The contents of this e-mail are intended for the named addressee only. It 
contains information that may be confidential. Unless you are the named 
addressee or an authorized designee, you may not copy or use it, or disclose it 
to anyone else. If you received it in error please notify us immediately and 
then destroy it. Compuware Austria GmbH (registration number FN 91482h) is a 
company registered in Vienna whose registered office is at 1120 Wien, Austria, 
Am Euro Platz 2 / Gebäude G.

Re: Beacon API

[Reitbauer, Alois]

My personal experience is different. We found that using img tags is not that 
reliable. Especially in Firefox we recently saw some problems. Img tags in 
general have the disadvantage that the amount of data that can be set is rather 
limited. While this obviously should be kept as small as possible, the 
information available via resource timing will increase the amount of data that 
gets sent.

Is there a way we can integrate this into a W3C test suite to check how 
different browsers behave in this case

// Alois

From: Andy Davies mailto:dajdav...@gmail.com>>
Date: Sunday, February 17, 2013 11:12 PM
To: Maciej Stachowiak mailto:m...@apple.com>>
Cc: "ife...@google.com" 
mailto:ife...@google.com>>, Anne van Kesteren 
mailto:ann...@annevk.nl>>, Web Applications Working Group WG 
mailto:public-webapps@w3.org>>, 
"public-web-p...@w3.org" 
mailto:public-web-p...@w3.org>>
Subject: Re: Beacon API
Resent-From: "public-web-p...@w3.org" 
mailto:public-web-p...@w3.org>>
Resent-Date: Sunday, February 17, 2013 11:13 PM

On 16 February 2013 05:25, Maciej Stachowiak 
mailto:m...@apple.com>> wrote:

BTW as far as I know the best current nonblocking technique to phone home on 
unload is to create an  in your unload handler pointing to the ping URL, 
this will result in reliable delivery without blocking at least in IE and 
WebKit-based browsers. I've found it hard to convince even knowledgable web 
developers to use this technique or  over synchronous XHR, even sites 
that are otherwise willing to do Safari-specific optimizations. I am not sure 
why sync XHR in unload is so tantalizing.


I asked Phillip Tellis (creater of Boomerang) about this and his experience is: 
(copied because he can't post to the list)

"A beacon may be sent using a GET request via an Image object in a listener for 
either the onunload,the onbeforeunload or the onpagehide events.  No browser 
blocks if this is sent via the onbeforeunload event, however Opera does not 
support this event.  Firefox is the only browser that blocks for Image requests 
in the onunload event.

Additionally, attaching a listener to the onunload event affects Page Cache in 
most browsers, so one should avoid that in any event.

The solution we've come up with for boomerang (a general purpose beaconing 
solution primarily used for web performance beacons) is to attach a listener to 
the beforeunload event and either the pagehide or unload, depending on which is 
available (pagehide is checked first).  The first of these events to fire sends 
the beacon out and removes the listener from the other.  This has allowed us to 
fire beacons on every page unload without affecting unload time and without 
beacon loss."

Andy
The contents of this e-mail are intended for the named addressee only. It 
contains information that may be confidential. Unless you are the named 
addressee or an authorized designee, you may not copy or use it, or disclose it 
to anyone else. If you received it in error please notify us immediately and 
then destroy it. Compuware Austria GmbH (registration number FN 91482h) is a 
company registered in Vienna whose registered office is at 1120 Wien, Austria, 
Am Euro Platz 2 / Geb?ude G.