Re: Lazy Blob

[Glenn Adams]

On Wed, Aug 1, 2012 at 10:44 PM, Ian Hickson  wrote:

> On Wed, 1 Aug 2012, Glenn Adams wrote:
> >
> > Of course, implementers are free to ignore whatever they want, but last
> > time I checked, the W3C was a consensus based standards organization
> > which means agreement needs to be reached on what specs go out the door
> > and what are in those specs.
>
> Doesn't really matter what's in the specs going "out the door" if it's not
> what's implemented...
>
> I don't really care about the XHR side of this (happy to let Anne figure
> that out), but since WebSockets was mentioned: what's the use case that
> involves Web Socket? I don't really understand what problem we're trying
> to solve here.
>

based on the pattern proposed by

partial interface BlobBuilder {
Blob getBlobFromURL (XMLHttpRequest xhr);
};

i would like to support two use cases:

(1) simple - single blob send, single blob response
(2) multiple - multiple instances of simple, i.e., send/response pairs

these could be handled with the following:

partial interface BlobBuilder {
// simple
Blob getBlobFromSource (WebSocket ws, Blob send);
// multiple
Blob getBlobFromSource (WebSocket ws, EventHandler sendHandler);
};

in the simple case, the creator of the lazy blob provides the initial send
blob, which is sent by the underlying lazy blob implementation upon a read
on the lazy blob, then the next (and only) response blob is returned as a
result from the read

in the multiple case, the creator of the lazy blob provides an event
handler to invoke to send a blob corresponding to a read on the lazy blob,
thus providing for multiple send/receive blob message pairs, with one lazy
blob for each pair

of course, the simple case could be folded into the multiple case, leaving
only one method to define:

partial interface BlobBuilder {
Blob getBlobFromSource (WebSocket ws, EventHandler sendHandler);
};

a use of this might be as follows:

var bb = new BlobBuilder();
var ws = new WebSocket("ws://example.com:/test");
var lb = [];
ws.onopen = function() {
lb.push ( bb.getBlobFromSource(ws, function() {
ws.send(getSendMessageAsBlob(1)); }) );
lb.push ( bb.getBlobFromSource(ws, function() {
ws.send(getSendMessageAsBlob(2)); }) );
lb.push ( bb.getBlobFromSource(ws, function() {
ws.send(getSendMessageAsBlob(3)); }) );
setTimeout(sendAndReceive);
}
function getSendMessageAsBlob(msgNum) {
return new Blob ( [ String(msgNum) ] );
}
function sendAndReceive() {
var numMsgs = 0;
var numBytes = 0;
// trigger read on queued lazy blobs


while ( lb.length > 0 ) {
b = lb.shift();
// read on size triggers stored send 'promise'


numBytes += b.size;
numMsgs += 1;
}
alert('Received ' + numMsgs + ' messages, containing ' + numBytes + '
bytes.');
ws.close();
}

of course, this example make use of a particular message paradigm
(send/recv pairs); while this may capture only a subset of interchange
patterns, one could easily generalize the above to provide more flexibility;

Re: Lazy Blob

[Jonas Sicking]

On Wed, Aug 1, 2012 at 7:59 AM, Robin Berjon  wrote:
> Hi all,
>
> with the likes of postMessage and Web Intents that we are getting access to 
> now, it is increasingly common that data may flow from a server to an 
> in-browser page, that may then pass that data on to another in-browser page 
> (typically running at a different origin). In a many cases, such data will be 
> captured as Blobs. There can be several reasons why one may wish to toss a 
> Blob over postMessage rather than a URL:
>
> • Origin restrictions and lack of CORS support (not necessarily fixable 
> by the hacker working on the site);
> • CSP on the receiving end;
> • Cookies or authorisation issues;
> • Hotlinking prevention;
> • Secrecy of the actual URL to the data (possibly for privacy reasons);
> • Probably a bunch of others I haven't thought of.
>
> The problem is that the way that things currently work, the sending end needs 
> to load the data in full before transmitting it. That's okay when it's small 
> and there are only few resources, or if you know that the receiving end will 
> read it all right away anyway. But there are cases in which there is a lot of 
> data (e.g. an image gallery) or where the receiving end might filter the 
> data, only load it item per item, etc.
>
> This is something that we noted while working on implementing some Web 
> Intents based services but that applies more broadly. The most acute example 
> was Jungkee Song's Pick Image Intent that could easily find itself in a 
> position to load several hundreds of megabytes that might then just be thrown 
> away by the receiving end.
>
> The proposals below were hashed out with Jungkee Song and Richard Tibbett, 
> based on the above use case as well as several people in the Intents and DAP 
> groups stating that they bumped into situations where they would have needed 
> this (feel free to send additional use cases folks).
>
>
> POTENTIAL SOLUTIONS
> ===
>
> The overall idea is simple irrespective of which approach is chosen: create a 
> Blob that can lazily fetch the content of a URL. Blobs are designed to be 
> transparently lazy if they need to, so nothing needs to change at the Blob 
> level, or for that matter at the level of anything that may end up reading 
> from one. In fact, a good implementation that reads file-based Blobs is 
> probably already lazy. A Lazy Blob is just a Blob.
>
> Things start to get messier (unless we've missed an option) when you plug 
> that into URLs (don't they always). That's where the multiple approaches 
> outlined below kick in.
>
> In all approaches it is assumed that if there is information to be inferred 
> from context (e.g. access to cookies) then the relevant context is the one in 
> which the Blob is created. Reading of the Blob performed in another context 
> changes nothing to that.
>
> User agents are always allowed to access the actual data at any arbitrary 
> moment, between immediately and when code requests it. Of course, quality of 
> implementation may dictate a strategy that doesn't make it just as bad as 
> loading everything immediately.
>
> Also, none of the options below are detailed anywhere near where they should 
> be — but if we agree on the need and on a general direction I'll be happy to 
> take care of that. If there's agreement on this, I volunteer as editor 
> (Jungkee and perhaps Richard may be interested as well).
>
>
> === The Simple Approach
>
> partial interface BlobBuilder {
> Blob getBlobFromURL (DOMString url);
> };
>
> Usage:
> var bb = new BlobBuilder()
> ,   blob = bb.getBlobFromURL("http://specifiction.com/kitten.png";);
>
> This is the simplest possible approach. When called, essentially the 
> equivalent of this happens:
>
> var xhr = new XMLHttpRequest();
> xhr.open("GET", url, true);
> xhr.responseType = "blob";
>
> And upon reading the blob, send() is triggered and everything else happens as 
> if the XHR blob from xhr.response were accessed directly.
>
> Pro: Extremely simple.
> Con: Does not allow much control over the request that gets made, notably 
> some uses will likely require setting headers.
>
>
>
> === The Somewhat Less Simple Approach
>
> partial interface BlobBuilder {
> Blob getBlobFromURL (DOMString url, optional DOMString method, optional 
> Object headers);
> };
>
> Usage:
> var bb = new BlobBuilder()
> ,   blob = bb.getBlobFromURL("http://specifiction.com/kitten.png";, "GET", { 
> Authorization: "Basic DEADBEEF" });
>
> Everything is the same as the previous version but the method and some 
> headers can be set by enumerating the Object. I *think* that those are all 
> that would ever be needed.
>
> This is currently my preferred approach. If you like it too, you may consider 
> the following ones as merely curiosities.
>
>
> === Using XHR For Options
>
> partial interface BlobBuilder {
> Blob getBlobFromURL (XMLHttpRequest xhr);
> };
>
> Usage:
> var bb = new BlobBuilder()
> ,   xhr = new XMLHttpR

Re: Lazy Blob

[Ian Hickson]

On Wed, 1 Aug 2012, Glenn Adams wrote:
> 
> Of course, implementers are free to ignore whatever they want, but last 
> time I checked, the W3C was a consensus based standards organization 
> which means agreement needs to be reached on what specs go out the door 
> and what are in those specs.

Doesn't really matter what's in the specs going "out the door" if it's not 
what's implemented...


I don't really care about the XHR side of this (happy to let Anne figure 
that out), but since WebSockets was mentioned: what's the use case that 
involves Web Socket? I don't really understand what problem we're trying 
to solve here.

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: Lazy Blob

[Glenn Adams]

On Wed, Aug 1, 2012 at 2:13 PM, Glenn Adams  wrote:

>
> On Wed, Aug 1, 2012 at 2:04 PM, Glenn Maynard  wrote:
>
>> Can we please stop saying "lazy blob"?  It's a confused and confusing
>> phrase.  Blobs are "lazy" by design.
>>
>> On Wed, Aug 1, 2012 at 2:26 PM, Glenn Adams  wrote:
>>
>>> So? Why should lazy blob be specific to HTTP specific semantics when an
>>> arbitrary URL is not specific to HTTP?
>>>
>>
>> XHR is no more specific to HTTP than it is to XML.  It serves as the
>> primary JavaScript API for performing generic network fetches.  WebSockets
>> has an entirely different API from blobs, and bringing them up is only
>> derailing the thread.
>>
>
> The subject line says Lazy Blob, not Lazy Blob and XHR. For the record, I
> will object to a LazyBlob solution that is tied solely to XHR, so deal with
> it now rather than later.
>

Just to make it clear, I support the idea of defining a "lazy blob"
mechanism. However, I am not satisfied that a solution that is tied solely
to XHR is sufficient. I would like to see a mechanism that supports both
XHR and WS [and others?]. Despite the repeated claims of Florian and GlennM
that it doesn't make sense, etc., I think it does make sense and can be
reasonably (and simply) defined to handle such use cases. If necessary I
can volunteer a strawman to that end. However, I would prefer that DAR or
other proposers take the time to consider this use case and factor it into
their proposals.

Re: Lazy Blob

[Boris Zbarsky]

On 8/1/12 9:50 PM, Glenn Maynard wrote:

(if the broken HEAD cases are detectable, anyway)


I wish.

Some of them are, but a lot are not.  We used to use HEAD for things 
like "save link as" in Firefox (for prompting the user for a filename 
and such), but had to stop because it was broken so often in totally 
undetectable ways.  :(


-Boris

Re: Lazy Blob

[Glenn Adams]

On Wed, Aug 1, 2012 at 9:35 PM, Glenn Maynard  wrote:

> On Wed, Aug 1, 2012 at 9:54 PM, Glenn Adams  wrote:
>
>> I don't particularly care if a default behavior for WS is provided that
>> buffers the entire read stream.
>>
>
> Sorry, but that doesn't make sense.  You don't access a message-based
> protocol (Web Sockets) using a character-based API (Blob).  They're utterly
> different APIs.
>

Have you read the Blob interface spec? To quote:

This interface represents *immutable* raw data. It provides a method to
slice  data objects
between ranges of bytes into further chunks of raw data.

The last time I checked, bytes are bytes, not characters. The fact that the
interface provides access to those bytes via a particular string encoding
is irrelevant.



>
> I'll leave the details of defining this to the proposers of lazy blob.
>>
>
> You're free to come up with your own proposal, of course, and editors and
> vendors will choose among them (or come up with something else, or reject
> the idea entirely) as they always do, but others are not obligated to twist
> their proposals to your demands.
>

Of course, implementers are free to ignore whatever they want, but last
time I checked, the W3C was a consensus based standards organization which
means agreement needs to be reached on what specs go out the door and what
are in those specs. Since this is a W3C ML and not an implementers' forum,
then I will continue to assume that the W3C process applies.

There is a fixed obligation for editors and WG to address comments. They
can't simply be rejected because they require work on the part of the
editors or proposers.

Re: Lazy Blob

[Glenn Maynard]

On Wed, Aug 1, 2012 at 9:54 PM, Glenn Adams  wrote:

> I don't particularly care if a default behavior for WS is provided that
> buffers the entire read stream.
>

Sorry, but that doesn't make sense.  You don't access a message-based
protocol (Web Sockets) using a character-based API (Blob).  They're utterly
different APIs.

I'll leave the details of defining this to the proposers of lazy blob.
>

You're free to come up with your own proposal, of course, and editors and
vendors will choose among them (or come up with something else, or reject
the idea entirely) as they always do, but others are not obligated to twist
their proposals to your demands.

-- 
Glenn Maynard

Re: Lazy Blob

[Glenn Adams]

On Wed, Aug 1, 2012 at 2:35 PM, Florian Bösch  wrote:

> On Wed, Aug 1, 2012 at 10:13 PM, Glenn Adams  wrote:
>
>> The subject line says Lazy Blob, not Lazy Blob and XHR. For the record, I
>> will object to a LazyBlob solution that is tied solely to XHR, so deal with
>> it now rather than later.
>>
>
> Then you better get onto specifying a resource/range transfer protocol on
> top of websockets alongside with web-server modules/extensions to be able
> to understand that protocol, because other than that there is no way that
> you'll get what you want.
>

I don't think so. There is nothing about Blob that would require a data
source to implement range access. Blob.slice() does not require the
underlying source to provide range access. The source could be read in
entirety and buffered by a Blob instance.

If a reasonable WS enabled mechanism were defined for a "Lazy Blob" that
permitted an application injected range access, then that could be used to
perform actual range access. There is no need for WS/WSP to support those
semantics directly.

I don't particularly care if a default behavior for WS is provided that
buffers the entire read stream. It's fine to mandate that an application
defined function implement those semantics on a WS instance.

My concern is that use of WS be recognized as a legitimate source for
filling a lazy blob, and that an author should have an option to use WS,
depending on app injected code as needed, instead of mandating XHR for this
purpose. I'll leave the details of defining this to the proposers of lazy
blob.

Re: Lazy Blob

[Glenn Maynard]

On Wed, Aug 1, 2012 at 8:16 PM, Boris Zbarsky  wrote:

> Unless the server is one of the common ones with broken HEAD handling. Or
> unless the resource is served with Content-Encoding:gzip, in which case
> your Content-Range is all sorta broken.  :(


Those could fall back on fetching the whole resource, too, so these issues
would still be transparent to scripts (if the broken HEAD cases are
detectable, anyway).  gzip/deflated content could still work (without
fetching the entire resource) in the common case of reading a Blob
sequentially.  It'd only need to fall back on retrieving the entire
resource if you're doing nonsequential reads.

It's not ideal, but it doesn't seem like such a big problem that it'd make
the feature not worth it.

It still doesn't help the "creating lots of resources without making lots
of network requests" use case, though.

-- 
Glenn Maynard

Re: Lazy Blob

[Boris Zbarsky]

On 8/1/12 11:44 AM, Glenn Maynard wrote:

Ideally, a new responseType could be added to XHR which operates like
"blob", except instead of reading the whole resource, it just performs a
HEAD to retrieve the response length and immediately returns the Blob,
which can be read to perform further Content-Range reads.


Unless the server is one of the common ones with broken HEAD handling. 
Or unless the resource is served with Content-Encoding:gzip, in which 
case your Content-Range is all sorta broken.  :(



The trouble is chunked responses, where the length of the resource isn't
known in advance, since Blobs have a static length and aren't designed
for streaming.


That's another trouble, yes.

-Boris

Re: Lazy Blob

[Glenn Adams]

On Wed, Aug 1, 2012 at 2:04 PM, Glenn Maynard  wrote:

> Can we please stop saying "lazy blob"?  It's a confused and confusing
> phrase.  Blobs are "lazy" by design.
>
> On Wed, Aug 1, 2012 at 2:26 PM, Glenn Adams  wrote:
>
>> So? Why should lazy blob be specific to HTTP specific semantics when an
>> arbitrary URL is not specific to HTTP?
>>
>
> XHR is no more specific to HTTP than it is to XML.  It serves as the
> primary JavaScript API for performing generic network fetches.  WebSockets
> has an entirely different API from blobs, and bringing them up is only
> derailing the thread.
>

The subject line says Lazy Blob, not Lazy Blob and XHR. For the record, I
will object to a LazyBlob solution that is tied solely to XHR, so deal with
it now rather than later.

Re: Lazy Blob

[Glenn Adams]

On Wed, Aug 1, 2012 at 1:54 PM, Florian Bösch  wrote:

> On Wed, Aug 1, 2012 at 9:50 PM, Glenn Adams  wrote:
>
>> Further, a default behavior in the absence of such an injection might be
>> defined simply to read data from the WS and stuff into the blob.
>>
> Which kind of defeats the purpose because you wanted to read ranges, so
> not a whole resource has to be transferred, and you can already read binary
> data from websockets if you wish to do that, without having to invent
> another blob.


A default behavior does not have to handle all uses cases. App specific
code injection could handle this if the author wished it, provided a
mechanism supported it.

Re: Lazy Blob

[Glenn Maynard]

Can we please stop saying "lazy blob"?  It's a confused and confusing
phrase.  Blobs are "lazy" by design.

On Wed, Aug 1, 2012 at 2:26 PM, Glenn Adams  wrote:

> So? Why should lazy blob be specific to HTTP specific semantics when an
> arbitrary URL is not specific to HTTP?
>

XHR is no more specific to HTTP than it is to XML.  It serves as the
primary JavaScript API for performing generic network fetches.  WebSockets
has an entirely different API from blobs, and bringing them up is only
derailing the thread.

On Wed, Aug 1, 2012 at 2:32 PM, Michael Nordman  wrote:

> Maybe another XHR based way to phrase this: define a new response type for
> XHR that results in the construction of a lazyBlob. I guess that's similar
> to the more explicit xhr.makeLazyBlob() method, but allows its construction
> to be async, and for greater reuse of the same signaling for progress and
> errors, and to capture the results of a POST as a lazyBlob as well.
>
> Knowing the length and to some degree the content-type upfront is trouble.
>

I like the idea of being able to retrieve a Blob for a URL via XHR, but I'm
uneasy about exposing the transfer-encoding (eg. whether Content-Length is
known in advance or not).  It's very confusing to move your code between
servers and to have things mysteriously break because the server uses a
different Transfer-Encoding.  I think web APIs should avoid exposing this
at all.

A middle-ground, if not an ideal one, would be for the Blob to return after
the HEAD request if Content-Length is present.  However, if it's absent,
perform the complete fetch before returning the Blob.  That would at least
abstract away the HTTP specifics, so code wouldn't break.

That said, this approach (with or without "greedy downloading for
chunked") may not quite fit some of the OP's use cases.  In particular, if
you have 1000 URLs and you want to hand Blobs off to another page in this
way, then while this wouldn't retrieve them all, it would still result in
1000 HEAD requests.

Sending XHR itself between contexts seems like it would be prickly from a
security standpoint.  A possible solution would be a higher-level wrapper,
eg. a thin abstraction of XHR that can only send a request and receive the
result (eg. exposing a very small subset of XMLHttpRequest's interface).
 That's much more heavyweight than the Blob approach, though, and I don't
have an opinion on whether the use cases justify it.

-- 
Glenn Maynard

Re: Lazy Blob

[Glenn Adams]

On Wed, Aug 1, 2012 at 1:47 PM, Glenn Adams  wrote:

>
> On Wed, Aug 1, 2012 at 1:36 PM, Florian Bösch  wrote:
>
>>  On Wed, Aug 1, 2012 at 9:26 PM, Glenn Adams  wrote:
>>
>>> So? Why should lazy blob be specific to HTTP specific semantics when an
>>> arbitrary URL is not specific to HTTP?
>>>
>>
>> So if you want to have a lazy reader on Websockets you have either:
>> 1) respecify the websocket protocol to include content semantics for
>> accessing resources defined by an URL and having a specified size OR
>> 2) define an additional protocol on top of websockets, which websockets
>> know nothing about, that allows a custom implementation at the server side
>> to respond in a meaningful fashion to resource range requests.
>>
>
> OR define a mechanism for LazyBlob that permits the injection of app
> specific code into the underlying LazyBlob reader loop.
>

Further, a default behavior in the absence of such an injection might be
defined simply to read data from the WS and stuff into the blob.

Re: Lazy Blob

[Glenn Adams]

On Wed, Aug 1, 2012 at 1:36 PM, Florian Bösch  wrote:

> On Wed, Aug 1, 2012 at 9:26 PM, Glenn Adams  wrote:
>
>> So? Why should lazy blob be specific to HTTP specific semantics when an
>> arbitrary URL is not specific to HTTP?
>>
>
> So if you want to have a lazy reader on Websockets you have either:
> 1) respecify the websocket protocol to include content semantics for
> accessing resources defined by an URL and having a specified size OR
> 2) define an additional protocol on top of websockets, which websockets
> know nothing about, that allows a custom implementation at the server side
> to respond in a meaningful fashion to resource range requests.
>

OR define a mechanism for LazyBlob that permits the injection of app
specific code into the underlying LazyBlob reader loop.

Re: Lazy Blob

[Michael Nordman]

Maybe another XHR based way to phrase this: define a new response type for
XHR that results in the construction of a lazyBlob. I guess that's similar
to the more explicit xhr.makeLazyBlob() method, but allows its construction
to be async, and for greater reuse of the same signaling for progress and
errors, and to capture the results of a POST as a lazyBlob as well.

Knowing the length and to some degree the content-type upfront is trouble.
An xhr.responseType of 'stream' would have some trouble with the
content-type too.


On Wed, Aug 1, 2012 at 10:57 AM, Charles Pritchard  wrote:

> On Aug 1, 2012, at 8:44 AM, Glenn Maynard  wrote:
>
> On Wed, Aug 1, 2012 at 9:59 AM, Robin Berjon  wrote:
>
>> var bb = new BlobBuilder()
>
> ,   blob = bb.getBlobFromURL("http://specifiction.com/kitten.png";, "GET",
>> { Authorization: "Basic DEADBEEF" });
>>
>> Everything is the same as the previous version but the method and some
>> headers can be set by enumerating the Object. I *think* that those are all
>> that would ever be needed.
>>
>
> We already have an API to allow scripts to make network requests: XHR.
>  Please don't create a new API that will end up duplicating all of that.
>  However this might be done, it should hang off of XHR.
>
> Ideally, a new responseType could be added to XHR which operates like
> "blob", except instead of reading the whole resource, it just performs a
> HEAD to retrieve the response length and immediately returns the Blob,
> which can be read to perform further Content-Range reads.  This sits
> relatively cleanly within the XHR API.  It also has nice security
> properties, as if you send that Blob somewhere else--possibly to a
> different origin--it can do nothing with the Blob but read it as it was
> given.
>
> The trouble is chunked responses, where the length of the resource isn't
> known in advance, since Blobs have a static length and aren't designed for
> streaming.
>
>
>
> The "HEAD" request seems a little like a FileEntry for use with FileReader.
>
> Something similar (very similar) is done for http+fuse on several
> platforms.
>
> It's quite useful for working with large archives such as PDF or ZIP.
>
>
> -Charles
>

Re: Lazy Blob

[Glenn Adams]

On Wed, Aug 1, 2012 at 12:03 PM, Florian Bösch  wrote:

> On Wed, Aug 1, 2012 at 7:57 PM, Glenn Adams  wrote:
>
>> blob = 
>> bb.getBlobFromURL("ws://specifiction.com/image/kitten.png
>> ")
>>
>
> There is no application layer transfer protocol inherent in websockets.
> Requesting a resource does not have any inherent meaning other than that
> you are opening a "channel" onto /image/kitten.png. Whoever receives that
> request is free to respond to that however he likes. You would need to
> introduce an application layer content protocol on top of websockets, and
> introduce a default websocket server framework capable of understanding
> such content requests. You're not just extending lazy reading to
> websockets. You're putting the burden on yourself to also specify a
> completely new standard application layer protocol for transfer and range
> and acquisition of resources over websocket channels.
>

So? Why should lazy blob be specific to HTTP specific semantics when an
arbitrary URL is not specific to HTTP?

Re: Lazy Blob

[Glenn Adams]

On Wed, Aug 1, 2012 at 11:13 AM, Florian Bösch  wrote:

> On Wed, Aug 1, 2012 at 6:51 PM, Glenn Adams  wrote:
>
>> I'm questioning defining a "LazyBlob" that is solely usable with XHR. It
>> would be better to have a more generic version IMO.
>>
> Websockets have no content semantics, therefore any lazy content
> negotiating reader cannot deal with websockets unless an additional
> application layer protocol and implementation on the server side is
> introduced, something that does not exist for websockets otherwise. You
> could for instance implement HTTP over websockets to get the content
> semantics, and if your server gets a websocket request, it could be proxied
> to a domain socket which happend to have a webserver listening which would
> understand the HTTP request and deliver the resource/range.
>
> Now instead of implementing HTTP over websockets over HTTP over sockets,
> you could just use XHRs which implement HTTP over sockets. Which is why
> generalising lazy readers to websockets does not make sense.
>

Given the "Simple approach" suggested by DAR:

partial interface BlobBuilder {
> Blob getBlobFromURL (DOMString url);
> };
> Usage:
> var bb = new BlobBuilder()
> ,   blob = bb.getBlobFromURL("http://specifiction.com/kitten.png";);


I don't see why the following isn't feasible:

blob = 
bb.getBlobFromURL("ws://specifiction.com/image/kitten.png
")

Or, given the "Using XHR for Options" approach:

partial interface BlobBuilder {
> Blob getBlobFromURL (XMLHttpRequest xhr);
> };
> Usage:
> var bb = new BlobBuilder()
> ,   xhr = new XMLHttpRequest();
> xhr.open("GET", "/kitten.png", true);
> xhr.setRequestHeader("Authorization", "Basic DEADBEEF");
> var blob = bb.getBlobFromURL(xhr);


why one couldn't have:

partial interface BlobBuilder {
Blob getBlobFromURL (WebSocket ws);
};

var bb = new BlobBuilder()
,   ws = new 
WebSocket("ws://specifiction.com/image
");
ws.onopen = function(){ws.send("kitten.png");}
var blob = bb.getBlobFromURL(ws);

Re: Lazy Blob

[Charles Pritchard]

On Aug 1, 2012, at 8:44 AM, Glenn Maynard  wrote:

> On Wed, Aug 1, 2012 at 9:59 AM, Robin Berjon  wrote:
> var bb = new BlobBuilder()
> ,   blob = bb.getBlobFromURL("http://specifiction.com/kitten.png";, "GET", { 
> Authorization: "Basic DEADBEEF" });
> 
> Everything is the same as the previous version but the method and some 
> headers can be set by enumerating the Object. I *think* that those are all 
> that would ever be needed.
> 
> We already have an API to allow scripts to make network requests: XHR.  
> Please don't create a new API that will end up duplicating all of that.  
> However this might be done, it should hang off of XHR.
> 
> Ideally, a new responseType could be added to XHR which operates like "blob", 
> except instead of reading the whole resource, it just performs a HEAD to 
> retrieve the response length and immediately returns the Blob, which can be 
> read to perform further Content-Range reads.  This sits relatively cleanly 
> within the XHR API.  It also has nice security properties, as if you send 
> that Blob somewhere else--possibly to a different origin--it can do nothing 
> with the Blob but read it as it was given.
> 
> The trouble is chunked responses, where the length of the resource isn't 
> known in advance, since Blobs have a static length and aren't designed for 
> streaming.


The "HEAD" request seems a little like a FileEntry for use with FileReader.

Something similar (very similar) is done for http+fuse on several platforms.

It's quite useful for working with large archives such as PDF or ZIP.


-Charles

Re: [whatwg] allowfullscreen vs sandbox="allow-fullscreen", and mimicking for pointer lock

[Vincent Scheib]

Thank you Adam, ROC, Anne. I commented on issue.


On Wed, Aug 1, 2012 at 2:05 AM, Anne van Kesteren  wrote:

> On Wed, Aug 1, 2012 at 11:04 AM, Anne van Kesteren 
> wrote:
> > https://www.w3.org/Bugs/Public/show_bug.cgi?id=17838
> >
> > I did not want to define new HTML features in a separate draft.
>
> So Vincent, if you want to argue for a different design, that bug
> would be the place I think. I do not really care either way, I just
> went with what roc came up with.
>
>
> --
> http://annevankesteren.nl/
>

Re: Lazy Blob

[Glenn Adams]

On Wed, Aug 1, 2012 at 10:46 AM, Florian Bösch  wrote:

> On Wed, Aug 1, 2012 at 6:40 PM, Glenn Adams  wrote:
>
>> Why restrict to XHR? How about WebSocket as data source?
>>
> Websockets support array buffers and therefore by extension any blob/file
> object. However as a stream oriented API websockets have no content
> aquisition, negotation, range and transfer semantics unless you prop those
> up by yourself as an application layer protocol.


I'm questioning defining a "LazyBlob" that is solely usable with XHR. It
would be better to have a more generic version IMO.

Re: Lazy Blob

[Glenn Adams]

On Wed, Aug 1, 2012 at 9:44 AM, Glenn Maynard  wrote:

> On Wed, Aug 1, 2012 at 9:59 AM, Robin Berjon  wrote:
>
>> var bb = new BlobBuilder()
>
> ,   blob = bb.getBlobFromURL("http://specifiction.com/kitten.png";, "GET",
>> { Authorization: "Basic DEADBEEF" });
>>
>> Everything is the same as the previous version but the method and some
>> headers can be set by enumerating the Object. I *think* that those are all
>> that would ever be needed.
>>
>
> We already have an API to allow scripts to make network requests: XHR.
>  Please don't create a new API that will end up duplicating all of that.
>  However this might be done, it should hang off of XHR.
>

Why restrict to XHR? How about WebSocket as data source?

Re: Lazy Blob

[Glenn Maynard]

On Wed, Aug 1, 2012 at 9:59 AM, Robin Berjon  wrote:

> var bb = new BlobBuilder()

,   blob = bb.getBlobFromURL("http://specifiction.com/kitten.png";, "GET", {
> Authorization: "Basic DEADBEEF" });
>
> Everything is the same as the previous version but the method and some
> headers can be set by enumerating the Object. I *think* that those are all
> that would ever be needed.
>

We already have an API to allow scripts to make network requests: XHR.
 Please don't create a new API that will end up duplicating all of that.
 However this might be done, it should hang off of XHR.

Ideally, a new responseType could be added to XHR which operates like
"blob", except instead of reading the whole resource, it just performs a
HEAD to retrieve the response length and immediately returns the Blob,
which can be read to perform further Content-Range reads.  This sits
relatively cleanly within the XHR API.  It also has nice security
properties, as if you send that Blob somewhere else--possibly to a
different origin--it can do nothing with the Blob but read it as it was
given.

The trouble is chunked responses, where the length of the resource isn't
known in advance, since Blobs have a static length and aren't designed for
streaming.

-- 
Glenn Maynard

Lazy Blob

[Robin Berjon]

Hi all,

with the likes of postMessage and Web Intents that we are getting access to 
now, it is increasingly common that data may flow from a server to an 
in-browser page, that may then pass that data on to another in-browser page 
(typically running at a different origin). In a many cases, such data will be 
captured as Blobs. There can be several reasons why one may wish to toss a Blob 
over postMessage rather than a URL:

• Origin restrictions and lack of CORS support (not necessarily fixable by 
the hacker working on the site);
• CSP on the receiving end;
• Cookies or authorisation issues;
• Hotlinking prevention;
• Secrecy of the actual URL to the data (possibly for privacy reasons);
• Probably a bunch of others I haven't thought of.

The problem is that the way that things currently work, the sending end needs 
to load the data in full before transmitting it. That's okay when it's small 
and there are only few resources, or if you know that the receiving end will 
read it all right away anyway. But there are cases in which there is a lot of 
data (e.g. an image gallery) or where the receiving end might filter the data, 
only load it item per item, etc.

This is something that we noted while working on implementing some Web Intents 
based services but that applies more broadly. The most acute example was 
Jungkee Song's Pick Image Intent that could easily find itself in a position to 
load several hundreds of megabytes that might then just be thrown away by the 
receiving end.

The proposals below were hashed out with Jungkee Song and Richard Tibbett, 
based on the above use case as well as several people in the Intents and DAP 
groups stating that they bumped into situations where they would have needed 
this (feel free to send additional use cases folks).


POTENTIAL SOLUTIONS
===

The overall idea is simple irrespective of which approach is chosen: create a 
Blob that can lazily fetch the content of a URL. Blobs are designed to be 
transparently lazy if they need to, so nothing needs to change at the Blob 
level, or for that matter at the level of anything that may end up reading from 
one. In fact, a good implementation that reads file-based Blobs is probably 
already lazy. A Lazy Blob is just a Blob.

Things start to get messier (unless we've missed an option) when you plug that 
into URLs (don't they always). That's where the multiple approaches outlined 
below kick in.

In all approaches it is assumed that if there is information to be inferred 
from context (e.g. access to cookies) then the relevant context is the one in 
which the Blob is created. Reading of the Blob performed in another context 
changes nothing to that.

User agents are always allowed to access the actual data at any arbitrary 
moment, between immediately and when code requests it. Of course, quality of 
implementation may dictate a strategy that doesn't make it just as bad as 
loading everything immediately.

Also, none of the options below are detailed anywhere near where they should be 
— but if we agree on the need and on a general direction I'll be happy to take 
care of that. If there's agreement on this, I volunteer as editor (Jungkee and 
perhaps Richard may be interested as well).


=== The Simple Approach

partial interface BlobBuilder {
Blob getBlobFromURL (DOMString url);
};

Usage:
var bb = new BlobBuilder()
,   blob = bb.getBlobFromURL("http://specifiction.com/kitten.png";);

This is the simplest possible approach. When called, essentially the equivalent 
of this happens:

var xhr = new XMLHttpRequest();
xhr.open("GET", url, true);
xhr.responseType = "blob";

And upon reading the blob, send() is triggered and everything else happens as 
if the XHR blob from xhr.response were accessed directly.

Pro: Extremely simple.
Con: Does not allow much control over the request that gets made, notably some 
uses will likely require setting headers.



=== The Somewhat Less Simple Approach

partial interface BlobBuilder {
Blob getBlobFromURL (DOMString url, optional DOMString method, optional 
Object headers);
};

Usage:
var bb = new BlobBuilder()
,   blob = bb.getBlobFromURL("http://specifiction.com/kitten.png";, "GET", { 
Authorization: "Basic DEADBEEF" });

Everything is the same as the previous version but the method and some headers 
can be set by enumerating the Object. I *think* that those are all that would 
ever be needed.

This is currently my preferred approach. If you like it too, you may consider 
the following ones as merely curiosities.


=== Using XHR For Options

partial interface BlobBuilder {
Blob getBlobFromURL (XMLHttpRequest xhr);
};

Usage:
var bb = new BlobBuilder()
,   xhr = new XMLHttpRequest();
xhr.open("GET", "/kitten.png", true);
xhr.setRequestHeader("Authorization", "Basic DEADBEEF");
var blob = bb.getBlobFromURL(xhr);

This avails the developer the full flexibility and power of XHR, and uses the 
configured XHR object to make the request (w

Re: [whatwg] allowfullscreen vs sandbox="allow-fullscreen", and mimicking for pointer lock

[Anne van Kesteren]

On Wed, Aug 1, 2012 at 11:04 AM, Anne van Kesteren  wrote:
> https://www.w3.org/Bugs/Public/show_bug.cgi?id=17838
>
> I did not want to define new HTML features in a separate draft.

So Vincent, if you want to argue for a different design, that bug
would be the place I think. I do not really care either way, I just
went with what roc came up with.


-- 
http://annevankesteren.nl/

Re: [whatwg] allowfullscreen vs sandbox="allow-fullscreen", and mimicking for pointer lock

[Anne van Kesteren]

On Wed, Aug 1, 2012 at 12:33 AM, Adam Barth  wrote:
> Main frame: a.html
>   -> 
> -> 
>
> Can c.html go full screen?  Where is that specified?

https://www.w3.org/Bugs/Public/show_bug.cgi?id=17838

I did not want to define new HTML features in a separate draft.


-- 
http://annevankesteren.nl/