[whatwg] asynchronous JSON.parse
right now JSON.parse blocks the mainloop, this gets more and more of an
issue as JSON documents get bigger and are also used as serialization
format to communicate with web workers.
To handle large JSON Documents there is a need for an async JSON.parse,
something like:
JSON.parse(data, function(obj) { ... });
or more like FileReader:
var json = new JSONReader();
json.addEventListener('load', function(event) {
//parsed JSON document in: this.result
});
json.parse(data);
While my major need is asynchronous parsing of JSON data, the same is
also true for serialization into JSON.
var json = new JSONWriter();
json.addEventListener('load', function(event) {
// serialized JSON string in: this.result
});
json.serialize(obj);
Re: [whatwg] asynchronous JSON.parse
(It's hard to talk to somebody called j, by the way. :) On Thu, Mar 7, 2013 at 2:06 AM, j...@mailb.org wrote: right now JSON.parse blocks the mainloop, this gets more and more of an issue as JSON documents get bigger Just load the data you want to parse inside a worker, and perform the parsing there. Computationally-expensive work is exactly something Web Workers are meant for. and are also used as serialization format to communicate with web workers. There's no need to serialize to JSON before sending data to a worker; there's nothing that JSON can represent that postMessage can't post directly. Just postMessage the object itself. -- Glenn Maynard
Re: [whatwg] asynchronous JSON.parse
The JSON object and its API are part of the ECMAScript language
specification which is standardized by Ecma/TC39, not whatwg.
Rick
On Thursday, March 7, 2013, wrote:
right now JSON.parse blocks the mainloop, this gets more and more of an
issue as JSON documents get bigger and are also used as serialization
format to communicate with web workers.
To handle large JSON Documents there is a need for an async JSON.parse,
something like:
JSON.parse(data, function(obj) { ... });
or more like FileReader:
var json = new JSONReader();
json.addEventListener('load', function(event) {
//parsed JSON document in: this.result
});
json.parse(data);
While my major need is asynchronous parsing of JSON data, the same is
also true for serialization into JSON.
var json = new JSONWriter();
json.addEventListener('load', function(event) {
// serialized JSON string in: this.result
});
json.serialize(obj);
Re: [whatwg] asynchronous JSON.parse
On Thu, Mar 7, 2013 at 9:29 AM, Rick Waldron waldron.r...@gmail.com wrote: The JSON object and its API are part of the ECMAScript language specification which is standardized by Ecma/TC39, not whatwg. He's talking about an async interface to it, not the core parser. It's a higher level of abstraction than the core language, which doesn't know anything about eg. DOM Events and doesn't typically define asynchronous interfaces. If an API like this was to be exposed (which I believe is unnecessary), it would belong here or webapps, not at the language level. -- Glenn Maynard
Re: [whatwg] asynchronous JSON.parse
On Thu, Mar 7, 2013 at 10:42 AM, Glenn Maynard gl...@zewt.org wrote: On Thu, Mar 7, 2013 at 9:29 AM, Rick Waldron waldron.r...@gmail.comwrote: The JSON object and its API are part of the ECMAScript language specification which is standardized by Ecma/TC39, not whatwg. He's talking about an async interface to it, not the core parser. It's a higher level of abstraction than the core language, which doesn't know anything about eg. DOM Events and doesn't typically define asynchronous interfaces. If an API like this was to be exposed (which I believe is unnecessary), it would belong here or webapps, not at the language level. Yes, and as a member of ECMA/TC39 I felt that it was my responsibility to clarify the specification ownership—but thanks for filling me in ;) Rick -- Glenn Maynard
Re: [whatwg] Fetch: Origin header
On Wed, Mar 6, 2013 at 3:21 PM, Anne van Kesteren ann...@annevk.nl wrote: Unless PHP does not expose Origin under HTTP_ORIGIN in $_SERVER as one would expect... (It does btw.) So I also tested the fetch from an origin in the specification http://dump.testsuite.org/fetch/form.html and it turns out that only WebKit exhibits this behavior. Other browsers do not include Origin in a navigation that uses the POST method. Adam, is that something you think we should keep? -- http://annevankesteren.nl/
Re: [whatwg] Fetch: Origin header
On Thu, Mar 7, 2013 at 9:07 AM, Anne van Kesteren ann...@annevk.nl wrote: On Wed, Mar 6, 2013 at 3:21 PM, Anne van Kesteren ann...@annevk.nl wrote: Unless PHP does not expose Origin under HTTP_ORIGIN in $_SERVER as one would expect... (It does btw.) So I also tested the fetch from an origin in the specification http://dump.testsuite.org/fetch/form.html and it turns out that only WebKit exhibits this behavior. Other browsers do not include Origin in a navigation that uses the POST method. Adam, is that something you think we should keep? I don't have strong feelings one way or another. Generally, I think it's a good idea if the presence of the Origin header isn't synonymous with the request being a CORS request because that could limit our ability to use the Origin header in the future. Adam
Re: [whatwg] asynchronous JSON.parse
(Note: New on this list, please be gentle if I'm debating an
inappropriate issue in an inappropriate place.)
Actually, communicating large JSON objects between threads may cause
performance issues. I do not have the means to measure reception speed
simply (which would be used to implement asynchronous JSON.parse), but
it is easy to measure main thread blocks caused by sending (which would
be used to implement asynchronous JSON.stringify).
I have put together a small test here - warning, this may kill your browser:
http://yoric.github.com/Bugzilla-832664/
While there are considerable fluctuations, even inside one browser, on
my system, I witness janks that last 300ms to 3s.
Consequently, I am convinced that we need asynchronous variants of
JSON.{parse, stringify}.
Best regards,
David
Glenn Maynard wrote
(It's hard to talk to somebody called j, by the way. :)
On Thu, Mar 7, 2013 at 2:06 AM, j at mailb.org wrote:
right now JSON.parse blocks the mainloop, this gets more and more of an
issue as JSON documents get bigger
Just load the data you want to parse inside a worker, and perform the
parsing there. Computationally-expensive work is exactly something Web
Workers are meant for.
and are also used as serialization
format to communicate with web workers.
There's no need to serialize to JSON before sending data to a worker;
there's nothing that JSON can represent that postMessage can't post
directly. Just postMessage the object itself.
--
David Rajchenbach-Teller, PhD
Performance Team, Mozilla
Re: [whatwg] asynchronous JSON.parse
I'd like to hear about the use cases a bit more.
Generally, structured data gets bulky because it contains more items, not
because items get bigger.
In which case, isn't part of the solution to paginate your data, and parse
those pages separately?
Even if an async API for JSON existed, wouldn't the perf bottleneck then simply
fall on whatever processing needs to be done afterwards?
Wouldn't some form of event-based API be more indicated? E.g.:
var parser = JSON.parser();
parser.parse(src);
parser.onparse = function(e) {
doSomething(e.data);
};
And wouldn't this be highly dependent on how the data is structured, and thus
very much app-specific?
--tobie
On Thursday, March 7, 2013 at 11:18 PM, David Rajchenbach-Teller wrote:
(Note: New on this list, please be gentle if I'm debating an
inappropriate issue in an inappropriate place.)
Actually, communicating large JSON objects between threads may cause
performance issues. I do not have the means to measure reception speed
simply (which would be used to implement asynchronous JSON.parse), but
it is easy to measure main thread blocks caused by sending (which would
be used to implement asynchronous JSON.stringify).
I have put together a small test here - warning, this may kill your browser:
http://yoric.github.com/Bugzilla-832664/
While there are considerable fluctuations, even inside one browser, on
my system, I witness janks that last 300ms to 3s.
Consequently, I am convinced that we need asynchronous variants of
JSON.{parse, stringify}.
Best regards,
David
Glenn Maynard wrote
(It's hard to talk to somebody called j, by the way. :)
On Thu, Mar 7, 2013 at 2:06 AM, j at mailb.org (http://mailb.org) wrote:
right now JSON.parse blocks the mainloop, this gets more and more of an
issue as JSON documents get bigger
Just load the data you want to parse inside a worker, and perform the
parsing there. Computationally-expensive work is exactly something Web
Workers are meant for.
and are also used as serialization
format to communicate with web workers.
There's no need to serialize to JSON before sending data to a worker;
there's nothing that JSON can represent that postMessage can't post
directly. Just postMessage the object itself.
--
David Rajchenbach-Teller, PhD
Performance Team, Mozilla
Re: [whatwg] asynchronous JSON.parse
On Thu, Mar 7, 2013 at 2:18 PM, David Rajchenbach-Teller
dtel...@mozilla.com wrote:
(Note: New on this list, please be gentle if I'm debating an
inappropriate issue in an inappropriate place.)
Actually, communicating large JSON objects between threads may cause
performance issues. I do not have the means to measure reception speed
simply (which would be used to implement asynchronous JSON.parse), but
it is easy to measure main thread blocks caused by sending (which would
be used to implement asynchronous JSON.stringify).
Isn't this precisely what Transferable objects are for?
http://www.whatwg.org/specs/web-apps/current-work/multipage/common-dom-interfaces.html#transferable-objects
--
Dan Beam
db...@chromium.org
I have put together a small test here - warning, this may kill your
browser:
http://yoric.github.com/Bugzilla-832664/
While there are considerable fluctuations, even inside one browser, on
my system, I witness janks that last 300ms to 3s.
Consequently, I am convinced that we need asynchronous variants of
JSON.{parse, stringify}.
Best regards,
David
Glenn Maynard wrote
(It's hard to talk to somebody called j, by the way. :)
On Thu, Mar 7, 2013 at 2:06 AM, j at mailb.org wrote:
right now JSON.parse blocks the mainloop, this gets more and more of an
issue as JSON documents get bigger
Just load the data you want to parse inside a worker, and perform the
parsing there. Computationally-expensive work is exactly something Web
Workers are meant for.
and are also used as serialization
format to communicate with web workers.
There's no need to serialize to JSON before sending data to a worker;
there's nothing that JSON can represent that postMessage can't post
directly. Just postMessage the object itself.
--
David Rajchenbach-Teller, PhD
Performance Team, Mozilla
Re: [whatwg] asynchronous JSON.parse
It is. However, to use Transferable objects for purpose of implementing asynchronous parse/stringify, one needs conversions of JSON objects from/to Transferable objects. As it turns out, these conversions are just variants on JSON parse/stringify, so we have not simplified the issue. Note that I would be quite satisfied with an efficient, asynchronous, implementation of these [de]serializations of JSON from/to Transferable objects. Best regards, David On Thu Mar 7 23:37:43 2013, Dan Beam wrote: Isn't this precisely what Transferable objects are for? http://www.whatwg.org/specs/web-apps/current-work/multipage/common-dom-interfaces.html#transferable-objects -- David Rajchenbach-Teller, PhD Performance Team, Mozilla
Re: [whatwg] asynchronous JSON.parse
On Thu, Mar 7, 2013 at 4:18 PM, David Rajchenbach-Teller dtel...@mozilla.com wrote: (Note: New on this list, please be gentle if I'm debating an inappropriate issue in an inappropriate place.) (To my understanding of this list, it's completely acceptable to discuss this here.) Actually, communicating large JSON objects between threads may cause performance issues. I do not have the means to measure reception speed simply (which would be used to implement asynchronous JSON.parse), but it is easy to measure main thread blocks caused by sending (which would be used to implement asynchronous JSON.stringify). If you're dealing with lots of data, you should be loading or creating the data in the worker in the first place, not creating it in the UI thread and then shuffling it off to a worker. For example, if you're reading a large file provided by the user, post the File object (received eg. from input) to the worker, then do the heavy lifting there in the first place. Benchmarks are always good, but it'd be better to talk about a real-world use case, since it gives us something concrete to talk about. What's a practical case where you would actually have to create the big object in the UI thread? On Thu, Mar 7, 2013 at 5:25 PM, David Rajchenbach-Teller dtel...@mozilla.com wrote: However, to use Transferable objects for purpose of implementing asynchronous parse/stringify, one needs conversions of JSON objects from/to Transferable objects. As it turns out, these conversions are just variants on JSON parse/stringify, so we have not simplified the issue. (Not nitpicking, since I really wasn't sure what you meant at first, but I think you mean a JavaScript object. There's no such thing as a JSON object.) -- Glenn Maynard
