On Monday, 2 June 2014 at 00:18:19 UTC, David Soria Parra wrote:
Hi,
I have recently had to deal with large amounts of JSON data in
D. While doing that I've found that std.json is remarkable slow
in comparison to other languages standard json implementation.
I've create a small and simple benchmark parsing a local copy
of a github API call
"https://api.github.com/repos/D-Programming-Language/dmd/pulls";
and parsing it 100% times and writing the title to stdout.
My results as follows:
./d-test > /dev/null 3.54s user 0.02s system 99% cpu 3.560
total
./hs-test > /dev/null 0.02s user 0.00s system 93% cpu 0.023
total
python test.py > /dev/null 0.77s user 0.02s system 99% cpu
0.792 total
The concrete implementations (sorry for my terrible haskell
implementation) can be found here:
https://github.com/dsp/D-Json-Tests/
This is comapring D's std.json vs Haskells Data.Aeson and
python standard library json. I am a bit concerned with the
current state of our JSON parser given that a lot of
applications these day use JSON. I personally consider a high
speed implementation of JSON a critical part of a standard
library.
Would it make sense to start thinking about using ujson4c as an
external library, or maybe come up with a better
implementation. I know Orvid has something and might add some
analysis as to why std.json is slow. Any ideas or pointers as
to how to start with that?
I don't know the status of another D based JSON library.
If you can install yajl library, then yajl-d is an another
candidate.
% time ./yajl_test > /dev/null
./yajl_test > /dev/null 0.42s user 0.01s system 99% cpu 0.434
total
% time python test.py> /dev/null
python test.py > /dev/null 0.65s user 0.02s system 99% cpu 0.671
total
% time ./test > /dev/null
./test > /dev/null 3.10s user 0.02s system 99% cpu 3.125 total
import yajl.yajl, std.datetime, std.file, std.stdio;
void parse() {
foreach(elem; readText("test.json").decode.array) {
writeln(elem.object["title"]);
}
}
int main(string[] args) {
for(uint i = 0; i < 100; i++) {
parse();
}
return 0;
}
http://code.dlang.org/packages/yajl
NOTE: yajl-d doesn't expose yajl's SAX style API unlike Sean's
implementation
