On Mon, 02 Jun 2014 00:18:18 +0000 David Soria Parra via Digitalmars-d <[email protected]> wrote:
> Hi, > > I have recently had to deal with large amounts of JSON data in D. > While doing that I've found that std.json is remarkable slow in > comparison to other languages standard json implementation. I've > create a small and simple benchmark parsing a local copy of a > github API call > "https://api.github.com/repos/D-Programming-Language/dmd/pulls"; > and parsing it 100% times and writing the title to stdout. > > My results as follows: > ./d-test > /dev/null 3.54s user 0.02s system 99% cpu 3.560 > total > ./hs-test > /dev/null 0.02s user 0.00s system 93% cpu 0.023 > total > python test.py > /dev/null 0.77s user 0.02s system 99% cpu > 0.792 total > > The concrete implementations (sorry for my terrible haskell > implementation) can be found here: > > https://github.com/dsp/D-Json-Tests/ > > This is comapring D's std.json vs Haskells Data.Aeson and python > standard library json. I am a bit concerned with the current > state of our JSON parser given that a lot of applications these > day use JSON. I personally consider a high speed implementation > of JSON a critical part of a standard library. > > Would it make sense to start thinking about using ujson4c as an > external library, or maybe come up with a better implementation. > I know Orvid has something and might add some analysis as to why > std.json is slow. Any ideas or pointers as to how to start with > that? It's my understanding that the current design of std.json is considered to be poor, but I don't haven't used it, so I don't know any the details. But if it's as slow as you're finding to be the case, then I think that that supports the idea that it needs a redesign. The question then is what a new std.json should look like and who would do it. And that pretty much comes down to an interested and motivated developer coming up with and implementing a new design and then proposing it here. And until someone takes up that torch, we'll be stuck with what we have. Certainly, there's no fundamental reason why we can't have a lightening fast std.json. With ranges and slices, parsing in D in general should be faster than C/C++ (and definitely faster than Haskell of python), and if it isn't, that indicates that the implementation (if not the whole design) of that code needs to be redone. I know that vibe.d uses its own json implementation, but I don't know how much of that is part of its public API and how much of that is simply used internally: http://vibed.org - Jonathan M Davis
