Evan, yes! Luis linked your blog post earlier and it was really helpful. The other advantage of json4s-jackson is that the interface is mostly compatible with lift-json. I made the (few and trivial) changes necessary to get everything in Spark switched over earlier today and will write it up in a PR for further discussion later tonight or tomorrow.
best, wb ----- Original Message ----- > From: "Evan Chan" <e...@ooyala.com> > To: dev@spark.incubator.apache.org > Sent: Monday, February 10, 2014 6:31:55 PM > Subject: Re: proposal: replace lift-json with spray-json > > By the way, I did a benchmark on JSON parsing performance recently. > Based on that, spray-json was about 10x slower than the Jackson-based > parsers. I recommend json4s-jackson, because jackson is almost > certainly already a dependency of Sparks (many other Java libraries > use it), so the dependencies are very lightweight. I didn't > benchmark Argonaut or play-json, partly because play-json pulled in > all the Play dependencies, tho as someone else commented in this > thread, they plan to split it out. > > On Mon, Feb 10, 2014 at 12:22 AM, Pascal Voitot Dev > <pascal.voitot....@gmail.com> wrote: > > Hi again, > > > > If spark just need Json with serialization/deserialization of basic > > structures and some potential simple validations for webUI, let's remind > > that play/json (without any other dependency than Jackson) is about 200-300 > > lines of code... the only dependency is jackson which is the best json > > parser that I know. The rest of code is about typesafety & composition... > > If Spark need some json veryveryvery performant JSON (de)serialization, we > > will have to look on things like pickling and potentially some streaming > > parsers (I think this is a domain under work right now...) > > > > Pascal > > > > > > On Mon, Feb 10, 2014 at 1:50 AM, Will Benton <wi...@redhat.com> wrote: > > > >> Matei, sorry if I was unclear: I'm referring to downstream operating > >> system distributions (like Fedora or Debian) that have policies requiring > >> that all packages are built from source (using only tools already packaged > >> in the distribution). So end-users (and distributions with different > >> policies) don't have to build Lift to get the lift-json artifact, but it > >> is > >> a concern for many open-source communities. > >> > >> best, > >> wb > >> > >> > >> ----- Original Message ----- > >> > From: "Matei Zaharia" <matei.zaha...@gmail.com> > >> > To: dev@spark.incubator.apache.org > >> > Sent: Sunday, February 9, 2014 4:29:20 PM > >> > Subject: Re: proposal: replace lift-json with spray-json > >> > > >> > Will, why are you saying that downstream distributes need to build all > >> > of > >> > Lift to package lift-json? Spark just downloads it from Maven Central, > >> where > >> > it's a JAR with no external dependencies. We don't have any dependency > >> > on > >> > the rest of lift. > >> > > >> > Matei > >> > > >> > On Feb 9, 2014, at 11:28 AM, Will Benton <wi...@redhat.com> wrote: > >> > > >> > > lift-json is a nice library, but Lift is a pretty heavyweight > >> dependency to > >> > > track just for its JSON support. (lift-json is relatively > >> self-contained > >> > > as a dependency from an end-user's perspective, but downstream > >> > > distributors need to build all of Lift in order to package the JSON > >> > > support.) I understand that this has come up before (cf. SPARK-883) > >> and > >> > > that the uncertain future of JSON support in the Scala standard > >> library is > >> > > the motivator for relying on an external library. > >> > > > >> > > I'm proposing replacing lift-json in Spark with something more > >> lightweight. > >> > > I've evaluated apparent project liveness and dependency scope for most > >> of > >> > > the current Scala JSON libraries and believe the best candidate is > >> > > spray-json (https://github.com/spray/spray-json), the JSON library > >> used by > >> > > the Spray HTTP toolkit. spray-json is Apache-licensed, actively > >> developed, > >> > > and builds and works independently of Spray with only one external > >> > > dependency. > >> > > > >> > > It looks to me like a pretty straightforward change (although > >> > > JsonProtocol.scala would be a little more verbose since it couldn't > >> > > use > >> > > the Lift JSON DSL), and I'd like to do it. I'm writing now to ask for > >> > > some community feedback before making the change (and submitting a > >> > > JIRA > >> > > and PR). If no one has any serious objections (to the effort in > >> general > >> > > or to to the choice of spark-json in particular), I'll go ahead and do > >> it, > >> > > but if anyone has concerns, I'd be happy to discuss and address them > >> > > before getting started. > >> > > > >> > > > >> > > thanks, > >> > > wb > >> > > >> > > >> > > > > -- > -- > Evan Chan > Staff Engineer > e...@ooyala.com | >
