lift-json is a nice library, but Lift is a pretty heavyweight dependency to track just for its JSON support. (lift-json is relatively self-contained as a dependency from an end-user's perspective, but downstream distributors need to build all of Lift in order to package the JSON support.) I understand that this has come up before (cf. SPARK-883) and that the uncertain future of JSON support in the Scala standard library is the motivator for relying on an external library.
I'm proposing replacing lift-json in Spark with something more lightweight. I've evaluated apparent project liveness and dependency scope for most of the current Scala JSON libraries and believe the best candidate is spray-json (https://github.com/spray/spray-json), the JSON library used by the Spray HTTP toolkit. spray-json is Apache-licensed, actively developed, and builds and works independently of Spray with only one external dependency. It looks to me like a pretty straightforward change (although JsonProtocol.scala would be a little more verbose since it couldn't use the Lift JSON DSL), and I'd like to do it. I'm writing now to ask for some community feedback before making the change (and submitting a JIRA and PR). If no one has any serious objections (to the effort in general or to to the choice of spark-json in particular), I'll go ahead and do it, but if anyone has concerns, I'd be happy to discuss and address them before getting started. thanks, wb
