Marcelo and Christopher, Thanks for your help! The problem turned out to arise from a different part of the code (we have multiple ObjectMappers), but because I am not very familiar with Jackson I had thought there was a problem with the Scala module.
Thank you again, Kevin From: Christopher Currie <[email protected]> Date: Wednesday, September 9, 2015 at 10:17 AM To: Kevin Chen <[email protected]>, "[email protected]" <[email protected]> Cc: Matt Cheah <[email protected]>, Mingyu Kim <[email protected]> Subject: Fwd: Deserializing JSON into Scala objects in Java code Kevin, I'm not a Spark dev, but I maintain the Scala module for Jackson. If you're continuing to have issues with parsing JSON using the Spark Scala datatypes, let me know or chime in on the jackson mailing list ([email protected]) and I'll see what I can do to help. Christopher Currie ---------- Forwarded message ---------- From: Paul Brown <[email protected]> Date: Tue, Sep 8, 2015 at 8:58 PM Subject: Fwd: Deserializing JSON into Scala objects in Java code To: Christopher Currie <[email protected]> Passing along. ---------- Forwarded message ---------- From: Kevin Chen <[email protected]> Date: Tuesday, September 8, 2015 Subject: Deserializing JSON into Scala objects in Java code To: "[email protected]" <[email protected]> Cc: Matt Cheah <[email protected]>, Mingyu Kim <[email protected]> Hello Spark Devs, I am trying to use the new Spark API json endpoints at /api/v1/[path] (added in SPARK-3454). In order to minimize maintenance on our end, I would like to use Retrofit/Jackson to parse the json directly into the Scala classes in org/apache/spark/status/api/v1/api.scala (ApplicationInfo, ApplicationAttemptInfo, etc…). However, Jackson does not seem to know how to handle Scala Seqs, and will throw an error when trying to parse the attempts: Seq[ApplicationAttemptInfo] field of ApplicationInfo. Our codebase is in Java. My questions are: 1. Do you have any recommendations on how to easily deserialize Scala objects from json? For example, do you have any current usage examples of SPARK-3454 with Java? 2. Alternatively, are you committed to the json formats of /api/v1/path? I would guess so, because of the ‘v1’, but wanted to confirm. If so, I could deserialize the json into instances of my own Java classes instead, without worrying about changing the class structure later due to changes in the Spark API. Some further information: * The error I am getting with Jackson when trying to deserialize the json into ApplicationInfo is Caused by: com.fasterxml.jackson.databind.JsonMappingException: Can not construct instance of scala.collection.Seq, problem: abstract types either need to be mapped to concrete types, have custom deserializer, or be instantiated with additional type information * I tried using Jackson’s DefaultScalaModule, which seems to have support for Scala Seqs, but got no luck. * Deserialization works if the Scala class does not have any Seq fields, and works if the fields are Java Lists instead of Seqs. Thanks very much for your help! Kevin Chen -- (Sent from mobile. Pardon brevity.)
smime.p7s
Description: S/MIME cryptographic signature
