getValuesMap is not very Java friendly. I have to do something like this String[] fieldNames = row.schema().fieldNames(); Seq<String> fieldNamesSeq = JavaConverters.asScalaIteratorConverter(Arrays.asList(fieldNames).iterator()).asScala().toSeq(); String json = row.getValuesMap(fieldNamesSeq).toString();
On Mon, Sep 11, 2017 at 12:39 AM, Riccardo Ferrari <ferra...@gmail.com> wrote: > Hi Ayan, yup that works very well however I believe Kant's other mail > "Queries with streaming sources must be executed with writeStream.start()" > is adding more context. > I think he is trying leverage on structured streaming and applying the rdd > conversion to a streaming dataset is breaking the streaming lineage. > Best, > > On Mon, Sep 11, 2017 at 3:51 AM, ayan guha <guha.a...@gmail.com> wrote: > >> Sorry for side-line question, but for Python, isn't following the easiest: >> >> >>> import json >> >>>* df1 = df.rdd.map(lambda r: json.dumps(r.asDict()))* >> >>> df1.take(10) >> ['{"id": 1}', '{"id": 2}', '{"id": 3}', '{"id": 4}', '{"id": 5}'] >> >> >> >> >> On Mon, Sep 11, 2017 at 4:22 AM, Riccardo Ferrari <ferra...@gmail.com> >> wrote: >> >>> Hi Kant, >>> >>> You can check the getValuesMap >>> <https://spark.apache.org/docs/2.2.0/api/java/org/apache/spark/sql/Row.html#getValuesMap-scala.collection.Seq->. >>> I found this post <https://stackoverflow.com/a/41602178/8356352> >>> useful, it is in Scala but should be a good starting point. >>> An alternative approach is combine the 'struct' and 'to_json' functions >>> <https://spark.apache.org/docs/2.2.0/api/java/org/apache/spark/sql/functions.html>. >>> I have not tested this in Java but I am using it in Python. >>> >>> Best, >>> >>> On Sun, Sep 10, 2017 at 1:45 AM, kant kodali <kanth...@gmail.com> wrote: >>> >>>> toJSON on Row object. >>>> >>>> On Sat, Sep 9, 2017 at 4:18 PM, Felix Cheung <felixcheun...@hotmail.com >>>> > wrote: >>>> >>>>> toJSON on Dataset/DataFrame? >>>>> >>>>> ------------------------------ >>>>> *From:* kant kodali <kanth...@gmail.com> >>>>> *Sent:* Saturday, September 9, 2017 4:15:49 PM >>>>> *To:* user @spark >>>>> *Subject:* How to convert Row to JSON in Java? >>>>> >>>>> Hi All, >>>>> >>>>> How to convert Row to JSON in Java? It would be nice to have .toJson() >>>>> method in the Row class. >>>>> >>>>> Thanks, >>>>> kant >>>>> >>>> >>>> >>> >> >> >> -- >> Best Regards, >> Ayan Guha >> > >