I agree,
Java tend to be pretty verbose unfortunately. You can check the "alternate"
approach that should be more compact and readable.
Should be something like:
df.select(to_json(struct(col("*")).alias("value"))
Of course to_json, struct and col are from the
https://spark.apache.org/docs/2.2.0/api/java/org/apache/spark/sql/functions.html
On Mon, Sep 11, 2017 at 10:15 PM, kant kodali <kanth...@gmail.com> wrote:
> getValuesMap is not very Java friendly. I have to do something like this
>
> String[] fieldNames = row.schema().fieldNames();
> Seq<String> fieldNamesSeq =
> JavaConverters.asScalaIteratorConverter(Arrays.asList(fieldNames).iterator()).asScala().toSeq();
> String json = row.getValuesMap(fieldNamesSeq).toString();
>
>
> On Mon, Sep 11, 2017 at 12:39 AM, Riccardo Ferrari <ferra...@gmail.com>
> wrote:
>
>> Hi Ayan, yup that works very well however I believe Kant's other mail
>> "Queries with streaming sources must be executed with writeStream.start()"
>> is adding more context.
>> I think he is trying leverage on structured streaming and applying the
>> rdd conversion to a streaming dataset is breaking the streaming lineage.
>> Best,
>>
>> On Mon, Sep 11, 2017 at 3:51 AM, ayan guha <guha.a...@gmail.com> wrote:
>>
>>> Sorry for side-line question, but for Python, isn't following the
>>> easiest:
>>>
>>> >>> import json
>>> >>>* df1 = df.rdd.map(lambda r: json.dumps(r.asDict()))*
>>> >>> df1.take(10)
>>> ['{"id": 1}', '{"id": 2}', '{"id": 3}', '{"id": 4}', '{"id": 5}']
>>>
>>>
>>>
>>>
>>> On Mon, Sep 11, 2017 at 4:22 AM, Riccardo Ferrari <ferra...@gmail.com>
>>> wrote:
>>>
>>>> Hi Kant,
>>>>
>>>> You can check the getValuesMap
>>>> <https://spark.apache.org/docs/2.2.0/api/java/org/apache/spark/sql/Row.html#getValuesMap-scala.collection.Seq->.
>>>> I found this post <https://stackoverflow.com/a/41602178/8356352>
>>>> useful, it is in Scala but should be a good starting point.
>>>> An alternative approach is combine the 'struct' and 'to_json' functions
>>>> <https://spark.apache.org/docs/2.2.0/api/java/org/apache/spark/sql/functions.html>.
>>>> I have not tested this in Java but I am using it in Python.
>>>>
>>>> Best,
>>>>
>>>> On Sun, Sep 10, 2017 at 1:45 AM, kant kodali <kanth...@gmail.com>
>>>> wrote:
>>>>
>>>>> toJSON on Row object.
>>>>>
>>>>> On Sat, Sep 9, 2017 at 4:18 PM, Felix Cheung <
>>>>> felixcheun...@hotmail.com> wrote:
>>>>>
>>>>>> toJSON on Dataset/DataFrame?
>>>>>>
>>>>>> ------------------------------
>>>>>> *From:* kant kodali <kanth...@gmail.com>
>>>>>> *Sent:* Saturday, September 9, 2017 4:15:49 PM
>>>>>> *To:* user @spark
>>>>>> *Subject:* How to convert Row to JSON in Java?
>>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> How to convert Row to JSON in Java? It would be nice to have
>>>>>> .toJson() method in the Row class.
>>>>>>
>>>>>> Thanks,
>>>>>> kant
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Best Regards,
>>> Ayan Guha
>>>
>>
>>
>
