subject:"How can I do the following simple scenario in spark"

Re: How can I do the following simple scenario in spark

2018-06-19 Thread Sonal Goyal

Try flatMapToPair instead of flatMap Thanks, Sonal Nube Technologies On Tue, Jun 19, 2018 at 11:08 PM, Soheil Pourbafrani wrote: > Hi, I have a JSON file in the following structure: > ++---+

How can I do the following simple scenario in spark

2018-06-19 Thread Soheil Pourbafrani

Hi, I have a JSON file in the following structure: ++---+ | full_text| id| ++---+ I want to tokenize each sentence into pairs of (word, id) for example, having the record : ("Hi, How are you?", id)