Hi Team,
I am new to Spark, my requirement is I have a huge list, which is converted
to spark dataset and I need to operate on this dataset and store computed
values in another object/dataset and store in memory for further processing.
Approach I tried is : list is retrieved from third party in
.collect()` to get a local list of `Person` objects
> first, then you can safely iterate on the local list and do any update you
> want.
>
> Thanks.
>
> On Tue, Dec 19, 2017 at 2:16 PM, Sunitha Chennareddy <
> chennareddysuni...@gmail.com> wrote:
>
>> Hi Deepak
global variable as in your snippet.
> This can to work because you work in a distributed setting.
> Probably the code will fail on a cluster or at random.
>
> On 19. Dec 2017, at 07:16, Sunitha Chennareddy <
> chennareddysuni...@gmail.com> wrote:
>
> Hi Deepak,
>
>
ing like
> df.rdd.map{ x => MyClass(x.getString(0),.)}
>
> HTH
>
> --Deepak
>
> On Dec 19, 2017 09:25, "Sunitha Chennareddy" > wrote:
>
> Hi All,
>
> I am new to Spark, I want to convert DataFrame to List with out
> using collect().
>
&
Hi All,
I am new to Spark, I want to convert DataFrame to List with out
using collect().
Main requirement is I need to iterate through the rows of dataframe and
call another function by passing column value of each row (person.getId())
Here is the snippet I have tried, Kindly help me to resolve
