Re: DataFrame Find/Filter Based on Input - Inside Map function
You can keep a joined dataset cached and filter that joined df with your
filter condition
On 2 Jul 2015 15:01, "Mailing List" wrote:
> I need to pass the value of the filter dynamically like where id=
> and that someVal exist in another RDD.
>
> How can I do this across JavaRDD and DataFrame ?
>
> Sent from my iPad
>
> On Jul 2, 2015, at 12:49 AM, ayan guha wrote:
>
> You can directly use filter on a Dataframe
> On 2 Jul 2015 12:15, "Ashish Soni" wrote:
>
>> Hi All ,
>>
>> I have an DataFrame Created as below
>>
>> options.put("dbtable", "(select * from user) as account");
>> DataFrame accountRdd =
>> sqlContext.read().format("jdbc").options(options).load();
>>
>> and i have another RDD which contains login name and i want to find the
>> userid from above DF RDD and return it
>>
>> Not sure how can i do that as when i apply a map function and say filter
>> on DF i get Null pointor exception.
>>
>> Please help.
>>
>>
>>
Re: DataFrame Find/Filter Based on Input - Inside Map function
I need to pass the value of the filter dynamically like where id= and
that someVal exist in another RDD.
How can I do this across JavaRDD and DataFrame ?
Sent from my iPad
> On Jul 2, 2015, at 12:49 AM, ayan guha wrote:
>
> You can directly use filter on a Dataframe
>
>> On 2 Jul 2015 12:15, "Ashish Soni" wrote:
>> Hi All ,
>>
>> I have an DataFrame Created as below
>>
>> options.put("dbtable", "(select * from user) as account");
>> DataFrame accountRdd =
>> sqlContext.read().format("jdbc").options(options).load();
>>
>> and i have another RDD which contains login name and i want to find the
>> userid from above DF RDD and return it
>>
>> Not sure how can i do that as when i apply a map function and say filter on
>> DF i get Null pointor exception.
>>
>> Please help.
Re: DataFrame Find/Filter Based on Input - Inside Map function
You can directly use filter on a Dataframe
On 2 Jul 2015 12:15, "Ashish Soni" wrote:
> Hi All ,
>
> I have an DataFrame Created as below
>
> options.put("dbtable", "(select * from user) as account");
> DataFrame accountRdd =
> sqlContext.read().format("jdbc").options(options).load();
>
> and i have another RDD which contains login name and i want to find the
> userid from above DF RDD and return it
>
> Not sure how can i do that as when i apply a map function and say filter
> on DF i get Null pointor exception.
>
> Please help.
>
>
>
