Re: DataFrame Filter Inside Another Data Frame Map

2015-07-02 Thread Raghavendra Pandey
You can collect the dataframe as array n then create map out of it..., On Jul 2, 2015 9:23 AM, wrote: > Any example how can i return a Hashmap from data frame ? > > Thanks , > Ashish > > On Jul 1, 2015, at 11:34 PM, Holden Karau wrote: > > Collecting it as a regular (Java/scala/Python) map. You

Re: DataFrame Filter Inside Another Data Frame Map

2015-07-01 Thread asoni . learn
Any example how can i return a Hashmap from data frame ? Thanks , Ashish > On Jul 1, 2015, at 11:34 PM, Holden Karau wrote: > > Collecting it as a regular (Java/scala/Python) map. You can also broadcast > the map if your going to use it multiple times. > >> On Wednesday, July 1, 2015, Ashish

Re: DataFrame Filter Inside Another Data Frame Map

2015-07-01 Thread Holden Karau
Collecting it as a regular (Java/scala/Python) map. You can also broadcast the map if your going to use it multiple times. On Wednesday, July 1, 2015, Ashish Soni wrote: > Thanks , So if i load some static data from database and then i need to > use than in my map function to filter records what

Re: DataFrame Filter Inside Another Data Frame Map

2015-07-01 Thread Ashish Soni
Thanks , So if i load some static data from database and then i need to use than in my map function to filter records what will be the best way to do it, Ashish On Wed, Jul 1, 2015 at 10:45 PM, Raghavendra Pandey < raghavendra.pan...@gmail.com> wrote: > You cannot refer to one rdd inside another

Re: DataFrame Filter Inside Another Data Frame Map

2015-07-01 Thread Raghavendra Pandey
You cannot refer to one rdd inside another rdd.map function... Rdd object is not serialiable. Whatever objects you use inside map function should be serializable as they get transferred to executor nodes. On Jul 2, 2015 6:13 AM, "Ashish Soni" wrote: > Hi All , > > I am not sure what is the wron

DataFrame Filter Inside Another Data Frame Map

2015-07-01 Thread Ashish Soni
Hi All , I am not sure what is the wrong with below code as it give below error when i access inside the map but it works outside JavaRDD rdd2 = rdd.map(new Function() { @Override public Charge call(Charge ch) throws Exception { * DataFrame df = accountR