Summarizing an offline message:
The following worked for Divya:
dffiltered = dffiltered.unionAll(dfresult.filter ...
On Mon, Mar 14, 2016 at 5:54 AM, Lohith Samaga M <lohith.sam...@mphasis.com>
wrote:
> If all sql results have same set of columns you could UNION all the
> dataframes
>
> Create an empty df and Union all
> Then reassign new df to original df before next union all
>
> Not sure if it is a good idea, but it works
>
> Lohith
>
> Sent from my Sony Xperia™ smartphone
>
>
> ---- Divya Gehlot wrote ----
>
>
> Hi,
>
> Please bear me for asking such a naive question
> I have list of conditions (dynamic sqls) sitting in hbase table .
> I need to iterate through those dynamic sqls and add the data to
> dataframes.
> As we know dataframes are immutable ,when I try to iterate in for loop as
> shown below I get only last dynamic sql result set .
>
> var dffiltered : DataFrame = sqlContext.emptyDataFrame
> for ( i <- 0 to (dfFilterSQLs.length - 1)) {
> println("Condition============="+dfFilterSQLs(i))
> dffiltered =
> dfresult.filter(dfFilterSQLs(i)).select("Col1","Col2","Col3","Col4","Col5")
> dffiltered.show
> }
>
>
> How can I keep on appending data to dataframe and get the final result
> having all the sql conditions.
>
> Thanks in advance for the help.
>
> Thanks,
> Divya
>
>
> Information transmitted by this e-mail is proprietary to Mphasis, its
> associated companies and/ or its customers and is intended
> for use only by the individual or entity to which it is addressed, and may
> contain information that is privileged, confidential or
> exempt from disclosure under applicable law. If you are not the intended
> recipient or it appears that this mail has been forwarded
> to you without proper authority, you are notified that any use or
> dissemination of this information in any manner is strictly
> prohibited. In such cases, please notify us immediately at
> mailmas...@mphasis.com and delete this mail from your records.
>
