Have you tried unionAll() method of DataFrame ? On Sun, Mar 13, 2016 at 8:44 PM, Divya Gehlot <divya.htco...@gmail.com> wrote:
> Hi, > > Please bear me for asking such a naive question > I have list of conditions (dynamic sqls) sitting in hbase table . > I need to iterate through those dynamic sqls and add the data to > dataframes. > As we know dataframes are immutable ,when I try to iterate in for loop as > shown below I get only last dynamic sql result set . > > var dffiltered : DataFrame = sqlContext.emptyDataFrame > for ( i <- 0 to (dfFilterSQLs.length - 1)) { > println("Condition============="+dfFilterSQLs(i)) > dffiltered = > dfresult.filter(dfFilterSQLs(i)).select("Col1","Col2","Col3","Col4","Col5") > dffiltered.show > } > > > How can I keep on appending data to dataframe and get the final result > having all the sql conditions. > > Thanks in advance for the help. > > Thanks, > Divya >