Thanks Ted. Actually a follow up question. I need to read multiple HDFS files
into RDD. What I am doing now is: for each file I read them into a RDD. Then
later on I union all these RDDs into one RDD. I am not sure if it is the best
way to do it.
ThanksSenqiang
On Tuesday, March 3, 2015 2:40 PM, Ted Yu <[email protected]> wrote:
Looking at scaladoc:
/** Get an RDD for a Hadoop file with an arbitrary new API InputFormat. */
def newAPIHadoopFile[K, V, F <: NewInputFormat[K, V]]
Your conclusion is confirmed.
On Tue, Mar 3, 2015 at 1:59 PM, S. Zhou <[email protected]> wrote:
I did some experiments and it seems not. But I like to get confirmation (or
perhaps I missed something). If it does support, could u let me know how to
specify multiple folders? Thanks.
Senqiang
