Re: DataFrame equivalent to RDD.partionByKey
I think you are looking for `def repartition(numPartitions: Int, partitionExprs: Column*)` On Tue, Aug 9, 2016 at 9:36 AM, Stephen Fletcher wrote: > Is there a DataFrameReader equivalent to the RDD's partitionByKey for RDD? > I'm reading data from a file data source and I want to partition this data > I'm reading in to be partitioned the same way as the data I'm processing > through a spark streaming RDD in the process. - To unsubscribe e-mail: user-unsubscr...@spark.apache.org
DataFrame equivalent to RDD.partionByKey
Is there a DataFrameReader equivalent to the RDD's partitionByKey for RDD? I'm reading data from a file data source and I want to partition this data I'm reading in to be partitioned the same way as the data I'm processing through a spark streaming RDD in the process.