Thanks everyone. I studied the source code and realized minPartitions is
passed over to Hadoop's InputFormat and its upto the InputFormat
implementation to use the parameter as a hint.
Thanks,
Aniket
On Fri, Jan 2, 2015, 7:13 AM Rishi Yadav ri...@infoobjects.com wrote:
Hi Ankit,
Optional
I am trying to read a file into a single partition but it seems like
sparkContext.textFile ignores the passed minPartitions value. I know I can
repartition the RDD but I was curious to know if this is expected or if
this is a bug that needs to be further investigated?
Hi Ankit,
Optional number of partitions value is to increase number of partitions not
reduce it from default value.
On Thu, Jan 1, 2015 at 10:43 AM, Aniket Bhatnagar
aniket.bhatna...@gmail.com wrote:
I am trying to read a file into a single partition but it seems like
sparkContext.textFile
