Does it retain the order if its pulling from the hdfs blocks, meaning if file1 => a, b, c partition in order if I convert to 2 partition read will it map to ab, c or a, bc or it can also be a, cb ?
Mayur Rustagi Ph: +1 (760) 203 3257 http://www.sigmoidanalytics.com @mayur_rustagi <https://twitter.com/mayur_rustagi> On Sat, Oct 18, 2014 at 9:09 AM, Ilya Ganelin <ilgan...@gmail.com> wrote: > Also - if you're doing a text file read you can pass the number of > resulting partitions as the second argument. > On Oct 17, 2014 9:05 PM, "Larry Liu" <larryli...@gmail.com> wrote: > >> Thanks, Andrew. What about reading out of local? >> >> On Fri, Oct 17, 2014 at 5:38 PM, Andrew Ash <and...@andrewash.com> wrote: >> >>> When reading out of HDFS it's the HDFS block size. >>> >>> On Fri, Oct 17, 2014 at 5:27 PM, Larry Liu <larryli...@gmail.com> wrote: >>> >>>> What is the default input split size? How to change it? >>>> >>> >>> >>
