In some cases, it should be possible to concatenate many files into one. If necessary, a unique byte sequence could be inserted at each "seam".
- Bob Futrelle On Nov 6, 2007 1:58 PM, Ted Dunning <[EMAIL PROTECTED]> wrote: > > I think that would help some, but the real problem for high performance is > disorganized behavior of the disk head. If the MFIFormat could organize > files according to disk location as well and avoid successive file opens, > you might be OK, but that is asking for the moon. > > > > On 11/6/07 8:14 AM, "Joydeep Sen Sarma" <[EMAIL PROTECTED]> wrote: > > > Would it help if the multifileinputformat bundled files into splits based on > > their location? (wondering if remote copy speed is a bottleneck in map) > >
