Re: FSDataInputStream.read(byte[]) only reads to a block boundary?

2009-06-28 Thread Raghu Angadi
This seems to be the case. I don't think there is any specific reason not to read across the block boundary... Even if HDFS does read across the blocks, it is still not a good idea to ignore the JavaDoc for read(). If you want all the bytes read, then you should have a while loop or one of

Re: FSDataInputStream.read(byte[]) only reads to a block boundary?

2009-06-28 Thread Matei Zaharia
This kind of partial read is often used by the OS to return to your application as soon as possible if trying to read more data would block, in case you can begin computing on the partial data. In some applications, it's not useful, but when you can begin computing on partial data, it allows the

Re: FSDataInputStream.read(byte[]) only reads to a block boundary?

2009-06-28 Thread M. C. Srivas
On Sun, Jun 28, 2009 at 3:01 PM, Matei Zaharia ma...@cloudera.com wrote: This kind of partial read is often used by the OS to return to your application as soon as possible if trying to read more data would block, in case you can begin computing on the partial data. In some applications, it's