Ananth T. Sarathy wrote:
it's on s3. and it always happens.
I have no experience with S3. You might want to check out S3 forums. It
can't be normal for S3 either.. there must be something missing
(configuration, ACLs... ).
Raghu.
Ananth T Sarathy
On Wed, Aug 19, 2009 at 4:35 PM, Raghu Angadi <[email protected]> wrote:
Ananth T. Sarathy wrote:
Also, I just want to clear... the delay seems to at the intial
(read = in.read(buf))
It the file on HDFS (over S3) or S3?
Does it always happen?
Raghu.
after the first time into the loop it flies...
Ananth T Sarathy
On Wed, Aug 19, 2009 at 1:58 PM, Raghu Angadi <[email protected]>
wrote:
Edward Capriolo wrote:
On Wed, Aug 19, 2009 at 11:11 AM, Edward Capriolo <
[email protected]
wrote:
It would be as fast as underlying filesystem goes.
I would not agree with that statement. There is overhead.
You might be misinterpreting my comment. There is of course some over
head
(at the least the procedure calls).. depending on you underlying
filesystem,
there could be extra buffer copies and CRC overhead. But none of that
explains transfer as slow as 1 MBps (if my interpretation of of results
is
correct).
Raghu.
