[jira] [Updated] (HADOOP-13169) Randomize file list in SimpleCopyListing

[ https://issues.apache.org/jira/browse/HADOOP-13169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13169: -- Attachment: HADOOP-13169-branch-2-006.patch Thank you very much for the review

[jira] [Updated] (HADOOP-13164) Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories

[ https://issues.apache.org/jira/browse/HADOOP-13164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13164: -- Attachment: HADOOP-13164-branch-2-004.patch > Optimize

[jira] [Updated] (HADOOP-13164) Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories

[ https://issues.apache.org/jira/browse/HADOOP-13164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13164: -- Status: Patch Available (was: Open) > Optimize

[jira] [Updated] (HADOOP-13164) Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories

[ https://issues.apache.org/jira/browse/HADOOP-13164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13164: -- Attachment: HADOOP-13164-branch-2-003.patch Thanks [~ste...@apache.org]. I have

[jira] [Updated] (HADOOP-13164) Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories

[ https://issues.apache.org/jira/browse/HADOOP-13164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13164: -- Status: Open (was: Patch Available) > Optimize

[jira] [Updated] (HADOOP-13164) Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories

[ https://issues.apache.org/jira/browse/HADOOP-13164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13164: -- Attachment: HADOOP-13164.branch-2-002.patch > Optimize

[jira] [Updated] (HADOOP-13164) Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories

[ https://issues.apache.org/jira/browse/HADOOP-13164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13164: -- Status: Patch Available (was: Open) > Optimize

[jira] [Updated] (HADOOP-13169) Randomize file list in SimpleCopyListing

[ https://issues.apache.org/jira/browse/HADOOP-13169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13169: -- Attachment: HADOOP-13169-branch-2-005.patch Sharing the HDFS based numbers based on

[jira] [Updated] (HADOOP-13169) Randomize file list in SimpleCopyListing

[ https://issues.apache.org/jira/browse/HADOOP-13169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13169: -- Attachment: HADOOP-13169-branch-2-004.patch I have run distcp in S3. Results show

[jira] [Updated] (HADOOP-13169) Randomize file list in SimpleCopyListing

[ https://issues.apache.org/jira/browse/HADOOP-13169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13169: -- Attachment: HADOOP-13169-branch-2-003.patch > Randomize file list in

[jira] [Created] (HADOOP-13432) S3A: Consider using TransferManager.download for copyToLocalFile

Rajesh Balamohan created HADOOP-13432: - Summary: S3A: Consider using TransferManager.download for copyToLocalFile Key: HADOOP-13432 URL: https://issues.apache.org/jira/browse/HADOOP-13432

[jira] [Updated] (HADOOP-13164) Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories

[ https://issues.apache.org/jira/browse/HADOOP-13164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13164: -- Attachment: HADOOP-13164.branch-2.WIP.002.patch > Optimize

[jira] [Commented] (HADOOP-13164) Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories

[ https://issues.apache.org/jira/browse/HADOOP-13164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15393498#comment-15393498 ] Rajesh Balamohan commented on HADOOP-13164: --- Thanks for the inputs Steve. I tried out a

[jira] [Updated] (HADOOP-13164) Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories

[ https://issues.apache.org/jira/browse/HADOOP-13164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13164: -- Attachment: HADOOP-13164.branch-2.WIP.patch Attaching the WIP patch for initial

[jira] [Commented] (HADOOP-13406) S3AFileSystem: Consider reusing filestatus in delete() and mkdirs()

[ https://issues.apache.org/jira/browse/HADOOP-13406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15389291#comment-15389291 ] Rajesh Balamohan commented on HADOOP-13406: --- This is from my local laptop connecting to west

[jira] [Updated] (HADOOP-13406) S3AFileSystem: Consider reusing filestatus in delete() and mkdirs()

[ https://issues.apache.org/jira/browse/HADOOP-13406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13406: -- Attachment: HADOOP-13406-branch-2-001.patch S3 test results are given below. Failures

[jira] [Updated] (HADOOP-13406) S3AFileSystem: Consider reusing filestatus in delete() and mkdirs()

[ https://issues.apache.org/jira/browse/HADOOP-13406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13406: -- Summary: S3AFileSystem: Consider reusing filestatus in delete() and mkdirs() (was:

[jira] [Updated] (HADOOP-13406) S3AFileSystem: Consider reusing filestatus in delete and mkdir

[ https://issues.apache.org/jira/browse/HADOOP-13406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13406: -- Affects Version/s: 2.8.0 > S3AFileSystem: Consider reusing filestatus in delete and

[jira] [Created] (HADOOP-13406) S3AFileSystem: Consider reusing filestatus in delete and mkdir

Rajesh Balamohan created HADOOP-13406: - Summary: S3AFileSystem: Consider reusing filestatus in delete and mkdir Key: HADOOP-13406 URL: https://issues.apache.org/jira/browse/HADOOP-13406 Project:

[jira] [Updated] (HADOOP-13212) Provide an option to set the socket buffers in S3AFileSystem

[ https://issues.apache.org/jira/browse/HADOOP-13212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13212: -- Attachment: HADOOP-13212-branch-2-002.patch >From my environment, i have been always

[jira] [Commented] (HADOOP-13212) Provide an option to set the socket buffers in S3AFileSystem

[ https://issues.apache.org/jira/browse/HADOOP-13212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15374250#comment-15374250 ] Rajesh Balamohan commented on HADOOP-13212: --- Without patch, here is the report which are

[jira] [Commented] (HADOOP-13212) Provide an option to set the socket buffers in S3AFileSystem

[ https://issues.apache.org/jira/browse/HADOOP-13212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15372718#comment-15372718 ] Rajesh Balamohan commented on HADOOP-13212: --- This is based on us-west-2 region. Ran the tests

[jira] [Updated] (HADOOP-13212) Provide an option to set the socket buffers in S3AFileSystem

[ https://issues.apache.org/jira/browse/HADOOP-13212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13212: -- Status: Patch Available (was: Open) > Provide an option to set the socket buffers in

[jira] [Assigned] (HADOOP-13212) Provide an option to set the socket buffers in S3AFileSystem

[ https://issues.apache.org/jira/browse/HADOOP-13212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned HADOOP-13212: - Assignee: Rajesh Balamohan > Provide an option to set the socket buffers in

[jira] [Updated] (HADOOP-13212) Provide an option to set the socket buffers in S3AFileSystem

[ https://issues.apache.org/jira/browse/HADOOP-13212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13212: -- Attachment: HADOOP-13212-branch-2-001.patch HttpClient internally would set it to

[jira] [Updated] (HADOOP-13203) S3a: Consider reducing the number of connection aborts by setting correct length in s3 request

[ https://issues.apache.org/jira/browse/HADOOP-13203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13203: -- Assignee: Steve Loughran (was: Rajesh Balamohan) > S3a: Consider reducing the number

[jira] [Updated] (HADOOP-13203) S3a: Consider reducing the number of connection aborts by setting correct length in s3 request

[ https://issues.apache.org/jira/browse/HADOOP-13203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13203: -- Attachment: stream_stats.tar.gz HADOOP-13203-branch-2-004.patch There

[jira] [Commented] (HADOOP-13203) S3a: Consider reducing the number of connection aborts by setting correct length in s3 request

[ https://issues.apache.org/jira/browse/HADOOP-13203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15328864#comment-15328864 ] Rajesh Balamohan commented on HADOOP-13203: --- Thanks for sharing the details. Would that mean

[jira] [Updated] (HADOOP-13203) S3a: Consider reducing the number of connection aborts by setting correct length in s3 request

[ https://issues.apache.org/jira/browse/HADOOP-13203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13203: -- Attachment: HADOOP-13203-branch-2-003.patch Thanks [~cnauroth]. Updated the patch.

[jira] [Comment Edited] (HADOOP-13203) S3a: Consider reducing the number of connection aborts by setting correct length in s3 request

[ https://issues.apache.org/jira/browse/HADOOP-13203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313391#comment-15313391 ] Rajesh Balamohan edited comment on HADOOP-13203 at 6/3/16 12:32 AM:

[jira] [Commented] (HADOOP-13203) S3a: Consider reducing the number of connection aborts by setting correct length in s3 request

[ https://issues.apache.org/jira/browse/HADOOP-13203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313391#comment-15313391 ] Rajesh Balamohan commented on HADOOP-13203: --- Thanks for the comments @cnauroth. In Hive,

[jira] [Comment Edited] (HADOOP-13203) S3a: Consider reducing the number of connection aborts by setting correct length in s3 request

[ https://issues.apache.org/jira/browse/HADOOP-13203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312227#comment-15312227 ] Rajesh Balamohan edited comment on HADOOP-13203 at 6/2/16 12:31 PM:

[jira] [Updated] (HADOOP-13203) S3a: Consider reducing the number of connection aborts by setting correct length in s3 request

[ https://issues.apache.org/jira/browse/HADOOP-13203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13203: -- Attachment: HADOOP-13203-branch-2-002.patch Attaching .2 patch with test coverage and

[jira] [Commented] (HADOOP-13164) Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories

[ https://issues.apache.org/jira/browse/HADOOP-13164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306506#comment-15306506 ] Rajesh Balamohan commented on HADOOP-13164: --- Thanks [~steve_l]. When a file is written to a

[jira] [Created] (HADOOP-13212) Provide an option to set the socket buffers in S3AFileSystem

Rajesh Balamohan created HADOOP-13212: - Summary: Provide an option to set the socket buffers in S3AFileSystem Key: HADOOP-13212 URL: https://issues.apache.org/jira/browse/HADOOP-13212 Project:

[jira] [Updated] (HADOOP-13162) Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs

[ https://issues.apache.org/jira/browse/HADOOP-13162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13162: -- Attachment: HADOOP-13162-branch-2-004.patch AWS endpoint: ap-southeast-1 AWS

[jira] [Updated] (HADOOP-13203) S3a: Consider reducing the number of connection aborts by setting correct length in s3 request

[ https://issues.apache.org/jira/browse/HADOOP-13203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13203: -- Target Version/s: 2.8.0 > S3a: Consider reducing the number of connection aborts by

[jira] [Updated] (HADOOP-13203) S3a: Consider reducing the number of connection aborts by setting correct length in s3 request

[ https://issues.apache.org/jira/browse/HADOOP-13203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13203: -- Attachment: HADOOP-13203-branch-2-001.patch Yes [~steve_l]. In workloads like hive,

[jira] [Updated] (HADOOP-13203) S3a: Consider reducing the number of connection aborts by setting correct length in s3 request

[ https://issues.apache.org/jira/browse/HADOOP-13203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13203: -- Issue Type: Sub-task (was: Bug) Parent: HADOOP-11694 > S3a: Consider

[jira] [Created] (HADOOP-13203) S3a: Consider reducing the number of connection aborts by setting correct length in s3 request

Rajesh Balamohan created HADOOP-13203: - Summary: S3a: Consider reducing the number of connection aborts by setting correct length in s3 request Key: HADOOP-13203 URL:

[jira] [Updated] (HADOOP-13162) Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs

[ https://issues.apache.org/jira/browse/HADOOP-13162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13162: -- Status: Patch Available (was: Open) > Consider reducing number of getFileStatus

[jira] [Updated] (HADOOP-13162) Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs

[ https://issues.apache.org/jira/browse/HADOOP-13162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13162: -- Attachment: HADOOP-13162-branch-2-003.patch > Consider reducing number of

[jira] [Updated] (HADOOP-13162) Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs

[ https://issues.apache.org/jira/browse/HADOOP-13162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13162: -- Status: Open (was: Patch Available) > Consider reducing number of getFileStatus

[jira] [Commented] (HADOOP-13164) Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories

[ https://issues.apache.org/jira/browse/HADOOP-13164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15296117#comment-15296117 ] Rajesh Balamohan commented on HADOOP-13164: --- Instead of optimizing

[jira] [Commented] (HADOOP-13162) Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs

[ https://issues.apache.org/jira/browse/HADOOP-13162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289227#comment-15289227 ] Rajesh Balamohan commented on HADOOP-13162: --- It should be possible to save one more additional

[jira] [Resolved] (HADOOP-13170) TestOptionsParser in o.a.h.tools fails

[ https://issues.apache.org/jira/browse/HADOOP-13170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan resolved HADOOP-13170. --- Resolution: Resolved Already got committed in HDFS-10397 yesterday in branch-2 and

[jira] [Comment Edited] (HADOOP-13170) TestOptionsParser in o.a.h.tools fails

[ https://issues.apache.org/jira/browse/HADOOP-13170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15288529#comment-15288529 ] Rajesh Balamohan edited comment on HADOOP-13170 at 5/18/16 6:51 AM:

[jira] [Comment Edited] (HADOOP-13169) Randomize file list in SimpleCopyListing

[ https://issues.apache.org/jira/browse/HADOOP-13169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15288525#comment-15288525 ] Rajesh Balamohan edited comment on HADOOP-13169 at 5/18/16 6:49 AM:

[jira] [Updated] (HADOOP-13169) Randomize file list in SimpleCopyListing

[ https://issues.apache.org/jira/browse/HADOOP-13169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13169: -- Attachment: HADOOP-13169-branch-2-002.patch Removed TestOptionsParser change which

[jira] [Updated] (HADOOP-13169) Randomize file list in SimpleCopyListing

[ https://issues.apache.org/jira/browse/HADOOP-13169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13169: -- Status: Patch Available (was: Open) > Randomize file list in SimpleCopyListing >

[jira] [Updated] (HADOOP-13169) Randomize file list in SimpleCopyListing

[ https://issues.apache.org/jira/browse/HADOOP-13169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13169: -- Attachment: HADOOP-13169-branch-2-001.patch TestCopyListing covers this codepath.

[jira] [Created] (HADOOP-13170) TestOptionsParser in o.a.h.tools fails

Rajesh Balamohan created HADOOP-13170: - Summary: TestOptionsParser in o.a.h.tools fails Key: HADOOP-13170 URL: https://issues.apache.org/jira/browse/HADOOP-13170 Project: Hadoop Common

[jira] [Updated] (HADOOP-13169) Randomize file list in SimpleCopyListing

[ https://issues.apache.org/jira/browse/HADOOP-13169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13169: -- Component/s: tools/distcp > Randomize file list in SimpleCopyListing >

[jira] [Created] (HADOOP-13169) Randomize file list in SimpleCopyListing

Rajesh Balamohan created HADOOP-13169: - Summary: Randomize file list in SimpleCopyListing Key: HADOOP-13169 URL: https://issues.apache.org/jira/browse/HADOOP-13169 Project: Hadoop Common

[jira] [Updated] (HADOOP-13162) Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs

[ https://issues.apache.org/jira/browse/HADOOP-13162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13162: -- Attachment: HADOOP-13162-branch-2-002.patch > Consider reducing number of

[jira] [Updated] (HADOOP-13162) Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs

[ https://issues.apache.org/jira/browse/HADOOP-13162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13162: -- Target Version/s: 2.8.0 Priority: Minor (was: Major) > Consider reducing

[jira] [Updated] (HADOOP-13162) Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs

[ https://issues.apache.org/jira/browse/HADOOP-13162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13162: -- Issue Type: Sub-task (was: Bug) Parent: HADOOP-1169 > Consider reducing

[jira] [Created] (HADOOP-13165) S3AFileSystem toString can throw NPE when cannedACL is not set

Rajesh Balamohan created HADOOP-13165: - Summary: S3AFileSystem toString can throw NPE when cannedACL is not set Key: HADOOP-13165 URL: https://issues.apache.org/jira/browse/HADOOP-13165 Project:

[jira] [Created] (HADOOP-13164) Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories

Rajesh Balamohan created HADOOP-13164: - Summary: Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories Key: HADOOP-13164 URL: https://issues.apache.org/jira/browse/HADOOP-13164 Project: Hadoop

[jira] [Updated] (HADOOP-13163) Reuse pre-computed filestatus in Distcp-CopyMapper

[ https://issues.apache.org/jira/browse/HADOOP-13163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13163: -- Attachment: HADOOP-13163.001.patch Attaching the patch for review. Existing test

[jira] [Created] (HADOOP-13163) Reuse pre-computed filestatus in Distcp-CopyMapper

Rajesh Balamohan created HADOOP-13163: - Summary: Reuse pre-computed filestatus in Distcp-CopyMapper Key: HADOOP-13163 URL: https://issues.apache.org/jira/browse/HADOOP-13163 Project: Hadoop Common

[jira] [Updated] (HADOOP-13162) Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs

[ https://issues.apache.org/jira/browse/HADOOP-13162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13162: -- Attachment: HADOOP-13162.001.patch \cc [~steve_l] [~cnauroth] > Consider reducing

[jira] [Updated] (HADOOP-13162) Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs

[ https://issues.apache.org/jira/browse/HADOOP-13162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13162: -- Description: getFileStatus is relatively expensive call and mkdirs invokes it

[jira] [Created] (HADOOP-13162) Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs

Rajesh Balamohan created HADOOP-13162: - Summary: Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs Key: HADOOP-13162 URL: https://issues.apache.org/jira/browse/HADOOP-13162

[jira] [Commented] (HADOOP-13145) In DistCp, prevent unnecessary getFileStatus call when not preserving metadata.

[ https://issues.apache.org/jira/browse/HADOOP-13145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284211#comment-15284211 ] Rajesh Balamohan commented on HADOOP-13145: --- Thanks for sharing the patch [~cnauroth]. I tried

[jira] [Updated] (HADOOP-10694) Remove synchronized input streams from Writable deserialization

[ https://issues.apache.org/jira/browse/HADOOP-10694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-10694: -- Status: Patch Available (was: Open) > Remove synchronized input streams from

[jira] [Updated] (HADOOP-10694) Remove synchronized input streams from Writable deserialization

[ https://issues.apache.org/jira/browse/HADOOP-10694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-10694: -- Attachment: (was: HADOOP-10694.2.patch) > Remove synchronized input streams from

[jira] [Updated] (HADOOP-10694) Remove synchronized input streams from Writable deserialization

[ https://issues.apache.org/jira/browse/HADOOP-10694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-10694: -- Attachment: HADOOP-10694.2.patch > Remove synchronized input streams from Writable

[jira] [Updated] (HADOOP-10694) Remove synchronized input streams from Writable deserialization

[ https://issues.apache.org/jira/browse/HADOOP-10694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-10694: -- Attachment: HADOOP-10694.2.patch > Remove synchronized input streams from Writable

[jira] [Updated] (HADOOP-13047) S3a Forward seek in stream length to be configurable

[ https://issues.apache.org/jira/browse/HADOOP-13047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13047: -- Attachment: HADOOP-13047.WIP.2.patch Agreed. Created "fs.s3a.readahead.buffer.size"

[jira] [Updated] (HADOOP-13047) S3a Forward seek in stream length to be configurable

[ https://issues.apache.org/jira/browse/HADOOP-13047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13047: -- Attachment: HADOOP-13047.WIP.patch Attaching the high level WIP patch. Based on the

[jira] [Commented] (HADOOP-12444) Consider implementing lazy seek in S3AInputStream

[ https://issues.apache.org/jira/browse/HADOOP-12444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15227184#comment-15227184 ] Rajesh Balamohan commented on HADOOP-12444: --- Patch is based on HADOOP-12994 and compilation

[jira] [Updated] (HADOOP-12444) Consider implementing lazy seek in S3AInputStream

[ https://issues.apache.org/jira/browse/HADOOP-12444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-12444: -- Attachment: HADOOP-12444-006.patch Changes in 006: 1. Modified patch after applying

[jira] [Commented] (HADOOP-12444) Consider implementing lazy seek in S3AInputStream

[ https://issues.apache.org/jira/browse/HADOOP-12444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223756#comment-15223756 ] Rajesh Balamohan commented on HADOOP-12444: --- Above changes are related to 005 patch. >

[jira] [Updated] (HADOOP-12444) Consider implementing lazy seek in S3AInputStream

[ https://issues.apache.org/jira/browse/HADOOP-12444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-12444: -- Attachment: HADOOP-12444-005.patch Changes: 1. EOF validation is not done in seek and

[jira] [Updated] (HADOOP-12444) Consider implementing lazy seek in S3AInputStream

[ https://issues.apache.org/jira/browse/HADOOP-12444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-12444: -- Attachment: hadoop-aws-test-reports.tar.gz Uploading AWS test report for reference.

[jira] [Updated] (HADOOP-12444) Consider implementing lazy seek in S3AInputStream

[ https://issues.apache.org/jira/browse/HADOOP-12444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-12444: -- Attachment: HADOOP-12444.3.patch Thanks [~thodemoor]. Attaching the revised patch.

[jira] [Updated] (HADOOP-12444) Consider implementing lazy seek in S3AInputStream

[ https://issues.apache.org/jira/browse/HADOOP-12444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-12444: -- Attachment: HADOOP-12444.2.patch Attaching revised .2 version (takes care of EOF

[jira] [Commented] (HADOOP-12444) Consider implementing lazy seek in S3AInputStream

[ https://issues.apache.org/jira/browse/HADOOP-12444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14958563#comment-14958563 ] Rajesh Balamohan commented on HADOOP-12444: --- Sure, whenever you have time. Thanks a lot

[jira] [Updated] (HADOOP-12444) Consider implementing lazy seek in S3AInputStream

[ https://issues.apache.org/jira/browse/HADOOP-12444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-12444: -- Status: Patch Available (was: Open) > Consider implementing lazy seek in

[jira] [Commented] (HADOOP-12444) Consider implementing lazy seek in S3AInputStream

[ https://issues.apache.org/jira/browse/HADOOP-12444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952688#comment-14952688 ] Rajesh Balamohan commented on HADOOP-12444: --- [~ste...@apache.org], [~thodemoor] - Can you

[jira] [Assigned] (HADOOP-12444) Consider implementing lazy seek in S3AInputStream

[ https://issues.apache.org/jira/browse/HADOOP-12444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned HADOOP-12444: - Assignee: Rajesh Balamohan > Consider implementing lazy seek in S3AInputStream

[jira] [Updated] (HADOOP-12444) Consider implementing lazy seek in S3AInputStream

[ https://issues.apache.org/jira/browse/HADOOP-12444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-12444: -- Attachment: HADOOP-12444.1.patch Thanks [~thodemoor]. Attaching .1 patch. Saw around

[jira] [Updated] (HADOOP-12444) Consider implementing lazy seek in S3AInputStream

[ https://issues.apache.org/jira/browse/HADOOP-12444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-12444: -- Attachment: HADOOP-12444.WIP.patch Attaching the WIP patch as first cut. > Consider

[jira] [Commented] (HADOOP-10674) Rewrite the PureJavaCrc32 loop for performance improvement

[ https://issues.apache.org/jira/browse/HADOOP-10674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14026142#comment-14026142 ] Rajesh Balamohan commented on HADOOP-10674: --- Posting the results for 1.8 (even

[jira] [Commented] (HADOOP-9331) Hadoop crypto codec framework and crypto codec implementations

[ https://issues.apache.org/jira/browse/HADOOP-9331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13753281#comment-13753281 ] Rajesh Balamohan commented on HADOOP-9331: -- First of all, thanks for adding the

[jira] [Updated] (HADOOP-5380) Calls from running tasks to TaskTracker methods sometimes fail and incur a 60s timeout

[ https://issues.apache.org/jira/browse/HADOOP-5380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-5380: - Status: Patch Available (was: Open) Calls from running tasks to TaskTracker methods

[jira] [Updated] (HADOOP-5380) Calls from running tasks to TaskTracker methods sometimes fail and incur a 60s timeout

[ https://issues.apache.org/jira/browse/HADOOP-5380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-5380: - Fix Version/s: 0.23.0 Status: Open (was: Patch Available) Calls from

[jira] [Updated] (HADOOP-5380) Calls from running tasks to TaskTracker methods sometimes fail and incur a 60s timeout

[ https://issues.apache.org/jira/browse/HADOOP-5380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-5380: - Affects Version/s: (was: 0.20.1) 0.23.0

[jira] Updated: (HADOOP-5380) Calls from running tasks to TaskTracker methods sometimes fail and incur a 60s timeout

[ https://issues.apache.org/jira/browse/HADOOP-5380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-5380: - Attachment: HADOOP_5380-Y.0.20.20x.patch Attaching the patch which fixes the issue in

[jira] Commented: (HADOOP-5380) Calls from running tasks to TaskTracker methods sometimes fail and incur a 60s timeout

[ https://issues.apache.org/jira/browse/HADOOP-5380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12978738#action_12978738 ] Rajesh Balamohan commented on HADOOP-5380: -- Hi Kan, Thanks for the comments.

[jira] Updated: (HADOOP-5380) Calls from running tasks to TaskTracker methods sometimes fail and incur a 60s timeout

[ https://issues.apache.org/jira/browse/HADOOP-5380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-5380: - Attachment: HADOOP-5380.Y.20.branch.patch Investigation: === 1.

[jira] Updated: (HADOOP-7016) Optimize locking codepath in LocalDirAllocator.getLocalPathToRead() and reduce creating many Path objects

[ https://issues.apache.org/jira/browse/HADOOP-7016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-7016: - Attachment: HADOOP-7016-v3.Yahoo.patch Removed the unwanted isFilePresent() method, and

[jira] Updated: (HADOOP-7016) Optimize locking codepath in LocalDirAllocator.getLocalPathToRead() and reduce creating many Path objects

[ https://issues.apache.org/jira/browse/HADOOP-7016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-7016: - Attachment: HADOOP-7016-v2.Yahoo.patch minor modification to the older patch and

[jira] Created: (HADOOP-7016) Optimize locking codepath in LocalDirAllocator.getLocalPathToRead() and reduce creating many Path objects

Optimize locking codepath in LocalDirAllocator.getLocalPathToRead() and reduce creating many Path objects - Key: HADOOP-7016 URL:

[jira] Updated: (HADOOP-7016) Optimize locking codepath in LocalDirAllocator.getLocalPathToRead() and reduce creating many Path objects

[ https://issues.apache.org/jira/browse/HADOOP-7016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-7016: - Attachment: HADOOP-7016-Yahoo.patch Optimize locking codepath in

[jira] Created: (HADOOP-6535) Ability to measure the throughput of individual components at node level

Ability to measure the throughput of individual components at node level Key: HADOOP-6535 URL: https://issues.apache.org/jira/browse/HADOOP-6535 Project: Hadoop Common

<    1   2