[jira] [Created] (HADOOP-18447) Vectored IO: Threadpool should be closed on interrupts or during close calls

2022-09-07 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created HADOOP-18447: - Summary: Vectored IO: Threadpool should be closed on interrupts or during close calls Key: HADOOP-18447 URL: https://issues.apache.org/jira/browse/HADOOP-18447

[jira] [Created] (HADOOP-18347) Restrict vectoredIO threadpool to reduce memory pressure

2022-07-18 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created HADOOP-18347: - Summary: Restrict vectoredIO threadpool to reduce memory pressure Key: HADOOP-18347 URL: https://issues.apache.org/jira/browse/HADOOP-18347 Project: Hadoop

[jira] [Commented] (HADOOP-18106) Handle memory fragmentation in S3 Vectored IO implementation.

2022-06-06 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17550781#comment-17550781 ] Rajesh Balamohan commented on HADOOP-18106: --- This will be applicable mainly for direct byte

[jira] [Created] (HADOOP-18115) EvaluatingStatisticsMap::entrySet may not need parallelstream

2022-02-03 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created HADOOP-18115: - Summary: EvaluatingStatisticsMap::entrySet may not need parallelstream Key: HADOOP-18115 URL: https://issues.apache.org/jira/browse/HADOOP-18115 Project:

[jira] [Commented] (HADOOP-17531) DistCp: Reduce memory usage on copying huge directories

2021-02-17 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286328#comment-17286328 ] Rajesh Balamohan commented on HADOOP-17531: --- [~ayushtkn]: Was the test tried out with HDFS or

[jira] [Commented] (HADOOP-17347) ABFS: Read optimizations

2021-01-03 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17257885#comment-17257885 ] Rajesh Balamohan commented on HADOOP-17347: --- >> If the read is for the last 8 bytes, read

[jira] [Updated] (HADOOP-17156) Clear readahead requests on stream close

2020-07-27 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-17156: -- Priority: Minor (was: Major) > Clear readahead requests on stream close >

[jira] [Created] (HADOOP-17156) Clear readahead requests on stream close

2020-07-27 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created HADOOP-17156: - Summary: Clear readahead requests on stream close Key: HADOOP-17156 URL: https://issues.apache.org/jira/browse/HADOOP-17156 Project: Hadoop Common

[jira] [Comment Edited] (HADOOP-17020) RawFileSystem could localize default block size to avoid sync bottleneck in config

2020-05-08 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17102474#comment-17102474 ] Rajesh Balamohan edited comment on HADOOP-17020 at 5/8/20, 11:09 AM:

[jira] [Commented] (HADOOP-17020) RawFileSystem could localize default block size to avoid sync bottleneck in config

2020-05-08 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17102474#comment-17102474 ] Rajesh Balamohan commented on HADOOP-17020: --- Sure. ThanksĀ  [~ste...@apache.org] . PR:

[jira] [Updated] (HADOOP-17020) RawFileSystem could localize default block size to avoid sync bottleneck in config

2020-05-01 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-17020: -- Status: Open (was: Patch Available) > RawFileSystem could localize default block

[jira] [Updated] (HADOOP-17020) RawFileSystem could localize default block size to avoid sync bottleneck in config

2020-05-01 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-17020: -- Status: Patch Available (was: Open) > RawFileSystem could localize default block

[jira] [Updated] (HADOOP-17020) RawFileSystem could localize default block size to avoid sync bottleneck in config

2020-04-30 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-17020: -- Attachment: HADOOP-17020.1.patch > RawFileSystem could localize default block size

[jira] [Commented] (HADOOP-17020) RawFileSystem could localize default block size to avoid sync bottleneck in config

2020-04-30 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097127#comment-17097127 ] Rajesh Balamohan commented on HADOOP-17020: --- Also found similar kind of issue in mkdirs.

[jira] [Comment Edited] (HADOOP-17020) RawFileSystem could localize default block size to avoid sync bottleneck in config

2020-04-30 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097127#comment-17097127 ] Rajesh Balamohan edited comment on HADOOP-17020 at 5/1/20, 2:17 AM:

[jira] [Updated] (HADOOP-17020) RawFileSystem could localize default block size to avoid sync bottleneck in config

2020-04-30 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-17020: -- Attachment: Screenshot 2020-05-01 at 7.12.06 AM.png > RawFileSystem could localize

[jira] [Updated] (HADOOP-17020) RawFileSystem could localize default block size to avoid sync bottleneck in config

2020-04-29 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-17020: -- Description: RawLocalFileSystem could localize default block size to avoid sync

[jira] [Created] (HADOOP-17020) RawFileSystem could localize default block size to avoid sync bottleneck in config

2020-04-29 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created HADOOP-17020: - Summary: RawFileSystem could localize default block size to avoid sync bottleneck in config Key: HADOOP-17020 URL: https://issues.apache.org/jira/browse/HADOOP-17020

[jira] [Created] (HADOOP-16751) DurationInfo text parsing/formatting should be moved out of hotpath

2019-12-08 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created HADOOP-16751: - Summary: DurationInfo text parsing/formatting should be moved out of hotpath Key: HADOOP-16751 URL: https://issues.apache.org/jira/browse/HADOOP-16751

[jira] [Commented] (HADOOP-16711) With S3Guard + authmode, consider skipping "verifyBuckets" check in S3A fs init()

2019-11-18 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976966#comment-16976966 ] Rajesh Balamohan commented on HADOOP-16711: --- ORC-570 is specific to lazy init on ORC side and

[jira] [Updated] (HADOOP-16711) With S3Guard + authmode, consider skipping "verifyBuckets" check in S3A fs init()

2019-11-14 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-16711: -- Attachment: HADOOP-16711.prelim.1.patch > With S3Guard + authmode, consider skipping

[jira] [Created] (HADOOP-16711) With S3Guard + authmode, consider skipping "verifyBuckets" check in S3A fs init()

2019-11-14 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created HADOOP-16711: - Summary: With S3Guard + authmode, consider skipping "verifyBuckets" check in S3A fs init() Key: HADOOP-16711 URL: https://issues.apache.org/jira/browse/HADOOP-16711

[jira] [Created] (HADOOP-16709) Consider having the ability to turn off TTL in S3Guard + Authoritative mode

2019-11-13 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created HADOOP-16709: - Summary: Consider having the ability to turn off TTL in S3Guard + Authoritative mode Key: HADOOP-16709 URL: https://issues.apache.org/jira/browse/HADOOP-16709

[jira] [Resolved] (HADOOP-16648) HDFS Native Client does not build correctly

2019-10-11 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan resolved HADOOP-16648. --- Resolution: Duplicate Marking this as dup of HDFS-14900 > HDFS Native Client does

[jira] [Commented] (HADOOP-16648) HDFS Native Client does not build correctly

2019-10-11 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16949175#comment-16949175 ] Rajesh Balamohan commented on HADOOP-16648: --- Closing this ticket as HDFS-14900 fixes the

[jira] [Commented] (HADOOP-16648) HDFS Native Client does not build correctly

2019-10-10 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16948984#comment-16948984 ] Rajesh Balamohan commented on HADOOP-16648: --- Sure, thanks [~ayushtkn], [~ste...@apache.org].

[jira] [Updated] (HADOOP-16648) HDFS Native Client does not build correctly

2019-10-10 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-16648: -- Affects Version/s: 3.3.0 > HDFS Native Client does not build correctly >

[jira] [Created] (HADOOP-16648) HDFS Native Client does not build correctly

2019-10-10 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created HADOOP-16648: - Summary: HDFS Native Client does not build correctly Key: HADOOP-16648 URL: https://issues.apache.org/jira/browse/HADOOP-16648 Project: Hadoop Common

[jira] [Commented] (HADOOP-16604) Provide copy functionality for cloud native applications

2019-10-04 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16944262#comment-16944262 ] Rajesh Balamohan commented on HADOOP-16604: --- Thanks for sharing the details

[jira] [Updated] (HADOOP-16629) support copyFile in s3afilesystem

2019-10-04 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-16629: -- Affects Version/s: 3.2.1 > support copyFile in s3afilesystem >

[jira] [Created] (HADOOP-16629) support copyFile in s3afilesystem

2019-10-04 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created HADOOP-16629: - Summary: support copyFile in s3afilesystem Key: HADOOP-16629 URL: https://issues.apache.org/jira/browse/HADOOP-16629 Project: Hadoop Common Issue

[jira] [Updated] (HADOOP-16629) support copyFile in s3afilesystem

2019-10-04 Thread Rajesh Balamohan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-16629: -- Component/s: fs/s3 > support copyFile in s3afilesystem >

[jira] [Created] (HADOOP-16604) Provide copy functionality for cloud native applications

2019-09-25 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created HADOOP-16604: - Summary: Provide copy functionality for cloud native applications Key: HADOOP-16604 URL: https://issues.apache.org/jira/browse/HADOOP-16604 Project: Hadoop

[jira] [Updated] (HADOOP-15042) Azure::PageBlobInputStream::skip can return -ve value when numberOfPagesRemaining is 0

2017-11-15 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-15042: -- Assignee: Rajesh Balamohan Status: Patch Available (was: Open) >

[jira] [Comment Edited] (HADOOP-15042) Azure::PageBlobInputStream::skip can return -ve value when numberOfPagesRemaining is 0

2017-11-15 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254718#comment-16254718 ] Rajesh Balamohan edited comment on HADOOP-15042 at 11/16/17 4:04 AM: -

[jira] [Updated] (HADOOP-15042) Azure::PageBlobInputStream::skip can return -ve value when numberOfPagesRemaining is 0

2017-11-15 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-15042: -- Attachment: HADOOP-15042.001.patch {noformat} mvn test -Dtest=Test\*,ITest\*

[jira] [Created] (HADOOP-15042) Azure::PageBlobInputStream::skip can return -ve value when numberOfPagesRemaining is 0

2017-11-15 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created HADOOP-15042: - Summary: Azure::PageBlobInputStream::skip can return -ve value when numberOfPagesRemaining is 0 Key: HADOOP-15042 URL:

[jira] [Created] (HADOOP-14988) WASB: Expose WASB status metrics as counters in Hadoop

2017-10-26 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created HADOOP-14988: - Summary: WASB: Expose WASB status metrics as counters in Hadoop Key: HADOOP-14988 URL: https://issues.apache.org/jira/browse/HADOOP-14988 Project: Hadoop

[jira] [Commented] (HADOOP-14965) s3a input stream "normal" fadvise mode to be adaptive

2017-10-22 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16214478#comment-16214478 ] Rajesh Balamohan commented on HADOOP-14965: --- It would be good to tune itself based on the seek

[jira] [Commented] (HADOOP-14680) Azure: IndexOutOfBoundsException in BlockBlobInputStream

2017-07-25 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100923#comment-16100923 ] Rajesh Balamohan commented on HADOOP-14680: --- Thanks for the patch [~tmarquardt]. Patch lgtm

[jira] [Created] (HADOOP-14680) Azure: IndexOutOfBoundsException in BlockBlobInputStream

2017-07-23 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created HADOOP-14680: - Summary: Azure: IndexOutOfBoundsException in BlockBlobInputStream Key: HADOOP-14680 URL: https://issues.apache.org/jira/browse/HADOOP-14680 Project: Hadoop

[jira] [Assigned] (HADOOP-11572) s3a delete() operation fails during a concurrent delete of child entries

2017-07-18 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-11572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned HADOOP-11572: - Assignee: Steve Loughran (was: Rajesh Balamohan) > s3a delete() operation

[jira] [Assigned] (HADOOP-11572) s3a delete() operation fails during a concurrent delete of child entries

2017-07-18 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-11572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned HADOOP-11572: - Assignee: Rajesh Balamohan (was: Steve Loughran) > s3a delete() operation

[jira] [Created] (HADOOP-14612) Reduce memory copy in BlobOutputStreamInternal::dispatchWrite

2017-06-28 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created HADOOP-14612: - Summary: Reduce memory copy in BlobOutputStreamInternal::dispatchWrite Key: HADOOP-14612 URL: https://issues.apache.org/jira/browse/HADOOP-14612 Project:

[jira] [Commented] (HADOOP-14596) AWS SDK 1.11+ aborts() on close() if > 0 bytes in stream; logs error

2017-06-28 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16066377#comment-16066377 ] Rajesh Balamohan commented on HADOOP-14596: --- Thanks for sharing the patch

[jira] [Commented] (HADOOP-14500) Azure: TestFileSystemOperationExceptionHandling{,MultiThreaded} fails

2017-06-07 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16041527#comment-16041527 ] Rajesh Balamohan commented on HADOOP-14500: --- It was simulation related I believe. When earlier

[jira] [Updated] (HADOOP-14500) Azure: TestFileSystemOperationExceptionHandling{,MultiThreaded} fails

2017-06-06 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-14500: -- Status: Patch Available (was: Open) > Azure:

[jira] [Updated] (HADOOP-14500) Azure: TestFileSystemOperationExceptionHandling{,MultiThreaded} fails

2017-06-06 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-14500: -- Attachment: HADOOP-14500-001.patch Tests were executed against WASB/Japan region.

[jira] [Assigned] (HADOOP-14500) Azure: TestFileSystemOperationExceptionHandling{,MultiThreaded} fails

2017-06-06 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned HADOOP-14500: - Assignee: Rajesh Balamohan > Azure:

[jira] [Commented] (HADOOP-14478) Optimize NativeAzureFsInputStream for positional reads

2017-06-05 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16037914#comment-16037914 ] Rajesh Balamohan commented on HADOOP-14478: --- [~liuml07] - Perf improvement would be observed

[jira] [Updated] (HADOOP-14473) Optimize NativeAzureFileSystem::seek for forward seeks

2017-06-05 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-14473: -- Resolution: Resolved Status: Resolved (was: Patch Available) Closing this

[jira] [Commented] (HADOOP-14478) Optimize NativeAzureFsInputStream for positional reads

2017-06-05 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16037890#comment-16037890 ] Rajesh Balamohan commented on HADOOP-14478: --- Thanks [~liuml07], [~ste...@apache.org]. >

[jira] [Commented] (HADOOP-14478) Optimize NativeAzureFsInputStream for positional reads

2017-06-02 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16035724#comment-16035724 ] Rajesh Balamohan commented on HADOOP-14478: --- Thanks [~liuml07] > Optimize

[jira] [Updated] (HADOOP-14478) Optimize NativeAzureFsInputStream for positional reads

2017-06-02 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-14478: -- Attachment: HADOOP-14478.003.patch Attaching .3 patch to address checkstyle issue

[jira] [Commented] (HADOOP-14473) Optimize NativeAzureFileSystem::seek for forward seeks

2017-06-02 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16034683#comment-16034683 ] Rajesh Balamohan commented on HADOOP-14473: --- Since it was easier to combine this patch with

[jira] [Assigned] (HADOOP-14478) Optimize NativeAzureFsInputStream for positional reads

2017-06-02 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned HADOOP-14478: - Assignee: Rajesh Balamohan > Optimize NativeAzureFsInputStream for positional

[jira] [Updated] (HADOOP-14478) Optimize NativeAzureFsInputStream for positional reads

2017-06-02 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-14478: -- Attachment: HADOOP-14478.002.patch Attaching .2 version with fixes in seek(). Also

[jira] [Updated] (HADOOP-14478) Optimize NativeAzureFsInputStream for positional reads

2017-06-02 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-14478: -- Description: Azure's {{BlobbInputStream}} internally buffers 4 MB of data

[jira] [Updated] (HADOOP-14478) Optimize NativeAzureFsInputStream for positional reads

2017-06-02 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-14478: -- Attachment: HADOOP-14478.001.patch Attaching .1 patch for review. This includes

[jira] [Created] (HADOOP-14478) Optimize NativeAzureFsInputStream for positional reads

2017-06-02 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created HADOOP-14478: - Summary: Optimize NativeAzureFsInputStream for positional reads Key: HADOOP-14478 URL: https://issues.apache.org/jira/browse/HADOOP-14478 Project: Hadoop

[jira] [Updated] (HADOOP-14473) Optimize NativeAzureFileSystem::seek for forward seeks

2017-05-31 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-14473: -- Attachment: HADOOP-14473-001.patch > Optimize NativeAzureFileSystem::seek for forward

[jira] [Created] (HADOOP-14473) Optimize NativeAzureFileSystem::seek for forward seeks

2017-05-31 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created HADOOP-14473: - Summary: Optimize NativeAzureFileSystem::seek for forward seeks Key: HADOOP-14473 URL: https://issues.apache.org/jira/browse/HADOOP-14473 Project: Hadoop

[jira] [Commented] (HADOOP-13926) S3Guard: S3AFileSystem::listLocatedStatus() to employ MetadataStore

2017-04-03 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15954481#comment-15954481 ] Rajesh Balamohan commented on HADOOP-13926: --- Thanks for the patch [~liuml07]. Patch LGTM.

[jira] [Updated] (HADOOP-14154) Set isAuthoritative flag when creating DirListingMetadata in DynamoDBMetaStore

2017-03-09 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-14154: -- Status: Open (was: Patch Available) > Set isAuthoritative flag when creating

[jira] [Commented] (HADOOP-14154) Set isAuthoritative flag when creating DirListingMetadata in DynamoDBMetaStore

2017-03-09 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904013#comment-15904013 ] Rajesh Balamohan commented on HADOOP-14154: --- Thanks for the clarification [~fabbri]. Would

[jira] [Created] (HADOOP-14165) Add S3Guard.dirListingUnion in S3AFileSystem#listFiles, listLocatedStatus

2017-03-08 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created HADOOP-14165: - Summary: Add S3Guard.dirListingUnion in S3AFileSystem#listFiles, listLocatedStatus Key: HADOOP-14165 URL: https://issues.apache.org/jira/browse/HADOOP-14165

[jira] [Updated] (HADOOP-14154) Set isAuthoritative flag when creating DirListingMetadata in DynamoDBMetaStore

2017-03-08 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-14154: -- Attachment: HADOOP-14154-HADOOP-13345.002.patch Fixing checkstyle issues. > Set

[jira] [Updated] (HADOOP-14154) Set isAuthoritative flag when creating DirListingMetadata in DynamoDBMetaStore

2017-03-07 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-14154: -- Status: Patch Available (was: Open) > Set isAuthoritative flag when creating

[jira] [Updated] (HADOOP-14154) Set isAuthoritative flag when creating DirListingMetadata in DynamoDBMetaStore

2017-03-07 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-14154: -- Attachment: HADOOP-14154-HADOOP-13345.001.patch > Set isAuthoritative flag when

[jira] [Created] (HADOOP-14154) Set isAuthoritative flag when creating DirListingMetadata in DynamoDBMetaStore

2017-03-07 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created HADOOP-14154: - Summary: Set isAuthoritative flag when creating DirListingMetadata in DynamoDBMetaStore Key: HADOOP-14154 URL: https://issues.apache.org/jira/browse/HADOOP-14154

[jira] [Commented] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2017-03-07 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15898949#comment-15898949 ] Rajesh Balamohan commented on HADOOP-13914: --- {noformat} S3AFileStatus innerGetFileStatus(final

[jira] [Commented] (HADOOP-14081) S3A: Consider avoiding array copy in S3ABlockOutputStream (ByteArrayBlock)

2017-02-20 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15875140#comment-15875140 ] Rajesh Balamohan commented on HADOOP-14081: --- Thanks [~ste...@apache.org]. I ran with "mvn test

[jira] [Commented] (HADOOP-14081) S3A: Consider avoiding array copy in S3ABlockOutputStream (ByteArrayBlock)

2017-02-17 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871711#comment-15871711 ] Rajesh Balamohan commented on HADOOP-14081: --- Thanks [~ste...@apache.org]. Here are the test

[jira] [Updated] (HADOOP-14081) S3A: Consider avoiding array copy in S3ABlockOutputStream (ByteArrayBlock)

2017-02-15 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-14081: -- Status: Patch Available (was: Open) > S3A: Consider avoiding array copy in

[jira] [Updated] (HADOOP-14081) S3A: Consider avoiding array copy in S3ABlockOutputStream (ByteArrayBlock)

2017-02-15 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-14081: -- Attachment: HADOOP-14081.001.patch > S3A: Consider avoiding array copy in

[jira] [Created] (HADOOP-14081) S3A: Consider avoiding array copy in S3ABlockOutputStream (ByteArrayBlock)

2017-02-14 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created HADOOP-14081: - Summary: S3A: Consider avoiding array copy in S3ABlockOutputStream (ByteArrayBlock) Key: HADOOP-14081 URL: https://issues.apache.org/jira/browse/HADOOP-14081

[jira] [Updated] (HADOOP-14081) S3A: Consider avoiding array copy in S3ABlockOutputStream (ByteArrayBlock)

2017-02-14 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-14081: -- Issue Type: Sub-task (was: Improvement) Parent: HADOOP-13204 > S3A: Consider

[jira] [Commented] (HADOOP-13926) S3Guard: Improve listLocatedStatus

2017-01-10 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816583#comment-15816583 ] Rajesh Balamohan commented on HADOOP-13926: --- Agreed. That would need change in s3guard to

[jira] [Updated] (HADOOP-13926) S3Guard: Improve listLocatedStatus

2016-12-22 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13926: -- Attachment: HADOOP-13926.wip.proto.branch-13345.1.patch > S3Guard: Improve

[jira] [Created] (HADOOP-13936) S3Guard: DynamoDB can go out of sync with S3AFileSystem::delete operation

2016-12-22 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created HADOOP-13936: - Summary: S3Guard: DynamoDB can go out of sync with S3AFileSystem::delete operation Key: HADOOP-13936 URL: https://issues.apache.org/jira/browse/HADOOP-13936

[jira] [Updated] (HADOOP-13934) S3Guard: DynamoDBMetaStore::move could be throwing exception due to BatchWriteItem limits

2016-12-21 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13934: -- Summary: S3Guard: DynamoDBMetaStore::move could be throwing exception due to

[jira] [Commented] (HADOOP-13934) S3Guard: DynamoDBMetaStore::move throws exception with limited info

2016-12-21 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15769320#comment-15769320 ] Rajesh Balamohan commented on HADOOP-13934: --- Suspecting the API limit in batchitem

[jira] [Created] (HADOOP-13934) S3Guard: DynamoDBMetaStore::move throws exception with limited info

2016-12-21 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created HADOOP-13934: - Summary: S3Guard: DynamoDBMetaStore::move throws exception with limited info Key: HADOOP-13934 URL: https://issues.apache.org/jira/browse/HADOOP-13934

[jira] [Created] (HADOOP-13931) S3AGuard: Use BatchWriteItem in DynamoDBMetadataStore::put(DirListingMetadata)

2016-12-21 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created HADOOP-13931: - Summary: S3AGuard: Use BatchWriteItem in DynamoDBMetadataStore::put(DirListingMetadata) Key: HADOOP-13931 URL: https://issues.apache.org/jira/browse/HADOOP-13931

[jira] [Resolved] (HADOOP-13925) S3Guard: NPE when table is already populated in dynamodb and user specifies "fs.s3a.s3guard.ddb.table.create=false"

2016-12-21 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan resolved HADOOP-13925. --- Resolution: Duplicate > S3Guard: NPE when table is already populated in dynamodb

[jira] [Created] (HADOOP-13926) S3Guard: Improve listLocatedStatus

2016-12-21 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created HADOOP-13926: - Summary: S3Guard: Improve listLocatedStatus Key: HADOOP-13926 URL: https://issues.apache.org/jira/browse/HADOOP-13926 Project: Hadoop Common Issue

[jira] [Created] (HADOOP-13925) S3Guard: NPE when table is already populated in dynamodb and user specifies "fs.s3a.s3guard.ddb.table.create=false"

2016-12-21 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created HADOOP-13925: - Summary: S3Guard: NPE when table is already populated in dynamodb and user specifies "fs.s3a.s3guard.ddb.table.create=false" Key: HADOOP-13925 URL:

[jira] [Created] (HADOOP-13757) Remove verifyBuckets overhead in S3AFileSystem::initialize()

2016-10-24 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created HADOOP-13757: - Summary: Remove verifyBuckets overhead in S3AFileSystem::initialize() Key: HADOOP-13757 URL: https://issues.apache.org/jira/browse/HADOOP-13757 Project:

[jira] [Commented] (HADOOP-13727) S3A: Reduce high number of connections to EC2 Instance Metadata Service caused by InstanceProfileCredentialsProvider.

2016-10-21 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15594685#comment-15594685 ] Rajesh Balamohan commented on HADOOP-13727: --- I have tried out this patch in AWS test

[jira] [Commented] (HADOOP-13560) S3ABlockOutputStream to support huge (many GB) file writes

2016-09-29 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15535127#comment-15535127 ] Rajesh Balamohan commented on HADOOP-13560: --- S3ABlockOutputStream::initiateMultiPartUpload()

[jira] [Updated] (HADOOP-13169) Randomize file list in SimpleCopyListing

2016-09-16 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13169: -- Attachment: HADOOP-13169-branch-2-010.patch Thank you [~ste...@apache.org],

[jira] [Updated] (HADOOP-13169) Randomize file list in SimpleCopyListing

2016-09-15 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13169: -- Attachment: HADOOP-13169-branch-2-009.patch Thank you [~cnauroth]. Changes in the

[jira] [Updated] (HADOOP-13169) Randomize file list in SimpleCopyListing

2016-09-14 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13169: -- Status: Patch Available (was: Open) > Randomize file list in SimpleCopyListing >

[jira] [Updated] (HADOOP-13169) Randomize file list in SimpleCopyListing

2016-09-14 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13169: -- Attachment: HADOOP-13169-branch-2-008.patch > Randomize file list in

[jira] [Updated] (HADOOP-13169) Randomize file list in SimpleCopyListing

2016-09-14 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13169: -- Attachment: (was: HADOOP-13169-branch-2-008.patch) > Randomize file list in

[jira] [Updated] (HADOOP-13169) Randomize file list in SimpleCopyListing

2016-09-14 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13169: -- Status: Open (was: Patch Available) > Randomize file list in SimpleCopyListing >

[jira] [Updated] (HADOOP-13169) Randomize file list in SimpleCopyListing

2016-09-14 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13169: -- Attachment: HADOOP-13169-branch-2-008.patch Thanks [~ste...@apache.org]. Added

[jira] [Updated] (HADOOP-13169) Randomize file list in SimpleCopyListing

2016-09-14 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13169: -- Attachment: HADOOP-13169-branch-2-007.patch > Randomize file list in

[jira] [Updated] (HADOOP-13169) Randomize file list in SimpleCopyListing

2016-09-14 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13169: -- Status: Patch Available (was: Open) > Randomize file list in SimpleCopyListing >

[jira] [Updated] (HADOOP-13169) Randomize file list in SimpleCopyListing

2016-09-14 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HADOOP-13169: -- Status: Open (was: Patch Available) > Randomize file list in SimpleCopyListing >

  1   2   >