[GitHub] [hadoop] hadoop-yetus commented on pull request #3170: HDFS-16107.Split RPC configuration to isolate RPC.

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3170: URL: https://github.com/apache/hadoop/pull/3170#issuecomment-24322 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17682) ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17682?focusedWorklogId=631004=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-631004 ] ASF GitHub Bot logged work on HADOOP-17682: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #2975: HADOOP-17682. ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #2975: URL: https://github.com/apache/hadoop/pull/2975#issuecomment-23045 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] hadoop-yetus commented on pull request #1808: MAPREDUCE-7258. HistoryServerRest.html#Task_Counters_API, modify the jobTaskCounters's itemName from taskcounterGroup to taskCounterGrou

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #1808: URL: https://github.com/apache/hadoop/pull/1808#issuecomment-19372 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Updated] (HADOOP-12491) Hadoop-common - Avoid unsafe split and append on fields that might be IPv6 literals

2021-07-28 Thread Hemanth Boyina (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-12491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hemanth Boyina updated HADOOP-12491: Attachment: HADOOP-12491-HADOOP-17800.003.patch > Hadoop-common - Avoid unsafe split and

[GitHub] [hadoop] jojochuang merged pull request #2836: HDFS-15936.Solve BlockSender#sendPacket() does not record SocketTimeout exception.

2021-07-28 Thread GitBox
jojochuang merged pull request #2836: URL: https://github.com/apache/hadoop/pull/2836 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] jojochuang commented on pull request #2836: HDFS-15936.Solve BlockSender#sendPacket() does not record SocketTimeout exception.

2021-07-28 Thread GitBox
jojochuang commented on pull request #2836: URL: https://github.com/apache/hadoop/pull/2836#issuecomment-00975 Merging it per Viraj and cxorm's review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[jira] [Work logged] (HADOOP-17682) ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17682?focusedWorklogId=630980=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630980 ] ASF GitHub Bot logged work on HADOOP-17682: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #2975: HADOOP-17682. ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #2975: URL: https://github.com/apache/hadoop/pull/2975#issuecomment-888794912 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17817) HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17817?focusedWorklogId=630973=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630973 ] ASF GitHub Bot logged work on HADOOP-17817: --- Author: ASF GitHub Bot

[GitHub] [hadoop] mehakmeet commented on pull request #3239: HADOOP-17817. Throw an exception if S3 client-side encryption is enabled on S3Guard enabled bucket

2021-07-28 Thread GitBox
mehakmeet commented on pull request #3239: URL: https://github.com/apache/hadoop/pull/3239#issuecomment-888781408 I ran the tests in S3-CSE ON S3Guard ON, S3-CSE OFF S3-Guard ON, and S3-CSE OFF S3-Guard OFF. More of a mistake that I thought I had run the S3-CSE ON and S3-Guard OFF test

[jira] [Work logged] (HADOOP-17812) NPE in S3AInputStream read() after failure to reconnect to store

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17812?focusedWorklogId=630971=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630971 ] ASF GitHub Bot logged work on HADOOP-17812: --- Author: ASF GitHub Bot

[GitHub] [hadoop] wbo4958 commented on pull request #3222: HADOOP-17812. NPE in S3AInputStream read() after failure to reconnect to store

2021-07-28 Thread GitBox
wbo4958 commented on pull request #3222: URL: https://github.com/apache/hadoop/pull/3222#issuecomment-888777338 Thx @steveloughran, I re-tested the integration test in AWS. and the result can be found at

[jira] [Commented] (HADOOP-17812) NPE in S3AInputStream read() after failure to reconnect to store

2021-07-28 Thread Bobby Wang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17389239#comment-17389239 ] Bobby Wang commented on HADOOP-17812: - Hi Steve, I just found an aws account from my colleague and

[GitHub] [hadoop] hadoop-yetus commented on pull request #3235: HDFS-16143. De-flake TestEditLogTailer#testStandbyTriggersLogRollsWhenTailInProgressEdits

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3235: URL: https://github.com/apache/hadoop/pull/3235#issuecomment-888775284 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Updated] (HADOOP-17812) NPE in S3AInputStream read() after failure to reconnect to store

2021-07-28 Thread Bobby Wang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bobby Wang updated HADOOP-17812: Attachment: failsafe-report.html.gz > NPE in S3AInputStream read() after failure to reconnect to

[jira] [Commented] (HADOOP-17763) DistCp job fails when AM is killed

2021-07-28 Thread Bilwa S T (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17389217#comment-17389217 ] Bilwa S T commented on HADOOP-17763: Hi [~ayushtkn] [~epayne] can you please take a look at updated

[GitHub] [hadoop] jianghuazhu commented on pull request #2836: HDFS-15936.Solve BlockSender#sendPacket() does not record SocketTimeout exception.

2021-07-28 Thread GitBox
jianghuazhu commented on pull request #2836: URL: https://github.com/apache/hadoop/pull/2836#issuecomment-888762016 @cxorm , thank you very much for your work. Can this pr be merged into the trunk branch? If there is still a lack of other work, I am willing to contribute. thank you

[GitHub] [hadoop] hadoop-yetus commented on pull request #3235: HDFS-16143. De-flake TestEditLogTailer#testStandbyTriggersLogRollsWhenTailInProgressEdits

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3235: URL: https://github.com/apache/hadoop/pull/3235#issuecomment-888751847 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] hadoop-yetus commented on pull request #3235: HDFS-16143. De-flake TestEditLogTailer#testStandbyTriggersLogRollsWhenTailInProgressEdits

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3235: URL: https://github.com/apache/hadoop/pull/3235#issuecomment-888747021 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] hadoop-yetus commented on pull request #3235: HDFS-16143. De-flake TestEditLogTailer#testStandbyTriggersLogRollsWhenTailInProgressEdits

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3235: URL: https://github.com/apache/hadoop/pull/3235#issuecomment-888701605 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Resolved] (HADOOP-17819) Add extensions to ProtobufRpcEngine RequestHeaderProto

2021-07-28 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved HADOOP-17819. -- Fix Version/s: 3.2.3 2.10.2 3.4.0

[jira] [Work logged] (HADOOP-17819) Add extensions to ProtobufRpcEngine RequestHeaderProto

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17819?focusedWorklogId=630816=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630816 ] ASF GitHub Bot logged work on HADOOP-17819: --- Author: ASF GitHub Bot

[GitHub] [hadoop] shvachko commented on pull request #3242: HADOOP-17819. Add extensions to ProtobufRpcEngine RequestHeaderProto

2021-07-28 Thread GitBox
shvachko commented on pull request #3242: URL: https://github.com/apache/hadoop/pull/3242#issuecomment-888661912 +1 Will be committing this shortly -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[jira] [Work logged] (HADOOP-17811) ABFS ExponentialRetryPolicy doesn't pick up configuration values

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17811?focusedWorklogId=630799=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630799 ] ASF GitHub Bot logged work on HADOOP-17811: --- Author: ASF GitHub Bot

[GitHub] [hadoop] brianloss commented on pull request #3221: HADOOP-17811: ABFS ExponentialRetryPolicy doesn't pick up configuration values

2021-07-28 Thread GitBox
brianloss commented on pull request #3221: URL: https://github.com/apache/hadoop/pull/3221#issuecomment-888635485 > Can you do the cherrypick and test of the branch-3.3 locally, let me know if it's all good there and I'll merge it in there too. @steveloughran my local cherry pick

[GitHub] [hadoop] hadoop-yetus commented on pull request #3246: YARN-10848. Vcore allocation problem with DefaultResourceCalculator

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3246: URL: https://github.com/apache/hadoop/pull/3246#issuecomment-888614796 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] hadoop-yetus commented on pull request #2971: MAPREDUCE-7341. Intermediate Manifest Committer

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #2971: URL: https://github.com/apache/hadoop/pull/2971#issuecomment-888603337 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17812) NPE in S3AInputStream read() after failure to reconnect to store

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17812?focusedWorklogId=630731=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630731 ] ASF GitHub Bot logged work on HADOOP-17812: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #3222: HADOOP-17812. NPE in S3AInputStream read() after failure to reconnect to store

2021-07-28 Thread GitBox
steveloughran commented on pull request #3222: URL: https://github.com/apache/hadoop/pull/3222#issuecomment-888573781 bq. BTW, I will upload the integration test results in JIRA later. don't need the full results, just add a comment here about where you tested (e.g.

[jira] [Work logged] (HADOOP-17139) Re-enable optimized copyFromLocal implementation in S3AFileSystem

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17139?focusedWorklogId=630730=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630730 ] ASF GitHub Bot logged work on HADOOP-17139: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on a change in pull request #3101: HADOOP-17139 Re-enable optimized copyFromLocal implementation in S3AFileSystem

2021-07-28 Thread GitBox
steveloughran commented on a change in pull request #3101: URL: https://github.com/apache/hadoop/pull/3101#discussion_r678596071 ## File path: hadoop-common-project/hadoop-common/src/site/markdown/filesystem/filesystem.md ## @@ -1427,57 +1427,91 @@ set to TRUE. If destination

[jira] [Updated] (HADOOP-17811) ABFS ExponentialRetryPolicy doesn't pick up configuration values

2021-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17811: Summary: ABFS ExponentialRetryPolicy doesn't pick up configuration values (was: ABFS:

[jira] [Work logged] (HADOOP-17811) ABFS: Allow all ExponentialRetryPolicy properties to be configured

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17811?focusedWorklogId=630729=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630729 ] ASF GitHub Bot logged work on HADOOP-17811: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #3221: HADOOP-17811: ABFS ExponentialRetryPolicy doesn't pick up configuration values

2021-07-28 Thread GitBox
steveloughran commented on pull request #3221: URL: https://github.com/apache/hadoop/pull/3221#issuecomment-888563527 > ITestAzureBlobFileSystemLease talk to @billierinaldi there. w.r.t ITestAbfsStreamStatistics, if there's no JIRA on that test failure, file one, with the stack

[jira] [Comment Edited] (HADOOP-17612) Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0

2021-07-28 Thread Viraj Jasani (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388979#comment-17388979 ] Viraj Jasani edited comment on HADOOP-17612 at 7/28/21, 7:24 PM: - Thank

[jira] [Work logged] (HADOOP-17811) ABFS: Allow all ExponentialRetryPolicy properties to be configured

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17811?focusedWorklogId=630728=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630728 ] ASF GitHub Bot logged work on HADOOP-17811: --- Author: ASF GitHub Bot

[GitHub] [hadoop] brianloss commented on pull request #3221: HADOOP-17811: ABFS ExponentialRetryPolicy doesn't pick up configuration values

2021-07-28 Thread GitBox
brianloss commented on pull request #3221: URL: https://github.com/apache/hadoop/pull/3221#issuecomment-888561184 > +1 for this, merging to trunk and then soon 3.3.2. > > interesting you are seeing all the test failures. I don't go near Oauth myself (long story). Yeah,

[jira] [Updated] (HADOOP-17811) ABFS: Allow all ExponentialRetryPolicy properties to be configured

2021-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17811: Fix Version/s: 3.4.0 > ABFS: Allow all ExponentialRetryPolicy properties to be

[jira] [Resolved] (HADOOP-17811) ABFS: Allow all ExponentialRetryPolicy properties to be configured

2021-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17811. - Resolution: Fixed > ABFS: Allow all ExponentialRetryPolicy properties to be configured

[jira] [Work logged] (HADOOP-17811) ABFS: Allow all ExponentialRetryPolicy properties to be configured

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17811?focusedWorklogId=630726=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630726 ] ASF GitHub Bot logged work on HADOOP-17811: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran merged pull request #3221: HADOOP-17811: ABFS ExponentialRetryPolicy doesn't pick up configuration values

2021-07-28 Thread GitBox
steveloughran merged pull request #3221: URL: https://github.com/apache/hadoop/pull/3221 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Work logged] (HADOOP-17811) ABFS: Allow all ExponentialRetryPolicy properties to be configured

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17811?focusedWorklogId=630722=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630722 ] ASF GitHub Bot logged work on HADOOP-17811: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on a change in pull request #3221: HADOOP-17811: Configure ExponentialRetryPolicy

2021-07-28 Thread GitBox
steveloughran commented on a change in pull request #3221: URL: https://github.com/apache/hadoop/pull/3221#discussion_r678585653 ## File path: hadoop-tools/hadoop-azure/src/site/markdown/testing_azure.md ## @@ -448,7 +448,7 @@ use requires the presence of secret credentials,

[jira] [Commented] (HADOOP-17612) Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0

2021-07-28 Thread Viraj Jasani (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388979#comment-17388979 ] Viraj Jasani commented on HADOOP-17612: --- Thank you [~eolivelli] for Curator release! Here is the

[jira] [Work logged] (HADOOP-17817) HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17817?focusedWorklogId=630718=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630718 ] ASF GitHub Bot logged work on HADOOP-17817: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #3239: HADOOP-17817. Throw an exception if S3 client-side encryption is enabled on S3Guard enabled bucket

2021-07-28 Thread GitBox
steveloughran commented on pull request #3239: URL: https://github.com/apache/hadoop/pull/3239#issuecomment-888553613 hmm. Yes, that'll be an interesting problem. Either test setup() checks for CSE on and skips the s3guard-enabled tests, or we catch the raised PathIOE and covert

[jira] [Commented] (HADOOP-17784) hadoop-aws landsat-pds test bucket will be deleted after Jul 1, 2021

2021-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388963#comment-17388963 ] Steve Loughran commented on HADOOP-17784: - update, we're trying to convince the relevant AWS

[GitHub] [hadoop] hadoop-yetus commented on pull request #2971: MAPREDUCE-7341. Intermediate Manifest Committer

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #2971: URL: https://github.com/apache/hadoop/pull/2971#issuecomment-888535151 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] minni31 opened a new pull request #3246: YARN-10848. Vcore allocation problem with DefaultResourceCalculator

2021-07-28 Thread GitBox
minni31 opened a new pull request #3246: URL: https://github.com/apache/hadoop/pull/3246 ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number. (e.g.

[jira] [Updated] (HADOOP-17814) Provide fallbacks for identity/cost providers and backoff enable

2021-07-28 Thread Takanobu Asanuma (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takanobu Asanuma updated HADOOP-17814: -- Fix Version/s: 3.4.0 Resolution: Fixed Status: Resolved (was:

[jira] [Work logged] (HADOOP-17814) Provide fallbacks for identity/cost providers and backoff enable

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17814?focusedWorklogId=630654=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630654 ] ASF GitHub Bot logged work on HADOOP-17814: --- Author: ASF GitHub Bot

[GitHub] [hadoop] tasanuma commented on pull request #3230: HADOOP-17814. Provide fallbacks for identity/cost providers and backoff enable

2021-07-28 Thread GitBox
tasanuma commented on pull request #3230: URL: https://github.com/apache/hadoop/pull/3230#issuecomment-888477003 Thanks for your contribution, @virajjasani! Thanks for your review, @jojochuang! -- This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Work logged] (HADOOP-17814) Provide fallbacks for identity/cost providers and backoff enable

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17814?focusedWorklogId=630653=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630653 ] ASF GitHub Bot logged work on HADOOP-17814: --- Author: ASF GitHub Bot

[GitHub] [hadoop] tasanuma merged pull request #3230: HADOOP-17814. Provide fallbacks for identity/cost providers and backoff enable

2021-07-28 Thread GitBox
tasanuma merged pull request #3230: URL: https://github.com/apache/hadoop/pull/3230 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] hadoop-yetus commented on pull request #3235: HDFS-16143. De-flake TestEditLogTailer#testStandbyTriggersLogRollsWhenTailInProgressEdits

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3235: URL: https://github.com/apache/hadoop/pull/3235#issuecomment-888463500 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17628) Distcp contract test is really slow with ABFS and S3A; timing out

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17628?focusedWorklogId=630639=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630639 ] ASF GitHub Bot logged work on HADOOP-17628: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #3240: HADOOP-17628. Distcp contract test is really slow with ABFS and S3A; timing out.

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3240: URL: https://github.com/apache/hadoop/pull/3240#issuecomment-888452283 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17817) HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17817?focusedWorklogId=630632=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630632 ] ASF GitHub Bot logged work on HADOOP-17817: --- Author: ASF GitHub Bot

[GitHub] [hadoop] mehakmeet edited a comment on pull request #3239: HADOOP-17817. Throw an exception if S3 client-side encryption is enabled on S3Guard enabled bucket

2021-07-28 Thread GitBox
mehakmeet edited a comment on pull request #3239: URL: https://github.com/apache/hadoop/pull/3239#issuecomment-888421643 Seems like, I've broken tests for S3-CSE ON because alot of S3Guard tests don't require your bucket to be s3guard enabled, and force the metastore to be dynamoDB. My

[jira] [Work logged] (HADOOP-17817) HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17817?focusedWorklogId=630607=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630607 ] ASF GitHub Bot logged work on HADOOP-17817: --- Author: ASF GitHub Bot

[GitHub] [hadoop] mehakmeet commented on pull request #3239: HADOOP-17817. Throw an exception if S3 client-side encryption is enabled on S3Guard enabled bucket

2021-07-28 Thread GitBox
mehakmeet commented on pull request #3239: URL: https://github.com/apache/hadoop/pull/3239#issuecomment-888421643 Seems like, I've broken tests for S3-CSE ON because some S3Guard tests don't require your bucket to be s3guard enabled, and force the metastore to be dynamoDB. My lapse on

[GitHub] [hadoop] hadoop-yetus commented on pull request #3235: HDFS-16143. De-flake TestEditLogTailer#testStandbyTriggersLogRollsWhenTailInProgressEdits

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3235: URL: https://github.com/apache/hadoop/pull/3235#issuecomment-888419375 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Resolved] (HADOOP-17789) S3 read performance with Spark with Hadoop 3.3.1 is slower than older Hadoop

2021-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17789. - Resolution: Works for Me > S3 read performance with Spark with Hadoop 3.3.1 is slower

[jira] [Commented] (HADOOP-17789) S3 read performance with Spark with Hadoop 3.3.1 is slower than older Hadoop

2021-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388847#comment-17388847 ] Steve Loughran commented on HADOOP-17789: - bq. Can we use the latest one wildfly-openssl 2.1.x

[GitHub] [hadoop] hadoop-yetus commented on pull request #3235: HDFS-16143. De-flake TestEditLogTailer#testStandbyTriggersLogRollsWhenTailInProgressEdits

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3235: URL: https://github.com/apache/hadoop/pull/3235#issuecomment-888374231 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Resolved] (HADOOP-17817) HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled

2021-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17817. - Resolution: Fixed > HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled

[jira] [Updated] (HADOOP-17817) HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled

2021-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17817: Component/s: fs/s3 > HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled >

[jira] [Updated] (HADOOP-13887) Encrypt S3A data client-side with AWS SDK (S3-CSE)

2021-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13887: Description: Expose the client-side encryption option documented in Amazon S3

[jira] [Updated] (HADOOP-17817) HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled

2021-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17817: Fix Version/s: 3.4.0 > HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled

[jira] [Updated] (HADOOP-17817) HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled

2021-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17817: Priority: Major (was: Minor) > HADOOP-17817. S3A to raise IOE if both S3-CSE and

[jira] [Updated] (HADOOP-17817) HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled

2021-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17817: Affects Version/s: 3.4.0 > HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard

[jira] [Updated] (HADOOP-17817) HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled

2021-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17817: Priority: Minor (was: Major) > HADOOP-17817. S3A to raise IOE if both S3-CSE and

[jira] [Updated] (HADOOP-17817) HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled

2021-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17817: Summary: HADOOP-17817. S3A to raise IOE if both S3-CSE and S3Guard enabled (was: Throw

[jira] [Work logged] (HADOOP-17817) Throw an exception if S3 client-side encryption is enabled on S3Guard enabled bucket

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17817?focusedWorklogId=630561=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630561 ] ASF GitHub Bot logged work on HADOOP-17817: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran merged pull request #3239: HADOOP-17817. Throw an exception if S3 client-side encryption is enabled on S3Guard enabled bucket

2021-07-28 Thread GitBox
steveloughran merged pull request #3239: URL: https://github.com/apache/hadoop/pull/3239 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] hadoop-yetus commented on pull request #3243: HDFS-14529. SetTimes to throw FileNotFoundException if inode is not found

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3243: URL: https://github.com/apache/hadoop/pull/3243#issuecomment-888337965 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Commented] (HADOOP-17789) S3 read performance with Spark with Hadoop 3.3.1 is slower than older Hadoop

2021-07-28 Thread Arghya Saha (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388790#comment-17388790 ] Arghya Saha commented on HADOOP-17789: -- [~ste...@apache.org] Thanks again for your suggestions,

[GitHub] [hadoop] hadoop-yetus commented on pull request #3220: YARN-10355. Refactor NM ContainerLaunch.java#orderEnvByDependencies

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3220: URL: https://github.com/apache/hadoop/pull/3220#issuecomment-888301811 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] hadoop-yetus commented on pull request #3195: YARN-10459. containerLaunchedOnNode method not need to hold scheduler…

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3195: URL: https://github.com/apache/hadoop/pull/3195#issuecomment-888281625 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17628) Distcp contract test is really slow with ABFS and S3A; timing out

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17628?focusedWorklogId=630485=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630485 ] ASF GitHub Bot logged work on HADOOP-17628: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #3240: HADOOP-17628. Distcp contract test is really slow with ABFS and S3A; timing out.

2021-07-28 Thread GitBox
steveloughran commented on pull request #3240: URL: https://github.com/apache/hadoop/pull/3240#issuecomment-888247513 Latest release * Address review comments * log IOStats after each test case. Important: as the cached FS retains statistics, the numbers get bigger over

[GitHub] [hadoop] hadoop-yetus commented on pull request #3235: HDFS-16143. De-flake TestEditLogTailer#testStandbyTriggersLogRollsWhenTailInProgressEdits

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3235: URL: https://github.com/apache/hadoop/pull/3235#issuecomment-888263127 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17628) Distcp contract test is really slow with ABFS and S3A; timing out

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17628?focusedWorklogId=630471=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630471 ] ASF GitHub Bot logged work on HADOOP-17628: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on a change in pull request #3240: HADOOP-17628. Distcp contract test is really slow with ABFS and S3A; timing out.

2021-07-28 Thread GitBox
steveloughran commented on a change in pull request #3240: URL: https://github.com/apache/hadoop/pull/3240#discussion_r678227498 ## File path: hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/contract/AbstractContractDistCpTest.java ## @@ -532,13 +549,15 @@

[jira] [Work logged] (HADOOP-17628) Distcp contract test is really slow with ABFS and S3A; timing out

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17628?focusedWorklogId=629905=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-629905 ] ASF GitHub Bot logged work on HADOOP-17628: --- Author: ASF GitHub Bot

[GitHub] [hadoop] mehakmeet commented on a change in pull request #3240: HADOOP-17628. Distcp contract test is really slow with ABFS and S3A; timing out.

2021-07-28 Thread GitBox
mehakmeet commented on a change in pull request #3240: URL: https://github.com/apache/hadoop/pull/3240#discussion_r678208587 ## File path: hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/contract/AbstractContractDistCpTest.java ## @@ -532,13 +549,15 @@

[jira] [Work logged] (HADOOP-17628) Distcp contract test is really slow with ABFS and S3A; timing out

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17628?focusedWorklogId=629408=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-629408 ] ASF GitHub Bot logged work on HADOOP-17628: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on a change in pull request #3240: HADOOP-17628. Distcp contract test is really slow with ABFS and S3A; timing out.

2021-07-28 Thread GitBox
steveloughran commented on a change in pull request #3240: URL: https://github.com/apache/hadoop/pull/3240#discussion_r678208290 ## File path: hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/contract/AbstractContractDistCpTest.java ## @@ -758,7 +828,7 @@

[jira] [Work logged] (HADOOP-17628) Distcp contract test is really slow with ABFS and S3A; timing out

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17628?focusedWorklogId=629248=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-629248 ] ASF GitHub Bot logged work on HADOOP-17628: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #3240: HADOOP-17628. Distcp contract test is really slow with ABFS and S3A; timing out.

2021-07-28 Thread GitBox
hadoop-yetus removed a comment on pull request #3240: URL: https://github.com/apache/hadoop/pull/3240#issuecomment-887887092 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:|

[jira] [Work logged] (HADOOP-17628) Distcp contract test is really slow with ABFS and S3A; timing out

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17628?focusedWorklogId=629011=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-629011 ] ASF GitHub Bot logged work on HADOOP-17628: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran edited a comment on pull request #3240: HADOOP-17628. Distcp contract test is really slow with ABFS and S3A; timing out.

2021-07-28 Thread GitBox
steveloughran edited a comment on pull request #3240: URL: https://github.com/apache/hadoop/pull/3240#issuecomment-888217775 optional OptionalTestHDFSContractDistCp runs fine from IDE'; 4 minutes. All the performance issues of the HDFS contract are related to the large file tests

[jira] [Work logged] (HADOOP-17628) Distcp contract test is really slow with ABFS and S3A; timing out

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17628?focusedWorklogId=629009=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-629009 ] ASF GitHub Bot logged work on HADOOP-17628: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #3240: HADOOP-17628. Distcp contract test is really slow with ABFS and S3A; timing out.

2021-07-28 Thread GitBox
steveloughran commented on pull request #3240: URL: https://github.com/apache/hadoop/pull/3240#issuecomment-888217775 optional OptionalTestHDFSContractDistCp runs fine from IDE'; 4 minutes. Probably mini DFSCluster overheads -- This is an automated message from the Apache Git Service.

[jira] [Work logged] (HADOOP-17628) Distcp contract test is really slow with ABFS and S3A; timing out

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17628?focusedWorklogId=629008=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-629008 ] ASF GitHub Bot logged work on HADOOP-17628: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #3240: HADOOP-17628. Distcp contract test is really slow with ABFS and S3A; timing out.

2021-07-28 Thread GitBox
steveloughran commented on pull request #3240: URL: https://github.com/apache/hadoop/pull/3240#issuecomment-888216640 Will add to release notes the fact you can turn off the large file uploads through ```xml scale.test.distcp.file.size.kb 0 ``` This is

[jira] [Work logged] (HADOOP-17817) Throw an exception if S3 client-side encryption is enabled on S3Guard enabled bucket

2021-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17817?focusedWorklogId=628995=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-628995 ] ASF GitHub Bot logged work on HADOOP-17817: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #3239: HADOOP-17817. Throw an exception if S3 client-side encryption is enabled on S3Guard enabled bucket

2021-07-28 Thread GitBox
hadoop-yetus commented on pull request #3239: URL: https://github.com/apache/hadoop/pull/3239#issuecomment-888206357 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Commented] (HADOOP-17612) Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0

2021-07-28 Thread Enrico Olivelli (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388672#comment-17388672 ] Enrico Olivelli commented on HADOOP-17612: -- Curator 5.2.0 has been released, with ZookKeeper

  1   2   >