[GitHub] [hadoop] steveloughran opened a new pull request, #4909: HADOOP-18456. NullPointerException in ObjectListingIterator.

2022-09-19 Thread GitBox
steveloughran opened a new pull request, #4909: URL: https://github.com/apache/hadoop/pull/4909 The fix for HADOOP-18456/IMPALA-11592 is, if our hypothesis is correct, in WeakReferenceMap.create() where a strong reference to the new value is kept in a local variable *and referred

[jira] [Updated] (HADOOP-18448) s3a endpoint per bucket configuration in pyspark is ignored (on EMR

2022-09-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18448: Summary: s3a endpoint per bucket configuration in pyspark is ignored (on EMR (was: s3a

[GitHub] [hadoop] hadoop-yetus commented on pull request #4908: YARN-11308. Router Page display the db username and password in mask mode.

2022-09-19 Thread GitBox
hadoop-yetus commented on PR #4908: URL: https://github.com/apache/hadoop/pull/4908#issuecomment-1251199268 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Resolved] (HADOOP-18448) s3a endpoint per bucket configuration in pyspark is ignored

2022-09-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18448. - Resolution: Invalid > s3a endpoint per bucket configuration in pyspark is ignored >

[GitHub] [hadoop] slfan1989 opened a new pull request, #4908: YARN-11308. Router Page display the db username and password in mask mode.

2022-09-19 Thread GitBox
slfan1989 opened a new pull request, #4908: URL: https://github.com/apache/hadoop/pull/4908 JIRA: YARN-11308. Router Page display the db username and password in mask mode. When using YRAN-Federation's SQLFederationStateStore, we need to configure

[jira] [Updated] (HADOOP-18448) s3a endpoint per bucket configuration in pyspark is ignored on EMR

2022-09-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18448: Summary: s3a endpoint per bucket configuration in pyspark is ignored on EMR (was: s3a

[jira] [Comment Edited] (HADOOP-18448) s3a endpoint per bucket configuration in pyspark is ignored

2022-09-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606558#comment-17606558 ] Steve Loughran edited comment on HADOOP-18448 at 9/19/22 1:02 PM: --

[jira] [Updated] (HADOOP-18448) s3a endpoint per bucket configuration in pyspark is ignored

2022-09-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18448: Summary: s3a endpoint per bucket configuration in pyspark is ignored (was: s3a endpoint

[jira] [Updated] (HADOOP-18448) s3a endpoint per bucket configuration in pyspark is ignored

2022-09-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18448: Affects Version/s: (was: 3.2.1) > s3a endpoint per bucket configuration in pyspark

[jira] [Updated] (HADOOP-18448) s3a endpoint per bucket configuration in pyspark is ignored

2022-09-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18448: Environment: Amazon EMR emr-6.5.0 cluster > s3a endpoint per bucket configuration in

[GitHub] [hadoop] steveloughran commented on a diff in pull request #4907: HADOOP-18457: ABFS:Enable throttling at account level

2022-09-19 Thread GitBox
steveloughran commented on code in PR #4907: URL: https://github.com/apache/hadoop/pull/4907#discussion_r974227285 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientThrottlingInterceptFactory.java: ## @@ -0,0 +1,59 @@ +/** + * Licensed

[jira] [Commented] (HADOOP-18457) ABFS: Support for account level throttling

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606595#comment-17606595 ] ASF GitHub Bot commented on HADOOP-18457: - steveloughran commented on code in PR #4907: URL:

[jira] [Assigned] (HADOOP-18456) NullPointerException in ObjectListingIterator's constructor

2022-09-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-18456: --- Assignee: Steve Loughran > NullPointerException in ObjectListingIterator's

[jira] [Commented] (HADOOP-18456) NullPointerException in ObjectListingIterator's constructor

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606651#comment-17606651 ] ASF GitHub Bot commented on HADOOP-18456: - steveloughran opened a new pull request, #4909: URL:

[jira] [Updated] (HADOOP-18456) NullPointerException in ObjectListingIterator's constructor

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18456: Labels: pull-request-available (was: ) > NullPointerException in

[jira] [Commented] (HADOOP-18456) NullPointerException in ObjectListingIterator's constructor

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606652#comment-17606652 ] ASF GitHub Bot commented on HADOOP-18456: - steveloughran commented on PR #4909: URL:

[GitHub] [hadoop] steveloughran commented on pull request #4909: HADOOP-18456. NullPointerException in ObjectListingIterator.

2022-09-19 Thread GitBox
steveloughran commented on PR #4909: URL: https://github.com/apache/hadoop/pull/4909#issuecomment-1251207353 s3a testing in progress -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[jira] [Commented] (HADOOP-18448) s3a endpoint per bucket configuration in pyspark is ignored

2022-09-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606558#comment-17606558 ] Steve Loughran commented on HADOOP-18448: - this is emr; emr's problem. use their s3a connector.

[GitHub] [hadoop] hadoop-yetus commented on pull request #4039: HADOOP-18146: ABFS: Added changes for expect hundred continue header

2022-09-19 Thread GitBox
hadoop-yetus commented on PR #4039: URL: https://github.com/apache/hadoop/pull/4039#issuecomment-1251012502 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18146) ABFS: Add changes for expect hundred continue header with append requests

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606597#comment-17606597 ] ASF GitHub Bot commented on HADOOP-18146: - hadoop-yetus commented on PR #4039: URL:

[GitHub] [hadoop] xkrogen commented on pull request #4900: HADOOP-18118.fix test case fail

2022-09-19 Thread GitBox
xkrogen commented on PR #4900: URL: https://github.com/apache/hadoop/pull/4900#issuecomment-1251231000 LGTM thanks for fixing this! Merging to trunk. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hadoop] ashutoshcipher commented on pull request #4910: YARN-11262: [hadoop-yarn-server-resourcemanager] Upgrade to Junit 5

2022-09-19 Thread GitBox
ashutoshcipher commented on PR #4910: URL: https://github.com/apache/hadoop/pull/4910#issuecomment-1251343476 Hi @andreoss, Thanks for you PR. I have already been working on it in my local and make some good progress as well. It would be great if you can ask on JIRA before starting the

[GitHub] [hadoop] andreoss commented on pull request #4910: YARN-11262: [hadoop-yarn-server-resourcemanager] Upgrade to Junit 5

2022-09-19 Thread GitBox
andreoss commented on PR #4910: URL: https://github.com/apache/hadoop/pull/4910#issuecomment-1251360519 @ashutoshcipher Understood. Any chance this can be merged? In this PR, only `Parameterized` tests are left intact, the rest are converted to Junit 5. The `Parameterized` tests require

[GitHub] [hadoop] hadoop-yetus commented on pull request #4909: HADOOP-18456. NullPointerException in ObjectListingIterator.

2022-09-19 Thread GitBox
hadoop-yetus commented on PR #4909: URL: https://github.com/apache/hadoop/pull/4909#issuecomment-1251465207 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] goiri merged pull request #4852: YARN-11293. [Federation] StoreNewMasterKey/removeStoredMasterKey With MemoryStateStore.

2022-09-19 Thread GitBox
goiri merged PR #4852: URL: https://github.com/apache/hadoop/pull/4852 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] ashutoshcipher commented on pull request #4906: HADOOP-18443. Upgrade snakeyaml to 1.32

2022-09-19 Thread GitBox
ashutoshcipher commented on PR #4906: URL: https://github.com/apache/hadoop/pull/4906#issuecomment-1251512085 ``` hadoop.http.TestHttpServer ``` is unrelated to this change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[jira] [Commented] (HADOOP-18443) Upgrade snakeyaml to 1.32

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606754#comment-17606754 ] ASF GitHub Bot commented on HADOOP-18443: - ashutoshcipher commented on PR #4906: URL:

[jira] [Commented] (HADOOP-18118) Fix KMS Accept Queue Size default value to 500

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606661#comment-17606661 ] ASF GitHub Bot commented on HADOOP-18118: - xkrogen commented on PR #4900: URL:

[GitHub] [hadoop] xkrogen merged pull request #4900: HADOOP-18118. [Follow on] Fix test failure in TestHttpServer

2022-09-19 Thread GitBox
xkrogen merged PR #4900: URL: https://github.com/apache/hadoop/pull/4900 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-18118) Fix KMS Accept Queue Size default value to 500

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606663#comment-17606663 ] ASF GitHub Bot commented on HADOOP-18118: - xkrogen merged PR #4900: URL:

[jira] [Commented] (HADOOP-18013) ABFS: add cloud trash policy with per-schema policy selection

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606660#comment-17606660 ] ASF GitHub Bot commented on HADOOP-18013: - steveloughran commented on code in PR #4729: URL:

[GitHub] [hadoop] steveloughran commented on a diff in pull request #4729: HADOOP-18013. ABFS: add cloud trash policy with per-schema policy selection

2022-09-19 Thread GitBox
steveloughran commented on code in PR #4729: URL: https://github.com/apache/hadoop/pull/4729#discussion_r974432220 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CloudStoreTrashPolicy.java: ## @@ -0,0 +1,220 @@ +/* + * Licensed to the Apache Software

[GitHub] [hadoop] hadoop-yetus commented on pull request #4873: Upgrade snakeyaml to 1.32

2022-09-19 Thread GitBox
hadoop-yetus commented on PR #4873: URL: https://github.com/apache/hadoop/pull/4873#issuecomment-1251286555 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #4906: HADOOP-18443. Upgrade snakeyaml to 1.32

2022-09-19 Thread GitBox
hadoop-yetus commented on PR #4906: URL: https://github.com/apache/hadoop/pull/4906#issuecomment-1251441934 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18443) Upgrade snakeyaml to 1.32

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606727#comment-17606727 ] ASF GitHub Bot commented on HADOOP-18443: - hadoop-yetus commented on PR #4906: URL:

[GitHub] [hadoop] goiri commented on a diff in pull request #4908: YARN-11308. Router Page display the db username and password in mask mode.

2022-09-19 Thread GitBox
goiri commented on code in PR #4908: URL: https://github.com/apache/hadoop/pull/4908#discussion_r974648275 ## hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfServlet.java: ## @@ -59,6 +59,8 @@ public class TestConfServlet { new HashMap();

[GitHub] [hadoop] goiri merged pull request #4890: HDFS-16772. refreshHostsReader should use the latest configuration

2022-09-19 Thread GitBox
goiri merged PR #4890: URL: https://github.com/apache/hadoop/pull/4890 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] pjfanning commented on pull request #4578: HADOOP-18341: upgrade commons-configuration2 to 2.8.0

2022-09-19 Thread GitBox
pjfanning commented on PR #4578: URL: https://github.com/apache/hadoop/pull/4578#issuecomment-1251577861 Thanks @ashutoshcipher - thanks for looking. Just in case, this is from the issue description. Current version 2.1.1 has no CVEs but all higher versions have CVEs except for the

[jira] [Updated] (HADOOP-18456) NullPointerException in ObjectListingIterator's constructor

2022-09-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18456: Affects Version/s: 3.3.9 > NullPointerException in ObjectListingIterator's constructor >

[GitHub] [hadoop] andreoss opened a new pull request, #4910: YARN-11262: [hadoop-yarn-server-resourcemanager] Upgrade to Junit 5

2022-09-19 Thread GitBox
andreoss opened a new pull request, #4910: URL: https://github.com/apache/hadoop/pull/4910 ### Description of PR Upgrade to Junit 5 ### How was this patch tested? mvn test ### For code changes: - [x] Does the title or this PR starts with the

[GitHub] [hadoop] ashutoshcipher commented on pull request #4578: HADOOP-18341: upgrade commons-configuration2 to 2.8.0

2022-09-19 Thread GitBox
ashutoshcipher commented on PR #4578: URL: https://github.com/apache/hadoop/pull/4578#issuecomment-1251580589 Thanks @pjfanning for more explanation about this upgrade. Given >Current version 2.1.1 has no CVEs but all higher versions have CVEs except for the latest release

[GitHub] [hadoop] hadoop-yetus commented on pull request #4873: Upgrade snakeyaml to 1.32

2022-09-19 Thread GitBox
hadoop-yetus commented on PR #4873: URL: https://github.com/apache/hadoop/pull/4873#issuecomment-1251304147 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] pjfanning commented on pull request #4578: HADOOP-18341: upgrade commons-configuration2 to 2.8.0

2022-09-19 Thread GitBox
pjfanning commented on PR #4578: URL: https://github.com/apache/hadoop/pull/4578#issuecomment-1251547177 @ashutoshcipher would you be able to review this? - I've seen you've been busy working on other lib upgrades. This PR is only aimed at hadoop 3.4 and not something that needs to go to

[jira] [Commented] (HADOOP-18341) upgrade to commons-configuration2 2.8.0

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606763#comment-17606763 ] ASF GitHub Bot commented on HADOOP-18341: - pjfanning commented on PR #4578: URL:

[jira] [Commented] (HADOOP-18341) upgrade to commons-configuration2 2.8.0

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606772#comment-17606772 ] ASF GitHub Bot commented on HADOOP-18341: - pjfanning commented on PR #4578: URL:

[jira] [Commented] (HADOOP-18456) NullPointerException in ObjectListingIterator's constructor

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606736#comment-17606736 ] ASF GitHub Bot commented on HADOOP-18456: - hadoop-yetus commented on PR #4909: URL:

[GitHub] [hadoop] goiri merged pull request #4899: YARN-11283. [Federation] Fix Typo of NodeManager AMRMProxy.

2022-09-19 Thread GitBox
goiri merged PR #4899: URL: https://github.com/apache/hadoop/pull/4899 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] goiri commented on a diff in pull request #4897: YARN-11306. [Federation] Refactor NM#FederationInterceptor#recover Code.

2022-09-19 Thread GitBox
goiri commented on code in PR #4897: URL: https://github.com/apache/hadoop/pull/4897#discussion_r974657324 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/amrmproxy/FederationInterceptor.java:

[jira] [Commented] (HADOOP-18341) upgrade to commons-configuration2 2.8.0

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606769#comment-17606769 ] ASF GitHub Bot commented on HADOOP-18341: - ashutoshcipher commented on PR #4578: URL:

[GitHub] [hadoop] ashutoshcipher commented on pull request #4578: HADOOP-18341: upgrade commons-configuration2 to 2.8.0

2022-09-19 Thread GitBox
ashutoshcipher commented on PR #4578: URL: https://github.com/apache/hadoop/pull/4578#issuecomment-1251557986 Thanks @pjfanning. I can help in reviewing. My only concern here is that - As there is no security related issue or any major bug encountered in the current version. Is it worth

[jira] [Commented] (HADOOP-18341) upgrade to commons-configuration2 2.8.0

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606773#comment-17606773 ] ASF GitHub Bot commented on HADOOP-18341: - ashutoshcipher commented on PR #4578: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #4910: YARN-11262: [hadoop-yarn-server-resourcemanager] Upgrade to Junit 5

2022-09-19 Thread GitBox
hadoop-yetus commented on PR #4910: URL: https://github.com/apache/hadoop/pull/4910#issuecomment-1251639019 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] omalley opened a new pull request, #4911: HDFS-16778. Separate out the logger for which DN was picked by a DFSInputStream.

2022-09-19 Thread GitBox
omalley opened a new pull request, #4911: URL: https://github.com/apache/hadoop/pull/4911 Just adds a new logger. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hadoop] slfan1989 commented on pull request #4852: YARN-11293. [Federation] StoreNewMasterKey/removeStoredMasterKey With MemoryStateStore.

2022-09-19 Thread GitBox
slfan1989 commented on PR #4852: URL: https://github.com/apache/hadoop/pull/4852#issuecomment-1251733241 @goiri Thank you very much for your help reviewing the code! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hadoop] slfan1989 commented on pull request #4899: YARN-11283. [Federation] Fix Typo of NodeManager AMRMProxy.

2022-09-19 Thread GitBox
slfan1989 commented on PR #4899: URL: https://github.com/apache/hadoop/pull/4899#issuecomment-1251733108 @goiri Thank you very much for your help reviewing the code! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[jira] [Commented] (HADOOP-18458) AliyunOSS: AliyunOSSBlockOutputStream to support heap/off-heap buffer before uploading data to OSS

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606838#comment-17606838 ] ASF GitHub Bot commented on HADOOP-18458: - wujinhu opened a new pull request, #4912: URL:

[jira] [Updated] (HADOOP-18458) AliyunOSS: AliyunOSSBlockOutputStream to support heap/off-heap buffer before uploading data to OSS

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18458: Labels: pull-request-available (was: ) > AliyunOSS: AliyunOSSBlockOutputStream to

[GitHub] [hadoop] hadoop-yetus commented on pull request #4905: YARN-11307. Fix Yarn Router Broken Link.

2022-09-19 Thread GitBox
hadoop-yetus commented on PR #4905: URL: https://github.com/apache/hadoop/pull/4905#issuecomment-1251846737 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Updated] (HADOOP-18377) hadoop-aws maven build to add a prefetch profile to run all tests with prefetching

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18377: Labels: pull-request-available (was: ) > hadoop-aws maven build to add a prefetch

[jira] [Commented] (HADOOP-18377) hadoop-aws maven build to add a prefetch profile to run all tests with prefetching

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606896#comment-17606896 ] ASF GitHub Bot commented on HADOOP-18377: - virajjasani commented on PR #4913: URL:

[GitHub] [hadoop] ZanderXu commented on a diff in pull request #4903: HDFS-16774.Improve async delete replica on datanode

2022-09-19 Thread GitBox
ZanderXu commented on code in PR #4903: URL: https://github.com/apache/hadoop/pull/4903#discussion_r974787840 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetAsyncDiskService.java: ## @@ -359,6 +371,89 @@ public

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4908: YARN-11308. Router Page display the db username and password in mask mode.

2022-09-19 Thread GitBox
slfan1989 commented on code in PR #4908: URL: https://github.com/apache/hadoop/pull/4908#discussion_r974812563 ## hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfServlet.java: ## @@ -247,4 +254,67 @@ public void testBadFormat() throws Exception

[jira] [Created] (HADOOP-18458) AliyunOSS: AliyunOSSBlockOutputStream to support heap/off-heap buffer before uploading data to OSS

2022-09-19 Thread wujinhu (Jira)
wujinhu created HADOOP-18458: Summary: AliyunOSS: AliyunOSSBlockOutputStream to support heap/off-heap buffer before uploading data to OSS Key: HADOOP-18458 URL: https://issues.apache.org/jira/browse/HADOOP-18458

[jira] [Updated] (HADOOP-18458) AliyunOSS: AliyunOSSBlockOutputStream to support heap/off-heap buffer before uploading data to OSS

2022-09-19 Thread wujinhu (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wujinhu updated HADOOP-18458: - Description: Recently, our customers raise a requirement: AliyunOSSBlockOutputStream should support

[GitHub] [hadoop] ashutoshcipher commented on pull request #4884: YARN-11255. Support loading alternative docker client config from system environment

2022-09-19 Thread GitBox
ashutoshcipher commented on PR #4884: URL: https://github.com/apache/hadoop/pull/4884#issuecomment-1251784604 Thanks @PrabhuJoseph for your reviewing and approving. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[jira] [Commented] (HADOOP-18458) AliyunOSS: AliyunOSSBlockOutputStream to support heap/off-heap buffer before uploading data to OSS

2022-09-19 Thread Yuwei Xiao (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606862#comment-17606862 ] Yuwei Xiao commented on HADOOP-18458: - Glad to see this feature to be available for the community!

[jira] [Commented] (HADOOP-18341) upgrade to commons-configuration2 2.8.0

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606894#comment-17606894 ] ASF GitHub Bot commented on HADOOP-18341: - tasanuma commented on PR #4578: URL:

[GitHub] [hadoop] virajjasani commented on pull request #4913: HADOOP-18377. hadoop-aws maven build to add a prefetch profile to run all tests with prefetching

2022-09-19 Thread GitBox
virajjasani commented on PR #4913: URL: https://github.com/apache/hadoop/pull/4913#issuecomment-1251850569 Tested against `us-west-2`: With `prefetch`: ``` $ mvn verify -Dparallel-tests -Dprefetch -Dscale -DtestsThreadCount=8 [INFO] ---

[jira] [Commented] (HADOOP-18443) Upgrade snakeyaml to 1.32

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606805#comment-17606805 ] ASF GitHub Bot commented on HADOOP-18443: - goiri commented on PR #4906: URL:

[GitHub] [hadoop] goiri commented on pull request #4906: HADOOP-18443. Upgrade snakeyaml to 1.32

2022-09-19 Thread GitBox
goiri commented on PR #4906: URL: https://github.com/apache/hadoop/pull/4906#issuecomment-1251708092 Let's get another build. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[jira] [Updated] (HADOOP-18458) AliyunOSS: AliyunOSSBlockOutputStream to support heap/off-heap buffer before uploading data to OSS

2022-09-19 Thread wujinhu (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wujinhu updated HADOOP-18458: - Description: Recently, our customers raise requirements that  AliyunOSSBlockOutputStream buffers data

[GitHub] [hadoop] ashutoshcipher commented on pull request #4895: YARN-11303. Upgrade jquery ui to 1.13.2 to mitigate CVE-2022-31160

2022-09-19 Thread GitBox
ashutoshcipher commented on PR #4895: URL: https://github.com/apache/hadoop/pull/4895#issuecomment-1251779567 Thanks @goiri for reviewing and approving. We do that have a clean build except the following and I hope it should be fine for this PR. ``` -1 ❌ | test4tests | 0m 0s |  

[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #4907: HADOOP-18457: ABFS:Enable throttling at account level

2022-09-19 Thread GitBox
anmolanmol1234 commented on code in PR #4907: URL: https://github.com/apache/hadoop/pull/4907#discussion_r974899360 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientThrottlingInterceptFactory.java: ## @@ -0,0 +1,59 @@ +/** + * Licensed

[jira] [Commented] (HADOOP-18457) ABFS: Support for account level throttling

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606912#comment-17606912 ] ASF GitHub Bot commented on HADOOP-18457: - anmolanmol1234 commented on code in PR #4907: URL:

[jira] [Commented] (HADOOP-18457) ABFS: Support for account level throttling

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606911#comment-17606911 ] ASF GitHub Bot commented on HADOOP-18457: - anmolanmol1234 commented on code in PR #4907: URL:

[jira] [Comment Edited] (HADOOP-18443) Upgrade snakeyaml to 1.32

2022-09-19 Thread PJ Fanning (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606797#comment-17606797 ] PJ Fanning edited comment on HADOOP-18443 at 9/19/22 11:43 PM: --- [~groot]

[jira] [Commented] (HADOOP-18443) Upgrade snakeyaml to 1.32

2022-09-19 Thread PJ Fanning (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606797#comment-17606797 ] PJ Fanning commented on HADOOP-18443: - [~groot] it looks like snakeyaml now limits the size of the

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4908: YARN-11308. Router Page display the db username and password in mask mode.

2022-09-19 Thread GitBox
slfan1989 commented on code in PR #4908: URL: https://github.com/apache/hadoop/pull/4908#discussion_r974809560 ## hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfServlet.java: ## @@ -59,6 +59,8 @@ public class TestConfServlet { new

[GitHub] [hadoop] slfan1989 commented on pull request #4905: YARN-11307. Fix Yarn Router Broken Link.

2022-09-19 Thread GitBox
slfan1989 commented on PR #4905: URL: https://github.com/apache/hadoop/pull/4905#issuecomment-1251782120 > Overall this looks good. @slfan1989 could you pls address the 6 new checkstyle warnings generated by your patch? > > Thank you for your contribution. @dineshchitlangia

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4897: YARN-11306. [Federation] Refactor NM#FederationInterceptor#recover Code.

2022-09-19 Thread GitBox
slfan1989 commented on code in PR #4897: URL: https://github.com/apache/hadoop/pull/4897#discussion_r974848893 ##

[jira] [Commented] (HADOOP-18457) ABFS: Support for account level throttling

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606904#comment-17606904 ] ASF GitHub Bot commented on HADOOP-18457: - anmolanmol1234 commented on code in PR #4907: URL:

[jira] [Commented] (HADOOP-18457) ABFS: Support for account level throttling

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606903#comment-17606903 ] ASF GitHub Bot commented on HADOOP-18457: - anmolanmol1234 commented on code in PR #4907: URL:

[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #4907: HADOOP-18457: ABFS:Enable throttling at account level

2022-09-19 Thread GitBox
anmolanmol1234 commented on code in PR #4907: URL: https://github.com/apache/hadoop/pull/4907#discussion_r974897835 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsOperationMetrics.java: ## @@ -0,0 +1,74 @@ +/** + * Licensed to the Apache

[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #4907: HADOOP-18457: ABFS:Enable throttling at account level

2022-09-19 Thread GitBox
anmolanmol1234 commented on code in PR #4907: URL: https://github.com/apache/hadoop/pull/4907#discussion_r974897716 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientThrottlingInterceptFactory.java: ## @@ -0,0 +1,59 @@ +/** + * Licensed

[jira] [Commented] (HADOOP-18457) ABFS: Support for account level throttling

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606902#comment-17606902 ] ASF GitHub Bot commented on HADOOP-18457: - anmolanmol1234 commented on code in PR #4907: URL:

[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #4907: HADOOP-18457: ABFS:Enable throttling at account level

2022-09-19 Thread GitBox
anmolanmol1234 commented on code in PR #4907: URL: https://github.com/apache/hadoop/pull/4907#discussion_r974897495 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientThrottlingIntercept.java: ## @@ -82,15 +97,15 @@ static void

[GitHub] [hadoop] ZanderXu commented on pull request #4911: HDFS-16778. Separate out the logger for which DN was picked by a DFSInputStream.

2022-09-19 Thread GitBox
ZanderXu commented on PR #4911: URL: https://github.com/apache/hadoop/pull/4911#issuecomment-1251696996 @omalley Why not change all the DFSClient.LOG in DFSInputStream to this new LOG? -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[jira] [Comment Edited] (HADOOP-18443) Upgrade snakeyaml to 1.32

2022-09-19 Thread PJ Fanning (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606797#comment-17606797 ] PJ Fanning edited comment on HADOOP-18443 at 9/20/22 12:17 AM: --- [~groot]

[GitHub] [hadoop] goiri commented on pull request #4895: YARN-11303. Upgrade jquery ui to 1.13.2 to mitigate CVE-2022-31160

2022-09-19 Thread GitBox
goiri commented on PR #4895: URL: https://github.com/apache/hadoop/pull/4895#issuecomment-1251707985 Can we get a clean build? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hadoop] wujinhu opened a new pull request, #4912: HADOOP-18458: AliyunOSSBlockOutputStream to support heap/off-heap buffer before uploading data to OSS

2022-09-19 Thread GitBox
wujinhu opened a new pull request, #4912: URL: https://github.com/apache/hadoop/pull/4912 ### Description of PR Jira issue: https://issues.apache.org/jira/browse/HADOOP-18458 ### How was this patch tested? New unit tests. ### For code changes: - [X]

[GitHub] [hadoop] slfan1989 commented on pull request #4893: YARN-11305. Fix TestLogAggregationService#testLocalFileDeletionAfterUpload Failed After YARN-11241(#4703).

2022-09-19 Thread GitBox
slfan1989 commented on PR #4893: URL: https://github.com/apache/hadoop/pull/4893#issuecomment-1251794396 @ayushtkn Please help to review the code again, thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hadoop] Hexiaoqiao commented on a diff in pull request #4903: HDFS-16774.Improve async delete replica on datanode

2022-09-19 Thread GitBox
Hexiaoqiao commented on code in PR #4903: URL: https://github.com/apache/hadoop/pull/4903#discussion_r974869052 ## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java: ## @@ -1830,4 +1833,92 @@ public void

[GitHub] [hadoop] tasanuma commented on pull request #4578: HADOOP-18341: upgrade commons-configuration2 to 2.8.0

2022-09-19 Thread GitBox
tasanuma commented on PR #4578: URL: https://github.com/apache/hadoop/pull/4578#issuecomment-1251849439 Thanks for your PR and your discussion, @pjfanning and @ashutoshcipher. Our source code analyzer also detected CVE-2022-33980 in commons-configuration2-2.1.1. I do not think this

[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #4907: HADOOP-18457: ABFS:Enable throttling at account level

2022-09-19 Thread GitBox
anmolanmol1234 commented on code in PR #4907: URL: https://github.com/apache/hadoop/pull/4907#discussion_r974897314 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientThrottlingInterceptFactory.java: ## @@ -0,0 +1,59 @@ +/** + * Licensed

[jira] [Commented] (HADOOP-18457) ABFS: Support for account level throttling

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606909#comment-17606909 ] ASF GitHub Bot commented on HADOOP-18457: - anmolanmol1234 commented on code in PR #4907: URL:

[jira] [Commented] (HADOOP-18457) ABFS: Support for account level throttling

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606910#comment-17606910 ] ASF GitHub Bot commented on HADOOP-18457: - anmolanmol1234 commented on code in PR #4907: URL:

[jira] [Commented] (HADOOP-18457) ABFS: Support for account level throttling

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606908#comment-17606908 ] ASF GitHub Bot commented on HADOOP-18457: - anmolanmol1234 commented on code in PR #4907: URL:

[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #4907: HADOOP-18457: ABFS:Enable throttling at account level

2022-09-19 Thread GitBox
anmolanmol1234 commented on code in PR #4907: URL: https://github.com/apache/hadoop/pull/4907#discussion_r974898835 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientThrottlingInterceptFactory.java: ## @@ -0,0 +1,59 @@ +/** + * Licensed

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4908: YARN-11308. Router Page display the db username and password in mask mode.

2022-09-19 Thread GitBox
slfan1989 commented on code in PR #4908: URL: https://github.com/apache/hadoop/pull/4908#discussion_r974841557 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/HttpServer2.java: ## @@ -214,6 +215,12 @@ public final class HttpServer2 implements

[jira] [Commented] (HADOOP-18443) Upgrade snakeyaml to 1.32

2022-09-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606841#comment-17606841 ] ASF GitHub Bot commented on HADOOP-18443: - ashutoshcipher commented on PR #4906: URL:

  1   2   >