[GitHub] [hadoop] jianghuazhu commented on pull request #3376: HDFS-16208. Support Delete API with FGL.

2022-07-28 Thread GitBox
jianghuazhu commented on PR #3376: URL: https://github.com/apache/hadoop/pull/3376#issuecomment-1198838825 Thanks @prasad-acit . I have read your update carefully, here are some of my personal insights: 1. We should hold child locks in FSDirDeleteOp, here are some of my suggestions.

[jira] [Work logged] (HADOOP-18079) Upgrade Netty to 4.1.77.Final

2022-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18079?focusedWorklogId=796223=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796223 ] ASF GitHub Bot logged work on HADOOP-18079: --- Author: ASF GitHub Bot

[GitHub] [hadoop] jasonwzs commented on pull request #4593: HADOOP-18079. Upgrade Netty to 4.1.77. (#3977)

2022-07-28 Thread GitBox
jasonwzs commented on PR #4593: URL: https://github.com/apache/hadoop/pull/4593#issuecomment-1198823378 https://infra.apache.org/licensing-howto.html#deps-of-deps is about license, but this PR has only change to pom.xml. In case that downstream application explicitly imports netty from

[GitHub] [hadoop] slfan1989 commented on pull request #4614: YARN-11212. [Federation] Add getNodeToLabels REST APIs for Router.

2022-07-28 Thread GitBox
slfan1989 commented on PR #4614: URL: https://github.com/apache/hadoop/pull/4614#issuecomment-1198787934 @goiri Thanks for your help reviewing the code! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4650: YARN-11029. Improve logs to print askCount, allocatedCount in AMRMProxy service.

2022-07-28 Thread GitBox
slfan1989 commented on code in PR #4650: URL: https://github.com/apache/hadoop/pull/4650#discussion_r932775232 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/amrmproxy/AMRMProxyService.java:

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4650: YARN-11029. Improve logs to print askCount, allocatedCount in AMRMProxy service.

2022-07-28 Thread GitBox
slfan1989 commented on code in PR #4650: URL: https://github.com/apache/hadoop/pull/4650#discussion_r932774919 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/amrmproxy/AMRMProxyService.java:

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4650: YARN-11029. Improve logs to print askCount, allocatedCount in AMRMProxy service.

2022-07-28 Thread GitBox
slfan1989 commented on code in PR #4650: URL: https://github.com/apache/hadoop/pull/4650#discussion_r932774778 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/amrmproxy/AMRMProxyService.java:

[jira] [Work logged] (HADOOP-18340) deleteOnExit does not work with S3AFileSystem

2022-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18340?focusedWorklogId=796206=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796206 ] ASF GitHub Bot logged work on HADOOP-18340: --- Author: ASF GitHub Bot

[GitHub] [hadoop] huaxiangsun commented on pull request #4608: HADOOP-18340 deleteOnExit does not work with S3AFileSystem

2022-07-28 Thread GitBox
huaxiangsun commented on PR #4608: URL: https://github.com/apache/hadoop/pull/4608#issuecomment-1198721463 > code in s3afs and the tests all look good. > > But i think the changes should be restricted to s3a fs, even if duplicates a bit of the superclass. > > That whole

[GitHub] [hadoop] cnauroth commented on a diff in pull request #4237: MAPREDUCE-7375 JobSubmissionFiles don't set right permission after mkdirs

2022-07-28 Thread GitBox
cnauroth commented on code in PR #4237: URL: https://github.com/apache/hadoop/pull/4237#discussion_r932699216 ## hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJobSubmissionFiles.java: ## @@ -139,4

[jira] [Work logged] (HADOOP-18345) Enhance client protocol to propagate last seen state IDs for multiple nameservices.

2022-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18345?focusedWorklogId=796189=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796189 ] ASF GitHub Bot logged work on HADOOP-18345: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #4584: HADOOP-18345: Enhance client protocol to propagate last seen state IDs for multiple nameservices.

2022-07-28 Thread GitBox
hadoop-yetus commented on PR #4584: URL: https://github.com/apache/hadoop/pull/4584#issuecomment-1198631578 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] goiri merged pull request #4614: YARN-11212. [Federation] Add getNodeToLabels REST APIs for Router.

2022-07-28 Thread GitBox
goiri merged PR #4614: URL: https://github.com/apache/hadoop/pull/4614 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] goiri commented on a diff in pull request #4606: HDFS-16678. RBF should supports disable getNodeUsage() in RBFMetrics

2022-07-28 Thread GitBox
goiri commented on code in PR #4606: URL: https://github.com/apache/hadoop/pull/4606#discussion_r932570164 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/metrics/RBFMetrics.java: ## @@ -537,35 +547,34 @@ public int

[GitHub] [hadoop] goiri commented on a diff in pull request #4656: YARN-11235. Refactor Policy Code and Define getReservationHomeSubcluster

2022-07-28 Thread GitBox
goiri commented on code in PR #4656: URL: https://github.com/apache/hadoop/pull/4656#discussion_r932565594 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/policies/router/AbstractRouterPolicy.java:

[GitHub] [hadoop] goiri commented on a diff in pull request #4650: YARN-11029. Improve logs to print askCount, allocatedCount in AMRMProxy service.

2022-07-28 Thread GitBox
goiri commented on code in PR #4650: URL: https://github.com/apache/hadoop/pull/4650#discussion_r932561002 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/amrmproxy/AMRMProxyService.java:

[GitHub] [hadoop] cnauroth commented on a diff in pull request #4237: MAPREDUCE-7375 JobSubmissionFiles don't set right permission after mkdirs

2022-07-28 Thread GitBox
cnauroth commented on code in PR #4237: URL: https://github.com/apache/hadoop/pull/4237#discussion_r932562817 ## hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobSubmissionFiles.java: ## @@ -160,6 +160,7

[jira] [Work logged] (HADOOP-18379) rebase feature/HADOOP-18028-s3a-prefetch to trunk

2022-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18379?focusedWorklogId=796157=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796157 ] ASF GitHub Bot logged work on HADOOP-18379: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #4654: HADOOP-18379 rebase feature/HADOOP-18028-s3a-prefetch to trunk

2022-07-28 Thread GitBox
hadoop-yetus commented on PR #4654: URL: https://github.com/apache/hadoop/pull/4654#issuecomment-1198469544 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Work logged] (HADOOP-18079) Upgrade Netty to 4.1.77.Final

2022-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18079?focusedWorklogId=796155=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796155 ] ASF GitHub Bot logged work on HADOOP-18079: --- Author: ASF GitHub Bot

[GitHub] [hadoop] jojochuang commented on pull request #4593: HADOOP-18079. Upgrade Netty to 4.1.77. (#3977)

2022-07-28 Thread GitBox
jojochuang commented on PR #4593: URL: https://github.com/apache/hadoop/pull/4593#issuecomment-1198461140 [INFO] +- io.netty:netty-all:jar:4.1.77.Final:compile [INFO] | +- io.netty:netty-buffer:jar:4.1.77.Final:compile [INFO] | +- io.netty:netty-codec:jar:4.1.77.Final:compile

[jira] [Work logged] (HADOOP-18079) Upgrade Netty to 4.1.77.Final

2022-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18079?focusedWorklogId=796154=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796154 ] ASF GitHub Bot logged work on HADOOP-18079: --- Author: ASF GitHub Bot

[GitHub] [hadoop] jojochuang commented on pull request #4593: HADOOP-18079. Upgrade Netty to 4.1.77. (#3977)

2022-07-28 Thread GitBox
jojochuang commented on PR #4593: URL: https://github.com/apache/hadoop/pull/4593#issuecomment-1198459811 It is my understanding that transitive dependencies should be included. https://infra.apache.org/licensing-howto.html#deps-of-deps Does it make sense to exclude transitive

[jira] [Work logged] (HADOOP-18379) rebase feature/HADOOP-18028-s3a-prefetch to trunk

2022-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18379?focusedWorklogId=796145=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796145 ] ASF GitHub Bot logged work on HADOOP-18379: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #4654: HADOOP-18379 rebase feature/HADOOP-18028-s3a-prefetch to trunk

2022-07-28 Thread GitBox
hadoop-yetus commented on PR #4654: URL: https://github.com/apache/hadoop/pull/4654#issuecomment-1198440835 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #4656: YARN-11235. Refactor Policy Code and Define getReservationHomeSubcluster

2022-07-28 Thread GitBox
hadoop-yetus commented on PR #4656: URL: https://github.com/apache/hadoop/pull/4656#issuecomment-1198436909 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Resolved] (HADOOP-18357) Retarget solution file to VS2019

2022-07-28 Thread Gautham Banasandra (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gautham Banasandra resolved HADOOP-18357. - Resolution: Abandoned Abandoning the PR, this is the reason -

[jira] [Commented] (HADOOP-18378) Implement readFully(long position, byte[] buffer, int offset, int length)

2022-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572549#comment-17572549 ] Steve Loughran commented on HADOOP-18378: - ideally do: ByteBufferPositionedReadable,

[jira] [Commented] (HADOOP-18378) Implement readFully(long position, byte[] buffer, int offset, int length)

2022-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572547#comment-17572547 ] Steve Loughran commented on HADOOP-18378: - related: have S3CachingInputStream.seek() be a noop

[jira] [Updated] (HADOOP-18378) Implement readFully(long position, byte[] buffer, int offset, int length)

2022-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18378: Component/s: fs/s3 > Implement readFully(long position, byte[] buffer, int offset, int

[jira] [Work logged] (HADOOP-18079) Upgrade Netty to 4.1.77.Final

2022-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18079?focusedWorklogId=796131=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796131 ] ASF GitHub Bot logged work on HADOOP-18079: --- Author: ASF GitHub Bot

[GitHub] [hadoop] jasonwzs commented on pull request #4593: HADOOP-18079. Upgrade Netty to 4.1.77. (#3977)

2022-07-28 Thread GitBox
jasonwzs commented on PR #4593: URL: https://github.com/apache/hadoop/pull/4593#issuecomment-1198375723 @jojochuang , is there a special reason why we need add other netty artifacts dependency versions declaration in this PR, e.g. netty-handler-proxy? I don't see where those lib

[jira] [Work logged] (HADOOP-18227) Add input stream IOstats for vectored IO api in S3A.

2022-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18227?focusedWorklogId=796130=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796130 ] ASF GitHub Bot logged work on HADOOP-18227: --- Author: ASF GitHub Bot

[GitHub] [hadoop] mukund-thakur merged pull request #4636: HADOOP-18227. Add input stream IOStats for vectored IO api in S3A.

2022-07-28 Thread GitBox
mukund-thakur merged PR #4636: URL: https://github.com/apache/hadoop/pull/4636 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Work logged] (HADOOP-18302) Remove WhiteBox in hadoop-common module.

2022-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18302?focusedWorklogId=796124=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796124 ] ASF GitHub Bot logged work on HADOOP-18302: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #4457: HADOOP-18302. Remove WhiteBox in hadoop-common module.

2022-07-28 Thread GitBox
hadoop-yetus commented on PR #4457: URL: https://github.com/apache/hadoop/pull/4457#issuecomment-1198347001 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] slfan1989 commented on pull request #4614: YARN-11212. [Federation] Add getNodeToLabels REST APIs for Router.

2022-07-28 Thread GitBox
slfan1989 commented on PR #4614: URL: https://github.com/apache/hadoop/pull/4614#issuecomment-1198347300 @goiri Thanks a lot for your help reviewing the code, I'll follow up with YARN-11220 after this pr is done. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [hadoop] slfan1989 commented on pull request #4656: YARN-11235. Refactor Policy Code and Define getReservationHomeSubcluster

2022-07-28 Thread GitBox
slfan1989 commented on PR #4656: URL: https://github.com/apache/hadoop/pull/4656#issuecomment-1198343021 @goiri This is the first pr after YARN-5871 Split. Please help to review the code, thank you very much! In YARN-11235, I uploaded a design document and described the changed

[GitHub] [hadoop] slfan1989 commented on pull request #4650: YARN-11029. Improve logs to print askCount, allocatedCount in AMRMProxy service.

2022-07-28 Thread GitBox
slfan1989 commented on PR #4650: URL: https://github.com/apache/hadoop/pull/4650#issuecomment-1198339902 @goiri Please help to review the code, thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hadoop] slfan1989 opened a new pull request, #4656: YARN-11235. Refactor Policy Code and Define getReservationHomeSubcluster

2022-07-28 Thread GitBox
slfan1989 opened a new pull request, #4656: URL: https://github.com/apache/hadoop/pull/4656 JIRA:YARN-11235. Refactor Policy Code and Define getReservationHomeSubcluster. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[jira] [Work logged] (HADOOP-18340) deleteOnExit does not work with S3AFileSystem

2022-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18340?focusedWorklogId=796112=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796112 ] ASF GitHub Bot logged work on HADOOP-18340: --- Author: ASF GitHub Bot

[jira] [Work logged] (HADOOP-18340) deleteOnExit does not work with S3AFileSystem

2022-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18340?focusedWorklogId=796111=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796111 ] ASF GitHub Bot logged work on HADOOP-18340: --- Author: ASF GitHub Bot

[GitHub] [hadoop] huaxiangsun commented on a diff in pull request #4608: HADOOP-18340 deleteOnExit does not work with S3AFileSystem

2022-07-28 Thread GitBox
huaxiangsun commented on code in PR #4608: URL: https://github.com/apache/hadoop/pull/4608#discussion_r932404542 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestS3ADeleteOnExit.java: ## @@ -0,0 +1,96 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [hadoop] huaxiangsun commented on a diff in pull request #4608: HADOOP-18340 deleteOnExit does not work with S3AFileSystem

2022-07-28 Thread GitBox
huaxiangsun commented on code in PR #4608: URL: https://github.com/apache/hadoop/pull/4608#discussion_r932404324 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java: ## @@ -3800,6 +3819,32 @@ UploadResult waitForUploadCompletion(String key,

[GitHub] [hadoop] 9uapaw closed pull request #4622: YARN-11063. Support auto queue creation template wildcards for arbitr…

2022-07-28 Thread GitBox
9uapaw closed pull request #4622: YARN-11063. Support auto queue creation template wildcards for arbitr… URL: https://github.com/apache/hadoop/pull/4622 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hadoop] 9uapaw commented on pull request #4622: YARN-11063. Support auto queue creation template wildcards for arbitr…

2022-07-28 Thread GitBox
9uapaw commented on PR #4622: URL: https://github.com/apache/hadoop/pull/4622#issuecomment-1198316088 Thank you for the review @brumi1024. Thank you for the PR @K0K0V0K. Committed to trunk. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [hadoop] hadoop-yetus commented on pull request #4655: YARN-11216. Avoid unnecessary reconstruction of ConfigurationProperties

2022-07-28 Thread GitBox
hadoop-yetus commented on PR #4655: URL: https://github.com/apache/hadoop/pull/4655#issuecomment-1198304739 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #4127: HDFS-13522. RBF: Support observer node from Router-Based Federation

2022-07-28 Thread GitBox
hadoop-yetus commented on PR #4127: URL: https://github.com/apache/hadoop/pull/4127#issuecomment-1198294890 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #4463: YARN-11187. Remove WhiteBox in yarn module.

2022-07-28 Thread GitBox
hadoop-yetus commented on PR #4463: URL: https://github.com/apache/hadoop/pull/4463#issuecomment-1198199681 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Created] (HADOOP-18380) fs.s3a.prefetch.block.size to be read through longBytesOption

2022-07-28 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18380: --- Summary: fs.s3a.prefetch.block.size to be read through longBytesOption Key: HADOOP-18380 URL: https://issues.apache.org/jira/browse/HADOOP-18380 Project:

[GitHub] [hadoop] hadoop-yetus commented on pull request #4462: MAPREDUCE-7390 Remove WhiteBox in mapreduce module.

2022-07-28 Thread GitBox
hadoop-yetus commented on PR #4462: URL: https://github.com/apache/hadoop/pull/4462#issuecomment-1198184311 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Work logged] (HADOOP-18379) rebase feature/HADOOP-18028-s3a-prefetch to trunk

2022-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18379?focusedWorklogId=796076=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796076 ] ASF GitHub Bot logged work on HADOOP-18379: --- Author: ASF GitHub Bot

[jira] [Updated] (HADOOP-18379) rebase feature/HADOOP-18028-s3a-prefetch to trunk

2022-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18379: Labels: pull-request-available (was: ) > rebase feature/HADOOP-18028-s3a-prefetch to

[GitHub] [hadoop] steveloughran commented on pull request #4654: HADOOP-18379 rebase feature/HADOOP-18028-s3a-prefetch to trunk

2022-07-28 Thread GitBox
steveloughran commented on PR #4654: URL: https://github.com/apache/hadoop/pull/4654#issuecomment-1198162616 test failure due to more than one stream read. implies that prefetching is still on by default ``` [ERROR]

[jira] [Assigned] (HADOOP-18379) rebase feature/HADOOP-18028-s3a-prefetch to trunk

2022-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-18379: --- Assignee: Steve Loughran > rebase feature/HADOOP-18028-s3a-prefetch to trunk >

[jira] [Created] (HADOOP-18379) rebase feature/HADOOP-18028-s3a-prefetch to trunk

2022-07-28 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18379: --- Summary: rebase feature/HADOOP-18028-s3a-prefetch to trunk Key: HADOOP-18379 URL: https://issues.apache.org/jira/browse/HADOOP-18379 Project: Hadoop Common

[jira] [Work started] (HADOOP-18379) rebase feature/HADOOP-18028-s3a-prefetch to trunk

2022-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HADOOP-18379 started by Steve Loughran. --- > rebase feature/HADOOP-18028-s3a-prefetch to trunk >

[jira] [Created] (HADOOP-18378) Implement readFully(long position, byte[] buffer, int offset, int length)

2022-07-28 Thread Ahmar Suhail (Jira)
Ahmar Suhail created HADOOP-18378: - Summary: Implement readFully(long position, byte[] buffer, int offset, int length) Key: HADOOP-18378 URL: https://issues.apache.org/jira/browse/HADOOP-18378

[jira] [Work logged] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569

2022-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18197?focusedWorklogId=796073=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796073 ] ASF GitHub Bot logged work on HADOOP-18197: --- Author: ASF GitHub Bot

[GitHub] [hadoop-thirdparty] steveloughran commented on pull request #19: HADOOP-18197. Upgrade protobuf to 3.21.1

2022-07-28 Thread GitBox
steveloughran commented on PR #19: URL: https://github.com/apache/hadoop-thirdparty/pull/19#issuecomment-1198157881 thinking of doing it differently * add the 3.21 stuff parallel to the 3.7 * all our own code switches to the new shaded release * delete the old version this

[jira] [Created] (HADOOP-18377) hadoop-aws maven build to add a prefetch profile to run all tests with prefetching

2022-07-28 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18377: --- Summary: hadoop-aws maven build to add a prefetch profile to run all tests with prefetching Key: HADOOP-18377 URL: https://issues.apache.org/jira/browse/HADOOP-18377

[jira] [Commented] (HADOOP-18028) High performance S3A input stream with prefetching & caching

2022-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572463#comment-17572463 ] Steve Loughran commented on HADOOP-18028: - rebasing to trunk in

[GitHub] [hadoop] K0K0V0K closed pull request #4631: YARN-11216. Avoid unnecessary reconstruction of ConfigurationProperties

2022-07-28 Thread GitBox
K0K0V0K closed pull request #4631: YARN-11216. Avoid unnecessary reconstruction of ConfigurationProperties URL: https://github.com/apache/hadoop/pull/4631 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hadoop] hadoop-yetus commented on pull request #4653: HDFS-16697.Randomly setting “dfs.namenode.resource.checked.volumes.minimum” will always prevent safe mode from being turned off

2022-07-28 Thread GitBox
hadoop-yetus commented on PR #4653: URL: https://github.com/apache/hadoop/pull/4653#issuecomment-1198144710 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Work logged] (HADOOP-18028) High performance S3A input stream with prefetching & caching

2022-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18028?focusedWorklogId=796067=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796067 ] ASF GitHub Bot logged work on HADOOP-18028: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran opened a new pull request, #4654: HADOOP-18028-s3a-prefetch

2022-07-28 Thread GitBox
steveloughran opened a new pull request, #4654: URL: https://github.com/apache/hadoop/pull/4654 ### Description of PR This is the feature branch feature/HADOOP-18028-s3a-prefetch rebased on trunk to see how yetus likes it. If all is good I will force push it to the asf branch,

[jira] [Commented] (HADOOP-18190) Collect IOStatistics during S3A prefetching

2022-07-28 Thread Ahmar Suhail (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572457#comment-17572457 ] Ahmar Suhail commented on HADOOP-18190: --- I ran this, initial results are not encouraging. For a

[jira] [Commented] (HADOOP-18181) move org.apache.hadoop.fs.common package into hadoop-common module

2022-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572428#comment-17572428 ] Steve Loughran commented on HADOOP-18181: - I would like to move them to common so that the

[jira] [Work logged] (HADOOP-16649) Defining hadoop-azure and hadoop-azure-datalake in HADOOP_OPTIONAL_TOOLS will ignore hadoop-azure

2022-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16649?focusedWorklogId=796059=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796059 ] ASF GitHub Bot logged work on HADOOP-16649: --- Author: ASF GitHub Bot

[jira] [Resolved] (HADOOP-18182) S3File to store reference to active S3Object in a field.

2022-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18182. - Resolution: Not A Problem > S3File to store reference to active S3Object in a field. >

[jira] [Commented] (HADOOP-18182) S3File to store reference to active S3Object in a field.

2022-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572426#comment-17572426 ] Steve Loughran commented on HADOOP-18182: - yeas, reviewed code. all looking good. > S3File to

[GitHub] [hadoop] hadoop-yetus commented on pull request #2385: HADOOP-16649. hadoop_add_param function : change regexp test by iterative equality test

2022-07-28 Thread GitBox
hadoop-yetus commented on PR #2385: URL: https://github.com/apache/hadoop/pull/2385#issuecomment-1198084989 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #4311: HDFS-13522: IPC changes to support observer reads through routers.

2022-07-28 Thread GitBox
hadoop-yetus commented on PR #4311: URL: https://github.com/apache/hadoop/pull/4311#issuecomment-1198065242 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Work logged] (HADOOP-18340) deleteOnExit does not work with S3AFileSystem

2022-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18340?focusedWorklogId=796052=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796052 ] ASF GitHub Bot logged work on HADOOP-18340: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on a diff in pull request #4608: HADOOP-18340 deleteOnExit does not work with S3AFileSystem

2022-07-28 Thread GitBox
steveloughran commented on code in PR #4608: URL: https://github.com/apache/hadoop/pull/4608#discussion_r932130167 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestS3ADeleteOnExit.java: ## @@ -0,0 +1,96 @@ +/* + * Licensed to the Apache Software Foundation

[jira] [Work logged] (HADOOP-18357) Retarget solution file to VS2019

2022-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18357?focusedWorklogId=796050=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796050 ] ASF GitHub Bot logged work on HADOOP-18357: --- Author: ASF GitHub Bot

[jira] [Work logged] (HADOOP-18357) Retarget solution file to VS2019

2022-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18357?focusedWorklogId=796051=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796051 ] ASF GitHub Bot logged work on HADOOP-18357: --- Author: ASF GitHub Bot

[GitHub] [hadoop] GauthamBanasandra closed pull request #4616: HADOOP-18357. Retarget solution file to VS2019

2022-07-28 Thread GitBox
GauthamBanasandra closed pull request #4616: HADOOP-18357. Retarget solution file to VS2019 URL: https://github.com/apache/hadoop/pull/4616 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hadoop] GauthamBanasandra commented on pull request #4616: HADOOP-18357. Retarget solution file to VS2019

2022-07-28 Thread GitBox
GauthamBanasandra commented on PR #4616: URL: https://github.com/apache/hadoop/pull/4616#issuecomment-1198055884 The vcxproj and solution files automatically get retargeted due to

[jira] [Work logged] (HADOOP-18357) Retarget solution file to VS2019

2022-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18357?focusedWorklogId=796048=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796048 ] ASF GitHub Bot logged work on HADOOP-18357: --- Author: ASF GitHub Bot

[GitHub] [hadoop] GauthamBanasandra commented on a diff in pull request #4616: HADOOP-18357. Retarget solution file to VS2019

2022-07-28 Thread GitBox
GauthamBanasandra commented on code in PR #4616: URL: https://github.com/apache/hadoop/pull/4616#discussion_r932136279 ## hadoop-common-project/hadoop-common/src/main/native/native.vcxproj: ## @@ -39,12 +37,14 @@ false true Unicode +v142 Review Comment:

[jira] [Work logged] (HADOOP-18344) AWS SDK update to 1.12.262 to address jackson CVE-2018-7489

2022-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18344?focusedWorklogId=796047=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796047 ] ASF GitHub Bot logged work on HADOOP-18344: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #4637: HADOOP-18344. Upgrade AWS SDK to 1.12.262

2022-07-28 Thread GitBox
hadoop-yetus commented on PR #4637: URL: https://github.com/apache/hadoop/pull/4637#issuecomment-1198052208 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #4650: YARN-11029. Improve logs to print askCount, allocatedCount in AMRMProxy service.

2022-07-28 Thread GitBox
hadoop-yetus commented on PR #4650: URL: https://github.com/apache/hadoop/pull/4650#issuecomment-1198044151 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18362) Running org.apache.hadoop.ha.TestZKFailoverController when "hadoop.security.groups.cache.secs" is zero or negative numbers will throw ambiguous exception

2022-07-28 Thread Jingxuan Fu (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572406#comment-17572406 ] Jingxuan Fu commented on HADOOP-18362: -- I commited a pr to trunk. Can assign this issue to me? >

[jira] [Commented] (HADOOP-18350) Support for hadoop-aws with aws-java-sdk-bundle with version greater than 1.12.220

2022-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572400#comment-17572400 ] Steve Loughran commented on HADOOP-18350: - what is the title of the other jira? > Support for

[GitHub] [hadoop] anmolanmol1234 closed pull request #4651: Hadoop 18325 temp

2022-07-28 Thread GitBox
anmolanmol1234 closed pull request #4651: Hadoop 18325 temp URL: https://github.com/apache/hadoop/pull/4651 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[GitHub] [hadoop] hadoop-yetus commented on pull request #4650: YARN-11029. Improve logs to print askCount, allocatedCount in AMRMProxy service.

2022-07-28 Thread GitBox
hadoop-yetus commented on PR #4650: URL: https://github.com/apache/hadoop/pull/4650#issuecomment-1198003193 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Resolved] (HADOOP-18344) AWS SDK update to 1.12.262 to address jackson CVE-2018-7489

2022-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18344. - Fix Version/s: 3.4.0 3.3.4 Resolution: Fixed > AWS SDK

[jira] [Work logged] (HADOOP-18344) AWS SDK update to 1.12.262 to address jackson CVE-2018-7489

2022-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18344?focusedWorklogId=796034=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796034 ] ASF GitHub Bot logged work on HADOOP-18344: --- Author: ASF GitHub Bot

[jira] [Work logged] (HADOOP-18344) AWS SDK update to 1.12.262 to address jackson CVE-2018-7489

2022-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18344?focusedWorklogId=796032=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796032 ] ASF GitHub Bot logged work on HADOOP-18344: --- Author: ASF GitHub Bot

[jira] [Work logged] (HADOOP-18344) AWS SDK update to 1.12.262 to address jackson CVE-2018-7489

2022-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18344?focusedWorklogId=796035=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796035 ] ASF GitHub Bot logged work on HADOOP-18344: --- Author: ASF GitHub Bot

[jira] [Work logged] (HADOOP-18344) AWS SDK update to 1.12.262 to address jackson CVE-2018-7489

2022-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18344?focusedWorklogId=796033=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796033 ] ASF GitHub Bot logged work on HADOOP-18344: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran closed pull request #4645: HADOOP-18344. Upgrade AWS SDK to 1.12.262

2022-07-28 Thread GitBox
steveloughran closed pull request #4645: HADOOP-18344. Upgrade AWS SDK to 1.12.262 URL: https://github.com/apache/hadoop/pull/4645 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hadoop] steveloughran commented on pull request #4645: HADOOP-18344. Upgrade AWS SDK to 1.12.262

2022-07-28 Thread GitBox
steveloughran commented on PR #4645: URL: https://github.com/apache/hadoop/pull/4645#issuecomment-1197975586 merged manually -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hadoop] steveloughran closed pull request #4646: HADOOP-18344. Upgrade AWS SDK to 1.12.262

2022-07-28 Thread GitBox
steveloughran closed pull request #4646: HADOOP-18344. Upgrade AWS SDK to 1.12.262 URL: https://github.com/apache/hadoop/pull/4646 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hadoop] steveloughran commented on pull request #4646: HADOOP-18344. Upgrade AWS SDK to 1.12.262

2022-07-28 Thread GitBox
steveloughran commented on PR #4646: URL: https://github.com/apache/hadoop/pull/4646#issuecomment-1197975091 merged manually -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[jira] [Work logged] (HADOOP-18344) AWS SDK update to 1.12.262 to address jackson CVE-2018-7489

2022-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18344?focusedWorklogId=796031=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796031 ] ASF GitHub Bot logged work on HADOOP-18344: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #4637: HADOOP-18344. Upgrade AWS SDK to 1.12.262

2022-07-28 Thread GitBox
steveloughran commented on PR #4637: URL: https://github.com/apache/hadoop/pull/4637#issuecomment-1197974480 aah, forgot that license-binary change. pushed up a followup, and for the backports merged the two changes -- This is an automated message from the Apache Git Service. To respond

[jira] [Work logged] (HADOOP-18344) AWS SDK update to 1.12.262 to address jackson CVE-2018-7489

2022-07-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18344?focusedWorklogId=796028=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796028 ] ASF GitHub Bot logged work on HADOOP-18344: --- Author: ASF GitHub Bot

  1   2   >