[jira] [Commented] (HADOOP-19039) Hadoop 3.4.0 Highlight big features and improvements.

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17808050#comment-17808050 ] ASF GitHub Bot commented on HADOOP-19039: - slfan1989 commented on PR #6462: URL:

Re: [PR] HADOOP-19039. Hadoop 3.4.0 Highlight big features and improvements. [hadoop]

2024-01-17 Thread via GitHub
slfan1989 commented on PR #6462: URL: https://github.com/apache/hadoop/pull/6462#issuecomment-1897928836 @ayushtkn @Hexiaoqiao @iwasakims @goiri @steveloughran @tasanuma Hadoop 3.4.0 includes numerous commits involving significant improvements such as RBF, EC, Yarn Federation, DN,

Re: [PR] HDFS-17332 DFSInputStream: avoid logging stacktrace until when we really need to fail a read request with a MissingBlockException [hadoop]

2024-01-17 Thread via GitHub
xinglin commented on PR #6446: URL: https://github.com/apache/hadoop/pull/6446#issuecomment-1897926271 We also added "Retry with the next available datanode." explicitly to the WARN log line. -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] HDFS-17332 DFSInputStream: avoid logging stacktrace until when we really need to fail a read request with a MissingBlockException [hadoop]

2024-01-17 Thread via GitHub
xinglin commented on PR #6446: URL: https://github.com/apache/hadoop/pull/6446#issuecomment-1897924023 > > > Ok so it seems we didn't remove any exception logging right? We are just additionally keeping track of them and logging them again at the end? > > > > > > correct. store

[jira] [Commented] (HADOOP-19039) Hadoop 3.4.0 Highlight big features and improvements.

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17808046#comment-17808046 ] ASF GitHub Bot commented on HADOOP-19039: - slfan1989 commented on PR #6462: URL:

Re: [PR] HADOOP-19039. Hadoop 3.4.0 Highlight big features and improvements. [hadoop]

2024-01-17 Thread via GitHub
slfan1989 commented on PR #6462: URL: https://github.com/apache/hadoop/pull/6462#issuecomment-1897918922 @steveloughran Thank you for the information! I will add more information about log4j2 and java 17 soon. -- This is an automated message from the Apache Git Service. To respond to the

[jira] [Commented] (HADOOP-19039) Hadoop 3.4.0 Highlight big features and improvements.

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17808044#comment-17808044 ] ASF GitHub Bot commented on HADOOP-19039: - slfan1989 opened a new pull request, #6462: URL:

[jira] [Updated] (HADOOP-19039) Hadoop 3.4.0 Highlight big features and improvements.

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-19039: Labels: pull-request-available (was: ) > Hadoop 3.4.0 Highlight big features and

[jira] [Commented] (HADOOP-18656) ABFS: Support for Pagination in Recursive Directory Delete

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17808043#comment-17808043 ] ASF GitHub Bot commented on HADOOP-18656: - anmolanmol1234 commented on code in PR #6409: URL:

[jira] [Commented] (HADOOP-18656) ABFS: Support for Pagination in Recursive Directory Delete

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17808042#comment-17808042 ] ASF GitHub Bot commented on HADOOP-18656: - anmolanmol1234 commented on code in PR #6409: URL:

Re: [PR] HADOOP-18656: [ABFS] Adding Support for Paginated Delete for Large Directories in HNS Account [hadoop]

2024-01-17 Thread via GitHub
anmolanmol1234 commented on code in PR #6409: URL: https://github.com/apache/hadoop/pull/6409#discussion_r1456987897 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/ITestAbfsPaginatedDelete.java: ## @@ -0,0 +1,279 @@ +/** + * Licensed to the

Re: [PR] HADOOP-18656: [ABFS] Adding Support for Paginated Delete for Large Directories in HNS Account [hadoop]

2024-01-17 Thread via GitHub
anmolanmol1234 commented on code in PR #6409: URL: https://github.com/apache/hadoop/pull/6409#discussion_r1456987897 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/ITestAbfsPaginatedDelete.java: ## @@ -0,0 +1,279 @@ +/** + * Licensed to the

[jira] [Commented] (HADOOP-19038) Improve create-release RUN script

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17808035#comment-17808035 ] ASF GitHub Bot commented on HADOOP-19038: - slfan1989 commented on PR #6448: URL:

Re: [PR] HADOOP-19038. Improve create-release RUN script. [hadoop]

2024-01-17 Thread via GitHub
slfan1989 commented on PR #6448: URL: https://github.com/apache/hadoop/pull/6448#issuecomment-1897876438 @Hexiaoqiao Can you help review this PR again? Thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] HDFS-17343. Revert "HDFS-16016. BPServiceActor to provide new thread to handle IBR (#2998)" [hadoop]

2024-01-17 Thread via GitHub
slfan1989 commented on PR #6457: URL: https://github.com/apache/hadoop/pull/6457#issuecomment-1897875338 @tasanuma @Hexiaoqiao @virajjasani Can you help review this PR? Thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] HDFS-17332 DFSInputStream: avoid logging stacktrace until when we really need to fail a read request with a MissingBlockException [hadoop]

2024-01-17 Thread via GitHub
mccormickt12 commented on PR #6446: URL: https://github.com/apache/hadoop/pull/6446#issuecomment-1897870432 > > Ok so it seems we didn't remove any exception logging right? We are just additionally keeping track of them and logging them again at the end? > > correct. store all

[jira] [Commented] (HADOOP-14837) Handle S3A "glacier" data

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17808029#comment-17808029 ] ASF GitHub Bot commented on HADOOP-14837: - bpahuja commented on code in PR #6407: URL:

Re: [PR] HADOOP-14837 : Support Read Restored Glacier Objects [hadoop]

2024-01-17 Thread via GitHub
bpahuja commented on code in PR #6407: URL: https://github.com/apache/hadoop/pull/6407#discussion_r1456959404 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Constants.java: ## @@ -1486,6 +1486,17 @@ private Constants() { */ public static final int

[jira] [Commented] (HADOOP-14837) Handle S3A "glacier" data

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17808028#comment-17808028 ] ASF GitHub Bot commented on HADOOP-14837: - bpahuja commented on code in PR #6407: URL:

[jira] [Commented] (HADOOP-14837) Handle S3A "glacier" data

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17808027#comment-17808027 ] ASF GitHub Bot commented on HADOOP-14837: - bpahuja commented on code in PR #6407: URL:

Re: [PR] HADOOP-14837 : Support Read Restored Glacier Objects [hadoop]

2024-01-17 Thread via GitHub
bpahuja commented on code in PR #6407: URL: https://github.com/apache/hadoop/pull/6407#discussion_r1456959208 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java: ## @@ -2466,8 +2474,8 @@ public RemoteIterator listFilesAndDirectoryMarkers(

Re: [PR] HADOOP-14837 : Support Read Restored Glacier Objects [hadoop]

2024-01-17 Thread via GitHub
bpahuja commented on code in PR #6407: URL: https://github.com/apache/hadoop/pull/6407#discussion_r1456958842 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3ObjectStorageClassFilter.java: ## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software

[jira] [Commented] (HADOOP-19024) change to bouncy castle jdk1.8 jars

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17808023#comment-17808023 ] ASF GitHub Bot commented on HADOOP-19024: - slfan1989 commented on PR #6410: URL:

Re: [PR] HADOOP-19024. Use bouncycastle jdk18 1.77 [hadoop]

2024-01-17 Thread via GitHub
slfan1989 commented on PR #6410: URL: https://github.com/apache/hadoop/pull/6410#issuecomment-1897849525 @pjfanning Can we retrigger compilation? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] YARN-11647. use StandardCharsets.UTF_8 [hadoop]

2024-01-17 Thread via GitHub
slfan1989 commented on PR #6447: URL: https://github.com/apache/hadoop/pull/6447#issuecomment-1897848393 @pjfanning Thank you for your contribution! Merged into trunk. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] YARN-11647. use StandardCharsets.UTF_8 [hadoop]

2024-01-17 Thread via GitHub
slfan1989 merged PR #6447: URL: https://github.com/apache/hadoop/pull/6447 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] HDFS-17334. FSEditLogAsync#enqueueEdit does not synchronized this before invoke wait method. [hadoop]

2024-01-17 Thread via GitHub
hfutatzhanghb commented on PR #6434: URL: https://github.com/apache/hadoop/pull/6434#issuecomment-1897837901 > Line211 has already ensured that we have a monitor for this object: > >

Re: [PR] HDFS-17341: Support dedicated user queues in Namenode FairCallQueue [hadoop]

2024-01-17 Thread via GitHub
hadoop-yetus commented on PR #6461: URL: https://github.com/apache/hadoop/pull/6461#issuecomment-1897743558 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17334. FSEditLogAsync#enqueueEdit does not synchronized this before invoke wait method. [hadoop]

2024-01-17 Thread via GitHub
zhangshuyan0 commented on PR #6434: URL: https://github.com/apache/hadoop/pull/6434#issuecomment-1897739440 Line211 has already ensured that we have a monitor for this object:

Re: [PR] HDFS-17331: Fix Blocks are always -1 and DataNode`s version are always UNKNOWN in federationhealth.html. [hadoop]

2024-01-17 Thread via GitHub
zhangshuyan0 commented on PR #6429: URL: https://github.com/apache/hadoop/pull/6429#issuecomment-1897716060 @ThinkerLei Please check the failed tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] HDFS-17339:Skip cacheReport when one blockPool does not have CacheBlock on this DataNode [hadoop]

2024-01-17 Thread via GitHub
zhangshuyan0 commented on PR #6456: URL: https://github.com/apache/hadoop/pull/6456#issuecomment-1897714361 I don't know what changes have happened to CI recently. This patch looks great to me and can alleviate the pressure on NN. Let's wait and see if others have any opinions. -- This

Re: [PR] HDFS-17337. RPC RESPONSE time seems not exactly accurate when using FSEditLogAsync. [hadoop]

2024-01-17 Thread via GitHub
zhangshuyan0 merged PR #6439: URL: https://github.com/apache/hadoop/pull/6439 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] HDFS-17332 DFSInputStream: avoid logging stacktrace until when we really need to fail a read request with a MissingBlockException [hadoop]

2024-01-17 Thread via GitHub
xinglin commented on code in PR #6446: URL: https://github.com/apache/hadoop/pull/6446#discussion_r1456687729 ## hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java: ## @@ -786,6 +786,7 @@ private synchronized int

Re: [PR] HDFS-17332 DFSInputStream: avoid logging stacktrace until when we really need to fail a read request with a MissingBlockException [hadoop]

2024-01-17 Thread via GitHub
xinglin commented on PR #6446: URL: https://github.com/apache/hadoop/pull/6446#issuecomment-1897575748 > Ok so it seems we didn't remove any exception logging right? We are just additionally keeping track of them and logging them again at the end? correct. store all exceptions in a

Re: [PR] Preparing for 3.5.0 development [hadoop]

2024-01-17 Thread via GitHub
hadoop-yetus commented on PR #6411: URL: https://github.com/apache/hadoop/pull/6411#issuecomment-1897517144 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17332 DFSInputStream: avoid logging stacktrace until when we really need to fail a read request with a MissingBlockException [hadoop]

2024-01-17 Thread via GitHub
mccormickt12 commented on code in PR #6446: URL: https://github.com/apache/hadoop/pull/6446#discussion_r1456635545 ## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRead.java: ## @@ -25,24 +25,49 @@ import java.nio.channels.ClosedByInterruptException;

[PR] HDFS-17341: Support dedicated user queues in Namenode FairCallQueue [hadoop]

2024-01-17 Thread via GitHub
li-leyang opened a new pull request, #6461: URL: https://github.com/apache/hadoop/pull/6461 ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g.

Re: [PR] HDFS-17332 DFSInputStream: avoid logging stacktrace until when we really need to fail a read request with a MissingBlockException [hadoop]

2024-01-17 Thread via GitHub
mccormickt12 commented on code in PR #6446: URL: https://github.com/apache/hadoop/pull/6446#discussion_r1456619911 ## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRead.java: ## @@ -25,24 +25,49 @@ import java.nio.channels.ClosedByInterruptException;

Re: [PR] HDFS-17332 DFSInputStream: avoid logging stacktrace until when we really need to fail a read request with a MissingBlockException [hadoop]

2024-01-17 Thread via GitHub
mccormickt12 commented on code in PR #6446: URL: https://github.com/apache/hadoop/pull/6446#discussion_r1456618755 ## hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java: ## @@ -880,6 +914,8 @@ protected synchronized int

Re: [PR] HDFS-17332 DFSInputStream: avoid logging stacktrace until when we really need to fail a read request with a MissingBlockException [hadoop]

2024-01-17 Thread via GitHub
mccormickt12 commented on code in PR #6446: URL: https://github.com/apache/hadoop/pull/6446#discussion_r1456617867 ## hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java: ## @@ -786,6 +786,7 @@ private synchronized int

Re: [PR] HDFS-17332 DFSInputStream: avoid logging stacktrace until when we really need to fail a read request with a MissingBlockException [hadoop]

2024-01-17 Thread via GitHub
mccormickt12 commented on PR #6446: URL: https://github.com/apache/hadoop/pull/6446#issuecomment-1897448148 @xinglin isn't it useful to have the stacktrace for all the failures? My understanding is its 3 failures before the block missing exception. The stacktraces for each is valuable

[jira] [Commented] (HADOOP-18184) s3a prefetching stream to support unbuffer()

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807918#comment-17807918 ] ASF GitHub Bot commented on HADOOP-18184: - hadoop-yetus commented on PR #5832: URL:

Re: [PR] HADOOP-18184. S3A Prefetching unbuffer. [hadoop]

2024-01-17 Thread via GitHub
hadoop-yetus commented on PR #5832: URL: https://github.com/apache/hadoop/pull/5832#issuecomment-1897084310 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18184) s3a prefetching stream to support unbuffer()

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807893#comment-17807893 ] ASF GitHub Bot commented on HADOOP-18184: - hadoop-yetus commented on PR #5832: URL:

Re: [PR] HADOOP-18184. S3A Prefetching unbuffer. [hadoop]

2024-01-17 Thread via GitHub
hadoop-yetus commented on PR #5832: URL: https://github.com/apache/hadoop/pull/5832#issuecomment-1896707194 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19043) S3A: Regression: ITestS3AOpenCost fails on prefetch test runs

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807889#comment-17807889 ] ASF GitHub Bot commented on HADOOP-19043: - hadoop-yetus commented on PR #6460: URL:

Re: [PR] HADOOP-19043. S3A: Regression: ITestS3AOpenCost fails on prefetch test runs [hadoop]

2024-01-17 Thread via GitHub
hadoop-yetus commented on PR #6460: URL: https://github.com/apache/hadoop/pull/6460#issuecomment-1896682323 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] Revert "HDFS-16016. BPServiceActor to provide new thread to handle IBR (#2998)" [hadoop]

2024-01-17 Thread via GitHub
hadoop-yetus commented on PR #6457: URL: https://github.com/apache/hadoop/pull/6457#issuecomment-1896599536 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19041) further use of StandardCharsets

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807857#comment-17807857 ] ASF GitHub Bot commented on HADOOP-19041: - hadoop-yetus commented on PR #6449: URL:

Re: [PR] HADOOP-19041. Use StandardCharsets in more places [hadoop]

2024-01-17 Thread via GitHub
hadoop-yetus commented on PR #6449: URL: https://github.com/apache/hadoop/pull/6449#issuecomment-1896460625 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19043) S3A: Regression: ITestS3AOpenCost fails on prefetch test runs

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807849#comment-17807849 ] ASF GitHub Bot commented on HADOOP-19043: - steveloughran opened a new pull request, #6460: URL:

[jira] [Updated] (HADOOP-19043) S3A: Regression: ITestS3AOpenCost fails on prefetch test runs

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-19043: Labels: pull-request-available (was: ) > S3A: Regression: ITestS3AOpenCost fails on

[PR] HADOOP-19043. S3A: Regression: ITestS3AOpenCost fails on prefetch test runs [hadoop]

2024-01-17 Thread via GitHub
steveloughran opened a new pull request, #6460: URL: https://github.com/apache/hadoop/pull/6460 This is actually trickier than it seems as we will need to go deep into the implementation of caching. Specifically: the prefetcher knows the file length and if you open a file

[jira] [Commented] (HADOOP-19033) S3A: disable checksum validation

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807846#comment-17807846 ] ASF GitHub Bot commented on HADOOP-19033: - steveloughran merged PR #6441: URL:

Re: [PR] HADOOP-19033. S3A: disable checksums when fs.s3a.checksum.validation == false [hadoop]

2024-01-17 Thread via GitHub
steveloughran merged PR #6441: URL: https://github.com/apache/hadoop/pull/6441 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-18894) upgrade sshd-core due to CVEs

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807836#comment-17807836 ] ASF GitHub Bot commented on HADOOP-18894: - pjfanning commented on PR #6060: URL:

Re: [PR] HADOOP-18894: upgrade sshd-core due to CVEs [hadoop]

2024-01-17 Thread via GitHub
pjfanning commented on PR #6060: URL: https://github.com/apache/hadoop/pull/6060#issuecomment-1896311836 @steveloughran these are test-only-dependencies. Still nice to keep them up to date especially when the old versions have CVEs. -- This is an automated message from the Apache Git

[jira] [Commented] (HADOOP-19033) S3A: disable checksum validation

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807823#comment-17807823 ] ASF GitHub Bot commented on HADOOP-19033: - hadoop-yetus commented on PR #6441: URL:

Re: [PR] HADOOP-19033. S3A: disable checksums when fs.s3a.checksum.validation == false [hadoop]

2024-01-17 Thread via GitHub
hadoop-yetus commented on PR #6441: URL: https://github.com/apache/hadoop/pull/6441#issuecomment-1896269981 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] YARN-11607: TestTimelineAuthFilterForV2 fails intermittently [hadoop]

2024-01-17 Thread via GitHub
hadoop-yetus commented on PR #6459: URL: https://github.com/apache/hadoop/pull/6459#issuecomment-1896242011 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-14837) Handle S3A "glacier" data

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807799#comment-17807799 ] ASF GitHub Bot commented on HADOOP-14837: - ahmarsuhail commented on code in PR #6407: URL:

Re: [PR] HADOOP-14837 : Support Read Restored Glacier Objects [hadoop]

2024-01-17 Thread via GitHub
ahmarsuhail commented on code in PR #6407: URL: https://github.com/apache/hadoop/pull/6407#discussion_r1455953276 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3ObjectStorageClassFilter.java: ## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software

[jira] [Commented] (HADOOP-19033) S3A: disable checksum validation

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807778#comment-17807778 ] ASF GitHub Bot commented on HADOOP-19033: - steveloughran commented on PR #6441: URL:

Re: [PR] HADOOP-19033. S3A: disable checksums when fs.s3a.checksum.validation == false [hadoop]

2024-01-17 Thread via GitHub
steveloughran commented on PR #6441: URL: https://github.com/apache/hadoop/pull/6441#issuecomment-1896063124 retested, all good except when I ran the openfile cost test with -Dprefetch, which was from #6425. Filed HADOOP-19043 for that -- This is an automated message from the Apache Git

[jira] [Commented] (HADOOP-19043) S3A: Regression: ITestS3AOpenCost fails on prefetch test runs

2024-01-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807774#comment-17807774 ] Steve Loughran commented on HADOOP-19043: - {code} [ERROR] Tests run: 8, Failures: 2, Errors:

[jira] [Created] (HADOOP-19043) S3A: Regression: ITestS3AOpenCost fails on prefetch test runs

2024-01-17 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19043: --- Summary: S3A: Regression: ITestS3AOpenCost fails on prefetch test runs Key: HADOOP-19043 URL: https://issues.apache.org/jira/browse/HADOOP-19043 Project:

Re: [PR] HADOOP-19033. S3A: disable checksums when fs.s3a.checksum.validation == false [hadoop]

2024-01-17 Thread via GitHub
steveloughran commented on PR #6441: URL: https://github.com/apache/hadoop/pull/6441#issuecomment-1895993631 need to rebase as the imports from the error handing pr conflict -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[jira] [Commented] (HADOOP-19033) S3A: disable checksum validation

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807759#comment-17807759 ] ASF GitHub Bot commented on HADOOP-19033: - steveloughran commented on PR #6441: URL:

[jira] [Commented] (HADOOP-18894) upgrade sshd-core due to CVEs

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807757#comment-17807757 ] ASF GitHub Bot commented on HADOOP-18894: - steveloughran commented on PR #6060: URL:

Re: [PR] HADOOP-18894: upgrade sshd-core due to CVEs [hadoop]

2024-01-17 Thread via GitHub
steveloughran commented on PR #6060: URL: https://github.com/apache/hadoop/pull/6060#issuecomment-1895991882 would need to be tagged as incompatible if sshd ftp jar is now needed in installations which want to use it. (I'd have moved it to its own module if SSH wasn't apparently used

Re: [PR] YARN-11607: TestTimelineAuthFilterForV2 fails intermittently [hadoop]

2024-01-17 Thread via GitHub
susheelgupta7 commented on PR #6459: URL: https://github.com/apache/hadoop/pull/6459#issuecomment-1895981465 Since the testcase failure cannot be reproduced. The sleep interval between retries is relatively short, I believe that the execution time of the asynchronous call might be

[PR] YARN-11607: TestTimelineAuthFilterForV2 fails intermittently [hadoop]

2024-01-17 Thread via GitHub
susheelgupta7 opened a new pull request, #6459: URL: https://github.com/apache/hadoop/pull/6459 ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g.

[jira] [Commented] (HADOOP-18154) S3A Authentication to support WebIdentity

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807734#comment-17807734 ] ASF GitHub Bot commented on HADOOP-18154: - steveloughran commented on PR #4070: URL:

Re: [PR] HADOOP-18154. S3A Authentication to support WebIdentity [hadoop]

2024-01-17 Thread via GitHub
steveloughran commented on PR #4070: URL: https://github.com/apache/hadoop/pull/4070#issuecomment-1895893894 Trunk is on aws v2 sdk now; it's where features go in. once something is in there a patch for branch-3.3 may be considered; that'd have to be a v1 implementation of the same feature

Re: [PR] Revert "HDFS-16016. BPServiceActor to provide new thread to handle IBR (#2998)" [hadoop]

2024-01-17 Thread via GitHub
slfan1989 commented on PR #6457: URL: https://github.com/apache/hadoop/pull/6457#issuecomment-1895855668 > @yuanboliu > I don't object to fixing [HDFS-16016](https://issues.apache.org/jira/browse/HDFS-16016) and [HDFS-17129](https://issues.apache.org/jira/browse/HDFS-17129). However, no

Re: [PR] Revert "HADOOP-16822. Provide source artifacts for hadoop-client-api" [hadoop]

2024-01-17 Thread via GitHub
hadoop-yetus commented on PR #6458: URL: https://github.com/apache/hadoop/pull/6458#issuecomment-1895825097 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-16822) Provide source artifacts for hadoop-client-api

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807723#comment-17807723 ] ASF GitHub Bot commented on HADOOP-16822: - hadoop-yetus commented on PR #6458: URL:

[jira] [Commented] (HADOOP-19019) Parallel Maven Build Support for Apache Hadoop

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807721#comment-17807721 ] ASF GitHub Bot commented on HADOOP-19019: - hadoop-yetus commented on PR #6373: URL:

Re: [PR] HADOOP-19019: Parallel Maven Build Support for Apache Hadoop [hadoop]

2024-01-17 Thread via GitHub
hadoop-yetus commented on PR #6373: URL: https://github.com/apache/hadoop/pull/6373#issuecomment-1895784931 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19041) further use of StandardCharsets

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807704#comment-17807704 ] ASF GitHub Bot commented on HADOOP-19041: - hadoop-yetus commented on PR #6449: URL:

Re: [PR] HADOOP-19041. Use StandardCharsets in more places [hadoop]

2024-01-17 Thread via GitHub
hadoop-yetus commented on PR #6449: URL: https://github.com/apache/hadoop/pull/6449#issuecomment-1895644308 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-15869. Network issue while FSEditLogAsync is executing RpcEdit.logSyncNotify can cause the namenode to hang [hadoop]

2024-01-17 Thread via GitHub
ZanderXu commented on PR #2737: URL: https://github.com/apache/hadoop/pull/2737#issuecomment-1895594762 @functioner Can you move forward with this MR? I can help with review. If you don't have time, I can resign this ticket to other contributor. -- This is an automated message from the

[jira] [Commented] (HADOOP-19042) S3A: detect and recover from SSL ConnectionReset exceptions

2024-01-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807685#comment-17807685 ] Steve Loughran commented on HADOOP-19042: - {code} Caused by: javax.net.ssl.SSLException:

[jira] [Created] (HADOOP-19042) S3A: detect and recover from SSL ConnectionReset exceptions

2024-01-17 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19042: --- Summary: S3A: detect and recover from SSL ConnectionReset exceptions Key: HADOOP-19042 URL: https://issues.apache.org/jira/browse/HADOOP-19042 Project: Hadoop

[jira] [Commented] (HADOOP-16822) Provide source artifacts for hadoop-client-api

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807674#comment-17807674 ] ASF GitHub Bot commented on HADOOP-16822: - pan3793 commented on PR #6458: URL:

Re: [PR] Revert "HADOOP-16822. Provide source artifacts for hadoop-client-api" [hadoop]

2024-01-17 Thread via GitHub
pan3793 commented on PR #6458: URL: https://github.com/apache/hadoop/pull/6458#issuecomment-1895499366 cc @jojochuang @steveloughran -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[jira] [Updated] (HADOOP-16822) Provide source artifacts for hadoop-client-api

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-16822: Labels: pull-request-available (was: ) > Provide source artifacts for hadoop-client-api

[jira] [Commented] (HADOOP-16822) Provide source artifacts for hadoop-client-api

2024-01-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807673#comment-17807673 ] ASF GitHub Bot commented on HADOOP-16822: - pan3793 opened a new pull request, #6458: URL:

[PR] Revert "HADOOP-16822. Provide source artifacts for hadoop-client-api" [hadoop]

2024-01-17 Thread via GitHub
pan3793 opened a new pull request, #6458: URL: https://github.com/apache/hadoop/pull/6458 This reverts commit 2c4ab72a60113e4dd4ef2375e6f9413e519b1044. ### Description of PR TL;DR, it generate the wrong source code and messes the code jump in IDEA. See details in

Re: [PR] Revert "HDFS-16016. BPServiceActor to provide new thread to handle IBR (#2998)" [hadoop]

2024-01-17 Thread via GitHub
hadoop-yetus commented on PR #6457: URL: https://github.com/apache/hadoop/pull/6457#issuecomment-1895378347 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] Revert "HDFS-16016. BPServiceActor to provide new thread to handle IBR (#2998)" [hadoop]

2024-01-17 Thread via GitHub
tasanuma commented on PR #6457: URL: https://github.com/apache/hadoop/pull/6457#issuecomment-1895367367 @yuanboliu I don't object to fixing HDFS-16016 and HDFS-17129. However, no conclusion has been reached on HDFS-17129, and it is acting as a blocker for 3.4.0, which we are about to