Re: [PR] HADOOP-18889. use getDuration() on http connection settings [hadoop]

2023-10-17 Thread via GitHub
virajjasani commented on code in PR #6180: URL: https://github.com/apache/hadoop/pull/6180#discussion_r1363235836 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/AWSClientConfig.java: ## @@ -378,17 +388,39 @@ private static void initSigner(Configuration

Re: [PR] HDFS-16791 Add getEnclosingRoot API to filesystem interface and all i… [hadoop]

2023-10-17 Thread via GitHub
hadoop-yetus commented on PR #6198: URL: https://github.com/apache/hadoop/pull/6198#issuecomment-1767687124 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18942) Upgrade ZooKeeper to 3.7.2

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776492#comment-17776492 ] ASF GitHub Bot commented on HADOOP-18942: - iwasakims commented on PR #6200: URL:

Re: [PR] HADOOP-18942. Upgrade ZooKeeper to 3.7.2. [hadoop]

2023-10-17 Thread via GitHub
iwasakims commented on PR #6200: URL: https://github.com/apache/hadoop/pull/6200#issuecomment-1767683237 I built it with `-Dhbase.profile=2.0` and ran relevant tests as done in #5988. ``` mvn clean install -DskipTests -DskipShade -Dhbase.profile=2.0 cd

[jira] [Commented] (HADOOP-18942) Upgrade ZooKeeper to 3.7.2

2023-10-17 Thread Masatake Iwasaki (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776491#comment-17776491 ] Masatake Iwasaki commented on HADOOP-18942: --- Thanks, [~groot]. I've already worked on it and

[jira] [Commented] (HADOOP-18942) Upgrade ZooKeeper to 3.7.2

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776490#comment-17776490 ] ASF GitHub Bot commented on HADOOP-18942: - iwasakims commented on PR #6200: URL:

[jira] [Updated] (HADOOP-18942) Upgrade ZooKeeper to 3.7.2

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18942: Labels: pull-request-available (was: ) > Upgrade ZooKeeper to 3.7.2 >

Re: [PR] HADOOP-18942. Upgrade ZooKeeper to 3.7.2. [hadoop]

2023-10-17 Thread via GitHub
iwasakims commented on PR #6200: URL: https://github.com/apache/hadoop/pull/6200#issuecomment-1767678688 jetty-client must be excluded to address dependency convergence error. ``` [WARNING] Dependency convergence error for

[PR] HADOOP-18942. Upgrade ZooKeeper to 3.7.2. [hadoop]

2023-10-17 Thread via GitHub
iwasakims opened a new pull request, #6200: URL: https://github.com/apache/hadoop/pull/6200 https://issues.apache.org/jira/browse/HADOOP-18942 While [HADOOP-18613](https://issues.apache.org/jira/browse/HADOOP-18613) is proposing to upgrade ZooKeeper to 3.8, it will bring dependency

Re: [PR] HADOOP-18889. use getDuration() on http connection settings [hadoop]

2023-10-17 Thread via GitHub
virajjasani commented on code in PR #6180: URL: https://github.com/apache/hadoop/pull/6180#discussion_r1363203939 ## hadoop-common-project/hadoop-common/src/main/resources/core-default.xml: ## @@ -1596,14 +1596,15 @@ fs.s3a.connection.establish.timeout - 5000 - Socket

[jira] [Commented] (HADOOP-18889) S3A: V2 SDK client does not work with third-party store

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776488#comment-17776488 ] ASF GitHub Bot commented on HADOOP-18889: - virajjasani commented on code in PR #6180: URL:

Re: [PR] HDFS-17220. Fix same available space policy in AvailableSpaceVolumeChoosingPolicy [hadoop]

2023-10-17 Thread via GitHub
tomscut commented on PR #6174: URL: https://github.com/apache/hadoop/pull/6174#issuecomment-1767646253 Thanks @GuoPhilipse for your contribution! Thanks @Hexiaoqiao and @zhangshuyan0 for your review! -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] HDFS-17220. Fix same available space policy in AvailableSpaceVolumeChoosingPolicy [hadoop]

2023-10-17 Thread via GitHub
tomscut merged PR #6174: URL: https://github.com/apache/hadoop/pull/6174 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-17679) Upgrade Protobuf from 3.7.1 to 3.17.3

2023-10-17 Thread Colm O hEigeartaigh (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776477#comment-17776477 ] Colm O hEigeartaigh commented on HADOOP-17679: -- Note we have many CVEs in protobuf 3.7.1:

Re: [PR] CVE-2023-2976 [hadoop-thirdparty]

2023-10-17 Thread via GitHub
coheigea commented on PR #25: URL: https://github.com/apache/hadoop-thirdparty/pull/25#issuecomment-1767618510 Note we already have a JIRA and PR https://github.com/apache/hadoop-thirdparty/pull/23 for this CVE -- This is an automated message from the Apache Git Service. To respond to

[jira] [Commented] (HADOOP-18931) FileSystem.getFileSystemClass() to log at debug the jar the .class came from

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776475#comment-17776475 ] ASF GitHub Bot commented on HADOOP-18931: - hadoop-yetus commented on PR #6197: URL:

[jira] [Commented] (HADOOP-18931) FileSystem.getFileSystemClass() to log at debug the jar the .class came from

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776476#comment-17776476 ] ASF GitHub Bot commented on HADOOP-18931: - hadoop-yetus commented on PR #6197: URL:

Re: [PR] HADOOP-18931. FileSystem.getFileSystemClass() to log at debug the jar the .class came from [hadoop]

2023-10-17 Thread via GitHub
hadoop-yetus commented on PR #6197: URL: https://github.com/apache/hadoop/pull/6197#issuecomment-1767612709 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HADOOP-18931. FileSystem.getFileSystemClass() to log at debug the jar the .class came from [hadoop]

2023-10-17 Thread via GitHub
hadoop-yetus commented on PR #6197: URL: https://github.com/apache/hadoop/pull/6197#issuecomment-1767612330 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[PR] YARN-11593. [Federation] Improve command line help information. [hadoop]

2023-10-17 Thread via GitHub
slfan1989 opened a new pull request, #6199: URL: https://github.com/apache/hadoop/pull/6199 ### Description of PR JIRA: YARN-11593. [Federation] Improve command line help information. ### How was this patch tested? ### For code changes: - [ ] Does the

[jira] [Updated] (HADOOP-18867) Upgrade ZooKeeper to 3.6.4

2023-10-17 Thread Masatake Iwasaki (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-18867: -- Fix Version/s: 3.3.7 > Upgrade ZooKeeper to 3.6.4 > -- > >

[jira] [Commented] (HADOOP-18942) Upgrade ZooKeeper to 3.7.2

2023-10-17 Thread Ashutosh Gupta (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776447#comment-17776447 ] Ashutosh Gupta commented on HADOOP-18942: - [~iwasakims] let me know if you not working on it. I

[jira] [Resolved] (HADOOP-18867) Upgrade ZooKeeper to 3.6.4

2023-10-17 Thread Masatake Iwasaki (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HADOOP-18867. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed >

[jira] [Commented] (HADOOP-18867) Upgrade ZooKeeper to 3.6.4

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776445#comment-17776445 ] ASF GitHub Bot commented on HADOOP-18867: - iwasakims commented on PR #5988: URL:

Re: [PR] HADOOP-18867. Upgrade ZooKeeper to 3.6.4. [hadoop]

2023-10-17 Thread via GitHub
iwasakims merged PR #5988: URL: https://github.com/apache/hadoop/pull/5988 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-18867) Upgrade ZooKeeper to 3.6.4

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776446#comment-17776446 ] ASF GitHub Bot commented on HADOOP-18867: - iwasakims merged PR #5988: URL:

Re: [PR] HADOOP-18867. Upgrade ZooKeeper to 3.6.4. [hadoop]

2023-10-17 Thread via GitHub
iwasakims commented on PR #5988: URL: https://github.com/apache/hadoop/pull/5988#issuecomment-1767457496 Thanks, @ayushtkn and @pjfanning. I filed HADOOP-18942 and merging this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[jira] [Created] (HADOOP-18942) Upgrade ZooKeeper to 3.7.2

2023-10-17 Thread Masatake Iwasaki (Jira)
Masatake Iwasaki created HADOOP-18942: - Summary: Upgrade ZooKeeper to 3.7.2 Key: HADOOP-18942 URL: https://issues.apache.org/jira/browse/HADOOP-18942 Project: Hadoop Common Issue Type:

[jira] [Commented] (HADOOP-18918) ITestS3GuardTool fails if SSE/DSSE encryption is used

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776436#comment-17776436 ] ASF GitHub Bot commented on HADOOP-18918: - virajjasani commented on PR #6165: URL:

Re: [PR] HADOOP-18918. ITestS3GuardTool fails if SSE/DSSE encryption is used [hadoop]

2023-10-17 Thread via GitHub
virajjasani commented on PR #6165: URL: https://github.com/apache/hadoop/pull/6165#issuecomment-1767407129 @ahmarsuhail @mehakmeet @mukund-thakur could you please review this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] Add getEnclosingRoot API to FileSystem interface and all implementations [hadoop]

2023-10-17 Thread via GitHub
hadoop-yetus commented on PR #6196: URL: https://github.com/apache/hadoop/pull/6196#issuecomment-1767405119 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] Branch 3.2 - hadoop per bucket endpoint configuration is ignored [hadoop]

2023-10-17 Thread via GitHub
ayushtkn closed pull request #4861: Branch 3.2 - hadoop per bucket endpoint configuration is ignored URL: https://github.com/apache/hadoop/pull/4861 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] HDFS-17228.Add documentation related to BlockManager. [hadoop]

2023-10-17 Thread via GitHub
ayushtkn merged PR #6195: URL: https://github.com/apache/hadoop/pull/6195 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-18931) FileSystem.getFileSystemClass() to log at debug the jar the .class came from

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776426#comment-17776426 ] ASF GitHub Bot commented on HADOOP-18931: - hadoop-yetus commented on PR #6197: URL:

Re: [PR] HADOOP-18931. FileSystem.getFileSystemClass() to log at debug the jar the .class came from [hadoop]

2023-10-17 Thread via GitHub
hadoop-yetus commented on PR #6197: URL: https://github.com/apache/hadoop/pull/6197#issuecomment-1767343446 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18931) FileSystem.getFileSystemClass() to log at debug the jar the .class came from

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776425#comment-17776425 ] ASF GitHub Bot commented on HADOOP-18931: - hadoop-yetus commented on PR #6197: URL:

Re: [PR] HADOOP-18931. FileSystem.getFileSystemClass() to log at debug the jar the .class came from [hadoop]

2023-10-17 Thread via GitHub
hadoop-yetus commented on PR #6197: URL: https://github.com/apache/hadoop/pull/6197#issuecomment-1767338765 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[PR] HDFS-16791 Add getEnclosingRoot API to filesystem interface and all i… [hadoop]

2023-10-17 Thread via GitHub
mccormickt12 opened a new pull request, #6198: URL: https://github.com/apache/hadoop/pull/6198 …mplementations ### Description of PR Add getEnclosingRoot API to filesystem interface. This API is needed in ViewFileSystem and RBF setups to get around the rename

[jira] [Commented] (HADOOP-18867) Upgrade ZooKeeper to 3.6.4

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776423#comment-17776423 ] ASF GitHub Bot commented on HADOOP-18867: - pjfanning commented on PR #5988: URL:

Re: [PR] HADOOP-18867. Upgrade ZooKeeper to 3.6.4. [hadoop]

2023-10-17 Thread via GitHub
pjfanning commented on PR #5988: URL: https://github.com/apache/hadoop/pull/5988#issuecomment-1767324560 @ayushtkn I agree this can be merged but ideally, a 2nd PR should be done soon to bring zookeeper up to 3..7.2 (or later). -- This is an automated message from the Apache Git Service.

[jira] [Commented] (HADOOP-18930) make fs.s3a.create.performance an option you can set for the entire bucket

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776398#comment-17776398 ] ASF GitHub Bot commented on HADOOP-18930: - hadoop-yetus commented on PR #6168: URL:

Re: [PR] HADOOP-18930. Make fs.s3a.create.performance a bucket-wide setting [hadoop]

2023-10-17 Thread via GitHub
hadoop-yetus commented on PR #6168: URL: https://github.com/apache/hadoop/pull/6168#issuecomment-1767214969 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18930) make fs.s3a.create.performance an option you can set for the entire bucket

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776395#comment-17776395 ] ASF GitHub Bot commented on HADOOP-18930: - hadoop-yetus commented on PR #6168: URL:

Re: [PR] HADOOP-18930. Make fs.s3a.create.performance a bucket-wide setting [hadoop]

2023-10-17 Thread via GitHub
hadoop-yetus commented on PR #6168: URL: https://github.com/apache/hadoop/pull/6168#issuecomment-1767191203 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18925) S3A: add option "fs.s3a.copy.from.local.enabled" to enable/disable CopyFromLocalOperation

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776390#comment-17776390 ] ASF GitHub Bot commented on HADOOP-18925: - hadoop-yetus commented on PR #6163: URL:

Re: [PR] HADOOP-18925. S3A: option "fs.s3a.optimized.copy.from.local.enabled" to control CopyFromLocalOperation [hadoop]

2023-10-17 Thread via GitHub
hadoop-yetus commented on PR #6163: URL: https://github.com/apache/hadoop/pull/6163#issuecomment-1767175080 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] CVE-2023-2976 [hadoop-thirdparty]

2023-10-17 Thread via GitHub
goiri commented on PR #25: URL: https://github.com/apache/hadoop-thirdparty/pull/25#issuecomment-1767154645 Approved but waiting for the CI run. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] HADOOP-18843. Guava version 32.0.1 bump to fix CVE-2023-2976 [hadoop-thirdparty]

2023-10-17 Thread via GitHub
Grimoren commented on PR #23: URL: https://github.com/apache/hadoop-thirdparty/pull/23#issuecomment-1767114027 > I've hit the approve and run button to see what the ci builds say. if things are good I'll merge Any chance you could merge this soon. Got a CVE reporting against

[PR] CVE-2023-2976 [hadoop-thirdparty]

2023-10-17 Thread via GitHub
Grimoren opened a new pull request, #25: URL: https://github.com/apache/hadoop-thirdparty/pull/25 Published Vulnerabilities CVE-2023-2976 Use of Java's default temporary directory for file creation in `FileBackedOutputStream` in Google Guava versions 1.0 to 31.1 on Unix systems

[jira] [Commented] (HADOOP-18919) Zookeeper SSL/TLS support in HDFS ZKFC

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776371#comment-17776371 ] ASF GitHub Bot commented on HADOOP-18919: - hadoop-yetus commented on PR #6194: URL:

Re: [PR] HADOOP-18919. Zookeeper SSL/TLS support in HDFS ZKFC [hadoop]

2023-10-17 Thread via GitHub
hadoop-yetus commented on PR #6194: URL: https://github.com/apache/hadoop/pull/6194#issuecomment-1767075251 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18931) FileSystem.getFileSystemClass() to log at debug the jar the .class came from

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776359#comment-17776359 ] ASF GitHub Bot commented on HADOOP-18931: - virajjasani opened a new pull request, #6197: URL:

[jira] [Updated] (HADOOP-18931) FileSystem.getFileSystemClass() to log at debug the jar the .class came from

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18931: Labels: pull-request-available (was: ) > FileSystem.getFileSystemClass() to log at

[jira] [Comment Edited] (HADOOP-18931) FileSystem.getFileSystemClass() to log at debug the jar the .class came from

2023-10-17 Thread Viraj Jasani (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776356#comment-17776356 ] Viraj Jasani edited comment on HADOOP-18931 at 10/17/23 7:16 PM: -

[PR] HADOOP-18931. FileSystem.getFileSystemClass() to log at debug the jar the .class came from [hadoop]

2023-10-17 Thread via GitHub
virajjasani opened a new pull request, #6197: URL: https://github.com/apache/hadoop/pull/6197 Jira: HADOOP-18931 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[jira] [Commented] (HADOOP-18931) FileSystem.getFileSystemClass() to log at debug the jar the .class came from

2023-10-17 Thread Viraj Jasani (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776356#comment-17776356 ] Viraj Jasani commented on HADOOP-18931: --- sounds good, it makes sense to log for all fs invocation

Re: [PR] Add getEnclosingRoot API to FileSystem interface and all implementations [hadoop]

2023-10-17 Thread via GitHub
mccormickt12 commented on PR #6196: URL: https://github.com/apache/hadoop/pull/6196#issuecomment-1766980795 cc @simbadzina @goiri (This is related to the [old pr](https://github.com/apache/hadoop/pull/4967/files#diff-a07f38553cfc52416c1a9bc82110837c8d0c078a0f1ef9c62c290d7de5b3e745) I

[jira] [Commented] (HADOOP-18487) Make protobuf 2.5 an optional runtime dependency.

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776333#comment-17776333 ] ASF GitHub Bot commented on HADOOP-18487: - steveloughran commented on PR #6185: URL:

Re: [PR] HADOOP-18487. Protobuf 2.5 removal part 2: stop exporting protobuf-2.5" [hadoop]

2023-10-17 Thread via GitHub
steveloughran commented on PR #6185: URL: https://github.com/apache/hadoop/pull/6185#issuecomment-1766974645 @apurtell can you review & approve the final bit of the protobuf 2.5 work -this cuts it out from the distributions -- This is an automated message from the Apache Git Service. To

[jira] [Commented] (HADOOP-18487) Make protobuf 2.5 an optional runtime dependency.

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776328#comment-17776328 ] ASF GitHub Bot commented on HADOOP-18487: - steveloughran merged PR #6184: URL:

Re: [PR] HADOOP-18487. Make protobuf 2.5 an optional runtime dependency. (#4996) [hadoop]

2023-10-17 Thread via GitHub
steveloughran merged PR #6184: URL: https://github.com/apache/hadoop/pull/6184 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] HDFS-17228.Add documentation related to BlockManager. [hadoop]

2023-10-17 Thread via GitHub
hadoop-yetus commented on PR #6195: URL: https://github.com/apache/hadoop/pull/6195#issuecomment-1766944233 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18925) S3A: add option "fs.s3a.copy.from.local.enabled" to enable/disable CopyFromLocalOperation

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776319#comment-17776319 ] ASF GitHub Bot commented on HADOOP-18925: - jojochuang commented on PR #6163: URL:

Re: [PR] HADOOP-18925. S3A: option "fs.s3a.optimized.copy.from.local.enabled" to control CopyFromLocalOperation [hadoop]

2023-10-17 Thread via GitHub
jojochuang commented on PR #6163: URL: https://github.com/apache/hadoop/pull/6163#issuecomment-1766909835 There are conflicts in the PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] YARN-9013. [BackPort] [GPG] fix order of steps cleaning Registry entries in ApplicationCleaner. [hadoop]

2023-10-17 Thread via GitHub
goiri commented on code in PR #6147: URL: https://github.com/apache/hadoop/pull/6147#discussion_r1362534634 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java: ## @@ -4367,7 +4367,7 @@ public static boolean

Re: [PR] YARN-11592. Add timeout to GPGUtils#invokeRMWebService. [hadoop]

2023-10-17 Thread via GitHub
goiri commented on code in PR #6189: URL: https://github.com/apache/hadoop/pull/6189#discussion_r1362528994 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-globalpolicygenerator/src/main/java/org/apache/hadoop/yarn/server/globalpolicygenerator/GPGUtils.java:

[jira] [Commented] (HADOOP-18919) Zookeeper SSL/TLS support in HDFS ZKFC

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776298#comment-17776298 ] ASF GitHub Bot commented on HADOOP-18919: - jojochuang commented on code in PR #6194: URL:

[jira] [Commented] (HADOOP-18919) Zookeeper SSL/TLS support in HDFS ZKFC

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776297#comment-17776297 ] ASF GitHub Bot commented on HADOOP-18919: - jojochuang commented on PR #6194: URL:

Re: [PR] HADOOP-18919. Zookeeper SSL/TLS support in HDFS ZKFC [hadoop]

2023-10-17 Thread via GitHub
jojochuang commented on code in PR #6194: URL: https://github.com/apache/hadoop/pull/6194#discussion_r1362493253 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SecurityUtil.java: ## @@ -786,4 +789,96 @@ public static List

Re: [PR] HADOOP-18919. Zookeeper SSL/TLS support in HDFS ZKFC [hadoop]

2023-10-17 Thread via GitHub
jojochuang commented on PR #6194: URL: https://github.com/apache/hadoop/pull/6194#issuecomment-1766842223 Unrelated question... in [HADOOP-18709](https://github.com/apache/hadoop/pull/5638/files#top) a new dependency was added (netty-transport-native-epoll) it's a native lib for Netty to

Re: [PR] HDFS-17227. EC: Fix bug in choosing targets when racks is not enough. [hadoop]

2023-10-17 Thread via GitHub
hadoop-yetus commented on PR #6192: URL: https://github.com/apache/hadoop/pull/6192#issuecomment-1766828501 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[PR] staging [hadoop]

2023-10-17 Thread via GitHub
mccormickt12 opened a new pull request, #6196: URL: https://github.com/apache/hadoop/pull/6196 ### Description of PR Original PR and comments https://github.com/apache/hadoop/pull/4967/files#diff-a07f38553cfc52416c1a9bc82110837c8d0c078a0f1ef9c62c290d7de5b3e745 ### How

Re: [PR] HDFS-17228.Add documentation related to BlockManager. [hadoop]

2023-10-17 Thread via GitHub
hadoop-yetus commented on PR #6195: URL: https://github.com/apache/hadoop/pull/6195#issuecomment-1766784532 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HADOOP-18711. Upgrade nimbus jwt jar due to issues in its embedded shaded json-smart code [hadoop]

2023-10-17 Thread via GitHub
pjfanning commented on PR #5639: URL: https://github.com/apache/hadoop/pull/5639#issuecomment-1766712078 @rohit-kb could you rebase this? The issue with the module-info classes has recently been resolved in branch-3.3. -- This is an automated message from the Apache Git Service. To

[jira] [Commented] (HADOOP-18938) S3A region logic to handle vpce and non standard endpoints

2023-10-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776243#comment-17776243 ] Steve Loughran commented on HADOOP-18938: - fail fast may not be the right strategy as the

[jira] [Resolved] (HADOOP-18908) Improve s3a region handling, including determining from endpoint

2023-10-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18908. - Fix Version/s: 3.4.0 Resolution: Fixed > Improve s3a region handling, including

[jira] [Updated] (HADOOP-18908) Improve s3a region handling, including determining from endpoint

2023-10-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18908: Description: S3A region logic improved for better inference and to be compatible with

[jira] [Commented] (HADOOP-18908) Improve s3a region handling, including determining from endpoint

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776241#comment-17776241 ] ASF GitHub Bot commented on HADOOP-18908: - steveloughran merged PR #6187: URL:

Re: [PR] HADOOP-18908. Improve S3A region handling. [hadoop]

2023-10-17 Thread via GitHub
steveloughran merged PR #6187: URL: https://github.com/apache/hadoop/pull/6187 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] HADOOP-18919. Zookeeper SSL/TLS support in HDFS ZKFC [hadoop]

2023-10-17 Thread via GitHub
ferdelyi commented on PR #6194: URL: https://github.com/apache/hadoop/pull/6194#issuecomment-1766546738 > Could you take a look at this @ferdelyi? I saw that you worked on both #5638 and #6114. > > I moved some code that you added from ZKCuratorManager to SecurityUtil, so I could

[jira] [Commented] (HADOOP-18919) Zookeeper SSL/TLS support in HDFS ZKFC

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776240#comment-17776240 ] ASF GitHub Bot commented on HADOOP-18919: - ferdelyi commented on PR #6194: URL:

Re: [PR] HADOOP-18939. NPE in AWS v2 SDK RetryOnErrorCodeCondition.shouldRetry() [hadoop]

2023-10-17 Thread via GitHub
steveloughran merged PR #6193: URL: https://github.com/apache/hadoop/pull/6193 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-18939) NPE in AWS v2 SDK RetryOnErrorCodeCondition.shouldRetry()

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776233#comment-17776233 ] ASF GitHub Bot commented on HADOOP-18939: - steveloughran merged PR #6193: URL:

[jira] [Commented] (HADOOP-18939) NPE in AWS v2 SDK RetryOnErrorCodeCondition.shouldRetry()

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776232#comment-17776232 ] ASF GitHub Bot commented on HADOOP-18939: - steveloughran commented on PR #6193: URL:

Re: [PR] HADOOP-18939. NPE in AWS v2 SDK RetryOnErrorCodeCondition.shouldRetry() [hadoop]

2023-10-17 Thread via GitHub
steveloughran commented on PR #6193: URL: https://github.com/apache/hadoop/pull/6193#issuecomment-1766508017 > The NPE will get fixed in the SDK. yes, but that's a future tense event. we are having problems with our deployments already -- This is an automated message from the

Re: [PR] HDFS-17218. NameNode should remove its excess blocks from the ExcessRedundancyMap When a DN registers [hadoop]

2023-10-17 Thread via GitHub
Hexiaoqiao commented on PR #6176: URL: https://github.com/apache/hadoop/pull/6176#issuecomment-1766458976 Back to this PR. > IMO, adding a timeout mechanism may not add much pressure on NameNode. However, it seems that the implementation of that solution is more complex than the

Re: [PR] HADOOP-18830. Cut S3 Select [hadoop]

2023-10-17 Thread via GitHub
ahmarsuhail commented on PR #6144: URL: https://github.com/apache/hadoop/pull/6144#issuecomment-1766391559 And agree with your comment, should cut `org.apache.hadoop.fs.s3a.select.BlockingEnumeration` and add the commit to the package-info. -- This is an automated message from the

[jira] [Commented] (HADOOP-18830) Cut S3 Select

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776209#comment-17776209 ] ASF GitHub Bot commented on HADOOP-18830: - ahmarsuhail commented on PR #6144: URL:

[jira] [Commented] (HADOOP-18830) Cut S3 Select

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776207#comment-17776207 ] ASF GitHub Bot commented on HADOOP-18830: - ahmarsuhail commented on code in PR #6144: URL:

Re: [PR] HADOOP-18830. Cut S3 Select [hadoop]

2023-10-17 Thread via GitHub
ahmarsuhail commented on code in PR #6144: URL: https://github.com/apache/hadoop/pull/6144#discussion_r1362087831 ## hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md: ## @@ -35,7 +35,6 @@ full details. * [Encryption](./encryption.html) *

[jira] [Commented] (HADOOP-18919) Zookeeper SSL/TLS support in HDFS ZKFC

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776205#comment-17776205 ] ASF GitHub Bot commented on HADOOP-18919: - dombizita commented on PR #6194: URL:

Re: [PR] HADOOP-18919. Zookeeper SSL/TLS support in HDFS ZKFC [hadoop]

2023-10-17 Thread via GitHub
dombizita commented on PR #6194: URL: https://github.com/apache/hadoop/pull/6194#issuecomment-1766365751 Could you take a look at this @ferdelyi? I saw that you worked on both #5638 and #6114. I moved some code that you added from ZKCuratorManager to SecurityUtil, so I could use it

Re: [PR] HDFS-17218. NameNode should remove its excess blocks from the ExcessRedundancyMap When a DN registers [hadoop]

2023-10-17 Thread via GitHub
zhangshuyan0 commented on code in PR #6176: URL: https://github.com/apache/hadoop/pull/6176#discussion_r1362023878 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java: ## @@ -1007,6 +1013,7 @@ public void

Re: [PR] HDFS-17227. EC: Fix bug in choosing targets when racks is not enough. [hadoop]

2023-10-17 Thread via GitHub
hadoop-yetus commented on PR #6192: URL: https://github.com/apache/hadoop/pull/6192#issuecomment-1766291730 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18932) upgrade AWS v1 and v2 SDKs

2023-10-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776175#comment-17776175 ] ASF GitHub Bot commented on HADOOP-18932: - steveloughran merged PR #6178: URL:

Re: [PR] HADOOP-18932. upgrade AWS v1 and v2 SDKs [hadoop]

2023-10-17 Thread via GitHub
steveloughran merged PR #6178: URL: https://github.com/apache/hadoop/pull/6178 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] HDFS-17218. NameNode should remove its excess blocks from the ExcessRedundancyMap When a DN registers [hadoop]

2023-10-17 Thread via GitHub
zhangshuyan0 commented on code in PR #6176: URL: https://github.com/apache/hadoop/pull/6176#discussion_r1361994647 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java: ## @@ -1007,6 +1013,7 @@ public void

Re: [PR] HDFS-17218. NameNode should remove its excess blocks from the ExcessRedundancyMap When a DN registers [hadoop]

2023-10-17 Thread via GitHub
ZanderXu commented on code in PR #6176: URL: https://github.com/apache/hadoop/pull/6176#discussion_r1361993679 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java: ## @@ -1007,6 +1013,7 @@ public void

Re: [PR] HDFS-17218. NameNode should remove its excess blocks from the ExcessRedundancyMap When a DN registers [hadoop]

2023-10-17 Thread via GitHub
ZanderXu commented on code in PR #6176: URL: https://github.com/apache/hadoop/pull/6176#discussion_r1361950390 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java: ## @@ -1007,6 +1013,7 @@ public void

[jira] [Updated] (HADOOP-18919) Zookeeper SSL/TLS support in HDFS ZKFC

2023-10-17 Thread Zita Dombi (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zita Dombi updated HADOOP-18919: Summary: Zookeeper SSL/TLS support in HDFS ZKFC (was: Zookeeper SSL/TLS support in HDFS ) >

  1   2   >