[jira] [Commented] (HADOOP-18325) ABFS: Add correlated metric support for ABFS operations

2024-05-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848052#comment-17848052 ] ASF GitHub Bot commented on HADOOP-18325: - anmolanmol1234 commented on PR #6314: URL:

Re: [PR] HADOOP-18325: ABFS: Add correlated metric support for ABFS operations [hadoop]

2024-05-20 Thread via GitHub
anmolanmol1234 commented on PR #6314: URL: https://github.com/apache/hadoop/pull/6314#issuecomment-2121794214 > well lets that as a next step and we can merge this in? makes sense. but it will be your next piece of homework... Sure @steveloughran, sounds good. -- This is an

[jira] [Commented] (HADOOP-19163) Upgrade protobuf version to 3.25.3

2024-05-20 Thread Bilwa S T (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848049#comment-17848049 ] Bilwa S T commented on HADOOP-19163: [~ste...@apache.org] [~ayushtkn] can we merge this? Thanks >

Re: [PR] HDFS-17529. Improve router state store cache entry deletion [hadoop]

2024-05-20 Thread via GitHub
hadoop-yetus commented on PR #6833: URL: https://github.com/apache/hadoop/pull/6833#issuecomment-2121778835 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17529. Improve router state store cache entry deletion [hadoop]

2024-05-20 Thread via GitHub
ZanderXu commented on code in PR #6833: URL: https://github.com/apache/hadoop/pull/6833#discussion_r1607542930 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/driver/StateStoreRecordOperations.java: ## @@ -127,6 +128,17 @@

Re: [PR] HDFS-17529. Improve router state store cache entry deletion [hadoop]

2024-05-20 Thread via GitHub
ZanderXu commented on code in PR #6833: URL: https://github.com/apache/hadoop/pull/6833#discussion_r1607499422 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/driver/impl/StateStoreBaseImpl.java: ## @@ -86,4 +89,37 @@ public

Re: [PR] HDFS-17464. Improve some logs output in class FsDatasetImpl [hadoop]

2024-05-20 Thread via GitHub
haiyang1987 commented on PR #6724: URL: https://github.com/apache/hadoop/pull/6724#issuecomment-2121550133 Committed to trunk. Thanks @hfutatzhanghb for your contributions and @ZanderXu @ayushtkn review! -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] HDFS-17464. Improve some logs output in class FsDatasetImpl [hadoop]

2024-05-20 Thread via GitHub
haiyang1987 merged PR #6724: URL: https://github.com/apache/hadoop/pull/6724 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-19173) Upgrade org.apache.derby:derby to 10.17.1.0

2024-05-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848018#comment-17848018 ] ASF GitHub Bot commented on HADOOP-19173: - slfan1989 commented on PR #6836: URL:

Re: [PR] HADOOP-19173. Upgrade org.apache.derby:derby to 10.17.1.0. [hadoop]

2024-05-20 Thread via GitHub
slfan1989 closed pull request #6836: HADOOP-19173. Upgrade org.apache.derby:derby to 10.17.1.0. URL: https://github.com/apache/hadoop/pull/6836 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] HADOOP-19173. Upgrade org.apache.derby:derby to 10.17.1.0. [hadoop]

2024-05-20 Thread via GitHub
slfan1989 commented on PR #6836: URL: https://github.com/apache/hadoop/pull/6836#issuecomment-2121501694 I will close this pr, and then continue to follow up on the derby version upgrade. -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] HDFS-17531. [Discuss] RBF: Aynchronous router RPC. [hadoop]

2024-05-20 Thread via GitHub
slfan1989 commented on PR #6838: URL: https://github.com/apache/hadoop/pull/6838#issuecomment-2121500478 > @ayushtkn @slfan1989 hi, thanks for you replay, I sent the discussion to [common-...@hadoop.apache.org](mailto:common-...@hadoop.apache.org). This pr has too many changes and

Re: [PR] Revert "Bump org.apache.derby:derby from 10.14.2.0 to 10.17.1.0 in /hadoop-project" [hadoop]

2024-05-20 Thread via GitHub
hadoop-yetus commented on PR #6841: URL: https://github.com/apache/hadoop/pull/6841#issuecomment-2121498754 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HADOOP-19173. Upgrade org.apache.derby:derby to 10.17.1.0. [hadoop]

2024-05-20 Thread via GitHub
slfan1989 commented on PR #6836: URL: https://github.com/apache/hadoop/pull/6836#issuecomment-2121496565 @simbadzina I have rolled back #6816. Sorry for the inconvenience caused. In the future, when upgrading jar versions, I will pay attention to the compatibility of the jar packages with

Re: [PR] Revert "Bump org.apache.derby:derby from 10.14.2.0 to 10.17.1.0 in /hadoop-project" [hadoop]

2024-05-20 Thread via GitHub
slfan1989 merged PR #6841: URL: https://github.com/apache/hadoop/pull/6841 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[PR] Revert "Bump org.apache.derby:derby from 10.14.2.0 to 10.17.1.0 in /hadoop-project" [hadoop]

2024-05-20 Thread via GitHub
slfan1989 opened a new pull request, #6841: URL: https://github.com/apache/hadoop/pull/6841 Reverts apache/hadoop#6816 We have founed some HDFS RBF errors. We can refer to this issue https://issues.apache.org/jira/browse/HDFS-17533. -- This is an automated message from the Apache

Re: [PR] HADOOP-19173. Upgrade org.apache.derby:derby to 10.17.1.0. [hadoop]

2024-05-20 Thread via GitHub
simbadzina commented on PR #6836: URL: https://github.com/apache/hadoop/pull/6836#issuecomment-2121445583 @slfan1989 derby 10.17.1.0 isn't compatible with Java 8. There are two RBF test that fail here: https://issues.apache.org/jira/browse/HDFS-17533 -- This is an automated message from

[jira] [Commented] (HADOOP-18508) support multiple s3a integration test runs on same bucket in parallel

2024-05-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847991#comment-17847991 ] ASF GitHub Bot commented on HADOOP-18508: - mukund-thakur commented on PR #5081: URL:

Re: [PR] HADOOP-18508. S3A: Support parallel integration test runs on same bucket [hadoop]

2024-05-20 Thread via GitHub
mukund-thakur commented on PR #5081: URL: https://github.com/apache/hadoop/pull/5081#issuecomment-2121184602 This is continuously failing. `[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AEncryptionSSEC [ERROR] Tests run: 24, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 110.261 s

[jira] [Commented] (HADOOP-18679) Add API for bulk/paged delete of files and objects

2024-05-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847976#comment-17847976 ] ASF GitHub Bot commented on HADOOP-18679: - steveloughran commented on PR #6726: URL:

Re: [PR] HADOOP-18679. Add API for bulk/paged delete of files [hadoop]

2024-05-20 Thread via GitHub
steveloughran commented on PR #6726: URL: https://github.com/apache/hadoop/pull/6726#issuecomment-2121154089 fyi #6686 adds dynamic load for the wrapped methods, and calls them through the tests. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] HDFS-17464. Improve some logs output in class FsDatasetImpl [hadoop]

2024-05-20 Thread via GitHub
ayushtkn commented on PR #6724: URL: https://github.com/apache/hadoop/pull/6724#issuecomment-2121001399 @haiyang1987 / @ZanderXu anyone hitting the merge button? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[jira] [Resolved] (HADOOP-13147) Constructors must not call overrideable methods in PureJavaCrc32C

2024-05-20 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HADOOP-13147. --- Fix Version/s: 3.5.0 Hadoop Flags: Reviewed Resolution: Fixed > Constructors

[jira] [Commented] (HADOOP-13147) Constructors must not call overrideable methods in PureJavaCrc32C

2024-05-20 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847960#comment-17847960 ] Ayush Saxena commented on HADOOP-13147: --- Committed to trunk. Thanx [~sebb] for the

[jira] [Assigned] (HADOOP-13147) Constructors must not call overrideable methods in PureJavaCrc32C

2024-05-20 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned HADOOP-13147: - Assignee: Sebb > Constructors must not call overrideable methods in PureJavaCrc32C >

[jira] [Commented] (HADOOP-13147) Constructors must not call overrideable methods

2024-05-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847959#comment-17847959 ] ASF GitHub Bot commented on HADOOP-13147: - ayushtkn merged PR #6408: URL:

[jira] [Updated] (HADOOP-13147) Constructors must not call overrideable methods in PureJavaCrc32C

2024-05-20 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HADOOP-13147: -- Summary: Constructors must not call overrideable methods in PureJavaCrc32C (was:

Re: [PR] HADOOP-13147: Constructors must not call overrideable methods [hadoop]

2024-05-20 Thread via GitHub
ayushtkn merged PR #6408: URL: https://github.com/apache/hadoop/pull/6408 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-18508) support multiple s3a integration test runs on same bucket in parallel

2024-05-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847925#comment-17847925 ] ASF GitHub Bot commented on HADOOP-18508: - mukund-thakur commented on PR #5081: URL:

Re: [PR] HADOOP-18508. S3A: Support parallel integration test runs on same bucket [hadoop]

2024-05-20 Thread via GitHub
mukund-thakur commented on PR #5081: URL: https://github.com/apache/hadoop/pull/5081#issuecomment-2120796420 I checked out this branch yesterday and ran tests in parallel in two terminals and saw many failures. Could be my setup issue as well but let's wait on this till I figure out what

[jira] [Commented] (HADOOP-18679) Add API for bulk/paged delete of files and objects

2024-05-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847918#comment-17847918 ] ASF GitHub Bot commented on HADOOP-18679: - steveloughran commented on PR #6726: URL:

Re: [PR] HADOOP-18679. Add API for bulk/paged delete of files [hadoop]

2024-05-20 Thread via GitHub
steveloughran commented on PR #6726: URL: https://github.com/apache/hadoop/pull/6726#issuecomment-2120754706 merged to trunk. mukund, can you do the backport to branch-3.4.1, while we can think about what to do for 3.3.9? speaking of which, we should think about that... -- This is an

[jira] [Commented] (HADOOP-18679) Add API for bulk/paged delete of files and objects

2024-05-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847917#comment-17847917 ] ASF GitHub Bot commented on HADOOP-18679: - steveloughran merged PR #6726: URL:

Re: [PR] HADOOP-18679. Add API for bulk/paged delete of files [hadoop]

2024-05-20 Thread via GitHub
steveloughran merged PR #6726: URL: https://github.com/apache/hadoop/pull/6726 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-18325) ABFS: Add correlated metric support for ABFS operations

2024-05-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847915#comment-17847915 ] ASF GitHub Bot commented on HADOOP-18325: - steveloughran commented on PR #6314: URL:

Re: [PR] HADOOP-19148 Upgrade solr from 8.11.2 to 8.11.3 to address CVE-2023-50298 [hadoop]

2024-05-20 Thread via GitHub
steveloughran commented on PR #6830: URL: https://github.com/apache/hadoop/pull/6830#issuecomment-2120748921 raise this on the developer list -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] HADOOP-18325: ABFS: Add correlated metric support for ABFS operations [hadoop]

2024-05-20 Thread via GitHub
steveloughran commented on PR #6314: URL: https://github.com/apache/hadoop/pull/6314#issuecomment-2120747934 well lets that as a next step and we can merge this in? makes sense. but it will be your next piece of homework... -- This is an automated message from the Apache Git Service. To

[jira] [Commented] (HADOOP-18508) support multiple s3a integration test runs on same bucket in parallel

2024-05-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847914#comment-17847914 ] ASF GitHub Bot commented on HADOOP-18508: - steveloughran commented on PR #5081: URL:

Re: [PR] HADOOP-18508. S3A: Support parallel integration test runs on same bucket [hadoop]

2024-05-20 Thread via GitHub
steveloughran commented on PR #5081: URL: https://github.com/apache/hadoop/pull/5081#issuecomment-2120745475 @mukund-thakur retested, identified a race condition in terasorts and fixed. all should be good now -- This is an automated message from the Apache Git Service. To respond to the

[PR] HDFS-17098. DatanodeManager does not handle null storage type properly. [hadoop]

2024-05-20 Thread via GitHub
Hexiaoqiao opened a new pull request, #6840: URL: https://github.com/apache/hadoop/pull/6840 ### Description of PR 1. From https://github.com/apache/hadoop/pull/6035 which contributed by @teamconfx. 2. Fix checkstyle and try to trigger Yetus again. ### How was this

Re: [PR] HDFS-17529. Improve router state store cache entry deletion [hadoop]

2024-05-20 Thread via GitHub
hadoop-yetus commented on PR #6833: URL: https://github.com/apache/hadoop/pull/6833#issuecomment-2120301224 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18990) S3A: retry on credential expiry

2024-05-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847848#comment-17847848 ] Steve Loughran commented on HADOOP-18990: - noticed that an AWS product (greegrass) has

Re: [PR] HDFS-17532. Allow router state store cache update to overwrite and delete in parallel [hadoop]

2024-05-20 Thread via GitHub
hadoop-yetus commented on PR #6839: URL: https://github.com/apache/hadoop/pull/6839#issuecomment-2120145776 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19181) IAMCredentialsProvider throttle failures

2024-05-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847840#comment-17847840 ] Steve Loughran commented on HADOOP-19181: - reviewing AWS SDK JIRA, there's still a shorter

[jira] [Updated] (HADOOP-19181) IAMCredentialsProvider throttle failures

2024-05-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-19181: Description: Tests report throttling errors in IAM being remapped to noauth and failure

[jira] [Created] (HADOOP-19181) IAMCredentialsProvider throttle failures

2024-05-20 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19181: --- Summary: IAMCredentialsProvider throttle failures Key: HADOOP-19181 URL: https://issues.apache.org/jira/browse/HADOOP-19181 Project: Hadoop Common

Re: [PR] HDFS-17532. Allow router state store cache update to overwrite and delete in parallel [hadoop]

2024-05-20 Thread via GitHub
kokonguyen191 commented on PR #6839: URL: https://github.com/apache/hadoop/pull/6839#issuecomment-2120093100 @ZanderXu This is the other half split from https://github.com/apache/hadoop/pull/6833, can help me review if you're free, thanks! -- This is an automated message from the Apache

[PR] HDFS-17532. Allow router state store cache update to overwrite and delete in parallel [hadoop]

2024-05-20 Thread via GitHub
kokonguyen191 opened a new pull request, #6839: URL: https://github.com/apache/hadoop/pull/6839 ### Description of PR This ticket aims to allow the overwrite part and delete part of org.apache.hadoop.hdfs.server.federation.store.CachedRecordStore#overrideExpiredRecords to run in

Re: [PR] HDFS-17529. Improve router state store cache entry deletion [hadoop]

2024-05-20 Thread via GitHub
kokonguyen191 commented on PR #6833: URL: https://github.com/apache/hadoop/pull/6833#issuecomment-2120015079 @ZanderXu Thanks for the review, I have updated the codes + changed the ticket/PR title for the deletion part only, will open another PR for the async part later. I'm a bit confused

[jira] [Commented] (HADOOP-18325) ABFS: Add correlated metric support for ABFS operations

2024-05-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1784#comment-1784 ] ASF GitHub Bot commented on HADOOP-18325: - anmolanmol1234 commented on PR #6314: URL:

Re: [PR] HADOOP-18325: ABFS: Add correlated metric support for ABFS operations [hadoop]

2024-05-20 Thread via GitHub
anmolanmol1234 commented on PR #6314: URL: https://github.com/apache/hadoop/pull/6314#issuecomment-2119782683 > @anmolanmol1234 you got any timeline for changes here? Hi @steveloughran, since the changes requested would need a design refactor and scale tests from our side, I would