[jira] [Commented] (HADOOP-16822) Provide source artifacts for hadoop-client-api

2024-01-20 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-16822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809090#comment-17809090
 ] 

ASF GitHub Bot commented on HADOOP-16822:
-

pan3793 commented on PR #6458:
URL: https://github.com/apache/hadoop/pull/6458#issuecomment-1902529715

   Also cc @pjfanning @Hexiaoqiao 




> Provide source artifacts for hadoop-client-api
> --
>
> Key: HADOOP-16822
> URL: https://issues.apache.org/jira/browse/HADOOP-16822
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: Karel Kolman
>Assignee: Karel Kolman
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.3.1, 3.4.0, 3.2.3
>
> Attachments: HADOOP-16822-hadoop-client-api-source-jar.patch
>
>
> h5. Improvement request
> The third-party libraries shading hadoop-client-api (& hadoop-client-runtime) 
> artifacts are super useful.
>  
> Having uber source jar for hadoop-client-api (maybe even 
> hadoop-client-runtime) would be great for downstream development & debugging 
> purposes.
> Are there any obstacles or objections against providing fat jar with all the 
> hadoop client api as well ?
> h5. Dev links
> - *maven-shaded-plugin* and its *shadeSourcesContent* attribute
> - 
> https://maven.apache.org/plugins/maven-shade-plugin/shade-mojo.html#shadeSourcesContent



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



Re: [PR] Revert "HADOOP-16822. Provide source artifacts for hadoop-client-api" [hadoop]

2024-01-20 Thread via GitHub


pan3793 commented on PR #6458:
URL: https://github.com/apache/hadoop/pull/6458#issuecomment-1902529715

   Also cc @pjfanning @Hexiaoqiao 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-18045) Disable TestDynamometerInfra

2024-01-20 Thread Shilun Fan (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-18045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809089#comment-17809089
 ] 

Shilun Fan commented on HADOOP-18045:
-

[~tasanuma] Thanks for the reminder! I will suspend the current operation. I do 
have some questions about the jira version. We will discuss it in the 
Hadoop-3.4 Release Update.

> Disable TestDynamometerInfra
> 
>
> Key: HADOOP-18045
> URL: https://issues.apache.org/jira/browse/HADOOP-18045
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: test
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.3.2
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> This test is broken and there is no fix provided for a long time. Let's 
> disable the test to reduce the noise in the daily qbt job.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-18045) Disable TestDynamometerInfra

2024-01-20 Thread Takanobu Asanuma (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-18045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809088#comment-17809088
 ] 

Takanobu Asanuma commented on HADOOP-18045:
---

By the way, let's continue this discussion on the mailing list. I'll reply to 
the 'Hadoop-3.4 Release Update' thread.

> Disable TestDynamometerInfra
> 
>
> Key: HADOOP-18045
> URL: https://issues.apache.org/jira/browse/HADOOP-18045
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: test
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.3.2
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> This test is broken and there is no fix provided for a long time. Let's 
> disable the test to reduce the noise in the daily qbt job.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-18045) Disable TestDynamometerInfra

2024-01-20 Thread Takanobu Asanuma (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-18045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809087#comment-17809087
 ] 

Takanobu Asanuma commented on HADOOP-18045:
---

[~slfan1989] 
{quote}I found that the change log of hadoop-3.4.0-RC0 will contain the JIRA 
that have been released
{quote}
I think this is something we are doing on purpose. If the minor versions are 
different, they are separate. For example, regarding this JIRA issue, I believe 
the changelogs for both 3.3.2 and 3.4.0 should include HADOOP-18045. If I 
understand correctly, this has been our practice so far. Without 3.4.x in the 
Fix version, it's hard for users to know which 3.4.x it's in.

> Disable TestDynamometerInfra
> 
>
> Key: HADOOP-18045
> URL: https://issues.apache.org/jira/browse/HADOOP-18045
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: test
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.3.2
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> This test is broken and there is no fix provided for a long time. Let's 
> disable the test to reduce the noise in the daily qbt job.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-17957) Replace Guava VisibleForTesting by Hadoop's own annotation in hadoop-hdfs-project modules

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-17957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-17957:

Component/s: build
 common

> Replace Guava VisibleForTesting by Hadoop's own annotation in 
> hadoop-hdfs-project modules
> -
>
> Key: HADOOP-17957
> URL: https://issues.apache.org/jira/browse/HADOOP-17957
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: build, common
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-17957) Replace Guava VisibleForTesting by Hadoop's own annotation in hadoop-hdfs-project modules

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-17957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-17957:

Affects Version/s: 3.4.0

> Replace Guava VisibleForTesting by Hadoop's own annotation in 
> hadoop-hdfs-project modules
> -
>
> Key: HADOOP-17957
> URL: https://issues.apache.org/jira/browse/HADOOP-17957
> Project: Hadoop Common
>  Issue Type: Sub-task
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-17957) Replace Guava VisibleForTesting by Hadoop's own annotation in hadoop-hdfs-project modules

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-17957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-17957:

Hadoop Flags: Reviewed

> Replace Guava VisibleForTesting by Hadoop's own annotation in 
> hadoop-hdfs-project modules
> -
>
> Key: HADOOP-17957
> URL: https://issues.apache.org/jira/browse/HADOOP-17957
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: build, common
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-17959) Replace Guava VisibleForTesting by Hadoop's own annotation in hadoop-cloud-storage-project and hadoop-mapreduce-project modules

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-17959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-17959:

Hadoop Flags: Reviewed

> Replace Guava VisibleForTesting by Hadoop's own annotation in 
> hadoop-cloud-storage-project and hadoop-mapreduce-project modules
> ---
>
> Key: HADOOP-17959
> URL: https://issues.apache.org/jira/browse/HADOOP-17959
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: build, common
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-17959) Replace Guava VisibleForTesting by Hadoop's own annotation in hadoop-cloud-storage-project and hadoop-mapreduce-project modules

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-17959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-17959:

Affects Version/s: 3.4.0

> Replace Guava VisibleForTesting by Hadoop's own annotation in 
> hadoop-cloud-storage-project and hadoop-mapreduce-project modules
> ---
>
> Key: HADOOP-17959
> URL: https://issues.apache.org/jira/browse/HADOOP-17959
> Project: Hadoop Common
>  Issue Type: Sub-task
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-17959) Replace Guava VisibleForTesting by Hadoop's own annotation in hadoop-cloud-storage-project and hadoop-mapreduce-project modules

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-17959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-17959:

Component/s: build
 common

> Replace Guava VisibleForTesting by Hadoop's own annotation in 
> hadoop-cloud-storage-project and hadoop-mapreduce-project modules
> ---
>
> Key: HADOOP-17959
> URL: https://issues.apache.org/jira/browse/HADOOP-17959
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: build, common
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Comment Edited] (HADOOP-18045) Disable TestDynamometerInfra

2024-01-20 Thread Shilun Fan (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-18045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809086#comment-17809086
 ] 

Shilun Fan edited comment on HADOOP-18045 at 1/21/24 5:03 AM:
--

[~tasanuma] I found that the change log of hadoop-3.4.0-RC0 will contain the 
JIRA that have been released, so I modified the fix version of this part of 
JIRA. HADOOP-18045 has been released in the release version of 3.3.2. I think 
the change-log of 3.4.0 should not include this jira


was (Author: slfan1989):
[~tasanuma] I found that the change log of hadoop-3.4.0-RC0 will contain the 
JIRA that have been released, so I modified the fix version of this part of 
JIRA.

> Disable TestDynamometerInfra
> 
>
> Key: HADOOP-18045
> URL: https://issues.apache.org/jira/browse/HADOOP-18045
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: test
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.3.2
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> This test is broken and there is no fix provided for a long time. Let's 
> disable the test to reduce the noise in the daily qbt job.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-18045) Disable TestDynamometerInfra

2024-01-20 Thread Shilun Fan (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-18045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809086#comment-17809086
 ] 

Shilun Fan commented on HADOOP-18045:
-

[~tasanuma] I found that the change log of hadoop-3.4.0-RC0 will contain the 
JIRA that have been released, so I modified the fix version of this part of 
JIRA.

> Disable TestDynamometerInfra
> 
>
> Key: HADOOP-18045
> URL: https://issues.apache.org/jira/browse/HADOOP-18045
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: test
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.3.2
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> This test is broken and there is no fix provided for a long time. Let's 
> disable the test to reduce the noise in the daily qbt job.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-18045) Disable TestDynamometerInfra

2024-01-20 Thread Takanobu Asanuma (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-18045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809085#comment-17809085
 ] 

Takanobu Asanuma commented on HADOOP-18045:
---

Hi [~slfan1989], why did you remove 3.4.0 from the Fix versions? 'Fix version: 
3.3.x' doesn't imply that it includes 3.4.0. If the upcoming 3.4.0 release is 
going to include this JIRA issue, then we should add 3.4.0 to the Fix version.

> Disable TestDynamometerInfra
> 
>
> Key: HADOOP-18045
> URL: https://issues.apache.org/jira/browse/HADOOP-18045
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: test
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.3.2
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> This test is broken and there is no fix provided for a long time. Let's 
> disable the test to reduce the noise in the daily qbt job.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-17962) Replace Guava VisibleForTesting by Hadoop's own annotation in hadoop-tools modules

2024-01-20 Thread Shilun Fan (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-17962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809084#comment-17809084
 ] 

Shilun Fan commented on HADOOP-17962:
-

add Affects Version/s = 3.4.0 & Component/s = build,common & Hadoop Flags = 
Reviewed

> Replace Guava VisibleForTesting by Hadoop's own annotation in hadoop-tools 
> modules
> --
>
> Key: HADOOP-17962
> URL: https://issues.apache.org/jira/browse/HADOOP-17962
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: build, common
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-17962) Replace Guava VisibleForTesting by Hadoop's own annotation in hadoop-tools modules

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-17962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-17962:

Hadoop Flags: Reviewed

> Replace Guava VisibleForTesting by Hadoop's own annotation in hadoop-tools 
> modules
> --
>
> Key: HADOOP-17962
> URL: https://issues.apache.org/jira/browse/HADOOP-17962
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: build, common
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-17962) Replace Guava VisibleForTesting by Hadoop's own annotation in hadoop-tools modules

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-17962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-17962:

Component/s: build
 common

> Replace Guava VisibleForTesting by Hadoop's own annotation in hadoop-tools 
> modules
> --
>
> Key: HADOOP-17962
> URL: https://issues.apache.org/jira/browse/HADOOP-17962
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: build, common
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-17963) Replace Guava VisibleForTesting by Hadoop's own annotation in hadoop-yarn-project modules

2024-01-20 Thread Shilun Fan (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-17963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809083#comment-17809083
 ] 

Shilun Fan commented on HADOOP-17963:
-

add Affects Version/s = 3.4.0 & Component/s = build,common & Hadoop Flags = 
Reviewed

> Replace Guava VisibleForTesting by Hadoop's own annotation in 
> hadoop-yarn-project modules
> -
>
> Key: HADOOP-17963
> URL: https://issues.apache.org/jira/browse/HADOOP-17963
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: build, common
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-17967) Keep restrict-imports-enforcer-rule for Guava VisibleForTesting in hadoop-main pom

2024-01-20 Thread Shilun Fan (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-17967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809082#comment-17809082
 ] 

Shilun Fan commented on HADOOP-17967:
-

add Affects Version/s = 3.4.0 & Component/s = build,common & Hadoop Flags = 
Reviewed

> Keep restrict-imports-enforcer-rule for Guava VisibleForTesting in 
> hadoop-main pom
> --
>
> Key: HADOOP-17967
> URL: https://issues.apache.org/jira/browse/HADOOP-17967
> Project: Hadoop Common
>  Issue Type: Task
>  Components: build, common
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-17963) Replace Guava VisibleForTesting by Hadoop's own annotation in hadoop-yarn-project modules

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-17963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-17963:

Component/s: build
 common

> Replace Guava VisibleForTesting by Hadoop's own annotation in 
> hadoop-yarn-project modules
> -
>
> Key: HADOOP-17963
> URL: https://issues.apache.org/jira/browse/HADOOP-17963
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: build, common
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-17963) Replace Guava VisibleForTesting by Hadoop's own annotation in hadoop-yarn-project modules

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-17963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-17963:

Affects Version/s: 3.4.0

> Replace Guava VisibleForTesting by Hadoop's own annotation in 
> hadoop-yarn-project modules
> -
>
> Key: HADOOP-17963
> URL: https://issues.apache.org/jira/browse/HADOOP-17963
> Project: Hadoop Common
>  Issue Type: Sub-task
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-17963) Replace Guava VisibleForTesting by Hadoop's own annotation in hadoop-yarn-project modules

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-17963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-17963:

Hadoop Flags: Reviewed

> Replace Guava VisibleForTesting by Hadoop's own annotation in 
> hadoop-yarn-project modules
> -
>
> Key: HADOOP-17963
> URL: https://issues.apache.org/jira/browse/HADOOP-17963
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: build, common
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-17962) Replace Guava VisibleForTesting by Hadoop's own annotation in hadoop-tools modules

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-17962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-17962:

Affects Version/s: 3.4.0

> Replace Guava VisibleForTesting by Hadoop's own annotation in hadoop-tools 
> modules
> --
>
> Key: HADOOP-17962
> URL: https://issues.apache.org/jira/browse/HADOOP-17962
> Project: Hadoop Common
>  Issue Type: Sub-task
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-17967) Keep restrict-imports-enforcer-rule for Guava VisibleForTesting in hadoop-main pom

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-17967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-17967:

Hadoop Flags: Reviewed

> Keep restrict-imports-enforcer-rule for Guava VisibleForTesting in 
> hadoop-main pom
> --
>
> Key: HADOOP-17967
> URL: https://issues.apache.org/jira/browse/HADOOP-17967
> Project: Hadoop Common
>  Issue Type: Task
>  Components: build, common
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-17967) Keep restrict-imports-enforcer-rule for Guava VisibleForTesting in hadoop-main pom

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-17967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-17967:

Component/s: build
 common

> Keep restrict-imports-enforcer-rule for Guava VisibleForTesting in 
> hadoop-main pom
> --
>
> Key: HADOOP-17967
> URL: https://issues.apache.org/jira/browse/HADOOP-17967
> Project: Hadoop Common
>  Issue Type: Task
>  Components: build, common
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-17967) Keep restrict-imports-enforcer-rule for Guava VisibleForTesting in hadoop-main pom

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-17967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-17967:

Affects Version/s: 3.4.0

> Keep restrict-imports-enforcer-rule for Guava VisibleForTesting in 
> hadoop-main pom
> --
>
> Key: HADOOP-17967
> URL: https://issues.apache.org/jira/browse/HADOOP-17967
> Project: Hadoop Common
>  Issue Type: Task
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-17968) Migrate checkstyle module illegalimport to maven enforcer banned-illegal-imports

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-17968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-17968:

Component/s: build

> Migrate checkstyle module illegalimport to maven enforcer 
> banned-illegal-imports
> 
>
> Key: HADOOP-17968
> URL: https://issues.apache.org/jira/browse/HADOOP-17968
> Project: Hadoop Common
>  Issue Type: Task
>  Components: build
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> As discussed on PR [3503|https://github.com/apache/hadoop/pull/3503], we 
> should migrate existing imports provided in IllegalImport tag in 
> checkstyle.xml to maven-enforcer-plugin's banned-illegal-imports enforcer 
> rule so that build never succeeds in the presence of any of the illegal 
> imports.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-17968) Migrate checkstyle module illegalimport to maven enforcer banned-illegal-imports

2024-01-20 Thread Shilun Fan (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-17968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809081#comment-17809081
 ] 

Shilun Fan commented on HADOOP-17968:
-

add Affects Version/s = 3.4.0 & Component/s = build & Hadoop Flags = Reviewed

> Migrate checkstyle module illegalimport to maven enforcer 
> banned-illegal-imports
> 
>
> Key: HADOOP-17968
> URL: https://issues.apache.org/jira/browse/HADOOP-17968
> Project: Hadoop Common
>  Issue Type: Task
>  Components: build
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> As discussed on PR [3503|https://github.com/apache/hadoop/pull/3503], we 
> should migrate existing imports provided in IllegalImport tag in 
> checkstyle.xml to maven-enforcer-plugin's banned-illegal-imports enforcer 
> rule so that build never succeeds in the presence of any of the illegal 
> imports.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-17968) Migrate checkstyle module illegalimport to maven enforcer banned-illegal-imports

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-17968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-17968:

Hadoop Flags: Reviewed

> Migrate checkstyle module illegalimport to maven enforcer 
> banned-illegal-imports
> 
>
> Key: HADOOP-17968
> URL: https://issues.apache.org/jira/browse/HADOOP-17968
> Project: Hadoop Common
>  Issue Type: Task
>  Components: build
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> As discussed on PR [3503|https://github.com/apache/hadoop/pull/3503], we 
> should migrate existing imports provided in IllegalImport tag in 
> checkstyle.xml to maven-enforcer-plugin's banned-illegal-imports enforcer 
> rule so that build never succeeds in the presence of any of the illegal 
> imports.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-17968) Migrate checkstyle module illegalimport to maven enforcer banned-illegal-imports

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-17968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-17968:

Affects Version/s: 3.4.0

> Migrate checkstyle module illegalimport to maven enforcer 
> banned-illegal-imports
> 
>
> Key: HADOOP-17968
> URL: https://issues.apache.org/jira/browse/HADOOP-17968
> Project: Hadoop Common
>  Issue Type: Task
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> As discussed on PR [3503|https://github.com/apache/hadoop/pull/3503], we 
> should migrate existing imports provided in IllegalImport tag in 
> checkstyle.xml to maven-enforcer-plugin's banned-illegal-imports enforcer 
> rule so that build never succeeds in the presence of any of the illegal 
> imports.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-17971) Exclude IBM Java security classes from being shaded/relocated

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-17971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-17971:

Fix Version/s: (was: 3.4.0)

> Exclude IBM Java security classes from being shaded/relocated
> -
>
> Key: HADOOP-17971
> URL: https://issues.apache.org/jira/browse/HADOOP-17971
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: build
>Affects Versions: 3.3.1
>Reporter: Nicholas Marion
>Assignee: Nicholas Marion
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.2.3, 3.3.2
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> IBM Java classes are shaded in Hadoop libraries, e.g. hadoop-client-api. When 
> loaded by Spark, UserGroupInformation has exception:
> {noformat}
> org.apache.hadoop.security.KerberosAuthException: failure to login: 
> javax.security.auth.login.LoginException: unable to find LoginModule class: 
> org.apache.hadoop.shaded.com.ibm.security.auth.module.JAASLoginModule
>   at 
> org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1986)
>   at 
> org.apache.hadoop.security.UserGroupInformation.createLoginUser(UserGroupInformation.java:719)
>   at 
> org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:669)
>   at 
> org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:579)
>   at 
> org.apache.spark.util.Utils$.$anonfun$getCurrentUserName$1(Utils.scala:2609)
>   at 
> org.apache.spark.util.Utils$$$Lambda$1388/0x338e9c30.apply(Unknown 
> Source)
>   at scala.Option.getOrElse(Option.scala:189)
> {noformat}
> When I manually compile UserGroupInformation.java without maven (aka 
> relocation) and inject the class files into hadoop-client-api jar; it works.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-17971) Exclude IBM Java security classes from being shaded/relocated

2024-01-20 Thread Shilun Fan (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-17971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809080#comment-17809080
 ] 

Shilun Fan commented on HADOOP-17971:
-

3.3.2 release has been fixed, fix version removed 3.4.0

> Exclude IBM Java security classes from being shaded/relocated
> -
>
> Key: HADOOP-17971
> URL: https://issues.apache.org/jira/browse/HADOOP-17971
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: build
>Affects Versions: 3.3.1
>Reporter: Nicholas Marion
>Assignee: Nicholas Marion
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.2.3, 3.3.2
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> IBM Java classes are shaded in Hadoop libraries, e.g. hadoop-client-api. When 
> loaded by Spark, UserGroupInformation has exception:
> {noformat}
> org.apache.hadoop.security.KerberosAuthException: failure to login: 
> javax.security.auth.login.LoginException: unable to find LoginModule class: 
> org.apache.hadoop.shaded.com.ibm.security.auth.module.JAASLoginModule
>   at 
> org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1986)
>   at 
> org.apache.hadoop.security.UserGroupInformation.createLoginUser(UserGroupInformation.java:719)
>   at 
> org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:669)
>   at 
> org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:579)
>   at 
> org.apache.spark.util.Utils$.$anonfun$getCurrentUserName$1(Utils.scala:2609)
>   at 
> org.apache.spark.util.Utils$$$Lambda$1388/0x338e9c30.apply(Unknown 
> Source)
>   at scala.Option.getOrElse(Option.scala:189)
> {noformat}
> When I manually compile UserGroupInformation.java without maven (aka 
> relocation) and inject the class files into hadoop-client-api jar; it works.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-17998) Allow get command to run with multi threads.

2024-01-20 Thread Shilun Fan (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-17998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809079#comment-17809079
 ] 

Shilun Fan commented on HADOOP-17998:
-

3.3.2 release has been fixed, fix version removed 3.4.0

> Allow get command to run with multi threads.
> 
>
> Key: HADOOP-17998
> URL: https://issues.apache.org/jira/browse/HADOOP-17998
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: fs
>Affects Versions: 3.3.1
>Reporter: Chengwei Wang
>Assignee: Chengwei Wang
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.2.3, 3.3.2, 3.2.4
>
> Attachments: HADOOP-17998.001.patch, HADOOP-17998.002.patch
>
>  Time Spent: 3h 50m
>  Remaining Estimate: 0h
>
> CopyFromLocal/Put is enabled to run with multi-thread with HDFS-11786 and 
> HADOOP-14698, and make put dirs or multiple files faster.
> So, It's necessary to enable get and copyToLocal command run with 
> multi-thread.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-17998) Allow get command to run with multi threads.

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-17998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-17998:

Fix Version/s: (was: 3.4.0)

> Allow get command to run with multi threads.
> 
>
> Key: HADOOP-17998
> URL: https://issues.apache.org/jira/browse/HADOOP-17998
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: fs
>Affects Versions: 3.3.1
>Reporter: Chengwei Wang
>Assignee: Chengwei Wang
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.2.3, 3.3.2, 3.2.4
>
> Attachments: HADOOP-17998.001.patch, HADOOP-17998.002.patch
>
>  Time Spent: 3h 50m
>  Remaining Estimate: 0h
>
> CopyFromLocal/Put is enabled to run with multi-thread with HDFS-11786 and 
> HADOOP-14698, and make put dirs or multiple files faster.
> So, It's necessary to enable get and copyToLocal command run with 
> multi-thread.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18002) abfs rename idempotency broken -remove recovery

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18002:

Fix Version/s: (was: 3.4.0)

> abfs rename idempotency broken -remove recovery
> ---
>
> Key: HADOOP-18002
> URL: https://issues.apache.org/jira/browse/HADOOP-18002
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: fs/azure
>Affects Versions: 3.3.2
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.3.2
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> rename idempotency logic of HADOOP-17105   is broken as modtimes aren't 
> uodated on rename.
> remove, with the changes from the PR of HADOOP-17981.
> also fix delete recovery test after HADOOP-17934



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-18006) maven-enforcer-plugin's execution of banned-illegal-imports gets overridden in child poms

2024-01-20 Thread Shilun Fan (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-18006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809077#comment-17809077
 ] 

Shilun Fan commented on HADOOP-18006:
-

add Affects Version/s = 3.4.0 & Component/s = build

> maven-enforcer-plugin's execution of banned-illegal-imports gets overridden 
> in child poms
> -
>
> Key: HADOOP-18006
> URL: https://issues.apache.org/jira/browse/HADOOP-18006
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: build
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> When we specify any maven plugin with execution tag in the parent as well as 
> child modules, child module plugin overrides parent plugin. For instance, 
> when {{banned-illegal-imports}} is applied for any child module with only one 
> banned import (let’s say {{{}Preconditions{}}}), then only that banned import 
> is covered by that child module and all imports defined in parent module (e.g 
> Sets, Lists etc) are overridden and they are no longer applied.
> After this 
> [commit|https://github.com/apache/hadoop/commit/62c86eaa0e539a4307ca794e0fcd502a77ebceb8],
>  hadoop-hdfs module will not complain about {{Sets}} even if i import it from 
> guava banned imports but on the other hand, hadoop-yarn module doesn’t have 
> any child level {{banned-illegal-imports}} defined so yarn modules will fail 
> if {{Sets}} guava import is used.
> So going forward, it would be good to replace guava imports with Hadoop’s own 
> imports module-by-module and only at the end, we should add new entry to 
> parent pom {{banned-illegal-imports}} list.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18006) maven-enforcer-plugin's execution of banned-illegal-imports gets overridden in child poms

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18006:

Affects Version/s: 3.4.0

> maven-enforcer-plugin's execution of banned-illegal-imports gets overridden 
> in child poms
> -
>
> Key: HADOOP-18006
> URL: https://issues.apache.org/jira/browse/HADOOP-18006
> Project: Hadoop Common
>  Issue Type: Bug
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> When we specify any maven plugin with execution tag in the parent as well as 
> child modules, child module plugin overrides parent plugin. For instance, 
> when {{banned-illegal-imports}} is applied for any child module with only one 
> banned import (let’s say {{{}Preconditions{}}}), then only that banned import 
> is covered by that child module and all imports defined in parent module (e.g 
> Sets, Lists etc) are overridden and they are no longer applied.
> After this 
> [commit|https://github.com/apache/hadoop/commit/62c86eaa0e539a4307ca794e0fcd502a77ebceb8],
>  hadoop-hdfs module will not complain about {{Sets}} even if i import it from 
> guava banned imports but on the other hand, hadoop-yarn module doesn’t have 
> any child level {{banned-illegal-imports}} defined so yarn modules will fail 
> if {{Sets}} guava import is used.
> So going forward, it would be good to replace guava imports with Hadoop’s own 
> imports module-by-module and only at the end, we should add new entry to 
> parent pom {{banned-illegal-imports}} list.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18006) maven-enforcer-plugin's execution of banned-illegal-imports gets overridden in child poms

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18006:

Component/s: build

> maven-enforcer-plugin's execution of banned-illegal-imports gets overridden 
> in child poms
> -
>
> Key: HADOOP-18006
> URL: https://issues.apache.org/jira/browse/HADOOP-18006
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: build
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> When we specify any maven plugin with execution tag in the parent as well as 
> child modules, child module plugin overrides parent plugin. For instance, 
> when {{banned-illegal-imports}} is applied for any child module with only one 
> banned import (let’s say {{{}Preconditions{}}}), then only that banned import 
> is covered by that child module and all imports defined in parent module (e.g 
> Sets, Lists etc) are overridden and they are no longer applied.
> After this 
> [commit|https://github.com/apache/hadoop/commit/62c86eaa0e539a4307ca794e0fcd502a77ebceb8],
>  hadoop-hdfs module will not complain about {{Sets}} even if i import it from 
> guava banned imports but on the other hand, hadoop-yarn module doesn’t have 
> any child level {{banned-illegal-imports}} defined so yarn modules will fail 
> if {{Sets}} guava import is used.
> So going forward, it would be good to replace guava imports with Hadoop’s own 
> imports module-by-module and only at the end, we should add new entry to 
> parent pom {{banned-illegal-imports}} list.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18017) unguava: remove Preconditions from hadoop-yarn-project modules

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18017:

Affects Version/s: 3.4.0

> unguava: remove Preconditions from hadoop-yarn-project modules
> --
>
> Key: HADOOP-18017
> URL: https://issues.apache.org/jira/browse/HADOOP-18017
> Project: Hadoop Common
>  Issue Type: Sub-task
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Replace guava Preconditions by internal implementations that rely on java8+ 
> APIs in the hadoop.util for all modules in hadoop-yarn-project.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18017) unguava: remove Preconditions from hadoop-yarn-project modules

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18017:

Component/s: common

> unguava: remove Preconditions from hadoop-yarn-project modules
> --
>
> Key: HADOOP-18017
> URL: https://issues.apache.org/jira/browse/HADOOP-18017
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: common
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Replace guava Preconditions by internal implementations that rely on java8+ 
> APIs in the hadoop.util for all modules in hadoop-yarn-project.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18022) Add restrict-imports-enforcer-rule for Guava Preconditions in hadoop-main pom

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18022:

Component/s: common

> Add restrict-imports-enforcer-rule for Guava Preconditions in hadoop-main pom
> -
>
> Key: HADOOP-18022
> URL: https://issues.apache.org/jira/browse/HADOOP-18022
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: common
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 4.5h
>  Remaining Estimate: 0h
>
> Add restrict-imports-enforcer-rule for Guava Preconditions in hadoop-main pom 
> to restrict any new import in future. Remove any remaining usages of Guava 
> Preconditions from the codebase.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-18023) Allow cp command to run with multi threads.

2024-01-20 Thread Shilun Fan (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-18023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809076#comment-17809076
 ] 

Shilun Fan commented on HADOOP-18023:
-

3.3.2 release has been fixed, fix version removed 3.4.0

> Allow cp command to run with multi threads.
> ---
>
> Key: HADOOP-18023
> URL: https://issues.apache.org/jira/browse/HADOOP-18023
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: fs
>Reporter: Chengwei Wang
>Assignee: Chengwei Wang
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.2.3, 3.3.2
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Allow _*hadoop fs -cp*_ command to  run with multi-thread to improve copy 
> speed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18023) Allow cp command to run with multi threads.

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18023:

Fix Version/s: (was: 3.4.0)

> Allow cp command to run with multi threads.
> ---
>
> Key: HADOOP-18023
> URL: https://issues.apache.org/jira/browse/HADOOP-18023
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: fs
>Reporter: Chengwei Wang
>Assignee: Chengwei Wang
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.2.3, 3.3.2
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Allow _*hadoop fs -cp*_ command to  run with multi-thread to improve copy 
> speed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18025) Upgrade HBase version to 1.7.1 for hbase1 profile

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18025:

Component/s: build

> Upgrade HBase version to 1.7.1 for hbase1 profile
> -
>
> Key: HADOOP-18025
> URL: https://issues.apache.org/jira/browse/HADOOP-18025
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: build
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18022) Add restrict-imports-enforcer-rule for Guava Preconditions in hadoop-main pom

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18022:

Affects Version/s: 3.4.0

> Add restrict-imports-enforcer-rule for Guava Preconditions in hadoop-main pom
> -
>
> Key: HADOOP-18022
> URL: https://issues.apache.org/jira/browse/HADOOP-18022
> Project: Hadoop Common
>  Issue Type: Sub-task
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 4.5h
>  Remaining Estimate: 0h
>
> Add restrict-imports-enforcer-rule for Guava Preconditions in hadoop-main pom 
> to restrict any new import in future. Remove any remaining usages of Guava 
> Preconditions from the codebase.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18025) Upgrade HBase version to 1.7.1 for hbase1 profile

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18025:

Affects Version/s: 3.4.0

> Upgrade HBase version to 1.7.1 for hbase1 profile
> -
>
> Key: HADOOP-18025
> URL: https://issues.apache.org/jira/browse/HADOOP-18025
> Project: Hadoop Common
>  Issue Type: Sub-task
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18027) Include static imports in the maven plugin rules

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18027:

Affects Version/s: 3.4.0

> Include static imports in the maven plugin rules
> 
>
> Key: HADOOP-18027
> URL: https://issues.apache.org/jira/browse/HADOOP-18027
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: build
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> Maven enforcer plugin to ban illegal imports require explicit mention of 
> static imports in order to evaluate whether any publicly accessible static 
> entities from the banned classes are directly imported by Hadoop code.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18027) Include static imports in the maven plugin rules

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18027:

Component/s: build

> Include static imports in the maven plugin rules
> 
>
> Key: HADOOP-18027
> URL: https://issues.apache.org/jira/browse/HADOOP-18027
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: build
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> Maven enforcer plugin to ban illegal imports require explicit mention of 
> static imports in order to evaluate whether any publicly accessible static 
> entities from the banned classes are directly imported by Hadoop code.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Assigned] (HADOOP-18039) Upgrade hbase2 version and fix TestTimelineWriterHBaseDown

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan reassigned HADOOP-18039:
---

Assignee: Viraj Jasani  (was: Shilun Fan)

> Upgrade hbase2 version and fix TestTimelineWriterHBaseDown
> --
>
> Key: HADOOP-18039
> URL: https://issues.apache.org/jira/browse/HADOOP-18039
> Project: Hadoop Common
>  Issue Type: Sub-task
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> As mentioned on the parent Jira, we can't upgrade hbase2 profile version 
> beyond 2.2.4 until we either have hbase 2 artifacts available that are built 
> with hadoop 3 profile by default or hbase 3 is rolled out (hbase 3 is 
> compatible with hadoop 3 versions only).
> Let's upgrade hbase2 profile version to 2.2.4 as part of this Jira and also 
> fix TestTimelineWriterHBaseDown to create connection only after mini cluster 
> is up.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18039) Upgrade hbase2 version and fix TestTimelineWriterHBaseDown

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18039:

Component/s: build

> Upgrade hbase2 version and fix TestTimelineWriterHBaseDown
> --
>
> Key: HADOOP-18039
> URL: https://issues.apache.org/jira/browse/HADOOP-18039
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: build
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> As mentioned on the parent Jira, we can't upgrade hbase2 profile version 
> beyond 2.2.4 until we either have hbase 2 artifacts available that are built 
> with hadoop 3 profile by default or hbase 3 is rolled out (hbase 3 is 
> compatible with hadoop 3 versions only).
> Let's upgrade hbase2 profile version to 2.2.4 as part of this Jira and also 
> fix TestTimelineWriterHBaseDown to create connection only after mini cluster 
> is up.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-18040) Use maven.test.failure.ignore instead of ignoreTestFailure

2024-01-20 Thread Shilun Fan (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-18040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809075#comment-17809075
 ] 

Shilun Fan commented on HADOOP-18040:
-

3.3.2 release has been fixed, fix version removed 3.4.0

> Use maven.test.failure.ignore instead of ignoreTestFailure
> --
>
> Key: HADOOP-18040
> URL: https://issues.apache.org/jira/browse/HADOOP-18040
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: build
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
>  Labels: pull-request-available
> Fix For: 2.10.2, 3.3.2, 3.2.4
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> In HADOOP-16596, "ignoreTestFailure" variable was introduced to ignore unit 
> test failure, however, Maven property "maven.test.failure.ignore" can be used 
> instead and it can simplify the pom.xml.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18039) Upgrade hbase2 version and fix TestTimelineWriterHBaseDown

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18039:

Affects Version/s: 3.4.0

> Upgrade hbase2 version and fix TestTimelineWriterHBaseDown
> --
>
> Key: HADOOP-18039
> URL: https://issues.apache.org/jira/browse/HADOOP-18039
> Project: Hadoop Common
>  Issue Type: Sub-task
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> As mentioned on the parent Jira, we can't upgrade hbase2 profile version 
> beyond 2.2.4 until we either have hbase 2 artifacts available that are built 
> with hadoop 3 profile by default or hbase 3 is rolled out (hbase 3 is 
> compatible with hadoop 3 versions only).
> Let's upgrade hbase2 profile version to 2.2.4 as part of this Jira and also 
> fix TestTimelineWriterHBaseDown to create connection only after mini cluster 
> is up.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Assigned] (HADOOP-18039) Upgrade hbase2 version and fix TestTimelineWriterHBaseDown

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan reassigned HADOOP-18039:
---

Assignee: Shilun Fan  (was: Viraj Jasani)

> Upgrade hbase2 version and fix TestTimelineWriterHBaseDown
> --
>
> Key: HADOOP-18039
> URL: https://issues.apache.org/jira/browse/HADOOP-18039
> Project: Hadoop Common
>  Issue Type: Sub-task
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Shilun Fan
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> As mentioned on the parent Jira, we can't upgrade hbase2 profile version 
> beyond 2.2.4 until we either have hbase 2 artifacts available that are built 
> with hadoop 3 profile by default or hbase 3 is rolled out (hbase 3 is 
> compatible with hadoop 3 versions only).
> Let's upgrade hbase2 profile version to 2.2.4 as part of this Jira and also 
> fix TestTimelineWriterHBaseDown to create connection only after mini cluster 
> is up.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18040) Use maven.test.failure.ignore instead of ignoreTestFailure

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18040:

Fix Version/s: (was: 3.4.0)

> Use maven.test.failure.ignore instead of ignoreTestFailure
> --
>
> Key: HADOOP-18040
> URL: https://issues.apache.org/jira/browse/HADOOP-18040
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: build
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
>  Labels: pull-request-available
> Fix For: 2.10.2, 3.3.2, 3.2.4
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> In HADOOP-16596, "ignoreTestFailure" variable was introduced to ignore unit 
> test failure, however, Maven property "maven.test.failure.ignore" can be used 
> instead and it can simplify the pom.xml.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18043) Use mina-core 2.0.22 to fix LDAP unit test failures

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18043:

Affects Version/s: 3.4.0

> Use mina-core 2.0.22 to fix LDAP unit test failures
> ---
>
> Key: HADOOP-18043
> URL: https://issues.apache.org/jira/browse/HADOOP-18043
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: test
>Affects Versions: 3.4.0
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> TestMultiSchemeAuthenticationHandler fails on trunk
> {code:java}
> [INFO] Running 
> org.apache.hadoop.security.authentication.server.TestMultiSchemeAuthenticationHandler
> [ERROR] Tests run: 4, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 7.957 
> s <<< FAILURE! - in 
> org.apache.hadoop.security.authentication.server.TestMultiSchemeAuthenticationHandler
> [ERROR] 
> testRequestWithLdapAuthorization(org.apache.hadoop.security.authentication.server.TestMultiSchemeAuthenticationHandler)
>   Time elapsed: 1.663 s  <<< ERROR!
> org.apache.hadoop.security.authentication.client.AuthenticationException: 
> Error validating LDAP user
>   at 
> org.apache.hadoop.security.authentication.server.LdapAuthenticationHandler.authenticateWithoutTlsExtension(LdapAuthenticationHandler.java:310)
>   at 
> org.apache.hadoop.security.authentication.server.LdapAuthenticationHandler.authenticateUser(LdapAuthenticationHandler.java:240)
>   at 
> org.apache.hadoop.security.authentication.server.LdapAuthenticationHandler.authenticate(LdapAuthenticationHandler.java:202)
>   at 
> org.apache.hadoop.security.authentication.server.MultiSchemeAuthenticationHandler.authenticate(MultiSchemeAuthenticationHandler.java:197)
>   at 
> org.apache.hadoop.security.authentication.server.TestMultiSchemeAuthenticationHandler.testRequestWithLdapAuthorization(TestMultiSchemeAuthenticationHandler.java:161)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>   at 
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
>   at 
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at java.lang.Thread.run(Thread.java:748)
> Caused by: javax.naming.NamingException: LDAP connection has been closed
>   at com.sun.jndi.ldap.LdapRequest.getReplyBer(LdapRequest.java:133)
>   at com.sun.jndi.ldap.Connection.readReply(Connection.java:469)
>   at com.sun.jndi.ldap.LdapClient.ldapBind(LdapClient.java:365)
>   at com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:214)
>   at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2897)
>   at com.sun.jndi.ldap.LdapCtx.(LdapCtx.java:347)
>   at 
> com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxFromUrl(LdapCtxFactory.java:225)
>   at com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:189)
>   at 
> com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:243)
>   at 
> com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:154)
>   at 
> com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:84)
>   at 
> javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:695)
>   at 
> javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:313)
>   at javax.naming.InitialContext.init(InitialContext.java:244)
>   at javax.naming.InitialContext.(InitialContext.java:216)
>   at 
> javax.naming.directory.InitialDirContext.(InitialDirContext.java:101)
>   at 
> org.apache.hadoop.security.authentication.server.LdapAuthenticationHandler.authenticateWithoutTlsExtension(LdapAuthenticationHandler.java:305)
>   ... 16 more {code}
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional 

[jira] [Updated] (HADOOP-18052) Support Apple Silicon in start-build-env.sh

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18052:

Fix Version/s: (was: 3.4.0)

> Support Apple Silicon in start-build-env.sh
> ---
>
> Key: HADOOP-18052
> URL: https://issues.apache.org/jira/browse/HADOOP-18052
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: build
> Environment: M1 Pro. MacOS 12.0.1. Docker for Mac.
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.3.2
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> start-build-env.sh uses Dockerfile for x86 in M1 Mac, and the Dockerfile sets 
> wrong JAVA_HOME. Dockerfile_aarch64 should be used instead.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18045) Disable TestDynamometerInfra

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18045:

Fix Version/s: (was: 3.4.0)

> Disable TestDynamometerInfra
> 
>
> Key: HADOOP-18045
> URL: https://issues.apache.org/jira/browse/HADOOP-18045
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: test
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.3.2
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> This test is broken and there is no fix provided for a long time. Let's 
> disable the test to reduce the noise in the daily qbt job.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-18045) Disable TestDynamometerInfra

2024-01-20 Thread Shilun Fan (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-18045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809074#comment-17809074
 ] 

Shilun Fan commented on HADOOP-18045:
-

3.3.2 release has been fixed, fix version removed 3.4.0

> Disable TestDynamometerInfra
> 
>
> Key: HADOOP-18045
> URL: https://issues.apache.org/jira/browse/HADOOP-18045
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: test
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.3.2
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> This test is broken and there is no fix provided for a long time. Let's 
> disable the test to reduce the noise in the daily qbt job.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-18052) Support Apple Silicon in start-build-env.sh

2024-01-20 Thread Shilun Fan (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-18052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809073#comment-17809073
 ] 

Shilun Fan commented on HADOOP-18052:
-

3.3.2 release has been fixed, fix version removed 3.4.0

> Support Apple Silicon in start-build-env.sh
> ---
>
> Key: HADOOP-18052
> URL: https://issues.apache.org/jira/browse/HADOOP-18052
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: build
> Environment: M1 Pro. MacOS 12.0.1. Docker for Mac.
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.3.2
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> start-build-env.sh uses Dockerfile for x86 in M1 Mac, and the Dockerfile sets 
> wrong JAVA_HOME. Dockerfile_aarch64 should be used instead.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18077) ProfileOutputServlet unable to proceed due to NPE

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18077:

Component/s: common

> ProfileOutputServlet unable to proceed due to NPE
> -
>
> Key: HADOOP-18077
> URL: https://issues.apache.org/jira/browse/HADOOP-18077
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> ProfileOutputServlet context doesn't have Hadoop configs available and hence 
> async profiler redirection to output servlet is failing to identify if admin 
> access is allowed:
> {code:java}
> HTTP ERROR 500 java.lang.NullPointerException
> URI:    /prof-output-hadoop/async-prof-pid-98613-cpu-2.html
> STATUS:    500
> MESSAGE:    java.lang.NullPointerException
> SERVLET:    org.apache.hadoop.http.ProfileOutputServlet-58c34bb3
> CAUSED BY:    java.lang.NullPointerException
> Caused by:
> java.lang.NullPointerException
>     at 
> org.apache.hadoop.http.HttpServer2.isInstrumentationAccessAllowed(HttpServer2.java:1619)
>     at 
> org.apache.hadoop.http.ProfileOutputServlet.doGet(ProfileOutputServlet.java:51)
>     at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
>     at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
>     at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
>     at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:550)
>     at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
>     at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
>     at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
>     at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
>     at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
>     at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
>     at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>     at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:234)
>     at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
>     at 
> org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:179)
>     at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
>     at org.eclipse.jetty.server.Server.handle(Server.java:516)
>     at 
> org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:400)
>     at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:645)
>     at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:392)
>     at 
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
>     at 
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
>     at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
>     at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
>     at 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
>     at 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
>     at 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
>     at 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
>     at 
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
>     at 
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
>     at 
> org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
>     at java.lang.Thread.run(Thread.java:748){code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18077) ProfileOutputServlet unable to proceed due to NPE

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18077:

Affects Version/s: 3.4.0

> ProfileOutputServlet unable to proceed due to NPE
> -
>
> Key: HADOOP-18077
> URL: https://issues.apache.org/jira/browse/HADOOP-18077
> Project: Hadoop Common
>  Issue Type: Bug
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> ProfileOutputServlet context doesn't have Hadoop configs available and hence 
> async profiler redirection to output servlet is failing to identify if admin 
> access is allowed:
> {code:java}
> HTTP ERROR 500 java.lang.NullPointerException
> URI:    /prof-output-hadoop/async-prof-pid-98613-cpu-2.html
> STATUS:    500
> MESSAGE:    java.lang.NullPointerException
> SERVLET:    org.apache.hadoop.http.ProfileOutputServlet-58c34bb3
> CAUSED BY:    java.lang.NullPointerException
> Caused by:
> java.lang.NullPointerException
>     at 
> org.apache.hadoop.http.HttpServer2.isInstrumentationAccessAllowed(HttpServer2.java:1619)
>     at 
> org.apache.hadoop.http.ProfileOutputServlet.doGet(ProfileOutputServlet.java:51)
>     at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
>     at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
>     at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
>     at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:550)
>     at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
>     at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
>     at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
>     at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
>     at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
>     at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
>     at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>     at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:234)
>     at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
>     at 
> org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:179)
>     at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
>     at org.eclipse.jetty.server.Server.handle(Server.java:516)
>     at 
> org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:400)
>     at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:645)
>     at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:392)
>     at 
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
>     at 
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
>     at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
>     at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
>     at 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
>     at 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
>     at 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
>     at 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
>     at 
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
>     at 
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
>     at 
> org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
>     at java.lang.Thread.run(Thread.java:748){code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18089) Test coverage for Async profiler servlets

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18089:

Component/s: common

> Test coverage for Async profiler servlets
> -
>
> Key: HADOOP-18089
> URL: https://issues.apache.org/jira/browse/HADOOP-18089
> Project: Hadoop Common
>  Issue Type: Test
>  Components: common
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> As discussed in HADOOP-18077, we should provide sufficient test coverage to 
> discover any potential regression in async profiler servlets: ProfileServlet 
> and ProfileOutputServlet.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18089) Test coverage for Async profiler servlets

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18089:

Affects Version/s: 3.4.0

> Test coverage for Async profiler servlets
> -
>
> Key: HADOOP-18089
> URL: https://issues.apache.org/jira/browse/HADOOP-18089
> Project: Hadoop Common
>  Issue Type: Test
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> As discussed in HADOOP-18077, we should provide sufficient test coverage to 
> discover any potential regression in async profiler servlets: ProfileServlet 
> and ProfileOutputServlet.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18098) Basic verification for the release candidate vote

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18098:

Component/s: build

> Basic verification for the release candidate vote
> -
>
> Key: HADOOP-18098
> URL: https://issues.apache.org/jira/browse/HADOOP-18098
> Project: Hadoop Common
>  Issue Type: Task
>  Components: build
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> We should provide script for the basic sanity of Hadoop release candidates. 
> It should include:
>  * Signature
>  * Checksum
>  * Rat check
>  * Build from src
>  * Build tarball from src
>  
> Although we can include unit test as well, but overall unit test run is going 
> to be significantly higher, and precommit Jenkins builds provide better view 
> of UT sanity.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18098) Basic verification for the release candidate vote

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18098:

Affects Version/s: 3.4.0

> Basic verification for the release candidate vote
> -
>
> Key: HADOOP-18098
> URL: https://issues.apache.org/jira/browse/HADOOP-18098
> Project: Hadoop Common
>  Issue Type: Task
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> We should provide script for the basic sanity of Hadoop release candidates. 
> It should include:
>  * Signature
>  * Checksum
>  * Rat check
>  * Build from src
>  * Build tarball from src
>  
> Although we can include unit test as well, but overall unit test run is going 
> to be significantly higher, and precommit Jenkins builds provide better view 
> of UT sanity.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18110) ViewFileSystem: Add Support for Localized Trash Root

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18110:

Affects Version/s: 3.4.0

> ViewFileSystem: Add Support for Localized Trash Root
> 
>
> Key: HADOOP-18110
> URL: https://issues.apache.org/jira/browse/HADOOP-18110
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: common
>Affects Versions: 3.4.0
>Reporter: Xing Lin
>Assignee: Xing Lin
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0, 2.10.2
>
>  Time Spent: 5h
>  Remaining Estimate: 0h
>
> getTrashRoot() in ViewFileSystem calls getTrashRoot() from underlying 
> filesystem, to return the trash root. Most of the time, we get a trash root 
> in user home dir. This can lead to problems when an application wants to 
> delete a file in a mounted point using moveToTrash() in TrashPolicyDefault, 
> because we can not rename across multiple filesystems/hdfs namenodes. 
>  
> We propose the following extension to getTrashRoot/getTrashRoots in 
> ViewFileSystem: add a flag to return a localized trash root for 
> ViewFileSystem. A localized trash root is a trash root which starts from the 
> root of a mount point (e.g., /mountpointRoot/.Trash/\{user}). 
> * If CONFIG_VIEWFS_MOUNT_POINT_LOCAL_TRASH is not set to true, or
> * when the path p is in a snapshot or an encryption zone, return
> * the default trash root in user home dir.
> *
> * when CONFIG_VIEWFS_MOUNT_POINT_LOCAL_TRASH is set to true,
> * 1) if path p is mounted from the same targetFS as user home dir,
> * return a trash root in user home dir.
> * 2) else, return a trash root in the mounted targetFS
> *



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18131) Upgrade maven enforcer plugin and relevant dependencies

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18131:

Component/s: build

> Upgrade maven enforcer plugin and relevant dependencies
> ---
>
> Key: HADOOP-18131
> URL: https://issues.apache.org/jira/browse/HADOOP-18131
> Project: Hadoop Common
>  Issue Type: Task
>  Components: build
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 4h 50m
>  Remaining Estimate: 0h
>
> Maven enforcer plugin's latest version 3.0.0 has some noticeable improvements 
> (e.g. MENFORCER-350, MENFORCER-388, MENFORCER-353) and fixes for us to 
> incorporate. Besides, some of the relevant enforcer dependencies (e.g. extra 
> enforcer rules and restrict import enforcer) too have good improvements.
> We should upgrade maven enforcer plugin and the relevant dependencies.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18131) Upgrade maven enforcer plugin and relevant dependencies

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18131:

Affects Version/s: 3.4.0

> Upgrade maven enforcer plugin and relevant dependencies
> ---
>
> Key: HADOOP-18131
> URL: https://issues.apache.org/jira/browse/HADOOP-18131
> Project: Hadoop Common
>  Issue Type: Task
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 4h 50m
>  Remaining Estimate: 0h
>
> Maven enforcer plugin's latest version 3.0.0 has some noticeable improvements 
> (e.g. MENFORCER-350, MENFORCER-388, MENFORCER-353) and fixes for us to 
> incorporate. Besides, some of the relevant enforcer dependencies (e.g. extra 
> enforcer rules and restrict import enforcer) too have good improvements.
> We should upgrade maven enforcer plugin and the relevant dependencies.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-18144) getTrashRoot/s in ViewFileSystem should return viewFS path, not targetFS path

2024-01-20 Thread Shilun Fan (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-18144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809072#comment-17809072
 ] 

Shilun Fan commented on HADOOP-18144:
-

3.3.5 release has been fixed, fix version removed 3.4.0

> getTrashRoot/s in ViewFileSystem should return viewFS path, not targetFS path
> -
>
> Key: HADOOP-18144
> URL: https://issues.apache.org/jira/browse/HADOOP-18144
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: common
>Reporter: Xing Lin
>Assignee: Xing Lin
>Priority: Major
>  Labels: pull-request-available
> Fix For: 2.10.2, 3.3.5
>
>  Time Spent: 6h 10m
>  Remaining Estimate: 0h
>
> It is probably incorrect that we return a targetFS path from getTrashRoot() 
> in ViewFileSystem, as that path will be used later on by ViewFileSystem in 
> other operations, such as rename. ViewFileSystem is assuming the path that it 
> receives is a viewFS path, but not a target FS path. For example, rename() in 
> ViewFileSystem will call getUriPath() for src/dst path, which will remove the 
> scheme/authority and then try to resolve the path-only component. It thus 
> sometimes leads to incorrect path resolution, as we are doing the path 
> resolution again on a targetFS path. 
>  
> On the other hand, it is not always trivial/feasible to determine the correct 
> viewFS path for a given trash root in targetFS path. 
> Example:
> Assume we have a mount point for /user/foo -> abfs:/containerA
> User foo calls getTrashRoot("/a/b/c") and "/a/b/c" does not match any mount 
> point. We fall back to the fallback hdfs, which by default returns 
> hdfs://localhost/user/foo/.Trash. In this case, it is incorrect to return the 
> trash root as viewfs:/user/foo, as it will be resolved to the abfs mount 
> point, instead of the fallback hdfs.
>   



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18144) getTrashRoot/s in ViewFileSystem should return viewFS path, not targetFS path

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18144:

Fix Version/s: (was: 3.4.0)

> getTrashRoot/s in ViewFileSystem should return viewFS path, not targetFS path
> -
>
> Key: HADOOP-18144
> URL: https://issues.apache.org/jira/browse/HADOOP-18144
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: common
>Reporter: Xing Lin
>Assignee: Xing Lin
>Priority: Major
>  Labels: pull-request-available
> Fix For: 2.10.2, 3.3.5
>
>  Time Spent: 6h 10m
>  Remaining Estimate: 0h
>
> It is probably incorrect that we return a targetFS path from getTrashRoot() 
> in ViewFileSystem, as that path will be used later on by ViewFileSystem in 
> other operations, such as rename. ViewFileSystem is assuming the path that it 
> receives is a viewFS path, but not a target FS path. For example, rename() in 
> ViewFileSystem will call getUriPath() for src/dst path, which will remove the 
> scheme/authority and then try to resolve the path-only component. It thus 
> sometimes leads to incorrect path resolution, as we are doing the path 
> resolution again on a targetFS path. 
>  
> On the other hand, it is not always trivial/feasible to determine the correct 
> viewFS path for a given trash root in targetFS path. 
> Example:
> Assume we have a mount point for /user/foo -> abfs:/containerA
> User foo calls getTrashRoot("/a/b/c") and "/a/b/c" does not match any mount 
> point. We fall back to the fallback hdfs, which by default returns 
> hdfs://localhost/user/foo/.Trash. In this case, it is incorrect to return the 
> trash root as viewfs:/user/foo, as it will be resolved to the abfs mount 
> point, instead of the fallback hdfs.
>   



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18172) Change scope of getRootFallbackLink for InodeTree to make them accessible from outside package

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18172:

Fix Version/s: (was: 3.4.0)

> Change scope of getRootFallbackLink for InodeTree to make them accessible 
> from outside package
> --
>
> Key: HADOOP-18172
> URL: https://issues.apache.org/jira/browse/HADOOP-18172
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: Xing Lin
>Assignee: Xing Lin
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 2.10.2, 3.3.5
>
>  Time Spent: 5h 50m
>  Remaining Estimate: 0h
>
> Sometimes, we need to access rootFallBackLink in InodeTree from another 
> package. One such case is we extend from ViewFileSystem but want to put the 
> new filesystem in org.apache.hadoop.fs package, instead of 
> org.apache.hadoop.fs.viewfs package. As a result, we need make them public, 
> similar as what we did for getMountPoints() in HADOOP-18100. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-18172) Change scope of getRootFallbackLink for InodeTree to make them accessible from outside package

2024-01-20 Thread Shilun Fan (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-18172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809070#comment-17809070
 ] 

Shilun Fan commented on HADOOP-18172:
-

3.3.5 release has been fixed, fix version removed 3.4.0

> Change scope of getRootFallbackLink for InodeTree to make them accessible 
> from outside package
> --
>
> Key: HADOOP-18172
> URL: https://issues.apache.org/jira/browse/HADOOP-18172
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: Xing Lin
>Assignee: Xing Lin
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 2.10.2, 3.3.5
>
>  Time Spent: 5h 50m
>  Remaining Estimate: 0h
>
> Sometimes, we need to access rootFallBackLink in InodeTree from another 
> package. One such case is we extend from ViewFileSystem but want to put the 
> new filesystem in org.apache.hadoop.fs package, instead of 
> org.apache.hadoop.fs.viewfs package. As a result, we need make them public, 
> similar as what we did for getMountPoints() in HADOOP-18100. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18169) getDelegationTokens in ViewFs should also fetch the token from the fallback FS

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18169:

Fix Version/s: (was: 3.4.0)

> getDelegationTokens in ViewFs should also fetch the token from the fallback FS
> --
>
> Key: HADOOP-18169
> URL: https://issues.apache.org/jira/browse/HADOOP-18169
> Project: Hadoop Common
>  Issue Type: Bug
>Reporter: Xing Lin
>Assignee: Xing Lin
>Priority: Major
>  Labels: pull-request-available
> Fix For: 2.10.2, 3.3.5
>
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> getDelegationTokens in ViewFs does not include the delegationToken from the 
> fallback FS, while it should. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-18169) getDelegationTokens in ViewFs should also fetch the token from the fallback FS

2024-01-20 Thread Shilun Fan (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-18169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809071#comment-17809071
 ] 

Shilun Fan commented on HADOOP-18169:
-

3.3.5 release has been fixed, fix version removed 3.4.0

> getDelegationTokens in ViewFs should also fetch the token from the fallback FS
> --
>
> Key: HADOOP-18169
> URL: https://issues.apache.org/jira/browse/HADOOP-18169
> Project: Hadoop Common
>  Issue Type: Bug
>Reporter: Xing Lin
>Assignee: Xing Lin
>Priority: Major
>  Labels: pull-request-available
> Fix For: 2.10.2, 3.3.5
>
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> getDelegationTokens in ViewFs does not include the delegationToken from the 
> fallback FS, while it should. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18188) Support touch command for directory

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18188:

Component/s: common

> Support touch command for directory
> ---
>
> Key: HADOOP-18188
> URL: https://issues.apache.org/jira/browse/HADOOP-18188
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: common
>Affects Versions: 3.4.0
>Reporter: Akira Ajisaka
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> Currently hadoop fs -touch command cannot update the mtime and the atime of 
> directory. The feature would be useful when we check whether the filesystem 
> is ready to write or not without creating any file.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18188) Support touch command for directory

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18188:

Affects Version/s: 3.4.0

> Support touch command for directory
> ---
>
> Key: HADOOP-18188
> URL: https://issues.apache.org/jira/browse/HADOOP-18188
> Project: Hadoop Common
>  Issue Type: Improvement
>Affects Versions: 3.4.0
>Reporter: Akira Ajisaka
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> Currently hadoop fs -touch command cannot update the mtime and the atime of 
> directory. The feature would be useful when we check whether the filesystem 
> is ready to write or not without creating any file.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-18191) Log retry count while handling exceptions in RetryInvocationHandler

2024-01-20 Thread Shilun Fan (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-18191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809069#comment-17809069
 ] 

Shilun Fan commented on HADOOP-18191:
-

3.3.5 release has been fixed, fix version removed 3.4.0

> Log retry count while handling exceptions in RetryInvocationHandler
> ---
>
> Key: HADOOP-18191
> URL: https://issues.apache.org/jira/browse/HADOOP-18191
> Project: Hadoop Common
>  Issue Type: Task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 2.10.2, 3.2.4, 3.3.5
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> As part of failure handling in RetryInvocationHandler, we log details of the 
> Exception details with which API was invoked, failover attempts, delay.
> For the purpose of better debugging as well as fine-tuning of retry params, 
> it would be good to also log retry count that we already maintain in the 
> Counter object.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18196) Remove replace-guava from replacer plugin

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18196:

Component/s: build

> Remove replace-guava from replacer plugin
> -
>
> Key: HADOOP-18196
> URL: https://issues.apache.org/jira/browse/HADOOP-18196
> Project: Hadoop Common
>  Issue Type: Task
>  Components: build
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> While running the build, realized that all replacer plugin executions run 
> only after "banned-illegal-imports" enforcer plugin.
> For instance,
> {code:java}
> [INFO] --- maven-enforcer-plugin:3.0.0:enforce (banned-illegal-imports) @ 
> hadoop-cloud-storage ---
> [INFO] 
> [INFO] --- replacer:1.5.3:replace (replace-generated-sources) @ 
> hadoop-cloud-storage ---
> [INFO] Skipping
> [INFO] 
> [INFO] --- replacer:1.5.3:replace (replace-sources) @ hadoop-cloud-storage ---
> [INFO] Skipping
> [INFO] 
> [INFO] --- replacer:1.5.3:replace (replace-guava) @ hadoop-cloud-storage ---
> [INFO] Replacement run on 0 file.
> [INFO]  {code}
> Hence, if our source code uses com.google.common, banned-illegal-imports will 
> cause the build failure and replacer plugin would not even get executed.
> We should remove it as it is only redundant execution step.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18196) Remove replace-guava from replacer plugin

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18196:

Affects Version/s: 3.4.0

> Remove replace-guava from replacer plugin
> -
>
> Key: HADOOP-18196
> URL: https://issues.apache.org/jira/browse/HADOOP-18196
> Project: Hadoop Common
>  Issue Type: Task
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> While running the build, realized that all replacer plugin executions run 
> only after "banned-illegal-imports" enforcer plugin.
> For instance,
> {code:java}
> [INFO] --- maven-enforcer-plugin:3.0.0:enforce (banned-illegal-imports) @ 
> hadoop-cloud-storage ---
> [INFO] 
> [INFO] --- replacer:1.5.3:replace (replace-generated-sources) @ 
> hadoop-cloud-storage ---
> [INFO] Skipping
> [INFO] 
> [INFO] --- replacer:1.5.3:replace (replace-sources) @ hadoop-cloud-storage ---
> [INFO] Skipping
> [INFO] 
> [INFO] --- replacer:1.5.3:replace (replace-guava) @ hadoop-cloud-storage ---
> [INFO] Replacement run on 0 file.
> [INFO]  {code}
> Hence, if our source code uses com.google.common, banned-illegal-imports will 
> cause the build failure and replacer plugin would not even get executed.
> We should remove it as it is only redundant execution step.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18224) Upgrade maven compiler plugin to 3.10.1

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18224:

Component/s: build

> Upgrade maven compiler plugin to 3.10.1
> ---
>
> Key: HADOOP-18224
> URL: https://issues.apache.org/jira/browse/HADOOP-18224
> Project: Hadoop Common
>  Issue Type: Task
>  Components: build
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 6h 50m
>  Remaining Estimate: 0h
>
> Currently we are using maven-compiler-plugin 3.1 version, which is quite old 
> (2013) and it's also pulling in vulnerable log4j dependency:
> {code:java}
> [INFO]
> org.apache.maven.plugins:maven-compiler-plugin:maven-plugin:3.1:runtime
> [INFO]   org.apache.maven.plugins:maven-compiler-plugin:jar:3.1
> [INFO]   org.apache.maven:maven-plugin-api:jar:2.0.9
> [INFO]   org.apache.maven:maven-artifact:jar:2.0.9
> [INFO]   org.codehaus.plexus:plexus-utils:jar:1.5.1
> [INFO]   org.apache.maven:maven-core:jar:2.0.9
> [INFO]   org.apache.maven:maven-settings:jar:2.0.9
> [INFO]   org.apache.maven:maven-plugin-parameter-documenter:jar:2.0.9
> ...
> ...
> ...
> [INFO]   log4j:log4j:jar:1.2.12
> [INFO]   commons-logging:commons-logging-api:jar:1.1
> [INFO]   com.google.collections:google-collections:jar:1.0
> [INFO]   junit:junit:jar:3.8.2
>  {code}
>  
> We should upgrade to 3.10.1 (latest Mar, 2022) version of 
> maven-compiler-plugin.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18191) Log retry count while handling exceptions in RetryInvocationHandler

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18191:

Fix Version/s: (was: 3.4.0)

> Log retry count while handling exceptions in RetryInvocationHandler
> ---
>
> Key: HADOOP-18191
> URL: https://issues.apache.org/jira/browse/HADOOP-18191
> Project: Hadoop Common
>  Issue Type: Task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 2.10.2, 3.2.4, 3.3.5
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> As part of failure handling in RetryInvocationHandler, we log details of the 
> Exception details with which API was invoked, failover attempts, delay.
> For the purpose of better debugging as well as fine-tuning of retry params, 
> it would be good to also log retry count that we already maintain in the 
> Counter object.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18224) Upgrade maven compiler plugin to 3.10.1

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18224:

Affects Version/s: 3.4.0

> Upgrade maven compiler plugin to 3.10.1
> ---
>
> Key: HADOOP-18224
> URL: https://issues.apache.org/jira/browse/HADOOP-18224
> Project: Hadoop Common
>  Issue Type: Task
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 6h 50m
>  Remaining Estimate: 0h
>
> Currently we are using maven-compiler-plugin 3.1 version, which is quite old 
> (2013) and it's also pulling in vulnerable log4j dependency:
> {code:java}
> [INFO]
> org.apache.maven.plugins:maven-compiler-plugin:maven-plugin:3.1:runtime
> [INFO]   org.apache.maven.plugins:maven-compiler-plugin:jar:3.1
> [INFO]   org.apache.maven:maven-plugin-api:jar:2.0.9
> [INFO]   org.apache.maven:maven-artifact:jar:2.0.9
> [INFO]   org.codehaus.plexus:plexus-utils:jar:1.5.1
> [INFO]   org.apache.maven:maven-core:jar:2.0.9
> [INFO]   org.apache.maven:maven-settings:jar:2.0.9
> [INFO]   org.apache.maven:maven-plugin-parameter-documenter:jar:2.0.9
> ...
> ...
> ...
> [INFO]   log4j:log4j:jar:1.2.12
> [INFO]   commons-logging:commons-logging-api:jar:1.1
> [INFO]   com.google.collections:google-collections:jar:1.0
> [INFO]   junit:junit:jar:3.8.2
>  {code}
>  
> We should upgrade to 3.10.1 (latest Mar, 2022) version of 
> maven-compiler-plugin.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18228) Update hadoop-vote to use HADOOP_RC_VERSION dir

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18228:

Component/s: build

> Update hadoop-vote to use HADOOP_RC_VERSION dir
> ---
>
> Key: HADOOP-18228
> URL: https://issues.apache.org/jira/browse/HADOOP-18228
> Project: Hadoop Common
>  Issue Type: Task
>  Components: build
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> The recent changes in release script requires a minor change in hadoop-vote 
> to use Hadoop RC version dir before verifying signature and checksum of 
> .tar.gz files.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Assigned] (HADOOP-18228) Update hadoop-vote to use HADOOP_RC_VERSION dir

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan reassigned HADOOP-18228:
---

Assignee: Shilun Fan  (was: Viraj Jasani)

> Update hadoop-vote to use HADOOP_RC_VERSION dir
> ---
>
> Key: HADOOP-18228
> URL: https://issues.apache.org/jira/browse/HADOOP-18228
> Project: Hadoop Common
>  Issue Type: Task
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Shilun Fan
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> The recent changes in release script requires a minor change in hadoop-vote 
> to use Hadoop RC version dir before verifying signature and checksum of 
> .tar.gz files.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Assigned] (HADOOP-18228) Update hadoop-vote to use HADOOP_RC_VERSION dir

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan reassigned HADOOP-18228:
---

Assignee: Viraj Jasani  (was: Shilun Fan)

> Update hadoop-vote to use HADOOP_RC_VERSION dir
> ---
>
> Key: HADOOP-18228
> URL: https://issues.apache.org/jira/browse/HADOOP-18228
> Project: Hadoop Common
>  Issue Type: Task
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> The recent changes in release script requires a minor change in hadoop-vote 
> to use Hadoop RC version dir before verifying signature and checksum of 
> .tar.gz files.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18228) Update hadoop-vote to use HADOOP_RC_VERSION dir

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18228:

Affects Version/s: 3.4.0

> Update hadoop-vote to use HADOOP_RC_VERSION dir
> ---
>
> Key: HADOOP-18228
> URL: https://issues.apache.org/jira/browse/HADOOP-18228
> Project: Hadoop Common
>  Issue Type: Task
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> The recent changes in release script requires a minor change in hadoop-vote 
> to use Hadoop RC version dir before verifying signature and checksum of 
> .tar.gz files.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18240) Upgrade Yetus to 0.14.0

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18240:

Fix Version/s: (was: 3.4.0)

> Upgrade Yetus to 0.14.0
> ---
>
> Key: HADOOP-18240
> URL: https://issues.apache.org/jira/browse/HADOOP-18240
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: build
>Reporter: Akira Ajisaka
>Assignee: Ashutosh Gupta
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.2.4, 3.3.5, 2.10.3
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Yetus 0.14.0 is released. Let's upgrade.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18288) Total requests and total requests per sec served by RPC servers

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18288:

Fix Version/s: (was: 3.4.0)

> Total requests and total requests per sec served by RPC servers
> ---
>
> Key: HADOOP-18288
> URL: https://issues.apache.org/jira/browse/HADOOP-18288
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.3.5
>
>  Time Spent: 4h
>  Remaining Estimate: 0h
>
> RPC Servers provide bunch of useful information like num of open connections, 
> slow requests, num of in-progress handlers, RPC processing time, queue time 
> etc, however so far it doesn't provide accumulation of all requests as well 
> as current snapshot of requests per second served by the server. Exposing 
> them would benefit from operational viewpoint in identifying how busy the 
> servers have been and how much load they are currently serving in the 
> presence of cluster wide high load.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-18390) Fix out of sync import for HADOOP-18321

2024-01-20 Thread Shilun Fan (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-18390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809065#comment-17809065
 ] 

Shilun Fan commented on HADOOP-18390:
-

3.3.5 release has been fixed, fix version removed 3.4.0

> Fix out of sync import for HADOOP-18321
> ---
>
> Key: HADOOP-18390
> URL: https://issues.apache.org/jira/browse/HADOOP-18390
> Project: Hadoop Common
>  Issue Type: Bug
>Reporter: Ashutosh Gupta
>Assignee: Ashutosh Gupta
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 3.3.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Fix out of sync import for added as part of HADOOP-18321



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-18288) Total requests and total requests per sec served by RPC servers

2024-01-20 Thread Shilun Fan (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-18288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809067#comment-17809067
 ] 

Shilun Fan commented on HADOOP-18288:
-

3.3.5 release has been fixed, fix version removed 3.4.0

> Total requests and total requests per sec served by RPC servers
> ---
>
> Key: HADOOP-18288
> URL: https://issues.apache.org/jira/browse/HADOOP-18288
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.3.5
>
>  Time Spent: 4h
>  Remaining Estimate: 0h
>
> RPC Servers provide bunch of useful information like num of open connections, 
> slow requests, num of in-progress handlers, RPC processing time, queue time 
> etc, however so far it doesn't provide accumulation of all requests as well 
> as current snapshot of requests per second served by the server. Exposing 
> them would benefit from operational viewpoint in identifying how busy the 
> servers have been and how much load they are currently serving in the 
> presence of cluster wide high load.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18300) Update google-gson to 2.9.0

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18300:

Fix Version/s: (was: 3.4.0)

> Update google-gson to 2.9.0
> ---
>
> Key: HADOOP-18300
> URL: https://issues.apache.org/jira/browse/HADOOP-18300
> Project: Hadoop Common
>  Issue Type: Task
>  Components: build
>Reporter: Igor Dvorzhak
>Assignee: Igor Dvorzhak
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 3.2.4, 3.3.5
>
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> Update to the Gson 2.9.0 that has many 
> [fixes|https://github.com/google/gson/releases/tag/gson-parent-2.9.0], and 
> backward-compatible as long as Java 7+ is used.
> Fixes CVE-2022-25647; Dos when unmarshalling data



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18390) Fix out of sync import for HADOOP-18321

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18390:

Fix Version/s: (was: 3.4.0)

> Fix out of sync import for HADOOP-18321
> ---
>
> Key: HADOOP-18390
> URL: https://issues.apache.org/jira/browse/HADOOP-18390
> Project: Hadoop Common
>  Issue Type: Bug
>Reporter: Ashutosh Gupta
>Assignee: Ashutosh Gupta
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 3.3.5
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Fix out of sync import for added as part of HADOOP-18321



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-18397) Shutdown AWSSecurityTokenService when its resources are no longer in use

2024-01-20 Thread Shilun Fan (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-18397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809064#comment-17809064
 ] 

Shilun Fan commented on HADOOP-18397:
-

3.3.5 release has been fixed, fix version removed 3.4.0

> Shutdown AWSSecurityTokenService when its resources are no longer in use
> 
>
> Key: HADOOP-18397
> URL: https://issues.apache.org/jira/browse/HADOOP-18397
> Project: Hadoop Common
>  Issue Type: Task
>  Components: fs/s3
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.3.5
>
>
> AWSSecurityTokenService resources can be released whenever they are no longer 
> in use. The documentation of AWSSecurityTokenService#shutdown says while it 
> is not important for client to compulsorily shutdown the token service, 
> client can definitely perform early release whenever client no longer 
> requires token service resources. We achieve this by making STSClient 
> closable, so we can certainly utilize it in all places where it's suitable.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-18400) Fix file split duplicating records from a succeeding split when reading BZip2 text files

2024-01-20 Thread Shilun Fan (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-18400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809063#comment-17809063
 ] 

Shilun Fan commented on HADOOP-18400:
-

3.3.5 release has been fixed, fix version removed 3.4.0

>  Fix file split duplicating records from a succeeding split when reading 
> BZip2 text files 
> --
>
> Key: HADOOP-18400
> URL: https://issues.apache.org/jira/browse/HADOOP-18400
> Project: Hadoop Common
>  Issue Type: Bug
>Affects Versions: 3.3.3, 3.3.4
>Reporter: Ashutosh Gupta
>Assignee: Ashutosh Gupta
>Priority: Critical
>  Labels: pull-request-available
> Fix For: 3.3.5
>
>
> Fix data correctness issue with TextInputFormat that can occur when reading 
> BZip2 compressed text files. When a file split's range does not include the 
> start position of a BZip2 block, then it is expected to contain no records 
> (i.e. the split is empty). However, if it so happens that the end of this 
> split (exclusive) is at the start of a BZip2 block, then LineRecordReader 
> ends up returning all the records for that BZip2 block. This ends up 
> duplicating records read by a job because the next split would also end up 
> returning all the records for the same block (since its range would include 
> the start of that block).
> This bug does not get triggered when the file split's range does include the 
> start of at least one block and ends just before the start of another block. 
> The reason for this has to do with when BZip2CompressionInputStream updates 
> its position when using the BYBLOCK READMODE. Using this read mode, the 
> stream's position while reading only gets updated when reading the first byte 
> past an end of a block marker. The bug is that if the stream, when 
> initialized, was adjusted to be at the end of one block, then we don't update 
> the position after we read the first byte of the next block. Rather, we keep 
> the position to be equal to the next block marker we've initialized to. If 
> the exclusive end position of the split is equal to stream's position, 
> LineRecordReader will continue to read lines until the position is updated 
> (an an additional record in the next block is read if needed).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18397) Shutdown AWSSecurityTokenService when its resources are no longer in use

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18397:

Fix Version/s: (was: 3.4.0)

> Shutdown AWSSecurityTokenService when its resources are no longer in use
> 
>
> Key: HADOOP-18397
> URL: https://issues.apache.org/jira/browse/HADOOP-18397
> Project: Hadoop Common
>  Issue Type: Task
>  Components: fs/s3
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.3.5
>
>
> AWSSecurityTokenService resources can be released whenever they are no longer 
> in use. The documentation of AWSSecurityTokenService#shutdown says while it 
> is not important for client to compulsorily shutdown the token service, 
> client can definitely perform early release whenever client no longer 
> requires token service resources. We achieve this by making STSClient 
> closable, so we can certainly utilize it in all places where it's suitable.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18400) Fix file split duplicating records from a succeeding split when reading BZip2 text files

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18400:

Fix Version/s: (was: 3.4.0)

>  Fix file split duplicating records from a succeeding split when reading 
> BZip2 text files 
> --
>
> Key: HADOOP-18400
> URL: https://issues.apache.org/jira/browse/HADOOP-18400
> Project: Hadoop Common
>  Issue Type: Bug
>Affects Versions: 3.3.3, 3.3.4
>Reporter: Ashutosh Gupta
>Assignee: Ashutosh Gupta
>Priority: Critical
>  Labels: pull-request-available
> Fix For: 3.3.5
>
>
> Fix data correctness issue with TextInputFormat that can occur when reading 
> BZip2 compressed text files. When a file split's range does not include the 
> start position of a BZip2 block, then it is expected to contain no records 
> (i.e. the split is empty). However, if it so happens that the end of this 
> split (exclusive) is at the start of a BZip2 block, then LineRecordReader 
> ends up returning all the records for that BZip2 block. This ends up 
> duplicating records read by a job because the next split would also end up 
> returning all the records for the same block (since its range would include 
> the start of that block).
> This bug does not get triggered when the file split's range does include the 
> start of at least one block and ends just before the start of another block. 
> The reason for this has to do with when BZip2CompressionInputStream updates 
> its position when using the BYBLOCK READMODE. Using this read mode, the 
> stream's position while reading only gets updated when reading the first byte 
> past an end of a block marker. The bug is that if the stream, when 
> initialized, was adjusted to be at the end of one block, then we don't update 
> the position after we read the first byte of the next block. Rather, we keep 
> the position to be equal to the next block marker we've initialized to. If 
> the exclusive end position of the split is equal to stream's position, 
> LineRecordReader will continue to read lines until the position is updated 
> (an an additional record in the next block is read if needed).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-18444) Add Support for localized trash for ViewFileSystem in Trash.moveToAppropriateTrash

2024-01-20 Thread Shilun Fan (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-18444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809061#comment-17809061
 ] 

Shilun Fan commented on HADOOP-18444:
-

3.3.5 release has been fixed, fix version removed 3.4.0

> Add Support for localized trash for ViewFileSystem in 
> Trash.moveToAppropriateTrash
> --
>
> Key: HADOOP-18444
> URL: https://issues.apache.org/jira/browse/HADOOP-18444
> Project: Hadoop Common
>  Issue Type: Bug
>Reporter: Xing Lin
>Assignee: Xing Lin
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.3.5
>
>
> Trash.moveToAppropriateTrash is used by _hadoop cli -rm_ and hive, to move 
> files to trash. However, its current implementation does not support 
> localized trash policy we added to ViewFileSystem in HADOOP-18144.
> The reason is in moveToAppropriateTrash, it first resolves a path and then 
> uses the resolvedFs, to initialize the trash. As a result, it uses 
> getTrashRoot() implementation from targetFs, not ViewFileSystem. The new 
> localized trash policy we implemented in ViewFileSystem is not invoked.
> With the new localized trash policy for ViewFileSystem, the trash root would 
> be local to a mount point, thus, for ViewFileSystem with this flag turned on, 
> there is no need to resolve the path in moveToAppropriateTrash. Rename in 
> ViewFileSystem can resolve the logical paths correctly and be able to move a 
> file to trash within a mount point. 
> Code section of current moveToAppropriateTrash implementation.
> {code:java}
> public static boolean moveToAppropriateTrash(FileSystem fs, Path p,
> Configuration conf) throws IOException {
>   Path fullyResolvedPath = fs.resolvePath(p);
>   FileSystem fullyResolvedFs =
>   FileSystem.get(fullyResolvedPath.toUri(), conf);
>   ...
>   Trash trash = new Trash(fullyResolvedFs, conf);
>   return trash.moveToTrash(fullyResolvedPath);
> }{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18443) Upgrade snakeyaml to 1.32

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18443:

Fix Version/s: (was: 3.4.0)

> Upgrade snakeyaml to 1.32
> -
>
> Key: HADOOP-18443
> URL: https://issues.apache.org/jira/browse/HADOOP-18443
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: security
>Affects Versions: 3.3.3, 3.3.4
>Reporter: Ashutosh Gupta
>Assignee: Ashutosh Gupta
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.3.5, 3.2.5
>
>
> Upgrade snakeyaml to 1.32 to mitigate CVE-2022-25857 and 
> [CVE-2022-38752|https://github.com/advisories/GHSA-9w3m-gqgf-c4p9]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18444) Add Support for localized trash for ViewFileSystem in Trash.moveToAppropriateTrash

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18444:

Fix Version/s: (was: 3.4.0)

> Add Support for localized trash for ViewFileSystem in 
> Trash.moveToAppropriateTrash
> --
>
> Key: HADOOP-18444
> URL: https://issues.apache.org/jira/browse/HADOOP-18444
> Project: Hadoop Common
>  Issue Type: Bug
>Reporter: Xing Lin
>Assignee: Xing Lin
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.3.5
>
>
> Trash.moveToAppropriateTrash is used by _hadoop cli -rm_ and hive, to move 
> files to trash. However, its current implementation does not support 
> localized trash policy we added to ViewFileSystem in HADOOP-18144.
> The reason is in moveToAppropriateTrash, it first resolves a path and then 
> uses the resolvedFs, to initialize the trash. As a result, it uses 
> getTrashRoot() implementation from targetFs, not ViewFileSystem. The new 
> localized trash policy we implemented in ViewFileSystem is not invoked.
> With the new localized trash policy for ViewFileSystem, the trash root would 
> be local to a mount point, thus, for ViewFileSystem with this flag turned on, 
> there is no need to resolve the path in moveToAppropriateTrash. Rename in 
> ViewFileSystem can resolve the logical paths correctly and be able to move a 
> file to trash within a mount point. 
> Code section of current moveToAppropriateTrash implementation.
> {code:java}
> public static boolean moveToAppropriateTrash(FileSystem fs, Path p,
> Configuration conf) throws IOException {
>   Path fullyResolvedPath = fs.resolvePath(p);
>   FileSystem fullyResolvedFs =
>   FileSystem.get(fullyResolvedPath.toUri(), conf);
>   ...
>   Trash trash = new Trash(fullyResolvedFs, conf);
>   return trash.moveToTrash(fullyResolvedPath);
> }{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-18493) update jackson-databind 2.12.7.1 due to CVE fixes

2024-01-20 Thread Shilun Fan (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-18493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shilun Fan updated HADOOP-18493:

Fix Version/s: (was: 3.4.0)

> update jackson-databind 2.12.7.1 due to CVE fixes
> -
>
> Key: HADOOP-18493
> URL: https://issues.apache.org/jira/browse/HADOOP-18493
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: PJ Fanning
>Assignee: PJ Fanning
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.3.5
>
>
> * [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-42003]
>  * [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-42004]
>  * both fixes have been backported (the CVEs themselves need to be updated to 
> reflect this)
>  * [https://github.com/FasterXML/jackson-databind/pull/3622]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



  1   2   3   >