[jira] [Commented] (HADOOP-18959) Use builder for prefetch CachingBlockManager
[ https://issues.apache.org/jira/browse/HADOOP-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809057#comment-17809057 ] ASF GitHub Bot commented on HADOOP-18959: - slfan1989 commented on PR #6302: URL: https://github.com/apache/hadoop/pull/6302#issuecomment-1902493160 @virajjasani Thanks for the contribution! merge into branch-3.3. > Use builder for prefetch CachingBlockManager > > > Key: HADOOP-18959 > URL: https://issues.apache.org/jira/browse/HADOOP-18959 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > > Some of the recent changes (HADOOP-18399, HADOOP-18291, HADOOP-18829 etc) > have added more params for prefetch CachingBlockManager c'tor to process > read/write block requests. They have added too many params and more are > likely to be introduced later. We should use builder pattern to pass params. > This would also help consolidating required prefetch params into one single > place within S3ACachingInputStream, from scattered locations. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18959) Use builder for prefetch CachingBlockManager
[ https://issues.apache.org/jira/browse/HADOOP-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809056#comment-17809056 ] ASF GitHub Bot commented on HADOOP-18959: - slfan1989 merged PR #6302: URL: https://github.com/apache/hadoop/pull/6302 > Use builder for prefetch CachingBlockManager > > > Key: HADOOP-18959 > URL: https://issues.apache.org/jira/browse/HADOOP-18959 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > > Some of the recent changes (HADOOP-18399, HADOOP-18291, HADOOP-18829 etc) > have added more params for prefetch CachingBlockManager c'tor to process > read/write block requests. They have added too many params and more are > likely to be introduced later. We should use builder pattern to pass params. > This would also help consolidating required prefetch params into one single > place within S3ACachingInputStream, from scattered locations. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18959) Use builder for prefetch CachingBlockManager
[ https://issues.apache.org/jira/browse/HADOOP-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17803410#comment-17803410 ] ASF GitHub Bot commented on HADOOP-18959: - slfan1989 commented on PR #6302: URL: https://github.com/apache/hadoop/pull/6302#issuecomment-1878155784 @ahmarsuhail Thanks for your reminder! I modified the version to 3.4.0, the PR will be merged to branch3.3 soon. > Use builder for prefetch CachingBlockManager > > > Key: HADOOP-18959 > URL: https://issues.apache.org/jira/browse/HADOOP-18959 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > > Some of the recent changes (HADOOP-18399, HADOOP-18291, HADOOP-18829 etc) > have added more params for prefetch CachingBlockManager c'tor to process > read/write block requests. They have added too many params and more are > likely to be introduced later. We should use builder pattern to pass params. > This would also help consolidating required prefetch params into one single > place within S3ACachingInputStream, from scattered locations. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18959) Use builder for prefetch CachingBlockManager
[ https://issues.apache.org/jira/browse/HADOOP-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17803401#comment-17803401 ] Viraj Jasani commented on HADOOP-18959: --- [~slfan1989] this is already committed to trunk, only backport PR is pending for merge. > Use builder for prefetch CachingBlockManager > > > Key: HADOOP-18959 > URL: https://issues.apache.org/jira/browse/HADOOP-18959 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > > Some of the recent changes (HADOOP-18399, HADOOP-18291, HADOOP-18829 etc) > have added more params for prefetch CachingBlockManager c'tor to process > read/write block requests. They have added too many params and more are > likely to be introduced later. We should use builder pattern to pass params. > This would also help consolidating required prefetch params into one single > place within S3ACachingInputStream, from scattered locations. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18959) Use builder for prefetch CachingBlockManager
[ https://issues.apache.org/jira/browse/HADOOP-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17802832#comment-17802832 ] Shilun Fan commented on HADOOP-18959: - Bulk update: moved all 3.4.0 non-blocker issues, please move back if it is a blocker. Retarget 3.5.0. > Use builder for prefetch CachingBlockManager > > > Key: HADOOP-18959 > URL: https://issues.apache.org/jira/browse/HADOOP-18959 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > > Some of the recent changes (HADOOP-18399, HADOOP-18291, HADOOP-18829 etc) > have added more params for prefetch CachingBlockManager c'tor to process > read/write block requests. They have added too many params and more are > likely to be introduced later. We should use builder pattern to pass params. > This would also help consolidating required prefetch params into one single > place within S3ACachingInputStream, from scattered locations. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18959) Use builder for prefetch CachingBlockManager
[ https://issues.apache.org/jira/browse/HADOOP-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17792126#comment-17792126 ] ASF GitHub Bot commented on HADOOP-18959: - virajjasani commented on PR #6302: URL: https://github.com/apache/hadoop/pull/6302#issuecomment-1836418036 Sounds good, thanks @ahmarsuhail. For branch-3.4, this will be clean cherry-pick since it was just cut recently. > Use builder for prefetch CachingBlockManager > > > Key: HADOOP-18959 > URL: https://issues.apache.org/jira/browse/HADOOP-18959 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > > Some of the recent changes (HADOOP-18399, HADOOP-18291, HADOOP-18829 etc) > have added more params for prefetch CachingBlockManager c'tor to process > read/write block requests. They have added too many params and more are > likely to be introduced later. We should use builder pattern to pass params. > This would also help consolidating required prefetch params into one single > place within S3ACachingInputStream, from scattered locations. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18959) Use builder for prefetch CachingBlockManager
[ https://issues.apache.org/jira/browse/HADOOP-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17792023#comment-17792023 ] ASF GitHub Bot commented on HADOOP-18959: - ahmarsuhail commented on PR #6302: URL: https://github.com/apache/hadoop/pull/6302#issuecomment-1836013073 @virajjasani I recently cut the 3.4.0 branch..so I'm just checking where things should be backported to now. Will do this once I have some clarity there. > Use builder for prefetch CachingBlockManager > > > Key: HADOOP-18959 > URL: https://issues.apache.org/jira/browse/HADOOP-18959 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > > Some of the recent changes (HADOOP-18399, HADOOP-18291, HADOOP-18829 etc) > have added more params for prefetch CachingBlockManager c'tor to process > read/write block requests. They have added too many params and more are > likely to be introduced later. We should use builder pattern to pass params. > This would also help consolidating required prefetch params into one single > place within S3ACachingInputStream, from scattered locations. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18959) Use builder for prefetch CachingBlockManager
[ https://issues.apache.org/jira/browse/HADOOP-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17791487#comment-17791487 ] ASF GitHub Bot commented on HADOOP-18959: - virajjasani commented on PR #6302: URL: https://github.com/apache/hadoop/pull/6302#issuecomment-1833208901 @ahmarsuhail since the original PR is merged to trunk and this is backport with clean build and scale testing done, could you please help review/merge so that upcoming 3.3.x or 3.4.0 release doesn't miss it? (otherwise we might have to deal with backward compatibility issues) :) > Use builder for prefetch CachingBlockManager > > > Key: HADOOP-18959 > URL: https://issues.apache.org/jira/browse/HADOOP-18959 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > > Some of the recent changes (HADOOP-18399, HADOOP-18291, HADOOP-18829 etc) > have added more params for prefetch CachingBlockManager c'tor to process > read/write block requests. They have added too many params and more are > likely to be introduced later. We should use builder pattern to pass params. > This would also help consolidating required prefetch params into one single > place within S3ACachingInputStream, from scattered locations. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18959) Use builder for prefetch CachingBlockManager
[
https://issues.apache.org/jira/browse/HADOOP-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790354#comment-17790354
]
ASF GitHub Bot commented on HADOOP-18959:
-
hadoop-yetus commented on PR #6302:
URL: https://github.com/apache/hadoop/pull/6302#issuecomment-1829043274
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 6m 38s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ branch-3.3 Compile Tests _ |
| +0 :ok: | mvndep | 14m 49s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 34m 56s | | branch-3.3 passed |
| +1 :green_heart: | compile | 18m 36s | | branch-3.3 passed |
| +1 :green_heart: | checkstyle | 2m 49s | | branch-3.3 passed |
| +1 :green_heart: | mvnsite | 2m 38s | | branch-3.3 passed |
| +1 :green_heart: | javadoc | 1m 40s | | branch-3.3 passed |
| +1 :green_heart: | spotbugs | 3m 56s | | branch-3.3 passed |
| +1 :green_heart: | shadedclient | 35m 28s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 33s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 39s | | the patch passed |
| +1 :green_heart: | compile | 17m 55s | | the patch passed |
| +1 :green_heart: | javac | 17m 55s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 2m 39s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 35s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 35s | | the patch passed |
| +1 :green_heart: | spotbugs | 4m 12s | | the patch passed |
| +1 :green_heart: | shadedclient | 35m 39s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 28s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 2m 44s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 59s | | The patch does not
generate ASF License warnings. |
| | | 215m 1s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6302/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6302 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 8975975f556c 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2
15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-3.3 / ea8bc8e8f81d6726d93125655936f30a3f2a4e9d |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~18.04-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6302/1/testReport/ |
| Max. process+thread count | 1252 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws
U: . |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6302/1/console |
| versions | git=2.17.1 maven=3.6.0 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> Use builder for prefetch CachingBlockManager
>
>
> Key: HADOOP-18959
> URL: https://issues.apache.org/jira/browse/HADOOP-18959
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
>
> Some of the recent changes (HADOOP-18399, HADOOP-18291, HADOOP-18829 etc)
> have added more params for prefetch CachingBlockManager c'tor to process
> read/write block requests. They have added too many params and more are
> likely to be introduced later. We should use builder pattern to pass params.
> This would also help
[jira] [Commented] (HADOOP-18959) Use builder for prefetch CachingBlockManager
[ https://issues.apache.org/jira/browse/HADOOP-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790347#comment-17790347 ] ASF GitHub Bot commented on HADOOP-18959: - virajjasani commented on PR #6302: URL: https://github.com/apache/hadoop/pull/6302#issuecomment-1828988579 Tested against `us-west-2` ``` mvn clean verify -Dparallel-tests -DtestsThreadCount=8 -Dscale -Dprefetch ``` > Use builder for prefetch CachingBlockManager > > > Key: HADOOP-18959 > URL: https://issues.apache.org/jira/browse/HADOOP-18959 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > > Some of the recent changes (HADOOP-18399, HADOOP-18291, HADOOP-18829 etc) > have added more params for prefetch CachingBlockManager c'tor to process > read/write block requests. They have added too many params and more are > likely to be introduced later. We should use builder pattern to pass params. > This would also help consolidating required prefetch params into one single > place within S3ACachingInputStream, from scattered locations. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18959) Use builder for prefetch CachingBlockManager
[ https://issues.apache.org/jira/browse/HADOOP-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790317#comment-17790317 ] ASF GitHub Bot commented on HADOOP-18959: - virajjasani commented on PR #6302: URL: https://github.com/apache/hadoop/pull/6302#issuecomment-1828873712 Testing in progress > Use builder for prefetch CachingBlockManager > > > Key: HADOOP-18959 > URL: https://issues.apache.org/jira/browse/HADOOP-18959 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > > Some of the recent changes (HADOOP-18399, HADOOP-18291, HADOOP-18829 etc) > have added more params for prefetch CachingBlockManager c'tor to process > read/write block requests. They have added too many params and more are > likely to be introduced later. We should use builder pattern to pass params. > This would also help consolidating required prefetch params into one single > place within S3ACachingInputStream, from scattered locations. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18959) Use builder for prefetch CachingBlockManager
[ https://issues.apache.org/jira/browse/HADOOP-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790318#comment-17790318 ] ASF GitHub Bot commented on HADOOP-18959: - virajjasani commented on PR #6302: URL: https://github.com/apache/hadoop/pull/6302#issuecomment-1828873892 FYI @ahmarsuhail > Use builder for prefetch CachingBlockManager > > > Key: HADOOP-18959 > URL: https://issues.apache.org/jira/browse/HADOOP-18959 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > > Some of the recent changes (HADOOP-18399, HADOOP-18291, HADOOP-18829 etc) > have added more params for prefetch CachingBlockManager c'tor to process > read/write block requests. They have added too many params and more are > likely to be introduced later. We should use builder pattern to pass params. > This would also help consolidating required prefetch params into one single > place within S3ACachingInputStream, from scattered locations. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18959) Use builder for prefetch CachingBlockManager
[ https://issues.apache.org/jira/browse/HADOOP-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790315#comment-17790315 ] ASF GitHub Bot commented on HADOOP-18959: - virajjasani opened a new pull request, #6302: URL: https://github.com/apache/hadoop/pull/6302 branch-3.3 backport PR of #6240 > Use builder for prefetch CachingBlockManager > > > Key: HADOOP-18959 > URL: https://issues.apache.org/jira/browse/HADOOP-18959 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > > Some of the recent changes (HADOOP-18399, HADOOP-18291, HADOOP-18829 etc) > have added more params for prefetch CachingBlockManager c'tor to process > read/write block requests. They have added too many params and more are > likely to be introduced later. We should use builder pattern to pass params. > This would also help consolidating required prefetch params into one single > place within S3ACachingInputStream, from scattered locations. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18959) Use builder for prefetch CachingBlockManager
[ https://issues.apache.org/jira/browse/HADOOP-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17789064#comment-17789064 ] ASF GitHub Bot commented on HADOOP-18959: - ahmarsuhail merged PR #6240: URL: https://github.com/apache/hadoop/pull/6240 > Use builder for prefetch CachingBlockManager > > > Key: HADOOP-18959 > URL: https://issues.apache.org/jira/browse/HADOOP-18959 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > > Some of the recent changes (HADOOP-18399, HADOOP-18291, HADOOP-18829 etc) > have added more params for prefetch CachingBlockManager c'tor to process > read/write block requests. They have added too many params and more are > likely to be introduced later. We should use builder pattern to pass params. > This would also help consolidating required prefetch params into one single > place within S3ACachingInputStream, from scattered locations. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18959) Use builder for prefetch CachingBlockManager
[
https://issues.apache.org/jira/browse/HADOOP-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788201#comment-17788201
]
ASF GitHub Bot commented on HADOOP-18959:
-
hadoop-yetus commented on PR #6240:
URL: https://github.com/apache/hadoop/pull/6240#issuecomment-1820053567
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 20s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 12s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 19m 31s | | trunk passed |
| +1 :green_heart: | compile | 8m 19s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 7m 24s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 2m 5s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 26s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 58s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 53s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 8s | | trunk passed |
| +1 :green_heart: | shadedclient | 19m 40s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 47s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 0m 53s | | the patch passed |
| +1 :green_heart: | compile | 7m 52s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 7m 52s | | the patch passed |
| +1 :green_heart: | compile | 7m 30s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 7m 30s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 2m 2s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 24s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 1s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 1s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 18s | | the patch passed |
| +1 :green_heart: | shadedclient | 19m 29s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 15m 48s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 2m 11s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 36s | | The patch does not
generate ASF License warnings. |
| | | 143m 19s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6240/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6240 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux d2a94196bca3 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2
15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 1df9eb70b23aa631bbbe7a598206a847a83a1b82 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6240/6/testReport/ |
| Max. process+thread count | 3153 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws
U: . |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6240/6/console |
|
[jira] [Commented] (HADOOP-18959) Use builder for prefetch CachingBlockManager
[
https://issues.apache.org/jira/browse/HADOOP-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788176#comment-17788176
]
ASF GitHub Bot commented on HADOOP-18959:
-
virajjasani commented on code in PR #6240:
URL: https://github.com/apache/hadoop/pull/6240#discussion_r1399837597
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/prefetch/BlockManagerParameters.java:
##
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hadoop.fs.impl.prefetch;
+
+import org.apache.hadoop.classification.InterfaceAudience;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.LocalDirAllocator;
+import org.apache.hadoop.fs.statistics.DurationTrackerFactory;
+
+/**
+ * This class is used to provide parameters to {@link BlockManager}.
+ */
+@InterfaceAudience.Private
+public final class BlockManagerParameters {
+
+ /**
+ * Asynchronous tasks are performed in this pool.
+ */
+ private ExecutorServiceFuturePool futurePool;
+
+ /**
+ * Information about each block of the underlying file.
+ */
+ private BlockData blockData;
+
+ /**
+ * Size of the in-memory cache in terms of number of blocks.
+ */
+ private int bufferPoolSize;
+
+ /**
+ * Statistics for the stream.
+ */
+ private PrefetchingStatistics prefetchingStatistics;
+
+ /**
+ * The configuration object.
+ */
+ private Configuration conf;
+
+ /**
+ * The local dir allocator instance.
+ */
+ private LocalDirAllocator localDirAllocator;
+
+ /**
+ * Max blocks count to be kept in cache at any time.
+ */
+ private int maxBlocksCount;
+
+ /**
+ * Tracker with statistics to update.
+ */
+ private DurationTrackerFactory trackerFactory;
+
+ public ExecutorServiceFuturePool getFuturePool() {
+return futurePool;
+ }
+
+ public BlockData getBlockData() {
+return blockData;
+ }
+
+ public int getBufferPoolSize() {
+return bufferPoolSize;
+ }
+
+ public PrefetchingStatistics getPrefetchingStatistics() {
+return prefetchingStatistics;
+ }
+
+ public Configuration getConf() {
+return conf;
+ }
+
+ public LocalDirAllocator getLocalDirAllocator() {
+return localDirAllocator;
+ }
+
+ public int getMaxBlocksCount() {
+return maxBlocksCount;
+ }
+
+ public DurationTrackerFactory getTrackerFactory() {
+return trackerFactory;
+ }
+
+ public BlockManagerParameters setFuturePool(
Review Comment:
done
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/prefetch/TestS3ACachingBlockManager.java:
##
@@ -61,50 +65,93 @@ public class TestS3ACachingBlockManager extends
AbstractHadoopTestBase {
private final BlockData blockData = new BlockData(FILE_SIZE, BLOCK_SIZE);
+ private static final Configuration CONF =
+ S3ATestUtils.prepareTestConfiguration(new Configuration());
+
@Test
public void testArgChecks() throws Exception {
MockS3ARemoteObject s3File = new MockS3ARemoteObject(FILE_SIZE, false);
S3ARemoteObjectReader reader = new S3ARemoteObjectReader(s3File);
Configuration conf = new Configuration();
+BlockManagerParameters blockManagerParamsBuilder1 =
Review Comment:
done
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/prefetch/TestS3ACachingBlockManager.java:
##
@@ -244,9 +292,18 @@ private void testPrefetchHelper(boolean
forcePrefetchFailure)
throws IOException, InterruptedException {
MockS3ARemoteObject s3File = new MockS3ARemoteObject(FILE_SIZE, false);
S3ARemoteObjectReader reader = new S3ARemoteObjectReader(s3File);
+BlockManagerParameters blockManagerParamsBuilder =
Review Comment:
done
> Use builder for prefetch CachingBlockManager
>
>
> Key: HADOOP-18959
> URL: https://issues.apache.org/jira/browse/HADOOP-18959
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
>
> Some of
[jira] [Commented] (HADOOP-18959) Use builder for prefetch CachingBlockManager
[
https://issues.apache.org/jira/browse/HADOOP-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787983#comment-17787983
]
ASF GitHub Bot commented on HADOOP-18959:
-
ahmarsuhail commented on code in PR #6240:
URL: https://github.com/apache/hadoop/pull/6240#discussion_r1399090061
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/prefetch/BlockManagerParameters.java:
##
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hadoop.fs.impl.prefetch;
+
+import org.apache.hadoop.classification.InterfaceAudience;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.LocalDirAllocator;
+import org.apache.hadoop.fs.statistics.DurationTrackerFactory;
+
+/**
+ * This class is used to provide parameters to {@link BlockManager}.
+ */
+@InterfaceAudience.Private
+public final class BlockManagerParameters {
+
+ /**
+ * Asynchronous tasks are performed in this pool.
+ */
+ private ExecutorServiceFuturePool futurePool;
+
+ /**
+ * Information about each block of the underlying file.
+ */
+ private BlockData blockData;
+
+ /**
+ * Size of the in-memory cache in terms of number of blocks.
+ */
+ private int bufferPoolSize;
+
+ /**
+ * Statistics for the stream.
+ */
+ private PrefetchingStatistics prefetchingStatistics;
+
+ /**
+ * The configuration object.
+ */
+ private Configuration conf;
+
+ /**
+ * The local dir allocator instance.
+ */
+ private LocalDirAllocator localDirAllocator;
+
+ /**
+ * Max blocks count to be kept in cache at any time.
+ */
+ private int maxBlocksCount;
+
+ /**
+ * Tracker with statistics to update.
+ */
+ private DurationTrackerFactory trackerFactory;
+
+ public ExecutorServiceFuturePool getFuturePool() {
+return futurePool;
+ }
+
+ public BlockData getBlockData() {
+return blockData;
+ }
+
+ public int getBufferPoolSize() {
+return bufferPoolSize;
+ }
+
+ public PrefetchingStatistics getPrefetchingStatistics() {
+return prefetchingStatistics;
+ }
+
+ public Configuration getConf() {
+return conf;
+ }
+
+ public LocalDirAllocator getLocalDirAllocator() {
+return localDirAllocator;
+ }
+
+ public int getMaxBlocksCount() {
+return maxBlocksCount;
+ }
+
+ public DurationTrackerFactory getTrackerFactory() {
+return trackerFactory;
+ }
+
+ public BlockManagerParameters setFuturePool(
Review Comment:
could you rename the setters to "withFuturePool" instead of set, keeps
things uniform with other builders in the project.
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/prefetch/TestS3ACachingBlockManager.java:
##
@@ -61,50 +65,93 @@ public class TestS3ACachingBlockManager extends
AbstractHadoopTestBase {
private final BlockData blockData = new BlockData(FILE_SIZE, BLOCK_SIZE);
+ private static final Configuration CONF =
+ S3ATestUtils.prepareTestConfiguration(new Configuration());
+
@Test
public void testArgChecks() throws Exception {
MockS3ARemoteObject s3File = new MockS3ARemoteObject(FILE_SIZE, false);
S3ARemoteObjectReader reader = new S3ARemoteObjectReader(s3File);
Configuration conf = new Configuration();
+BlockManagerParameters blockManagerParamsBuilder1 =
Review Comment:
with the builder, this test is getting really long now. How about we split
it into separate tests? One test for the valid blockManager and the assertions
on it that are done. And then split each of the illegal arg ones into a
separate test? eg: testNullFuturePool(), etc.
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/prefetch/BlockManagerParameters.java:
##
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the
[jira] [Commented] (HADOOP-18959) Use builder for prefetch CachingBlockManager
[
https://issues.apache.org/jira/browse/HADOOP-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787881#comment-17787881
]
ASF GitHub Bot commented on HADOOP-18959:
-
hadoop-yetus commented on PR #6240:
URL: https://github.com/apache/hadoop/pull/6240#issuecomment-1818459112
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 20s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 13m 35s | | Maven dependency ordering for branch |
| -1 :x: | mvninstall | 18m 45s |
[/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6240/5/artifact/out/branch-mvninstall-root.txt)
| root in trunk failed. |
| +1 :green_heart: | compile | 8m 20s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 7m 30s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 59s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 23s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 5s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 57s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 8s | | trunk passed |
| +1 :green_heart: | shadedclient | 19m 30s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 20s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 0m 47s | | the patch passed |
| +1 :green_heart: | compile | 7m 55s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 7m 55s | | the patch passed |
| +1 :green_heart: | compile | 7m 33s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 7m 33s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 54s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 23s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 58s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 1s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 22s | | the patch passed |
| +1 :green_heart: | shadedclient | 19m 43s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 15m 47s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 2m 13s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not
generate ASF License warnings. |
| | | 141m 36s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6240/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6240 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux e8edd1dac598 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2
15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 6f217d2c9bb795ee7b19ac333ddf5d5ac377c9d7 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6240/5/testReport/ |
| Max. process+thread count | 3152 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common
[jira] [Commented] (HADOOP-18959) Use builder for prefetch CachingBlockManager
[
https://issues.apache.org/jira/browse/HADOOP-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787811#comment-17787811
]
ASF GitHub Bot commented on HADOOP-18959:
-
hadoop-yetus commented on PR #6240:
URL: https://github.com/apache/hadoop/pull/6240#issuecomment-1818251148
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 6m 46s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 20s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 19m 38s | | trunk passed |
| +1 :green_heart: | compile | 8m 15s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 7m 27s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 2m 4s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 23s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 5s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 1s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 13s | | trunk passed |
| +1 :green_heart: | shadedclient | 19m 54s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 22s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 0m 48s | | the patch passed |
| +1 :green_heart: | compile | 7m 56s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 7m 56s | | the patch passed |
| +1 :green_heart: | compile | 7m 33s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 7m 33s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 58s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6240/4/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| +1 :green_heart: | mvnsite | 1m 18s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 56s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 1s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 22s | | the patch passed |
| +1 :green_heart: | shadedclient | 19m 33s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 15m 47s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 2m 14s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 36s | | The patch does not
generate ASF License warnings. |
| | | 150m 2s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6240/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6240 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux a2ef816d598f 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2
15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 792ea34beb9f3a1e7f306824897291a1f0b6c6b7 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6240/4/testReport/ |
| Max. process+thread count | 3134 (vs. ulimit of 5500) |
[jira] [Commented] (HADOOP-18959) Use builder for prefetch CachingBlockManager
[
https://issues.apache.org/jira/browse/HADOOP-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787787#comment-17787787
]
ASF GitHub Bot commented on HADOOP-18959:
-
virajjasani commented on code in PR #6240:
URL: https://github.com/apache/hadoop/pull/6240#discussion_r1398616555
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/prefetch/BlockManagerParams.java:
##
@@ -0,0 +1,169 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hadoop.fs.impl.prefetch;
+
+import org.apache.hadoop.classification.InterfaceAudience;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.LocalDirAllocator;
+import org.apache.hadoop.fs.statistics.DurationTrackerFactory;
+
+/**
+ * This class is used to provide params to {@link BlockManager}.
+ */
+@InterfaceAudience.Private
+public final class BlockManagerParams {
Review Comment:
done
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/prefetch/BlockManagerParams.java:
##
@@ -0,0 +1,169 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hadoop.fs.impl.prefetch;
+
+import org.apache.hadoop.classification.InterfaceAudience;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.LocalDirAllocator;
+import org.apache.hadoop.fs.statistics.DurationTrackerFactory;
+
+/**
+ * This class is used to provide params to {@link BlockManager}.
+ */
+@InterfaceAudience.Private
+public final class BlockManagerParams {
+
+ /**
+ * Asynchronous tasks are performed in this pool.
+ */
+ private final ExecutorServiceFuturePool futurePool;
Review Comment:
done
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/prefetch/BlockManagerParams.java:
##
@@ -0,0 +1,169 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hadoop.fs.impl.prefetch;
+
+import org.apache.hadoop.classification.InterfaceAudience;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.LocalDirAllocator;
+import org.apache.hadoop.fs.statistics.DurationTrackerFactory;
+
+/**
+ * This class is used to provide params to {@link BlockManager}.
+ */
+@InterfaceAudience.Private
+public final class BlockManagerParams {
+
+ /**
+ * Asynchronous tasks are performed in this pool.
+ */
+ private final ExecutorServiceFuturePool futurePool;
+ /**
+ * Information about each block of the underlying file.
+ */
+ private final BlockData blockData;
+ /**
+ * Size of the in-memory cache in terms of number of
[jira] [Commented] (HADOOP-18959) Use builder for prefetch CachingBlockManager
[
https://issues.apache.org/jira/browse/HADOOP-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17785574#comment-17785574
]
ASF GitHub Bot commented on HADOOP-18959:
-
ahmarsuhail commented on code in PR #6240:
URL: https://github.com/apache/hadoop/pull/6240#discussion_r1391235690
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/prefetch/BlockManagerParams.java:
##
@@ -0,0 +1,169 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hadoop.fs.impl.prefetch;
+
+import org.apache.hadoop.classification.InterfaceAudience;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.LocalDirAllocator;
+import org.apache.hadoop.fs.statistics.DurationTrackerFactory;
+
+/**
+ * This class is used to provide params to {@link BlockManager}.
+ */
+@InterfaceAudience.Private
+public final class BlockManagerParams {
Review Comment:
nit: could you please rename to `BlockManagerParameters` ?
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/prefetch/BlockManagerParams.java:
##
@@ -0,0 +1,169 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hadoop.fs.impl.prefetch;
+
+import org.apache.hadoop.classification.InterfaceAudience;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.LocalDirAllocator;
+import org.apache.hadoop.fs.statistics.DurationTrackerFactory;
+
+/**
+ * This class is used to provide params to {@link BlockManager}.
+ */
+@InterfaceAudience.Private
+public final class BlockManagerParams {
+
+ /**
+ * Asynchronous tasks are performed in this pool.
+ */
+ private final ExecutorServiceFuturePool futurePool;
Review Comment:
nit: could you add a new line after each of these, makes for slightly easier
reading.
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/prefetch/BlockManagerParams.java:
##
@@ -0,0 +1,169 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hadoop.fs.impl.prefetch;
Review Comment:
could you refactor this class, similar to
[S3ClientFactory.S3ClientCreationParameters](https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3ClientFactory.java#L98).
That feels like a cleaner way to get and set these parameters.
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/prefetch/BlockManagerParams.java:
##
@@ -0,0 +1,169 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+
[jira] [Commented] (HADOOP-18959) Use builder for prefetch CachingBlockManager
[
https://issues.apache.org/jira/browse/HADOOP-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781312#comment-17781312
]
ASF GitHub Bot commented on HADOOP-18959:
-
hadoop-yetus commented on PR #6240:
URL: https://github.com/apache/hadoop/pull/6240#issuecomment-1786980461
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 27s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 51s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 23s | | trunk passed |
| +1 :green_heart: | compile | 11m 46s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 10m 23s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 2m 34s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 59s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 35s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 26s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 41s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 14s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 25s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 0m 59s | | the patch passed |
| +1 :green_heart: | compile | 9m 12s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 9m 12s | | the patch passed |
| +1 :green_heart: | compile | 8m 53s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 8m 53s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 2m 19s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 57s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 29s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 25s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 52s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 18s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 16m 26s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 2m 31s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 52s | | The patch does not
generate ASF License warnings. |
| | | 166m 4s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6240/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6240 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 259dd65e823b 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4a6f726fb53b0950017a755dd34777453b5015e1 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6240/3/testReport/ |
| Max. process+thread count | 1268 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws
U: . |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6240/3/console |
|
[jira] [Commented] (HADOOP-18959) Use builder for prefetch CachingBlockManager
[
https://issues.apache.org/jira/browse/HADOOP-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781209#comment-17781209
]
ASF GitHub Bot commented on HADOOP-18959:
-
hadoop-yetus commented on PR #6240:
URL: https://github.com/apache/hadoop/pull/6240#issuecomment-1786427133
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 26s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 18s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 21m 4s | | trunk passed |
| +1 :green_heart: | compile | 9m 42s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 39s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 2m 20s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 55s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 33s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 34s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 51s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 49s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 25s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 0m 56s | | the patch passed |
| +1 :green_heart: | compile | 10m 1s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 10m 1s | | the patch passed |
| +1 :green_heart: | compile | 10m 4s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 10m 4s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 2m 35s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 55s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 28s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 20s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 41s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 4s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 17m 10s |
[/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6240/2/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt)
| hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 2m 37s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 51s | | The patch does not
generate ASF License warnings. |
| | | 170m 49s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.metrics2.source.TestJvmMetrics |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6240/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6240 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux bddf13d0de58 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4a6f726fb53b0950017a755dd34777453b5015e1 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
[jira] [Commented] (HADOOP-18959) Use builder for prefetch CachingBlockManager
[ https://issues.apache.org/jira/browse/HADOOP-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781185#comment-17781185 ] ASF GitHub Bot commented on HADOOP-18959: - virajjasani commented on PR #6240: URL: https://github.com/apache/hadoop/pull/6240#issuecomment-1786296729 @ahmarsuhail could you please review this PR? > Use builder for prefetch CachingBlockManager > > > Key: HADOOP-18959 > URL: https://issues.apache.org/jira/browse/HADOOP-18959 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > > Some of the recent changes (HADOOP-18399, HADOOP-18291, HADOOP-18829 etc) > have added more params for prefetch CachingBlockManager c'tor to process > read/write block requests. They have added too many params and more are > likely to be introduced later. We should use builder pattern to pass params. > This would also help consolidating required prefetch params into one single > place within S3ACachingInputStream, from scattered locations. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18959) Use builder for prefetch CachingBlockManager
[
https://issues.apache.org/jira/browse/HADOOP-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781170#comment-17781170
]
ASF GitHub Bot commented on HADOOP-18959:
-
hadoop-yetus commented on PR #6240:
URL: https://github.com/apache/hadoop/pull/6240#issuecomment-1786247282
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 26s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 53s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 25s | | trunk passed |
| +1 :green_heart: | compile | 9m 36s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 9m 5s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 2m 25s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 56s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 36s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 25s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 38s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 27s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 0m 59s | | the patch passed |
| +1 :green_heart: | compile | 9m 25s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 9m 25s | | the patch passed |
| +1 :green_heart: | compile | 8m 52s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 8m 52s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 2m 18s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6240/1/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 11 new + 0 unchanged - 0 fixed = 11 total (was 0)
|
| +1 :green_heart: | mvnsite | 1m 50s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 27s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 26s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 51s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 23s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 16m 29s | | hadoop-common in the patch
passed. |
| -1 :x: | unit | 1m 2s |
[/patch-unit-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6240/1/artifact/out/patch-unit-hadoop-tools_hadoop-aws.txt)
| hadoop-aws in the patch failed. |
| +1 :green_heart: | asflicense | 0m 53s | | The patch does not
generate ASF License warnings. |
| | | 162m 36s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6240/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6240 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 9fabeaf2e543 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 633cbd41562f9ce00d55584a4592647607eed9bc |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
|
[jira] [Commented] (HADOOP-18959) Use builder for prefetch CachingBlockManager
[ https://issues.apache.org/jira/browse/HADOOP-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781124#comment-17781124 ] ASF GitHub Bot commented on HADOOP-18959: - virajjasani opened a new pull request, #6240: URL: https://github.com/apache/hadoop/pull/6240 Jira: HADOOP-18959 > Use builder for prefetch CachingBlockManager > > > Key: HADOOP-18959 > URL: https://issues.apache.org/jira/browse/HADOOP-18959 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > > Some of the recent changes (HADOOP-18399, HADOOP-18291, HADOOP-18829 etc) > have added more params for prefetch CachingBlockManager c'tor to process > read/write block requests. They have added too many params and more are > likely to be introduced later. We should use builder pattern to pass params. > This would also help consolidating required prefetch params into one single > place within S3ACachingInputStream, from scattered locations. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18959) Use builder for prefetch CachingBlockManager
[ https://issues.apache.org/jira/browse/HADOOP-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781125#comment-17781125 ] ASF GitHub Bot commented on HADOOP-18959: - virajjasani commented on PR #6240: URL: https://github.com/apache/hadoop/pull/6240#issuecomment-1786090728 Tested against `us-west-2` `mvn clean verify -Dparallel-tests -DtestsThreadCount=8 -Dscale -Dprefetch` > Use builder for prefetch CachingBlockManager > > > Key: HADOOP-18959 > URL: https://issues.apache.org/jira/browse/HADOOP-18959 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > > Some of the recent changes (HADOOP-18399, HADOOP-18291, HADOOP-18829 etc) > have added more params for prefetch CachingBlockManager c'tor to process > read/write block requests. They have added too many params and more are > likely to be introduced later. We should use builder pattern to pass params. > This would also help consolidating required prefetch params into one single > place within S3ACachingInputStream, from scattered locations. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
