[jira] [Commented] (MAPREDUCE-7332) Fix SpillCallBackPathsFinder to use JDK7 on branch-2.10
[ https://issues.apache.org/jira/browse/MAPREDUCE-7332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309454#comment-17309454 ] Jim Brennan commented on MAPREDUCE-7332: Given that this is an isolated change that affects only one unit test, I am ok with using the patch in this instance. We really need to get the precommit build working for branch-2.10 though. > Fix SpillCallBackPathsFinder to use JDK7 on branch-2.10 > --- > > Key: MAPREDUCE-7332 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7332 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: job submission, security >Reporter: Ahmed Hussein >Assignee: Ahmed Hussein >Priority: Minor > Attachments: MAPREDUCE-7332.branch-2.10.001.patch, > MAPREDUCE-7332.branch-2.10.002.patch > > > I mistakenly uploaded a patch for branch-2.10 that uses JDK8. > Yetus did not fail though. It should be investigated why it was not failing > if JDK8+ is used in the code. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7332) Fix SpillCallBackPathsFinder to use JDK7 on branch-2.10
[ https://issues.apache.org/jira/browse/MAPREDUCE-7332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309172#comment-17309172 ] Akira Ajisaka commented on MAPREDUCE-7332: -- Hi [~Jim_Brennan], would you use GitHub Pull Request instead of JIRA for branch-2.10? We switched from Findbugs to SpotBugs in HADOOP-16870 and updated the config of https://ci-hadoop.apache.org/job/PreCommit-MAPREDUCE-Build to use SpotBugs. However, now HADOOP-16870 is not backported to branch-2.10. That's why the job fails. In GitHub PR, Jenkinsfile is used to configure the multibranch job (https://ci-hadoop.apache.org/job/hadoop-multibranch), and the file in branch-2.10 is still set to use FindBugs. > Fix SpillCallBackPathsFinder to use JDK7 on branch-2.10 > --- > > Key: MAPREDUCE-7332 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7332 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: job submission, security >Reporter: Ahmed Hussein >Assignee: Ahmed Hussein >Priority: Minor > Attachments: MAPREDUCE-7332.branch-2.10.001.patch, > MAPREDUCE-7332.branch-2.10.002.patch > > > I mistakenly uploaded a patch for branch-2.10 that uses JDK8. > Yetus did not fail though. It should be investigated why it was not failing > if JDK8+ is used in the code. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7332) Fix SpillCallBackPathsFinder to use JDK7 on branch-2.10
[
https://issues.apache.org/jira/browse/MAPREDUCE-7332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309098#comment-17309098
]
Hadoop QA commented on MAPREDUCE-7332:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 11m
27s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} yetus {color} | {color:red} 0m 15s{color}
| {color:red}{color} | {color:red} Unprocessed flag(s):
--spotbugs-strict-precheck {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/PreCommit-MAPREDUCE-Build/73/artifact/out/Dockerfile
|
| JIRA Issue | MAPREDUCE-7332 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/13022972/MAPREDUCE-7332.branch-2.10.002.patch
|
| Console output |
https://ci-hadoop.apache.org/job/PreCommit-MAPREDUCE-Build/73/console |
| versions | git=2.7.4 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
> Fix SpillCallBackPathsFinder to use JDK7 on branch-2.10
> ---
>
> Key: MAPREDUCE-7332
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7332
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission, security
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Minor
> Attachments: MAPREDUCE-7332.branch-2.10.001.patch,
> MAPREDUCE-7332.branch-2.10.002.patch
>
>
> I mistakenly uploaded a patch for branch-2.10 that uses JDK8.
> Yetus did not fail though. It should be investigated why it was not failing
> if JDK8+ is used in the code.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7332) Fix SpillCallBackPathsFinder to use JDK7 on branch-2.10
[ https://issues.apache.org/jira/browse/MAPREDUCE-7332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309079#comment-17309079 ] Ahmed Hussein commented on MAPREDUCE-7332: -- findbugs is not supported. We need to pull HADOOP-16870 into branch-2.10. https://issues.apache.org/jira/browse/HADOOP-16870?focusedCommentId=17309077=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17309077 > Fix SpillCallBackPathsFinder to use JDK7 on branch-2.10 > --- > > Key: MAPREDUCE-7332 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7332 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: job submission, security >Reporter: Ahmed Hussein >Assignee: Ahmed Hussein >Priority: Minor > Attachments: MAPREDUCE-7332.branch-2.10.001.patch > > > I mistakenly uploaded a patch for branch-2.10 that uses JDK8. > Yetus did not fail though. It should be investigated why it was not failing > if JDK8+ is used in the code. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7332) Fix SpillCallBackPathsFinder to use JDK7 on branch-2.10
[
https://issues.apache.org/jira/browse/MAPREDUCE-7332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17308974#comment-17308974
]
Jim Brennan commented on MAPREDUCE-7332:
[~aajisaka] do you know why this failure happens?
{noformat}
Re-exec mode detected. Continuing.
ERROR:
Unprocessed flag(s): --spotbugs-strict-precheck {noformat}
> Fix SpillCallBackPathsFinder to use JDK7 on branch-2.10
> ---
>
> Key: MAPREDUCE-7332
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7332
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission, security
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Minor
> Attachments: MAPREDUCE-7332.branch-2.10.001.patch
>
>
> I mistakenly uploaded a patch for branch-2.10 that uses JDK8.
> Yetus did not fail though. It should be investigated why it was not failing
> if JDK8+ is used in the code.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7332) Fix SpillCallBackPathsFinder to use JDK7 on branch-2.10
[ https://issues.apache.org/jira/browse/MAPREDUCE-7332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17308954#comment-17308954 ] Ahmed Hussein commented on MAPREDUCE-7332: -- [~Jim_Brennan] Can you please take a look at the fix? > Fix SpillCallBackPathsFinder to use JDK7 on branch-2.10 > --- > > Key: MAPREDUCE-7332 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7332 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: job submission, security >Reporter: Ahmed Hussein >Assignee: Ahmed Hussein >Priority: Minor > Attachments: MAPREDUCE-7332.branch-2.10.001.patch > > > I mistakenly uploaded a patch for branch-2.10 that uses JDK8. > Yetus did not fail though. It should be investigated why it was not failing > if JDK8+ is used in the code. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7332) Fix SpillCallBackPathsFinder to use JDK7 on branch-2.10
[
https://issues.apache.org/jira/browse/MAPREDUCE-7332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17308883#comment-17308883
]
Hadoop QA commented on MAPREDUCE-7332:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 11m
29s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} yetus {color} | {color:red} 0m 14s{color}
| {color:red}{color} | {color:red} Unprocessed flag(s):
--spotbugs-strict-precheck {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/PreCommit-MAPREDUCE-Build/72/artifact/out/Dockerfile
|
| JIRA Issue | MAPREDUCE-7332 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/13022961/MAPREDUCE-7332.branch-2.10.001.patch
|
| Console output |
https://ci-hadoop.apache.org/job/PreCommit-MAPREDUCE-Build/72/console |
| versions | git=2.7.4 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
> Fix SpillCallBackPathsFinder to use JDK7 on branch-2.10
> ---
>
> Key: MAPREDUCE-7332
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7332
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission, security
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Minor
> Attachments: MAPREDUCE-7332.branch-2.10.001.patch
>
>
> I mistakenly uploaded a patch for branch-2.10 that uses JDK8.
> Yetus did not fail though. It should be investigated why it was not failing
> if JDK8+ is used in the code.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7332) Fix SpillCallBackPathsFinder to use JDK7 on branch-2.10
[
https://issues.apache.org/jira/browse/MAPREDUCE-7332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17308876#comment-17308876
]
Ahmed Hussein commented on MAPREDUCE-7332:
--
To be on the safe side, I preferred to use the guava library which should be
compatible for both JDK7 and JDK8.
To achieve the same level of concurrency, I used {{LoadingCache}} which
guarantees atomicity of the operations.
I experimented {{guava.MapMaker().makeMap()}} but the unit test
{{TestMRIntermediateDataEncryption}} --- with multiple reducers -- failed.
The failure was due to the fact that the {{ConcurrentMap}} returned from
{{guava.MapMaker().makeMap()}} did not guarantee atomicity.
I tested the patch locally and reviewed the checkstyle.
{code:bash}
dev-support/bin/test-patch \
--plugins="maven,checkstyle" \
--test-parallel=true \
--branch=branch-2.10 \
patches/MAPREDUCE-7332.branch-2.10.001.patch
{code}
> Fix SpillCallBackPathsFinder to use JDK7 on branch-2.10
> ---
>
> Key: MAPREDUCE-7332
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7332
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission, security
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Minor
> Attachments: MAPREDUCE-7332.branch-2.10.001.patch
>
>
> I mistakenly uploaded a patch for branch-2.10 that uses JDK8.
> Yetus did not fail though. It should be investigated why it was not failing
> if JDK8+ is used in the code.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7332) Fix SpillCallBackPathsFinder to use JDK7 on branch-2.10
[
https://issues.apache.org/jira/browse/MAPREDUCE-7332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17308092#comment-17308092
]
Ahmed Hussein commented on MAPREDUCE-7332:
--
{code:bash}
[INFO] Checking unresolved references to org.codehaus.mojo.signature:java17:1.0
[ERROR]
/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/SpillCallBackPathsFinder.java:184:
Undefined reference: java.util.concurrent.ConcurrentHashMap.KeySetView
[ERROR]
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/SpillCallBackPathsFinder.java:184:
Undefined reference: java.util.concurrent.ConcurrentHashMap.KeySetView
java.util.concurrent.ConcurrentHashMap.keySet()
{code}
The root cause of the problem is that {{ConcurrentHashMap.keySet()}} changed
between JDK7 and JDK8 .
branch-2.10 fails if it is compiled without {{-Danimal.sniffer.skip}}
> Fix SpillCallBackPathsFinder to use JDK7 on branch-2.10
> ---
>
> Key: MAPREDUCE-7332
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7332
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission, security
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
>
> I mistakenly uploaded a patch for branch-2.10 that uses JDK8.
> Yetus did not fail though. It should be investigated why it was not failing
> if JDK8+ is used in the code.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
