[jira] [Commented] (TEZ-2849) Implement Specific Workaround for JDK-8026049 & JDK-8073093
[ https://issues.apache.org/jira/browse/TEZ-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14988526#comment-14988526 ] Rajesh Balamohan commented on TEZ-2849: --- [~hitesh] - Yes, .2 patch is good to commit. > Implement Specific Workaround for JDK-8026049 & JDK-8073093 > > > Key: TEZ-2849 > URL: https://issues.apache.org/jira/browse/TEZ-2849 > Project: Apache Tez > Issue Type: Sub-task >Affects Versions: 0.8.1-alpha >Reporter: Gopal V >Assignee: Gopal V > Attachments: TEZ-2849.1.patch, TEZ-2849.2.patch > > > The CPU performance overhead of the sort operation is tied to the JIT > handling of the aliased byte buffers without an explicit alignment (i.e a > byte[] buffer referenced as an IntBuffer). > https://bugs.openjdk.java.net/browse/JDK-8026049 > + > https://bugs.openjdk.java.net/browse/JDK-8073093 > Workaround that JIT limitation while keeping in mind the modern CPU > micro-code optimizations around alignment. > http://lemire.me/blog/archives/2012/05/31/data-alignment-for-speed-myth-or-reality/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2849) Implement Specific Workaround for JDK-8026049 & JDK-8073093
[ https://issues.apache.org/jira/browse/TEZ-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14988517#comment-14988517 ] Hitesh Shah commented on TEZ-2849: -- Is this good to commit? > Implement Specific Workaround for JDK-8026049 & JDK-8073093 > > > Key: TEZ-2849 > URL: https://issues.apache.org/jira/browse/TEZ-2849 > Project: Apache Tez > Issue Type: Sub-task >Affects Versions: 0.8.1-alpha >Reporter: Gopal V >Assignee: Gopal V > Attachments: TEZ-2849.1.patch, TEZ-2849.2.patch > > > The CPU performance overhead of the sort operation is tied to the JIT > handling of the aliased byte buffers without an explicit alignment (i.e a > byte[] buffer referenced as an IntBuffer). > https://bugs.openjdk.java.net/browse/JDK-8026049 > + > https://bugs.openjdk.java.net/browse/JDK-8073093 > Workaround that JIT limitation while keeping in mind the modern CPU > micro-code optimizations around alignment. > http://lemire.me/blog/archives/2012/05/31/data-alignment-for-speed-myth-or-reality/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2849) Implement Specific Workaround for JDK-8026049 & JDK-8073093
[ https://issues.apache.org/jira/browse/TEZ-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14976975#comment-14976975 ] Rajesh Balamohan commented on TEZ-2849: --- lgtm. +1. Is the explicit cast on SpanIterator intensional (i.e within SortSpan)? > Implement Specific Workaround for JDK-8026049 & JDK-8073093 > > > Key: TEZ-2849 > URL: https://issues.apache.org/jira/browse/TEZ-2849 > Project: Apache Tez > Issue Type: Sub-task >Affects Versions: 0.8.1-alpha >Reporter: Gopal V >Assignee: Gopal V > Attachments: TEZ-2849.1.patch, TEZ-2849.2.patch > > > The CPU performance overhead of the sort operation is tied to the JIT > handling of the aliased byte buffers without an explicit alignment (i.e a > byte[] buffer referenced as an IntBuffer). > https://bugs.openjdk.java.net/browse/JDK-8026049 > + > https://bugs.openjdk.java.net/browse/JDK-8073093 > Workaround that JIT limitation while keeping in mind the modern CPU > micro-code optimizations around alignment. > http://lemire.me/blog/archives/2012/05/31/data-alignment-for-speed-myth-or-reality/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2849) Implement Specific Workaround for JDK-8026049 & JDK-8073093
[ https://issues.apache.org/jira/browse/TEZ-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14960786#comment-14960786 ] Karam Singh commented on TEZ-2849: -- Again terasort against GB data with OracleJDK8 (After restarting cluster for OracleJDK8 and setting the same in hadoop-env.sh): java version "1.8.0_45" Also ran terasort against OracleJDK7 (After restarting cluster for OracleJDK7 and setting the same in hadoop-env.sh): java version "1.7.0_67" Java(TM) SE Runtime Environment (build 1.7.0_67-b01) Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode) AND against OpenJDK7 (After restarting cluster for OpenJDK8 and setting the same in hadoop-env.sh): java version "1.7.0_85" OpenJDK Runtime Environment (rhel-2.6.1.2.el7_1-x86_64 u85-b01) OpenJDK 64-Bit Server VM (build 24.85-b03, mixed mode) All terasort application ran sucessfully . I ran all application after applying patch on master bracnh > Implement Specific Workaround for JDK-8026049 & JDK-8073093 > > > Key: TEZ-2849 > URL: https://issues.apache.org/jira/browse/TEZ-2849 > Project: Apache Tez > Issue Type: Sub-task >Affects Versions: 0.8.1-alpha >Reporter: Gopal V >Assignee: Gopal V > Attachments: TEZ-2849.1.patch, TEZ-2849.2.patch > > > The CPU performance overhead of the sort operation is tied to the JIT > handling of the aliased byte buffers without an explicit alignment (i.e a > byte[] buffer referenced as an IntBuffer). > https://bugs.openjdk.java.net/browse/JDK-8026049 > + > https://bugs.openjdk.java.net/browse/JDK-8073093 > Workaround that JIT limitation while keeping in mind the modern CPU > micro-code optimizations around alignment. > http://lemire.me/blog/archives/2012/05/31/data-alignment-for-speed-myth-or-reality/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2849) Implement Specific Workaround for JDK-8026049 & JDK-8073093
[ https://issues.apache.org/jira/browse/TEZ-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14960211#comment-14960211 ] Karam Singh commented on TEZ-2849: -- Ran terasort on small scale cluster against OracleJDK8 and OpenJDK8, Did not faced any problems with that. OracleJDK8 version (cluster was by defualt running with version): Used Teragen to ran genetar 2G of data The ran terasort on it /usr/jdk64/jdk1.8.0_45/bin/java -version java version "1.8.0_45" Java(TM) SE Runtime Environment (build 1.8.0_45-b14) Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode) THEN ran default JAVA_HOME and JDK_HOME in hadoop-env.sh to OpenJDK8: /usr/lib/jvm/java-1.8.0-openjdk ran reran terasort again. Also user tergen to generate 6G of data again ran terasort on it successfully. OpenJDK8 version is: openjdk version "1.8.0_60" OpenJDK Runtime Environment (build 1.8.0_60-b27) OpenJDK 64-Bit Server VM (build 25.60-b23, mixed mode) While changing to OpenJDK8: I restarted all Hadoop daemons.sh to OpenJDK8 and verified containers ran with OpenJDK8 version by checking launch-container.sh of contaiers Will check agains JDK7 version today. > Implement Specific Workaround for JDK-8026049 & JDK-8073093 > > > Key: TEZ-2849 > URL: https://issues.apache.org/jira/browse/TEZ-2849 > Project: Apache Tez > Issue Type: Sub-task >Affects Versions: 0.8.1-alpha >Reporter: Gopal V >Assignee: Gopal V > Attachments: TEZ-2849.1.patch, TEZ-2849.2.patch > > > The CPU performance overhead of the sort operation is tied to the JIT > handling of the aliased byte buffers without an explicit alignment (i.e a > byte[] buffer referenced as an IntBuffer). > https://bugs.openjdk.java.net/browse/JDK-8026049 > + > https://bugs.openjdk.java.net/browse/JDK-8073093 > Workaround that JIT limitation while keeping in mind the modern CPU > micro-code optimizations around alignment. > http://lemire.me/blog/archives/2012/05/31/data-alignment-for-speed-myth-or-reality/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2849) Implement Specific Workaround for JDK-8026049 & JDK-8073093
[ https://issues.apache.org/jira/browse/TEZ-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14960176#comment-14960176 ] Gopal V commented on TEZ-2849: -- [~karams]: Can you test this patch with terasort on JDK7 and JDK8? [~ozawa]: This patch will definitely break if you've got any changes from porting the sort buffer to direct byte buffers (or on 0.5.x branch). > Implement Specific Workaround for JDK-8026049 & JDK-8073093 > > > Key: TEZ-2849 > URL: https://issues.apache.org/jira/browse/TEZ-2849 > Project: Apache Tez > Issue Type: Sub-task >Affects Versions: 0.8.1-alpha >Reporter: Gopal V >Assignee: Gopal V > Attachments: TEZ-2849.1.patch, TEZ-2849.2.patch > > > The CPU performance overhead of the sort operation is tied to the JIT > handling of the aliased byte buffers without an explicit alignment (i.e a > byte[] buffer referenced as an IntBuffer). > https://bugs.openjdk.java.net/browse/JDK-8026049 > + > https://bugs.openjdk.java.net/browse/JDK-8073093 > Workaround that JIT limitation while keeping in mind the modern CPU > micro-code optimizations around alignment. > http://lemire.me/blog/archives/2012/05/31/data-alignment-for-speed-myth-or-reality/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2849) Implement Specific Workaround for JDK-8026049 & JDK-8073093
[
https://issues.apache.org/jira/browse/TEZ-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14946126#comment-14946126
]
TezQA commented on TEZ-2849:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12765292/TEZ-2849.2.patch
against master revision dd0293f.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1192//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1192//console
This message is automatically generated.
> Implement Specific Workaround for JDK-8026049 & JDK-8073093
>
>
> Key: TEZ-2849
> URL: https://issues.apache.org/jira/browse/TEZ-2849
> Project: Apache Tez
> Issue Type: Sub-task
>Affects Versions: 0.8.1
>Reporter: Gopal V
>Assignee: Gopal V
> Attachments: TEZ-2849.1.patch, TEZ-2849.2.patch
>
>
> The CPU performance overhead of the sort operation is tied to the JIT
> handling of the aliased byte buffers without an explicit alignment (i.e a
> byte[] buffer referenced as an IntBuffer).
> https://bugs.openjdk.java.net/browse/JDK-8026049
> +
> https://bugs.openjdk.java.net/browse/JDK-8073093
> Workaround that JIT limitation while keeping in mind the modern CPU
> micro-code optimizations around alignment.
> http://lemire.me/blog/archives/2012/05/31/data-alignment-for-speed-myth-or-reality/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2849) Implement Specific Workaround for JDK-8026049 & JDK-8073093
[ https://issues.apache.org/jira/browse/TEZ-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14946109#comment-14946109 ] Gopal V commented on TEZ-2849: -- [~ozawa]: the patch works for me on JDK8 - can you tell me what's the actual ByteBuffer class generated for the slice() operation? In all my testing, the ByteBuffer.allocate() is array-backed & slices support .array(). > Implement Specific Workaround for JDK-8026049 & JDK-8073093 > > > Key: TEZ-2849 > URL: https://issues.apache.org/jira/browse/TEZ-2849 > Project: Apache Tez > Issue Type: Sub-task >Affects Versions: 0.8.1 >Reporter: Gopal V >Assignee: Gopal V > Attachments: TEZ-2849.1.patch, TEZ-2849.2.patch > > > The CPU performance overhead of the sort operation is tied to the JIT > handling of the aliased byte buffers without an explicit alignment (i.e a > byte[] buffer referenced as an IntBuffer). > https://bugs.openjdk.java.net/browse/JDK-8026049 > + > https://bugs.openjdk.java.net/browse/JDK-8073093 > Workaround that JIT limitation while keeping in mind the modern CPU > micro-code optimizations around alignment. > http://lemire.me/blog/archives/2012/05/31/data-alignment-for-speed-myth-or-reality/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2849) Implement Specific Workaround for JDK-8026049 & JDK-8073093
[ https://issues.apache.org/jira/browse/TEZ-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14940866#comment-14940866 ] Tsuyoshi Ozawa commented on TEZ-2849: - [~gopalv] thanks for your patch! I'll take a benchmark with your patch in this weekend. Please wait a moment. > Implement Specific Workaround for JDK-8026049 & JDK-8073093 > > > Key: TEZ-2849 > URL: https://issues.apache.org/jira/browse/TEZ-2849 > Project: Apache Tez > Issue Type: Sub-task >Affects Versions: 0.8.1 >Reporter: Gopal V >Assignee: Gopal V > Attachments: TEZ-2849.1.patch > > > The CPU performance overhead of the sort operation is tied to the JIT > handling of the aliased byte buffers without an explicit alignment (i.e a > byte[] buffer referenced as an IntBuffer). > https://bugs.openjdk.java.net/browse/JDK-8026049 > + > https://bugs.openjdk.java.net/browse/JDK-8073093 > Workaround that JIT limitation while keeping in mind the modern CPU > micro-code optimizations around alignment. > http://lemire.me/blog/archives/2012/05/31/data-alignment-for-speed-myth-or-reality/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2849) Implement Specific Workaround for JDK-8026049 & JDK-8073093
[
https://issues.apache.org/jira/browse/TEZ-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14907239#comment-14907239
]
TezQA commented on TEZ-2849:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12762229/TEZ-2849.1.patch
against master revision 4edba44.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.test.TestFaultTolerance
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1170//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1170//console
This message is automatically generated.
> Implement Specific Workaround for JDK-8026049 & JDK-8073093
>
>
> Key: TEZ-2849
> URL: https://issues.apache.org/jira/browse/TEZ-2849
> Project: Apache Tez
> Issue Type: Sub-task
>Affects Versions: 0.8.1
>Reporter: Gopal V
>Assignee: Gopal V
> Attachments: TEZ-2849.1.patch
>
>
> The CPU performance overhead of the sort operation is tied to the JIT
> handling of the aliased byte buffers without an explicit alignment (i.e a
> byte[] buffer referenced as an IntBuffer).
> https://bugs.openjdk.java.net/browse/JDK-8026049
> +
> https://bugs.openjdk.java.net/browse/JDK-8073093
> Workaround that JIT limitation while keeping in mind the modern CPU
> micro-code optimizations around alignment.
> http://lemire.me/blog/archives/2012/05/31/data-alignment-for-speed-myth-or-reality/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
