[jira] [Commented] (TEZ-2779) When ResourceManager is HA mode, How can I set up Tez UI configs?
[ https://issues.apache.org/jira/browse/TEZ-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15570415#comment-15570415 ] Sreenath Somarajapuram commented on TEZ-2779: - Thanks, got it. > When ResourceManager is HA mode, How can I set up Tez UI configs? > - > > Key: TEZ-2779 > URL: https://issues.apache.org/jira/browse/TEZ-2779 > Project: Apache Tez > Issue Type: Improvement > Components: UI >Affects Versions: 0.7.0 >Reporter: skrho >Assignee: Sreenath Somarajapuram > > I am using Tez UI 0.7.0 & hadoop 2.7.1 and It is a quite good. > I have a question. > When the resourcemanager is HA mode, How can I set up configs.js? > expecially RMWebUrl value > Now I am just setting RMWebUrl value to 'http://resourcemangerUrl1:8088' > But When the resoucemanager ACTIVE state was changed from resourcemangerUrl1 > to resourcemangerUrl2, the resourcemanger link is not working in Tez UI > because of not service resourcemangerUrl1 > Please tell me how to set configs.js ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-3465) Support broadcast edge into cartesian product vertex and forbid other edges
[ https://issues.apache.org/jira/browse/TEZ-3465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15570294#comment-15570294 ] Zhiyuan Yang commented on TEZ-3465: --- [~hitesh], [~mingma], please help review when you get time. Thanks! > Support broadcast edge into cartesian product vertex and forbid other edges > --- > > Key: TEZ-3465 > URL: https://issues.apache.org/jira/browse/TEZ-3465 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Zhiyuan Yang >Assignee: Zhiyuan Yang > Attachments: TEZ-3465.1.patch, TEZ-3465.2.patch > > > Cartesian product vertex manager should support other incoming edge type. > Currently only broadcast edge is necessary, although potentially more edge > types could also be. Custom edge need its own vertex manager which can't work > with Cartesian product VM, so it has to be forbade. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-3271) Provide mapreduce failures.maxpercent equivalent
[
https://issues.apache.org/jira/browse/TEZ-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15570232#comment-15570232
]
TezQA commented on TEZ-3271:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12832987/TEZ-3271.7.patch
against master revision c9b09cb.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.test.TestFaultTolerance
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/2036//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/2036//console
This message is automatically generated.
> Provide mapreduce failures.maxpercent equivalent
>
>
> Key: TEZ-3271
> URL: https://issues.apache.org/jira/browse/TEZ-3271
> Project: Apache Tez
> Issue Type: New Feature
>Reporter: Jonathan Eagles
>Assignee: Jonathan Eagles
> Attachments: Succeeded with Failures.png, TEZ-3271.1.patch,
> TEZ-3271.2.patch, TEZ-3271.3.patch, TEZ-3271.4.patch, TEZ-3271.5.patch,
> TEZ-3271.6.patch, TEZ-3271.7.patch
>
>
> There is a certain category of work that need not have 100% of tasks succeed
> to cause the work to be considered a success. To meet that end, I propose we
> provide a tez equivalent of mapreduce.map.failures.maxpercent and
> mapreduce.reduce.failures.maxpercent. In this way a vertex will be considered
> a success if the number of failures is below a configured threshold.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Failed: TEZ-3271 PreCommit Build #2036
Jira: https://issues.apache.org/jira/browse/TEZ-3271
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/2036/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 4620 lines...]
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] [Help 2]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :tez-tests
[INFO] Build failures were ignored.
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12832987/TEZ-3271.7.patch
against master revision c9b09cb.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.test.TestFaultTolerance
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/2036//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/2036//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
9ec053cdacf121e57e3257960084b8c7eba34c90 logged out
==
==
Finished build.
==
==
Build step 'Execute shell' marked build as failure
Archiving artifacts
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
###
## FAILED TESTS (if any)
##
1 tests failed.
FAILED: org.apache.tez.test.TestFaultTolerance.testBasicInputFailureWithoutExit
Error Message:
expected: but was:
Stack Trace:
java.lang.AssertionError: expected: but was:
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:144)
at
org.apache.tez.test.TestFaultTolerance.runDAGAndVerify(TestFaultTolerance.java:151)
at
org.apache.tez.test.TestFaultTolerance.runDAGAndVerify(TestFaultTolerance.java:134)
at
org.apache.tez.test.TestFaultTolerance.runDAGAndVerify(TestFaultTolerance.java:130)
at
org.apache.tez.test.TestFaultTolerance.testBasicInputFailureWithoutExit(TestFaultTolerance.java:293)
[jira] [Commented] (TEZ-3466) Tez classpath building to mimic mapreduce classpath building
[
https://issues.apache.org/jira/browse/TEZ-3466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15570143#comment-15570143
]
TezQA commented on TEZ-3466:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12832982/TEZ-3466.2.patch
against master revision c9b09cb.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/2035//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/2035//console
This message is automatically generated.
> Tez classpath building to mimic mapreduce classpath building
>
>
> Key: TEZ-3466
> URL: https://issues.apache.org/jira/browse/TEZ-3466
> Project: Apache Tez
> Issue Type: New Feature
>Reporter: Jonathan Eagles
>Assignee: Jason Lowe
> Attachments: TEZ-3466.1.patch, TEZ-3466.2.patch
>
>
> Tez builds the classpath very differently than MapReduce. For example, it
> unconditionally puts PWD/* in front of the framework classpath, whereas
> MapReduce will only do that if mapreduce.job.user.classpath.first=true.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Success: TEZ-3466 PreCommit Build #2035
Jira: https://issues.apache.org/jira/browse/TEZ-3466
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/2035/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 4811 lines...]
[INFO] Tez SUCCESS [ 0.033 s]
[INFO]
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 57:47 min
[INFO] Finished at: 2016-10-12T22:59:15+00:00
[INFO] Final Memory: 81M/1287M
[INFO]
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12832982/TEZ-3466.2.patch
against master revision c9b09cb.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/2035//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/2035//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
009358652ba1326f07c821273c5cb2ad747bd68b logged out
==
==
Finished build.
==
==
Archiving artifacts
[description-setter] Description set: TEZ-3466
Recording test results
Email was triggered for: Success
Sending email for trigger: Success
###
## FAILED TESTS (if any)
##
All tests passed
[jira] [Commented] (TEZ-3466) Tez classpath building to mimic mapreduce classpath building
[ https://issues.apache.org/jira/browse/TEZ-3466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15570072#comment-15570072 ] Hitesh Shah commented on TEZ-3466: -- New config TEZ_USER_CLASSPATH_FIRST needs a ConfigurationProperty annotation. +1 after that is addressed. > Tez classpath building to mimic mapreduce classpath building > > > Key: TEZ-3466 > URL: https://issues.apache.org/jira/browse/TEZ-3466 > Project: Apache Tez > Issue Type: New Feature >Reporter: Jonathan Eagles >Assignee: Jason Lowe > Attachments: TEZ-3466.1.patch, TEZ-3466.2.patch > > > Tez builds the classpath very differently than MapReduce. For example, it > unconditionally puts PWD/* in front of the framework classpath, whereas > MapReduce will only do that if mapreduce.job.user.classpath.first=true. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-3271) Provide mapreduce failures.maxpercent equivalent
[ https://issues.apache.org/jira/browse/TEZ-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3271: - Attachment: TEZ-3271.7.patch Attaching a new patch that moves the generation of DME events to the edge manager plugins. > Provide mapreduce failures.maxpercent equivalent > > > Key: TEZ-3271 > URL: https://issues.apache.org/jira/browse/TEZ-3271 > Project: Apache Tez > Issue Type: New Feature >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > Attachments: Succeeded with Failures.png, TEZ-3271.1.patch, > TEZ-3271.2.patch, TEZ-3271.3.patch, TEZ-3271.4.patch, TEZ-3271.5.patch, > TEZ-3271.6.patch, TEZ-3271.7.patch > > > There is a certain category of work that need not have 100% of tasks succeed > to cause the work to be considered a success. To meet that end, I propose we > provide a tez equivalent of mapreduce.map.failures.maxpercent and > mapreduce.reduce.failures.maxpercent. In this way a vertex will be considered > a success if the number of failures is below a configured threshold. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-3271) Provide mapreduce failures.maxpercent equivalent
[
https://issues.apache.org/jira/browse/TEZ-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569988#comment-15569988
]
TezQA commented on TEZ-3271:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12832971/TEZ-3271.6.patch
against master revision c9b09cb.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/2034//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-TEZ-Build/2034//artifact/patchprocess/newPatchFindbugsWarningstez-dag.html
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/2034//console
This message is automatically generated.
> Provide mapreduce failures.maxpercent equivalent
>
>
> Key: TEZ-3271
> URL: https://issues.apache.org/jira/browse/TEZ-3271
> Project: Apache Tez
> Issue Type: New Feature
>Reporter: Jonathan Eagles
>Assignee: Jonathan Eagles
> Attachments: Succeeded with Failures.png, TEZ-3271.1.patch,
> TEZ-3271.2.patch, TEZ-3271.3.patch, TEZ-3271.4.patch, TEZ-3271.5.patch,
> TEZ-3271.6.patch
>
>
> There is a certain category of work that need not have 100% of tasks succeed
> to cause the work to be considered a success. To meet that end, I propose we
> provide a tez equivalent of mapreduce.map.failures.maxpercent and
> mapreduce.reduce.failures.maxpercent. In this way a vertex will be considered
> a success if the number of failures is below a configured threshold.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Failed: TEZ-3271 PreCommit Build #2034
Jira: https://issues.apache.org/jira/browse/TEZ-3271
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/2034/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 4824 lines...]
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 58:31 min
[INFO] Finished at: 2016-10-12T21:59:09+00:00
[INFO] Final Memory: 80M/1488M
[INFO]
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12832971/TEZ-3271.6.patch
against master revision c9b09cb.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/2034//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-TEZ-Build/2034//artifact/patchprocess/newPatchFindbugsWarningstez-dag.html
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/2034//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
a30d2a0abff833483d0ff2da0dff973646ed46e5 logged out
==
==
Finished build.
==
==
Build step 'Execute shell' marked build as failure
Archiving artifacts
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
###
## FAILED TESTS (if any)
##
All tests passed
[jira] [Comment Edited] (TEZ-3106) Tez UI 2 - Features Umbrella
[ https://issues.apache.org/jira/browse/TEZ-3106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569969#comment-15569969 ] Hitesh Shah edited comment on TEZ-3106 at 10/12/16 9:55 PM: Tez UI 2 is done. It might be better to make all pending sub tasks as top-level jiras as improvements? was (Author: hitesh): Tez UI 2 is done. It might be better to make all pending jiras as top-level jiras as improvements? > Tez UI 2 - Features Umbrella > > > Key: TEZ-3106 > URL: https://issues.apache.org/jira/browse/TEZ-3106 > Project: Apache Tez > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > > A place to collate all the upcoming changes in Tez UI 2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-3106) Tez UI 2 - Features Umbrella
[ https://issues.apache.org/jira/browse/TEZ-3106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569969#comment-15569969 ] Hitesh Shah commented on TEZ-3106: -- Tez UI 2 is done. It might be better to make all pending jiras as top-level jiras as improvements? > Tez UI 2 - Features Umbrella > > > Key: TEZ-3106 > URL: https://issues.apache.org/jira/browse/TEZ-3106 > Project: Apache Tez > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > > A place to collate all the upcoming changes in Tez UI 2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-3466) Tez classpath building to mimic mapreduce classpath building
[ https://issues.apache.org/jira/browse/TEZ-3466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated TEZ-3466: Attachment: TEZ-3466.2.patch Updated the mappings in the deprecated keys and also updated the addtional classpath config doc to note the interaction with the user classpath first config. > Tez classpath building to mimic mapreduce classpath building > > > Key: TEZ-3466 > URL: https://issues.apache.org/jira/browse/TEZ-3466 > Project: Apache Tez > Issue Type: New Feature >Reporter: Jonathan Eagles >Assignee: Jason Lowe > Attachments: TEZ-3466.1.patch, TEZ-3466.2.patch > > > Tez builds the classpath very differently than MapReduce. For example, it > unconditionally puts PWD/* in front of the framework classpath, whereas > MapReduce will only do that if mapreduce.job.user.classpath.first=true. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-3466) Tez classpath building to mimic mapreduce classpath building
[ https://issues.apache.org/jira/browse/TEZ-3466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569794#comment-15569794 ] Hitesh Shah commented on TEZ-3466: -- Mostly looks good [~jeagles]. Minor nit though: TEZ_CLUSTER_ADDITIONAL_CLASSPATH_PREFIX - indicates that this will be at the front of the classpath but when user classpath first is set to false, it ends up at the back. Might be good to update the docs to indicate that this can be end up at the back of the classpath when the other config property is set to false. > Tez classpath building to mimic mapreduce classpath building > > > Key: TEZ-3466 > URL: https://issues.apache.org/jira/browse/TEZ-3466 > Project: Apache Tez > Issue Type: New Feature >Reporter: Jonathan Eagles >Assignee: Jason Lowe > Attachments: TEZ-3466.1.patch > > > Tez builds the classpath very differently than MapReduce. For example, it > unconditionally puts PWD/* in front of the framework classpath, whereas > MapReduce will only do that if mapreduce.job.user.classpath.first=true. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-3271) Provide mapreduce failures.maxpercent equivalent
[ https://issues.apache.org/jira/browse/TEZ-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3271: - Attachment: TEZ-3271.6.patch > Provide mapreduce failures.maxpercent equivalent > > > Key: TEZ-3271 > URL: https://issues.apache.org/jira/browse/TEZ-3271 > Project: Apache Tez > Issue Type: New Feature >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > Attachments: Succeeded with Failures.png, TEZ-3271.1.patch, > TEZ-3271.2.patch, TEZ-3271.3.patch, TEZ-3271.4.patch, TEZ-3271.5.patch, > TEZ-3271.6.patch > > > There is a certain category of work that need not have 100% of tasks succeed > to cause the work to be considered a success. To meet that end, I propose we > provide a tez equivalent of mapreduce.map.failures.maxpercent and > mapreduce.reduce.failures.maxpercent. In this way a vertex will be considered > a success if the number of failures is below a configured threshold. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (TEZ-2779) When ResourceManager is HA mode, How can I set up Tez UI configs?
[ https://issues.apache.org/jira/browse/TEZ-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569575#comment-15569575 ] Hitesh Shah edited comment on TEZ-2779 at 10/12/16 6:57 PM: [~Sreenath] Does the UI support a config which takes in a list of RM hostnames? If not, let us consider this scenario: - Tez UI config has rm1 as the hostname for the RM. - YARN has RM HA enabled with rm1, rm2 and rm3 as the hosts. - rm1 is completely down but rm2 and rm3 are up. At this point, I am assuming that the UI will not function properly even though the YARN RM layer is actually up. was (Author: hitesh): [~Sreenath] Does the UI support a config which takes in a list of RM hostnames? If not, let us consider this scenario: - Tez UI config has rm1 as the hostname for the RM. - YARN has RM HA enabled with rm1, rm2 and rm3 as the hosts. - rm1 is completely down but rm2 and rm3 are up. > When ResourceManager is HA mode, How can I set up Tez UI configs? > - > > Key: TEZ-2779 > URL: https://issues.apache.org/jira/browse/TEZ-2779 > Project: Apache Tez > Issue Type: Improvement > Components: UI >Affects Versions: 0.7.0 >Reporter: skrho >Assignee: Sreenath Somarajapuram > > I am using Tez UI 0.7.0 & hadoop 2.7.1 and It is a quite good. > I have a question. > When the resourcemanager is HA mode, How can I set up configs.js? > expecially RMWebUrl value > Now I am just setting RMWebUrl value to 'http://resourcemangerUrl1:8088' > But When the resoucemanager ACTIVE state was changed from resourcemangerUrl1 > to resourcemangerUrl2, the resourcemanger link is not working in Tez UI > because of not service resourcemangerUrl1 > Please tell me how to set configs.js ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2779) When ResourceManager is HA mode, How can I set up Tez UI configs?
[ https://issues.apache.org/jira/browse/TEZ-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569575#comment-15569575 ] Hitesh Shah commented on TEZ-2779: -- [~Sreenath] Does the UI support a config which takes in a list of RM hostnames? If not, let us consider this scenario: - Tez UI config has rm1 as the hostname for the RM. - YARN has RM HA enabled with rm1, rm2 and rm3 as the hosts. - rm1 is completely down but rm2 and rm3 are up. > When ResourceManager is HA mode, How can I set up Tez UI configs? > - > > Key: TEZ-2779 > URL: https://issues.apache.org/jira/browse/TEZ-2779 > Project: Apache Tez > Issue Type: Improvement > Components: UI >Affects Versions: 0.7.0 >Reporter: skrho >Assignee: Sreenath Somarajapuram > > I am using Tez UI 0.7.0 & hadoop 2.7.1 and It is a quite good. > I have a question. > When the resourcemanager is HA mode, How can I set up configs.js? > expecially RMWebUrl value > Now I am just setting RMWebUrl value to 'http://resourcemangerUrl1:8088' > But When the resoucemanager ACTIVE state was changed from resourcemangerUrl1 > to resourcemangerUrl2, the resourcemanger link is not working in Tez UI > because of not service resourcemangerUrl1 > Please tell me how to set configs.js ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2779) When ResourceManager is HA mode, How can I set up Tez UI configs?
[ https://issues.apache.org/jira/browse/TEZ-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569532#comment-15569532 ] Sreenath Somarajapuram commented on TEZ-2779: - [~hitesh] Was having a discussion with YARN team about HA in yarn UI. From them, cam to know that if the request hits secondary RM, it would be redirected to the active RM. So is HA required? > When ResourceManager is HA mode, How can I set up Tez UI configs? > - > > Key: TEZ-2779 > URL: https://issues.apache.org/jira/browse/TEZ-2779 > Project: Apache Tez > Issue Type: Improvement > Components: UI >Affects Versions: 0.7.0 >Reporter: skrho >Assignee: Sreenath Somarajapuram > > I am using Tez UI 0.7.0 & hadoop 2.7.1 and It is a quite good. > I have a question. > When the resourcemanager is HA mode, How can I set up configs.js? > expecially RMWebUrl value > Now I am just setting RMWebUrl value to 'http://resourcemangerUrl1:8088' > But When the resoucemanager ACTIVE state was changed from resourcemangerUrl1 > to resourcemangerUrl2, the resourcemanger link is not working in Tez UI > because of not service resourcemangerUrl1 > Please tell me how to set configs.js ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-3269) Provide basic fair routing and scheduling functionality via custom VertexManager and EdgeManager
[ https://issues.apache.org/jira/browse/TEZ-3269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569378#comment-15569378 ] Zhiyuan Yang commented on TEZ-3269: --- +1 (non-binding) > Provide basic fair routing and scheduling functionality via custom > VertexManager and EdgeManager > > > Key: TEZ-3269 > URL: https://issues.apache.org/jira/browse/TEZ-3269 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Ming Ma > Attachments: TEZ-3269-2.patch, TEZ-3269-3.patch, TEZ-3269.patch > > > With TEZ-3206 and TEZ-3216, we can build a custom VertexManager and > EdgeManager that uses partition stats to do fair routing as well as the > scheduling based on destination tasks’ dependency on source tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-3269) Provide basic fair routing and scheduling functionality via custom VertexManager and EdgeManager
[ https://issues.apache.org/jira/browse/TEZ-3269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569234#comment-15569234 ] Ming Ma commented on TEZ-3269: -- [~aplusplus] [~sseth] or anyone else, any additional comments? If it looks good, appreciate you can give a +1 so that I can commit this and use it for our integration testing. Thanks. > Provide basic fair routing and scheduling functionality via custom > VertexManager and EdgeManager > > > Key: TEZ-3269 > URL: https://issues.apache.org/jira/browse/TEZ-3269 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Ming Ma > Attachments: TEZ-3269-2.patch, TEZ-3269-3.patch, TEZ-3269.patch > > > With TEZ-3206 and TEZ-3216, we can build a custom VertexManager and > EdgeManager that uses partition stats to do fair routing as well as the > scheduling based on destination tasks’ dependency on source tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-3466) Tez classpath building to mimic mapreduce classpath building
[ https://issues.apache.org/jira/browse/TEZ-3466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568797#comment-15568797 ] Jonathan Eagles commented on TEZ-3466: -- [~hitesh], [~sseth], what do you think about this classpath feature that allows tez to build its classpath in the same way as mapreduce? > Tez classpath building to mimic mapreduce classpath building > > > Key: TEZ-3466 > URL: https://issues.apache.org/jira/browse/TEZ-3466 > Project: Apache Tez > Issue Type: New Feature >Reporter: Jonathan Eagles >Assignee: Jason Lowe > Attachments: TEZ-3466.1.patch > > > Tez builds the classpath very differently than MapReduce. For example, it > unconditionally puts PWD/* in front of the framework classpath, whereas > MapReduce will only do that if mapreduce.job.user.classpath.first=true. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-3443) Remove a repeated method
[ https://issues.apache.org/jira/browse/TEZ-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568716#comment-15568716 ] ASF GitHub Bot commented on TEZ-3443: - Github user darionyaphet closed the pull request at: https://github.com/apache/tez/pull/15 > Remove a repeated method > > > Key: TEZ-3443 > URL: https://issues.apache.org/jira/browse/TEZ-3443 > Project: Apache Tez > Issue Type: Improvement >Affects Versions: 0.8.4 >Reporter: darion yaphet >Assignee: darion yaphet >Priority: Minor > Attachments: TEZ-3443.1.patch > > > MRTask.getFsStatistics seems not used , I try to remove it . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Success: TEZ-3465 PreCommit Build #2033
Jira: https://issues.apache.org/jira/browse/TEZ-3465
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/2033/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 4812 lines...]
[INFO] Tez SUCCESS [ 0.048 s]
[INFO]
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 56:03 min
[INFO] Finished at: 2016-10-12T07:32:50+00:00
[INFO] Final Memory: 85M/1301M
[INFO]
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12832821/TEZ-3465.2.patch
against master revision c9b09cb.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/2033//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/2033//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
d79d42bd5f2fde5f433d39047b0a166ee36e122e logged out
==
==
Finished build.
==
==
Archiving artifacts
[description-setter] Description set: TEZ-3465
Recording test results
Email was triggered for: Success
Sending email for trigger: Success
###
## FAILED TESTS (if any)
##
All tests passed
[jira] [Commented] (TEZ-3465) Support broadcast edge into cartesian product vertex and forbid other edges
[
https://issues.apache.org/jira/browse/TEZ-3465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15567921#comment-15567921
]
TezQA commented on TEZ-3465:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12832821/TEZ-3465.2.patch
against master revision c9b09cb.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/2033//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/2033//console
This message is automatically generated.
> Support broadcast edge into cartesian product vertex and forbid other edges
> ---
>
> Key: TEZ-3465
> URL: https://issues.apache.org/jira/browse/TEZ-3465
> Project: Apache Tez
> Issue Type: Sub-task
>Reporter: Zhiyuan Yang
>Assignee: Zhiyuan Yang
> Attachments: TEZ-3465.1.patch, TEZ-3465.2.patch
>
>
> Cartesian product vertex manager should support other incoming edge type.
> Currently only broadcast edge is necessary, although potentially more edge
> types could also be. Custom edge need its own vertex manager which can't work
> with Cartesian product VM, so it has to be forbade.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
