[jira] [Updated] (TEZ-2986) Tez UI 2: Implement All DAGs page
[ https://issues.apache.org/jira/browse/TEZ-2986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreenath Somarajapuram updated TEZ-2986: Attachment: TEZ-2986.1.patch Added All DAGs page and implemented the required routes, adapters, controllers, serializers, models & templates. > Tez UI 2: Implement All DAGs page > - > > Key: TEZ-2986 > URL: https://issues.apache.org/jira/browse/TEZ-2986 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Attachments: TEZ-2986.1.patch > > > - Create all DAG models, adapters & serializers > - Display DAGs table -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TEZ-3041) Tez UI 2: Create sub pages
Sreenath Somarajapuram created TEZ-3041: --- Summary: Tez UI 2: Create sub pages Key: TEZ-3041 URL: https://issues.apache.org/jira/browse/TEZ-3041 Project: Apache Tez Issue Type: Sub-task Reporter: Sreenath Somarajapuram - Create Vertex > task & attempt pages - Create task > attempt page - Create App > DAGs page -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TEZ-3040) Tez UI 2: Create all details pages
Sreenath Somarajapuram created TEZ-3040: --- Summary: Tez UI 2: Create all details pages Key: TEZ-3040 URL: https://issues.apache.org/jira/browse/TEZ-3040 Project: Apache Tez Issue Type: Sub-task Reporter: Sreenath Somarajapuram - Create Vertex details, Task details, Attempt details & App details pages -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TEZ-3038) Tez UI 2: Create DAG details page
Sreenath Somarajapuram created TEZ-3038: --- Summary: Tez UI 2: Create DAG details page Key: TEZ-3038 URL: https://issues.apache.org/jira/browse/TEZ-3038 Project: Apache Tez Issue Type: Sub-task Reporter: Sreenath Somarajapuram - DAG details must be displayed as it was in UI1 - A tabular navigation interface must be available -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2986) Tez UI 2: Implement All DAGs page
[ https://issues.apache.org/jira/browse/TEZ-2986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreenath Somarajapuram updated TEZ-2986: Attachment: TEZ-2986.2.patch Fixing an UT. > Tez UI 2: Implement All DAGs page > - > > Key: TEZ-2986 > URL: https://issues.apache.org/jira/browse/TEZ-2986 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Attachments: TEZ-2986.1.patch, TEZ-2986.2.patch > > > - Create all DAG models, adapters & serializers > - Display DAGs table -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TEZ-2986) Tez UI 2: Implement All DAGs page
[ https://issues.apache.org/jira/browse/TEZ-2986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreenath Somarajapuram resolved TEZ-2986. - Resolution: Fixed Committed to TEZ-2980 > Tez UI 2: Implement All DAGs page > - > > Key: TEZ-2986 > URL: https://issues.apache.org/jira/browse/TEZ-2986 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Attachments: TEZ-2986.1.patch, TEZ-2986.2.patch > > > - Create all DAG models, adapters & serializers > - Display DAGs table -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TEZ-3039) Tez UI 2: Create all sub-pages for DAG
Sreenath Somarajapuram created TEZ-3039: --- Summary: Tez UI 2: Create all sub-pages for DAG Key: TEZ-3039 URL: https://issues.apache.org/jira/browse/TEZ-3039 Project: Apache Tez Issue Type: Sub-task Reporter: Sreenath Somarajapuram - Create All Vertices, All Tasks, All Attempts pages -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TEZ-2986) Tez UI 2: Implement All DAGs page
[ https://issues.apache.org/jira/browse/TEZ-2986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreenath Somarajapuram reassigned TEZ-2986: --- Assignee: Sreenath Somarajapuram > Tez UI 2: Implement All DAGs page > - > > Key: TEZ-2986 > URL: https://issues.apache.org/jira/browse/TEZ-2986 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > > - Create all DAG models, adapters & serializers > - Display DAGs table -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TEZ-3042) Tez UI 2: Create Counters pages
Sreenath Somarajapuram created TEZ-3042: --- Summary: Tez UI 2: Create Counters pages Key: TEZ-3042 URL: https://issues.apache.org/jira/browse/TEZ-3042 Project: Apache Tez Issue Type: Sub-task Reporter: Sreenath Somarajapuram - Create DAG, Vertex, Task & Attempt counters pages -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TEZ-3043) Tez UI 2: Create configurations page
Sreenath Somarajapuram created TEZ-3043: --- Summary: Tez UI 2: Create configurations page Key: TEZ-3043 URL: https://issues.apache.org/jira/browse/TEZ-3043 Project: Apache Tez Issue Type: Sub-task Reporter: Sreenath Somarajapuram - Configurations page for App - Configurations page for vertex -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (TEZ-2164) Shade the guava version used by Tez and move to guava-18
[ https://issues.apache.org/jira/browse/TEZ-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101991#comment-15101991 ] Hitesh Shah edited comment on TEZ-2164 at 1/15/16 4:19 PM: --- The problem is due to module dependencies. If all tez modules are combined into a single tez-assembly.jar at the very end, then relocation would work.Doing this a module at a time does not work. However, if tez-api is relocated and tez-dag say depends on tez-api, this will create problems if any internal functions in tez-api use guava objects ( and tez-dag tries to invoke such functions ) as tez-dag will be referring to vanilla guava packages ( compile happens before relocation ) whereas tez-api will only know about the relocated package location. was (Author: hitesh): The problem is due to module dependencies. If all tez modules are combined into a single tez-assembly.jar at the very end, then relocation would work. However, if tez-api is relocated and tez-dag say depends on tez-api, this will create problems if any internal functions in tez-api use guava objects ( and tez-dag tries to invoke such functions ) as tez-dag will be referring to vanilla guava packages ( compile happens before relocation ) whereas tez-api will only know about the relocated package location. > Shade the guava version used by Tez and move to guava-18 > > > Key: TEZ-2164 > URL: https://issues.apache.org/jira/browse/TEZ-2164 > Project: Apache Tez > Issue Type: Improvement >Reporter: Siddharth Seth >Assignee: Hitesh Shah >Priority: Critical > Attachments: TEZ-2164.3.patch, TEZ-2164.4.patch, > TEZ-2164.wip.2.patch, allow-guava-16.0.1.patch > > > Should allow us to upgrade to a newer version without shipping a guava > dependency. > Would be good to do this in 0.7 so that we stop shipping guava as early as > possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2164) Shade the guava version used by Tez and move to guava-18
[ https://issues.apache.org/jira/browse/TEZ-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101991#comment-15101991 ] Hitesh Shah commented on TEZ-2164: -- The problem is due to module dependencies. If all tez modules are combined into a single tez-assembly.jar at the very end, then relocation would work. However, if tez-api is relocated and tez-dag say depends on tez-api, this will create problems if any internal functions in tez-api use guava objects ( and tez-dag tries to invoke such functions ) as tez-dag will be referring to vanilla guava packages ( compile happens before relocation ) whereas tez-api will only know about the relocated package location. > Shade the guava version used by Tez and move to guava-18 > > > Key: TEZ-2164 > URL: https://issues.apache.org/jira/browse/TEZ-2164 > Project: Apache Tez > Issue Type: Improvement >Reporter: Siddharth Seth >Assignee: Hitesh Shah >Priority: Critical > Attachments: TEZ-2164.3.patch, TEZ-2164.4.patch, > TEZ-2164.wip.2.patch, allow-guava-16.0.1.patch > > > Should allow us to upgrade to a newer version without shipping a guava > dependency. > Would be good to do this in 0.7 so that we stop shipping guava as early as > possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2164) Shade the guava version used by Tez and move to guava-18
[ https://issues.apache.org/jira/browse/TEZ-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101997#comment-15101997 ] Hitesh Shah commented on TEZ-2164: -- Also, both curator and YARN ( yarn depends on curator ) expose public APIs that use guava objects so there are still a bunch of issues that resolving before guava is cleanly separated out/removed. > Shade the guava version used by Tez and move to guava-18 > > > Key: TEZ-2164 > URL: https://issues.apache.org/jira/browse/TEZ-2164 > Project: Apache Tez > Issue Type: Improvement >Reporter: Siddharth Seth >Assignee: Hitesh Shah >Priority: Critical > Attachments: TEZ-2164.3.patch, TEZ-2164.4.patch, > TEZ-2164.wip.2.patch, allow-guava-16.0.1.patch > > > Should allow us to upgrade to a newer version without shipping a guava > dependency. > Would be good to do this in 0.7 so that we stop shipping guava as early as > possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TEZ-3044) Split TaskCommunicatorContext.taskHeartbeat into separate methods
Siddharth Seth created TEZ-3044: --- Summary: Split TaskCommunicatorContext.taskHeartbeat into separate methods Key: TEZ-3044 URL: https://issues.apache.org/jira/browse/TEZ-3044 Project: Apache Tez Issue Type: Improvement Reporter: Siddharth Seth Assignee: Siddharth Seth taskHeartbeat currently handles 3 different operations 1. The heartbeat itself to indicate that the container is alive. 2. Reporting new events sent over by the task 3. Fetching events which are to be consumed by the task. All three can be independent. This does not affect the Tez wire protocol itself, which will continue to use a single call for all three. However this would get split within the default Tez AM plugin. The eventual intent is to allow plugins to control this functionality independently. An example: The addition of a eventsAvailable(Task*) on the TaskCommunicator allows a Plugin to get events the moment they're available and push them into the executors. Meanwhile, the heartbeat becomes just that - a heartbeat - and the frequency can be reduced to multiple seconds instead of the current milliseconds. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2937) Can Processor.close() be called after closing inputs and outputs?
[ https://issues.apache.org/jira/browse/TEZ-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102209#comment-15102209 ] Siddharth Seth commented on TEZ-2937: - cc [~hagleitn] - does this cause issues with Hive, since it's the opposite of what was changed in TEZ-955. Looking at the Hive processor on master - it doesn't do anything in close. Tez should allow the processor to control when Inputs / Outputs are closed. > Can Processor.close() be called after closing inputs and outputs? > - > > Key: TEZ-2937 > URL: https://issues.apache.org/jira/browse/TEZ-2937 > Project: Apache Tez > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Jonathan Eagles > Fix For: 0.7.1, 0.8.3 > > Attachments: TEZ-2937.1.patch > > > Pig hit PIG-4722 as processor.close() which clears some static thread local > variables was called when SpillThread was still running the Combiner. > LogicalIOProcessorRuntimeTask.java calls processor.close() before closing the > inputs and outputs in both close() and cleanup() methods. Can Tez change to > closing the processor after the inputs and outputs are closed? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-3044) Split TaskCommunicatorContext.taskHeartbeat into separate methods
[ https://issues.apache.org/jira/browse/TEZ-3044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3044: Attachment: TEZ-3044.1.wip.txt WIP patch. [~hitesh] - thoughts? > Split TaskCommunicatorContext.taskHeartbeat into separate methods > - > > Key: TEZ-3044 > URL: https://issues.apache.org/jira/browse/TEZ-3044 > Project: Apache Tez > Issue Type: Improvement >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: TEZ-3044.1.wip.txt > > > taskHeartbeat currently handles 3 different operations > 1. The heartbeat itself to indicate that the container is alive. > 2. Reporting new events sent over by the task > 3. Fetching events which are to be consumed by the task. > All three can be independent. > This does not affect the Tez wire protocol itself, which will continue to use > a single call for all three. However this would get split within the default > Tez AM plugin. > The eventual intent is to allow plugins to control this functionality > independently. > An example: The addition of a eventsAvailable(Task*) on the TaskCommunicator > allows a Plugin to get events the moment they're available and push them into > the executors. Meanwhile, the heartbeat becomes just that - a heartbeat - and > the frequency can be reduced to multiple seconds instead of the current > milliseconds. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-3036) Tez AM can hang on startup with no indication of error
[ https://issues.apache.org/jira/browse/TEZ-3036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated TEZ-3036: Attachment: TEZ-3036.001.patch Attaching a prototype patch that seems to fix the issue. This has the ServiceThread invoke the state change callback for dependent services when starting the service throws. Still needs a unit test, but I manually tested by hardcoding WebUIService to throw an error when it starts. Initially I thought of a simpler approach where it simply converts any exception caught by the ServiceThread into an error and let the uncaught exception handler tear everything down. However this also hangs because the DAGAppMaster shutdown hook ends up waiting for the lock being held during service startup. > Tez AM can hang on startup with no indication of error > -- > > Key: TEZ-3036 > URL: https://issues.apache.org/jira/browse/TEZ-3036 > Project: Apache Tez > Issue Type: Bug >Affects Versions: 0.7.0 >Reporter: Jason Lowe >Priority: Critical > Attachments: TEZ-3036.001.patch > > > I've seen a couple of instances where the Tez AM fails to complete the > startup sequence. It never gets around to registering with the > ResourceManager, so the RM eventually times out the attempt and starts > another. The subsequent attempts do the same. There are no indications in > the logs that anything is wrong, rather it just seems to get stuck during > startup then a bit over 10 minutes later is killed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Failed: TEZ-3044 PreCommit Build #1425
Jira: https://issues.apache.org/jira/browse/TEZ-3044
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1425/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 3634 lines...]
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :tez-tests
[INFO] Build failures were ignored.
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12782593/TEZ-3044.1.wip.txt
against master revision b0ba133.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
See
https://builds.apache.org/job/PreCommit-TEZ-Build/1425//artifact/patchprocess/diffJavadocWarnings.txt
for details.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.test.TestFaultTolerance
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1425//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1425//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
197074b8ea60eeb48aab9a8d70c01096c7ed2f4f logged out
==
==
Finished build.
==
==
Build step 'Execute shell' marked build as failure
Archiving artifacts
Compressed 3.18 MB of artifacts by 23.6% relative to #1421
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
###
## FAILED TESTS (if any)
##
7 tests failed.
FAILED: org.apache.tez.test.TestFaultTolerance.testRandomFailingInputs
Error Message:
expected: but was:
Stack Trace:
java.lang.AssertionError: expected: but was:
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:144)
at
org.apache.tez.test.TestFaultTolerance.runDAGAndVerify(TestFaultTolerance.java:141)
at
org.apache.tez.test.TestFaultTolerance.runDAGAndVerify(TestFaultTolerance.java:124)
at
org.apache.tez.test.TestFaultTolerance.runDAGAndVerify(TestFaultTolerance.java:120)
at
org.apache.tez.test.TestFaultTolerance.testRandomFailingInputs(TestFaultTolerance.java:763)
FAILED: org.apache.tez.test.TestFaultTolerance.testBasicInputFailureWithExit
Error Message:
TezSession has already shutdown. No cluster diagnostics found.
Stack Trace:
org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown. No
cluster diagnostics found.
at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:784)
at
org.apache.tez.test.TestFaultTolerance.runDAGAndVerify(TestFaultTolerance.java:129)
at
org.apache.tez.test.TestFaultTolerance.runDAGAndVerify(TestFaultTolerance.java:124)
at
org.apache.tez.test.TestFaultTolerance.runDAGAndVerify(TestFaultTolerance.java:120)
at
org.apache.tez.test.TestFaultTolerance.testBasicInputFailureWithExit(TestFaultTolerance.java:261)
FAILED:
org.apache.tez.test.TestFaultTolerance.testInputFailureRerunCanSendOutputToTwoDownstreamVertices
Error Message:
TezSession has already shutdown. No cluster diagnostics found.
Stack Trace:
org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown. No
cluster diagnostics found.
at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:784)
at
org.apache.tez.test.TestFaultTolerance.runDAGAndVerify(TestFaultTolerance.java:129)
at
[jira] [Commented] (TEZ-3044) Split TaskCommunicatorContext.taskHeartbeat into separate methods
[
https://issues.apache.org/jira/browse/TEZ-3044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102495#comment-15102495
]
TezQA commented on TEZ-3044:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12782593/TEZ-3044.1.wip.txt
against master revision b0ba133.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
See
https://builds.apache.org/job/PreCommit-TEZ-Build/1425//artifact/patchprocess/diffJavadocWarnings.txt
for details.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.test.TestFaultTolerance
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1425//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1425//console
This message is automatically generated.
> Split TaskCommunicatorContext.taskHeartbeat into separate methods
> -
>
> Key: TEZ-3044
> URL: https://issues.apache.org/jira/browse/TEZ-3044
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Siddharth Seth
>Assignee: Siddharth Seth
> Attachments: TEZ-3044.1.wip.txt
>
>
> taskHeartbeat currently handles 3 different operations
> 1. The heartbeat itself to indicate that the container is alive.
> 2. Reporting new events sent over by the task
> 3. Fetching events which are to be consumed by the task.
> All three can be independent.
> This does not affect the Tez wire protocol itself, which will continue to use
> a single call for all three. However this would get split within the default
> Tez AM plugin.
> The eventual intent is to allow plugins to control this functionality
> independently.
> An example: The addition of a eventsAvailable(Task*) on the TaskCommunicator
> allows a Plugin to get events the moment they're available and push them into
> the executors. Meanwhile, the heartbeat becomes just that - a heartbeat - and
> the frequency can be reduced to multiple seconds instead of the current
> milliseconds.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-3036) Tez AM can hang on startup with no indication of error
[
https://issues.apache.org/jira/browse/TEZ-3036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102478#comment-15102478
]
Jason Lowe commented on TEZ-3036:
-
My apologies, I misread the heap dump info. NoSuchMethodError was being
propagated up rather than NoSuchMethodException (which is the cause of the
error). The issue occurs with no notification when an error is thrown rather
than an exception. If an exception is thrown then it will log it but I think
it will still hang.
Here's the relevant portions of the stacktrace when this occurs:
{noformat}
"ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerEventHandler" #34 prio=5
os_prio=0 tid=0x7f2be161c000 nid=0x65a4 in Object.wait()
[0x7f2bb87b5000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xf55ddd58> (a
org.apache.tez.dag.app.DAGAppMaster$ServiceWithDependency)
at
org.apache.tez.dag.app.DAGAppMaster$ServiceWithDependency.start(DAGAppMaster.java:1655)
- locked <0xf55ddd58> (a
org.apache.tez.dag.app.DAGAppMaster$ServiceWithDependency)
at
org.apache.tez.dag.app.DAGAppMaster$ServiceThread.run(DAGAppMaster.java:1693)
"main" #1 prio=5 os_prio=0 tid=0x7f2be0019800 nid=0x653b in Object.wait()
[0x7f2be57e1000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xf6857998> (a
org.apache.tez.dag.app.DAGAppMaster$ServiceThread)
at java.lang.Thread.join(Thread.java:1245)
- locked <0xf6857998> (a
org.apache.tez.dag.app.DAGAppMaster$ServiceThread)
at java.lang.Thread.join(Thread.java:1319)
at
org.apache.tez.dag.app.DAGAppMaster.startServices(DAGAppMaster.java:1730)
at
org.apache.tez.dag.app.DAGAppMaster.serviceStart(DAGAppMaster.java:1799)
- locked <0xa0326928> (a org.apache.tez.dag.app.DAGAppMaster)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
- locked <0xa0326ae8> (a java.lang.Object)
at org.apache.tez.dag.app.DAGAppMaster$6.run(DAGAppMaster.java:2369)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1679)
at
org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2365)
at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2173)
{noformat}
> Tez AM can hang on startup with no indication of error
> --
>
> Key: TEZ-3036
> URL: https://issues.apache.org/jira/browse/TEZ-3036
> Project: Apache Tez
> Issue Type: Bug
>Affects Versions: 0.7.0
>Reporter: Jason Lowe
>Priority: Critical
>
> I've seen a couple of instances where the Tez AM fails to complete the
> startup sequence. It never gets around to registering with the
> ResourceManager, so the RM eventually times out the attempt and starts
> another. The subsequent attempts do the same. There are no indications in
> the logs that anything is wrong, rather it just seems to get stuck during
> startup then a bit over 10 minutes later is killed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2164) Shade the guava version used by Tez and move to guava-18
[ https://issues.apache.org/jira/browse/TEZ-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15103010#comment-15103010 ] Rajat Jain commented on TEZ-2164: - Thanks for the explanation. > Shade the guava version used by Tez and move to guava-18 > > > Key: TEZ-2164 > URL: https://issues.apache.org/jira/browse/TEZ-2164 > Project: Apache Tez > Issue Type: Improvement >Reporter: Siddharth Seth >Assignee: Hitesh Shah >Priority: Critical > Attachments: TEZ-2164.3.patch, TEZ-2164.4.patch, > TEZ-2164.wip.2.patch, allow-guava-16.0.1.patch > > > Should allow us to upgrade to a newer version without shipping a guava > dependency. > Would be good to do this in 0.7 so that we stop shipping guava as early as > possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Failed: TEZ-3036 PreCommit Build #1426
Jira: https://issues.apache.org/jira/browse/TEZ-3036
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1426/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 3763 lines...]
[INFO] Total time: 01:02 h
[INFO] Finished at: 2016-01-16T00:21:55+00:00
[INFO] Final Memory: 60M/884M
[INFO]
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12782610/TEZ-3036.001.patch
against master revision b0ba133.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1426//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1426//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
a225abe6a8cc16224972b979f3f293e30838308b logged out
==
==
Finished build.
==
==
Build step 'Execute shell' marked build as failure
Archiving artifacts
Compressed 3.17 MB of artifacts by 27.6% relative to #1421
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
###
## FAILED TESTS (if any)
##
All tests passed
[jira] [Commented] (TEZ-3036) Tez AM can hang on startup with no indication of error
[
https://issues.apache.org/jira/browse/TEZ-3036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102770#comment-15102770
]
TezQA commented on TEZ-3036:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12782610/TEZ-3036.001.patch
against master revision b0ba133.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1426//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1426//console
This message is automatically generated.
> Tez AM can hang on startup with no indication of error
> --
>
> Key: TEZ-3036
> URL: https://issues.apache.org/jira/browse/TEZ-3036
> Project: Apache Tez
> Issue Type: Bug
>Affects Versions: 0.7.0
>Reporter: Jason Lowe
>Assignee: Jason Lowe
>Priority: Critical
> Attachments: TEZ-3036.001.patch
>
>
> I've seen a couple of instances where the Tez AM fails to complete the
> startup sequence. It never gets around to registering with the
> ResourceManager, so the RM eventually times out the attempt and starts
> another. The subsequent attempts do the same. There are no indications in
> the logs that anything is wrong, rather it just seems to get stuck during
> startup then a bit over 10 minutes later is killed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
