[jira] [Updated] (TEZ-3064) Tez UI 2: Add All DAGs filters
[ https://issues.apache.org/jira/browse/TEZ-3064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreenath Somarajapuram updated TEZ-3064: Attachment: TEZ-3064.1.patch - Added filters in All DAGs page - Updated pagination to just have row selector - Added refresh button into All DAGs page - Added client side filtering for records in All DAGs > Tez UI 2: Add All DAGs filters > -- > > Key: TEZ-3064 > URL: https://issues.apache.org/jira/browse/TEZ-3064 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Attachments: TEZ-3064.1.patch > > > Filter for - Name, Id, Submitter, Status, Application and Content ID -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TEZ-3064) Tez UI 2: Add All DAGs filters
[ https://issues.apache.org/jira/browse/TEZ-3064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreenath Somarajapuram resolved TEZ-3064. - Resolution: Cannot Reproduce Committed to TEZ-2980 branch > Tez UI 2: Add All DAGs filters > -- > > Key: TEZ-3064 > URL: https://issues.apache.org/jira/browse/TEZ-3064 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Attachments: TEZ-3064.1.patch > > > Filter for - Name, Id, Submitter, Status, Application and Content ID -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TEZ-3064) Tez UI 2: Add All DAGs filters
[ https://issues.apache.org/jira/browse/TEZ-3064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreenath Somarajapuram reassigned TEZ-3064: --- Assignee: Sreenath Somarajapuram > Tez UI 2: Add All DAGs filters > -- > > Key: TEZ-3064 > URL: https://issues.apache.org/jira/browse/TEZ-3064 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > > Filter for - Name, Id, Submitter, Status, Application and Content ID -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-3055) I got this jar is by the mvn package and run it on my cluster,occurred this problem.I can run the example which the tez given successful.But when I run this jar which is
[ https://issues.apache.org/jira/browse/TEZ-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110284#comment-15110284 ] Lvpenglin commented on TEZ-3055: Can you explain what mean this property(tez.aux.uris)?I searched some information,but none can be used.thank you. > I got this jar is by the mvn package and run it on my cluster,occurred this > problem.I can run the example which the tez given successful.But when I run > this jar which is packaged by mvn packsge myself is failed. > --- > > Key: TEZ-3055 > URL: https://issues.apache.org/jira/browse/TEZ-3055 > Project: Apache Tez > Issue Type: Improvement > Environment: this is the tez orderedwordcount example,I want it can > be run on my cluster,but occurred this problem.This code is packaged by > myself.When I run the tez's example jar on my cluster,it's ok.My environment > is hadoop2.6.0+tez0.7.0.Can you give me some suggestion? >Reporter: Lvpenglin > Labels: jar, maven-package, run-example, tez > > Vertex failed, vertexName=ASSOCIATE, vertexId=vertex_1452667827400_0022_1_00, > diagnostics= > Task failed, taskId=task_1452667827400_0022_1_00_00, diagnostics= > TaskAttempt 0 failed, info= > Error: Failure while running > task:org.apache.tez.dag.api.TezUncheckedException: Unable to load class: > com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor > at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:45) > at > org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:96) > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.createProcessor(LogicalIOProcessorRuntimeTask.java:584) > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.initialize(LogicalIOProcessorRuntimeTask.java:202) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167) > at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.ClassNotFoundException: > com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:274) > at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:43) > ... 15 more > TaskAttempt 1 failed, info= > Error: Failure while running > task:org.apache.tez.dag.api.TezUncheckedException: Unable to load class: > com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor > at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:45) > at > org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:96) > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.createProcessor(LogicalIOProcessorRuntimeTask.java:584) > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.initialize(LogicalIOProcessorRuntimeTask.java:202) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171) > at >
[jira] [Reopened] (TEZ-3064) Tez UI 2: Add All DAGs filters
[ https://issues.apache.org/jira/browse/TEZ-3064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah reopened TEZ-3064: -- > Tez UI 2: Add All DAGs filters > -- > > Key: TEZ-3064 > URL: https://issues.apache.org/jira/browse/TEZ-3064 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Attachments: TEZ-3064.1.patch > > > Filter for - Name, Id, Submitter, Status, Application and Content ID -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TEZ-3064) Tez UI 2: Add All DAGs filters
[ https://issues.apache.org/jira/browse/TEZ-3064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah resolved TEZ-3064. -- Resolution: Fixed > Tez UI 2: Add All DAGs filters > -- > > Key: TEZ-3064 > URL: https://issues.apache.org/jira/browse/TEZ-3064 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Attachments: TEZ-3064.1.patch > > > Filter for - Name, Id, Submitter, Status, Application and Content ID -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TEZ-3067) Links to tez configs documentation should be bubbled up to top-level release page
Hitesh Shah created TEZ-3067: Summary: Links to tez configs documentation should be bubbled up to top-level release page Key: TEZ-3067 URL: https://issues.apache.org/jira/browse/TEZ-3067 Project: Apache Tez Issue Type: Bug Reporter: Hitesh Shah http://tez.apache.org/releases/0.8.2/tez-api-javadocs/configs/TezConfiguration.html is hidden away in the api docs. Would you useful to update the release template to add direct links to the config docs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-3066) TaskAttemptFinishedEvent ConcurrentModificationException in recovery or history logging services
[ https://issues.apache.org/jira/browse/TEZ-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-3066: Attachment: TEZ-3066-1.patch > TaskAttemptFinishedEvent ConcurrentModificationException in recovery or > history logging services > > > Key: TEZ-3066 > URL: https://issues.apache.org/jira/browse/TEZ-3066 > Project: Apache Tez > Issue Type: Bug >Affects Versions: 0.7.0 >Reporter: Jason Lowe >Assignee: Jeff Zhang > Attachments: TEZ-3066-1.patch > > > A ConcurrentModificationException can occur if a TaskAttemptFinishedEvent is > processed by the recovery service or another history logging service. Sample > stacktraces to follow. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-3055) I got this jar is by the mvn package and run it on my cluster,occurred this problem.I can run the example which the tez given successful.But when I run this jar which is
[
https://issues.apache.org/jira/browse/TEZ-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15111036#comment-15111036
]
Hitesh Shah commented on TEZ-3055:
--
tez.aux.uris is a way to add additional jars/files to the runtime of the AM and
the task containers. The format is a comma-separated list of files or
directories on HDFS.
you can set tez.aux.uris to {{"${fs.defaultFS}/tmp/sgprocessor.jar}} for
example.
> I got this jar is by the mvn package and run it on my cluster,occurred this
> problem.I can run the example which the tez given successful.But when I run
> this jar which is packaged by mvn packsge myself is failed.
> ---
>
> Key: TEZ-3055
> URL: https://issues.apache.org/jira/browse/TEZ-3055
> Project: Apache Tez
> Issue Type: Improvement
> Environment: this is the tez orderedwordcount example,I want it can
> be run on my cluster,but occurred this problem.This code is packaged by
> myself.When I run the tez's example jar on my cluster,it's ok.My environment
> is hadoop2.6.0+tez0.7.0.Can you give me some suggestion?
>Reporter: Lvpenglin
> Labels: jar, maven-package, run-example, tez
>
> Vertex failed, vertexName=ASSOCIATE, vertexId=vertex_1452667827400_0022_1_00,
> diagnostics=
> Task failed, taskId=task_1452667827400_0022_1_00_00, diagnostics=
> TaskAttempt 0 failed, info=
> Error: Failure while running
> task:org.apache.tez.dag.api.TezUncheckedException: Unable to load class:
> com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor
> at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:45)
> at
> org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:96)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.createProcessor(LogicalIOProcessorRuntimeTask.java:584)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.initialize(LogicalIOProcessorRuntimeTask.java:202)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.ClassNotFoundException:
> com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:274)
> at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:43)
> ... 15 more
> TaskAttempt 1 failed, info=
> Error: Failure while running
> task:org.apache.tez.dag.api.TezUncheckedException: Unable to load class:
> com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor
> at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:45)
> at
> org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:96)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.createProcessor(LogicalIOProcessorRuntimeTask.java:584)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.initialize(LogicalIOProcessorRuntimeTask.java:202)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
>
[jira] [Commented] (TEZ-3066) TaskAttemptFinishedEvent ConcurrentModificationException in recovery or history logging services
[ https://issues.apache.org/jira/browse/TEZ-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15111081#comment-15111081 ] Jeff Zhang commented on TEZ-3066: - This issue may happen when task attempt is in the process of kill (TaskAttemptStateInternal.KILL_IN_PROGRESS), in this case we have logged the TaskAttemptFinishedEvent, but the task attempt may still alive and will heartbeat with AM. which cause the ConcurrentModificationException here. Although it might be better to log TaskAttemptFinishedEvent in the last state, but it require much change on the TaskAttempt state machine, the easier way is to copy a new list for dataEvents to avoid the ConcurrentModificationException. Also check taGeneratedEvents in TaskAttemptFinishedEvent, it won't be updated after TaskAttemptFinishedEvent is created, otherwise it is some bug of TaskAttempt's state machine. [~hitesh] [~bikassaha] Please help review it. > TaskAttemptFinishedEvent ConcurrentModificationException in recovery or > history logging services > > > Key: TEZ-3066 > URL: https://issues.apache.org/jira/browse/TEZ-3066 > Project: Apache Tez > Issue Type: Bug >Affects Versions: 0.7.0 >Reporter: Jason Lowe >Assignee: Jeff Zhang > Attachments: TEZ-3066-1.patch > > > A ConcurrentModificationException can occur if a TaskAttemptFinishedEvent is > processed by the recovery service or another history logging service. Sample > stacktraces to follow. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-3036) Tez AM can hang on startup with no indication of error
[ https://issues.apache.org/jira/browse/TEZ-3036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated TEZ-3036: Attachment: TEZ-3036.002.patch Attaching patch version with comments and volatile ServiceError.thread. Thanks for the review! Committing this. > Tez AM can hang on startup with no indication of error > -- > > Key: TEZ-3036 > URL: https://issues.apache.org/jira/browse/TEZ-3036 > Project: Apache Tez > Issue Type: Bug >Affects Versions: 0.7.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Critical > Attachments: TEZ-3036.001.patch, TEZ-3036.002.patch > > > I've seen a couple of instances where the Tez AM fails to complete the > startup sequence. It never gets around to registering with the > ResourceManager, so the RM eventually times out the attempt and starts > another. The subsequent attempts do the same. There are no indications in > the logs that anything is wrong, rather it just seems to get stuck during > startup then a bit over 10 minutes later is killed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-3066) TaskAttemptFinishedEvent ConcurrentModificationException in recovery or history logging services
[ https://issues.apache.org/jira/browse/TEZ-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15111659#comment-15111659 ] Bikas Saha commented on TEZ-3066: - Please avoid bulk import. "+import org.apache.tez.dag.app.dag.*;" Could you please create a jira for a better fix with some of the ideas discussed offline and reference that in a comment near the fix. +1. > TaskAttemptFinishedEvent ConcurrentModificationException in recovery or > history logging services > > > Key: TEZ-3066 > URL: https://issues.apache.org/jira/browse/TEZ-3066 > Project: Apache Tez > Issue Type: Bug >Affects Versions: 0.7.0 >Reporter: Jason Lowe >Assignee: Jeff Zhang > Attachments: TEZ-3066-1.patch, TEZ-3066-2.patch > > > A ConcurrentModificationException can occur if a TaskAttemptFinishedEvent is > processed by the recovery service or another history logging service. Sample > stacktraces to follow. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TEZ-3068) Log TaskAttemptFinishedEvent after TaskAttempt is really finished.
Jeff Zhang created TEZ-3068: --- Summary: Log TaskAttemptFinishedEvent after TaskAttempt is really finished. Key: TEZ-3068 URL: https://issues.apache.org/jira/browse/TEZ-3068 Project: Apache Tez Issue Type: Improvement Reporter: Jeff Zhang Assignee: Jeff Zhang -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-3066) TaskAttemptFinishedEvent ConcurrentModificationException in recovery or history logging services
[ https://issues.apache.org/jira/browse/TEZ-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15111539#comment-15111539 ] Jeff Zhang commented on TEZ-3066: - Discuss with [~bikassaha] offline, copy events for every task attempt will too costly. Attach another patch to avoid the event update in the middle of task attempt TERMINATING. > TaskAttemptFinishedEvent ConcurrentModificationException in recovery or > history logging services > > > Key: TEZ-3066 > URL: https://issues.apache.org/jira/browse/TEZ-3066 > Project: Apache Tez > Issue Type: Bug >Affects Versions: 0.7.0 >Reporter: Jason Lowe >Assignee: Jeff Zhang > Attachments: TEZ-3066-1.patch, TEZ-3066-2.patch > > > A ConcurrentModificationException can occur if a TaskAttemptFinishedEvent is > processed by the recovery service or another history logging service. Sample > stacktraces to follow. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-3066) TaskAttemptFinishedEvent ConcurrentModificationException in recovery or history logging services
[ https://issues.apache.org/jira/browse/TEZ-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-3066: Attachment: TEZ-3066-2.patch > TaskAttemptFinishedEvent ConcurrentModificationException in recovery or > history logging services > > > Key: TEZ-3066 > URL: https://issues.apache.org/jira/browse/TEZ-3066 > Project: Apache Tez > Issue Type: Bug >Affects Versions: 0.7.0 >Reporter: Jason Lowe >Assignee: Jeff Zhang > Attachments: TEZ-3066-1.patch, TEZ-3066-2.patch > > > A ConcurrentModificationException can occur if a TaskAttemptFinishedEvent is > processed by the recovery service or another history logging service. Sample > stacktraces to follow. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-3066) TaskAttemptFinishedEvent ConcurrentModificationException in recovery or history logging services
[
https://issues.apache.org/jira/browse/TEZ-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15111816#comment-15111816
]
TezQA commented on TEZ-3066:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12783712/TEZ-3066-3.patch
against master revision 92def52.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1431//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1431//console
This message is automatically generated.
> TaskAttemptFinishedEvent ConcurrentModificationException in recovery or
> history logging services
>
>
> Key: TEZ-3066
> URL: https://issues.apache.org/jira/browse/TEZ-3066
> Project: Apache Tez
> Issue Type: Bug
>Affects Versions: 0.7.0
>Reporter: Jason Lowe
>Assignee: Jeff Zhang
> Attachments: TEZ-3066-1.patch, TEZ-3066-2.patch, TEZ-3066-3.patch
>
>
> A ConcurrentModificationException can occur if a TaskAttemptFinishedEvent is
> processed by the recovery service or another history logging service. Sample
> stacktraces to follow.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Failed: TEZ-3066 PreCommit Build #1431
Jira: https://issues.apache.org/jira/browse/TEZ-3066
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1431/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 3767 lines...]
[INFO]
[INFO] Total time: 52:28 min
[INFO] Finished at: 2016-01-22T02:37:18+00:00
[INFO] Final Memory: 77M/895M
[INFO]
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12783712/TEZ-3066-3.patch
against master revision 92def52.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1431//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1431//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
12ae81b0f8e22ae1f658ddf8041a36067efaaca8 logged out
==
==
Finished build.
==
==
Build step 'Execute shell' marked build as failure
Archiving artifacts
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
###
## FAILED TESTS (if any)
##
All tests passed
[jira] [Updated] (TEZ-3066) TaskAttemptFinishedEvent ConcurrentModificationException in recovery or history logging services
[ https://issues.apache.org/jira/browse/TEZ-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-3066: Attachment: TEZ-3066-3.patch Minor update the patch, address the comments. > TaskAttemptFinishedEvent ConcurrentModificationException in recovery or > history logging services > > > Key: TEZ-3066 > URL: https://issues.apache.org/jira/browse/TEZ-3066 > Project: Apache Tez > Issue Type: Bug >Affects Versions: 0.7.0 >Reporter: Jason Lowe >Assignee: Jeff Zhang > Attachments: TEZ-3066-1.patch, TEZ-3066-2.patch, TEZ-3066-3.patch > > > A ConcurrentModificationException can occur if a TaskAttemptFinishedEvent is > processed by the recovery service or another history logging service. Sample > stacktraces to follow. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-3055) I got this jar is by the mvn package and run it on my cluster,occurred this problem.I can run the example which the tez given successful.But when I run this jar which is
[
https://issues.apache.org/jira/browse/TEZ-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15111782#comment-15111782
]
Hitesh Shah commented on TEZ-3055:
--
Your error seems to be this:
{code}
Vertex failed, vertexName=ASSOCIATE, vertexId=vertex_1453425211412_0004_1_00,
diagnostics=
Task failed, taskId=task_1453425211412_0004_1_00_00, diagnostics=
TaskAttempt 0 failed, info=
Error: Failure while running task:java.lang.NullPointerException
at
com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor.run(TezDataGeneratorTest.java:78){code}
It seems like it is running on the cluster. You can obtain the yarn logs via
"bin/yarn logs -application application_1453425211412_0004" to debug further.
> I got this jar is by the mvn package and run it on my cluster,occurred this
> problem.I can run the example which the tez given successful.But when I run
> this jar which is packaged by mvn packsge myself is failed.
> ---
>
> Key: TEZ-3055
> URL: https://issues.apache.org/jira/browse/TEZ-3055
> Project: Apache Tez
> Issue Type: Improvement
> Environment: this is the tez orderedwordcount example,I want it can
> be run on my cluster,but occurred this problem.This code is packaged by
> myself.When I run the tez's example jar on my cluster,it's ok.My environment
> is hadoop2.6.0+tez0.7.0.Can you give me some suggestion?
>Reporter: Lvpenglin
> Labels: jar, maven-package, run-example, tez
>
> Vertex failed, vertexName=ASSOCIATE, vertexId=vertex_1452667827400_0022_1_00,
> diagnostics=
> Task failed, taskId=task_1452667827400_0022_1_00_00, diagnostics=
> TaskAttempt 0 failed, info=
> Error: Failure while running
> task:org.apache.tez.dag.api.TezUncheckedException: Unable to load class:
> com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor
> at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:45)
> at
> org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:96)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.createProcessor(LogicalIOProcessorRuntimeTask.java:584)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.initialize(LogicalIOProcessorRuntimeTask.java:202)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.ClassNotFoundException:
> com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:274)
> at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:43)
> ... 15 more
> TaskAttempt 1 failed, info=
> Error: Failure while running
> task:org.apache.tez.dag.api.TezUncheckedException: Unable to load class:
> com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor
> at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:45)
> at
> org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:96)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.createProcessor(LogicalIOProcessorRuntimeTask.java:584)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.initialize(LogicalIOProcessorRuntimeTask.java:202)
> at
>
[jira] [Commented] (TEZ-3066) TaskAttemptFinishedEvent ConcurrentModificationException in recovery or history logging services
[
https://issues.apache.org/jira/browse/TEZ-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15111729#comment-15111729
]
TezQA commented on TEZ-3066:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12783694/TEZ-3066-2.patch
against master revision 92def52.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1430//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1430//console
This message is automatically generated.
> TaskAttemptFinishedEvent ConcurrentModificationException in recovery or
> history logging services
>
>
> Key: TEZ-3066
> URL: https://issues.apache.org/jira/browse/TEZ-3066
> Project: Apache Tez
> Issue Type: Bug
>Affects Versions: 0.7.0
>Reporter: Jason Lowe
>Assignee: Jeff Zhang
> Attachments: TEZ-3066-1.patch, TEZ-3066-2.patch, TEZ-3066-3.patch
>
>
> A ConcurrentModificationException can occur if a TaskAttemptFinishedEvent is
> processed by the recovery service or another history logging service. Sample
> stacktraces to follow.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Failed: TEZ-3066 PreCommit Build #1430
Jira: https://issues.apache.org/jira/browse/TEZ-3066
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1430/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 3767 lines...]
[INFO]
[INFO] Total time: 57:13 min
[INFO] Finished at: 2016-01-22T01:14:35+00:00
[INFO] Final Memory: 73M/1211M
[INFO]
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12783694/TEZ-3066-2.patch
against master revision 92def52.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1430//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1430//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
c919ea4852c15bbd79987a655794dd0db9384296 logged out
==
==
Finished build.
==
==
Build step 'Execute shell' marked build as failure
Archiving artifacts
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
###
## FAILED TESTS (if any)
##
All tests passed
[jira] [Commented] (TEZ-3055) I got this jar is by the mvn package and run it on my cluster,occurred this problem.I can run the example which the tez given successful.But when I run this jar which is
[ https://issues.apache.org/jira/browse/TEZ-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15111778#comment-15111778 ] Lvpenglin commented on TEZ-3055: thank you very much.but when I set this property,have some error as the following.I want to ask how to set the tez program which can be run on my cluster,because I only found how to set the program run in local mode,I don't know if need some special set.Can you give some suggestion or data? Vertex failed, vertexName=ASSOCIATE, vertexId=vertex_1453425211412_0004_1_00, diagnostics= Task failed, taskId=task_1453425211412_0004_1_00_00, diagnostics= TaskAttempt 0 failed, info= Error: Failure while running task:java.lang.NullPointerException at com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor.run(TezDataGeneratorTest.java:78) at org.apache.tez.runtime.library.processor.SimpleProcessor.run(SimpleProcessor.java:53) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:337) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) TaskAttempt 1 failed, info= Error: Failure while running task:java.lang.NullPointerException at com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor.run(TezDataGeneratorTest.java:78) at org.apache.tez.runtime.library.processor.SimpleProcessor.run(SimpleProcessor.java:53) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:337) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) TaskAttempt 2 failed, info= Error: Failure while running task:java.lang.NullPointerException at com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor.run(TezDataGeneratorTest.java:78) at org.apache.tez.runtime.library.processor.SimpleProcessor.run(SimpleProcessor.java:53) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:337) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) TaskAttempt 3 failed, info= Error: Failure while running task:java.lang.NullPointerException at com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor.run(TezDataGeneratorTest.java:78) at org.apache.tez.runtime.library.processor.SimpleProcessor.run(SimpleProcessor.java:53) at
[jira] [Commented] (TEZ-3055) I got this jar is by the mvn package and run it on my cluster,occurred this problem.I can run the example which the tez given successful.But when I run this jar which is
[ https://issues.apache.org/jira/browse/TEZ-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15111860#comment-15111860 ] Lvpenglin commented on TEZ-3055: no more log can be used.where can find the all default property and description of tez-site.xml?Can you give me some site to study?thank you. > I got this jar is by the mvn package and run it on my cluster,occurred this > problem.I can run the example which the tez given successful.But when I run > this jar which is packaged by mvn packsge myself is failed. > --- > > Key: TEZ-3055 > URL: https://issues.apache.org/jira/browse/TEZ-3055 > Project: Apache Tez > Issue Type: Improvement > Environment: this is the tez orderedwordcount example,I want it can > be run on my cluster,but occurred this problem.This code is packaged by > myself.When I run the tez's example jar on my cluster,it's ok.My environment > is hadoop2.6.0+tez0.7.0.Can you give me some suggestion? >Reporter: Lvpenglin > Labels: jar, maven-package, run-example, tez > > Vertex failed, vertexName=ASSOCIATE, vertexId=vertex_1452667827400_0022_1_00, > diagnostics= > Task failed, taskId=task_1452667827400_0022_1_00_00, diagnostics= > TaskAttempt 0 failed, info= > Error: Failure while running > task:org.apache.tez.dag.api.TezUncheckedException: Unable to load class: > com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor > at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:45) > at > org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:96) > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.createProcessor(LogicalIOProcessorRuntimeTask.java:584) > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.initialize(LogicalIOProcessorRuntimeTask.java:202) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167) > at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.ClassNotFoundException: > com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:274) > at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:43) > ... 15 more > TaskAttempt 1 failed, info= > Error: Failure while running > task:org.apache.tez.dag.api.TezUncheckedException: Unable to load class: > com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor > at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:45) > at > org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:96) > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.createProcessor(LogicalIOProcessorRuntimeTask.java:584) > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.initialize(LogicalIOProcessorRuntimeTask.java:202) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171) > at
[jira] [Comment Edited] (TEZ-3055) I got this jar is by the mvn package and run it on my cluster,occurred this problem.I can run the example which the tez given successful.But when I run this jar whic
[
https://issues.apache.org/jira/browse/TEZ-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15111782#comment-15111782
]
Hitesh Shah edited comment on TEZ-3055 at 1/22/16 2:04 AM:
---
Your error seems to be the NPE in your test code:
{code}
Vertex failed, vertexName=ASSOCIATE, vertexId=vertex_1453425211412_0004_1_00,
diagnostics=
Task failed, taskId=task_1453425211412_0004_1_00_00, diagnostics=
TaskAttempt 0 failed, info=
Error: Failure while running task:java.lang.NullPointerException
at
com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor.run(TezDataGeneratorTest.java:78){code}
It seems like it is running on the cluster. You can obtain the yarn logs via
"bin/yarn logs -application application_1453425211412_0004" to debug further.
was (Author: hitesh):
Your error seems to be this:
{code}
Vertex failed, vertexName=ASSOCIATE, vertexId=vertex_1453425211412_0004_1_00,
diagnostics=
Task failed, taskId=task_1453425211412_0004_1_00_00, diagnostics=
TaskAttempt 0 failed, info=
Error: Failure while running task:java.lang.NullPointerException
at
com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor.run(TezDataGeneratorTest.java:78){code}
It seems like it is running on the cluster. You can obtain the yarn logs via
"bin/yarn logs -application application_1453425211412_0004" to debug further.
> I got this jar is by the mvn package and run it on my cluster,occurred this
> problem.I can run the example which the tez given successful.But when I run
> this jar which is packaged by mvn packsge myself is failed.
> ---
>
> Key: TEZ-3055
> URL: https://issues.apache.org/jira/browse/TEZ-3055
> Project: Apache Tez
> Issue Type: Improvement
> Environment: this is the tez orderedwordcount example,I want it can
> be run on my cluster,but occurred this problem.This code is packaged by
> myself.When I run the tez's example jar on my cluster,it's ok.My environment
> is hadoop2.6.0+tez0.7.0.Can you give me some suggestion?
>Reporter: Lvpenglin
> Labels: jar, maven-package, run-example, tez
>
> Vertex failed, vertexName=ASSOCIATE, vertexId=vertex_1452667827400_0022_1_00,
> diagnostics=
> Task failed, taskId=task_1452667827400_0022_1_00_00, diagnostics=
> TaskAttempt 0 failed, info=
> Error: Failure while running
> task:org.apache.tez.dag.api.TezUncheckedException: Unable to load class:
> com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor
> at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:45)
> at
> org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:96)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.createProcessor(LogicalIOProcessorRuntimeTask.java:584)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.initialize(LogicalIOProcessorRuntimeTask.java:202)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.ClassNotFoundException:
> com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:274)
> at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:43)
> ... 15 more
> TaskAttempt 1 failed, info=
> Error:
[jira] [Commented] (TEZ-3066) TaskAttemptFinishedEvent ConcurrentModificationException in recovery or history logging services
[ https://issues.apache.org/jira/browse/TEZ-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15111968#comment-15111968 ] Jeff Zhang commented on TEZ-3066: - [~jlowe] Thanks for reporting this issue, [~bikassaha] Thanks for review, committed to master & branch-0.7 > TaskAttemptFinishedEvent ConcurrentModificationException in recovery or > history logging services > > > Key: TEZ-3066 > URL: https://issues.apache.org/jira/browse/TEZ-3066 > Project: Apache Tez > Issue Type: Bug >Affects Versions: 0.7.0 >Reporter: Jason Lowe >Assignee: Jeff Zhang > Fix For: 0.7.1, 0.8.3 > > Attachments: TEZ-3066-1.patch, TEZ-3066-2.patch, TEZ-3066-3.patch > > > A ConcurrentModificationException can occur if a TaskAttemptFinishedEvent is > processed by the recovery service or another history logging service. Sample > stacktraces to follow. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
