[jira] [Updated] (TEZ-2521) TEZ UI: Move sorting to web worker for making the UI responsive while searching.
[ https://issues.apache.org/jira/browse/TEZ-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreenath Somarajapuram updated TEZ-2521: Description: - Right now sorting is done in the main thread, causing the browser to be stuck while sorting is happening. - Web workers help in running code on a parallel thread, and hence keeps the browser responsive. > TEZ UI: Move sorting to web worker for making the UI responsive while > searching. > > > Key: TEZ-2521 > URL: https://issues.apache.org/jira/browse/TEZ-2521 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram >Priority: Minor > > - Right now sorting is done in the main thread, causing the browser to be > stuck while sorting is happening. > - Web workers help in running code on a parallel thread, and hence keeps the > browser responsive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2974) Tez tools: TFileRecordReader in tez-tools should support reading >2 GB tfiles
[ https://issues.apache.org/jira/browse/TEZ-2974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2974: -- Attachment: TEZ-2974.1.patch Attaching .1 patch. Instead of reading entire TFile into buffer (getValue() was doing that earlier with TFile scanner), it uses getValueStream() now. This helps in reading one line at a time and converts to tuple for processing. > Tez tools: TFileRecordReader in tez-tools should support reading >2 GB tfiles > - > > Key: TEZ-2974 > URL: https://issues.apache.org/jira/browse/TEZ-2974 > Project: Apache Tez > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan > Attachments: TEZ-2974.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TEZ-2974) Tez tools: TFileRecordReader in tez-tools should support reading >2 GB tfiles
[ https://issues.apache.org/jira/browse/TEZ-2974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-2974: - Assignee: Rajesh Balamohan > Tez tools: TFileRecordReader in tez-tools should support reading >2 GB tfiles > - > > Key: TEZ-2974 > URL: https://issues.apache.org/jira/browse/TEZ-2974 > Project: Apache Tez > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2022) java.lang.IllegalStateException: Vertex: got invalid start event
[
https://issues.apache.org/jira/browse/TEZ-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044410#comment-15044410
]
Jeff Zhang commented on TEZ-2022:
-
[~hitesh] Mind to help review it ?
> java.lang.IllegalStateException: Vertex: got invalid start event
>
>
> Key: TEZ-2022
> URL: https://issues.apache.org/jira/browse/TEZ-2022
> Project: Apache Tez
> Issue Type: Bug
>Reporter: Rajesh Balamohan
>Assignee: Jeff Zhang
> Attachments: TEZ-2022-1.patch, TEZ-2022-DAG.png,
> TEZ-2022.applog.1.txt, TEZ-2022.applog.txt
>
>
> Latest Tez (with tez-2020 patch) + Pig
> When running pig with rank function, the following exception happens
> consistently
> {code}
> java.lang.IllegalStateException: Vertex: vertex_1422270854961_0113_1_03
> [scope-47] got invalid start event
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:145)
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$StartWhileInitializingTransition.transition(VertexImpl.java:3178)
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$StartWhileInitializingTransition.transition(VertexImpl.java:3170)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:362)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> at
> org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:57)
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1547)
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:181)
> at
> org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1768)
> at
> org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1754)
> at
> org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:184)
> at
> org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:116)
> at java.lang.Thread.run(Thread.java:744)
> {code}
> {code}
> rawLogs = load '/tmp/logs/root/logs/application_1422270854961_0093/' using
> org.apache.tez.tools.TFileLoader() as (machine:chararray, key:chararray,
> line:chararray);
> raw = FOREACH rawLogs GENERATE TRIM(REGEX_EXTRACT(machine, '(.*)_(\\d+)', 1))
> as machine, key, line;
> machines = FOREACH raw GENERATE machine;
> distinctMachines = DISTINCT machines;
> sortByMachines = ORDER distinctMachines BY machine;
> ranked = RANK sortByMachines;
> dump ranked;
> {code}
> Will attach the DAG and applog asap.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (TEZ-2427) TestFaultTolerance NPE in RecoveryService
[ https://issues.apache.org/jira/browse/TEZ-2427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang resolved TEZ-2427. - Resolution: Won't Fix Close it for new since didn't see it in the recent build. Could open if see it again > TestFaultTolerance NPE in RecoveryService > - > > Key: TEZ-2427 > URL: https://issues.apache.org/jira/browse/TEZ-2427 > Project: Apache Tez > Issue Type: Bug >Reporter: Hitesh Shah >Assignee: Jeff Zhang >Priority: Critical > > From https://builds.apache.org/job/Tez-Build/1055/ > 2015-05-06 23:55:54,488 ERROR [Dispatcher thread: Central] > common.AsyncDispatcher: Error in dispatcher thread > java.lang.NullPointerException > at > org.apache.tez.dag.history.recovery.RecoveryService.doFlush(RecoveryService.java:458) > at > org.apache.tez.dag.history.recovery.RecoveryService.handle(RecoveryService.java:289) > at > org.apache.tez.dag.history.HistoryEventHandler.handleCriticalEvent(HistoryEventHandler.java:102) > at > org.apache.tez.dag.app.dag.impl.DAGImpl.logJobHistoryUnsuccesfulEvent(DAGImpl.java:1161) > at org.apache.tez.dag.app.dag.impl.DAGImpl.finished(DAGImpl.java:1275) > at org.apache.tez.dag.app.dag.impl.DAGImpl.access$2600(DAGImpl.java:144) > at > org.apache.tez.dag.app.dag.impl.DAGImpl$InternalErrorTransition.transition(DAGImpl.java:2151) > at > org.apache.tez.dag.app.dag.impl.DAGImpl$InternalErrorTransition.transition(DAGImpl.java:2140) > at > org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:362) > at > org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302) > at > org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46) > at > org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448) > at > org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:57) > at org.apache.tez.dag.app.dag.impl.DAGImpl.handle(DAGImpl.java:1079) > at org.apache.tez.dag.app.dag.impl.DAGImpl.handle(DAGImpl.java:143) > at > org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher.handle(DAGAppMaster.java:1871) > at > org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher.handle(DAGAppMaster.java:1862) > at > org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:183) > at org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:114) > at java.lang.Thread.run(Thread.java:662) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2113) Tez UI: show warning symbol for dags/vertices with failed task attempts
[ https://issues.apache.org/jira/browse/TEZ-2113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2113: Assignee: Sreenath Somarajapuram (was: Prakash Ramachandran) > Tez UI: show warning symbol for dags/vertices with failed task attempts > --- > > Key: TEZ-2113 > URL: https://issues.apache.org/jira/browse/TEZ-2113 > Project: Apache Tez > Issue Type: Bug >Reporter: Prakash Ramachandran >Assignee: Sreenath Somarajapuram > > The current implementation shows a warning sign for successful dags which > have failed task attempts however its not consistent across dag, vertex etc. > also showing a warning sign if just one task attempt failed may be a false > warning and will tend to be ignored over time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-987) Need a way to measure container reuse
[ https://issues.apache.org/jira/browse/TEZ-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044411#comment-15044411 ] Jeff Zhang commented on TEZ-987: Move to 0.7.2 > Need a way to measure container reuse > - > > Key: TEZ-987 > URL: https://issues.apache.org/jira/browse/TEZ-987 > Project: Apache Tez > Issue Type: New Feature >Reporter: Rohini Palaniswamy > > It is very important to know about the amount of container reuse to measure > the effectiveness of Tez. Currently nothing is available (AM API, counters, > etc) to get those stats programatically. Only way is probably to scrape the > AM logs (haven't tried it though). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-987) Need a way to measure container reuse
[ https://issues.apache.org/jira/browse/TEZ-987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-987: --- Target Version/s: 0.7.2 (was: 0.7.1) > Need a way to measure container reuse > - > > Key: TEZ-987 > URL: https://issues.apache.org/jira/browse/TEZ-987 > Project: Apache Tez > Issue Type: New Feature >Reporter: Rohini Palaniswamy > > It is very important to know about the amount of container reuse to measure > the effectiveness of Tez. Currently nothing is available (AM API, counters, > etc) to get those stats programatically. Only way is probably to scrape the > AM logs (haven't tried it though). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2439) Should not throw exception in DAGClientHandler#getVertexStatus when DAG is not initialized
[
https://issues.apache.org/jira/browse/TEZ-2439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Zhang updated TEZ-2439:
Target Version/s: 0.7.2 (was: 0.7.1)
> Should not throw exception in DAGClientHandler#getVertexStatus when DAG is
> not initialized
> --
>
> Key: TEZ-2439
> URL: https://issues.apache.org/jira/browse/TEZ-2439
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Jeff Zhang
>Assignee: Jeff Zhang
> Attachments: TEZ-2439-1.patch
>
>
> It would be possible that dag is not initialized when
> DAGClientHandler#getVertexStatus is invoked so that
> DAGClientHandler#getVertexStatus would throw exception.
> {code}
> 2015-05-10 04:29:26,554 INFO [IPC Server handler 0 on 51881] ipc.Server: IPC
> Server handler 0 on 51881, call
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB.getVertexStatus
> from 10.79.246.151:53112 Call#188 Retry#0^M
> org.apache.tez.dag.api.TezException: Unknown vertexName: initialmap^M
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getVertexStatus(DAGClientHandler.java:75)^M
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.getVertexStatus(DAGClientAMProtocolBlockingPBServerImpl.java:121)^M
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7467)^M
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)^M
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)^M
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)^M
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)^M
> at java.security.AccessController.doPrivileged(Native Method)^M
> at javax.security.auth.Subject.doAs(Subject.java:415)^M
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)^M
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)^M
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Failed: TEZ-2439 PreCommit Build #1352
Jira: https://issues.apache.org/jira/browse/TEZ-2439
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1352/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 3482 lines...]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] [Help 2]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :tez-history-parser
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12733434/TEZ-2439-1.patch
against master revision f237586.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.history.TestHistoryParser
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1352//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1352//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
8534b32f891bb6c61643abf0326e6d811de4df9d logged out
==
==
Finished build.
==
==
Build step 'Execute shell' marked build as failure
Archiving artifacts
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
###
## FAILED TESTS (if any)
##
2 tests failed.
FAILED: org.apache.tez.history.TestHistoryParser.testParserWithFailedJob
Error Message:
null
Stack Trace:
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.tez.history.TestHistoryParser.verifyDagInfo(TestHistoryParser.java:697)
at
org.apache.tez.history.TestHistoryParser.testParserWithFailedJob(TestHistoryParser.java:372)
FAILED: org.apache.tez.history.TestHistoryParser.testParserWithSuccessfulJob
Error Message:
null
Stack Trace:
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.tez.history.TestHistoryParser.verifyDagInfo(TestHistoryParser.java:698)
at
org.apache.tez.history.TestHistoryParser.testParserWithSuccessfulJob(TestHistoryParser.java:209)
[jira] [Commented] (TEZ-2439) Should not throw exception in DAGClientHandler#getVertexStatus when DAG is not initialized
[
https://issues.apache.org/jira/browse/TEZ-2439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044510#comment-15044510
]
TezQA commented on TEZ-2439:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12733434/TEZ-2439-1.patch
against master revision f237586.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.history.TestHistoryParser
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1352//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1352//console
This message is automatically generated.
> Should not throw exception in DAGClientHandler#getVertexStatus when DAG is
> not initialized
> --
>
> Key: TEZ-2439
> URL: https://issues.apache.org/jira/browse/TEZ-2439
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Jeff Zhang
>Assignee: Jeff Zhang
> Attachments: TEZ-2439-1.patch
>
>
> It would be possible that dag is not initialized when
> DAGClientHandler#getVertexStatus is invoked so that
> DAGClientHandler#getVertexStatus would throw exception.
> {code}
> 2015-05-10 04:29:26,554 INFO [IPC Server handler 0 on 51881] ipc.Server: IPC
> Server handler 0 on 51881, call
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB.getVertexStatus
> from 10.79.246.151:53112 Call#188 Retry#0^M
> org.apache.tez.dag.api.TezException: Unknown vertexName: initialmap^M
> at
> org.apache.tez.dag.api.client.DAGClientHandler.getVertexStatus(DAGClientHandler.java:75)^M
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.getVertexStatus(DAGClientAMProtocolBlockingPBServerImpl.java:121)^M
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7467)^M
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)^M
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)^M
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)^M
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)^M
> at java.security.AccessController.doPrivileged(Native Method)^M
> at javax.security.auth.Subject.doAs(Subject.java:415)^M
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)^M
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)^M
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (TEZ-2188) Verify vertex with bipartite source vertex and root input in client side
[
https://issues.apache.org/jira/browse/TEZ-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Zhang updated TEZ-2188:
Target Version/s: 0.7.2 (was: 0.7.1)
> Verify vertex with bipartite source vertex and root input in client side
>
>
> Key: TEZ-2188
> URL: https://issues.apache.org/jira/browse/TEZ-2188
> Project: Apache Tez
> Issue Type: Bug
>Reporter: Jeff Zhang
>Assignee: Jeff Zhang
> Attachments: TEZ-2188-1.patch, TEZ-2188-2.patch
>
>
> For this kind of case that vertex with bipartite source vertex and root
> input, there's no clear diagnosis message in client side, should do the
> verification in client side.
> {code}
> 16:31:57,333 - Thread( main) - (DAGClientImpl.java:541) - Waiting for DAG to
> start running
> 16:32:00,455 - Thread( main) - (DAGClientImpl.java:541) - DAG initialized:
> CurrentState=Running
> 16:32:00,977 - Thread( main) - (DAGClientImpl.java:541) - DAG: State: FAILED
> Progress: 0% TotalTasks: 1 Succeeded: 0 Running: 0 Failed: 0 Killed: 2
> 16:32:00,978 - Thread( main) - (DAGClientImpl.java:541) - DAG completed.
> FinalState=FAILED
> 16:32:00,978 - Thread( main) - (TezExampleBase.java:137) - DAG diagnostics:
> [Vertex failed, vertexName=v2, vertexId=vertex_142588246_0025_1_01,
> diagnostics=[Vertex vertex_142588246_0025_1_01 [v2] killed/failed due
> to:null], Vertex killed, vertexName=v1,
> vertexId=vertex_142588246_0025_1_00, diagnostics=[Vertex received Kill
> while in RUNNING state., Vertex killed as other vertex failed. failedTasks:0,
> Vertex vertex_142588246_0025_1_00 [v1] killed/failed due to:null], DAG
> failed due to vertex failure. failedVertices:1 killedVertices:1]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (TEZ-2430) Add test for RecoveryEvent Spec
[ https://issues.apache.org/jira/browse/TEZ-2430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang resolved TEZ-2430. - Resolution: Won't Fix Resolved in TEZ-2581 > Add test for RecoveryEvent Spec > --- > > Key: TEZ-2430 > URL: https://issues.apache.org/jira/browse/TEZ-2430 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Jeff Zhang >Assignee: Jeff Zhang > > * Ordering of RecoveryEvents. > ** DataMovementEvent must be logged before TaskAttemptFinishedEvent > ** InputDataInfoEvent must be logged before VertexInitializedEvent (already > covered in the existing test) > * Frequency of RecoveryEvent. e.g. TaskAttemptStartedEvent can only been > logged once, but TaskAttemptFinishedEvent can been logged twice. (TaskAttempt > transit from SUCCEEDED to FAILED) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Failed: TEZ-2974 PreCommit Build #1351
Jira: https://issues.apache.org/jira/browse/TEZ-2974
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1351/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 3483 lines...]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] [Help 2]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :tez-history-parser
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12776036/TEZ-2974.1.patch
against master revision f237586.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.history.TestHistoryParser
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1351//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1351//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
3729c448e7bda3609050c98077176ab2dbaa2342 logged out
==
==
Finished build.
==
==
Build step 'Execute shell' marked build as failure
Archiving artifacts
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
###
## FAILED TESTS (if any)
##
2 tests failed.
FAILED: org.apache.tez.history.TestHistoryParser.testParserWithFailedJob
Error Message:
null
Stack Trace:
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.tez.history.TestHistoryParser.verifyDagInfo(TestHistoryParser.java:697)
at
org.apache.tez.history.TestHistoryParser.testParserWithFailedJob(TestHistoryParser.java:372)
FAILED: org.apache.tez.history.TestHistoryParser.testParserWithSuccessfulJob
Error Message:
null
Stack Trace:
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.tez.history.TestHistoryParser.verifyDagInfo(TestHistoryParser.java:698)
at
org.apache.tez.history.TestHistoryParser.testParserWithSuccessfulJob(TestHistoryParser.java:209)
[jira] [Commented] (TEZ-2974) Tez tools: TFileRecordReader in tez-tools should support reading >2 GB tfiles
[
https://issues.apache.org/jira/browse/TEZ-2974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044466#comment-15044466
]
TezQA commented on TEZ-2974:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12776036/TEZ-2974.1.patch
against master revision f237586.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.history.TestHistoryParser
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1351//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1351//console
This message is automatically generated.
> Tez tools: TFileRecordReader in tez-tools should support reading >2 GB tfiles
> -
>
> Key: TEZ-2974
> URL: https://issues.apache.org/jira/browse/TEZ-2974
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
> Attachments: TEZ-2974.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (TEZ-2974) Tez tools: TFileRecordReader in tez-tools should support reading >2 GB tfiles
Rajesh Balamohan created TEZ-2974: - Summary: Tez tools: TFileRecordReader in tez-tools should support reading >2 GB tfiles Key: TEZ-2974 URL: https://issues.apache.org/jira/browse/TEZ-2974 Project: Apache Tez Issue Type: Improvement Reporter: Rajesh Balamohan -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2249) Wait for all task attempt finished before moving Task to finished state
[ https://issues.apache.org/jira/browse/TEZ-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2249: Target Version/s: 0.7.2 (was: 0.7.1) > Wait for all task attempt finished before moving Task to finished state > --- > > Key: TEZ-2249 > URL: https://issues.apache.org/jira/browse/TEZ-2249 > Project: Apache Tez > Issue Type: Bug >Reporter: Jeff Zhang >Assignee: Jeff Zhang > Attachments: TEZ-2249-1.patch > > > 2 cases: > * If Task needs to move the SUCCEEDED, then committing may happens while > there's still task attempt running. > * If Tasks needs to move to FAILED/KILLED/ERROD, then aborting may happens > while there's still task attempt running. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2824) Add javadocs for Vertex.setConf and DAG.setConf
[ https://issues.apache.org/jira/browse/TEZ-2824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044415#comment-15044415 ] Jeff Zhang commented on TEZ-2824: - ping [~sseth] > Add javadocs for Vertex.setConf and DAG.setConf > --- > > Key: TEZ-2824 > URL: https://issues.apache.org/jira/browse/TEZ-2824 > Project: Apache Tez > Issue Type: Task >Affects Versions: 0.7.0, 0.8.0-alpha >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: TEZ-2824.1.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2413) No need to check whether vertex has been committed as shared output in DAGImpl#vertexreRunning
[
https://issues.apache.org/jira/browse/TEZ-2413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Zhang updated TEZ-2413:
Target Version/s: 0.7.2 (was: 0.7.1)
> No need to check whether vertex has been committed as shared output in
> DAGImpl#vertexreRunning
> --
>
> Key: TEZ-2413
> URL: https://issues.apache.org/jira/browse/TEZ-2413
> Project: Apache Tez
> Issue Type: Bug
>Reporter: Jeff Zhang
>Assignee: Jeff Zhang
> Fix For: 0.7.1
>
>
> * Vertex only send out DAGEventVertexReRunning when
> vertex.commitVertexOutputs is false, that means it is not necessary to check
> whether it has been committed as shared output of vertex group in
> DAGImpl#vertexReRunning
> {code}
> private boolean vertexReRunning(Vertex vertex) {
> reRunningVertices.add(vertex.getVertexId());
> numSuccessfulVertices--;
> addDiagnostic("Vertex re-running"
> + ", vertexName=" + vertex.getName()
> + ", vertexId=" + vertex.getVertexId());
> if (!commitAllOutputsOnSuccess) {
> // partial output may already have been committed. fail if so
> List groupList = vertexGroupInfo.get(vertex.getName());
> if (groupList != null) {
> for (VertexGroupInfo groupInfo : groupList) {
> if (groupInfo.committed) {
> String msg = "Aborting job as committed vertex: "
> + vertex.getLogIdentifier() + " is re-running";
> LOG.info(msg);
> addDiagnostic(msg);
> enactKill(DAGTerminationCause.VERTEX_RERUN_AFTER_COMMIT,
> VertexTerminationCause.VERTEX_RERUN_AFTER_COMMIT);
> return true;
> } else {
> groupInfo.successfulMembers--;
> }
> }
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2824) Add javadocs for Vertex.setConf and DAG.setConf
[
https://issues.apache.org/jira/browse/TEZ-2824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044453#comment-15044453
]
TezQA commented on TEZ-2824:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12755790/TEZ-2824.1.txt
against master revision f237586.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.client.TestTezClient
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1353//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1353//console
This message is automatically generated.
> Add javadocs for Vertex.setConf and DAG.setConf
> ---
>
> Key: TEZ-2824
> URL: https://issues.apache.org/jira/browse/TEZ-2824
> Project: Apache Tez
> Issue Type: Task
>Affects Versions: 0.7.0, 0.8.0-alpha
>Reporter: Siddharth Seth
>Assignee: Siddharth Seth
> Attachments: TEZ-2824.1.txt
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Failed: TEZ-2824 PreCommit Build #1353
Jira: https://issues.apache.org/jira/browse/TEZ-2824
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1353/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 2508 lines...]
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :tez-api
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12755790/TEZ-2824.1.txt
against master revision f237586.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.client.TestTezClient
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1353//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1353//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
60b92311d6c173a12b2757f4d94c497d1236731e logged out
==
==
Finished build.
==
==
Build step 'Execute shell' marked build as failure
Archiving artifacts
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
###
## FAILED TESTS (if any)
##
1 tests failed.
FAILED: org.apache.tez.client.TestTezClient.testStopRetriesUntilTimeout
Error Message:
test timed out after 5000 milliseconds
Stack Trace:
java.lang.Exception: test timed out after 5000 milliseconds
at java.lang.Thread.sleep(Native Method)
at org.apache.tez.client.TezClient.stop(TezClient.java:589)
at
org.apache.tez.client.TestTezClient.testStopRetriesUntilTimeout(TestTezClient.java:557)
[jira] [Commented] (TEZ-2492) Tez UI: mouse events freeze workaround followup
[ https://issues.apache.org/jira/browse/TEZ-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044408#comment-15044408 ] Jeff Zhang commented on TEZ-2492: - Move to 0.7.2 > Tez UI: mouse events freeze workaround followup > --- > > Key: TEZ-2492 > URL: https://issues.apache.org/jira/browse/TEZ-2492 > Project: Apache Tez > Issue Type: Bug > Components: UI >Reporter: Prakash Ramachandran >Assignee: Sreenath Somarajapuram >Priority: Trivial > > Tez-2482 works around a issue in IE11 where clicking on an SVG element causes > the browser to stop responding to mouse events. This is a followup bug to fix > the issue without the extra dialog for IE11. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2492) Tez UI: mouse events freeze workaround followup
[ https://issues.apache.org/jira/browse/TEZ-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-2492: Target Version/s: 0.7.2 (was: 0.7.1) > Tez UI: mouse events freeze workaround followup > --- > > Key: TEZ-2492 > URL: https://issues.apache.org/jira/browse/TEZ-2492 > Project: Apache Tez > Issue Type: Bug > Components: UI >Reporter: Prakash Ramachandran >Assignee: Sreenath Somarajapuram >Priority: Trivial > > Tez-2482 works around a issue in IE11 where clicking on an SVG element causes > the browser to stop responding to mouse events. This is a followup bug to fix > the issue without the extra dialog for IE11. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
