[jira] [Commented] (TEZ-2958) Recovered TA, whose commit cannot be recovered, should move to killed state
[ https://issues.apache.org/jira/browse/TEZ-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15198653#comment-15198653 ] Jeff Zhang commented on TEZ-2958: - Thanks [~jlowe] +1 > Recovered TA, whose commit cannot be recovered, should move to killed state > --- > > Key: TEZ-2958 > URL: https://issues.apache.org/jira/browse/TEZ-2958 > Project: Apache Tez > Issue Type: Sub-task >Affects Versions: 0.7.0 >Reporter: Bikas Saha >Assignee: Jason Lowe > Attachments: TEZ-2958-branch-0.7.001.patch, TEZ-2958.001.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TEZ-3169) Counters should be updated for interrupted task
Zhiyuan Yang created TEZ-3169: - Summary: Counters should be updated for interrupted task Key: TEZ-3169 URL: https://issues.apache.org/jira/browse/TEZ-3169 Project: Apache Tez Issue Type: Bug Reporter: Zhiyuan Yang On master branch, if a task gets interrupted very shortly because container stop is requested, counters may not be fully updated. We should publish all counters for them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TEZ-3171) Tez UI: Swimlane - Tooltip, zoom & redirection
Sreenath Somarajapuram created TEZ-3171: --- Summary: Tez UI: Swimlane - Tooltip, zoom & redirection Key: TEZ-3171 URL: https://issues.apache.org/jira/browse/TEZ-3171 Project: Apache Tez Issue Type: Sub-task Reporter: Sreenath Somarajapuram - Implement tooltip -- Create a tooltip component with dynamic (Update at realtime) data display --- Create tooltip definition class - in line with tabel definition to use column definition classes --- Create tooltip component - Must display title, text & a list of values --- Attach tooltip to entities displayed --- When multiple event bubbles come together, the tooltip must display all of them - Ensure the data displayed can be changed with the column selector - Ensure the selector window have a meaningful title - Implement horizontal zoom & scroll -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2958) Recovered TA, whose commit cannot be recovered, should move to killed state
[
https://issues.apache.org/jira/browse/TEZ-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15197216#comment-15197216
]
TezQA commented on TEZ-2958:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12793631/TEZ-2958.001.patch
against master revision 42b61f4.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1565//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1565//console
This message is automatically generated.
> Recovered TA, whose commit cannot be recovered, should move to killed state
> ---
>
> Key: TEZ-2958
> URL: https://issues.apache.org/jira/browse/TEZ-2958
> Project: Apache Tez
> Issue Type: Sub-task
>Affects Versions: 0.7.0
>Reporter: Bikas Saha
>Assignee: Jason Lowe
> Attachments: TEZ-2958.001.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2967) Vertex start time should be that of first task start time in UI
[ https://issues.apache.org/jira/browse/TEZ-2967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15202066#comment-15202066 ] Rohini Palaniswamy commented on TEZ-2967: - +1. [~jeagles], Can we still leave this jira open after committing the fix to branch 0.7, so that [~Sreenath] makes the equivalent fix for the new UI? > Vertex start time should be that of first task start time in UI > --- > > Key: TEZ-2967 > URL: https://issues.apache.org/jira/browse/TEZ-2967 > Project: Apache Tez > Issue Type: Bug > Components: UI >Reporter: Rohini Palaniswamy >Assignee: Sreenath Somarajapuram > Attachments: TEZ-2967.1.patch > > > Currently it shows the vertex initialization time which is hardly of any > use. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Success: TEZ-3166 PreCommit Build #1572
Jira: https://issues.apache.org/jira/browse/TEZ-3166
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1572/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 4585 lines...]
[INFO] Tez ... SUCCESS [ 0.034 s]
[INFO]
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 55:13 min
[INFO] Finished at: 2016-03-17T20:38:27+00:00
[INFO] Final Memory: 74M/1015M
[INFO]
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12793871/TEZ-3166.3.patch
against master revision 191447e.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1572//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1572//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
aee74873cec4c18b34857c8fbfa0b1cb8b60e601 logged out
==
==
Finished build.
==
==
Archiving artifacts
[description-setter] Description set: TEZ-3166
Recording test results
Email was triggered for: Success
Sending email for trigger: Success
###
## FAILED TESTS (if any)
##
All tests passed
[jira] [Commented] (TEZ-3175) Add tez submit host
[
https://issues.apache.org/jira/browse/TEZ-3175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15200808#comment-15200808
]
TezQA commented on TEZ-3175:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12794085/TEZ-3175.1.patch
against master revision 191447e.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1574//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1574//console
This message is automatically generated.
> Add tez submit host
> ---
>
> Key: TEZ-3175
> URL: https://issues.apache.org/jira/browse/TEZ-3175
> Project: Apache Tez
> Issue Type: Bug
>Reporter: Jonathan Eagles
>Assignee: Jonathan Eagles
> Attachments: TEZ-3175.1.patch
>
>
> JobSubmitter sets MRJobConfig JOB_SUBMITHOST and JOB_SUBMITHOSTADDR in the
> client when submitting a job.
> Tez should log at least the submit host in the am config so that this detail
> can be tracked and job debugging can be less user intensive.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (TEZ-3173) Tez : Update Tez AM REST APIs to send back realtime events
[ https://issues.apache.org/jira/browse/TEZ-3173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreenath Somarajapuram updated TEZ-3173: Description: `/ui/ws/v2/tez/verticesInfo` endpoint in WebUIService must be upgraded to send back the following fields # startTime # endTime # firstTaskStartTime # lastTaskFinishTime # events in addition to the existing id, status, progress, totalTasks, succeededTasks, runningTasks, pendingTasks, failedTaskAttempts & killedTaskAttempts fields. was: `/ui/ws/v2/tez/verticesInfo` endpoint in WebUIService must be upgraded to send back the following fields #. startTime #. endTime #. firstTaskStartTime #. lastTaskFinishTime #. events in addition to the existing id, status, progress, totalTasks, succeededTasks, runningTasks, pendingTasks, failedTaskAttempts & killedTaskAttempts fields. > Tez : Update Tez AM REST APIs to send back realtime events > -- > > Key: TEZ-3173 > URL: https://issues.apache.org/jira/browse/TEZ-3173 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Sreenath Somarajapuram > > `/ui/ws/v2/tez/verticesInfo` endpoint in WebUIService must be upgraded to > send back the following fields > # startTime > # endTime > # firstTaskStartTime > # lastTaskFinishTime > # events > in addition to the existing id, status, progress, totalTasks, > succeededTasks, runningTasks, pendingTasks, failedTaskAttempts & > killedTaskAttempts fields. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-3168) Provide a more predictable approach for total resource guidance for wave/split calculation
[ https://issues.apache.org/jira/browse/TEZ-3168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15200521#comment-15200521 ] Jason Lowe commented on TEZ-3168: - bq. Do you know if headroom factors in the user limits ? The reported headroom from the RM does factor in user limits. > Provide a more predictable approach for total resource guidance for > wave/split calculation > --- > > Key: TEZ-3168 > URL: https://issues.apache.org/jira/browse/TEZ-3168 > Project: Apache Tez > Issue Type: Bug >Reporter: Hitesh Shah >Assignee: Hitesh Shah > Attachments: TEZ-3168.wip.patch > > > Currently, Tez uses headroom for checking total available resources. This is > flaky as it ends up causing the split count to be determined by a point in > time lookup at what is available in the cluster. A better approach would be > either the queue size or even cluster size to get a more predictable count. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (TEZ-3173) Update Tez AM REST APIs to send back realtime events
[ https://issues.apache.org/jira/browse/TEZ-3173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15198560#comment-15198560 ] Hitesh Shah edited comment on TEZ-3173 at 3/17/16 1:32 AM: --- What events are being referred to in (5)? was (Author: hitesh): What events are being referred to? > Update Tez AM REST APIs to send back realtime events > > > Key: TEZ-3173 > URL: https://issues.apache.org/jira/browse/TEZ-3173 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Sreenath Somarajapuram > > `/ui/ws/v2/tez/verticesInfo` endpoint in WebUIService must be upgraded to > send back the following fields > # startTime > # endTime > # firstTaskStartTime > # lastTaskFinishTime > # events > in addition to the existing id, status, progress, totalTasks, > succeededTasks, runningTasks, pendingTasks, failedTaskAttempts & > killedTaskAttempts fields. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-3166) Counters aren't fully updated and sent for failed tasks
[ https://issues.apache.org/jira/browse/TEZ-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15200700#comment-15200700 ] Siddharth Seth commented on TEZ-3166: - +1. Committing. Thanks [~aplusplus]. I'm going to create a follow up to potentially handle updates for a partly initialized task. > Counters aren't fully updated and sent for failed tasks > --- > > Key: TEZ-3166 > URL: https://issues.apache.org/jira/browse/TEZ-3166 > Project: Apache Tez > Issue Type: Bug >Reporter: Zhiyuan Yang >Assignee: Zhiyuan Yang > Attachments: TEZ-3166.1.patch, TEZ-3166.2.patch, TEZ-3166.3.patch, > TEZ-3166.branch-0.7.patch, TEZ-3166.branch-0.7.second.patch > > > On master branch, if an application fails very soon, counters aren't fully > updated so on tez-ui we can only see part of counters. We should publish all > counters even task attempt fails. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-3175) Add tez submit host
[ https://issues.apache.org/jira/browse/TEZ-3175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15200764#comment-15200764 ] Hitesh Shah commented on TEZ-3175: -- Minor nits: - typo in "The host name of the was unable to be retrieved" - "The host name of the tez client the dag was submitted from." - should probably indicate where the Tez App was submitted from as we are not tracking this on a per dag basis. Likewise for the host address. - Although this is not really a configuration property, adding the annotation and marking it Private will make it visible in the documentation. I am not too sure what makes sense here - whether to add the annotation or not as this will never be set by users - so this change is optional if it makes sense to you. +1 once above and canonicalhostname changes are addressed. > Add tez submit host > --- > > Key: TEZ-3175 > URL: https://issues.apache.org/jira/browse/TEZ-3175 > Project: Apache Tez > Issue Type: Bug >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > Attachments: TEZ-3175.1.patch > > > JobSubmitter sets MRJobConfig JOB_SUBMITHOST and JOB_SUBMITHOSTADDR in the > client when submitting a job. > Tez should log at least the submit host in the am config so that this detail > can be tracked and job debugging can be less user intensive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-3175) Add tez client submit host
[ https://issues.apache.org/jira/browse/TEZ-3175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3175: - Summary: Add tez client submit host (was: Add tez submit host) > Add tez client submit host > -- > > Key: TEZ-3175 > URL: https://issues.apache.org/jira/browse/TEZ-3175 > Project: Apache Tez > Issue Type: Bug >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > Attachments: TEZ-3175.1.patch, TEZ-3175.2-branch-0.7.patch, > TEZ-3175.2.patch > > > JobSubmitter sets MRJobConfig JOB_SUBMITHOST and JOB_SUBMITHOSTADDR in the > client when submitting a job. > Tez should log at least the submit host in the am config so that this detail > can be tracked and job debugging can be less user intensive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TEZ-3177) Non-DAG events should use the session domain or no domain if the data does not need protection
Hitesh Shah created TEZ-3177: Summary: Non-DAG events should use the session domain or no domain if the data does not need protection Key: TEZ-3177 URL: https://issues.apache.org/jira/browse/TEZ-3177 Project: Apache Tez Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah There have been issues noticed where when using dag specific domains, container events get generated under different dags causing issues as they are updated using different domains. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-3170) Tez UI: Swimlane - Display computed events, event bars & dependencies
[ https://issues.apache.org/jira/browse/TEZ-3170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreenath Somarajapuram updated TEZ-3170: Attachment: TEZ-3170.1.patch [~hitesh] Please review. > Tez UI: Swimlane - Display computed events, event bars & dependencies > - > > Key: TEZ-3170 > URL: https://issues.apache.org/jira/browse/TEZ-3170 > Project: Apache Tez > Issue Type: Sub-task >Affects Versions: 0.8.3 >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Attachments: TEZ-3170.1.patch > > > - Add computed events > -- Add firstTaskStartTime > -- Add lastTaskFinishTime > -- Add dependent vertices complete event. > - All the new events lines are displayed in the UI > - Add event bars into the swimlane > -- Draw the events bars using the computed events > - Implement Auto/manual colouring > - Show dependency -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-3173) Update Tez AM REST APIs to send back realtime events
[ https://issues.apache.org/jira/browse/TEZ-3173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15198568#comment-15198568 ] Sreenath Somarajapuram commented on TEZ-3173: - The events array property that comes from ATS. By default we get VERTEX_INITIALIZED, VERTEX_STARTED & VERTEX_FINISHED. But the UI is flexible enough to take in any number of events. Hence events like VERTEX_PARALLELISM_UPDATED can also be send when available. > Update Tez AM REST APIs to send back realtime events > > > Key: TEZ-3173 > URL: https://issues.apache.org/jira/browse/TEZ-3173 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Sreenath Somarajapuram > > `/ui/ws/v2/tez/verticesInfo` endpoint in WebUIService must be upgraded to > send back the following fields > # startTime > # endTime > # firstTaskStartTime > # lastTaskFinishTime > # events > in addition to the existing id, status, progress, totalTasks, > succeededTasks, runningTasks, pendingTasks, failedTaskAttempts & > killedTaskAttempts fields. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
