[jira] [Assigned] (TEZ-3999) Extend VertexManagerPlugin interface to allow for relevant events notification

2018-10-05 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yingda Chen reassigned TEZ-3999: Assignee: Yingda Chen > Extend VertexManagerPlugin interface to allow for relevant events

[jira] [Assigned] (TEZ-3997) Enable CONCURRENT edge

2018-10-05 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yingda Chen reassigned TEZ-3997: Assignee: Yingda Chen > Enable CONCURRENT edge > -- > > Key:

[jira] [Assigned] (TEZ-3998) Allow CONCURRENT edge property in DAG construction and introduce ConcurrentSchedulingType

2018-10-05 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yingda Chen reassigned TEZ-3998: Assignee: Yingda Chen > Allow CONCURRENT edge property in DAG construction and introduce >

[jira] [Created] (TEZ-4000) Enable downstream vertex connecting to an EPHEMERAL data source, to reason about network connections of upstream tasks.

2018-10-05 Thread Yingda Chen (JIRA)
Yingda Chen created TEZ-4000: Summary: Enable downstream vertex connecting to an EPHEMERAL data source, to reason about network connections of upstream tasks. Key: TEZ-4000 URL:

[jira] [Created] (TEZ-3998) Allow CONCURRENT edge property in DAG construction and introduce ConcurrentSchedulingType

2018-10-05 Thread Yingda Chen (JIRA)
Yingda Chen created TEZ-3998: Summary: Allow CONCURRENT edge property in DAG construction and introduce ConcurrentSchedulingType Key: TEZ-3998 URL: https://issues.apache.org/jira/browse/TEZ-3998 Project:

[jira] [Updated] (TEZ-3999) Extend VertexManagerPlugin interface to allow for relevant events notification

2018-10-05 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yingda Chen updated TEZ-3999: - External issue ID: TEZ-3997 > Extend VertexManagerPlugin interface to allow for relevant events

[jira] [Created] (TEZ-4001) Allow mixture of CONCURRENT/SEQUENTIAL incoming edges on the same vertex

2018-10-05 Thread Yingda Chen (JIRA)
Yingda Chen created TEZ-4001: Summary: Allow mixture of CONCURRENT/SEQUENTIAL incoming edges on the same vertex Key: TEZ-4001 URL: https://issues.apache.org/jira/browse/TEZ-4001 Project: Apache Tez

[jira] [Created] (TEZ-3997) Enable CONCURRENT edge

2018-10-05 Thread Yingda Chen (JIRA)
Yingda Chen created TEZ-3997: Summary: Enable CONCURRENT edge Key: TEZ-3997 URL: https://issues.apache.org/jira/browse/TEZ-3997 Project: Apache Tez Issue Type: New Feature Reporter:

[jira] [Created] (TEZ-3999) Extend VertexManagerPlugin interface to allow for relevant events notification

2018-10-05 Thread Yingda Chen (JIRA)
Yingda Chen created TEZ-3999: Summary: Extend VertexManagerPlugin interface to allow for relevant events notification Key: TEZ-3999 URL: https://issues.apache.org/jira/browse/TEZ-3999 Project: Apache Tez

[jira] [Updated] (TEZ-3997) Enable CONCURRENT edge

2018-10-12 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yingda Chen updated TEZ-3997: - Description: A better formatted (and commentable) google doc with figures can be found at 

[jira] [Commented] (TEZ-2222) Investigate moving to log4j2 for logging

2018-10-16 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16652896#comment-16652896 ] Yingda Chen commented on TEZ-: -- I am wondering if this is still being considered? log4j2 provides

[jira] [Commented] (TEZ-4010) Use InputReadyVertexManager for broadcast connection

2018-10-17 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16653168#comment-16653168 ] Yingda Chen commented on TEZ-4010: -- +1 agreed with replacing  ImmediateStartVertexManager with

[jira] [Assigned] (TEZ-3075) Revamp bad node handling

2018-10-16 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yingda Chen reassigned TEZ-3075: Assignee: Ying Han > Revamp bad node handling > > > Key:

[jira] [Commented] (TEZ-3075) Revamp bad node handling

2018-10-16 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16651765#comment-16651765 ] Yingda Chen commented on TEZ-3075: -- [~Chyler] and I will be looking at this together with TEZ-3822 >

[jira] [Updated] (TEZ-4017) Support proper AM failover on DAG with concurrent edge

2018-10-31 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yingda Chen updated TEZ-4017: - Description: This is an umbrella task for feature TEZ-3997. Initial support for concurrent edge in

[jira] [Updated] (TEZ-3997) Enable CONCURRENT edge

2018-10-31 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yingda Chen updated TEZ-3997: - External issue ID: TEZ-3998, TEZ-3999, TEZ-4000, TEZ-4001, TEZ-4017 (was: TEZ-3998, TEZ-3999, TEZ-4000,

[jira] [Created] (TEZ-4017) Support proper AM failover on DAG with concurrent edge

2018-10-31 Thread Yingda Chen (JIRA)
Yingda Chen created TEZ-4017: Summary: Support proper AM failover on DAG with concurrent edge Key: TEZ-4017 URL: https://issues.apache.org/jira/browse/TEZ-4017 Project: Apache Tez Issue Type:

[jira] [Updated] (TEZ-3997) Enable CONCURRENT edge

2018-10-31 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yingda Chen updated TEZ-3997: - External issue ID: TEZ-3998, TEZ-3999, TEZ-4000, TEZ-4001 > Enable CONCURRENT edge >

[jira] [Commented] (TEZ-2222) Investigate moving to log4j2 for logging

2018-11-09 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682169#comment-16682169 ] Yingda Chen commented on TEZ-: -- [~ewohlstadter] thanks for letting us know about the related work. We

[jira] [Commented] (TEZ-2222) Investigate moving to log4j2 for logging

2018-11-08 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680589#comment-16680589 ] Yingda Chen commented on TEZ-: -- [~jeagles] I would like to provide some more context about our usage

[jira] [Updated] (TEZ-4018) Allow conditional vertex in DAG execution

2018-11-15 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yingda Chen updated TEZ-4018: - Description: A high-level description is provided here for now, *will follow up with a proper design doc

[jira] [Updated] (TEZ-3998) Allow CONCURRENT edge property in DAG construction and introduce ConcurrentSchedulingType

2018-11-15 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yingda Chen updated TEZ-3998: - Attachment: TEZ-3998.001.patch.diff > Allow CONCURRENT edge property in DAG construction and introduce >

[jira] [Updated] (TEZ-4018) Allow conditional vertex in DAG execution

2018-11-13 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yingda Chen updated TEZ-4018: - Description: A high-level description is provided here for now, *will follow up with a proper design doc

[jira] [Updated] (TEZ-4018) Allow conditional vertex in DAG execution

2018-11-13 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yingda Chen updated TEZ-4018: - Description: A high-level description is provided here for now, *will follow up with a proper design doc

[jira] [Created] (TEZ-4018) Allow conditional vertex in DAG execution

2018-11-13 Thread Yingda Chen (JIRA)
Yingda Chen created TEZ-4018: Summary: Allow conditional vertex in DAG execution Key: TEZ-4018 URL: https://issues.apache.org/jira/browse/TEZ-4018 Project: Apache Tez Issue Type: New Feature

[jira] [Assigned] (TEZ-4018) Allow conditional vertex in DAG execution

2018-11-13 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yingda Chen reassigned TEZ-4018: Assignee: Yingda Chen > Allow conditional vertex in DAG execution >

[jira] [Assigned] (TEZ-3822) Default threshold for blacklisting a node is too high

2018-10-05 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yingda Chen reassigned TEZ-3822: Assignee: Ying Han > Default threshold for blacklisting a node is too high >

[jira] [Commented] (TEZ-3998) Allow CONCURRENT edge property in DAG construction and introduce ConcurrentSchedulingType

2018-11-16 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689930#comment-16689930 ] Yingda Chen commented on TEZ-3998: -- [~jeagles] could you please comment on the test result and how to

[jira] [Commented] (TEZ-3998) Allow CONCURRENT edge property in DAG construction and introduce ConcurrentSchedulingType

2018-11-20 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16693642#comment-16693642 ] Yingda Chen commented on TEZ-3998: -- [~jeagles] kindly ping... I was hoping this can go in before the

[jira] [Commented] (TEZ-4042) Speculative attempts should avoid running on the same node

2019-02-20 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16773743#comment-16773743 ] Yingda Chen commented on TEZ-4042: -- I am done with my review [~Chyler]. LGTM > Speculative attempts

[jira] [Updated] (TEZ-4045) Task should be accessible from TaskAttempt

2019-02-22 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yingda Chen updated TEZ-4045: - Priority: Major (was: Minor) > Task should be accessible from TaskAttempt >

[jira] [Created] (TEZ-4045) Task should be accessible from TaskAttempt

2019-02-22 Thread Yingda Chen (JIRA)
Yingda Chen created TEZ-4045: Summary: Task should be accessible from TaskAttempt Key: TEZ-4045 URL: https://issues.apache.org/jira/browse/TEZ-4045 Project: Apache Tez Issue Type: Improvement

[jira] [Assigned] (TEZ-4063) DAGClient:tryKillDAG taking long time

2019-04-26 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yingda Chen reassigned TEZ-4063: Assignee: Ying Han > DAGClient:tryKillDAG taking long time > -

[jira] [Assigned] (TEZ-4060) NoOpVertexManager schedules tasks that are not ready to run

2019-04-26 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yingda Chen reassigned TEZ-4060: Assignee: Ying Han > NoOpVertexManager schedules tasks that are not ready to run >

[jira] [Assigned] (TEZ-4062) Speculative attempt scheduling should be aborted when Task has complelted

2019-04-18 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yingda Chen reassigned TEZ-4062: Assignee: Ying Han > Speculative attempt scheduling should be aborted when Task has complelted >

[jira] [Created] (TEZ-4062) Speculative attempt scheduling should be aborted when Task has complelted

2019-04-18 Thread Yingda Chen (JIRA)
Yingda Chen created TEZ-4062: Summary: Speculative attempt scheduling should be aborted when Task has complelted Key: TEZ-4062 URL: https://issues.apache.org/jira/browse/TEZ-4062 Project: Apache Tez

[jira] [Commented] (TEZ-4062) Speculative attempt scheduling should be aborted when Task has completed

2019-04-26 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16827145#comment-16827145 ] Yingda Chen commented on TEZ-4062: -- [~jeagles] any progress on the build failure, and anything we can

[jira] [Commented] (TEZ-4062) Speculative attempt scheduling should be aborted when Task has completed

2019-04-26 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16827266#comment-16827266 ] Yingda Chen commented on TEZ-4062: -- Thanks [~jeagles] for getting the build problem resolved. We will keep

[jira] [Commented] (TEZ-4062) Speculative attempt scheduling should be aborted when Task has complelted

2019-04-22 Thread Yingda Chen (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823328#comment-16823328 ] Yingda Chen commented on TEZ-4062: -- Looking at the build failure above, most errors look like this:

[jira] [Commented] (TEZ-4317) Tez job can hang if new allocated container released because of speculative attempts avoid running on the same node

2021-07-06 Thread Yingda Chen (Jira)
[ https://issues.apache.org/jira/browse/TEZ-4317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17375251#comment-17375251 ] Yingda Chen commented on TEZ-4317: -- [~wei.wei], could you plase be more specifics as to what you believe