[jira] [Updated] (TEZ-4076) Add hadoop-cloud-storage jar to aws and azure mvn profiles

2019-07-16 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-4076: - Fix Version/s: 0.10.1 > Add hadoop-cloud-storage jar to aws and azure mvn profiles >

[jira] [Assigned] (TEZ-4076) Add hadoop-cloud-storage jar to aws and azure mvn profiles

2019-07-10 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V reassigned TEZ-4076: Assignee: Jesus Camacho Rodriguez > Add hadoop-cloud-storage jar to aws and azure mvn profiles >

[jira] [Commented] (TEZ-4076) Add hadoop-cloud-storage jar to aws and azure mvn profiles

2019-07-10 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882550#comment-16882550 ] Gopal V commented on TEZ-4076: -- LGTM - +1 > Add hadoop-cloud-storage jar to aws and azure mvn profiles >

[jira] [Commented] (TEZ-4075) Tez: Reimplement tez.runtime.transfer.data-via-events.enabled

2019-06-20 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868950#comment-16868950 ] Gopal V commented on TEZ-4075: -- The corresponding test-scenario for this is

[jira] [Created] (TEZ-4075) Tez: Reimplement tez.runtime.transfer.data-via-events.enabled

2019-06-20 Thread Gopal V (JIRA)
Gopal V created TEZ-4075: Summary: Tez: Reimplement tez.runtime.transfer.data-via-events.enabled Key: TEZ-4075 URL: https://issues.apache.org/jira/browse/TEZ-4075 Project: Apache Tez Issue Type:

[jira] [Commented] (TEZ-4073) Configuration: Reduce Vertex and DAG Payload Size

2019-06-01 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16853602#comment-16853602 ] Gopal V commented on TEZ-4073: -- The HiveConf source annotation, allows for a source filter version of {{public

[jira] [Commented] (TEZ-4073) Configuration: Reduce Vertex and DAG Payload Size

2019-05-31 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16853529#comment-16853529 ] Gopal V commented on TEZ-4073: -- We can also differentiate between the configs set via HiveConf::initialize()

[jira] [Commented] (TEZ-4073) Configuration: Reduce Vertex and DAG Payload Size

2019-05-31 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16853527#comment-16853527 ] Gopal V commented on TEZ-4073: -- The trivial version of this is to skip all the configs which have *.xml when

[jira] [Comment Edited] (TEZ-4073) Configuration: Reduce Vertex and DAG Payload Size

2019-05-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16851545#comment-16851545 ] Gopal V edited comment on TEZ-4073 at 5/30/19 6:07 AM: --- Async dispatcher CPU is

[jira] [Commented] (TEZ-4073) Configuration: Reduce Vertex and DAG Payload Size

2019-05-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16851545#comment-16851545 ] Gopal V commented on TEZ-4073: -- Async dispatcher CPU is mostly spent on the protobuf codepaths. >

[jira] [Updated] (TEZ-4073) Configuration: Reduce Vertex and DAG Payload Size

2019-05-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-4073: - Description: As the total number of vertices go up, the Tez protobuf transport starts to show up as a potential

[jira] [Updated] (TEZ-4073) Configuration: Reduce Vertex and DAG Payload Size

2019-05-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-4073: - Attachment: tez-am-protobuf-reading.png > Configuration: Reduce Vertex and DAG Payload Size >

[jira] [Updated] (TEZ-4073) Configuration: Reduce Vertex and DAG Payload Size

2019-05-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-4073: - Attachment: tez-protobuf-writing.png > Configuration: Reduce Vertex and DAG Payload Size >

[jira] [Created] (TEZ-4073) Configuration: Reduce Vertex and DAG Payload Size

2019-05-30 Thread Gopal V (JIRA)
Gopal V created TEZ-4073: Summary: Configuration: Reduce Vertex and DAG Payload Size Key: TEZ-4073 URL: https://issues.apache.org/jira/browse/TEZ-4073 Project: Apache Tez Issue Type: Bug

[jira] [Commented] (TEZ-3310) Handle splits grouping better when locality information is not available (or only when localhost is available)

2019-05-06 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16833956#comment-16833956 ] Gopal V commented on TEZ-3310: -- The reason "localhost" is scrubbed is because RM was waiting for "localhost"

[jira] [Updated] (TEZ-4057) Fix Unsorted broadcast shuffle umasks

2019-03-29 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-4057: - Fix Version/s: (was: 0.9.2) 0.9.3 > Fix Unsorted broadcast shuffle umasks >

[jira] [Updated] (TEZ-4057) Fix Unsorted broadcast shuffle umasks

2019-03-28 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-4057: - Fix Version/s: (was: 0.9.3) 0.9.2 > Fix Unsorted broadcast shuffle umasks >

[jira] [Updated] (TEZ-4057) Fix Unsorted broadcast shuffle umasks

2019-03-28 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-4057: - Fix Version/s: 0.10.1 > Fix Unsorted broadcast shuffle umasks > - > >

[jira] [Commented] (TEZ-4057) Fix Unsorted broadcast shuffle umasks

2019-03-28 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804265#comment-16804265 ] Gopal V commented on TEZ-4057: -- LGTM - +1 > Fix Unsorted broadcast shuffle umasks >

[jira] [Created] (TEZ-4057) Fix Unsorted broadcast shuffle umasks

2019-03-27 Thread Gopal V (JIRA)
Gopal V created TEZ-4057: Summary: Fix Unsorted broadcast shuffle umasks Key: TEZ-4057 URL: https://issues.apache.org/jira/browse/TEZ-4057 Project: Apache Tez Issue Type: Bug Affects Versions:

[jira] [Commented] (TEZ-4044) Zookeeper: exclude jline from Zookeeper client from tez dist

2019-03-12 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790803#comment-16790803 ] Gopal V commented on TEZ-4044: -- thanks [~jeagles]! > Zookeeper: exclude jline from Zookeeper client from tez

[jira] [Commented] (TEZ-4044) Zookeeper: exclude jline from Zookeeper client from tez dist

2019-03-11 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790132#comment-16790132 ] Gopal V commented on TEZ-4044: -- bq. I wonder how this is failing for you Beeline stops working - because it

[jira] [Commented] (TEZ-4048) Make proto history logger queue size configurable

2019-02-27 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16779880#comment-16779880 ] Gopal V commented on TEZ-4048: -- The real problem is that a huge number of events being logged aren't

[jira] [Commented] (TEZ-4048) Make proto history logger queue size configurable

2019-02-27 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16779879#comment-16779879 ] Gopal V commented on TEZ-4048: -- Since I'm overflowing 10k items right now easily (~200k items every 30s), I

[jira] [Commented] (TEZ-4048) Make proto history logger queue size configurable

2019-02-27 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16779833#comment-16779833 ] Gopal V commented on TEZ-4048: -- LGTM - +1 The events that overflow are just dropped right now - I suspect

[jira] [Assigned] (TEZ-4044) Zookeeper: exclude jline from Zookeeper client from tez dist

2019-02-21 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V reassigned TEZ-4044: Assignee: Gopal V > Zookeeper: exclude jline from Zookeeper client from tez dist >

[jira] [Created] (TEZ-4044) Zookeeper: exclude jline from Zookeeper client from tez dist

2019-02-21 Thread Gopal V (JIRA)
Gopal V created TEZ-4044: Summary: Zookeeper: exclude jline from Zookeeper client from tez dist Key: TEZ-4044 URL: https://issues.apache.org/jira/browse/TEZ-4044 Project: Apache Tez Issue Type: Bug

[jira] [Commented] (TEZ-3985) Correctness: Throw a clear exception for DMEs sent during cleanup

2019-02-08 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16763925#comment-16763925 ] Gopal V commented on TEZ-3985: -- Fixed minor nits. > Correctness: Throw a clear exception for DMEs sent during

[jira] [Assigned] (TEZ-4038) Add a /prof profiler endpoint like HiveServer2 has

2019-02-08 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V reassigned TEZ-4038: Assignee: Gopal V > Add a /prof profiler endpoint like HiveServer2 has >

[jira] [Created] (TEZ-4038) Add a /prof profiler endpoint like HiveServer2 has

2019-02-08 Thread Gopal V (JIRA)
Gopal V created TEZ-4038: Summary: Add a /prof profiler endpoint like HiveServer2 has Key: TEZ-4038 URL: https://issues.apache.org/jira/browse/TEZ-4038 Project: Apache Tez Issue Type: New Feature

[jira] [Updated] (TEZ-3985) Correctness: Throw a clear exception for DMEs sent during cleanup

2019-02-08 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-3985: - Attachment: TEZ-3985.3.patch > Correctness: Throw a clear exception for DMEs sent during cleanup >

[jira] [Updated] (TEZ-4033) Add a -Pozone flag to include Ozone/HDDS

2019-01-28 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-4033: - Issue Type: New Feature (was: Bug) > Add a -Pozone flag to include Ozone/HDDS >

[jira] [Assigned] (TEZ-4033) Add a -Pozone flag to include Ozone/HDDS

2019-01-28 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V reassigned TEZ-4033: Assignee: Gopal V > Add a -Pozone flag to include Ozone/HDDS > - >

[jira] [Created] (TEZ-4033) Add a -Pozone flag to include Ozone/HDDS

2019-01-28 Thread Gopal V (JIRA)
Gopal V created TEZ-4033: Summary: Add a -Pozone flag to include Ozone/HDDS Key: TEZ-4033 URL: https://issues.apache.org/jira/browse/TEZ-4033 Project: Apache Tez Issue Type: Bug

[jira] [Updated] (TEZ-4028) Events not visible from proto history logging for s3a filesystem until dag completes.

2019-01-27 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-4028: - Labels: history (was: ) > Events not visible from proto history logging for s3a filesystem until dag >

[jira] [Commented] (TEZ-4028) Events not visible from proto history logging for s3a filesystem until dag completes.

2019-01-25 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752776#comment-16752776 ] Gopal V commented on TEZ-4028: -- LGTM - +1 The DAG now gets a flush the moment it starts, so that the configs

[jira] [Commented] (TEZ-3957) Report TASK_DURATION_MILLIS as a Counter for completed tasks

2018-12-06 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711829#comment-16711829 ] Gopal V commented on TEZ-3957: -- LGTM - +1 This counter is aggregated for capacity planning models with

[jira] [Commented] (TEZ-3957) Report TASK_DURATION_MILLIS as a Counter for completed tasks

2018-11-06 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677669#comment-16677669 ] Gopal V commented on TEZ-3957: -- {code} [ERROR] Process Exit Code: 1 [ERROR] ExecutionException The forked VM

[jira] [Updated] (TEZ-3957) Report TASK_DURATION_MILLIS as a Counter for completed tasks

2018-11-06 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-3957: - Attachment: TEZ-3957.02.patch > Report TASK_DURATION_MILLIS as a Counter for completed tasks >

[jira] [Commented] (TEZ-3957) Report TASK_DURATION_MILLIS as a Counter for completed tasks

2018-11-06 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677425#comment-16677425 ] Gopal V commented on TEZ-3957: -- [~sershe]: can you fix the NS_TO_MS with the java time conversions? MapReduce

[jira] [Updated] (TEZ-3976) Batch ShuffleManager error report events

2018-10-23 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-3976: - Fix Version/s: 0.10.1 > Batch ShuffleManager error report events > > >

[jira] [Updated] (TEZ-3976) Batch ShuffleManager error report events

2018-10-23 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-3976: - Summary: Batch ShuffleManager error report events (was: ShuffleManager reporting too many errors) > Batch

[jira] [Commented] (TEZ-3976) ShuffleManager reporting too many errors

2018-10-23 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16661392#comment-16661392 ] Gopal V commented on TEZ-3976: -- LGTM - +1 > ShuffleManager reporting too many errors >

[jira] [Commented] (TEZ-3976) ShuffleManager reporting too many errors

2018-10-19 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657617#comment-16657617 ] Gopal V commented on TEZ-3976: -- [~jmarhuen]: I'm fine with this patch (+1), except for one detail - the

[jira] [Commented] (TEZ-4010) Use InputReadyVertexManager for broadcast connection

2018-10-17 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16654147#comment-16654147 ] Gopal V commented on TEZ-4010: -- [~jeagles]: you are right - the scenario in TEZ-3274 is related, but this

[jira] [Commented] (TEZ-4010) Use InputReadyVertexManager for broadcast connection

2018-10-17 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16653140#comment-16653140 ] Gopal V commented on TEZ-4010: -- bq. This seems to be inefficient as tasks are started way before they can run

[jira] [Updated] (TEZ-3991) Unmanaged tez sessions

2018-10-16 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-3991: - Labels: Kubernetes (was: ) > Unmanaged tez sessions > -- > > Key: TEZ-3991 >

[jira] [Updated] (TEZ-4003) Add gop...@apache.org to KEYS file

2018-10-09 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-4003: - Attachment: TEZ-4003.patch > Add gop...@apache.org to KEYS file > -- > >

[jira] [Assigned] (TEZ-4003) Add gop...@apache.org to KEYS file

2018-10-09 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V reassigned TEZ-4003: Assignee: Gopal V > Add gop...@apache.org to KEYS file > -- > >

[jira] [Created] (TEZ-4003) Add gop...@apache.org to KEYS file

2018-10-09 Thread Gopal V (JIRA)
Gopal V created TEZ-4003: Summary: Add gop...@apache.org to KEYS file Key: TEZ-4003 URL: https://issues.apache.org/jira/browse/TEZ-4003 Project: Apache Tez Issue Type: Task Reporter:

[jira] [Commented] (TEZ-3888) Update Jetty to org.eclipse.jetty 9.x

2018-10-08 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642412#comment-16642412 ] Gopal V commented on TEZ-3888: -- Sure, standardizing the versions for ABI makes sense - I think HIVE-19421

[jira] [Commented] (TEZ-3888) Update Jetty to org.eclipse.jetty 9.x

2018-10-08 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642405#comment-16642405 ] Gopal V commented on TEZ-3888: -- bq. this is causing NoSuchMethod errors in HiveServer2. Which version is

[jira] [Assigned] (TEZ-3984) Shuffle: Out of Band DME event sending causes errors

2018-09-09 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V reassigned TEZ-3984: Assignee: Jaume M (was: Gopal V) > Shuffle: Out of Band DME event sending causes errors >

[jira] [Updated] (TEZ-3984) Shuffle: Out of Band DME event sending causes errors

2018-09-08 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-3984: - Attachment: TEZ-3984.5.patch > Shuffle: Out of Band DME event sending causes errors >

[jira] [Assigned] (TEZ-3984) Shuffle: Out of Band DME event sending causes errors

2018-09-08 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V reassigned TEZ-3984: Assignee: Gopal V (was: Jaume M) > Shuffle: Out of Band DME event sending causes errors >

[jira] [Commented] (TEZ-3984) Shuffle: Out of Band DME event sending causes errors

2018-08-29 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596751#comment-16596751 ] Gopal V commented on TEZ-3984: -- The patch looks good - minor NIT, the OrderedPartitionedKVOutput event lists

[jira] [Commented] (TEZ-3985) Correctness: Throw a clear exception for DMEs sent during cleanup

2018-08-28 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595259#comment-16595259 ] Gopal V commented on TEZ-3985: -- Interestingly, this does not get logged at all - because the cleanup ignores

[jira] [Updated] (TEZ-3958) Add internal vertex priority information into the tez dag.dot debug information

2018-08-27 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-3958: - Attachment: TEZ-3958.5.patch > Add internal vertex priority information into the tez dag.dot debug > information

[jira] [Assigned] (TEZ-3958) Add internal vertex priority information into the tez dag.dot debug information

2018-08-27 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V reassigned TEZ-3958: Assignee: Jaume M (was: Gopal V) > Add internal vertex priority information into the tez dag.dot debug >

[jira] [Assigned] (TEZ-3958) Add internal vertex priority information into the tez dag.dot debug information

2018-08-27 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V reassigned TEZ-3958: Assignee: Gopal V (was: Jaume M) > Add internal vertex priority information into the tez dag.dot debug >

[jira] [Created] (TEZ-3985) Correctness: Throw a clear exception for DMEs sent during cleanup

2018-08-27 Thread Gopal V (JIRA)
Gopal V created TEZ-3985: Summary: Correctness: Throw a clear exception for DMEs sent during cleanup Key: TEZ-3985 URL: https://issues.apache.org/jira/browse/TEZ-3985 Project: Apache Tez Issue

[jira] [Commented] (TEZ-3984) Shuffle: Out of Band DME event sending causes errors

2018-08-27 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594304#comment-16594304 ] Gopal V commented on TEZ-3984: -- Specific sequence of events is - input throws exception. {code}

[jira] [Updated] (TEZ-3984) Shuffle: Out of Band DME event sending causes errors

2018-08-27 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-3984: - Labels: correctness (was: ) > Shuffle: Out of Band DME event sending causes errors >

[jira] [Created] (TEZ-3984) Shuffle: Out of Band DME event sending causes errors

2018-08-27 Thread Gopal V (JIRA)
Gopal V created TEZ-3984: Summary: Shuffle: Out of Band DME event sending causes errors Key: TEZ-3984 URL: https://issues.apache.org/jira/browse/TEZ-3984 Project: Apache Tez Issue Type: Bug

[jira] [Commented] (TEZ-3980) ShuffleRunner: the wake loop needs to check for shutdown

2018-08-16 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582757#comment-16582757 ] Gopal V commented on TEZ-3980: -- Testing issues with LLAP task pre-emption. When reducers doing the unsorted

[jira] [Commented] (TEZ-3980) ShuffleRunner: the wake loop needs to check for shutdown

2018-08-16 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582752#comment-16582752 ] Gopal V commented on TEZ-3980: -- The shufflescheduler has a check for shutdown.get() + a break inside the loop

[jira] [Updated] (TEZ-3980) ShuffleRunner: the wake loop needs to check for shutdown

2018-08-15 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-3980: - Attachment: (was: TEZ-3980.1.patch) > ShuffleRunner: the wake loop needs to check for shutdown >

[jira] [Updated] (TEZ-3980) ShuffleRunner: the wake loop needs to check for shutdown

2018-08-15 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-3980: - Attachment: TEZ-3980.1.patch > ShuffleRunner: the wake loop needs to check for shutdown >

[jira] [Updated] (TEZ-3980) ShuffleRunner: the wake loop needs to check for shutdown

2018-08-15 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-3980: - Attachment: TEZ-3980.1.patch > ShuffleRunner: the wake loop needs to check for shutdown >

[jira] [Created] (TEZ-3980) ShuffleRunner: the wake loop needs to check for shutdown

2018-08-15 Thread Gopal V (JIRA)
Gopal V created TEZ-3980: Summary: ShuffleRunner: the wake loop needs to check for shutdown Key: TEZ-3980 URL: https://issues.apache.org/jira/browse/TEZ-3980 Project: Apache Tez Issue Type: Bug

[jira] [Assigned] (TEZ-3980) ShuffleRunner: the wake loop needs to check for shutdown

2018-08-15 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V reassigned TEZ-3980: Assignee: Gopal V > ShuffleRunner: the wake loop needs to check for shutdown >

[jira] [Updated] (TEZ-3974) Tez: Correctness regression of TEZ-955 in TEZ-2937

2018-08-09 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-3974: - Fix Version/s: 0.10.0 > Tez: Correctness regression of TEZ-955 in TEZ-2937 >

[jira] [Updated] (TEZ-3974) Tez: Correctness regression of TEZ-955 in TEZ-2937

2018-08-09 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-3974: - Affects Version/s: 0.9.1 > Tez: Correctness regression of TEZ-955 in TEZ-2937 >

[jira] [Commented] (TEZ-3974) Tez: Correctness regression of TEZ-955 in TEZ-2937

2018-08-09 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16575211#comment-16575211 ] Gopal V commented on TEZ-3974: -- I see - https://builds.apache.org/job/PreCommit-TEZ-Build/2871/ has kicked off

[jira] [Commented] (TEZ-3974) Tez: Correctness regression of TEZ-955 in TEZ-2937

2018-08-09 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16575203#comment-16575203 ] Gopal V commented on TEZ-3974: -- Reuploading to force QE runs. > Tez: Correctness regression of TEZ-955 in

[jira] [Assigned] (TEZ-3974) Tez: Correctness regression of TEZ-955 in TEZ-2937

2018-08-09 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V reassigned TEZ-3974: Assignee: Gopal V (was: Jaume M) > Tez: Correctness regression of TEZ-955 in TEZ-2937 >

[jira] [Commented] (TEZ-3976) ShuffleManager reporting too many errors

2018-08-07 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16572034#comment-16572034 ] Gopal V commented on TEZ-3976: -- bq. Normally this scenario does not occur because one of the following

[jira] [Assigned] (TEZ-3974) Tez: Correctness regression of TEZ-955 in TEZ-2937

2018-08-03 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V reassigned TEZ-3974: Assignee: Jaume M > Tez: Correctness regression of TEZ-955 in TEZ-2937 >

[jira] [Commented] (TEZ-3974) Tez: Correctness regression of TEZ-955 in TEZ-2937

2018-08-03 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16568979#comment-16568979 ] Gopal V commented on TEZ-3974: -- LGTM - +1 [~jaume]: can you give brief note about testing, because this is an

[jira] [Resolved] (TEZ-3971) Incorrect query result in hive when hive.convert.join.bucket.mapjoin.tez=true

2018-07-16 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V resolved TEZ-3971. -- Resolution: Won't Do > Incorrect query result in hive when hive.convert.join.bucket.mapjoin.tez=true >

[jira] [Commented] (TEZ-3965) TestMROutput: Fix the hard-coded "/tmp/output" paths

2018-07-12 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16542533#comment-16542533 ] Gopal V commented on TEZ-3965: -- LGTM - +1 > TestMROutput: Fix the hard-coded "/tmp/output" paths >

[jira] [Commented] (TEZ-3967) DAGImpl: dag lock is unfair and can starve the writers

2018-07-12 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16542205#comment-16542205 ] Gopal V commented on TEZ-3967: -- {code} TEZ_DAG_STATUS_CHECK_INTERVAL("hive.tez.dag.status.check.interval",

[jira] [Commented] (TEZ-3968) Tez Job Fails with Shuffle failures too fast when NM returns a 401 error

2018-07-09 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16537776#comment-16537776 ] Gopal V commented on TEZ-3968: -- The core issue is that the 400+ retries happened in under a second and the

[jira] [Updated] (TEZ-3968) Tez Job Fails with Shuffle failures too fast when NM returns a 401 error

2018-07-09 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-3968: - Description: Tez Job failed with a reduce task failed on all four attempts while fetching a particular map output

[jira] [Updated] (TEZ-3968) Tez Job Fails with Shuffle failures too fast when NM returns a 401 error

2018-07-09 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-3968: - Summary: Tez Job Fails with Shuffle failures too fast when NM returns a 401 error (was: Tez Job Fails with

[jira] [Created] (TEZ-3969) TaskAttemptImpl: static fields initialized in instance ctor

2018-07-09 Thread Gopal V (JIRA)
Gopal V created TEZ-3969: Summary: TaskAttemptImpl: static fields initialized in instance ctor Key: TEZ-3969 URL: https://issues.apache.org/jira/browse/TEZ-3969 Project: Apache Tez Issue Type: Bug

[jira] [Comment Edited] (TEZ-3916) Add hadoop-azure-datalake jar to azure profile

2018-07-09 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16537541#comment-16537541 ] Gopal V edited comment on TEZ-3916 at 7/9/18 8:45 PM: -- These jars are in the

[jira] [Commented] (TEZ-3916) Add hadoop-azure-datalake jar to azure profile

2018-07-09 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16537541#comment-16537541 ] Gopal V commented on TEZ-3916: -- These jars are in the tez.tar.gz (similar to other hadoop-* jars) when

[jira] [Created] (TEZ-3967) DAGImpl: dag lock is unfair and can starve the writers

2018-07-06 Thread Gopal V (JIRA)
Gopal V created TEZ-3967: Summary: DAGImpl: dag lock is unfair and can starve the writers Key: TEZ-3967 URL: https://issues.apache.org/jira/browse/TEZ-3967 Project: Apache Tez Issue Type: Bug

[jira] [Assigned] (TEZ-3963) Possible InflaterInputStream leaked in TezCommonUtils and related classes

2018-07-05 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V reassigned TEZ-3963: Assignee: Jaume M > Possible InflaterInputStream leaked in TezCommonUtils and related classes >

[jira] [Assigned] (TEZ-3958) Add internal vertex priority information into the tez dag.dot debug information

2018-07-03 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V reassigned TEZ-3958: Assignee: Jaume M > Add internal vertex priority information into the tez dag.dot debug > information >

[jira] [Created] (TEZ-3965) TestMROutput: Fix the hard-coded "/tmp/output" paths

2018-07-03 Thread Gopal V (JIRA)
Gopal V created TEZ-3965: Summary: TestMROutput: Fix the hard-coded "/tmp/output" paths Key: TEZ-3965 URL: https://issues.apache.org/jira/browse/TEZ-3965 Project: Apache Tez Issue Type: Bug

[jira] [Commented] (TEZ-3963) Possible InflaterInputStream leaked in TezCommonUtils and related classes

2018-06-30 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528568#comment-16528568 ] Gopal V commented on TEZ-3963: -- +1 tests pending > Possible InflaterInputStream leaked in TezCommonUtils and

[jira] [Created] (TEZ-3962) Configuration decode leaks an Inflater object

2018-06-26 Thread Gopal V (JIRA)
Gopal V created TEZ-3962: Summary: Configuration decode leaks an Inflater object Key: TEZ-3962 URL: https://issues.apache.org/jira/browse/TEZ-3962 Project: Apache Tez Issue Type: Bug Affects

[jira] [Commented] (TEZ-3958) Add internal vertex priority information into the tez dag.dot debug information

2018-06-25 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522662#comment-16522662 ] Gopal V commented on TEZ-3958: -- Let me take a very specific case here and talk about a DAG in this order.

[jira] [Commented] (TEZ-3958) Add internal vertex priority information into the tez dag.dot debug information

2018-06-25 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522650#comment-16522650 ] Gopal V commented on TEZ-3958: -- [~jmarhuen]: yes, you are right - however the priority limits for the task

[jira] [Created] (TEZ-3958) Add internal vertex priority information into the tez dag.dot debug information

2018-06-21 Thread Gopal V (JIRA)
Gopal V created TEZ-3958: Summary: Add internal vertex priority information into the tez dag.dot debug information Key: TEZ-3958 URL: https://issues.apache.org/jira/browse/TEZ-3958 Project: Apache Tez

[jira] [Updated] (TEZ-3953) Restore ABI-compat for DAGClient for TEZ-3951

2018-06-11 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-3953: - Summary: Restore ABI-compat for DAGClient for TEZ-3951 (was: make interface change from TEZ-3951 non-breaking) >

[jira] [Updated] (TEZ-3953) make interface change from TEZ-3951 non-breaking

2018-06-11 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-3953: - Affects Version/s: 0.10.0 > make interface change from TEZ-3951 non-breaking >

[jira] [Commented] (TEZ-3953) make interface change from TEZ-3951 non-breaking

2018-06-11 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508933#comment-16508933 ] Gopal V commented on TEZ-3953: -- LGTM - +1 Is there a hive-3 branch impl patch for Hive SyncDagClient for

  1   2   3   4   5   6   >