[jira] [Updated] (TEZ-3955) Upgrade hadoop dependency to 3.0.3

2018-06-15 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3955: - Attachment: TEZ-3955.001.patch > Upgrade hadoop dependency to 3.0.3 >

[jira] [Created] (TEZ-3955) Upgrade hadoop dependency to 3.0.3

2018-06-15 Thread Jonathan Eagles (JIRA)
Jonathan Eagles created TEZ-3955: Summary: Upgrade hadoop dependency to 3.0.3 Key: TEZ-3955 URL: https://issues.apache.org/jira/browse/TEZ-3955 Project: Apache Tez Issue Type: Bug

[jira] [Updated] (TEZ-3954) Reduce Tez Shuffle Handler Memory needs for holding TezIndexRecords

2018-06-15 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3954: - Attachment: TEZ-3954.001.patch > Reduce Tez Shuffle Handler Memory needs for holding

[jira] [Created] (TEZ-3954) Reduce Tez Shuffle Handler Memory needs for holding TezIndexRecords

2018-06-15 Thread Jonathan Eagles (JIRA)
Jonathan Eagles created TEZ-3954: Summary: Reduce Tez Shuffle Handler Memory needs for holding TezIndexRecords Key: TEZ-3954 URL: https://issues.apache.org/jira/browse/TEZ-3954 Project: Apache Tez

[jira] [Commented] (TEZ-3885) Tez timeline server integration error

2018-06-14 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16512721#comment-16512721 ] Jonathan Eagles commented on TEZ-3885: -- As to the NullPointerException, that was addressed in TEZ-3834

[jira] [Commented] (TEZ-3885) Tez timeline server integration error

2018-06-14 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16512710#comment-16512710 ] Jonathan Eagles commented on TEZ-3885: -- [~karthikp], we upgraded our jersey dependency to 1.19 as

[jira] [Commented] (TEZ-3952) Allow Tez task speculation to allow for greater customization

2018-06-12 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16510055#comment-16510055 ] Jonathan Eagles commented on TEZ-3952: -- [~nishantdash], thanks for this patch. Couple of minor nits.

[jira] [Commented] (TEZ-3944) TestTaskScheduler times-out on Hadoop3

2018-06-06 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504001#comment-16504001 ] Jonathan Eagles commented on TEZ-3944: -- Made the change to move mockContainer3A to mockContainer3 and

[jira] [Updated] (TEZ-3944) TestTaskScheduler times-out on Hadoop3

2018-06-06 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3944: - Attachment: TEZ-3944.003.patch > TestTaskScheduler times-out on Hadoop3 >

[jira] [Commented] (TEZ-2218) Turn on speculation by default

2018-06-06 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16503385#comment-16503385 ] Jonathan Eagles commented on TEZ-2218: -- The other consideration, is that for Pig and Hive, they are

[jira] [Commented] (TEZ-2218) Turn on speculation by default

2018-06-06 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16503381#comment-16503381 ] Jonathan Eagles commented on TEZ-2218: -- [~sershe], there are some issues with enabling speculative

[jira] [Updated] (TEZ-2218) Turn on speculation by default

2018-06-06 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-2218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-2218: - Target Version/s: 0.10.0 > Turn on speculation by default > -- > >

[jira] [Commented] (TEZ-3945) TestHistoryParser fails with Hadoop3 build

2018-06-06 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16503308#comment-16503308 ] Jonathan Eagles commented on TEZ-3945: -- [~ewohlstadter], I don't see this failing anymore on master

[jira] [Commented] (TEZ-3944) TestTaskScheduler times-out on Hadoop3

2018-06-06 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16503306#comment-16503306 ] Jonathan Eagles commented on TEZ-3944: -- [~kshukla], can you have a review. 99% of the change is just

[jira] [Commented] (TEZ-3944) TestTaskScheduler times-out on Hadoop3

2018-06-05 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16502821#comment-16502821 ] Jonathan Eagles commented on TEZ-3944: -- Updated the tests to be in line with the level of mock that

[jira] [Updated] (TEZ-3944) TestTaskScheduler times-out on Hadoop3

2018-06-05 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3944: - Attachment: TEZ-3944.002.patch > TestTaskScheduler times-out on Hadoop3 >

[jira] [Commented] (TEZ-3951) TezClient wait too long for the DAGClient for prewarm; tries to shut down the wrong DAG

2018-06-05 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16502594#comment-16502594 ] Jonathan Eagles commented on TEZ-3951: -- [~sershe], is there a reason not to make the wait time

[jira] [Commented] (TEZ-3938) Task attempts failing due to not making progress

2018-06-05 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16502525#comment-16502525 ] Jonathan Eagles commented on TEZ-3938: -- +1. Committing to master and branch-0.9 > Task attempts

[jira] [Commented] (TEZ-3944) TestTaskScheduler times-out on Hadoop3

2018-06-05 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16502290#comment-16502290 ] Jonathan Eagles commented on TEZ-3944: -- Fixed my comment above and corrected to HADOOP-15450 >

[jira] [Comment Edited] (TEZ-3944) TestTaskScheduler times-out on Hadoop3

2018-06-05 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16500865#comment-16500865 ] Jonathan Eagles edited comment on TEZ-3944 at 6/5/18 6:18 PM: -- Test failure is

[jira] [Comment Edited] (TEZ-3944) TestTaskScheduler times-out on Hadoop3

2018-06-05 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16500865#comment-16500865 ] Jonathan Eagles edited comment on TEZ-3944 at 6/5/18 6:18 PM: -- Test failure is

[jira] [Commented] (TEZ-3950) Preempted task attempts intermittently marked as FAILED instead of KILLED

2018-06-05 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16502133#comment-16502133 ] Jonathan Eagles commented on TEZ-3950: -- The race is present in LocalTaskSchedulerService. However, the

[jira] [Commented] (TEZ-3950) Preempted task attempts intermittently marked as FAILED instead of KILLED

2018-06-05 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16502124#comment-16502124 ] Jonathan Eagles commented on TEZ-3950: -- Attaching a patch that helps the first message loop complete

[jira] [Created] (TEZ-3950) Preempted task attempts intermittently marked as FAILED instead of KILLED

2018-06-05 Thread Jonathan Eagles (JIRA)
Jonathan Eagles created TEZ-3950: Summary: Preempted task attempts intermittently marked as FAILED instead of KILLED Key: TEZ-3950 URL: https://issues.apache.org/jira/browse/TEZ-3950 Project: Apache

[jira] [Updated] (TEZ-3950) Preempted task attempts intermittently marked as FAILED instead of KILLED

2018-06-05 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3950: - Attachment: TEZ-3950.fail.patch > Preempted task attempts intermittently marked as FAILED instead

[jira] [Resolved] (TEZ-3005) TestMockDAGAppMaster.testInternalPreemption fails

2018-06-05 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles resolved TEZ-3005. -- Resolution: Cannot Reproduce > TestMockDAGAppMaster.testInternalPreemption fails >

[jira] [Updated] (TEZ-3005) TestMockDAGAppMaster.testInternalPreemption fails

2018-06-05 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3005: - Attachment: TEZ-3005.fail.patch > TestMockDAGAppMaster.testInternalPreemption fails >

[jira] [Updated] (TEZ-3005) TestMockDAGAppMaster.testInternalPreemption fails

2018-06-05 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3005: - Attachment: (was: TEZ-3005.fail.patch) > TestMockDAGAppMaster.testInternalPreemption fails >

[jira] [Reopened] (TEZ-3005) TestMockDAGAppMaster.testInternalPreemption fails

2018-06-05 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles reopened TEZ-3005: -- > TestMockDAGAppMaster.testInternalPreemption fails >

[jira] [Commented] (TEZ-3949) TestATSHistoryV15 is failing with hadoop3+

2018-06-04 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16501157#comment-16501157 ] Jonathan Eagles commented on TEZ-3949: -- TestTaskScheduler failure is due to TEZ-3944. >

[jira] [Updated] (TEZ-3949) TestATSHistoryV15 is failing with hadoop3+

2018-06-04 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3949: - Issue Type: Sub-task (was: Bug) Parent: TEZ-3928 > TestATSHistoryV15 is failing with

[jira] [Commented] (TEZ-3944) TestTaskScheduler times-out on Hadoop3

2018-06-04 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16500945#comment-16500945 ] Jonathan Eagles commented on TEZ-3944: -- Filed TEZ-3949 to address the TestATSHistoryV15 test failure

[jira] [Updated] (TEZ-3949) TestATSHistoryV15 is failing with hadoop3+

2018-06-04 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3949: - Attachment: TEZ-3949.001.patch > TestATSHistoryV15 is failing with hadoop3+ >

[jira] [Created] (TEZ-3949) TestATSHistoryV15 is failing with hadoop3+

2018-06-04 Thread Jonathan Eagles (JIRA)
Jonathan Eagles created TEZ-3949: Summary: TestATSHistoryV15 is failing with hadoop3+ Key: TEZ-3949 URL: https://issues.apache.org/jira/browse/TEZ-3949 Project: Apache Tez Issue Type: Bug

[jira] [Commented] (TEZ-3944) TestTaskScheduler times-out on Hadoop3

2018-06-04 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16500865#comment-16500865 ] Jonathan Eagles commented on TEZ-3944: -- Test failure is due to DiskChecker performance regression in

[jira] [Assigned] (TEZ-3948) Tez distribution broken with hadoop3

2018-06-04 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles reassigned TEZ-3948: Assignee: Jonathan Eagles > Tez distribution broken with hadoop3 >

[jira] [Commented] (TEZ-3948) Tez distribution broken with hadoop3

2018-06-04 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16500311#comment-16500311 ] Jonathan Eagles commented on TEZ-3948: -- There was an activationByDefault that was failing to activate

[jira] [Updated] (TEZ-3948) Tez distribution broken with hadoop3

2018-06-04 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3948: - Attachment: TEZ-3948.001.patch > Tez distribution broken with hadoop3 >

[jira] [Commented] (TEZ-3946) NoClassDefFoundError, org.apache.hadoop.mapred.ShuffleHandler. (Multiple tests with Hadoop3)

2018-06-01 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498822#comment-16498822 ] Jonathan Eagles commented on TEZ-3946: -- {noformat} [INFO] Reactor Summary: [INFO] [INFO] tez

[jira] [Created] (TEZ-3948) Tez distribution broken with hadoop3

2018-06-01 Thread Jonathan Eagles (JIRA)
Jonathan Eagles created TEZ-3948: Summary: Tez distribution broken with hadoop3 Key: TEZ-3948 URL: https://issues.apache.org/jira/browse/TEZ-3948 Project: Apache Tez Issue Type: Bug

[jira] [Commented] (TEZ-3946) NoClassDefFoundError, org.apache.hadoop.mapred.ShuffleHandler. (Multiple tests with Hadoop3)

2018-06-01 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498810#comment-16498810 ] Jonathan Eagles commented on TEZ-3946: -- Uploading the patch again. Build is working correctly on my

[jira] [Updated] (TEZ-3946) NoClassDefFoundError, org.apache.hadoop.mapred.ShuffleHandler. (Multiple tests with Hadoop3)

2018-06-01 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3946: - Attachment: TEZ-3946.2.patch > NoClassDefFoundError, org.apache.hadoop.mapred.ShuffleHandler.

[jira] [Comment Edited] (TEZ-3946) NoClassDefFoundError, org.apache.hadoop.mapred.ShuffleHandler. (Multiple tests with Hadoop3)

2018-06-01 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498732#comment-16498732 ] Jonathan Eagles edited comment on TEZ-3946 at 6/1/18 11:58 PM: ---

[jira] [Commented] (TEZ-3946) NoClassDefFoundError, org.apache.hadoop.mapred.ShuffleHandler. (Multiple tests with Hadoop3)

2018-06-01 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498732#comment-16498732 ] Jonathan Eagles commented on TEZ-3946: -- [~ewohlstadter], it will be better to not switch away from

[jira] [Commented] (TEZ-3929) Upgrade Jersey to 1.19

2018-06-01 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498728#comment-16498728 ] Jonathan Eagles commented on TEZ-3929: -- Committed to master branch. > Upgrade Jersey to 1.19 >

[jira] [Commented] (TEZ-3947) TestATSHistoryWithACLs fails with Hadoop3 and Jersey 1.19

2018-06-01 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498719#comment-16498719 ] Jonathan Eagles commented on TEZ-3947: -- This change was applied to branch-0.9 as well. >

[jira] [Commented] (TEZ-3947) TestATSHistoryWithACLs fails with Hadoop3 and Jersey 1.19

2018-06-01 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498715#comment-16498715 ] Jonathan Eagles commented on TEZ-3947: -- +1. Committing to master branch. > TestATSHistoryWithACLs

[jira] [Commented] (TEZ-3944) TestTaskScheduler times-out on Hadoop3

2018-06-01 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498501#comment-16498501 ] Jonathan Eagles commented on TEZ-3944: -- The hadoop 3 framework is less forgiving with mock objects so

[jira] [Updated] (TEZ-3944) TestTaskScheduler times-out on Hadoop3

2018-06-01 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3944: - Target Version/s: 0.9.2, 0.10.0 (was: 0.10.0) > TestTaskScheduler times-out on Hadoop3 >

[jira] [Commented] (TEZ-3929) Upgrade Jersey to 1.19

2018-06-01 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498428#comment-16498428 ] Jonathan Eagles commented on TEZ-3929: -- This will need to be checked-in jointly with TEZ-3923. >

[jira] [Updated] (TEZ-3944) TestTaskScheduler times-out on Hadoop3

2018-06-01 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3944: - Attachment: TEZ-3944.001.patch > TestTaskScheduler times-out on Hadoop3 >

[jira] [Assigned] (TEZ-3944) TestTaskScheduler times-out on Hadoop3

2018-06-01 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles reassigned TEZ-3944: Assignee: Jonathan Eagles > TestTaskScheduler times-out on Hadoop3 >

[jira] [Commented] (TEZ-3929) Upgrade Jersey to 1.19

2018-06-01 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498287#comment-16498287 ] Jonathan Eagles commented on TEZ-3929: -- +1. This change looks good. Tez little dependency on jersey

[jira] [Commented] (TEZ-3947) TestATSHistoryWithACLs fails with Hadoop3 and Jersey 1.19

2018-06-01 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498248#comment-16498248 ] Jonathan Eagles commented on TEZ-3947: -- [~ewohlstadter], please make use of the MediaType.isCompatible

[jira] [Assigned] (TEZ-3942) RPC getTask writable optimization invalid in hadoop 2.8+

2018-05-31 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles reassigned TEZ-3942: Assignee: Jonathan Eagles > RPC getTask writable optimization invalid in hadoop 2.8+ >

[jira] [Commented] (TEZ-3938) Task attempts failing due to not making progress

2018-05-31 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16496716#comment-16496716 ] Jonathan Eagles commented on TEZ-3938: -- Fix looks good in general. By updating the progress time stamp

[jira] [Updated] (TEZ-3938) Task attempts failing due to not making progress

2018-05-31 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3938: - Target Version/s: 0.9.2 > Task attempts failing due to not making progress >

[jira] [Commented] (TEZ-3944) TestTaskScheduler times-out on Hadoop3

2018-05-29 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16494262#comment-16494262 ] Jonathan Eagles commented on TEZ-3944: -- [~ewohlstadter], if you can reproduce this locally, can you

[jira] [Commented] (TEZ-3944) TestTaskScheduler times-out intermittently

2018-05-29 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16494157#comment-16494157 ] Jonathan Eagles commented on TEZ-3944: -- [~ewohlstadter], can you provide a log of the test failure and

[jira] [Updated] (TEZ-3942) RPC getTask writable optimization invalid in hadoop 2.8+

2018-05-25 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3942: - Attachment: TEZ-3942.001.patch > RPC getTask writable optimization invalid in hadoop 2.8+ >

[jira] [Updated] (TEZ-3942) RPC getTask writable optimization invalid in hadoop 2.8+

2018-05-25 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3942: - Attachment: TEZ-3942.test.patch > RPC getTask writable optimization invalid in hadoop 2.8+ >

[jira] [Created] (TEZ-3942) RPC getTask writable optimization invalid in hadoop 2.8+

2018-05-24 Thread Jonathan Eagles (JIRA)
Jonathan Eagles created TEZ-3942: Summary: RPC getTask writable optimization invalid in hadoop 2.8+ Key: TEZ-3942 URL: https://issues.apache.org/jira/browse/TEZ-3942 Project: Apache Tez

[jira] [Commented] (TEZ-3940) Reduce time to convert TaskFinishedEvent to string

2018-05-23 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16488045#comment-16488045 ] Jonathan Eagles commented on TEZ-3940: -- @jlowe, test failures are expected. Addressed your comments in

[jira] [Commented] (TEZ-3939) Remove performance hit of precondition check in AM for register running task attempt

2018-05-23 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16487821#comment-16487821 ] Jonathan Eagles commented on TEZ-3939: -- [~jlowe], test failures are expected until hadoop 3+ update. >

[jira] [Updated] (TEZ-3940) Reduce time to convert TaskFinishedEvent to string

2018-05-23 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3940: - Attachment: TEZ-3940.003.patch > Reduce time to convert TaskFinishedEvent to string >

[jira] [Assigned] (TEZ-3941) Reduce CPU needed for LegacySpeculator#speculationValue

2018-05-23 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles reassigned TEZ-3941: Assignee: Jonathan Eagles > Reduce CPU needed for LegacySpeculator#speculationValue >

[jira] [Updated] (TEZ-3941) Reduce CPU needed for LegacySpeculator#speculationValue

2018-05-23 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3941: - Description: Found this in 20% of the AM stack traces taken. {noformat} "Dispatcher thread

[jira] [Assigned] (TEZ-3940) Reduce time to convert TaskFinishedEvent to string

2018-05-23 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles reassigned TEZ-3940: Assignee: Jonathan Eagles > Reduce time to convert TaskFinishedEvent to string >

[jira] [Created] (TEZ-3941) Reduce CPU needed for LegacySpeculator#speculationValue

2018-05-23 Thread Jonathan Eagles (JIRA)
Jonathan Eagles created TEZ-3941: Summary: Reduce CPU needed for LegacySpeculator#speculationValue Key: TEZ-3941 URL: https://issues.apache.org/jira/browse/TEZ-3941 Project: Apache Tez Issue

[jira] [Commented] (TEZ-3940) Reduce time to convert TaskFinishedEvent to string

2018-05-23 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16487567#comment-16487567 ] Jonathan Eagles commented on TEZ-3940: -- [~jlowe], Addressed the comments above. In addition, added the

[jira] [Updated] (TEZ-3940) Reduce time to convert TaskFinishedEvent to string

2018-05-23 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3940: - Attachment: TEZ-3940.002.patch > Reduce time to convert TaskFinishedEvent to string >

[jira] [Commented] (TEZ-3939) Remove performance hit of precondition check in AM for register running task attempt

2018-05-23 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16487448#comment-16487448 ] Jonathan Eagles commented on TEZ-3939: -- Thanks, [~jlowe]. Went with the more succinct version like was

[jira] [Updated] (TEZ-3939) Remove performance hit of precondition check in AM for register running task attempt

2018-05-23 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3939: - Attachment: TEZ-3939.002.patch > Remove performance hit of precondition check in AM for register

[jira] [Commented] (TEZ-3940) Reduce time to convert TaskFinishedEvent to string

2018-05-23 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16487344#comment-16487344 ] Jonathan Eagles commented on TEZ-3940: -- Improvement seems to be about 3 seconds of CPU for large jobs

[jira] [Updated] (TEZ-3940) Reduce time to convert TaskFinishedEvent to string

2018-05-23 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3940: - Attachment: TEZ-3940.001.patch > Reduce time to convert TaskFinishedEvent to string >

[jira] [Updated] (TEZ-3940) Reduce time to convert TaskFinishedEvent to string

2018-05-23 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3940: - Description: Found a small CPU improvement while investigating a high CPU AM. {noformat}

[jira] [Updated] (TEZ-3940) Reduce time to convert TaskFinishedEvent to string

2018-05-23 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3940: - Summary: Reduce time to convert TaskFinishedEvent to string (was: Reduce time to convert

[jira] [Created] (TEZ-3940) Reduce time to convert TaskFinishedEvent/TaskAttemptFinishedEvent to string

2018-05-22 Thread Jonathan Eagles (JIRA)
Jonathan Eagles created TEZ-3940: Summary: Reduce time to convert TaskFinishedEvent/TaskAttemptFinishedEvent to string Key: TEZ-3940 URL: https://issues.apache.org/jira/browse/TEZ-3940 Project:

[jira] [Commented] (TEZ-3939) Remove performance hit of precondition check in AM for register running task attempt

2018-05-22 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486672#comment-16486672 ] Jonathan Eagles commented on TEZ-3939: -- These test failures are expected until hadoop dependency is

[jira] [Commented] (TEZ-3939) Remove performance hit of precondition check in AM for register running task attempt

2018-05-22 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486589#comment-16486589 ] Jonathan Eagles commented on TEZ-3939: -- The particular precondition check showed up in a stack trace

[jira] [Updated] (TEZ-3939) Remove performance hit of precondition check in AM for register running task attempt

2018-05-22 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3939: - Target Version/s: 0.9.2, 0.10.0 > Remove performance hit of precondition check in AM for register

[jira] [Updated] (TEZ-3939) Remove performance hit of precondition check in AM for register running task attempt

2018-05-22 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3939: - Attachment: TEZ-3939.001.patch > Remove performance hit of precondition check in AM for register

[jira] [Assigned] (TEZ-3939) Remove performance hit of precondition check in AM for register running task attempt

2018-05-22 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles reassigned TEZ-3939: Assignee: Jonathan Eagles > Remove performance hit of precondition check in AM for register

[jira] [Updated] (TEZ-3939) Remove performance hit of precondition check in AM for register running task attempt

2018-05-22 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3939: - Description: {noformat} java.lang.Thread.State: RUNNABLE at

[jira] [Created] (TEZ-3939) Remove performance hit of precondition check in AM for register running task attempt

2018-05-22 Thread Jonathan Eagles (JIRA)
Jonathan Eagles created TEZ-3939: Summary: Remove performance hit of precondition check in AM for register running task attempt Key: TEZ-3939 URL: https://issues.apache.org/jira/browse/TEZ-3939

[jira] [Commented] (TEZ-3885) Tez timeline server integration error

2018-05-22 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484010#comment-16484010 ] Jonathan Eagles commented on TEZ-3885: -- [~jerryjung], is this still an issue? Seems like we missed the

[jira] [Resolved] (TEZ-3890) Jenkins test patch scripts don't post results to Tez JIRA

2018-05-22 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles resolved TEZ-3890. -- Resolution: Fixed This issue has now been fixed. Thanks for reporting [~ewohlstadter] > Jenkins

[jira] [Commented] (TEZ-3904) an API to update tokens for Tez AM and the DAG

2018-05-22 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483952#comment-16483952 ] Jonathan Eagles commented on TEZ-3904: -- [~sershe], does MR or any other runtime framework allow for

[jira] [Commented] (TEZ-3917) Speculative task attempt's DMEs can cause downstream fetcher to NPE or duplicate fetch

2018-05-22 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483945#comment-16483945 ] Jonathan Eagles commented on TEZ-3917: -- [~kshukla], have you confirmed that this is/isn't a data

[jira] [Commented] (TEZ-3935) DAG aware scheduler should release unassigned new containers rather than hold them

2018-05-22 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483940#comment-16483940 ] Jonathan Eagles commented on TEZ-3935: -- +1. Thanks, [~jlowe]. Committing to branch-0.9 and master

[jira] [Commented] (TEZ-3935) DAG aware scheduler should release unassigned new containers rather than hold them

2018-05-22 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483921#comment-16483921 ] Jonathan Eagles commented on TEZ-3935: -- This will improve the many type of jobs. Does this setting have

[jira] [Resolved] (TEZ-3790) Unrecognized SSL message while building tez

2018-05-22 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles resolved TEZ-3790. -- Resolution: Fixed Believe this to be fixed as stated above. Please reopen if the issue persists.

[jira] [Commented] (TEZ-3902) Upgrade to netty-3.10.5.Final.jar

2018-05-21 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483115#comment-16483115 ] Jonathan Eagles commented on TEZ-3902: -- We expect the above test failures

[jira] [Commented] (TEZ-3937) Empty partition BitSet to byte[] conversion creates one extra byte in rounding error

2018-05-18 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16481018#comment-16481018 ] Jonathan Eagles commented on TEZ-3937: -- TestAMRecovery test is failing intermittently. [~jlowe], I have

[jira] [Updated] (TEZ-3937) Empty partition BitSet to byte[] conversion creates one extra byte in rounding error

2018-05-18 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3937: - Attachment: TEZ-3937.002.patch > Empty partition BitSet to byte[] conversion creates one extra byte

[jira] [Comment Edited] (TEZ-3902) Upgrade to netty-3.10.5.Final.jar

2018-05-17 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479837#comment-16479837 ] Jonathan Eagles edited comment on TEZ-3902 at 5/17/18 11:09 PM: [~jlowe],

[jira] [Commented] (TEZ-3902) Upgrade to netty-3.10.5.Final.jar

2018-05-17 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479837#comment-16479837 ] Jonathan Eagles commented on TEZ-3902: -- [~jlowe], now that we have 0.10 as the hadoop 3+ line. Please

[jira] [Created] (TEZ-3938) Task attempts failing due to not making progress

2018-05-17 Thread Jonathan Eagles (JIRA)
Jonathan Eagles created TEZ-3938: Summary: Task attempts failing due to not making progress Key: TEZ-3938 URL: https://issues.apache.org/jira/browse/TEZ-3938 Project: Apache Tez Issue Type:

[jira] [Updated] (TEZ-3937) Empty partition BitSet to byte[] conversion creates one extra byte in rounding error

2018-05-16 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3937: - Attachment: TEZ-3937.001.patch > Empty partition BitSet to byte[] conversion creates one extra byte

[jira] [Updated] (TEZ-3937) Empty partition BitSet to byte[] conversion creates one extra byte in rounding error

2018-05-16 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-3937: - Description: Byte array length calculation is defined as (bitset.length / 8) + 1 which has off by

<    1   2   3   4   5   6   7   8   9   10   >