[jira] [Commented] (TEZ-3904) an API to update tokens for Tez AM and the DAG

2018-06-07 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505201#comment-16505201 ] Jaume M commented on TEZ-3904: -- Spark seems to [push from the

[jira] [Updated] (TEZ-3904) an API to update tokens for Tez AM and the DAG

2018-06-13 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M updated TEZ-3904: - Attachment: TEZ-3904.1.patch > an API to update tokens for Tez AM and the DAG >

[jira] [Commented] (TEZ-3958) Add internal vertex priority information into the tez dag.dot debug information

2018-06-25 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522585#comment-16522585 ] Jaume M commented on TEZ-3958: -- [~gopalv] looking at the code seems like the priority is something task

[jira] [Updated] (TEZ-3958) Add internal vertex priority information into the tez dag.dot debug information

2018-07-02 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M updated TEZ-3958: - Attachment: TEZ-3958.1.patch > Add internal vertex priority information into the tez dag.dot debug > information

[jira] [Updated] (TEZ-3958) Add internal vertex priority information into the tez dag.dot debug information

2018-08-01 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M updated TEZ-3958: - Attachment: (was: TEZ-3958.3.patch) > Add internal vertex priority information into the tez dag.dot debug >

[jira] [Updated] (TEZ-3976) ShuffleManager reporting too many errors

2018-08-06 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M updated TEZ-3976: - Summary: ShuffleManager reporting too many errors (was: {{ShuffleManager}} reporting too many errors) >

[jira] [Created] (TEZ-3976) {{ShuffleManager}} reporting too many errors

2018-08-06 Thread Jaume M (JIRA)
Jaume M created TEZ-3976: Summary: {{ShuffleManager}} reporting too many errors Key: TEZ-3976 URL: https://issues.apache.org/jira/browse/TEZ-3976 Project: Apache Tez Issue Type: Bug

[jira] [Commented] (TEZ-3976) ShuffleManager reporting too many errors

2018-08-07 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16572043#comment-16572043 ] Jaume M commented on TEZ-3976: -- [~jlowe] for clarification the OOM is in the AM. Some tasks are killed by the

[jira] [Created] (TEZ-3979) Redact configuration when it's being logged, sent to the timeline server or displayed in some way

2018-08-15 Thread Jaume M (JIRA)
Jaume M created TEZ-3979: Summary: Redact configuration when it's being logged, sent to the timeline server or displayed in some way Key: TEZ-3979 URL: https://issues.apache.org/jira/browse/TEZ-3979 Project:

[jira] [Assigned] (TEZ-3979) Redact configuration when it's being logged, sent to the timeline server or displayed in some way

2018-08-15 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M reassigned TEZ-3979: Assignee: Jaume M > Redact configuration when it's being logged, sent to the timeline server or >

[jira] [Assigned] (TEZ-3976) ShuffleManager reporting too many errors

2018-08-14 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M reassigned TEZ-3976: Assignee: Jaume M > ShuffleManager reporting too many errors > > >

[jira] [Commented] (TEZ-3979) Redact configuration when it's being logged, sent to the timeline server or displayed in some way

2018-08-16 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16583031#comment-16583031 ] Jaume M commented on TEZ-3979: -- Finally added a new property for tez({{tez.dag.redacted-properties}}) instead

[jira] [Commented] (TEZ-3974) Tez: Correctness regression of TEZ-955 in TEZ-2937

2018-08-06 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570380#comment-16570380 ] Jaume M commented on TEZ-3974: -- Sure, to reproduce this the following step where taken: * Create a DAG with

[jira] [Commented] (TEZ-3974) Tez: Correctness regression of TEZ-955 in TEZ-2937

2018-08-06 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570649#comment-16570649 ] Jaume M commented on TEZ-3974: -- Added a test [~jeagles] to verify that the empty events aren't sent from the

[jira] [Created] (TEZ-3981) UnorderedPartitionedKVWriter.getInitialMemoryRequirement may return negative memory

2018-08-20 Thread Jaume M (JIRA)
Jaume M created TEZ-3981: Summary: UnorderedPartitionedKVWriter.getInitialMemoryRequirement may return negative memory Key: TEZ-3981 URL: https://issues.apache.org/jira/browse/TEZ-3981 Project: Apache Tez

[jira] [Assigned] (TEZ-3981) UnorderedPartitionedKVWriter.getInitialMemoryRequirement may return negative memory

2018-08-20 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M reassigned TEZ-3981: Assignee: Jaume M > UnorderedPartitionedKVWriter.getInitialMemoryRequirement may return negative > memory

[jira] [Assigned] (TEZ-3969) TaskAttemptImpl: static fields initialized in instance ctor

2018-08-20 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M reassigned TEZ-3969: Assignee: Jaume M > TaskAttemptImpl: static fields initialized in instance ctor >

[jira] [Assigned] (TEZ-3985) Correctness: Throw a clear exception for DMEs sent during cleanup

2018-08-27 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M reassigned TEZ-3985: Assignee: Jaume M > Correctness: Throw a clear exception for DMEs sent during cleanup >

[jira] [Assigned] (TEZ-3984) Shuffle: Out of Band DME event sending causes errors

2018-08-29 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M reassigned TEZ-3984: Assignee: Jaume M Attachment: TEZ-3984.1.patch > Shuffle: Out of Band DME event sending causes errors

[jira] [Assigned] (TEZ-3987) Schedule giving priorities based on topological order

2018-09-05 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M reassigned TEZ-3987: Assignee: Jaume M > Schedule giving priorities based on topological order >

[jira] [Created] (TEZ-3987) Schedule giving priorities based on topological order

2018-09-05 Thread Jaume M (JIRA)
Jaume M created TEZ-3987: Summary: Schedule giving priorities based on topological order Key: TEZ-3987 URL: https://issues.apache.org/jira/browse/TEZ-3987 Project: Apache Tez Issue Type: New Feature

[jira] [Commented] (TEZ-3987) Schedule giving priorities based on topological order

2018-09-05 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605143#comment-16605143 ] Jaume M commented on TEZ-3987: -- Thanks for pointing me to that [~jlowe], I wasn't aware of it. It's definitely

[jira] [Commented] (TEZ-3984) Shuffle: Out of Band DME event sending causes errors

2018-08-29 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596774#comment-16596774 ] Jaume M commented on TEZ-3984: -- Yeah, good point > Shuffle: Out of Band DME event sending causes errors >

[jira] [Commented] (TEZ-3964) Inflater not closed in some places

2018-07-05 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16534215#comment-16534215 ] Jaume M commented on TEZ-3964: -- Thank you for reviewing [~jlowe]. Looks like the method doesn't throw

[jira] [Assigned] (TEZ-3904) an API to update tokens for Tez AM and the DAG

2018-07-05 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M reassigned TEZ-3904: Assignee: Jaume M > an API to update tokens for Tez AM and the DAG >

[jira] [Assigned] (TEZ-3884) Hadoop3-beta1 fixes for Tez tests

2018-07-10 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M reassigned TEZ-3884: Assignee: Jaume M > Hadoop3-beta1 fixes for Tez tests > - > >

[jira] [Assigned] (TEZ-3866) add ability to pass information between plugins

2018-07-10 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M reassigned TEZ-3866: Assignee: Jaume M > add ability to pass information between plugins >

[jira] [Assigned] (TEZ-3884) Hadoop3-beta1 fixes for Tez tests

2018-07-10 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M reassigned TEZ-3884: Assignee: (was: Jaume M) > Hadoop3-beta1 fixes for Tez tests > - > >

[jira] [Assigned] (TEZ-3965) TestMROutput: Fix the hard-coded "/tmp/output" paths

2018-07-12 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M reassigned TEZ-3965: Assignee: Jaume M > TestMROutput: Fix the hard-coded "/tmp/output" paths >

[jira] [Commented] (TEZ-3967) DAGImpl: dag lock is unfair and can starve the writers

2018-07-12 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541957#comment-16541957 ] Jaume M commented on TEZ-3967: -- [~gopalv] the description of the problem makes sense to me but I'm having

[jira] [Reopened] (TEZ-3984) Shuffle: Out of Band DME event sending causes errors

2018-09-11 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M reopened TEZ-3984: -- > Shuffle: Out of Band DME event sending causes errors > > >

[jira] [Created] (TEZ-3963) Possible InflaterInputStream leaked in TezCommonUtils and related classes

2018-06-28 Thread Jaume M (JIRA)
Jaume M created TEZ-3963: Summary: Possible InflaterInputStream leaked in TezCommonUtils and related classes Key: TEZ-3963 URL: https://issues.apache.org/jira/browse/TEZ-3963 Project: Apache Tez

[jira] [Created] (TEZ-3964) Inflater not closed in some places

2018-06-28 Thread Jaume M (JIRA)
Jaume M created TEZ-3964: Summary: Inflater not closed in some places Key: TEZ-3964 URL: https://issues.apache.org/jira/browse/TEZ-3964 Project: Apache Tez Issue Type: Bug Affects Versions:

[jira] [Commented] (TEZ-3979) Redact configuration when it's being logged, sent to the timeline server or displayed in some way

2018-10-17 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16653779#comment-16653779 ] Jaume M commented on TEZ-3979: -- Thank you for the review [~harishjp], you mean redacting as well

[jira] [Updated] (TEZ-3976) ShuffleManager reporting too many errors

2018-10-19 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M updated TEZ-3976: - Attachment: (was: HIVE-20701.7.patch) > ShuffleManager reporting too many errors >

[jira] [Created] (TEZ-4015) Send killed diagnostics to the AM when ShuffleScheduler calls killSelf

2018-10-30 Thread Jaume M (JIRA)
Jaume M created TEZ-4015: Summary: Send killed diagnostics to the AM when ShuffleScheduler calls killSelf Key: TEZ-4015 URL: https://issues.apache.org/jira/browse/TEZ-4015 Project: Apache Tez Issue

[jira] [Commented] (TEZ-4004) Update jetty9 to align with Hadoop and Hive

2018-10-29 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16667442#comment-16667442 ] Jaume M commented on TEZ-4004: -- [~jeagles] looks like that. I'll reopen 

[jira] [Reopened] (TEZ-3995) Fix dot files produced by tests to prevent ASF license warnings in yetus

2018-10-29 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M reopened TEZ-3995: -- > Fix dot files produced by tests to prevent ASF license warnings in yetus >

[jira] [Commented] (TEZ-4004) Update jetty9 to align with Hadoop and Hive

2018-10-29 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16667443#comment-16667443 ] Jaume M commented on TEZ-4004: -- [~jeagles] looks like that. I'll reopen 

[jira] [Issue Comment Deleted] (TEZ-4004) Update jetty9 to align with Hadoop and Hive

2018-10-29 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M updated TEZ-4004: - Comment: was deleted (was: [~jeagles] looks like that. I'll reopen 

[jira] [Commented] (TEZ-3831) Reduce Unordered memory needed for storing empty completed events

2018-11-12 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684523#comment-16684523 ] Jaume M commented on TEZ-3831: -- [~jeagles] I'm looking into a bug in which tez hangs [in this take()|

[jira] [Commented] (TEZ-3979) Redact configuration when it's being logged, sent to the timeline server or displayed in some way

2018-10-02 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16636246#comment-16636246 ] Jaume M commented on TEZ-3979: -- [~harishjp] could you take a look at this and advise if you think there's

[jira] [Commented] (TEZ-3995) Fix dot files produced by tests to prevent ASF license warnings in yetus

2018-09-28 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632631#comment-16632631 ] Jaume M commented on TEZ-3995: -- Oh I see it now [~jeagles], the .dot happened in a [different

[jira] [Assigned] (TEZ-3995) Fix dot files produced by tests to prevent ASF license warnings in yetus

2018-09-28 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M reassigned TEZ-3995: Assignee: Jaume M > Fix dot files produced by tests to prevent ASF license warnings in yetus >

[jira] [Commented] (TEZ-3995) Fix dot files produced by tests to prevent ASF license warnings in yetus

2018-10-01 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16634321#comment-16634321 ] Jaume M commented on TEZ-3995: -- This will need a different patch for 0.9. Also we'll have to wait until

[jira] [Commented] (TEZ-3979) Redact configuration when it's being logged, sent to the timeline server or displayed in some way

2018-10-08 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642303#comment-16642303 ] Jaume M commented on TEZ-3979: -- ping [~harishjp] > Redact configuration when it's being logged, sent to the

[jira] [Reopened] (TEZ-3995) Fix dot files produced by tests to prevent ASF license warnings in yetus

2018-10-09 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M reopened TEZ-3995: -- > Fix dot files produced by tests to prevent ASF license warnings in yetus >

[jira] [Commented] (TEZ-3995) Fix dot files produced by tests to prevent ASF license warnings in yetus

2018-10-09 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16644151#comment-16644151 ] Jaume M commented on TEZ-3995: -- [~jeagles] thank you for the review and for committing, I believe this is only

[jira] [Updated] (TEZ-3984) Shuffle: Out of Band DME event sending causes errors

2018-08-31 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M updated TEZ-3984: - Attachment: (was: TEZ-3984.2.patch) > Shuffle: Out of Band DME event sending causes errors >

[jira] [Commented] (TEZ-3995) Fix dot files produced by tests to prevent ASF license warnings in yetus

2018-09-28 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632158#comment-16632158 ] Jaume M commented on TEZ-3995: -- https://issues.apache.org/jira/browse/TEZ-3958 will generate a new dot file in

[jira] [Comment Edited] (TEZ-3995) Fix dot files produced by tests to prevent ASF license warnings in yetus

2018-09-28 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632158#comment-16632158 ] Jaume M edited comment on TEZ-3995 at 9/28/18 5:32 PM: --- TEZ-3958 will generate a new

[jira] [Commented] (TEZ-3831) Reduce Unordered memory needed for storing empty completed events

2018-12-04 Thread Jaume M (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-3831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16709449#comment-16709449 ] Jaume M commented on TEZ-3831: -- {code} 2018-08-24T06:53:46,757 WARN [pool-156-thread-4 ()]