[jira] [Commented] (TEZ-3818) Support a new data routing policy for small partitions

[ https://issues.apache.org/jira/browse/TEZ-3818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126805#comment-16126805 ] TezQA commented on TEZ-3818: {color:red}-1 overall{color}. Here are the results of testing the latest

Failed: TEZ-3818 PreCommit Build #2616

Jira: https://issues.apache.org/jira/browse/TEZ-3818 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/2616/ ### ## LAST 60 LINES OF THE CONSOLE ###

[jira] [Updated] (TEZ-3818) Support a new data routing policy for small partitions

[ https://issues.apache.org/jira/browse/TEZ-3818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated TEZ-3818: - Attachment: TEZ-3818.patch Here is the draft patch. Besides the added unit test, I tested on a hadoop cluster with

[jira] [Commented] (TEZ-3819) Round robin partitioner make fair cartesian product not fault tolerant

[ https://issues.apache.org/jira/browse/TEZ-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126614#comment-16126614 ] Zhiyuan Yang commented on TEZ-3819: --- [~gopalv] Thanks for pointing out this! I wasn't aware this in Hive.

[jira] [Commented] (TEZ-3817) DAGs can hang after more than one uncaught Exception during doTransition.

[ https://issues.apache.org/jira/browse/TEZ-3817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126600#comment-16126600 ] TezQA commented on TEZ-3817: {color:red}-1 overall{color}. Here are the results of testing the latest

Failed: TEZ-3817 PreCommit Build #2615

Jira: https://issues.apache.org/jira/browse/TEZ-3817 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/2615/ ### ## LAST 60 LINES OF THE CONSOLE ###

[jira] [Commented] (TEZ-3819) Round robin partitioner make fair cartesian product not fault tolerant

[ https://issues.apache.org/jira/browse/TEZ-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126590#comment-16126590 ] Gopal V commented on TEZ-3819: -- bq. although skew caused by repeated key becomes inevitable. Is this using

[jira] [Commented] (TEZ-3819) Round robin partitioner make fair cartesian product not fault tolerant

[ https://issues.apache.org/jira/browse/TEZ-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126580#comment-16126580 ] TezQA commented on TEZ-3819: {color:red}-1 overall{color}. Here are the results of testing the latest

Failed: TEZ-3819 PreCommit Build #2614

Jira: https://issues.apache.org/jira/browse/TEZ-3819 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/2614/ ### ## LAST 60 LINES OF THE CONSOLE ###

[jira] [Commented] (TEZ-3819) Round robin partitioner make fair cartesian product not fault tolerant

[ https://issues.apache.org/jira/browse/TEZ-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126485#comment-16126485 ] Gopal V commented on TEZ-3819: -- [~aplusplus]: You're right, any edge in the 2nd stage isn't safe to re-run. >

[jira] [Commented] (TEZ-3819) Round robin partitioner make fair cartesian product not fault tolerant

[ https://issues.apache.org/jira/browse/TEZ-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126477#comment-16126477 ] Zhiyuan Yang commented on TEZ-3819: --- Yep. Another part is fetch order is guaranteed, which may also affect

[jira] [Comment Edited] (TEZ-3819) Round robin partitioner make fair cartesian product not fault tolerant

[ https://issues.apache.org/jira/browse/TEZ-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126473#comment-16126473 ] Gopal V edited comment on TEZ-3819 at 8/14/17 9:52 PM: --- Is the problem with the

[jira] [Commented] (TEZ-3819) Round robin partitioner make fair cartesian product not fault tolerant

[ https://issues.apache.org/jira/browse/TEZ-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126473#comment-16126473 ] Gopal V commented on TEZ-3819: -- Is the problem with the previous edge (i.e MRInput offers input order, but

[jira] [Commented] (TEZ-3819) Round robin partitioner make fair cartesian product not fault tolerant

[ https://issues.apache.org/jira/browse/TEZ-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126466#comment-16126466 ] Gopal V commented on TEZ-3819: -- bq. Round robin partitioner cannot give consistent partitioning Why?

[jira] [Updated] (TEZ-3817) DAGs can hang after more than one uncaught Exception during doTransition.

[ https://issues.apache.org/jira/browse/TEZ-3817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated TEZ-3817: - Attachment: TEZ-3817.001.patch Attaching first cut of the patch that catches exceptions in

[jira] [Updated] (TEZ-3819) Round robin partitioner make fair cartesian product not fault tolerant

[ https://issues.apache.org/jira/browse/TEZ-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhiyuan Yang updated TEZ-3819: -- Attachment: TEZ-3819.patch > Round robin partitioner make fair cartesian product not fault tolerant >

[jira] [Comment Edited] (TEZ-3819) Round robin partitioner make fair cartesian product not fault tolerant

[ https://issues.apache.org/jira/browse/TEZ-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126411#comment-16126411 ] Zhiyuan Yang edited comment on TEZ-3819 at 8/14/17 9:16 PM: Using hash

[jira] [Commented] (TEZ-3819) Round robin partitioner make fair cartesian product not fault tolerant

[ https://issues.apache.org/jira/browse/TEZ-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126411#comment-16126411 ] Zhiyuan Yang commented on TEZ-3819: --- Using hash partitioner can be a short term resolution, although skew

[jira] [Created] (TEZ-3819) Round robin partitioner make fair cartesian product not fault tolerant

Zhiyuan Yang created TEZ-3819: - Summary: Round robin partitioner make fair cartesian product not fault tolerant Key: TEZ-3819 URL: https://issues.apache.org/jira/browse/TEZ-3819 Project: Apache Tez

[jira] [Commented] (TEZ-3816) Ability to automatically speculate single-task vertices

[ https://issues.apache.org/jira/browse/TEZ-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126181#comment-16126181 ] TezQA commented on TEZ-3816: {color:green}+1 overall{color}. Here are the results of testing the latest

Success: TEZ-3816 PreCommit Build #2613

Jira: https://issues.apache.org/jira/browse/TEZ-3816 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/2613/ ### ## LAST 60 LINES OF THE CONSOLE ###

[jira] [Commented] (TEZ-3816) Ability to automatically speculate single-task vertices

[ https://issues.apache.org/jira/browse/TEZ-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126168#comment-16126168 ] Siddharth Seth commented on TEZ-3816: - +1. Thanks [~samkhan]. Will commit once jenkins gets back. >

[jira] [Commented] (TEZ-3159) Reduce memory utilization while serializing keys and values

[ https://issues.apache.org/jira/browse/TEZ-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126065#comment-16126065 ] Rohini Palaniswamy commented on TEZ-3159: - Some more comments: Just to be clear, when you

[jira] [Created] (TEZ-3818) Support a new data routing policy for small partitions

Ming Ma created TEZ-3818: Summary: Support a new data routing policy for small partitions Key: TEZ-3818 URL: https://issues.apache.org/jira/browse/TEZ-3818 Project: Apache Tez Issue Type: Sub-task

[jira] [Updated] (TEZ-3816) Ability to automatically speculate single-task vertices

[ https://issues.apache.org/jira/browse/TEZ-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Muhammad Samir Khan updated TEZ-3816: - Attachment: TEZ-3816.003.patch Added a couple of comments and speculationValue returns

[jira] [Commented] (TEZ-3159) Reduce memory utilization while serializing keys and values

[ https://issues.apache.org/jira/browse/TEZ-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126007#comment-16126007 ] Rohini Palaniswamy commented on TEZ-3159: - Actually my comment on MAX_BUFFER_SIZE may not be right.

[jira] [Updated] (TEZ-3816) Ability to automatically speculate single-task vertices

[ https://issues.apache.org/jira/browse/TEZ-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Muhammad Samir Khan updated TEZ-3816: - Attachment: TEZ-3816.002.patch Thanks [~sseth] for the comments. I updated the patch to put

[jira] [Commented] (TEZ-3159) Reduce memory utilization while serializing keys and values

[ https://issues.apache.org/jira/browse/TEZ-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16125983#comment-16125983 ] Rohini Palaniswamy commented on TEZ-3159: - General comments: 1) We need a equivalent class for

[jira] [Commented] (TEZ-3431) Add unit tests for container release

[ https://issues.apache.org/jira/browse/TEZ-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16125970#comment-16125970 ] Taklon Stephen Wu commented on TEZ-3431: the second patch TEZ-3431.1.patch was generated with