[jira] [Updated] (HIVE-14246) Tez: disable auto-reducer parallelism when CUSTOM_EDGE is in place
[ https://issues.apache.org/jira/browse/HIVE-14246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stamatis Zampetakis updated HIVE-14246: --- Fix Version/s: (was: 3.2.0) I cleared the fixVersion field since this ticket is not resolved. Please review this ticket and if the fix is already committed to a specific version please set the version accordingly and mark the ticket as RESOLVED. According to the JIRA guidelines (https://cwiki.apache.org/confluence/display/Hive/HowToContribute) the fixVersion should be set only when the issue is resolved/closed. > Tez: disable auto-reducer parallelism when CUSTOM_EDGE is in place > -- > > Key: HIVE-14246 > URL: https://issues.apache.org/jira/browse/HIVE-14246 > Project: Hive > Issue Type: Bug > Components: Tez >Affects Versions: 2.2.0 >Reporter: Gopal Vijayaraghavan >Assignee: Gopal Vijayaraghavan >Priority: Minor > Attachments: HIVE-14246.1.patch > > > The CUSTOM_SIMPLE_EDGE impl has differences between the size constraints of > either edge which cannot be represented by the ShuffleVertexManager presently. > Reducing the width based on the hashtable build side vs the streaming probe > side have different consequences since there is no order of runtime between > them. > Until the two parent vertices of the shuffle hash-join are related, this > feature causes massive inconsistency of performance across runs. > For inner & semi joins, the hashtable side should have a higher priority than > the streaming side and for left outer joins, the streaming side can over-take > the hashtable side, being the more dominant factor in the final row-counts. > Until such priorities can be bubbled up into ShuffleVertexManager, this > feature can be disabled. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-14246) Tez: disable auto-reducer parallelism when CUSTOM_EDGE is in place
[ https://issues.apache.org/jira/browse/HIVE-14246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-14246: --- Fix Version/s: (was: 3.1.0) 3.2.0 Deferring this to 3.2.0 since the branch for 3.1.0 has been cut off. > Tez: disable auto-reducer parallelism when CUSTOM_EDGE is in place > -- > > Key: HIVE-14246 > URL: https://issues.apache.org/jira/browse/HIVE-14246 > Project: Hive > Issue Type: Bug > Components: Tez >Affects Versions: 2.2.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Minor > Fix For: 3.2.0 > > Attachments: HIVE-14246.1.patch > > > The CUSTOM_SIMPLE_EDGE impl has differences between the size constraints of > either edge which cannot be represented by the ShuffleVertexManager presently. > Reducing the width based on the hashtable build side vs the streaming probe > side have different consequences since there is no order of runtime between > them. > Until the two parent vertices of the shuffle hash-join are related, this > feature causes massive inconsistency of performance across runs. > For inner & semi joins, the hashtable side should have a higher priority than > the streaming side and for left outer joins, the streaming side can over-take > the hashtable side, being the more dominant factor in the final row-counts. > Until such priorities can be bubbled up into ShuffleVertexManager, this > feature can be disabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-14246) Tez: disable auto-reducer parallelism when CUSTOM_EDGE is in place
[ https://issues.apache.org/jira/browse/HIVE-14246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-14246: --- Fix Version/s: (was: 3.0.0) 3.1.0 Deferring this to 3.1.0 since the branch for 3.0.0 has been cut off. Please update the JIRA if you would like to get your patch in 3.0.0. > Tez: disable auto-reducer parallelism when CUSTOM_EDGE is in place > -- > > Key: HIVE-14246 > URL: https://issues.apache.org/jira/browse/HIVE-14246 > Project: Hive > Issue Type: Bug > Components: Tez >Affects Versions: 2.2.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Minor > Fix For: 3.1.0 > > Attachments: HIVE-14246.1.patch > > > The CUSTOM_SIMPLE_EDGE impl has differences between the size constraints of > either edge which cannot be represented by the ShuffleVertexManager presently. > Reducing the width based on the hashtable build side vs the streaming probe > side have different consequences since there is no order of runtime between > them. > Until the two parent vertices of the shuffle hash-join are related, this > feature causes massive inconsistency of performance across runs. > For inner & semi joins, the hashtable side should have a higher priority than > the streaming side and for left outer joins, the streaming side can over-take > the hashtable side, being the more dominant factor in the final row-counts. > Until such priorities can be bubbled up into ShuffleVertexManager, this > feature can be disabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-14246) Tez: disable auto-reducer parallelism when CUSTOM_EDGE is in place
[ https://issues.apache.org/jira/browse/HIVE-14246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-14246: --- Priority: Minor (was: Major) > Tez: disable auto-reducer parallelism when CUSTOM_EDGE is in place > -- > > Key: HIVE-14246 > URL: https://issues.apache.org/jira/browse/HIVE-14246 > Project: Hive > Issue Type: Bug > Components: Tez >Affects Versions: 2.2.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Minor > Fix For: 2.2.0 > > Attachments: HIVE-14246.1.patch > > > The CUSTOM_SIMPLE_EDGE impl has differences between the size constraints of > either edge which cannot be represented by the ShuffleVertexManager presently. > Reducing the width based on the hashtable build side vs the streaming probe > side have different consequences since there is no order of runtime between > them. > Until the two parent vertices of the shuffle hash-join are related, this > feature causes massive inconsistency of performance across runs. > For inner & semi joins, the hashtable side should have a higher priority than > the streaming side and for left outer joins, the streaming side can over-take > the hashtable side, being the more dominant factor in the final row-counts. > Until such priorities can be bubbled up into ShuffleVertexManager, this > feature can be disabled. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14246) Tez: disable auto-reducer parallelism when CUSTOM_EDGE is in place
[ https://issues.apache.org/jira/browse/HIVE-14246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-14246: --- Status: Patch Available (was: Open) > Tez: disable auto-reducer parallelism when CUSTOM_EDGE is in place > -- > > Key: HIVE-14246 > URL: https://issues.apache.org/jira/browse/HIVE-14246 > Project: Hive > Issue Type: Bug > Components: Tez >Affects Versions: 2.2.0 >Reporter: Gopal V >Assignee: Gopal V > Fix For: 2.2.0 > > Attachments: HIVE-14246.1.patch > > > The CUSTOM_SIMPLE_EDGE impl has differences between the size constraints of > either edge which cannot be represented by the ShuffleVertexManager presently. > Reducing the width based on the hashtable build side vs the streaming probe > side have different consequences since there is no order of runtime between > them. > Until the two parent vertices of the shuffle hash-join are related, this > feature causes massive inconsistency of performance across runs. > For inner & semi joins, the hashtable side should have a higher priority than > the streaming side and for left outer joins, the streaming side can over-take > the hashtable side, being the more dominant factor in the final row-counts. > Until such priorities can be bubbled up into ShuffleVertexManager, this > feature can be disabled. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14246) Tez: disable auto-reducer parallelism when CUSTOM_EDGE is in place
[ https://issues.apache.org/jira/browse/HIVE-14246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-14246: --- Attachment: HIVE-14246.1.patch > Tez: disable auto-reducer parallelism when CUSTOM_EDGE is in place > -- > > Key: HIVE-14246 > URL: https://issues.apache.org/jira/browse/HIVE-14246 > Project: Hive > Issue Type: Bug > Components: Tez >Affects Versions: 2.2.0 >Reporter: Gopal V >Assignee: Gopal V > Fix For: 2.2.0 > > Attachments: HIVE-14246.1.patch > > > The CUSTOM_SIMPLE_EDGE impl has differences between the size constraints of > either edge which cannot be represented by the ShuffleVertexManager presently. > Reducing the width based on the hashtable build side vs the streaming probe > side have different consequences since there is no order of runtime between > them. > Until the two parent vertices of the shuffle hash-join are related, this > feature causes massive inconsistency of performance across runs. > For inner & semi joins, the hashtable side should have a higher priority than > the streaming side and for left outer joins, the streaming side can over-take > the hashtable side, being the more dominant factor in the final row-counts. > Until such priorities can be bubbled up into ShuffleVertexManager, this > feature can be disabled. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14246) Tez: disable auto-reducer parallelism when CUSTOM_EDGE is in place
[ https://issues.apache.org/jira/browse/HIVE-14246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-14246: --- Description: The CUSTOM_SIMPLE_EDGE impl has differences between the size constraints of either edge which cannot be represented by the ShuffleVertexManager presently. Reducing the width based on the hashtable build side vs the streaming probe side have different consequences since there is no order of runtime between them. Until the two parent vertices of the shuffle hash-join are related, this feature causes massive inconsistency of performance across runs. For inner & semi joins, the hashtable side should have a higher priority than the streaming side and for left outer joins, the streaming side can over-take the hashtable side, being the more dominant factor in the final row-counts. Until such priorities can be bubbled up into ShuffleVertexManager, this feature can be disabled. was: The CUSTOM_SIMPLE_EDGE impl has differences between the size constraints of either edge which cannot be represented by the ShuffleVertexManager presently. Reducing the width based on the hashtable build side vs the streaming probe side have different consequences since there is no order of runtime between them. Until the two parent vertices of the shuffle hash-join are related, this feature causes massive inconsistency of performance across runs. > Tez: disable auto-reducer parallelism when CUSTOM_EDGE is in place > -- > > Key: HIVE-14246 > URL: https://issues.apache.org/jira/browse/HIVE-14246 > Project: Hive > Issue Type: Bug > Components: Tez >Affects Versions: 2.2.0 >Reporter: Gopal V >Assignee: Gopal V > Fix For: 2.2.0 > > > The CUSTOM_SIMPLE_EDGE impl has differences between the size constraints of > either edge which cannot be represented by the ShuffleVertexManager presently. > Reducing the width based on the hashtable build side vs the streaming probe > side have different consequences since there is no order of runtime between > them. > Until the two parent vertices of the shuffle hash-join are related, this > feature causes massive inconsistency of performance across runs. > For inner & semi joins, the hashtable side should have a higher priority than > the streaming side and for left outer joins, the streaming side can over-take > the hashtable side, being the more dominant factor in the final row-counts. > Until such priorities can be bubbled up into ShuffleVertexManager, this > feature can be disabled. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
