[jira] [Commented] (HIVE-20252) Semijoin Reduction : Cycles due to semi join branch may remain undetected if small table side has a map join upstream.
[ https://issues.apache.org/jira/browse/HIVE-20252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566321#comment-16566321 ] Hive QA commented on HIVE-20252: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12933979/HIVE-20252.01-branch-3.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12992/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12992/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12992/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Tests exited with: Exception: Patch URL https://issues.apache.org/jira/secure/attachment/12933979/HIVE-20252.01-branch-3.patch was found in seen patch url's cache and a test was probably run already on it. Aborting... {noformat} This message is automatically generated. ATTACHMENT ID: 12933979 - PreCommit-HIVE-Build > Semijoin Reduction : Cycles due to semi join branch may remain undetected if > small table side has a map join upstream. > -- > > Key: HIVE-20252 > URL: https://issues.apache.org/jira/browse/HIVE-20252 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-20252.01-branch-3.patch, HIVE-20252.1.patch, > HIVE-20252.2.patch, HIVE-20252.3.patch, HIVE-20252.4.patch, > HIVE-20252.5.patch, HIVE-20252.6.patch > > > For eg, > {noformat} > # 2018-07-26T17:22:14,664 DEBUG [51377701-dc98-424f-82e0-bbb5d6c84316 main] > optimizer.SharedWorkOptimizer: Before SharedWorkOptimizer: > # > TS[0]-FIL[96]-SEL[2]-MAPJOIN[156]-MAPJOIN[157]-MAPJOIN[161]-MAPJOIN[162]-FIL[47]-SEL[48]-MAPJOIN[163]-FIL[66]-SEL[67]-TNK[105]-GBY[68]-RS[69]-GBY[70]-SEL[71]-RS[72]-SEL[73]-LIM[74]-FS[75] > # > -SEL[142]-GBY[143]-RS[144]-GBY[145]-RS[155] > # TS[3]-FIL[97]-SEL[5]-RS[34]-MAPJOIN[156] > # TS[6]-FIL[98]-SEL[8]-RS[37]-MAPJOIN[157] > # TS[9]-FIL[99]-SEL[11]-MAPJOIN[158]-GBY[40]-RS[42]-MAPJOIN[161] > # TS[12]-FIL[100]-SEL[14]-RS[16]-MAPJOIN[158] > # -SEL[131]-GBY[132]-EVENT[133] > # > TS[19]-FIL[101]-SEL[21]-MAPJOIN[159]-GBY[29]-RS[30]-GBY[31]-SEL[32]-RS[45]-MAPJOIN[162] > # TS[22]-FIL[102]-SEL[24]-RS[26]-MAPJOIN[159] > # -SEL[139]-GBY[140]-EVENT[141] > # > TS[49]-FIL[103]-SEL[51]-MAPJOIN[160]-GBY[59]-RS[60]-GBY[61]-SEL[62]-RS[64]-MAPJOIN[163] > # TS[52]-FIL[104]-SEL[54]-RS[56]-MAPJOIN[160] > # -SEL[147]-GBY[148]-EVENT[149] > # > # > # DPP information stored in the cache: \{TS[19]=[EVENT[141]], > TS[9]=[EVENT[133]], TS[49]=[RS[155], EVENT[149]]} > {noformat} > > The semi join branch in line 3 feeds into TS[49] in line 12 which feeds to > MAPJOIN[163] going back to parent of the semi join branch at line 2. > The logic to detect cycle may fail as there is a MAPJOIN[160] at line 12 > which could cause the logic to look for wrong TS. The logic to find TS > operator upstream must use findOperatorsUpstream() and examine each TS Op for > complete coverage. > Simplified image of task-cycle, without operator cycles - > http://people.apache.org/~gopalv/HIVE_20252_cycle1.svg > And the artificial edge introduced to trigger cycle detection (in red) - > http://people.apache.org/~gopalv/HIVE_20252_cycle_fix.svg > cc [~jcamachorodriguez] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20252) Semijoin Reduction : Cycles due to semi join branch may remain undetected if small table side has a map join upstream.
[ https://issues.apache.org/jira/browse/HIVE-20252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566149#comment-16566149 ] Hive QA commented on HIVE-20252: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12933979/HIVE-20252.01-branch-3.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 100 failed/errored test(s), 14407 tests executed *Failed tests:* {noformat} TestAddPartitions - did not produce a TEST-*.xml file (likely timed out) (batchId=227) TestAddPartitionsFromPartSpec - did not produce a TEST-*.xml file (likely timed out) (batchId=229) TestAdminUser - did not produce a TEST-*.xml file (likely timed out) (batchId=235) TestAggregateStatsCache - did not produce a TEST-*.xml file (likely timed out) (batchId=229) TestAlterPartitions - did not produce a TEST-*.xml file (likely timed out) (batchId=229) TestAppendPartitions - did not produce a TEST-*.xml file (likely timed out) (batchId=229) TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=272) TestCachedStore - did not produce a TEST-*.xml file (likely timed out) (batchId=235) TestCatalogCaching - did not produce a TEST-*.xml file (likely timed out) (batchId=235) TestCatalogNonDefaultClient - did not produce a TEST-*.xml file (likely timed out) (batchId=227) TestCatalogNonDefaultSvr - did not produce a TEST-*.xml file (likely timed out) (batchId=235) TestCatalogOldClient - did not produce a TEST-*.xml file (likely timed out) (batchId=227) TestCatalogs - did not produce a TEST-*.xml file (likely timed out) (batchId=229) TestCheckConstraint - did not produce a TEST-*.xml file (likely timed out) (batchId=227) TestDataSourceProviderFactory - did not produce a TEST-*.xml file (likely timed out) (batchId=237) TestDatabases - did not produce a TEST-*.xml file (likely timed out) (batchId=229) TestDeadline - did not produce a TEST-*.xml file (likely timed out) (batchId=235) TestDefaultConstraint - did not produce a TEST-*.xml file (likely timed out) (batchId=229) TestDropPartitions - did not produce a TEST-*.xml file (likely timed out) (batchId=227) TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=272) TestEmbeddedHiveMetaStore - did not produce a TEST-*.xml file (likely timed out) (batchId=230) TestExchangePartitions - did not produce a TEST-*.xml file (likely timed out) (batchId=229) TestFMSketchSerialization - did not produce a TEST-*.xml file (likely timed out) (batchId=237) TestFilterHooks - did not produce a TEST-*.xml file (likely timed out) (batchId=227) TestForeignKey - did not produce a TEST-*.xml file (likely timed out) (batchId=229) TestFunctions - did not produce a TEST-*.xml file (likely timed out) (batchId=227) TestGetPartitions - did not produce a TEST-*.xml file (likely timed out) (batchId=227) TestGetTableMeta - did not produce a TEST-*.xml file (likely timed out) (batchId=227) TestHLLNoBias - did not produce a TEST-*.xml file (likely timed out) (batchId=237) TestHLLSerialization - did not produce a TEST-*.xml file (likely timed out) (batchId=237) TestHdfsUtils - did not produce a TEST-*.xml file (likely timed out) (batchId=235) TestHiveAlterHandler - did not produce a TEST-*.xml file (likely timed out) (batchId=227) TestHiveMetaStoreGetMetaConf - did not produce a TEST-*.xml file (likely timed out) (batchId=235) TestHiveMetaStorePartitionSpecs - did not produce a TEST-*.xml file (likely timed out) (batchId=229) TestHiveMetaStoreSchemaMethods - did not produce a TEST-*.xml file (likely timed out) (batchId=235) TestHiveMetaStoreTimeout - did not produce a TEST-*.xml file (likely timed out) (batchId=237) TestHiveMetaStoreTxns - did not produce a TEST-*.xml file (likely timed out) (batchId=237) TestHiveMetaStoreWithEnvironmentContext - did not produce a TEST-*.xml file (likely timed out) (batchId=232) TestHiveMetastoreCli - did not produce a TEST-*.xml file (likely timed out) (batchId=227) TestHyperLogLog - did not produce a TEST-*.xml file (likely timed out) (batchId=237) TestHyperLogLogDense - did not produce a TEST-*.xml file (likely timed out) (batchId=237) TestHyperLogLogMerge - did not produce a TEST-*.xml file (likely timed out) (batchId=237) TestHyperLogLogSparse - did not produce a TEST-*.xml file (likely timed out) (batchId=237) TestJSONMessageDeserializer - did not produce a TEST-*.xml file (likely timed out) (batchId=235) TestListPartitions - did not produce a TEST-*.xml file (likely timed out) (batchId=227) TestLockRequestBuilder - did not produce a TEST-*.xml file (likely timed out) (batchId=227) TestMarkPartition - did not produce a TEST-*.xml file (likely timed out) (batchId=235) TestMarkPartitionRemote - did not produce a TEST-*.xml file (likely timed out) (batchId=237) TestMetaStoreConnectionUrlHook - did not produce a TEST-*.xml file
[jira] [Commented] (HIVE-20252) Semijoin Reduction : Cycles due to semi join branch may remain undetected if small table side has a map join upstream.
[ https://issues.apache.org/jira/browse/HIVE-20252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566080#comment-16566080 ] Hive QA commented on HIVE-20252: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 20s{color} | {color:red} /data/hiveptest/logs/PreCommit-HIVE-Build-12988/patches/PreCommit-HIVE-Build-12988.patch does not apply to master. Rebase required? Wrong Branch? See http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help. {color} | \\ \\ || Subsystem || Report/Notes || | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12988/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Semijoin Reduction : Cycles due to semi join branch may remain undetected if > small table side has a map join upstream. > -- > > Key: HIVE-20252 > URL: https://issues.apache.org/jira/browse/HIVE-20252 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-20252.01-branch-3.patch, HIVE-20252.1.patch, > HIVE-20252.2.patch, HIVE-20252.3.patch, HIVE-20252.4.patch, > HIVE-20252.5.patch, HIVE-20252.6.patch > > > For eg, > {noformat} > # 2018-07-26T17:22:14,664 DEBUG [51377701-dc98-424f-82e0-bbb5d6c84316 main] > optimizer.SharedWorkOptimizer: Before SharedWorkOptimizer: > # > TS[0]-FIL[96]-SEL[2]-MAPJOIN[156]-MAPJOIN[157]-MAPJOIN[161]-MAPJOIN[162]-FIL[47]-SEL[48]-MAPJOIN[163]-FIL[66]-SEL[67]-TNK[105]-GBY[68]-RS[69]-GBY[70]-SEL[71]-RS[72]-SEL[73]-LIM[74]-FS[75] > # > -SEL[142]-GBY[143]-RS[144]-GBY[145]-RS[155] > # TS[3]-FIL[97]-SEL[5]-RS[34]-MAPJOIN[156] > # TS[6]-FIL[98]-SEL[8]-RS[37]-MAPJOIN[157] > # TS[9]-FIL[99]-SEL[11]-MAPJOIN[158]-GBY[40]-RS[42]-MAPJOIN[161] > # TS[12]-FIL[100]-SEL[14]-RS[16]-MAPJOIN[158] > # -SEL[131]-GBY[132]-EVENT[133] > # > TS[19]-FIL[101]-SEL[21]-MAPJOIN[159]-GBY[29]-RS[30]-GBY[31]-SEL[32]-RS[45]-MAPJOIN[162] > # TS[22]-FIL[102]-SEL[24]-RS[26]-MAPJOIN[159] > # -SEL[139]-GBY[140]-EVENT[141] > # > TS[49]-FIL[103]-SEL[51]-MAPJOIN[160]-GBY[59]-RS[60]-GBY[61]-SEL[62]-RS[64]-MAPJOIN[163] > # TS[52]-FIL[104]-SEL[54]-RS[56]-MAPJOIN[160] > # -SEL[147]-GBY[148]-EVENT[149] > # > # > # DPP information stored in the cache: \{TS[19]=[EVENT[141]], > TS[9]=[EVENT[133]], TS[49]=[RS[155], EVENT[149]]} > {noformat} > > The semi join branch in line 3 feeds into TS[49] in line 12 which feeds to > MAPJOIN[163] going back to parent of the semi join branch at line 2. > The logic to detect cycle may fail as there is a MAPJOIN[160] at line 12 > which could cause the logic to look for wrong TS. The logic to find TS > operator upstream must use findOperatorsUpstream() and examine each TS Op for > complete coverage. > Simplified image of task-cycle, without operator cycles - > http://people.apache.org/~gopalv/HIVE_20252_cycle1.svg > And the artificial edge introduced to trigger cycle detection (in red) - > http://people.apache.org/~gopalv/HIVE_20252_cycle_fix.svg > cc [~jcamachorodriguez] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20252) Semijoin Reduction : Cycles due to semi join branch may remain undetected if small table side has a map join upstream.
[ https://issues.apache.org/jira/browse/HIVE-20252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565993#comment-16565993 ] Deepak Jaiswal commented on HIVE-20252: --- Committed to master. Preparing for branch-3. Thanks [~jcamachorodriguez] for the review. > Semijoin Reduction : Cycles due to semi join branch may remain undetected if > small table side has a map join upstream. > -- > > Key: HIVE-20252 > URL: https://issues.apache.org/jira/browse/HIVE-20252 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-20252.1.patch, HIVE-20252.2.patch, > HIVE-20252.3.patch, HIVE-20252.4.patch, HIVE-20252.5.patch, HIVE-20252.6.patch > > > For eg, > {noformat} > # 2018-07-26T17:22:14,664 DEBUG [51377701-dc98-424f-82e0-bbb5d6c84316 main] > optimizer.SharedWorkOptimizer: Before SharedWorkOptimizer: > # > TS[0]-FIL[96]-SEL[2]-MAPJOIN[156]-MAPJOIN[157]-MAPJOIN[161]-MAPJOIN[162]-FIL[47]-SEL[48]-MAPJOIN[163]-FIL[66]-SEL[67]-TNK[105]-GBY[68]-RS[69]-GBY[70]-SEL[71]-RS[72]-SEL[73]-LIM[74]-FS[75] > # > -SEL[142]-GBY[143]-RS[144]-GBY[145]-RS[155] > # TS[3]-FIL[97]-SEL[5]-RS[34]-MAPJOIN[156] > # TS[6]-FIL[98]-SEL[8]-RS[37]-MAPJOIN[157] > # TS[9]-FIL[99]-SEL[11]-MAPJOIN[158]-GBY[40]-RS[42]-MAPJOIN[161] > # TS[12]-FIL[100]-SEL[14]-RS[16]-MAPJOIN[158] > # -SEL[131]-GBY[132]-EVENT[133] > # > TS[19]-FIL[101]-SEL[21]-MAPJOIN[159]-GBY[29]-RS[30]-GBY[31]-SEL[32]-RS[45]-MAPJOIN[162] > # TS[22]-FIL[102]-SEL[24]-RS[26]-MAPJOIN[159] > # -SEL[139]-GBY[140]-EVENT[141] > # > TS[49]-FIL[103]-SEL[51]-MAPJOIN[160]-GBY[59]-RS[60]-GBY[61]-SEL[62]-RS[64]-MAPJOIN[163] > # TS[52]-FIL[104]-SEL[54]-RS[56]-MAPJOIN[160] > # -SEL[147]-GBY[148]-EVENT[149] > # > # > # DPP information stored in the cache: \{TS[19]=[EVENT[141]], > TS[9]=[EVENT[133]], TS[49]=[RS[155], EVENT[149]]} > {noformat} > > The semi join branch in line 3 feeds into TS[49] in line 12 which feeds to > MAPJOIN[163] going back to parent of the semi join branch at line 2. > The logic to detect cycle may fail as there is a MAPJOIN[160] at line 12 > which could cause the logic to look for wrong TS. The logic to find TS > operator upstream must use findOperatorsUpstream() and examine each TS Op for > complete coverage. > Simplified image of task-cycle, without operator cycles - > http://people.apache.org/~gopalv/HIVE_20252_cycle1.svg > And the artificial edge introduced to trigger cycle detection (in red) - > http://people.apache.org/~gopalv/HIVE_20252_cycle_fix.svg > cc [~jcamachorodriguez] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20252) Semijoin Reduction : Cycles due to semi join branch may remain undetected if small table side has a map join upstream.
[ https://issues.apache.org/jira/browse/HIVE-20252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565986#comment-16565986 ] Hive QA commented on HIVE-20252: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12933952/HIVE-20252.6.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:green}SUCCESS:{color} +1 due to 14838 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12986/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12986/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12986/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase {noformat} This message is automatically generated. ATTACHMENT ID: 12933952 - PreCommit-HIVE-Build > Semijoin Reduction : Cycles due to semi join branch may remain undetected if > small table side has a map join upstream. > -- > > Key: HIVE-20252 > URL: https://issues.apache.org/jira/browse/HIVE-20252 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-20252.1.patch, HIVE-20252.2.patch, > HIVE-20252.3.patch, HIVE-20252.4.patch, HIVE-20252.5.patch, HIVE-20252.6.patch > > > For eg, > {noformat} > # 2018-07-26T17:22:14,664 DEBUG [51377701-dc98-424f-82e0-bbb5d6c84316 main] > optimizer.SharedWorkOptimizer: Before SharedWorkOptimizer: > # > TS[0]-FIL[96]-SEL[2]-MAPJOIN[156]-MAPJOIN[157]-MAPJOIN[161]-MAPJOIN[162]-FIL[47]-SEL[48]-MAPJOIN[163]-FIL[66]-SEL[67]-TNK[105]-GBY[68]-RS[69]-GBY[70]-SEL[71]-RS[72]-SEL[73]-LIM[74]-FS[75] > # > -SEL[142]-GBY[143]-RS[144]-GBY[145]-RS[155] > # TS[3]-FIL[97]-SEL[5]-RS[34]-MAPJOIN[156] > # TS[6]-FIL[98]-SEL[8]-RS[37]-MAPJOIN[157] > # TS[9]-FIL[99]-SEL[11]-MAPJOIN[158]-GBY[40]-RS[42]-MAPJOIN[161] > # TS[12]-FIL[100]-SEL[14]-RS[16]-MAPJOIN[158] > # -SEL[131]-GBY[132]-EVENT[133] > # > TS[19]-FIL[101]-SEL[21]-MAPJOIN[159]-GBY[29]-RS[30]-GBY[31]-SEL[32]-RS[45]-MAPJOIN[162] > # TS[22]-FIL[102]-SEL[24]-RS[26]-MAPJOIN[159] > # -SEL[139]-GBY[140]-EVENT[141] > # > TS[49]-FIL[103]-SEL[51]-MAPJOIN[160]-GBY[59]-RS[60]-GBY[61]-SEL[62]-RS[64]-MAPJOIN[163] > # TS[52]-FIL[104]-SEL[54]-RS[56]-MAPJOIN[160] > # -SEL[147]-GBY[148]-EVENT[149] > # > # > # DPP information stored in the cache: \{TS[19]=[EVENT[141]], > TS[9]=[EVENT[133]], TS[49]=[RS[155], EVENT[149]]} > {noformat} > > The semi join branch in line 3 feeds into TS[49] in line 12 which feeds to > MAPJOIN[163] going back to parent of the semi join branch at line 2. > The logic to detect cycle may fail as there is a MAPJOIN[160] at line 12 > which could cause the logic to look for wrong TS. The logic to find TS > operator upstream must use findOperatorsUpstream() and examine each TS Op for > complete coverage. > Simplified image of task-cycle, without operator cycles - > http://people.apache.org/~gopalv/HIVE_20252_cycle1.svg > And the artificial edge introduced to trigger cycle detection (in red) - > http://people.apache.org/~gopalv/HIVE_20252_cycle_fix.svg > cc [~jcamachorodriguez] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20252) Semijoin Reduction : Cycles due to semi join branch may remain undetected if small table side has a map join upstream.
[ https://issues.apache.org/jira/browse/HIVE-20252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565940#comment-16565940 ] Hive QA commented on HIVE-20252: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m 32s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 15s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 44s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 34s{color} | {color:blue} ql in master has 2301 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 9s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 41s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 15s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 15s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 38s{color} | {color:red} ql: The patch generated 9 new + 59 unchanged - 11 fixed = 68 total (was 70) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m 52s{color} | {color:red} ql generated 1 new + 2301 unchanged - 0 fixed = 2302 total (was 2301) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 7s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 14s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 27m 33s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:ql | | | Unread public/protected field:At TezCompiler.java:[line 846] | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-12986/dev-support/hive-personality.sh | | git revision | master / 163e0de | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-12986/yetus/diff-checkstyle-ql.txt | | findbugs | http://104.198.109.242/logs//PreCommit-HIVE-Build-12986/yetus/new-findbugs-ql.html | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12986/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Semijoin Reduction : Cycles due to semi join branch may remain undetected if > small table side has a map join upstream. > -- > > Key: HIVE-20252 > URL: https://issues.apache.org/jira/browse/HIVE-20252 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-20252.1.patch, HIVE-20252.2.patch, > HIVE-20252.3.patch, HIVE-20252.4.patch, HIVE-20252.5.patch, HIVE-20252.6.patch > > > For eg, > {noformat} > # 2018-07-26T17:22:14,664 DEBUG [51377701-dc98-424f-82e0-bbb5d6c84316 main] > optimizer.SharedWorkOptimizer: Before SharedWorkOptimizer: > # > TS[0]-FIL[96]-SEL[2]-MAPJOIN[156]-MAPJOIN[157]-MAPJOIN[161]-MAPJOIN[162]-FIL[47]-SEL[48]-MAPJOIN[163]-FIL[66]-SEL[67]-TNK[105]-GBY[68]-RS[69]-GBY[70]-SEL[71]-RS[72]-SEL[73]-LIM[74]-FS[75] > # > -SEL[142]-GBY[143]-RS[144]-GBY[145]-RS[155] > # TS[3]-FIL[97]-SEL[5]-RS[34]-MAPJOIN[156] > # TS[6]-FIL[98]-SEL[8]-RS[37]-MAPJOIN[157] > # TS[9]-FIL[99]-SEL[11]-MAPJOIN[158]-GBY[40]-RS[42]-MAPJOIN[161] > #
[jira] [Commented] (HIVE-20252) Semijoin Reduction : Cycles due to semi join branch may remain undetected if small table side has a map join upstream.
[ https://issues.apache.org/jira/browse/HIVE-20252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565748#comment-16565748 ] Jesus Camacho Rodriguez commented on HIVE-20252: +1 > Semijoin Reduction : Cycles due to semi join branch may remain undetected if > small table side has a map join upstream. > -- > > Key: HIVE-20252 > URL: https://issues.apache.org/jira/browse/HIVE-20252 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-20252.1.patch, HIVE-20252.2.patch, > HIVE-20252.3.patch, HIVE-20252.4.patch, HIVE-20252.5.patch, HIVE-20252.6.patch > > > For eg, > {noformat} > # 2018-07-26T17:22:14,664 DEBUG [51377701-dc98-424f-82e0-bbb5d6c84316 main] > optimizer.SharedWorkOptimizer: Before SharedWorkOptimizer: > # > TS[0]-FIL[96]-SEL[2]-MAPJOIN[156]-MAPJOIN[157]-MAPJOIN[161]-MAPJOIN[162]-FIL[47]-SEL[48]-MAPJOIN[163]-FIL[66]-SEL[67]-TNK[105]-GBY[68]-RS[69]-GBY[70]-SEL[71]-RS[72]-SEL[73]-LIM[74]-FS[75] > # > -SEL[142]-GBY[143]-RS[144]-GBY[145]-RS[155] > # TS[3]-FIL[97]-SEL[5]-RS[34]-MAPJOIN[156] > # TS[6]-FIL[98]-SEL[8]-RS[37]-MAPJOIN[157] > # TS[9]-FIL[99]-SEL[11]-MAPJOIN[158]-GBY[40]-RS[42]-MAPJOIN[161] > # TS[12]-FIL[100]-SEL[14]-RS[16]-MAPJOIN[158] > # -SEL[131]-GBY[132]-EVENT[133] > # > TS[19]-FIL[101]-SEL[21]-MAPJOIN[159]-GBY[29]-RS[30]-GBY[31]-SEL[32]-RS[45]-MAPJOIN[162] > # TS[22]-FIL[102]-SEL[24]-RS[26]-MAPJOIN[159] > # -SEL[139]-GBY[140]-EVENT[141] > # > TS[49]-FIL[103]-SEL[51]-MAPJOIN[160]-GBY[59]-RS[60]-GBY[61]-SEL[62]-RS[64]-MAPJOIN[163] > # TS[52]-FIL[104]-SEL[54]-RS[56]-MAPJOIN[160] > # -SEL[147]-GBY[148]-EVENT[149] > # > # > # DPP information stored in the cache: \{TS[19]=[EVENT[141]], > TS[9]=[EVENT[133]], TS[49]=[RS[155], EVENT[149]]} > {noformat} > > The semi join branch in line 3 feeds into TS[49] in line 12 which feeds to > MAPJOIN[163] going back to parent of the semi join branch at line 2. > The logic to detect cycle may fail as there is a MAPJOIN[160] at line 12 > which could cause the logic to look for wrong TS. The logic to find TS > operator upstream must use findOperatorsUpstream() and examine each TS Op for > complete coverage. > > cc [~jcamachorodriguez] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20252) Semijoin Reduction : Cycles due to semi join branch may remain undetected if small table side has a map join upstream.
[ https://issues.apache.org/jira/browse/HIVE-20252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564599#comment-16564599 ] Hive QA commented on HIVE-20252: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12933837/HIVE-20252.4.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:green}SUCCESS:{color} +1 due to 14838 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12975/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12975/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12975/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase {noformat} This message is automatically generated. ATTACHMENT ID: 12933837 - PreCommit-HIVE-Build > Semijoin Reduction : Cycles due to semi join branch may remain undetected if > small table side has a map join upstream. > -- > > Key: HIVE-20252 > URL: https://issues.apache.org/jira/browse/HIVE-20252 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-20252.1.patch, HIVE-20252.2.patch, > HIVE-20252.3.patch, HIVE-20252.4.patch > > > For eg, > {noformat} > # 2018-07-26T17:22:14,664 DEBUG [51377701-dc98-424f-82e0-bbb5d6c84316 main] > optimizer.SharedWorkOptimizer: Before SharedWorkOptimizer: > # > TS[0]-FIL[96]-SEL[2]-MAPJOIN[156]-MAPJOIN[157]-MAPJOIN[161]-MAPJOIN[162]-FIL[47]-SEL[48]-MAPJOIN[163]-FIL[66]-SEL[67]-TNK[105]-GBY[68]-RS[69]-GBY[70]-SEL[71]-RS[72]-SEL[73]-LIM[74]-FS[75] > # > -SEL[142]-GBY[143]-RS[144]-GBY[145]-RS[155] > # TS[3]-FIL[97]-SEL[5]-RS[34]-MAPJOIN[156] > # TS[6]-FIL[98]-SEL[8]-RS[37]-MAPJOIN[157] > # TS[9]-FIL[99]-SEL[11]-MAPJOIN[158]-GBY[40]-RS[42]-MAPJOIN[161] > # TS[12]-FIL[100]-SEL[14]-RS[16]-MAPJOIN[158] > # -SEL[131]-GBY[132]-EVENT[133] > # > TS[19]-FIL[101]-SEL[21]-MAPJOIN[159]-GBY[29]-RS[30]-GBY[31]-SEL[32]-RS[45]-MAPJOIN[162] > # TS[22]-FIL[102]-SEL[24]-RS[26]-MAPJOIN[159] > # -SEL[139]-GBY[140]-EVENT[141] > # > TS[49]-FIL[103]-SEL[51]-MAPJOIN[160]-GBY[59]-RS[60]-GBY[61]-SEL[62]-RS[64]-MAPJOIN[163] > # TS[52]-FIL[104]-SEL[54]-RS[56]-MAPJOIN[160] > # -SEL[147]-GBY[148]-EVENT[149] > # > # > # DPP information stored in the cache: \{TS[19]=[EVENT[141]], > TS[9]=[EVENT[133]], TS[49]=[RS[155], EVENT[149]]} > {noformat} > > The semi join branch in line 3 feeds into TS[49] in line 12 which feeds to > MAPJOIN[163] going back to parent of the semi join branch at line 2. > The logic to detect cycle may fail as there is a MAPJOIN[160] at line 12 > which could cause the logic to look for wrong TS. The logic to find TS > operator upstream must use findOperatorsUpstream() and examine each TS Op for > complete coverage. > > cc [~jcamachorodriguez] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20252) Semijoin Reduction : Cycles due to semi join branch may remain undetected if small table side has a map join upstream.
[ https://issues.apache.org/jira/browse/HIVE-20252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564581#comment-16564581 ] Hive QA commented on HIVE-20252: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m 35s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 7s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 40s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 5s{color} | {color:blue} ql in master has 2301 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 0s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 25s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 6s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 6s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 39s{color} | {color:red} ql: The patch generated 10 new + 59 unchanged - 11 fixed = 69 total (was 70) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m 25s{color} | {color:red} ql generated 1 new + 2301 unchanged - 0 fixed = 2302 total (was 2301) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 4s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 13s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 24m 55s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:ql | | | Unread public/protected field:At TezCompiler.java:[line 846] | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-12975/dev-support/hive-personality.sh | | git revision | master / e9e1f8f | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-12975/yetus/diff-checkstyle-ql.txt | | findbugs | http://104.198.109.242/logs//PreCommit-HIVE-Build-12975/yetus/new-findbugs-ql.html | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12975/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Semijoin Reduction : Cycles due to semi join branch may remain undetected if > small table side has a map join upstream. > -- > > Key: HIVE-20252 > URL: https://issues.apache.org/jira/browse/HIVE-20252 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-20252.1.patch, HIVE-20252.2.patch, > HIVE-20252.3.patch, HIVE-20252.4.patch > > > For eg, > {noformat} > # 2018-07-26T17:22:14,664 DEBUG [51377701-dc98-424f-82e0-bbb5d6c84316 main] > optimizer.SharedWorkOptimizer: Before SharedWorkOptimizer: > # > TS[0]-FIL[96]-SEL[2]-MAPJOIN[156]-MAPJOIN[157]-MAPJOIN[161]-MAPJOIN[162]-FIL[47]-SEL[48]-MAPJOIN[163]-FIL[66]-SEL[67]-TNK[105]-GBY[68]-RS[69]-GBY[70]-SEL[71]-RS[72]-SEL[73]-LIM[74]-FS[75] > # > -SEL[142]-GBY[143]-RS[144]-GBY[145]-RS[155] > # TS[3]-FIL[97]-SEL[5]-RS[34]-MAPJOIN[156] > # TS[6]-FIL[98]-SEL[8]-RS[37]-MAPJOIN[157] > # TS[9]-FIL[99]-SEL[11]-MAPJOIN[158]-GBY[40]-RS[42]-MAPJOIN[161] > # TS[12]-FIL[100]-SEL[14]-RS[16]-MAPJOIN[158]
[jira] [Commented] (HIVE-20252) Semijoin Reduction : Cycles due to semi join branch may remain undetected if small table side has a map join upstream.
[ https://issues.apache.org/jira/browse/HIVE-20252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564534#comment-16564534 ] Hive QA commented on HIVE-20252: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12933832/HIVE-20252.3.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 14838 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druid_timestamptz] (batchId=193) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_joins] (batchId=193) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_masking] (batchId=193) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test1] (batchId=193) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12974/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12974/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12974/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 4 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12933832 - PreCommit-HIVE-Build > Semijoin Reduction : Cycles due to semi join branch may remain undetected if > small table side has a map join upstream. > -- > > Key: HIVE-20252 > URL: https://issues.apache.org/jira/browse/HIVE-20252 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-20252.1.patch, HIVE-20252.2.patch, > HIVE-20252.3.patch > > > For eg, > {noformat} > # 2018-07-26T17:22:14,664 DEBUG [51377701-dc98-424f-82e0-bbb5d6c84316 main] > optimizer.SharedWorkOptimizer: Before SharedWorkOptimizer: > # > TS[0]-FIL[96]-SEL[2]-MAPJOIN[156]-MAPJOIN[157]-MAPJOIN[161]-MAPJOIN[162]-FIL[47]-SEL[48]-MAPJOIN[163]-FIL[66]-SEL[67]-TNK[105]-GBY[68]-RS[69]-GBY[70]-SEL[71]-RS[72]-SEL[73]-LIM[74]-FS[75] > # > -SEL[142]-GBY[143]-RS[144]-GBY[145]-RS[155] > # TS[3]-FIL[97]-SEL[5]-RS[34]-MAPJOIN[156] > # TS[6]-FIL[98]-SEL[8]-RS[37]-MAPJOIN[157] > # TS[9]-FIL[99]-SEL[11]-MAPJOIN[158]-GBY[40]-RS[42]-MAPJOIN[161] > # TS[12]-FIL[100]-SEL[14]-RS[16]-MAPJOIN[158] > # -SEL[131]-GBY[132]-EVENT[133] > # > TS[19]-FIL[101]-SEL[21]-MAPJOIN[159]-GBY[29]-RS[30]-GBY[31]-SEL[32]-RS[45]-MAPJOIN[162] > # TS[22]-FIL[102]-SEL[24]-RS[26]-MAPJOIN[159] > # -SEL[139]-GBY[140]-EVENT[141] > # > TS[49]-FIL[103]-SEL[51]-MAPJOIN[160]-GBY[59]-RS[60]-GBY[61]-SEL[62]-RS[64]-MAPJOIN[163] > # TS[52]-FIL[104]-SEL[54]-RS[56]-MAPJOIN[160] > # -SEL[147]-GBY[148]-EVENT[149] > # > # > # DPP information stored in the cache: \{TS[19]=[EVENT[141]], > TS[9]=[EVENT[133]], TS[49]=[RS[155], EVENT[149]]} > {noformat} > > The semi join branch in line 3 feeds into TS[49] in line 12 which feeds to > MAPJOIN[163] going back to parent of the semi join branch at line 2. > The logic to detect cycle may fail as there is a MAPJOIN[160] at line 12 > which could cause the logic to look for wrong TS. The logic to find TS > operator upstream must use findOperatorsUpstream() and examine each TS Op for > complete coverage. > > cc [~jcamachorodriguez] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20252) Semijoin Reduction : Cycles due to semi join branch may remain undetected if small table side has a map join upstream.
[ https://issues.apache.org/jira/browse/HIVE-20252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564508#comment-16564508 ] Hive QA commented on HIVE-20252: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m 32s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 10s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 41s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 9s{color} | {color:blue} ql in master has 2301 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 0s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 31s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 8s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 8s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 42s{color} | {color:red} ql: The patch generated 8 new + 52 unchanged - 11 fixed = 60 total (was 63) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 22s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 1s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 13s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 25m 7s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-12974/dev-support/hive-personality.sh | | git revision | master / 5b2124b | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-12974/yetus/diff-checkstyle-ql.txt | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12974/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Semijoin Reduction : Cycles due to semi join branch may remain undetected if > small table side has a map join upstream. > -- > > Key: HIVE-20252 > URL: https://issues.apache.org/jira/browse/HIVE-20252 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-20252.1.patch, HIVE-20252.2.patch, > HIVE-20252.3.patch > > > For eg, > {noformat} > # 2018-07-26T17:22:14,664 DEBUG [51377701-dc98-424f-82e0-bbb5d6c84316 main] > optimizer.SharedWorkOptimizer: Before SharedWorkOptimizer: > # > TS[0]-FIL[96]-SEL[2]-MAPJOIN[156]-MAPJOIN[157]-MAPJOIN[161]-MAPJOIN[162]-FIL[47]-SEL[48]-MAPJOIN[163]-FIL[66]-SEL[67]-TNK[105]-GBY[68]-RS[69]-GBY[70]-SEL[71]-RS[72]-SEL[73]-LIM[74]-FS[75] > # > -SEL[142]-GBY[143]-RS[144]-GBY[145]-RS[155] > # TS[3]-FIL[97]-SEL[5]-RS[34]-MAPJOIN[156] > # TS[6]-FIL[98]-SEL[8]-RS[37]-MAPJOIN[157] > # TS[9]-FIL[99]-SEL[11]-MAPJOIN[158]-GBY[40]-RS[42]-MAPJOIN[161] > # TS[12]-FIL[100]-SEL[14]-RS[16]-MAPJOIN[158] > # -SEL[131]-GBY[132]-EVENT[133] > # > TS[19]-FIL[101]-SEL[21]-MAPJOIN[159]-GBY[29]-RS[30]-GBY[31]-SEL[32]-RS[45]-MAPJOIN[162] > # TS[22]-FIL[102]-SEL[24]-RS[26]-MAPJOIN[159] > # -SEL[139]-GBY[140]-EVENT[141] > # >
[jira] [Commented] (HIVE-20252) Semijoin Reduction : Cycles due to semi join branch may remain undetected if small table side has a map join upstream.
[ https://issues.apache.org/jira/browse/HIVE-20252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16563650#comment-16563650 ] Hive QA commented on HIVE-20252: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12933702/HIVE-20252.2.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:green}SUCCESS:{color} +1 due to 14834 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12964/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12964/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12964/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase {noformat} This message is automatically generated. ATTACHMENT ID: 12933702 - PreCommit-HIVE-Build > Semijoin Reduction : Cycles due to semi join branch may remain undetected if > small table side has a map join upstream. > -- > > Key: HIVE-20252 > URL: https://issues.apache.org/jira/browse/HIVE-20252 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-20252.1.patch, HIVE-20252.2.patch > > > For eg, > {noformat} > # 2018-07-26T17:22:14,664 DEBUG [51377701-dc98-424f-82e0-bbb5d6c84316 main] > optimizer.SharedWorkOptimizer: Before SharedWorkOptimizer: > # > TS[0]-FIL[96]-SEL[2]-MAPJOIN[156]-MAPJOIN[157]-MAPJOIN[161]-MAPJOIN[162]-FIL[47]-SEL[48]-MAPJOIN[163]-FIL[66]-SEL[67]-TNK[105]-GBY[68]-RS[69]-GBY[70]-SEL[71]-RS[72]-SEL[73]-LIM[74]-FS[75] > # > -SEL[142]-GBY[143]-RS[144]-GBY[145]-RS[155] > # TS[3]-FIL[97]-SEL[5]-RS[34]-MAPJOIN[156] > # TS[6]-FIL[98]-SEL[8]-RS[37]-MAPJOIN[157] > # TS[9]-FIL[99]-SEL[11]-MAPJOIN[158]-GBY[40]-RS[42]-MAPJOIN[161] > # TS[12]-FIL[100]-SEL[14]-RS[16]-MAPJOIN[158] > # -SEL[131]-GBY[132]-EVENT[133] > # > TS[19]-FIL[101]-SEL[21]-MAPJOIN[159]-GBY[29]-RS[30]-GBY[31]-SEL[32]-RS[45]-MAPJOIN[162] > # TS[22]-FIL[102]-SEL[24]-RS[26]-MAPJOIN[159] > # -SEL[139]-GBY[140]-EVENT[141] > # > TS[49]-FIL[103]-SEL[51]-MAPJOIN[160]-GBY[59]-RS[60]-GBY[61]-SEL[62]-RS[64]-MAPJOIN[163] > # TS[52]-FIL[104]-SEL[54]-RS[56]-MAPJOIN[160] > # -SEL[147]-GBY[148]-EVENT[149] > # > # > # DPP information stored in the cache: \{TS[19]=[EVENT[141]], > TS[9]=[EVENT[133]], TS[49]=[RS[155], EVENT[149]]} > {noformat} > > The semi join branch in line 3 feeds into TS[49] in line 12 which feeds to > MAPJOIN[163] going back to parent of the semi join branch at line 2. > The logic to detect cycle may fail as there is a MAPJOIN[160] at line 12 > which could cause the logic to look for wrong TS. The logic to find TS > operator upstream must use findOperatorsUpstream() and examine each TS Op for > complete coverage. > > cc [~jcamachorodriguez] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20252) Semijoin Reduction : Cycles due to semi join branch may remain undetected if small table side has a map join upstream.
[ https://issues.apache.org/jira/browse/HIVE-20252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16563607#comment-16563607 ] Hive QA commented on HIVE-20252: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m 12s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 12s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 43s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 21s{color} | {color:blue} ql in master has 2306 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 31s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 7s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 7s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 40s{color} | {color:red} ql: The patch generated 12 new + 58 unchanged - 10 fixed = 70 total (was 68) {color} | | {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m 38s{color} | {color:red} ql generated 1 new + 2306 unchanged - 0 fixed = 2307 total (was 2306) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 13s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 26m 13s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:ql | | | org.apache.hadoop.hive.ql.parse.TezCompiler.removeSemiJoinsDueToTaskLevelCycles(OptimizeTezProcContext) makes inefficient use of keySet iterator instead of entrySet iterator At TezCompiler.java:keySet iterator instead of entrySet iterator At TezCompiler.java:[line 1108] | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-12964/dev-support/hive-personality.sh | | git revision | master / edc53cc | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-12964/yetus/diff-checkstyle-ql.txt | | whitespace | http://104.198.109.242/logs//PreCommit-HIVE-Build-12964/yetus/whitespace-eol.txt | | findbugs | http://104.198.109.242/logs//PreCommit-HIVE-Build-12964/yetus/new-findbugs-ql.html | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12964/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Semijoin Reduction : Cycles due to semi join branch may remain undetected if > small table side has a map join upstream. > -- > > Key: HIVE-20252 > URL: https://issues.apache.org/jira/browse/HIVE-20252 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-20252.1.patch, HIVE-20252.2.patch > > > For eg, > {noformat} > # 2018-07-26T17:22:14,664 DEBUG [51377701-dc98-424f-82e0-bbb5d6c84316 main] > optimizer.SharedWorkOptimizer: Before SharedWorkOptimizer: > # >
[jira] [Commented] (HIVE-20252) Semijoin Reduction : Cycles due to semi join branch may remain undetected if small table side has a map join upstream.
[ https://issues.apache.org/jira/browse/HIVE-20252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559729#comment-16559729 ] Hive QA commented on HIVE-20252: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12933290/HIVE-20252.1.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14812 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testMultiInsert (batchId=312) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12896/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12896/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12896/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12933290 - PreCommit-HIVE-Build > Semijoin Reduction : Cycles due to semi join branch may remain undetected if > small table side has a map join upstream. > -- > > Key: HIVE-20252 > URL: https://issues.apache.org/jira/browse/HIVE-20252 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-20252.1.patch > > > For eg, > {noformat} > # 2018-07-26T17:22:14,664 DEBUG [51377701-dc98-424f-82e0-bbb5d6c84316 main] > optimizer.SharedWorkOptimizer: Before SharedWorkOptimizer: > # > TS[0]-FIL[96]-SEL[2]-MAPJOIN[156]-MAPJOIN[157]-MAPJOIN[161]-MAPJOIN[162]-FIL[47]-SEL[48]-MAPJOIN[163]-FIL[66]-SEL[67]-TNK[105]-GBY[68]-RS[69]-GBY[70]-SEL[71]-RS[72]-SEL[73]-LIM[74]-FS[75] > # > -SEL[142]-GBY[143]-RS[144]-GBY[145]-RS[155] > # TS[3]-FIL[97]-SEL[5]-RS[34]-MAPJOIN[156] > # TS[6]-FIL[98]-SEL[8]-RS[37]-MAPJOIN[157] > # TS[9]-FIL[99]-SEL[11]-MAPJOIN[158]-GBY[40]-RS[42]-MAPJOIN[161] > # TS[12]-FIL[100]-SEL[14]-RS[16]-MAPJOIN[158] > # -SEL[131]-GBY[132]-EVENT[133] > # > TS[19]-FIL[101]-SEL[21]-MAPJOIN[159]-GBY[29]-RS[30]-GBY[31]-SEL[32]-RS[45]-MAPJOIN[162] > # TS[22]-FIL[102]-SEL[24]-RS[26]-MAPJOIN[159] > # -SEL[139]-GBY[140]-EVENT[141] > # > TS[49]-FIL[103]-SEL[51]-MAPJOIN[160]-GBY[59]-RS[60]-GBY[61]-SEL[62]-RS[64]-MAPJOIN[163] > # TS[52]-FIL[104]-SEL[54]-RS[56]-MAPJOIN[160] > # -SEL[147]-GBY[148]-EVENT[149] > # > # > # DPP information stored in the cache: \{TS[19]=[EVENT[141]], > TS[9]=[EVENT[133]], TS[49]=[RS[155], EVENT[149]]} > {noformat} > > The semi join branch in line 3 feeds into TS[49] in line 12 which feeds to > MAPJOIN[163] going back to parent of the semi join branch at line 2. > The logic to detect cycle may fail as there is a MAPJOIN[160] at line 12 > which could cause the logic to look for wrong TS. The logic to find TS > operator upstream must use findOperatorsUpstream() and examine each TS Op for > complete coverage. > > cc [~jcamachorodriguez] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20252) Semijoin Reduction : Cycles due to semi join branch may remain undetected if small table side has a map join upstream.
[ https://issues.apache.org/jira/browse/HIVE-20252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559686#comment-16559686 ] Hive QA commented on HIVE-20252: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m 11s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 8s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 39s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 0s{color} | {color:blue} ql in master has 2296 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 0s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 25s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 8s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 8s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 44s{color} | {color:red} ql: The patch generated 7 new + 34 unchanged - 7 fixed = 41 total (was 41) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 15s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 0s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 13s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 24m 14s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-12896/dev-support/hive-personality.sh | | git revision | master / 94ec368 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-12896/yetus/diff-checkstyle-ql.txt | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12896/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Semijoin Reduction : Cycles due to semi join branch may remain undetected if > small table side has a map join upstream. > -- > > Key: HIVE-20252 > URL: https://issues.apache.org/jira/browse/HIVE-20252 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-20252.1.patch > > > For eg, > {noformat} > # 2018-07-26T17:22:14,664 DEBUG [51377701-dc98-424f-82e0-bbb5d6c84316 main] > optimizer.SharedWorkOptimizer: Before SharedWorkOptimizer: > # > TS[0]-FIL[96]-SEL[2]-MAPJOIN[156]-MAPJOIN[157]-MAPJOIN[161]-MAPJOIN[162]-FIL[47]-SEL[48]-MAPJOIN[163]-FIL[66]-SEL[67]-TNK[105]-GBY[68]-RS[69]-GBY[70]-SEL[71]-RS[72]-SEL[73]-LIM[74]-FS[75] > # > -SEL[142]-GBY[143]-RS[144]-GBY[145]-RS[155] > # TS[3]-FIL[97]-SEL[5]-RS[34]-MAPJOIN[156] > # TS[6]-FIL[98]-SEL[8]-RS[37]-MAPJOIN[157] > # TS[9]-FIL[99]-SEL[11]-MAPJOIN[158]-GBY[40]-RS[42]-MAPJOIN[161] > # TS[12]-FIL[100]-SEL[14]-RS[16]-MAPJOIN[158] > # -SEL[131]-GBY[132]-EVENT[133] > # > TS[19]-FIL[101]-SEL[21]-MAPJOIN[159]-GBY[29]-RS[30]-GBY[31]-SEL[32]-RS[45]-MAPJOIN[162] > # TS[22]-FIL[102]-SEL[24]-RS[26]-MAPJOIN[159] > # -SEL[139]-GBY[140]-EVENT[141] > # >