[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16578034#comment-16578034 ] Sankar Hariappan commented on HIVE-19924: - 14.patch committed to branch-3 as well. > Tag distcp jobs run by Repl Load > > > Key: HIVE-19924 > URL: https://issues.apache.org/jira/browse/HIVE-19924 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: DR, replication > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-19924.01.patch, HIVE-19924.02.patch, > HIVE-19924.03.patch, HIVE-19924.04.patch, HIVE-19924.05.patch, > HIVE-19924.06.patch, HIVE-19924.07.patch, HIVE-19924.08.patch, > HIVE-19924.09.patch, HIVE-19924.10.patch, HIVE-19924.11.patch, > HIVE-19924.12.patch, HIVE-19924.13.patch, HIVE-19924.14-branch-3.patch, > HIVE-19924.14.patch > > > Add tags in jobconf for distcp related jobs started by replication. This will > allow hive to kill these jobs in case beacon retries, or hs2 dies and beacon > issues a kill command. > * one of the tags should definitely be the query_id that starts the job : > With this flow beacon before retrying the bootstrap load, will issue a kill > command to hs2 with the query id of the previous issued command. hs2 will > then kill an running jobs on yarn tagged with the Query_id. > * To get around the additional failure point as mentioned above. The jobs > can be tagged with an additional unique tag_id provided by Beacon in the WITH > clause in repl load command to be used to tag distcp jobs ). Enhance the kill > api to take the tag as input and kill jobs associated with that tag. Problem > here is how do we validate the association of the tag with a hive query id to > make sure this api is not used to kill jobs run by other components, however > we can provide this capability to only admins and should be ok in that case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16578029#comment-16578029 ] mahesh kumar behera commented on HIVE-19924: [~sankarh] The test failures are not related to this files and failing for other patches also. Please check if this can be committed to branch-3. > Tag distcp jobs run by Repl Load > > > Key: HIVE-19924 > URL: https://issues.apache.org/jira/browse/HIVE-19924 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: DR, replication > Fix For: 4.0.0 > > Attachments: HIVE-19924.01.patch, HIVE-19924.02.patch, > HIVE-19924.03.patch, HIVE-19924.04.patch, HIVE-19924.05.patch, > HIVE-19924.06.patch, HIVE-19924.07.patch, HIVE-19924.08.patch, > HIVE-19924.09.patch, HIVE-19924.10.patch, HIVE-19924.11.patch, > HIVE-19924.12.patch, HIVE-19924.13.patch, HIVE-19924.14-branch-3.patch, > HIVE-19924.14.patch > > > Add tags in jobconf for distcp related jobs started by replication. This will > allow hive to kill these jobs in case beacon retries, or hs2 dies and beacon > issues a kill command. > * one of the tags should definitely be the query_id that starts the job : > With this flow beacon before retrying the bootstrap load, will issue a kill > command to hs2 with the query id of the previous issued command. hs2 will > then kill an running jobs on yarn tagged with the Query_id. > * To get around the additional failure point as mentioned above. The jobs > can be tagged with an additional unique tag_id provided by Beacon in the WITH > clause in repl load command to be used to tag distcp jobs ). Enhance the kill > api to take the tag as input and kill jobs associated with that tag. Problem > here is how do we validate the association of the tag with a hive query id to > make sure this api is not used to kill jobs run by other components, however > we can provide this capability to only admins and should be ok in that case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16578018#comment-16578018 ] Hive QA commented on HIVE-19924: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12935322/HIVE-19924.14-branch-3.patch {color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 103 failed/errored test(s), 14426 tests executed *Failed tests:* {noformat} TestAddPartitions - did not produce a TEST-*.xml file (likely timed out) (batchId=228) TestAddPartitionsFromPartSpec - did not produce a TEST-*.xml file (likely timed out) (batchId=230) TestAdminUser - did not produce a TEST-*.xml file (likely timed out) (batchId=236) TestAggregateStatsCache - did not produce a TEST-*.xml file (likely timed out) (batchId=230) TestAlterPartitions - did not produce a TEST-*.xml file (likely timed out) (batchId=230) TestAppendPartitions - did not produce a TEST-*.xml file (likely timed out) (batchId=230) TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=273) TestCachedStore - did not produce a TEST-*.xml file (likely timed out) (batchId=236) TestCatalogCaching - did not produce a TEST-*.xml file (likely timed out) (batchId=236) TestCatalogNonDefaultClient - did not produce a TEST-*.xml file (likely timed out) (batchId=228) TestCatalogNonDefaultSvr - did not produce a TEST-*.xml file (likely timed out) (batchId=236) TestCatalogOldClient - did not produce a TEST-*.xml file (likely timed out) (batchId=228) TestCatalogs - did not produce a TEST-*.xml file (likely timed out) (batchId=230) TestCheckConstraint - did not produce a TEST-*.xml file (likely timed out) (batchId=228) TestDataSourceProviderFactory - did not produce a TEST-*.xml file (likely timed out) (batchId=238) TestDatabases - did not produce a TEST-*.xml file (likely timed out) (batchId=230) TestDeadline - did not produce a TEST-*.xml file (likely timed out) (batchId=236) TestDefaultConstraint - did not produce a TEST-*.xml file (likely timed out) (batchId=230) TestDropPartitions - did not produce a TEST-*.xml file (likely timed out) (batchId=228) TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=273) TestEmbeddedHiveMetaStore - did not produce a TEST-*.xml file (likely timed out) (batchId=231) TestExchangePartitions - did not produce a TEST-*.xml file (likely timed out) (batchId=230) TestFMSketchSerialization - did not produce a TEST-*.xml file (likely timed out) (batchId=238) TestFilterHooks - did not produce a TEST-*.xml file (likely timed out) (batchId=228) TestForeignKey - did not produce a TEST-*.xml file (likely timed out) (batchId=230) TestFunctions - did not produce a TEST-*.xml file (likely timed out) (batchId=228) TestGetPartitions - did not produce a TEST-*.xml file (likely timed out) (batchId=228) TestGetTableMeta - did not produce a TEST-*.xml file (likely timed out) (batchId=228) TestHLLNoBias - did not produce a TEST-*.xml file (likely timed out) (batchId=238) TestHLLSerialization - did not produce a TEST-*.xml file (likely timed out) (batchId=238) TestHdfsUtils - did not produce a TEST-*.xml file (likely timed out) (batchId=236) TestHiveAlterHandler - did not produce a TEST-*.xml file (likely timed out) (batchId=228) TestHiveMetaStoreGetMetaConf - did not produce a TEST-*.xml file (likely timed out) (batchId=236) TestHiveMetaStorePartitionSpecs - did not produce a TEST-*.xml file (likely timed out) (batchId=230) TestHiveMetaStoreSchemaMethods - did not produce a TEST-*.xml file (likely timed out) (batchId=236) TestHiveMetaStoreTimeout - did not produce a TEST-*.xml file (likely timed out) (batchId=238) TestHiveMetaStoreTxns - did not produce a TEST-*.xml file (likely timed out) (batchId=238) TestHiveMetaStoreWithEnvironmentContext - did not produce a TEST-*.xml file (likely timed out) (batchId=233) TestHiveMetastoreCli - did not produce a TEST-*.xml file (likely timed out) (batchId=228) TestHyperLogLog - did not produce a TEST-*.xml file (likely timed out) (batchId=238) TestHyperLogLogDense - did not produce a TEST-*.xml file (likely timed out) (batchId=238) TestHyperLogLogMerge - did not produce a TEST-*.xml file (likely timed out) (batchId=238) TestHyperLogLogSparse - did not produce a TEST-*.xml file (likely timed out) (batchId=238) TestJSONMessageDeserializer - did not produce a TEST-*.xml file (likely timed out) (batchId=236) TestListPartitions - did not produce a TEST-*.xml file (likely timed out) (batchId=228) TestLockRequestBuilder - did not produce a TEST-*.xml file (likely timed out) (batchId=228) TestMarkPartition - did not produce a TEST-*.xml file (likely timed out) (batchId=236) TestMarkPartitionRemote - did not produce a TEST-*.xml file (likely timed out) (batchId=238) TestMetaStoreConnectionUrlHook - did not produce a TE
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16577950#comment-16577950 ] Hive QA commented on HIVE-19924: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 11s{color} | {color:red} /data/hiveptest/logs/PreCommit-HIVE-Build-13187/patches/PreCommit-HIVE-Build-13187.patch does not apply to master. Rebase required? Wrong Branch? See http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help. {color} | \\ \\ || Subsystem || Report/Notes || | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-13187/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Tag distcp jobs run by Repl Load > > > Key: HIVE-19924 > URL: https://issues.apache.org/jira/browse/HIVE-19924 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: DR, replication > Fix For: 4.0.0 > > Attachments: HIVE-19924.01.patch, HIVE-19924.02.patch, > HIVE-19924.03.patch, HIVE-19924.04.patch, HIVE-19924.05.patch, > HIVE-19924.06.patch, HIVE-19924.07.patch, HIVE-19924.08.patch, > HIVE-19924.09.patch, HIVE-19924.10.patch, HIVE-19924.11.patch, > HIVE-19924.12.patch, HIVE-19924.13.patch, HIVE-19924.14-branch-3.patch, > HIVE-19924.14.patch > > > Add tags in jobconf for distcp related jobs started by replication. This will > allow hive to kill these jobs in case beacon retries, or hs2 dies and beacon > issues a kill command. > * one of the tags should definitely be the query_id that starts the job : > With this flow beacon before retrying the bootstrap load, will issue a kill > command to hs2 with the query id of the previous issued command. hs2 will > then kill an running jobs on yarn tagged with the Query_id. > * To get around the additional failure point as mentioned above. The jobs > can be tagged with an additional unique tag_id provided by Beacon in the WITH > clause in repl load command to be used to tag distcp jobs ). Enhance the kill > api to take the tag as input and kill jobs associated with that tag. Problem > here is how do we validate the association of the tag with a hive query id to > make sure this api is not used to kill jobs run by other components, however > we can provide this capability to only admins and should be ok in that case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16577854#comment-16577854 ] Sankar Hariappan commented on HIVE-19924: - 14.patch is committed to master. Thanks [~maheshk114] for the patch! > Tag distcp jobs run by Repl Load > > > Key: HIVE-19924 > URL: https://issues.apache.org/jira/browse/HIVE-19924 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: DR, replication > Fix For: 4.0.0 > > Attachments: HIVE-19924.01.patch, HIVE-19924.02.patch, > HIVE-19924.03.patch, HIVE-19924.04.patch, HIVE-19924.05.patch, > HIVE-19924.06.patch, HIVE-19924.07.patch, HIVE-19924.08.patch, > HIVE-19924.09.patch, HIVE-19924.10.patch, HIVE-19924.11.patch, > HIVE-19924.12.patch, HIVE-19924.13.patch, HIVE-19924.14.patch > > > Add tags in jobconf for distcp related jobs started by replication. This will > allow hive to kill these jobs in case beacon retries, or hs2 dies and beacon > issues a kill command. > * one of the tags should definitely be the query_id that starts the job : > With this flow beacon before retrying the bootstrap load, will issue a kill > command to hs2 with the query id of the previous issued command. hs2 will > then kill an running jobs on yarn tagged with the Query_id. > * To get around the additional failure point as mentioned above. The jobs > can be tagged with an additional unique tag_id provided by Beacon in the WITH > clause in repl load command to be used to tag distcp jobs ). Enhance the kill > api to take the tag as input and kill jobs associated with that tag. Problem > here is how do we validate the association of the tag with a hive query id to > make sure this api is not used to kill jobs run by other components, however > we can provide this capability to only admins and should be ok in that case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16577853#comment-16577853 ] Sankar Hariappan commented on HIVE-19924: - +1 > Tag distcp jobs run by Repl Load > > > Key: HIVE-19924 > URL: https://issues.apache.org/jira/browse/HIVE-19924 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: DR, replication > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-19924.01.patch, HIVE-19924.02.patch, > HIVE-19924.03.patch, HIVE-19924.04.patch, HIVE-19924.05.patch, > HIVE-19924.06.patch, HIVE-19924.07.patch, HIVE-19924.08.patch, > HIVE-19924.09.patch, HIVE-19924.10.patch, HIVE-19924.11.patch, > HIVE-19924.12.patch, HIVE-19924.13.patch, HIVE-19924.14.patch > > > Add tags in jobconf for distcp related jobs started by replication. This will > allow hive to kill these jobs in case beacon retries, or hs2 dies and beacon > issues a kill command. > * one of the tags should definitely be the query_id that starts the job : > With this flow beacon before retrying the bootstrap load, will issue a kill > command to hs2 with the query id of the previous issued command. hs2 will > then kill an running jobs on yarn tagged with the Query_id. > * To get around the additional failure point as mentioned above. The jobs > can be tagged with an additional unique tag_id provided by Beacon in the WITH > clause in repl load command to be used to tag distcp jobs ). Enhance the kill > api to take the tag as input and kill jobs associated with that tag. Problem > here is how do we validate the association of the tag with a hive query id to > make sure this api is not used to kill jobs run by other components, however > we can provide this capability to only admins and should be ok in that case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16577841#comment-16577841 ] Hive QA commented on HIVE-19924: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12935304/HIVE-19924.14.patch {color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified. {color:green}SUCCESS:{color} +1 due to 14877 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/13185/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13185/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13185/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase {noformat} This message is automatically generated. ATTACHMENT ID: 12935304 - PreCommit-HIVE-Build > Tag distcp jobs run by Repl Load > > > Key: HIVE-19924 > URL: https://issues.apache.org/jira/browse/HIVE-19924 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: DR, replication > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-19924.01.patch, HIVE-19924.02.patch, > HIVE-19924.03.patch, HIVE-19924.04.patch, HIVE-19924.05.patch, > HIVE-19924.06.patch, HIVE-19924.07.patch, HIVE-19924.08.patch, > HIVE-19924.09.patch, HIVE-19924.10.patch, HIVE-19924.11.patch, > HIVE-19924.12.patch, HIVE-19924.13.patch, HIVE-19924.14.patch > > > Add tags in jobconf for distcp related jobs started by replication. This will > allow hive to kill these jobs in case beacon retries, or hs2 dies and beacon > issues a kill command. > * one of the tags should definitely be the query_id that starts the job : > With this flow beacon before retrying the bootstrap load, will issue a kill > command to hs2 with the query id of the previous issued command. hs2 will > then kill an running jobs on yarn tagged with the Query_id. > * To get around the additional failure point as mentioned above. The jobs > can be tagged with an additional unique tag_id provided by Beacon in the WITH > clause in repl load command to be used to tag distcp jobs ). Enhance the kill > api to take the tag as input and kill jobs associated with that tag. Problem > here is how do we validate the association of the tag with a hive query id to > make sure this api is not used to kill jobs run by other components, however > we can provide this capability to only admins and should be ok in that case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16577830#comment-16577830 ] Hive QA commented on HIVE-19924: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 32s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m 30s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 16s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 16s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 41s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 8s{color} | {color:blue} ql in master has 2306 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 41s{color} | {color:blue} service in master has 48 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 45s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 8s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 40s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 16s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 16s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 22s{color} | {color:red} itests/hive-unit: The patch generated 4 new + 230 unchanged - 0 fixed = 234 total (was 230) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 43s{color} | {color:red} ql: The patch generated 2 new + 306 unchanged - 13 fixed = 308 total (was 319) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 55s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 46s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 14s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 35m 2s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-13185/dev-support/hive-personality.sh | | git revision | master / 4a30574 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-13185/yetus/diff-checkstyle-itests_hive-unit.txt | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-13185/yetus/diff-checkstyle-ql.txt | | modules | C: itests/hive-unit ql service U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-13185/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Tag distcp jobs run by Repl Load > > > Key: HIVE-19924 > URL: https://issues.apache.org/jira/browse/HIVE-19924 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: DR, replication > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-19924.01.patch, HIVE-19924.02.patch, > HIVE-19924.03.patch, HI
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16577092#comment-16577092 ] Hive QA commented on HIVE-19924: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12935234/HIVE-19924.12.patch {color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 14876 tests executed *Failed tests:* {noformat} org.apache.hive.jdbc.TestJdbcWithMiniLlapArrow.testKillQuery (batchId=252) org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testCancelRenewTokenFlow (batchId=265) org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testConnection (batchId=265) org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testIsValid (batchId=265) org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testIsValidNeg (batchId=265) org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testNegativeProxyAuth (batchId=265) org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testNegativeTokenAuth (batchId=265) org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testProxyAuth (batchId=265) org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testRenewDelegationToken (batchId=265) org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testTokenAuth (batchId=265) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/13169/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13169/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13169/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 10 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12935234 - PreCommit-HIVE-Build > Tag distcp jobs run by Repl Load > > > Key: HIVE-19924 > URL: https://issues.apache.org/jira/browse/HIVE-19924 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: DR, replication > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-19924.01.patch, HIVE-19924.02.patch, > HIVE-19924.03.patch, HIVE-19924.04.patch, HIVE-19924.05.patch, > HIVE-19924.06.patch, HIVE-19924.07.patch, HIVE-19924.08.patch, > HIVE-19924.09.patch, HIVE-19924.10.patch, HIVE-19924.11.patch, > HIVE-19924.12.patch > > > Add tags in jobconf for distcp related jobs started by replication. This will > allow hive to kill these jobs in case beacon retries, or hs2 dies and beacon > issues a kill command. > * one of the tags should definitely be the query_id that starts the job : > With this flow beacon before retrying the bootstrap load, will issue a kill > command to hs2 with the query id of the previous issued command. hs2 will > then kill an running jobs on yarn tagged with the Query_id. > * To get around the additional failure point as mentioned above. The jobs > can be tagged with an additional unique tag_id provided by Beacon in the WITH > clause in repl load command to be used to tag distcp jobs ). Enhance the kill > api to take the tag as input and kill jobs associated with that tag. Problem > here is how do we validate the association of the tag with a hive query id to > make sure this api is not used to kill jobs run by other components, however > we can provide this capability to only admins and should be ok in that case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16577085#comment-16577085 ] Hive QA commented on HIVE-19924: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 25s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m 57s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 21s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 21s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 42s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 11s{color} | {color:blue} ql in master has 2305 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 41s{color} | {color:blue} service in master has 48 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 46s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 46s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 21s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 21s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 20s{color} | {color:red} itests/hive-unit: The patch generated 17 new + 230 unchanged - 0 fixed = 247 total (was 230) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 43s{color} | {color:red} ql: The patch generated 2 new + 306 unchanged - 13 fixed = 308 total (was 319) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m 3s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 49s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 15s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 36m 1s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-13169/dev-support/hive-personality.sh | | git revision | master / 380946c | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-13169/yetus/diff-checkstyle-itests_hive-unit.txt | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-13169/yetus/diff-checkstyle-ql.txt | | modules | C: itests/hive-unit ql service U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-13169/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Tag distcp jobs run by Repl Load > > > Key: HIVE-19924 > URL: https://issues.apache.org/jira/browse/HIVE-19924 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: DR, replication > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-19924.01.patch, HIVE-19924.02.patch, > HIVE-19924.03.patch, H
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16574798#comment-16574798 ] Hive QA commented on HIVE-19924: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12934944/HIVE-19924.11.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14873 tests executed *Failed tests:* {noformat} org.apache.hive.jdbc.TestJdbcWithMiniHS2.testKillQuery (batchId=252) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/13122/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13122/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13122/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12934944 - PreCommit-HIVE-Build > Tag distcp jobs run by Repl Load > > > Key: HIVE-19924 > URL: https://issues.apache.org/jira/browse/HIVE-19924 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: DR, replication > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-19924.01.patch, HIVE-19924.02.patch, > HIVE-19924.03.patch, HIVE-19924.04.patch, HIVE-19924.05.patch, > HIVE-19924.06.patch, HIVE-19924.07.patch, HIVE-19924.08.patch, > HIVE-19924.09.patch, HIVE-19924.10.patch, HIVE-19924.11.patch > > > Add tags in jobconf for distcp related jobs started by replication. This will > allow hive to kill these jobs in case beacon retries, or hs2 dies and beacon > issues a kill command. > * one of the tags should definitely be the query_id that starts the job : > With this flow beacon before retrying the bootstrap load, will issue a kill > command to hs2 with the query id of the previous issued command. hs2 will > then kill an running jobs on yarn tagged with the Query_id. > * To get around the additional failure point as mentioned above. The jobs > can be tagged with an additional unique tag_id provided by Beacon in the WITH > clause in repl load command to be used to tag distcp jobs ). Enhance the kill > api to take the tag as input and kill jobs associated with that tag. Problem > here is how do we validate the association of the tag with a hive query id to > make sure this api is not used to kill jobs run by other components, however > we can provide this capability to only admins and should be ok in that case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16574753#comment-16574753 ] Hive QA commented on HIVE-19924: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 40s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 56s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 14s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 15s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 43s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 16s{color} | {color:blue} ql in master has 2305 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 42s{color} | {color:blue} service in master has 48 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 46s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 36s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 17s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 17s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 43s{color} | {color:red} ql: The patch generated 3 new + 306 unchanged - 13 fixed = 309 total (was 319) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 14s{color} | {color:red} service: The patch generated 2 new + 123 unchanged - 0 fixed = 125 total (was 123) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 52s{color} | {color:red} service generated 1 new + 48 unchanged - 0 fixed = 49 total (was 48) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 54s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 15s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 34m 55s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:service | | | Redundant nullcheck of operation, which is known to be non-null in org.apache.hive.service.cli.operation.OperationManager.closeOperation(OperationHandle) Redundant null check at OperationManager.java:is known to be non-null in org.apache.hive.service.cli.operation.OperationManager.closeOperation(OperationHandle) Redundant null check at OperationManager.java:[line 307] | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-13122/dev-support/hive-personality.sh | | git revision | master / 873d31f | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-13122/yetus/diff-checkstyle-ql.txt | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-13122/yetus/diff-checkstyle-service.txt | | findbugs | http://104.198.109.242/logs//PreCommit-HIVE-Build-13122/yetus/new-findbugs-service.html | | modules | C: itests/hive-unit ql service U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-13122/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Tag
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16571757#comment-16571757 ] Hive QA commented on HIVE-19924: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 41s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 21s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m 12s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 32s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 38s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 54s{color} | {color:blue} ql in master has 2304 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 38s{color} | {color:blue} service in master has 48 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m 18s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 7s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m 53s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 45s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m 45s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 39s{color} | {color:red} ql: The patch generated 1 new + 55 unchanged - 12 fixed = 56 total (was 67) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 14s{color} | {color:red} service: The patch generated 2 new + 123 unchanged - 0 fixed = 125 total (was 123) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 46s{color} | {color:red} service generated 1 new + 48 unchanged - 0 fixed = 49 total (was 48) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 15m 25s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 18s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 71m 47s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:service | | | Redundant nullcheck of operation, which is known to be non-null in org.apache.hive.service.cli.operation.OperationManager.closeOperation(OperationHandle) Redundant null check at OperationManager.java:is known to be non-null in org.apache.hive.service.cli.operation.OperationManager.closeOperation(OperationHandle) Redundant null check at OperationManager.java:[line 307] | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense xml javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-13081/dev-support/hive-personality.sh | | git revision | master / 5391651 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-13081/yetus/diff-checkstyle-ql.txt | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-13081/yetus/diff-checkstyle-service.txt | | findbugs | http://104.198.109.242/logs//PreCommit-HIVE-Build-13081/yetus/new-findbugs-service.html | | modules | C: . itests/hive-unit ql service U: . | | Console output | http://104.198.109.242/l
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16571741#comment-16571741 ] Hive QA commented on HIVE-19924: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12934639/HIVE-19924.10.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 14868 tests executed *Failed tests:* {noformat} org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitions (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsMultiInsert (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsUnionAll (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedFiles (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerDagRawInputSplitsKill (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerDefaultRawInputSplits (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighShuffleBytes (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerVertexRawInputSplitsKill (batchId=251) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/13081/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13081/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13081/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 8 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12934639 - PreCommit-HIVE-Build > Tag distcp jobs run by Repl Load > > > Key: HIVE-19924 > URL: https://issues.apache.org/jira/browse/HIVE-19924 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: DR, replication > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-19924.01.patch, HIVE-19924.02.patch, > HIVE-19924.03.patch, HIVE-19924.04.patch, HIVE-19924.05.patch, > HIVE-19924.06.patch, HIVE-19924.07.patch, HIVE-19924.08.patch, > HIVE-19924.09.patch, HIVE-19924.10.patch > > > Add tags in jobconf for distcp related jobs started by replication. This will > allow hive to kill these jobs in case beacon retries, or hs2 dies and beacon > issues a kill command. > * one of the tags should definitely be the query_id that starts the job : > With this flow beacon before retrying the bootstrap load, will issue a kill > command to hs2 with the query id of the previous issued command. hs2 will > then kill an running jobs on yarn tagged with the Query_id. > * To get around the additional failure point as mentioned above. The jobs > can be tagged with an additional unique tag_id provided by Beacon in the WITH > clause in repl load command to be used to tag distcp jobs ). Enhance the kill > api to take the tag as input and kill jobs associated with that tag. Problem > here is how do we validate the association of the tag with a hive query id to > make sure this api is not used to kill jobs run by other components, however > we can provide this capability to only admins and should be ok in that case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16568135#comment-16568135 ] Hive QA commented on HIVE-19924: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 49s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m 23s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m 44s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 38s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 44s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 17s{color} | {color:blue} ql in master has 2301 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 45s{color} | {color:blue} service in master has 48 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 9m 7s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 8s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m 52s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m 51s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m 51s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 41s{color} | {color:red} ql: The patch generated 1 new + 55 unchanged - 12 fixed = 56 total (was 67) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 15s{color} | {color:red} service: The patch generated 2 new + 123 unchanged - 0 fixed = 125 total (was 123) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 43s{color} | {color:red} patch/itests/hive-unit cannot run setBugDatabaseInfo from findbugs {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 6m 16s{color} | {color:red} patch/ql cannot run setBugDatabaseInfo from findbugs {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 25s{color} | {color:red} patch/service cannot run setBugDatabaseInfo from findbugs {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 14m 5s{color} | {color:green} root generated 0 new + 378 unchanged - 7 fixed = 378 total (was 385) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s{color} | {color:green} hive-unit in the patch passed. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 27s{color} | {color:green} ql in the patch passed. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 22s{color} | {color:green} service in the patch passed. {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 17s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 80m 49s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense xml javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-13021/dev-support/hive-personality.sh | | git revision | master / 66ea326 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//Pre
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16568119#comment-16568119 ] Hive QA commented on HIVE-19924: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12934243/HIVE-19924.09.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 23 failed/errored test(s), 14582 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.org.apache.hadoop.hive.cli.TestMiniDruidCliDriver (batchId=191) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.org.apache.hadoop.hive.cli.TestMiniDruidCliDriver (batchId=192) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.org.apache.hadoop.hive.cli.TestMiniDruidCliDriver (batchId=193) org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver (batchId=263) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver (batchId=150) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver (batchId=151) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver (batchId=152) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver (batchId=153) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver (batchId=154) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver (batchId=155) org.apache.hadoop.hive.cli.TestMiniTezCliDriver.org.apache.hadoop.hive.cli.TestMiniTezCliDriver (batchId=106) org.apache.hadoop.hive.cli.TestMiniTezCliDriver.org.apache.hadoop.hive.cli.TestMiniTezCliDriver (batchId=107) org.apache.hadoop.hive.cli.TestTezPerfCliDriver.org.apache.hadoop.hive.cli.TestTezPerfCliDriver (batchId=263) org.apache.hive.beeline.TestBeeLineWithArgs.org.apache.hive.beeline.TestBeeLineWithArgs (batchId=245) org.apache.hive.jdbc.TestJdbcDriver2.testGetQueryId (batchId=250) org.apache.hive.jdbc.TestJdbcGenericUDTFGetSplits.org.apache.hive.jdbc.TestJdbcGenericUDTFGetSplits (batchId=248) org.apache.hive.jdbc.TestJdbcWithMiniHS2.testKillQuery (batchId=251) org.apache.hive.jdbc.TestJdbcWithMiniLlapArrow.org.apache.hive.jdbc.TestJdbcWithMiniLlapArrow (batchId=251) org.apache.hive.jdbc.TestJdbcWithMiniLlapRow.org.apache.hive.jdbc.TestJdbcWithMiniLlapRow (batchId=248) org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.org.apache.hive.jdbc.TestTriggersMoveWorkloadManager (batchId=251) org.apache.hive.jdbc.TestTriggersNoTezSessionPool.org.apache.hive.jdbc.TestTriggersNoTezSessionPool (batchId=248) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.org.apache.hive.jdbc.TestTriggersTezSessionPoolManager (batchId=251) org.apache.hive.service.cli.operation.TestOperationLoggingAPIWithTez.org.apache.hive.service.cli.operation.TestOperationLoggingAPIWithTez (batchId=247) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/13021/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13021/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13021/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 23 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12934243 - PreCommit-HIVE-Build > Tag distcp jobs run by Repl Load > > > Key: HIVE-19924 > URL: https://issues.apache.org/jira/browse/HIVE-19924 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: DR, replication > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-19924.01.patch, HIVE-19924.02.patch, > HIVE-19924.03.patch, HIVE-19924.04.patch, HIVE-19924.05.patch, > HIVE-19924.06.patch, HIVE-19924.07.patch, HIVE-19924.08.patch, > HIVE-19924.09.patch > > > Add tags in jobconf for distcp related jobs started by replication. This will > allow hive to kill these jobs in case beacon retries, or hs2 dies and beacon > issues a kill command. > * one of the tags should definitely be the query_id that starts the job : > With this flow beacon before retrying the bootstrap load, will issue a kill > command to hs2 with the query id of the previous issued command. hs2 will > then kill an running jobs on yarn ta
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563547#comment-16563547 ] Hive QA commented on HIVE-19924: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12933701/HIVE-19924.08.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 29 failed/errored test(s), 14802 tests executed *Failed tests:* {noformat} TestJdbcWithMiniHS2 - did not produce a TEST-*.xml file (likely timed out) (batchId=251) TestJdbcWithMiniHS2ErasureCoding - did not produce a TEST-*.xml file (likely timed out) (batchId=251) TestNoSaslAuth - did not produce a TEST-*.xml file (likely timed out) (batchId=251) org.apache.hive.jdbc.TestJdbcDriver2.testGetQueryId (batchId=250) org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveAndKill (batchId=251) org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveBackKill (batchId=251) org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveConflictKill (batchId=251) org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerDAGTotalTasks (batchId=248) org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerSlowQueryExecutionTime (batchId=248) org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerTotalLaunchedTasks (batchId=248) org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerVertexTotalTasks (batchId=248) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testMultipleTriggers1 (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testMultipleTriggers2 (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitions (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsMultiInsert (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsUnionAll (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedFiles (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomReadOps (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerDagRawInputSplitsKill (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerDagTotalTasks (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerDefaultRawInputSplits (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighBytesRead (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighBytesWrite (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighShuffleBytes (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerShortQueryElapsedTime (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerSlowQueryElapsedTime (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerSlowQueryExecutionTime (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerTotalTasks (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerVertexRawInputSplitsKill (batchId=251) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12963/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12963/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12963/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 29 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12933701 - PreCommit-HIVE-Build > Tag distcp jobs run by Repl Load > > > Key: HIVE-19924 > URL: https://issues.apache.org/jira/browse/HIVE-19924 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: DR, replication > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-19924.01.patch, HIVE-19924.02.patch, > HIVE-19924.03.patch, HIVE-19924.04.patch, HIVE-19924.05.patch, > HIVE-19924.06.patch, HIVE-19924.07.patch, HIVE-19924.08.patch > > > Add tags in jobconf for distcp related jobs started by replication. This will > allow hive to kill these jobs in case beacon retries, or hs2 dies and beacon > issues a kill command. > * one of the tags should
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563504#comment-16563504 ] Hive QA commented on HIVE-19924: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 51s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 48s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 20s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 14s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 40s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 25s{color} | {color:blue} ql in master has 2306 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 41s{color} | {color:blue} service in master has 48 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 49s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 41s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 17s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 17s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 39s{color} | {color:red} ql: The patch generated 1 new + 55 unchanged - 12 fixed = 56 total (was 67) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 15s{color} | {color:red} service: The patch generated 2 new + 123 unchanged - 0 fixed = 125 total (was 123) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 51s{color} | {color:red} service generated 1 new + 48 unchanged - 0 fixed = 49 total (was 48) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 50s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 14s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 35m 23s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:service | | | Redundant nullcheck of operation, which is known to be non-null in org.apache.hive.service.cli.operation.OperationManager.closeOperation(OperationHandle) Redundant null check at OperationManager.java:is known to be non-null in org.apache.hive.service.cli.operation.OperationManager.closeOperation(OperationHandle) Redundant null check at OperationManager.java:[line 307] | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-12963/dev-support/hive-personality.sh | | git revision | master / edc53cc | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-12963/yetus/diff-checkstyle-ql.txt | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-12963/yetus/diff-checkstyle-service.txt | | findbugs | http://104.198.109.242/logs//PreCommit-HIVE-Build-12963/yetus/new-findbugs-service.html | | modules | C: itests/hive-unit ql service U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12963/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Tag dis
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16561035#comment-16561035 ] Hive QA commented on HIVE-19924: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12933502/HIVE-19924.07.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 33 failed/errored test(s), 14782 tests executed *Failed tests:* {noformat} TestJdbcWithMiniHS2 - did not produce a TEST-*.xml file (likely timed out) (batchId=251) TestJdbcWithMiniHS2ErasureCoding - did not produce a TEST-*.xml file (likely timed out) (batchId=251) TestNoSaslAuth - did not produce a TEST-*.xml file (likely timed out) (batchId=251) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druid_timestamptz] (batchId=193) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_joins] (batchId=193) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_masking] (batchId=193) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test1] (batchId=193) org.apache.hive.jdbc.TestJdbcDriver2.testGetQueryId (batchId=250) org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveAndKill (batchId=251) org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveBackKill (batchId=251) org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveConflictKill (batchId=251) org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerDAGTotalTasks (batchId=248) org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerSlowQueryExecutionTime (batchId=248) org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerTotalLaunchedTasks (batchId=248) org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerVertexTotalTasks (batchId=248) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testMultipleTriggers1 (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testMultipleTriggers2 (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitions (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsMultiInsert (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsUnionAll (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedFiles (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomReadOps (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerDagRawInputSplitsKill (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerDagTotalTasks (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerDefaultRawInputSplits (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighBytesRead (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighBytesWrite (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighShuffleBytes (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerShortQueryElapsedTime (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerSlowQueryElapsedTime (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerSlowQueryExecutionTime (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerTotalTasks (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerVertexRawInputSplitsKill (batchId=251) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12926/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12926/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12926/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 33 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12933502 - PreCommit-HIVE-Build > Tag distcp jobs run by Repl Load > > > Key: HIVE-19924 > URL: https://issues.apache.org/jira/browse/HIVE-19924 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: DR, replication > Fix For: 4.0.0, 3.2.0 > > Attachments: HI
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16561030#comment-16561030 ] Hive QA commented on HIVE-19924: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 31s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 49s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 9s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 6s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 37s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 1s{color} | {color:blue} ql in master has 2297 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 39s{color} | {color:blue} service in master has 48 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 40s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 34s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 18s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 18s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 42s{color} | {color:red} ql: The patch generated 1 new + 55 unchanged - 12 fixed = 56 total (was 67) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 14s{color} | {color:red} service: The patch generated 2 new + 123 unchanged - 0 fixed = 125 total (was 123) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 54s{color} | {color:red} service generated 1 new + 48 unchanged - 0 fixed = 49 total (was 48) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 52s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 14s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 34m 20s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:service | | | Redundant nullcheck of operation, which is known to be non-null in org.apache.hive.service.cli.operation.OperationManager.closeOperation(OperationHandle) Redundant null check at OperationManager.java:is known to be non-null in org.apache.hive.service.cli.operation.OperationManager.closeOperation(OperationHandle) Redundant null check at OperationManager.java:[line 307] | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-12926/dev-support/hive-personality.sh | | git revision | master / beb57c6 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-12926/yetus/diff-checkstyle-ql.txt | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-12926/yetus/diff-checkstyle-service.txt | | findbugs | http://104.198.109.242/logs//PreCommit-HIVE-Build-12926/yetus/new-findbugs-service.html | | modules | C: itests/hive-unit ql service U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12926/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Tag dis
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16552047#comment-16552047 ] Hive QA commented on HIVE-19924: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12932599/HIVE-19924.06.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 29 failed/errored test(s), 14649 tests executed *Failed tests:* {noformat} TestJdbcWithMiniHS2 - did not produce a TEST-*.xml file (likely timed out) (batchId=251) TestJdbcWithMiniHS2ErasureCoding - did not produce a TEST-*.xml file (likely timed out) (batchId=251) TestNoSaslAuth - did not produce a TEST-*.xml file (likely timed out) (batchId=251) org.apache.hive.jdbc.TestJdbcDriver2.testGetQueryId (batchId=250) org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveAndKill (batchId=251) org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveBackKill (batchId=251) org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveConflictKill (batchId=251) org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerDAGTotalTasks (batchId=248) org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerSlowQueryExecutionTime (batchId=248) org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerTotalLaunchedTasks (batchId=248) org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerVertexTotalTasks (batchId=248) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testMultipleTriggers1 (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testMultipleTriggers2 (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitions (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsMultiInsert (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsUnionAll (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedFiles (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomReadOps (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerDagRawInputSplitsKill (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerDagTotalTasks (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerDefaultRawInputSplits (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighBytesRead (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighBytesWrite (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighShuffleBytes (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerShortQueryElapsedTime (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerSlowQueryElapsedTime (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerSlowQueryExecutionTime (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerTotalTasks (batchId=251) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerVertexRawInputSplitsKill (batchId=251) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12779/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12779/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12779/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 29 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12932599 - PreCommit-HIVE-Build > Tag distcp jobs run by Repl Load > > > Key: HIVE-19924 > URL: https://issues.apache.org/jira/browse/HIVE-19924 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: DR, replication > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-19924.01.patch, HIVE-19924.02.patch, > HIVE-19924.03.patch, HIVE-19924.04.patch, HIVE-19924.05.patch, > HIVE-19924.06.patch > > > Add tags in jobconf for distcp related jobs started by replication. This will > allow hive to kill these jobs in case beacon retries, or hs2 dies and beacon > issues a kill command. > * one of the tags should definitely be the query_id that starts th
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16552034#comment-16552034 ] Hive QA commented on HIVE-19924: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 1s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 40s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 8s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 2s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 7s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 38s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 52s{color} | {color:blue} ql in master has 2280 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 39s{color} | {color:blue} service in master has 48 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 34s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 29s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 7s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 7s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 36s{color} | {color:red} ql: The patch generated 1 new + 55 unchanged - 12 fixed = 56 total (was 67) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 13s{color} | {color:red} service: The patch generated 2 new + 123 unchanged - 0 fixed = 125 total (was 123) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 46s{color} | {color:red} service generated 1 new + 48 unchanged - 0 fixed = 49 total (was 48) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 34s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 14s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 31m 53s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:service | | | Redundant nullcheck of operation, which is known to be non-null in org.apache.hive.service.cli.operation.OperationManager.closeOperation(OperationHandle) Redundant null check at OperationManager.java:is known to be non-null in org.apache.hive.service.cli.operation.OperationManager.closeOperation(OperationHandle) Redundant null check at OperationManager.java:[line 307] | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-12779/dev-support/hive-personality.sh | | git revision | master / cce3a05 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-12779/yetus/diff-checkstyle-ql.txt | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-12779/yetus/diff-checkstyle-service.txt | | findbugs | http://104.198.109.242/logs//PreCommit-HIVE-Build-12779/yetus/new-findbugs-service.html | | modules | C: itests/hive-unit ql service U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12779/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Tag dis
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16551408#comment-16551408 ] Hive QA commented on HIVE-19924: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12932369/HIVE-19924.05.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 29 failed/errored test(s), 14648 tests executed *Failed tests:* {noformat} TestJdbcWithMiniHS2 - did not produce a TEST-*.xml file (likely timed out) (batchId=250) TestJdbcWithMiniHS2ErasureCoding - did not produce a TEST-*.xml file (likely timed out) (batchId=250) TestNoSaslAuth - did not produce a TEST-*.xml file (likely timed out) (batchId=250) org.apache.hive.jdbc.TestJdbcDriver2.testGetQueryId (batchId=249) org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveAndKill (batchId=250) org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveBackKill (batchId=250) org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveConflictKill (batchId=250) org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerDAGTotalTasks (batchId=247) org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerSlowQueryExecutionTime (batchId=247) org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerTotalLaunchedTasks (batchId=247) org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerVertexTotalTasks (batchId=247) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testMultipleTriggers1 (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testMultipleTriggers2 (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitions (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsMultiInsert (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsUnionAll (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedFiles (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomReadOps (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerDagRawInputSplitsKill (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerDagTotalTasks (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerDefaultRawInputSplits (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighBytesRead (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighBytesWrite (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighShuffleBytes (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerShortQueryElapsedTime (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerSlowQueryElapsedTime (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerSlowQueryExecutionTime (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerTotalTasks (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerVertexRawInputSplitsKill (batchId=250) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12744/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12744/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12744/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 29 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12932369 - PreCommit-HIVE-Build > Tag distcp jobs run by Repl Load > > > Key: HIVE-19924 > URL: https://issues.apache.org/jira/browse/HIVE-19924 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: DR, replication > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-19924.01.patch, HIVE-19924.02.patch, > HIVE-19924.03.patch, HIVE-19924.04.patch, HIVE-19924.05.patch > > > Add tags in jobconf for distcp related jobs started by replication. This will > allow hive to kill these jobs in case beacon retries, or hs2 dies and beacon > issues a kill command. > * one of the tags should definitely be the query_id that starts the job : > With this flo
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16551370#comment-16551370 ] Hive QA commented on HIVE-19924: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 48s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 44s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 15s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 12s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 40s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 10s{color} | {color:blue} ql in master has 2281 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 39s{color} | {color:blue} service in master has 48 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 45s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 34s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 17s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 17s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 41s{color} | {color:red} ql: The patch generated 1 new + 55 unchanged - 12 fixed = 56 total (was 67) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 14s{color} | {color:red} service: The patch generated 2 new + 123 unchanged - 0 fixed = 125 total (was 123) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 46s{color} | {color:red} service generated 1 new + 48 unchanged - 0 fixed = 49 total (was 48) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 44s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 14s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 34m 16s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:service | | | Redundant nullcheck of operation, which is known to be non-null in org.apache.hive.service.cli.operation.OperationManager.closeOperation(OperationHandle) Redundant null check at OperationManager.java:is known to be non-null in org.apache.hive.service.cli.operation.OperationManager.closeOperation(OperationHandle) Redundant null check at OperationManager.java:[line 307] | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-12744/dev-support/hive-personality.sh | | git revision | master / 06a4f98 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-12744/yetus/diff-checkstyle-ql.txt | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-12744/yetus/diff-checkstyle-service.txt | | findbugs | http://104.198.109.242/logs//PreCommit-HIVE-Build-12744/yetus/new-findbugs-service.html | | modules | C: itests/hive-unit ql service U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12744/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Tag dis
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16550105#comment-16550105 ] Hive QA commented on HIVE-19924: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12932219/HIVE-19924.04.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12713/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12713/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12713/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Tests exited with: Exception: Patch URL https://issues.apache.org/jira/secure/attachment/12932219/HIVE-19924.04.patch was found in seen patch url's cache and a test was probably run already on it. Aborting... {noformat} This message is automatically generated. ATTACHMENT ID: 12932219 - PreCommit-HIVE-Build > Tag distcp jobs run by Repl Load > > > Key: HIVE-19924 > URL: https://issues.apache.org/jira/browse/HIVE-19924 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: DR, replication > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-19924.01.patch, HIVE-19924.02.patch, > HIVE-19924.03.patch, HIVE-19924.04.patch > > > Add tags in jobconf for distcp related jobs started by replication. This will > allow hive to kill these jobs in case beacon retries, or hs2 dies and beacon > issues a kill command. > * one of the tags should definitely be the query_id that starts the job : > With this flow beacon before retrying the bootstrap load, will issue a kill > command to hs2 with the query id of the previous issued command. hs2 will > then kill an running jobs on yarn tagged with the Query_id. > * To get around the additional failure point as mentioned above. The jobs > can be tagged with an additional unique tag_id provided by Beacon in the WITH > clause in repl load command to be used to tag distcp jobs ). Enhance the kill > api to take the tag as input and kill jobs associated with that tag. Problem > here is how do we validate the association of the tag with a hive query id to > make sure this api is not used to kill jobs run by other components, however > we can provide this capability to only admins and should be ok in that case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16550103#comment-16550103 ] Hive QA commented on HIVE-19924: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12932219/HIVE-19924.04.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 29 failed/errored test(s), 14648 tests executed *Failed tests:* {noformat} TestJdbcWithMiniHS2 - did not produce a TEST-*.xml file (likely timed out) (batchId=250) TestJdbcWithMiniHS2ErasureCoding - did not produce a TEST-*.xml file (likely timed out) (batchId=250) TestNoSaslAuth - did not produce a TEST-*.xml file (likely timed out) (batchId=250) org.apache.hive.jdbc.TestJdbcDriver2.testGetQueryId (batchId=249) org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveAndKill (batchId=250) org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveBackKill (batchId=250) org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveConflictKill (batchId=250) org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerDAGTotalTasks (batchId=247) org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerSlowQueryExecutionTime (batchId=247) org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerTotalLaunchedTasks (batchId=247) org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerVertexTotalTasks (batchId=247) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testMultipleTriggers1 (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testMultipleTriggers2 (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitions (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsMultiInsert (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsUnionAll (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedFiles (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomReadOps (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerDagRawInputSplitsKill (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerDagTotalTasks (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerDefaultRawInputSplits (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighBytesRead (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighBytesWrite (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighShuffleBytes (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerShortQueryElapsedTime (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerSlowQueryElapsedTime (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerSlowQueryExecutionTime (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerTotalTasks (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerVertexRawInputSplitsKill (batchId=250) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12712/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12712/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12712/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 29 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12932219 - PreCommit-HIVE-Build > Tag distcp jobs run by Repl Load > > > Key: HIVE-19924 > URL: https://issues.apache.org/jira/browse/HIVE-19924 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: DR, replication > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-19924.01.patch, HIVE-19924.02.patch, > HIVE-19924.03.patch, HIVE-19924.04.patch > > > Add tags in jobconf for distcp related jobs started by replication. This will > allow hive to kill these jobs in case beacon retries, or hs2 dies and beacon > issues a kill command. > * one of the tags should definitely be the query_id that starts the job : > With this flow beacon before retry
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16550085#comment-16550085 ] Hive QA commented on HIVE-19924: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 50s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 41s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 16s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 10s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 39s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 5s{color} | {color:blue} ql in master has 2281 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 40s{color} | {color:blue} service in master has 48 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 42s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 37s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 13s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 13s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 42s{color} | {color:red} ql: The patch generated 1 new + 55 unchanged - 12 fixed = 56 total (was 67) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 13s{color} | {color:red} service: The patch generated 2 new + 123 unchanged - 0 fixed = 125 total (was 123) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 49s{color} | {color:red} service generated 1 new + 48 unchanged - 0 fixed = 49 total (was 48) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 42s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 13s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 34m 9s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:service | | | Redundant nullcheck of operation, which is known to be non-null in org.apache.hive.service.cli.operation.OperationManager.closeOperation(OperationHandle) Redundant null check at OperationManager.java:is known to be non-null in org.apache.hive.service.cli.operation.OperationManager.closeOperation(OperationHandle) Redundant null check at OperationManager.java:[line 307] | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-12712/dev-support/hive-personality.sh | | git revision | master / 851c8ab | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-12712/yetus/diff-checkstyle-ql.txt | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-12712/yetus/diff-checkstyle-service.txt | | findbugs | http://104.198.109.242/logs//PreCommit-HIVE-Build-12712/yetus/new-findbugs-service.html | | modules | C: itests/hive-unit ql service U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12712/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Tag dis
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16549102#comment-16549102 ] Sankar Hariappan commented on HIVE-19924: - +1 for 04.patch, pending tests > Tag distcp jobs run by Repl Load > > > Key: HIVE-19924 > URL: https://issues.apache.org/jira/browse/HIVE-19924 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: DR, replication > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-19924.01.patch, HIVE-19924.02.patch, > HIVE-19924.03.patch, HIVE-19924.04.patch > > > Add tags in jobconf for distcp related jobs started by replication. This will > allow hive to kill these jobs in case beacon retries, or hs2 dies and beacon > issues a kill command. > * one of the tags should definitely be the query_id that starts the job : > With this flow beacon before retrying the bootstrap load, will issue a kill > command to hs2 with the query id of the previous issued command. hs2 will > then kill an running jobs on yarn tagged with the Query_id. > * To get around the additional failure point as mentioned above. The jobs > can be tagged with an additional unique tag_id provided by Beacon in the WITH > clause in repl load command to be used to tag distcp jobs ). Enhance the kill > api to take the tag as input and kill jobs associated with that tag. Problem > here is how do we validate the association of the tag with a hive query id to > make sure this api is not used to kill jobs run by other components, however > we can provide this capability to only admins and should be ok in that case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16548593#comment-16548593 ] Hive QA commented on HIVE-19924: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12932029/HIVE-19924.03.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 39 failed/errored test(s), 14663 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.ql.parse.TestMacroSemanticAnalyzer.testDropMacro (batchId=288) org.apache.hadoop.hive.ql.parse.TestMacroSemanticAnalyzer.testDropMacroDoesNotExist (batchId=288) org.apache.hadoop.hive.ql.parse.TestMacroSemanticAnalyzer.testDropMacroExistsDoNotIgnoreErrors (batchId=288) org.apache.hadoop.hive.ql.parse.TestMacroSemanticAnalyzer.testDropMacroNonExistentWithIfExists (batchId=288) org.apache.hadoop.hive.ql.parse.TestMacroSemanticAnalyzer.testDropMacroNonExistentWithIfExistsDoNotIgnoreNonExistent (batchId=288) org.apache.hadoop.hive.ql.parse.TestMacroSemanticAnalyzer.testOneInputParamters (batchId=288) org.apache.hadoop.hive.ql.parse.TestMacroSemanticAnalyzer.testThreeInputParamters (batchId=288) org.apache.hadoop.hive.ql.parse.TestMacroSemanticAnalyzer.testTwoInputParamters (batchId=288) org.apache.hadoop.hive.ql.parse.TestMacroSemanticAnalyzer.testZeroInputParamters (batchId=288) org.apache.hive.jdbc.TestJdbcDriver2.testGetQueryId (batchId=249) org.apache.hive.jdbc.TestJdbcDriver2.testReplErrorScenarios (batchId=249) org.apache.hive.jdbc.TestJdbcWithMiniHS2.testKillQuery (batchId=250) org.apache.hive.jdbc.TestJdbcWithMiniLlapArrow.testKillQuery (batchId=250) org.apache.hive.jdbc.TestJdbcWithMiniLlapRow.testKillQuery (batchId=247) org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveAndKill (batchId=250) org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveBackKill (batchId=250) org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveConflictKill (batchId=250) org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerDAGTotalTasks (batchId=247) org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerSlowQueryExecutionTime (batchId=247) org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerTotalLaunchedTasks (batchId=247) org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerVertexTotalTasks (batchId=247) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testMultipleTriggers1 (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testMultipleTriggers2 (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitions (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsMultiInsert (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsUnionAll (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedFiles (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomReadOps (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerDagRawInputSplitsKill (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerDagTotalTasks (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerDefaultRawInputSplits (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighBytesRead (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighBytesWrite (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighShuffleBytes (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerShortQueryElapsedTime (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerSlowQueryElapsedTime (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerSlowQueryExecutionTime (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerTotalTasks (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerVertexRawInputSplitsKill (batchId=250) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12681/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12681/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12681/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 39 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16548568#comment-16548568 ] Hive QA commented on HIVE-19924: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 40s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 22s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 5s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 8s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 56s{color} | {color:blue} ql in master has 2274 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 40s{color} | {color:blue} service in master has 48 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 36s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 42s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 24s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 30s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 6s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 6s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 40s{color} | {color:red} ql: The patch generated 1 new + 55 unchanged - 12 fixed = 56 total (was 67) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 13s{color} | {color:red} service: The patch generated 2 new + 123 unchanged - 0 fixed = 125 total (was 123) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 48s{color} | {color:red} service generated 1 new + 48 unchanged - 0 fixed = 49 total (was 48) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 40s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 13s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 32m 51s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:service | | | Redundant nullcheck of operation, which is known to be non-null in org.apache.hive.service.cli.operation.OperationManager.closeOperation(OperationHandle) Redundant null check at OperationManager.java:is known to be non-null in org.apache.hive.service.cli.operation.OperationManager.closeOperation(OperationHandle) Redundant null check at OperationManager.java:[line 306] | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-12681/dev-support/hive-personality.sh | | git revision | master / da1f758 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-12681/yetus/diff-checkstyle-ql.txt | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-12681/yetus/diff-checkstyle-service.txt | | findbugs | http://104.198.109.242/logs//PreCommit-HIVE-Build-12681/yetus/new-findbugs-service.html | | modules | C: ql service itests/hive-unit U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12681/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Tag dis
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16542398#comment-16542398 ] Hive QA commented on HIVE-19924: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12931299/HIVE-19924.02.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 37 failed/errored test(s), 14619 tests executed *Failed tests:* {noformat} TestJdbcWithMiniHS2 - did not produce a TEST-*.xml file (likely timed out) (batchId=250) TestJdbcWithMiniHS2ErasureCoding - did not produce a TEST-*.xml file (likely timed out) (batchId=250) TestNoSaslAuth - did not produce a TEST-*.xml file (likely timed out) (batchId=250) org.apache.hadoop.hive.ql.parse.TestMacroSemanticAnalyzer.testDropMacro (batchId=288) org.apache.hadoop.hive.ql.parse.TestMacroSemanticAnalyzer.testDropMacroDoesNotExist (batchId=288) org.apache.hadoop.hive.ql.parse.TestMacroSemanticAnalyzer.testDropMacroExistsDoNotIgnoreErrors (batchId=288) org.apache.hadoop.hive.ql.parse.TestMacroSemanticAnalyzer.testDropMacroNonExistentWithIfExists (batchId=288) org.apache.hadoop.hive.ql.parse.TestMacroSemanticAnalyzer.testDropMacroNonExistentWithIfExistsDoNotIgnoreNonExistent (batchId=288) org.apache.hadoop.hive.ql.parse.TestMacroSemanticAnalyzer.testOneInputParamters (batchId=288) org.apache.hadoop.hive.ql.parse.TestMacroSemanticAnalyzer.testThreeInputParamters (batchId=288) org.apache.hadoop.hive.ql.parse.TestMacroSemanticAnalyzer.testTwoInputParamters (batchId=288) org.apache.hadoop.hive.ql.parse.TestMacroSemanticAnalyzer.testZeroInputParamters (batchId=288) org.apache.hive.jdbc.TestJdbcDriver2.testGetQueryId (batchId=249) org.apache.hive.jdbc.TestJdbcDriver2.testReplErrorScenarios (batchId=249) org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveAndKill (batchId=250) org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveBackKill (batchId=250) org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveConflictKill (batchId=250) org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerSlowQueryExecutionTime (batchId=247) org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerTotalLaunchedTasks (batchId=247) org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerVertexTotalTasks (batchId=247) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testMultipleTriggers1 (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testMultipleTriggers2 (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitions (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsMultiInsert (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsUnionAll (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedFiles (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomReadOps (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerDagRawInputSplitsKill (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerDagTotalTasks (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerDefaultRawInputSplits (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighBytesRead (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighShuffleBytes (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerShortQueryElapsedTime (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerSlowQueryElapsedTime (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerSlowQueryExecutionTime (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerTotalTasks (batchId=250) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerVertexRawInputSplitsKill (batchId=250) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12568/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12568/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12568/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 37 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12931299 - PreCommit-HIVE-Build > Tag distcp jobs run by Repl Load > > > Key:
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16542386#comment-16542386 ] Hive QA commented on HIVE-19924: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 58s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 42s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 36s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 21s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 39s{color} | {color:blue} ql in master has 2289 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 46s{color} | {color:blue} service in master has 48 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 48s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 58s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 29s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 1s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 36s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 36s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 45s{color} | {color:red} ql: The patch generated 1 new + 55 unchanged - 12 fixed = 56 total (was 67) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 16s{color} | {color:red} service: The patch generated 2 new + 123 unchanged - 0 fixed = 125 total (was 123) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m 48s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 59s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 15s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 39m 19s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-12568/dev-support/hive-personality.sh | | git revision | master / 57dd304 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-12568/yetus/diff-checkstyle-ql.txt | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-12568/yetus/diff-checkstyle-service.txt | | modules | C: ql service itests/hive-unit U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12568/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Tag distcp jobs run by Repl Load > > > Key: HIVE-19924 > URL: https://issues.apache.org/jira/browse/HIVE-19924 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: DR, replication > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-19924.01.patch, HIVE-19924.02.patch > > > Add tags in jobconf for distcp related job
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16525443#comment-16525443 ] Vineet Garg commented on HIVE-19924: Deferring this to 3.2.0 since the branch for 3.1.0 has been cut off. > Tag distcp jobs run by Repl Load > > > Key: HIVE-19924 > URL: https://issues.apache.org/jira/browse/HIVE-19924 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-19924.01.patch > > > Add tags in jobconf for distcp related jobs started by replication. This will > allow hive to kill these jobs in case beacon retries, or hs2 dies and beacon > issues a kill command. > * one of the tags should definitely be the query_id that starts the job : > With this flow beacon before retrying the bootstrap load, will issue a kill > command to hs2 with the query id of the previous issued command. hs2 will > then kill an running jobs on yarn tagged with the Query_id. > * To get around the additional failure point as mentioned above. The jobs > can be tagged with an additional unique tag_id provided by Beacon in the WITH > clause in repl load command to be used to tag distcp jobs ). Enhance the kill > api to take the tag as input and kill jobs associated with that tag. Problem > here is how do we validate the association of the tag with a hive query id to > make sure this api is not used to kill jobs run by other components, however > we can provide this capability to only admins and should be ok in that case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522048#comment-16522048 ] Hive QA commented on HIVE-19924: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12928918/HIVE-19924.01.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12086/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12086/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12086/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N' 2018-06-25 08:53:50.165 + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]] + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'MAVEN_OPTS=-Xmx1g ' + MAVEN_OPTS='-Xmx1g ' + cd /data/hiveptest/working/ + tee /data/hiveptest/logs/PreCommit-HIVE-Build-12086/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ git = \s\v\n ]] + [[ git = \g\i\t ]] + [[ -z master ]] + [[ -d apache-github-source-source ]] + [[ ! -d apache-github-source-source/.git ]] + [[ ! -d apache-github-source-source ]] + date '+%Y-%m-%d %T.%3N' 2018-06-25 08:53:50.168 + cd apache-github-source-source + git fetch origin + git reset --hard HEAD HEAD is now at 2277661 HIVE-19922: TestMiniDruidKafkaCliDriver[druidkafkamini_basic] is flaky (Peter Vary, reviewed by Jason Dere) + git clean -f -d + git checkout master Already on 'master' Your branch is up-to-date with 'origin/master'. + git reset --hard origin/master HEAD is now at 2277661 HIVE-19922: TestMiniDruidKafkaCliDriver[druidkafkamini_basic] is flaky (Peter Vary, reviewed by Jason Dere) + git merge --ff-only origin/master Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2018-06-25 08:53:50.782 + rm -rf ../yetus_PreCommit-HIVE-Build-12086 + mkdir ../yetus_PreCommit-HIVE-Build-12086 + git gc + cp -R . ../yetus_PreCommit-HIVE-Build-12086 + mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-12086/yetus + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch error: a/itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcDriver2.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/QueryState.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/parse/ReplicationSemanticAnalyzer.java: does not exist in index error: a/service/src/java/org/apache/hive/service/cli/operation/OperationManager.java: does not exist in index error: a/service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java: does not exist in index error: a/service/src/java/org/apache/hive/service/cli/session/HiveSession.java: does not exist in index error: a/service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java: does not exist in index error: a/service/src/java/org/apache/hive/service/server/KillQueryImpl.java: does not exist in index Going to apply patch with: git apply -p1 + [[ maven == \m\a\v\e\n ]] + rm -rf /data/hiveptest/working/maven/org/apache/hive + mvn -B clean install -DskipTests -T 4 -q -Dmaven.repo.local=/data/hiveptest/working/maven protoc-jar: executing: [/tmp/protoc8452245142444521116.exe, --version] libprotoc 2.5.0 protoc-jar: executing: [/tmp/protoc8452245142444521116.exe, -I/data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/protobuf/org/apache/hadoop/hive/metastore, --java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/target/generated-sources, /data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto] ANTLR Parser Generator Version 3.5.2 Output file /data/hiveptest/working/apache-github-source-source/standalone-metastore/target/generated-sources/org/apache/hadoop/hive/metastore/parser/FilterParser.java does not exist: must build /data/hiveptest/working/apache-github-source-s
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16521875#comment-16521875 ] mahesh kumar behera commented on HIVE-19924: https://github.com/apache/hive/pull/376 > Tag distcp jobs run by Repl Load > > > Key: HIVE-19924 > URL: https://issues.apache.org/jira/browse/HIVE-19924 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-19924.01.patch > > > Add tags in jobconf for distcp related jobs started by replication. This will > allow hive to kill these jobs in case beacon retries, or hs2 dies and beacon > issues a kill command. > * one of the tags should definitely be the query_id that starts the job : > With this flow beacon before retrying the bootstrap load, will issue a kill > command to hs2 with the query id of the previous issued command. hs2 will > then kill an running jobs on yarn tagged with the Query_id. > * To get around the additional failure point as mentioned above. The jobs > can be tagged with an additional unique tag_id provided by Beacon in the WITH > clause in repl load command to be used to tag distcp jobs ). Enhance the kill > api to take the tag as input and kill jobs associated with that tag. Problem > here is how do we validate the association of the tag with a hive query id to > make sure this api is not used to kill jobs run by other components, however > we can provide this capability to only admins and should be ok in that case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
[ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16521871#comment-16521871 ] mahesh kumar behera commented on HIVE-19924: [~sankarh] pease review the patch > Tag distcp jobs run by Repl Load > > > Key: HIVE-19924 > URL: https://issues.apache.org/jira/browse/HIVE-19924 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-19924.01.patch > > > Add tags in jobconf for distcp related jobs started by replication. This will > allow hive to kill these jobs in case beacon retries, or hs2 dies and beacon > issues a kill command. > * one of the tags should definitely be the query_id that starts the job : > With this flow beacon before retrying the bootstrap load, will issue a kill > command to hs2 with the query id of the previous issued command. hs2 will > then kill an running jobs on yarn tagged with the Query_id. > * To get around the additional failure point as mentioned above. The jobs > can be tagged with an additional unique tag_id provided by Beacon in the WITH > clause in repl load command to be used to tag distcp jobs ). Enhance the kill > api to take the tag as input and kill jobs associated with that tag. Problem > here is how do we validate the association of the tag with a hive query id to > make sure this api is not used to kill jobs run by other components, however > we can provide this capability to only admins and should be ok in that case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)