[jira] [Commented] (HIVE-18885) Cascaded alter table + notifications = disaster
[ https://issues.apache.org/jira/browse/HIVE-18885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394393#comment-16394393 ] Vihang Karajgaonkar commented on HIVE-18885: The issue happens due to a deadlock between two concurrent transactions which acquire the same blocking db locks and the java object lock {{NOTIFICATION_TBL_LOCK}} in DbNotificationListener.java. The issue is more likely to happen in systems where DbNotificationListener is configured as a transactional listener because the db locks are not released until the top level transaction completes. Here is an example: 1. Two transactions call alter_partitions on a List of non-overlapping partitions (Typical scenario happening from StatsTask from multiple concurrent queries) 2. In alter_partitions both the transactions are executing the following loop in alter_partitions {code} for (Partition tmpPart : new_parts) { Partition oldTmpPart = null; if (olditr.hasNext()) { oldTmpPart = olditr.next(); } else { throw new InvalidOperationException("failed to alterpartitions"); } if (table == null) { table = getMS().getTable(db_name, tbl_name); } if (!listeners.isEmpty()) { MetaStoreListenerNotifier.notifyEvent(listeners, EventType.ALTER_PARTITION, new AlterPartitionEvent(oldTmpPart, tmpPart, table, true, this)); } } {code} 3. Transaction 1 acquires the dblock on notification_sequence table in notifyEvent method and releases the lock on {{NOTIFICATION_TBL_LOCK}} object. The fact that notification_sequence table is a single row table makes matters worse. 4. Transaction 2 thrift thread is scheduled and it tries to do same thing above. But now it blocks on the dbLock *while holding the lock on {{NOTIFICATION_TBL_LOCK}}*. 5. Transaction 1 thrift thread is scheduled and it blocks on lock {{NOTIFICATION_TBL_LOCK}} which held by Transaction 2 above. Eventually, DB times out one of the transaction with DB lock time-out errors and rollbacks one of them. But in a highly concurrent workload this keeps repeating and HMS slows down so much that it practically becomes unusable. > Cascaded alter table + notifications = disaster > --- > > Key: HIVE-18885 > URL: https://issues.apache.org/jira/browse/HIVE-18885 > Project: Hive > Issue Type: Bug > Components: Hive, Metastore >Affects Versions: 2.3.2 >Reporter: Alexander Kolbasov >Assignee: Vihang Karajgaonkar >Priority: Major > > You can see the problem from looking at the code, but it actually created > severe problems for real life Hive user. > When {{alter table}} has {{cascade}} option it does the following: > {code:java} > msdb.openTransaction() > ... > List parts = msdb.getPartitions(dbname, name, -1); > for (Partition part : parts) { > List oldCols = part.getSd().getCols(); > part.getSd().setCols(newt.getSd().getCols()); > String oldPartName = > Warehouse.makePartName(oldt.getPartitionKeys(), part.getValues()); > updatePartColumnStatsForAlterColumns(msdb, part, oldPartName, > part.getValues(), oldCols, part); > msdb.alterPartition(dbname, name, part.getValues(), part); > } > {code} > So it walks all partitions (and this may be huge list) and does some > non-trivial operations in one single uber-transaction. > When DbNotificationListener is enabled, it adds an event for each partition, > all while > holding a row lock on NOTIFICATION_SEQUENCE table. As a result, while this is > happening no other write DDL can proceed. This can sometimes cause DB lock > timeouts which cause HMS level operation retries which make things even worse. > In one particular case this pretty much made HMS unusable. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18885) Cascaded alter table + notifications = disaster
[ https://issues.apache.org/jira/browse/HIVE-18885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar updated HIVE-18885: --- Affects Version/s: (was: 3.0.0) 2.3.2 > Cascaded alter table + notifications = disaster > --- > > Key: HIVE-18885 > URL: https://issues.apache.org/jira/browse/HIVE-18885 > Project: Hive > Issue Type: Bug > Components: Hive, Metastore >Affects Versions: 2.3.2 >Reporter: Alexander Kolbasov >Assignee: Vihang Karajgaonkar >Priority: Major > > You can see the problem from looking at the code, but it actually created > severe problems for real life Hive user. > When {{alter table}} has {{cascade}} option it does the following: > {code:java} > msdb.openTransaction() > ... > List parts = msdb.getPartitions(dbname, name, -1); > for (Partition part : parts) { > List oldCols = part.getSd().getCols(); > part.getSd().setCols(newt.getSd().getCols()); > String oldPartName = > Warehouse.makePartName(oldt.getPartitionKeys(), part.getValues()); > updatePartColumnStatsForAlterColumns(msdb, part, oldPartName, > part.getValues(), oldCols, part); > msdb.alterPartition(dbname, name, part.getValues(), part); > } > {code} > So it walks all partitions (and this may be huge list) and does some > non-trivial operations in one single uber-transaction. > When DbNotificationListener is enabled, it adds an event for each partition, > all while > holding a row lock on NOTIFICATION_SEQUENCE table. As a result, while this is > happening no other write DDL can proceed. This can sometimes cause DB lock > timeouts which cause HMS level operation retries which make things even worse. > In one particular case this pretty much made HMS unusable. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18885) Cascaded alter table + notifications = disaster
[ https://issues.apache.org/jira/browse/HIVE-18885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar reassigned HIVE-18885: -- Assignee: Vihang Karajgaonkar > Cascaded alter table + notifications = disaster > --- > > Key: HIVE-18885 > URL: https://issues.apache.org/jira/browse/HIVE-18885 > Project: Hive > Issue Type: Bug > Components: Hive, Metastore >Affects Versions: 2.3.2 >Reporter: Alexander Kolbasov >Assignee: Vihang Karajgaonkar >Priority: Major > > You can see the problem from looking at the code, but it actually created > severe problems for real life Hive user. > When {{alter table}} has {{cascade}} option it does the following: > {code:java} > msdb.openTransaction() > ... > List parts = msdb.getPartitions(dbname, name, -1); > for (Partition part : parts) { > List oldCols = part.getSd().getCols(); > part.getSd().setCols(newt.getSd().getCols()); > String oldPartName = > Warehouse.makePartName(oldt.getPartitionKeys(), part.getValues()); > updatePartColumnStatsForAlterColumns(msdb, part, oldPartName, > part.getValues(), oldCols, part); > msdb.alterPartition(dbname, name, part.getValues(), part); > } > {code} > So it walks all partitions (and this may be huge list) and does some > non-trivial operations in one single uber-transaction. > When DbNotificationListener is enabled, it adds an event for each partition, > all while > holding a row lock on NOTIFICATION_SEQUENCE table. As a result, while this is > happening no other write DDL can proceed. This can sometimes cause DB lock > timeouts which cause HMS level operation retries which make things even worse. > In one particular case this pretty much made HMS unusable. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18917) Add spark.home to hive.conf.restricted.list
[ https://issues.apache.org/jira/browse/HIVE-18917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18917: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Pushed to master. Thanks Aihua for the review. > Add spark.home to hive.conf.restricted.list > --- > > Key: HIVE-18917 > URL: https://issues.apache.org/jira/browse/HIVE-18917 > Project: Hive > Issue Type: Task >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18917.1.patch > > > Add spark.home to hive.conf.restricted.list so its not settable by users. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17580) Remove dependency of get_fields_with_environment_context API to serde
[ https://issues.apache.org/jira/browse/HIVE-17580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394361#comment-16394361 ] Vihang Karajgaonkar commented on HIVE-17580: Fixed TestAvroObjectInspectorGenerator. Rest of the failures are unrelated. > Remove dependency of get_fields_with_environment_context API to serde > - > > Key: HIVE-17580 > URL: https://issues.apache.org/jira/browse/HIVE-17580 > Project: Hive > Issue Type: Sub-task > Components: Standalone Metastore >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > Labels: pull-request-available > Attachments: HIVE-17580.003-standalone-metastore.patch, > HIVE-17580.04-standalone-metastore.patch, > HIVE-17580.05-standalone-metastore.patch, > HIVE-17580.06-standalone-metastore.patch, > HIVE-17580.07-standalone-metastore.patch, > HIVE-17580.08-standalone-metastore.patch, > HIVE-17580.09-standalone-metastore.patch, > HIVE-17580.092-standalone-metastore.patch, > HIVE-17580.093-standalone-metastore.patch > > > {{get_fields_with_environment_context}} metastore API uses {{Deserializer}} > class to access the fields metadata for the cases where it is stored along > with the data files (avro tables). The problem is Deserializer classes is > defined in hive-serde module and in order to make metastore independent of > Hive we will have to remove this dependency (atleast we should change it to > runtime dependency instead of compile time). > The other option is investigate if we can use SearchArgument to provide this > functionality. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18910) Migrate to Murmur hash for shuffle and bucketing
[ https://issues.apache.org/jira/browse/HIVE-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394274#comment-16394274 ] Hive QA commented on HIVE-18910: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12913877/HIVE-18910.3.patch {color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 677 failed/errored test(s), 13350 tests executed *Failed tests:* {noformat} TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=95)
[jira] [Commented] (HIVE-18910) Migrate to Murmur hash for shuffle and bucketing
[ https://issues.apache.org/jira/browse/HIVE-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394264#comment-16394264 ] Hive QA commented on HIVE-18910: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 32s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 16s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 13s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 27s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 15s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 8s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 15s{color} | {color:red} streaming in the patch failed. {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 41s{color} | {color:red} ql in the patch failed. {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 13s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 13s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 11s{color} | {color:red} storage-api: The patch generated 3 new + 88 unchanged - 3 fixed = 91 total (was 91) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 13s{color} | {color:red} serde: The patch generated 5 new + 215 unchanged - 2 fixed = 220 total (was 217) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 40s{color} | {color:red} ql: The patch generated 12 new + 604 unchanged - 3 fixed = 616 total (was 607) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 7s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 13s{color} | {color:red} The patch generated 49 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 21m 36s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-9592/dev-support/hive-personality.sh | | git revision | master / 41afa4e | | Default Java | 1.8.0_111 | | mvninstall | http://104.198.109.242/logs//PreCommit-HIVE-Build-9592/yetus/patch-mvninstall-hcatalog_streaming.txt | | mvninstall | http://104.198.109.242/logs//PreCommit-HIVE-Build-9592/yetus/patch-mvninstall-ql.txt | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-9592/yetus/diff-checkstyle-storage-api.txt | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-9592/yetus/diff-checkstyle-serde.txt | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-9592/yetus/diff-checkstyle-ql.txt | | asflicense | http://104.198.109.242/logs//PreCommit-HIVE-Build-9592/yetus/patch-asflicense-problems.txt | | modules | C: storage-api serde hcatalog/streaming ql standalone-metastore U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-9592/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Migrate to Murmur hash for shuffle and bucketing > > > Key: HIVE-18910 > URL: https://issues.apache.org/jira/browse/HIVE-18910 > Project: Hive > Issue Type: Task >Reporter: Deepak Jaiswal >
[jira] [Commented] (HIVE-18831) Differentiate errors that are thrown by Spark tasks
[ https://issues.apache.org/jira/browse/HIVE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394251#comment-16394251 ] Hive QA commented on HIVE-18831: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12913870/HIVE-18831.4.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 22 failed/errored test(s), 13352 tests executed *Failed tests:* {noformat} TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=95)
[jira] [Commented] (HIVE-18831) Differentiate errors that are thrown by Spark tasks
[ https://issues.apache.org/jira/browse/HIVE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394212#comment-16394212 ] Hive QA commented on HIVE-18831: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 34s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m 53s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 18s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 45s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 6s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 7s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 42s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 18s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 18s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 34s{color} | {color:red} ql: The patch generated 1 new + 44 unchanged - 4 fixed = 45 total (was 48) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 4s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 12s{color} | {color:red} The patch generated 49 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 15m 10s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-9591/dev-support/hive-personality.sh | | git revision | master / 41afa4e | | Default Java | 1.8.0_111 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-9591/yetus/diff-checkstyle-ql.txt | | asflicense | http://104.198.109.242/logs//PreCommit-HIVE-Build-9591/yetus/patch-asflicense-problems.txt | | modules | C: itests itests/util ql U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-9591/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Differentiate errors that are thrown by Spark tasks > --- > > Key: HIVE-18831 > URL: https://issues.apache.org/jira/browse/HIVE-18831 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Attachments: HIVE-18831.1.patch, HIVE-18831.2.patch, > HIVE-18831.3.patch, HIVE-18831.4.patch > > > We propagate exceptions from Spark task failures to the client well, but we > don't differentiate between errors from HS2 / RSC vs. errors thrown by > individual tasks. > Main motivation is that when the client sees a propagated Spark exception its > difficult to know what part of the excution threw the exception. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18739) Add support for Export from unpartitioned Acid table
[ https://issues.apache.org/jira/browse/HIVE-18739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394201#comment-16394201 ] Hive QA commented on HIVE-18739: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12913860/HIVE-18739.06.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 115 failed/errored test(s), 13358 tests executed *Failed tests:* {noformat} TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=94)
[jira] [Commented] (HIVE-18739) Add support for Export from unpartitioned Acid table
[ https://issues.apache.org/jira/browse/HIVE-18739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394178#comment-16394178 ] Hive QA commented on HIVE-18739: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 9s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 54s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 41s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 14s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 55s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 55s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 45s{color} | {color:red} ql: The patch generated 107 new + 681 unchanged - 6 fixed = 788 total (was 687) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 13s{color} | {color:red} The patch generated 50 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 13m 48s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-9590/dev-support/hive-personality.sh | | git revision | master / 41afa4e | | Default Java | 1.8.0_111 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-9590/yetus/diff-checkstyle-ql.txt | | asflicense | http://104.198.109.242/logs//PreCommit-HIVE-Build-9590/yetus/patch-asflicense-problems.txt | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-9590/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Add support for Export from unpartitioned Acid table > > > Key: HIVE-18739 > URL: https://issues.apache.org/jira/browse/HIVE-18739 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Major > Attachments: HIVE-18739.01.patch, HIVE-18739.04.patch, > HIVE-18739.04.patch, HIVE-18739.06.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17580) Remove dependency of get_fields_with_environment_context API to serde
[ https://issues.apache.org/jira/browse/HIVE-17580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394170#comment-16394170 ] Hive QA commented on HIVE-17580: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12913856/HIVE-17580.093-standalone-metastore.patch {color:green}SUCCESS:{color} +1 due to 8 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 25 failed/errored test(s), 13214 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries] (batchId=241) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2] (batchId=49) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook] (batchId=13) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=79) org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl] (batchId=175) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb] (batchId=152) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1] (batchId=172) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata] (batchId=167) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid] (batchId=171) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast] (batchId=162) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan] (batchId=164) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] (batchId=161) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_opt_shuffle_serde] (batchId=180) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5] (batchId=122) org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query1] (batchId=251) org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut (batchId=222) org.apache.hadoop.hive.serde2.avro.TestAvroObjectInspectorGenerator.canHandleArrays (batchId=300) org.apache.hadoop.hive.serde2.avro.TestAvroObjectInspectorGenerator.canHandleMapsWithPrimitiveValueTypes (batchId=300) org.apache.hadoop.hive.serde2.avro.TestAvroObjectInspectorGenerator.canHandleRecords (batchId=300) org.apache.hadoop.hive.serde2.avro.TestAvroObjectInspectorGenerator.convertsMapsWithNullablePrimitiveTypes (batchId=300) org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188) org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=235) org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=235) org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=235) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9589/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9589/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9589/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 25 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12913856 - PreCommit-HIVE-Build > Remove dependency of get_fields_with_environment_context API to serde > - > > Key: HIVE-17580 > URL: https://issues.apache.org/jira/browse/HIVE-17580 > Project: Hive > Issue Type: Sub-task > Components: Standalone Metastore >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > Labels: pull-request-available > Attachments: HIVE-17580.003-standalone-metastore.patch, > HIVE-17580.04-standalone-metastore.patch, > HIVE-17580.05-standalone-metastore.patch, > HIVE-17580.06-standalone-metastore.patch, > HIVE-17580.07-standalone-metastore.patch, > HIVE-17580.08-standalone-metastore.patch, > HIVE-17580.09-standalone-metastore.patch, > HIVE-17580.092-standalone-metastore.patch, > HIVE-17580.093-standalone-metastore.patch > > > {{get_fields_with_environment_context}} metastore API uses {{Deserializer}} > class to access the fields metadata for the cases where it is stored along > with the data files (avro tables). The problem is Deserializer classes is > defined in hive-serde module and in order to make metastore independent of > Hive we will have to remove this dependency (atleast we should change it to > runtime dependency instead of compile time). > The other option is
[jira] [Commented] (HIVE-18560) qtests: QTestUtil refactor/split - QOutProcessor
[ https://issues.apache.org/jira/browse/HIVE-18560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394147#comment-16394147 ] Hive QA commented on HIVE-18560: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12913855/HIVE-18560.01.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 110 failed/errored test(s), 13351 tests executed *Failed tests:* {noformat} TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=94)
[jira] [Comment Edited] (HIVE-18862) qfiles: prepare .q files for using datasets
[ https://issues.apache.org/jira/browse/HIVE-18862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387742#comment-16387742 ] Laszlo Bodor edited comment on HIVE-18862 at 3/10/18 12:03 PM: --- source tables from pom.xml: src,src1,srcbucket,srcbucket2,src_json,src_thrift,src_sequencefile,srcpart,alltypesorc,alltypesparquet,src_hbase,cbo_t1,cbo_t2,cbo_t3,src_cbo,part,lineitem was (Author: abstractdog): {code} cat data/scripts/q_test_init.sql | grep "CREATE TABLE" | cut -d "(" -f 1 | cut -d " " -f 3 {code} {code} src src1 src_json src_sequencefile src_thrift srcbucket_tmp srcbucket srcbucket_tmp srcbucket2 srcpart alltypesorc alltypesparquet primitives dest1 dest2 dest3 dest4 dest4_sequencefile part lineitem {code} > qfiles: prepare .q files for using datasets > --- > > Key: HIVE-18862 > URL: https://issues.apache.org/jira/browse/HIVE-18862 > Project: Hive > Issue Type: Sub-task >Reporter: Laszlo Bodor >Assignee: Laszlo Bodor >Priority: Major > Attachments: HIVE-18862.01.patch > > > # Parse .q files for source table usage > # Add needed dataset annotations > # Remove setInitScript("q_test_init.sql"); from CliConfig -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18560) qtests: QTestUtil refactor/split - QOutProcessor
[ https://issues.apache.org/jira/browse/HIVE-18560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394132#comment-16394132 ] Hive QA commented on HIVE-18560: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 42s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 24s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 13s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 17s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 24s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 24s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 24s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 12s{color} | {color:red} itests/util: The patch generated 8 new + 101 unchanged - 7 fixed = 109 total (was 108) {color} | | {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s{color} | {color:red} The patch has 8 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 16s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 12s{color} | {color:red} The patch generated 49 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 9m 19s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-9588/dev-support/hive-personality.sh | | git revision | master / 41afa4e | | Default Java | 1.8.0_111 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-9588/yetus/diff-checkstyle-itests_util.txt | | whitespace | http://104.198.109.242/logs//PreCommit-HIVE-Build-9588/yetus/whitespace-eol.txt | | asflicense | http://104.198.109.242/logs//PreCommit-HIVE-Build-9588/yetus/patch-asflicense-problems.txt | | modules | C: itests/util U: itests/util | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-9588/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > qtests: QTestUtil refactor/split - QOutProcessor > > > Key: HIVE-18560 > URL: https://issues.apache.org/jira/browse/HIVE-18560 > Project: Hive > Issue Type: Task > Components: Tests >Reporter: Laszlo Bodor >Assignee: Laszlo Bodor >Priority: Major > Attachments: HIVE-18560.01.patch > > > QTestUtil needs some refactoring, as it has become a large, hard-to-maintain, > hard-to-test class with a bunch of responsibilities: > * running qtests > * parsing/evaluating results > * masking/processing results > First, I would extract a processing class from it (e.g. QOutProcessor or > something like that), which should be capable of being tested easily, and > contains all the logic what's needed to make the final q.out results from the > original q.out file. > This processing class shall remain untouched in case the testing > infrastructure (QTestUtil) is replaced or changed in some ways. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-14792) AvroSerde reads the remote schema-file at least once per mapper, per table reference.
[ https://issues.apache.org/jira/browse/HIVE-14792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394129#comment-16394129 ] Hive QA commented on HIVE-14792: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12913840/HIVE-14792.5.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 22 failed/errored test(s), 13351 tests executed *Failed tests:* {noformat} TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=95)
[jira] [Commented] (HIVE-14792) AvroSerde reads the remote schema-file at least once per mapper, per table reference.
[ https://issues.apache.org/jira/browse/HIVE-14792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394122#comment-16394122 ] Hive QA commented on HIVE-14792: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 6s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 57s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 35s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 49s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 13s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 55s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 55s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 34s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 11s{color} | {color:red} The patch generated 49 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 13m 28s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-9587/dev-support/hive-personality.sh | | git revision | master / 41afa4e | | Default Java | 1.8.0_111 | | asflicense | http://104.198.109.242/logs//PreCommit-HIVE-Build-9587/yetus/patch-asflicense-problems.txt | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-9587/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > AvroSerde reads the remote schema-file at least once per mapper, per table > reference. > - > > Key: HIVE-14792 > URL: https://issues.apache.org/jira/browse/HIVE-14792 > Project: Hive > Issue Type: Bug >Affects Versions: 1.2.1, 2.1.0 >Reporter: Mithun Radhakrishnan >Assignee: Aihua Xu >Priority: Major > Labels: TODOC2.2, TODOC2.4 > Fix For: 3.0.0, 2.4.0, 2.2.1 > > Attachments: HIVE-14792.1.patch, HIVE-14792.3.patch, > HIVE-14792.4.patch, HIVE-14792.5.patch > > > Avro tables that use "external" schema files stored on HDFS can cause > excessive calls to {{FileSystem::open()}}, especially for queries that spawn > large numbers of mappers. > This is because of the following code in {{AvroSerDe::initialize()}}: > {code:title=AvroSerDe.java|borderStyle=solid} > public void initialize(Configuration configuration, Properties properties) > throws SerDeException { > // ... > if (hasExternalSchema(properties) > || columnNameProperty == null || columnNameProperty.isEmpty() > || columnTypeProperty == null || columnTypeProperty.isEmpty()) { > schema = determineSchemaOrReturnErrorSchema(configuration, properties); > } else { > // Get column names and sort order > columnNames = Arrays.asList(columnNameProperty.split(",")); > columnTypes = > TypeInfoUtils.getTypeInfosFromTypeString(columnTypeProperty); > schema = getSchemaFromCols(properties, columnNames, columnTypes, > columnCommentProperty); > > properties.setProperty(AvroSerdeUtils.AvroTableProperties.SCHEMA_LITERAL.getPropName(), > schema.toString());
[jira] [Commented] (HIVE-18846) Query results cache: Allow queries to refer to the pending results of a query that has not finished yet
[ https://issues.apache.org/jira/browse/HIVE-18846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394113#comment-16394113 ] Hive QA commented on HIVE-18846: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12913837/HIVE-18846.2.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 25 failed/errored test(s), 12955 tests executed *Failed tests:* {noformat} TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=94)
[jira] [Commented] (HIVE-18846) Query results cache: Allow queries to refer to the pending results of a query that has not finished yet
[ https://issues.apache.org/jira/browse/HIVE-18846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394104#comment-16394104 ] Hive QA commented on HIVE-18846: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 26s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 22s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 10s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 53s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 0s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 7s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 26s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 14s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 14s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 39s{color} | {color:red} ql: The patch generated 2 new + 574 unchanged - 0 fixed = 576 total (was 574) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 1s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 12s{color} | {color:red} The patch generated 49 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 15m 5s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-9586/dev-support/hive-personality.sh | | git revision | master / 41afa4e | | Default Java | 1.8.0_111 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-9586/yetus/diff-checkstyle-ql.txt | | asflicense | http://104.198.109.242/logs//PreCommit-HIVE-Build-9586/yetus/patch-asflicense-problems.txt | | modules | C: common ql U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-9586/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Query results cache: Allow queries to refer to the pending results of a query > that has not finished yet > --- > > Key: HIVE-18846 > URL: https://issues.apache.org/jira/browse/HIVE-18846 > Project: Hive > Issue Type: Sub-task > Components: Query Planning >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > Attachments: HIVE-18846.1.patch, HIVE-18846.2.patch > > > Currently, a query's results can only be looked up in the cache if the query > has completely finished execution. Allow new queries to use the results cache > to find queries that are still executing so they can re-use the results when > the query has finished. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18264) CachedStore: Store cached partitions/col stats within the table cache and make prewarm non-blocking
[ https://issues.apache.org/jira/browse/HIVE-18264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394099#comment-16394099 ] Hive QA commented on HIVE-18264: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12913826/HIVE-18264.6.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9585/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9585/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9585/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N' 2018-03-10 08:21:42.803 + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]] + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'MAVEN_OPTS=-Xmx1g ' + MAVEN_OPTS='-Xmx1g ' + cd /data/hiveptest/working/ + tee /data/hiveptest/logs/PreCommit-HIVE-Build-9585/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ git = \s\v\n ]] + [[ git = \g\i\t ]] + [[ -z master ]] + [[ -d apache-github-source-source ]] + [[ ! -d apache-github-source-source/.git ]] + [[ ! -d apache-github-source-source ]] + date '+%Y-%m-%d %T.%3N' 2018-03-10 08:21:42.806 + cd apache-github-source-source + git fetch origin + git reset --hard HEAD HEAD is now at 41afa4e HIVE-18436: Upgrade to Spark 2.3.0 (Sahil Takiar, reviewed by Rui Li) + git clean -f -d + git checkout master Already on 'master' Your branch is up-to-date with 'origin/master'. + git reset --hard origin/master HEAD is now at 41afa4e HIVE-18436: Upgrade to Spark 2.3.0 (Sahil Takiar, reviewed by Rui Li) + git merge --ff-only origin/master Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2018-03-10 08:21:46.190 + rm -rf ../yetus_PreCommit-HIVE-Build-9585 + mkdir ../yetus_PreCommit-HIVE-Build-9585 + git gc + cp -R . ../yetus_PreCommit-HIVE-Build-9585 + mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-9585/yetus + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch error: a/itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java: does not exist in index error: a/service/src/java/org/apache/hive/service/server/HiveServer2.java: does not exist in index error: a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java: does not exist in index error: a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java: does not exist in index error: a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java: does not exist in index error: a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java: does not exist in index error: a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java: does not exist in index error: a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java: does not exist in index error: a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreUtils.java: does not exist in index error: a/standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java: does not exist in index error: a/standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java: does not exist in index error: a/standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java: does not exist in index error: a/standalone-metastore/src/test/resources/log4j2.properties: does not exist in index error: patch failed: standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java:71 Falling back to three-way merge... Applied patch to 'standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java' cleanly. error: patch failed: standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java:9964 Falling back to three-way merge... Applied patch to
[jira] [Commented] (HIVE-18863) trunc() calls itself trunk() in an error message
[ https://issues.apache.org/jira/browse/HIVE-18863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394098#comment-16394098 ] Hive QA commented on HIVE-18863: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12913822/HIVE-18863.1.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 21 failed/errored test(s), 13350 tests executed *Failed tests:* {noformat} TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=95)