[jira] [Updated] (HIVE-21962) Replacing ArrayList params with List in and around PlanUtils and MapWork
[ https://issues.apache.org/jira/browse/HIVE-21962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21962: --- Attachment: HIVE-21962.2.patch > Replacing ArrayList params with List in and around PlanUtils and MapWork > > > Key: HIVE-21962 > URL: https://issues.apache.org/jira/browse/HIVE-21962 > Project: Hive > Issue Type: Improvement >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Minor > Attachments: HIVE-21962.1.patch, HIVE-21962.1.patch, > HIVE-21962.2.patch > > > Using the implementing class is usually a bad practice. OO suggest to use the > least restrictive interface instead. ArrayList is used in many-many methods > as a parameter - this is just a tiny part of this work. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21861) ClassCastException during CTAS over external table using KafkaStorageHandler
[
https://issues.apache.org/jira/browse/HIVE-21861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881746#comment-16881746
]
Jesus Camacho Rodriguez commented on HIVE-21861:
{code}
objectInspector =
ObjectInspectorFactory.getStandardStructObjectInspector(columnNames,
inspectors);
{code}
That creates a {{StandardStructObjectInspector}}. That does not seem correct,
since we are creating lazy objects after deserialization step. Basically, the
object inspector should match the objects that you are creating when
deserializing the writable objects.
> ClassCastException during CTAS over external table using KafkaStorageHandler
>
>
> Key: HIVE-21861
> URL: https://issues.apache.org/jira/browse/HIVE-21861
> Project: Hive
> Issue Type: Bug
> Components: kafka integration
>Affects Versions: 4.0.0
>Reporter: Justin Leet
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-21861.patch
>
>
> To reproduce, create a table similar to the following:
> {code}
> CREATE EXTERNAL TABLE
> (raw_value STRING)
> ROW FORMAT DELIMITED
> LINES TERMINATED BY '\n'
> STORED BY 'org.apache.hadoop.hive.kafka.KafkaStorageHandler'
> TBLPROPERTIES(
> "kafka.topic"="",
> "kafka.bootstrap.servers"="",
> "kafka.consumer.security.protocol"="PLAINTEXT",
> "kafka.serde.class"="org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe");
> {code}
> Note the SerDe isn't the default SerDe. Additionally, this error occurs when
> vectorization is enabled.
> Basic queries work fine:
> {code}
> SELECT * FROM LIMIT 1;
> {code}
> Doing a CTAS to bring it into a managed table fails:
> {code}
> CREATE TABLE AS
> SELECT * FROM ;
> {code}
> The exception is:
> {code}
> Caused by: java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.lazy.LazyString cannot be cast to
> org.apache.hadoop.io.TextCaused by: java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.lazy.LazyString cannot be cast to
> org.apache.hadoop.io.Text at
> org.apache.hadoop.hive.ql.exec.vector.VectorAssignRow.assignRowColumn(VectorAssignRow.java:471)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorAssignRow.assignRowColumn(VectorAssignRow.java:350)
> at
> org.apache.hadoop.hive.kafka.VectorizedKafkaRecordReader.readNextBatch(VectorizedKafkaRecordReader.java:159)
> at
> org.apache.hadoop.hive.kafka.VectorizedKafkaRecordReader.next(VectorizedKafkaRecordReader.java:113)
> at
> org.apache.hadoop.hive.kafka.VectorizedKafkaRecordReader.next(VectorizedKafkaRecordReader.java:47)
> at
> org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:360)
> ... 24 more
> {code}
> A workaround to this is to disable vectorization via:
> {code}
> set hive.vectorized.execution.enabled = false;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-21861) ClassCastException during CTAS over external table using KafkaStorageHandler
[
https://issues.apache.org/jira/browse/HIVE-21861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881737#comment-16881737
]
Rajkumar Singh edited comment on HIVE-21861 at 7/10/19 5:24 AM:
I noticed that for column raw_value LazySimpleSerDe create primitive object
Inspector (LazyStringObjectInspector) since column type is a string.
https://github.com/apache/hive/blob/8a606abdec0f92d60653d892b2f92ff729f1c020/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java#L435
{code:java}
objectInspector =
ObjectInspectorFactory.getStandardStructObjectInspector(columnNames,
inspectors);
{code}
objectInspector has the same LazyStringObjectInspector with other inspectors
for other kafka meta columns.
so it's seem objectInspector are expected?
was (Author: rajkumar singh):
I noticed that for column raw_value LazySimpleSerDe create primitive object
Inspector (LazyStringObjectInspector) since column type is a string.
https://github.com/apache/hive/blob/8a606abdec0f92d60653d892b2f92ff729f1c020/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java#L435
{code:java}
objectInspector =
ObjectInspectorFactory.getStandardStructObjectInspector(columnNames,
inspectors);
{code}
objectInspector has the same LazyStringObjectInspector with other inspectors
for other kafka meta columns.
> ClassCastException during CTAS over external table using KafkaStorageHandler
>
>
> Key: HIVE-21861
> URL: https://issues.apache.org/jira/browse/HIVE-21861
> Project: Hive
> Issue Type: Bug
> Components: kafka integration
>Affects Versions: 4.0.0
>Reporter: Justin Leet
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-21861.patch
>
>
> To reproduce, create a table similar to the following:
> {code}
> CREATE EXTERNAL TABLE
> (raw_value STRING)
> ROW FORMAT DELIMITED
> LINES TERMINATED BY '\n'
> STORED BY 'org.apache.hadoop.hive.kafka.KafkaStorageHandler'
> TBLPROPERTIES(
> "kafka.topic"="",
> "kafka.bootstrap.servers"="",
> "kafka.consumer.security.protocol"="PLAINTEXT",
> "kafka.serde.class"="org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe");
> {code}
> Note the SerDe isn't the default SerDe. Additionally, this error occurs when
> vectorization is enabled.
> Basic queries work fine:
> {code}
> SELECT * FROM LIMIT 1;
> {code}
> Doing a CTAS to bring it into a managed table fails:
> {code}
> CREATE TABLE AS
> SELECT * FROM ;
> {code}
> The exception is:
> {code}
> Caused by: java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.lazy.LazyString cannot be cast to
> org.apache.hadoop.io.TextCaused by: java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.lazy.LazyString cannot be cast to
> org.apache.hadoop.io.Text at
> org.apache.hadoop.hive.ql.exec.vector.VectorAssignRow.assignRowColumn(VectorAssignRow.java:471)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorAssignRow.assignRowColumn(VectorAssignRow.java:350)
> at
> org.apache.hadoop.hive.kafka.VectorizedKafkaRecordReader.readNextBatch(VectorizedKafkaRecordReader.java:159)
> at
> org.apache.hadoop.hive.kafka.VectorizedKafkaRecordReader.next(VectorizedKafkaRecordReader.java:113)
> at
> org.apache.hadoop.hive.kafka.VectorizedKafkaRecordReader.next(VectorizedKafkaRecordReader.java:47)
> at
> org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:360)
> ... 24 more
> {code}
> A workaround to this is to disable vectorization via:
> {code}
> set hive.vectorized.execution.enabled = false;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21912) Implement BlacklistingLlapMetricsListener
[
https://issues.apache.org/jira/browse/HIVE-21912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881740#comment-16881740
]
Hive QA commented on HIVE-21912:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
42s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
33s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
29s{color} | {color:blue} llap-common in master has 90 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
23s{color} | {color:blue} llap-client in master has 26 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
27s{color} | {color:blue} llap-tez in master has 17 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
43s{color} | {color:blue} llap-server in master has 83 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
20s{color} | {color:red} llap-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
22s{color} | {color:red} llap-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 22s{color}
| {color:red} llap-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
9s{color} | {color:red} llap-common: The patch generated 7 new + 0 unchanged -
0 fixed = 7 total (was 0) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
10s{color} | {color:red} llap-client: The patch generated 1 new + 16 unchanged
- 2 fixed = 17 total (was 18) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
13s{color} | {color:red} llap-server: The patch generated 1 new + 0 unchanged -
7 fixed = 1 total (was 7) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
34s{color} | {color:red} llap-tez generated 1 new + 17 unchanged - 0 fixed = 18
total (was 17) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
17s{color} | {color:red} llap-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
10s{color} | {color:red} llap-tez generated 1 new + 1 unchanged - 0 fixed = 2
total (was 1) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 10s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:llap-tez |
| |
org.apache.hadoop.hive.llap.tezplugins.metrics.BlacklistingLlapMetricsListener.newClusterMetrics(Map)
makes inefficient use of keySet iterator instead of entrySet iterator At
BlacklistingLlapMetricsListener.java:keySet iterator instead of entrySet
iterator At BlacklistingLlapMetricsListener.java:[line 112] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Commented] (HIVE-21861) ClassCastException during CTAS over external table using KafkaStorageHandler
[
https://issues.apache.org/jira/browse/HIVE-21861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881737#comment-16881737
]
Rajkumar Singh commented on HIVE-21861:
---
I noticed that for column raw_value LazySimpleSerDe create primitive object
Inspector (LazyStringObjectInspector) since column type is a string.
https://github.com/apache/hive/blob/8a606abdec0f92d60653d892b2f92ff729f1c020/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java#L435
{code:java}
objectInspector =
ObjectInspectorFactory.getStandardStructObjectInspector(columnNames,
inspectors);
{code}
objectInspector has the same LazyStringObjectInspector with other inspectors
for other kafka meta columns.
> ClassCastException during CTAS over external table using KafkaStorageHandler
>
>
> Key: HIVE-21861
> URL: https://issues.apache.org/jira/browse/HIVE-21861
> Project: Hive
> Issue Type: Bug
> Components: kafka integration
>Affects Versions: 4.0.0
>Reporter: Justin Leet
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-21861.patch
>
>
> To reproduce, create a table similar to the following:
> {code}
> CREATE EXTERNAL TABLE
> (raw_value STRING)
> ROW FORMAT DELIMITED
> LINES TERMINATED BY '\n'
> STORED BY 'org.apache.hadoop.hive.kafka.KafkaStorageHandler'
> TBLPROPERTIES(
> "kafka.topic"="",
> "kafka.bootstrap.servers"="",
> "kafka.consumer.security.protocol"="PLAINTEXT",
> "kafka.serde.class"="org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe");
> {code}
> Note the SerDe isn't the default SerDe. Additionally, this error occurs when
> vectorization is enabled.
> Basic queries work fine:
> {code}
> SELECT * FROM LIMIT 1;
> {code}
> Doing a CTAS to bring it into a managed table fails:
> {code}
> CREATE TABLE AS
> SELECT * FROM ;
> {code}
> The exception is:
> {code}
> Caused by: java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.lazy.LazyString cannot be cast to
> org.apache.hadoop.io.TextCaused by: java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.lazy.LazyString cannot be cast to
> org.apache.hadoop.io.Text at
> org.apache.hadoop.hive.ql.exec.vector.VectorAssignRow.assignRowColumn(VectorAssignRow.java:471)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorAssignRow.assignRowColumn(VectorAssignRow.java:350)
> at
> org.apache.hadoop.hive.kafka.VectorizedKafkaRecordReader.readNextBatch(VectorizedKafkaRecordReader.java:159)
> at
> org.apache.hadoop.hive.kafka.VectorizedKafkaRecordReader.next(VectorizedKafkaRecordReader.java:113)
> at
> org.apache.hadoop.hive.kafka.VectorizedKafkaRecordReader.next(VectorizedKafkaRecordReader.java:47)
> at
> org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:360)
> ... 24 more
> {code}
> A workaround to this is to disable vectorization via:
> {code}
> set hive.vectorized.execution.enabled = false;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21637) Synchronized metastore cache
[ https://issues.apache.org/jira/browse/HIVE-21637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-21637: -- Attachment: HIVE-21637.23.patch > Synchronized metastore cache > > > Key: HIVE-21637 > URL: https://issues.apache.org/jira/browse/HIVE-21637 > Project: Hive > Issue Type: New Feature >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-21637-1.patch, HIVE-21637.10.patch, > HIVE-21637.11.patch, HIVE-21637.12.patch, HIVE-21637.13.patch, > HIVE-21637.14.patch, HIVE-21637.15.patch, HIVE-21637.16.patch, > HIVE-21637.17.patch, HIVE-21637.18.patch, HIVE-21637.19.patch, > HIVE-21637.19.patch, HIVE-21637.2.patch, HIVE-21637.20.patch, > HIVE-21637.21.patch, HIVE-21637.22.patch, HIVE-21637.23.patch, > HIVE-21637.3.patch, HIVE-21637.4.patch, HIVE-21637.5.patch, > HIVE-21637.6.patch, HIVE-21637.7.patch, HIVE-21637.8.patch, HIVE-21637.9.patch > > > Currently, HMS has a cache implemented by CachedStore. The cache is > asynchronized and in HMS HA setting, we can only get eventual consistency. In > this Jira, we try to make it synchronized. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21965) Implement parallel processing in HiveStrictManagedMigration
[
https://issues.apache.org/jira/browse/HIVE-21965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881727#comment-16881727
]
Hive QA commented on HIVE-21965:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12974051/HIVE-21965.2.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 16601 tests
executed
*Failed tests:*
{noformat}
TestDataSourceProviderFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=232)
TestObjectStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=232)
org.apache.hadoop.hive.metastore.TestMetastoreHousekeepingLeader.testHouseKeepingThreadExistence
(batchId=240)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17936/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17936/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17936/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12974051 - PreCommit-HIVE-Build
> Implement parallel processing in HiveStrictManagedMigration
> ---
>
> Key: HIVE-21965
> URL: https://issues.apache.org/jira/browse/HIVE-21965
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.0
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21965.1.patch, HIVE-21965.2.patch
>
>
> This process, kicked off from Ambari can take many days for systems with
> 1000's of tables. The process needs to support parallel execution as it
> iterates through the Databases and Tables.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21164) ACID: explore how we can avoid a move step during inserts/compaction
[ https://issues.apache.org/jira/browse/HIVE-21164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-21164: Attachment: HIVE-21164.5.patch > ACID: explore how we can avoid a move step during inserts/compaction > > > Key: HIVE-21164 > URL: https://issues.apache.org/jira/browse/HIVE-21164 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.1.1 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta >Priority: Major > Attachments: HIVE-21164.1.patch, HIVE-21164.2.patch, > HIVE-21164.3.patch, HIVE-21164.4.patch, HIVE-21164.5.patch > > > Currently, we write compacted data to a temporary location and then move the > files to a final location, which is an expensive operation on some cloud file > systems. Since HIVE-20823 is already in, it can control the visibility of > compacted data for the readers. Therefore, we can perhaps avoid writing data > to a temporary location and directly write compacted data to the intended > final path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21978) mr data skew while druid handler indexing data
[
https://issues.apache.org/jira/browse/HIVE-21978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881718#comment-16881718
]
tangyun commented on HIVE-21978:
{{modify partitioning key from basing on floor(1/rand) %
targetShardsPerGranularity to on floor(targetShardsPerGranularity * rand) %
targetShardsPerGranularity ,which is Linear transformation}}
> mr data skew while druid handler indexing data
> --
>
> Key: HIVE-21978
> URL: https://issues.apache.org/jira/browse/HIVE-21978
> Project: Hive
> Issue Type: Improvement
> Components: Druid integration
>Affects Versions: 2.3.2, 3.1.1
>Reporter: tangyun
>Assignee: tangyun
>Priority: Major
> Attachments: optimized_algorithm.png, origi n_algorithm.png
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21978) mr data skew while druid handler indexing data
[ https://issues.apache.org/jira/browse/HIVE-21978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tangyun updated HIVE-21978: --- Attachment: origi n_algorithm.png optimized_algorithm.png > mr data skew while druid handler indexing data > -- > > Key: HIVE-21978 > URL: https://issues.apache.org/jira/browse/HIVE-21978 > Project: Hive > Issue Type: Improvement > Components: Druid integration >Affects Versions: 2.3.2, 3.1.1 >Reporter: tangyun >Assignee: tangyun >Priority: Major > Attachments: optimized_algorithm.png, origi n_algorithm.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21978) mr data skew while druid handler indexing data
[ https://issues.apache.org/jira/browse/HIVE-21978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tangyun updated HIVE-21978: --- Attachment: (was: WechatIMG12.jpg) > mr data skew while druid handler indexing data > -- > > Key: HIVE-21978 > URL: https://issues.apache.org/jira/browse/HIVE-21978 > Project: Hive > Issue Type: Improvement > Components: Druid integration >Affects Versions: 2.3.2, 3.1.1 >Reporter: tangyun >Assignee: tangyun >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21978) mr data skew while druid handler indexing data
[ https://issues.apache.org/jira/browse/HIVE-21978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tangyun updated HIVE-21978: --- Attachment: WechatIMG12.jpg > mr data skew while druid handler indexing data > -- > > Key: HIVE-21978 > URL: https://issues.apache.org/jira/browse/HIVE-21978 > Project: Hive > Issue Type: Improvement > Components: Druid integration >Affects Versions: 2.3.2, 3.1.1 >Reporter: tangyun >Assignee: tangyun >Priority: Major > Attachments: WechatIMG12.jpg > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21978) mr data skew while druid handler indexing data
[ https://issues.apache.org/jira/browse/HIVE-21978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tangyun updated HIVE-21978: --- Attachment: (was: WechatIMG12.tiff) > mr data skew while druid handler indexing data > -- > > Key: HIVE-21978 > URL: https://issues.apache.org/jira/browse/HIVE-21978 > Project: Hive > Issue Type: Improvement > Components: Druid integration >Affects Versions: 2.3.2, 3.1.1 >Reporter: tangyun >Assignee: tangyun >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21978) mr data skew while druid handler indexing data
[ https://issues.apache.org/jira/browse/HIVE-21978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tangyun updated HIVE-21978: --- Attachment: WechatIMG12.tiff > mr data skew while druid handler indexing data > -- > > Key: HIVE-21978 > URL: https://issues.apache.org/jira/browse/HIVE-21978 > Project: Hive > Issue Type: Improvement > Components: Druid integration >Affects Versions: 2.3.2, 3.1.1 >Reporter: tangyun >Assignee: tangyun >Priority: Major > Attachments: WechatIMG12.tiff > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21965) Implement parallel processing in HiveStrictManagedMigration
[
https://issues.apache.org/jira/browse/HIVE-21965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881710#comment-16881710
]
Hive QA commented on HIVE-21965:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
4s{color} | {color:blue} ql in master has 2252 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 14 new + 62 unchanged - 17
fixed = 76 total (was 79) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
16s{color} | {color:red} ql generated 1 new + 2252 unchanged - 0 fixed = 2253
total (was 2252) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 36s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Exception is caught when Exception is not thrown in
org.apache.hadoop.hive.ql.util.HiveStrictManagedMigration.processDatabase(String,
ForkJoinPool) At HiveStrictManagedMigration.java:is not thrown in
org.apache.hadoop.hive.ql.util.HiveStrictManagedMigration.processDatabase(String,
ForkJoinPool) At HiveStrictManagedMigration.java:[line 610] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17936/dev-support/hive-personality.sh
|
| git revision | master / 33d2995 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17936/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17936/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17936/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Implement parallel processing in HiveStrictManagedMigration
> ---
>
> Key: HIVE-21965
> URL: https://issues.apache.org/jira/browse/HIVE-21965
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.0
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21965.1.patch, HIVE-21965.2.patch
>
>
> This process, kicked off from Ambari can take many days for systems with
> 1000's of tables. The process needs to support parallel execution as it
> iterates through the Databases and Tables.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally
[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881708#comment-16881708 ] Eugene Koifman commented on HIVE-21225: --- Full Acid uses stmtId for Insert ... as select union all which it does somewhere in MoveTask I think MM tables retain these subdirs since its MoveTask doesn't rename any files > ACID: getAcidState() should cache a recursive dir listing locally > - > > Key: HIVE-21225 > URL: https://issues.apache.org/jira/browse/HIVE-21225 > Project: Hive > Issue Type: Improvement > Components: Transactions >Reporter: Gopal V >Assignee: Vaibhav Gumashta >Priority: Major > Attachments: HIVE-21225.1.patch, HIVE-21225.2.patch, > HIVE-21225.3.patch, HIVE-21225.4.patch, HIVE-21225.4.patch, > HIVE-21225.5.patch, HIVE-21225.6.patch, HIVE-21225.7.patch, > HIVE-21225.7.patch, HIVE-21225.8.patch, HIVE-21225.9.patch, async-pid-44-2.svg > > > Currently getAcidState() makes 3 calls into the FS api which could be > answered by making a single recursive listDir call and reusing the same data > to check for isRawFormat() and isValidBase(). > All delta operations for a single partition can go against a single listed > directory snapshot instead of interacting with the NameNode or ObjectStore > within the inner loop. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally
[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881703#comment-16881703 ] Gopal V commented on HIVE-21225: bq. union_mm/delta_001_001_0001/HIVE_UNION_SUBDIR_1 Are you seeing stuff like that? That looks like a bug to me. The output dir should have delta_001_001_0001 & delta_001_001_0002, but no further sub-dirs in it. > ACID: getAcidState() should cache a recursive dir listing locally > - > > Key: HIVE-21225 > URL: https://issues.apache.org/jira/browse/HIVE-21225 > Project: Hive > Issue Type: Improvement > Components: Transactions >Reporter: Gopal V >Assignee: Vaibhav Gumashta >Priority: Major > Attachments: HIVE-21225.1.patch, HIVE-21225.2.patch, > HIVE-21225.3.patch, HIVE-21225.4.patch, HIVE-21225.4.patch, > HIVE-21225.5.patch, HIVE-21225.6.patch, HIVE-21225.7.patch, > HIVE-21225.7.patch, HIVE-21225.8.patch, HIVE-21225.9.patch, async-pid-44-2.svg > > > Currently getAcidState() makes 3 calls into the FS api which could be > answered by making a single recursive listDir call and reusing the same data > to check for isRawFormat() and isValidBase(). > All delta operations for a single partition can go against a single listed > directory snapshot instead of interacting with the NameNode or ObjectStore > within the inner loop. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally
[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881703#comment-16881703 ] Gopal V edited comment on HIVE-21225 at 7/10/19 3:45 AM: - bq. union_mm/delta_001_001_0001/HIVE_UNION_SUBDIR_1 Are you seeing stuff like that? That looks like a bug to me. The output dir should have delta_001_001_0001 & delta_001_001_0002, but no further sub-dirs in it (i.e two dirs for a single write-id range, but different statement ids for each) was (Author: gopalv): bq. union_mm/delta_001_001_0001/HIVE_UNION_SUBDIR_1 Are you seeing stuff like that? That looks like a bug to me. The output dir should have delta_001_001_0001 & delta_001_001_0002, but no further sub-dirs in it. > ACID: getAcidState() should cache a recursive dir listing locally > - > > Key: HIVE-21225 > URL: https://issues.apache.org/jira/browse/HIVE-21225 > Project: Hive > Issue Type: Improvement > Components: Transactions >Reporter: Gopal V >Assignee: Vaibhav Gumashta >Priority: Major > Attachments: HIVE-21225.1.patch, HIVE-21225.2.patch, > HIVE-21225.3.patch, HIVE-21225.4.patch, HIVE-21225.4.patch, > HIVE-21225.5.patch, HIVE-21225.6.patch, HIVE-21225.7.patch, > HIVE-21225.7.patch, HIVE-21225.8.patch, HIVE-21225.9.patch, async-pid-44-2.svg > > > Currently getAcidState() makes 3 calls into the FS api which could be > answered by making a single recursive listDir call and reusing the same data > to check for isRawFormat() and isValidBase(). > All delta operations for a single partition can go against a single listed > directory snapshot instead of interacting with the NameNode or ObjectStore > within the inner loop. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21958) The list of table expression in the inclusion and exclusion list should be separated by '|' instead of comma.
[
https://issues.apache.org/jira/browse/HIVE-21958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mahesh kumar behera updated HIVE-21958:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
[^HIVE-21958.05.patch] committed to master. Thanks [~sankarh] for review.
> The list of table expression in the inclusion and exclusion list should be
> separated by '|' instead of comma.
> -
>
> Key: HIVE-21958
> URL: https://issues.apache.org/jira/browse/HIVE-21958
> Project: Hive
> Issue Type: Sub-task
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21958.01.patch, HIVE-21958.02.patch,
> HIVE-21958.03.patch, HIVE-21958.04.patch, HIVE-21958.05.patch
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> Java regex expression does not support comma. If user wants multiple
> expression to be present in the include or exclude list, then the expressions
> can be provided separated by pipe ('|') character. The policy will look
> something like db_name.'(t1*)|(t3)'.'t100'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21962) Replacing ArrayList params with List in and around PlanUtils and MapWork
[
https://issues.apache.org/jira/browse/HIVE-21962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881696#comment-16881696
]
Hive QA commented on HIVE-21962:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12974045/HIVE-21962.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 16632 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.checkExpectedLocks
(batchId=340)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testMetadataOperationLocks
(batchId=340)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17935/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17935/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17935/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12974045 - PreCommit-HIVE-Build
> Replacing ArrayList params with List in and around PlanUtils and MapWork
>
>
> Key: HIVE-21962
> URL: https://issues.apache.org/jira/browse/HIVE-21962
> Project: Hive
> Issue Type: Improvement
>Reporter: Ivan Suller
>Assignee: Ivan Suller
>Priority: Minor
> Attachments: HIVE-21962.1.patch, HIVE-21962.1.patch
>
>
> Using the implementing class is usually a bad practice. OO suggest to use the
> least restrictive interface instead. ArrayList is used in many-many methods
> as a parameter - this is just a tiny part of this work.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21962) Replacing ArrayList params with List in and around PlanUtils and MapWork
[
https://issues.apache.org/jira/browse/HIVE-21962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881676#comment-16881676
]
Hive QA commented on HIVE-21962:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
59s{color} | {color:blue} ql in master has 2252 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
44s{color} | {color:red} ql: The patch generated 1 new + 583 unchanged - 1
fixed = 584 total (was 584) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17935/dev-support/hive-personality.sh
|
| git revision | master / 33d2995 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17935/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17935/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Replacing ArrayList params with List in and around PlanUtils and MapWork
>
>
> Key: HIVE-21962
> URL: https://issues.apache.org/jira/browse/HIVE-21962
> Project: Hive
> Issue Type: Improvement
>Reporter: Ivan Suller
>Assignee: Ivan Suller
>Priority: Minor
> Attachments: HIVE-21962.1.patch, HIVE-21962.1.patch
>
>
> Using the implementing class is usually a bad practice. OO suggest to use the
> least restrictive interface instead. ArrayList is used in many-many methods
> as a parameter - this is just a tiny part of this work.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-21978) mr data skew while druid handler indexing data
[ https://issues.apache.org/jira/browse/HIVE-21978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tangyun reassigned HIVE-21978: -- > mr data skew while druid handler indexing data > -- > > Key: HIVE-21978 > URL: https://issues.apache.org/jira/browse/HIVE-21978 > Project: Hive > Issue Type: Improvement > Components: Druid integration >Affects Versions: 3.1.1, 2.3.2 >Reporter: tangyun >Assignee: tangyun >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21968) Remove index related codes
[ https://issues.apache.org/jira/browse/HIVE-21968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21968: -- Attachment: HIVE-21968.02.patch > Remove index related codes > -- > > Key: HIVE-21968 > URL: https://issues.apache.org/jira/browse/HIVE-21968 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-21968.01.patch, HIVE-21968.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Hive doesn't support indexes since 3.0.0, still some index related tests were > left behind, and some code to disable them. Also some index related code is > still in the codebase. They should be removed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21968) Remove index related codes
[ https://issues.apache.org/jira/browse/HIVE-21968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21968: -- Attachment: (was: HIVE-21968.02.patch) > Remove index related codes > -- > > Key: HIVE-21968 > URL: https://issues.apache.org/jira/browse/HIVE-21968 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-21968.01.patch, HIVE-21968.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Hive doesn't support indexes since 3.0.0, still some index related tests were > left behind, and some code to disable them. Also some index related code is > still in the codebase. They should be removed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21968) Remove index related codes
[
https://issues.apache.org/jira/browse/HIVE-21968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881657#comment-16881657
]
Hive QA commented on HIVE-21968:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12974040/HIVE-21968.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 16632 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.llap.cache.TestBuddyAllocator.testMTT[2] (batchId=350)
org.apache.hadoop.hive.ql.exec.tez.TestCustomPartitionVertex.testGetBytePayload
(batchId=331)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17934/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17934/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17934/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12974040 - PreCommit-HIVE-Build
> Remove index related codes
> --
>
> Key: HIVE-21968
> URL: https://issues.apache.org/jira/browse/HIVE-21968
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.1
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21968.01.patch, HIVE-21968.02.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hive doesn't support indexes since 3.0.0, still some index related tests were
> left behind, and some code to disable them. Also some index related code is
> still in the codebase. They should be removed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21968) Remove index related codes
[
https://issues.apache.org/jira/browse/HIVE-21968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881642#comment-16881642
]
Hive QA commented on HIVE-21968:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
51s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
7s{color} | {color:blue} ql in master has 2252 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
22s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
40s{color} | {color:red} ql: The patch generated 3 new + 20 unchanged - 320
fixed = 23 total (was 340) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} itests/hive-unit: The patch generated 0 new + 44
unchanged - 7 fixed = 44 total (was 51) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 32m 18s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17934/dev-support/hive-personality.sh
|
| git revision | master / 33d2995 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17934/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql itests itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17934/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove index related codes
> --
>
> Key: HIVE-21968
> URL: https://issues.apache.org/jira/browse/HIVE-21968
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.1
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21968.01.patch, HIVE-21968.02.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hive doesn't support indexes since 3.0.0, still some index related tests were
> left behind, and some code to disable them. Also some index related code is
> still in the codebase. They should be removed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21976) Offset should be null instead of zero in Calcite HiveSortLimit
[
https://issues.apache.org/jira/browse/HIVE-21976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-21976:
---
Attachment: HIVE-21976.patch
> Offset should be null instead of zero in Calcite HiveSortLimit
> --
>
> Key: HIVE-21976
> URL: https://issues.apache.org/jira/browse/HIVE-21976
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
> Attachments: HIVE-21976.patch
>
>
> Calcite expects a value equal or greater than 1. Otherwise, it may generate
> SQL from a plan incorrectly ({{offset 0}}).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21976) Offset should be null instead of zero in Calcite HiveSortLimit
[
https://issues.apache.org/jira/browse/HIVE-21976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-21976:
---
Status: Patch Available (was: In Progress)
> Offset should be null instead of zero in Calcite HiveSortLimit
> --
>
> Key: HIVE-21976
> URL: https://issues.apache.org/jira/browse/HIVE-21976
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
> Attachments: HIVE-21976.patch
>
>
> Calcite expects a value equal or greater than 1. Otherwise, it may generate
> SQL from a plan incorrectly ({{offset 0}}).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (HIVE-21976) Offset should be null instead of zero in Calcite HiveSortLimit
[
https://issues.apache.org/jira/browse/HIVE-21976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-21976 started by Jesus Camacho Rodriguez.
--
> Offset should be null instead of zero in Calcite HiveSortLimit
> --
>
> Key: HIVE-21976
> URL: https://issues.apache.org/jira/browse/HIVE-21976
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
>
> Calcite expects a value equal or greater than 1. Otherwise, it may generate
> SQL from a plan incorrectly ({{offset 0}}).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21637) Synchronized metastore cache
[
https://issues.apache.org/jira/browse/HIVE-21637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881634#comment-16881634
]
Hive QA commented on HIVE-21637:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
20s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
28s{color} | {color:blue} storage-api in master has 48 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
26s{color} | {color:blue} standalone-metastore/metastore-common in master has
31 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
19s{color} | {color:blue} standalone-metastore/metastore-server in master has
179 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
20s{color} | {color:blue} ql in master has 2252 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
35s{color} | {color:blue} beeline in master has 44 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
30s{color} | {color:blue} hcatalog/server-extensions in master has 3 extant
Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
24s{color} | {color:blue}
standalone-metastore/metastore-tools/metastore-benchmarks in master has 3
extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
42s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
49s{color} | {color:blue} itests/util in master has 44 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m
33s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
31s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} storage-api: The patch generated 1 new + 5 unchanged -
0 fixed = 6 total (was 5) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
15s{color} | {color:red} standalone-metastore/metastore-common: The patch
generated 9 new + 498 unchanged - 2 fixed = 507 total (was 500) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
40s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 164 new + 2193 unchanged - 65 fixed = 2357 total (was 2258) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
50s{color} | {color:red} ql: The patch generated 28 new + 1055 unchanged - 10
fixed = 1083 total (was 1065) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
11s{color} | {color:red} standalone-metastore/metastore-tools/tools-common: The
patch generated 5 new + 31 unchanged - 0 fixed = 36 total (was 31) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
11s{color} | {color:red} itests/hcatalog-unit: The patch generated 2 new + 24
unchanged - 3 fixed = 26 total (was 27) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} itests/hive-unit: The patch generated 3 new + 163
unchanged - 1 fixed = 166 total (was 164) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} |
[jira] [Updated] (HIVE-21977) Clean up DescTableOperation
[ https://issues.apache.org/jira/browse/HIVE-21977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21977: -- Attachment: HIVE-21977.01.patch > Clean up DescTableOperation > --- > > Key: HIVE-21977 > URL: https://issues.apache.org/jira/browse/HIVE-21977 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-21977.01.patch > > > DescTable operation is messy, with some unclean solutions. It should be > refactored. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21977) Clean up DescTableOperation
[ https://issues.apache.org/jira/browse/HIVE-21977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21977: -- Status: Patch Available (was: Open) > Clean up DescTableOperation > --- > > Key: HIVE-21977 > URL: https://issues.apache.org/jira/browse/HIVE-21977 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-21977.01.patch > > > DescTable operation is messy, with some unclean solutions. It should be > refactored. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21977) Clean up DescTableOperation
[ https://issues.apache.org/jira/browse/HIVE-21977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21977: -- Labels: refactor-ddl (was: ) > Clean up DescTableOperation > --- > > Key: HIVE-21977 > URL: https://issues.apache.org/jira/browse/HIVE-21977 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-ddl > Fix For: 4.0.0 > > > DescTable operation is messy, with some unclean solutions. It should be > refactored. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21977) Clean up DescTableOperation
[ https://issues.apache.org/jira/browse/HIVE-21977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely reassigned HIVE-21977: - > Clean up DescTableOperation > --- > > Key: HIVE-21977 > URL: https://issues.apache.org/jira/browse/HIVE-21977 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > > DescTable operation is messy, with some unclean solutions. It should be > refactored. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21976) Offset should be null instead of zero in Calcite HiveSortLimit
[
https://issues.apache.org/jira/browse/HIVE-21976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-21976:
--
> Offset should be null instead of zero in Calcite HiveSortLimit
> --
>
> Key: HIVE-21976
> URL: https://issues.apache.org/jira/browse/HIVE-21976
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
>
> Calcite expects a value equal or greater than 1. Otherwise, it may generate
> SQL from a plan incorrectly ({{offset 0}}).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally
[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881618#comment-16881618 ] Vineet Garg commented on HIVE-21225: bq. While building directory snapshots, check for isValidBase, isCompactedBase and isRawFormated and cache those for later within each snapshot. [~vgumashta] I think it makes more sense to cache them whenever they are called the first time. It looks like these functions might not be called for all the base directories and not all the time. So we should pay penalty only when necessary. bq. For the union case, let me look at what the rest of the code is doing, but in any case, we will need to map union_mm/delta_001_001_0002/HIVE_UNION_SUBDIR_2 to delta_001_001_0002 I guess doing this should fix the second issue > ACID: getAcidState() should cache a recursive dir listing locally > - > > Key: HIVE-21225 > URL: https://issues.apache.org/jira/browse/HIVE-21225 > Project: Hive > Issue Type: Improvement > Components: Transactions >Reporter: Gopal V >Assignee: Vaibhav Gumashta >Priority: Major > Attachments: HIVE-21225.1.patch, HIVE-21225.2.patch, > HIVE-21225.3.patch, HIVE-21225.4.patch, HIVE-21225.4.patch, > HIVE-21225.5.patch, HIVE-21225.6.patch, HIVE-21225.7.patch, > HIVE-21225.7.patch, HIVE-21225.8.patch, HIVE-21225.9.patch, async-pid-44-2.svg > > > Currently getAcidState() makes 3 calls into the FS api which could be > answered by making a single recursive listDir call and reusing the same data > to check for isRawFormat() and isValidBase(). > All delta operations for a single partition can go against a single listed > directory snapshot instead of interacting with the NameNode or ObjectStore > within the inner loop. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally
[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881614#comment-16881614 ] Vaibhav Gumashta commented on HIVE-21225: - [~vgarg] Thanks for the review: let me update the patch with feedback. On the 2 issues you raised, here is what I am thinking: 1. While building directory snapshots, check for isValidBase, isCompactedBase and isRawFormated and cache those for later within each snapshot. 2. For the union case, let me look at what the rest of the code is doing, but in any case, we will need to map union_mm/delta_001_001_0002/HIVE_UNION_SUBDIR_2 to delta_001_001_0002. I am guessing in other places as well we may be parsing delta_001_001_0002 portion of it, but I can verify and use the same approach. > ACID: getAcidState() should cache a recursive dir listing locally > - > > Key: HIVE-21225 > URL: https://issues.apache.org/jira/browse/HIVE-21225 > Project: Hive > Issue Type: Improvement > Components: Transactions >Reporter: Gopal V >Assignee: Vaibhav Gumashta >Priority: Major > Attachments: HIVE-21225.1.patch, HIVE-21225.2.patch, > HIVE-21225.3.patch, HIVE-21225.4.patch, HIVE-21225.4.patch, > HIVE-21225.5.patch, HIVE-21225.6.patch, HIVE-21225.7.patch, > HIVE-21225.7.patch, HIVE-21225.8.patch, HIVE-21225.9.patch, async-pid-44-2.svg > > > Currently getAcidState() makes 3 calls into the FS api which could be > answered by making a single recursive listDir call and reusing the same data > to check for isRawFormat() and isValidBase(). > All delta operations for a single partition can go against a single listed > directory snapshot instead of interacting with the NameNode or ObjectStore > within the inner loop. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (HIVE-21967) Clean up CreateTableLikeOperation
[ https://issues.apache.org/jira/browse/HIVE-21967?focusedWorklogId=274557=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-274557 ] ASF GitHub Bot logged work on HIVE-21967: - Author: ASF GitHub Bot Created on: 09/Jul/19 23:38 Start Date: 09/Jul/19 23:38 Worklog Time Spent: 10m Work Description: miklosgergely commented on pull request #708: HIVE-21967 Clean up CreateTableLikeOperation URL: https://github.com/apache/hive/pull/708 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 274557) Time Spent: 20m (was: 10m) > Clean up CreateTableLikeOperation > - > > Key: HIVE-21967 > URL: https://issues.apache.org/jira/browse/HIVE-21967 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-21967.01.patch > > Time Spent: 20m > Remaining Estimate: 0h > > CreateTableLikeOperation has two sub types, creating from view or table. A > lot of their codes is common, they should be reused. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally
[ https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881610#comment-16881610 ] Vineet Garg commented on HIVE-21225: The patch is building directory snapshot (dir path to file lists). For example in this case snapshot keys look like as following: * union_mm/delta_001_001_0002/HIVE_UNION_SUBDIR_2 * union_mm/delta_001_001_0001/HIVE_UNION_SUBDIR_1 This breaks the logic where base and delta directories are identified using directory name prefix. Since none of these begin with base_ or delta_ they are skipped. We could update this logic to look for base_ or delta_ in the whole path but I don't know if this is correct and safe. > ACID: getAcidState() should cache a recursive dir listing locally > - > > Key: HIVE-21225 > URL: https://issues.apache.org/jira/browse/HIVE-21225 > Project: Hive > Issue Type: Improvement > Components: Transactions >Reporter: Gopal V >Assignee: Vaibhav Gumashta >Priority: Major > Attachments: HIVE-21225.1.patch, HIVE-21225.2.patch, > HIVE-21225.3.patch, HIVE-21225.4.patch, HIVE-21225.4.patch, > HIVE-21225.5.patch, HIVE-21225.6.patch, HIVE-21225.7.patch, > HIVE-21225.7.patch, HIVE-21225.8.patch, HIVE-21225.9.patch, async-pid-44-2.svg > > > Currently getAcidState() makes 3 calls into the FS api which could be > answered by making a single recursive listDir call and reusing the same data > to check for isRawFormat() and isValidBase(). > All delta operations for a single partition can go against a single listed > directory snapshot instead of interacting with the NameNode or ObjectStore > within the inner loop. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21971) HS2 leaks classloader due to `ReflectionUtils::CONSTRUCTOR_CACHE` with temporary functions + GenericUDF
[
https://issues.apache.org/jira/browse/HIVE-21971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881608#comment-16881608
]
Hive QA commented on HIVE-21971:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12974026/HIVE-21971.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16605 tests
executed
*Failed tests:*
{noformat}
TestReplAcrossInstancesWithJsonMessageFormat - did not produce a TEST-*.xml
file (likely timed out) (batchId=255)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17932/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17932/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17932/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12974026 - PreCommit-HIVE-Build
> HS2 leaks classloader due to `ReflectionUtils::CONSTRUCTOR_CACHE` with
> temporary functions + GenericUDF
> ---
>
> Key: HIVE-21971
> URL: https://issues.apache.org/jira/browse/HIVE-21971
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 2.3.4
>Reporter: Rajesh Balamohan
>Priority: Critical
> Attachments: HIVE-21971.1.patch
>
>
> https://issues.apache.org/jira/browse/HIVE-10329 helped in moving away from
> hadoop's ReflectionUtils constructor cache issue
> (https://issues.apache.org/jira/browse/HADOOP-10513).
> However, there are corner cases where hadoop's {{ReflectionUtils}} is in use
> and this causes gradual build up of memory in HS2.
> I have observed this in Hive 2.3. But the codepath in master for this has not
> changed much.
> Easiest way to repro would be to add a temp function which extends
> {{GenericUDF}}. In {{FunctionRegistry::cloneGenericUDF,}} this would
> end up using {{org.apache.hadoop.util.ReflectionUtils.newInstance}} which in
> turn lands up in COSNTRUCTOR_CACHE of ReflectionUtils.
> {noformat}
> CREATE TEMPORARY FUNCTION dummy AS 'com.hive.test.DummyGenericUDF' USING JAR
> 'file:///home/test/udf/dummy.jar';
> select dummy();
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> at
> org.apache.hadoop.hive.ql.exec.FunctionRegistry.cloneGenericUDF(FunctionRegistry.java:1353)
> at
> org.apache.hadoop.hive.ql.exec.FunctionInfo.getGenericUDF(FunctionInfo.java:122)
> at
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.getXpathOrFuncExprNodeDesc(TypeCheckProcFactory.java:983)
> at
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.process(TypeCheckProcFactory.java:1359)
> at
> org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89)
> at
> org.apache.hadoop.hive.ql.lib.ExpressionWalker.walk(ExpressionWalker.java:76)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:120)
> {noformat}
> Note: Reflection based invocation of hadoop's {{ReflectionUtils::clear}} was
> removed in 2.x.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21861) ClassCastException during CTAS over external table using KafkaStorageHandler
[
https://issues.apache.org/jira/browse/HIVE-21861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881607#comment-16881607
]
Jesus Camacho Rodriguez commented on HIVE-21861:
For instance, deserialize would create a {{LazyString}} object and
{{LazyStringObjectInspector}} would extract a {{Text}} value from it, which is
what the method {{assignRowColumn}} is expecting.
I think the problem is in KafkaSerDe. Since you are creating lazy objects, it
seems to me that {{public ObjectInspector getObjectInspector()}} method may be
not be returning the expected object inspector? It seems it is always created
as standard and it is different from {{delegateSerDe.getObjectInspector}}:
{code}
objectInspector =
ObjectInspectorFactory.getStandardStructObjectInspector(columnNames,
inspectors);
{code}
> ClassCastException during CTAS over external table using KafkaStorageHandler
>
>
> Key: HIVE-21861
> URL: https://issues.apache.org/jira/browse/HIVE-21861
> Project: Hive
> Issue Type: Bug
> Components: kafka integration
>Affects Versions: 4.0.0
>Reporter: Justin Leet
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-21861.patch
>
>
> To reproduce, create a table similar to the following:
> {code}
> CREATE EXTERNAL TABLE
> (raw_value STRING)
> ROW FORMAT DELIMITED
> LINES TERMINATED BY '\n'
> STORED BY 'org.apache.hadoop.hive.kafka.KafkaStorageHandler'
> TBLPROPERTIES(
> "kafka.topic"="",
> "kafka.bootstrap.servers"="",
> "kafka.consumer.security.protocol"="PLAINTEXT",
> "kafka.serde.class"="org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe");
> {code}
> Note the SerDe isn't the default SerDe. Additionally, this error occurs when
> vectorization is enabled.
> Basic queries work fine:
> {code}
> SELECT * FROM LIMIT 1;
> {code}
> Doing a CTAS to bring it into a managed table fails:
> {code}
> CREATE TABLE AS
> SELECT * FROM ;
> {code}
> The exception is:
> {code}
> Caused by: java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.lazy.LazyString cannot be cast to
> org.apache.hadoop.io.TextCaused by: java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.lazy.LazyString cannot be cast to
> org.apache.hadoop.io.Text at
> org.apache.hadoop.hive.ql.exec.vector.VectorAssignRow.assignRowColumn(VectorAssignRow.java:471)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorAssignRow.assignRowColumn(VectorAssignRow.java:350)
> at
> org.apache.hadoop.hive.kafka.VectorizedKafkaRecordReader.readNextBatch(VectorizedKafkaRecordReader.java:159)
> at
> org.apache.hadoop.hive.kafka.VectorizedKafkaRecordReader.next(VectorizedKafkaRecordReader.java:113)
> at
> org.apache.hadoop.hive.kafka.VectorizedKafkaRecordReader.next(VectorizedKafkaRecordReader.java:47)
> at
> org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:360)
> ... 24 more
> {code}
> A workaround to this is to disable vectorization via:
> {code}
> set hive.vectorized.execution.enabled = false;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21971) HS2 leaks classloader due to `ReflectionUtils::CONSTRUCTOR_CACHE` with temporary functions + GenericUDF
[
https://issues.apache.org/jira/browse/HIVE-21971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881600#comment-16881600
]
Hive QA commented on HIVE-21971:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
3s{color} | {color:blue} ql in master has 2252 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
40s{color} | {color:red} ql: The patch generated 1 new + 74 unchanged - 0 fixed
= 75 total (was 74) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 30s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17932/dev-support/hive-personality.sh
|
| git revision | master / 33d2995 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17932/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17932/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> HS2 leaks classloader due to `ReflectionUtils::CONSTRUCTOR_CACHE` with
> temporary functions + GenericUDF
> ---
>
> Key: HIVE-21971
> URL: https://issues.apache.org/jira/browse/HIVE-21971
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 2.3.4
>Reporter: Rajesh Balamohan
>Priority: Critical
> Attachments: HIVE-21971.1.patch
>
>
> https://issues.apache.org/jira/browse/HIVE-10329 helped in moving away from
> hadoop's ReflectionUtils constructor cache issue
> (https://issues.apache.org/jira/browse/HADOOP-10513).
> However, there are corner cases where hadoop's {{ReflectionUtils}} is in use
> and this causes gradual build up of memory in HS2.
> I have observed this in Hive 2.3. But the codepath in master for this has not
> changed much.
> Easiest way to repro would be to add a temp function which extends
> {{GenericUDF}}. In {{FunctionRegistry::cloneGenericUDF,}} this would
> end up using {{org.apache.hadoop.util.ReflectionUtils.newInstance}} which in
> turn lands up in COSNTRUCTOR_CACHE of ReflectionUtils.
> {noformat}
> CREATE TEMPORARY FUNCTION dummy AS 'com.hive.test.DummyGenericUDF' USING JAR
> 'file:///home/test/udf/dummy.jar';
> select dummy();
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> at
>
[jira] [Updated] (HIVE-21973) "show locks" print the header twice.
[
https://issues.apache.org/jira/browse/HIVE-21973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajkumar Singh updated HIVE-21973:
--
Status: Open (was: Patch Available)
> "show locks" print the header twice.
>
>
> Key: HIVE-21973
> URL: https://issues.apache.org/jira/browse/HIVE-21973
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.1
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Minor
> Attachments: HIVE-21973.01.patch, HIVE-21973.patch
>
>
> show locks;
> -- output
> {code:java}
> +--+---+++-+-++-+-+--+---+---+-+
> | lockid | database | table | partition | lock_state | blocked_by |
> lock_type | transaction_id | last_heartbeat | acquired_at | user |
> hostname | agent_info |
> +--+---+++-+-++-+-+--+---+---+-+
> | Lock ID | Database | Table | Partition | State | Blocked By |
> Type | Transaction ID | Last Heartbeat | Acquired At | User |
> Hostname | Agent Info |
> +--+---+++-+-++-+-+--+---+---+-+
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21973) "show locks" print the header twice.
[
https://issues.apache.org/jira/browse/HIVE-21973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881597#comment-16881597
]
Rajkumar Singh commented on HIVE-21973:
---
test failure seems unrelated, attaching a patch again for a fresh clean run.
> "show locks" print the header twice.
>
>
> Key: HIVE-21973
> URL: https://issues.apache.org/jira/browse/HIVE-21973
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.1
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Minor
> Attachments: HIVE-21973.01.patch, HIVE-21973.patch
>
>
> show locks;
> -- output
> {code:java}
> +--+---+++-+-++-+-+--+---+---+-+
> | lockid | database | table | partition | lock_state | blocked_by |
> lock_type | transaction_id | last_heartbeat | acquired_at | user |
> hostname | agent_info |
> +--+---+++-+-++-+-+--+---+---+-+
> | Lock ID | Database | Table | Partition | State | Blocked By |
> Type | Transaction ID | Last Heartbeat | Acquired At | User |
> Hostname | Agent Info |
> +--+---+++-+-++-+-+--+---+---+-+
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21973) "show locks" print the header twice.
[
https://issues.apache.org/jira/browse/HIVE-21973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajkumar Singh updated HIVE-21973:
--
Attachment: HIVE-21973.01.patch
Status: Patch Available (was: Open)
> "show locks" print the header twice.
>
>
> Key: HIVE-21973
> URL: https://issues.apache.org/jira/browse/HIVE-21973
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.1
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Minor
> Attachments: HIVE-21973.01.patch, HIVE-21973.patch
>
>
> show locks;
> -- output
> {code:java}
> +--+---+++-+-++-+-+--+---+---+-+
> | lockid | database | table | partition | lock_state | blocked_by |
> lock_type | transaction_id | last_heartbeat | acquired_at | user |
> hostname | agent_info |
> +--+---+++-+-++-+-+--+---+---+-+
> | Lock ID | Database | Table | Partition | State | Blocked By |
> Type | Transaction ID | Last Heartbeat | Acquired At | User |
> Hostname | Agent Info |
> +--+---+++-+-++-+-+--+---+---+-+
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21861) ClassCastException during CTAS over external table using KafkaStorageHandler
[
https://issues.apache.org/jira/browse/HIVE-21861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881595#comment-16881595
]
Rajkumar Singh commented on HIVE-21861:
---
As per my understanding, it looks that it is expected to get the lazystring
during assignrow. this is how the code flow looks like.
Kafka Serde delegate to LazySimpleSerDe
https://github.com/apache/hive/blob/eba668eed6fecce6bae87fb77ca056b8e34ad5e2/kafka-handler/src/java/org/apache/hadoop/hive/kafka/KafkaSerDe.java#L102
LazySimpleSerDe initialize and set
https://github.com/apache/hive/blob/8a606abdec0f92d60653d892b2f92ff729f1c020/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazySimpleSerDe.java#L119
and set the object inspector to
LazySimpleStructObjectInspector
now during deserialization, lazy serde deserialize the text writable and return
row as lazystruct
https://github.com/apache/hive/blob/eba668eed6fecce6bae87fb77ca056b8e34ad5e2/kafka-handler/src/java/org/apache/hadoop/hive/kafka/VectorizedKafkaRecordReader.java#L157
https://github.com/apache/hive/blob/eba668eed6fecce6bae87fb77ca056b8e34ad5e2/kafka-handler/src/java/org/apache/hadoop/hive/kafka/KafkaSerDe.java#L206
since delegateDeserializerOI is LazyString object inspector hence the following
piece of code return the lazy string.
https://github.com/apache/hive/blob/eba668eed6fecce6bae87fb77ca056b8e34ad5e2/kafka-handler/src/java/org/apache/hadoop/hive/kafka/KafkaSerDe.java#L210
> ClassCastException during CTAS over external table using KafkaStorageHandler
>
>
> Key: HIVE-21861
> URL: https://issues.apache.org/jira/browse/HIVE-21861
> Project: Hive
> Issue Type: Bug
> Components: kafka integration
>Affects Versions: 4.0.0
>Reporter: Justin Leet
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-21861.patch
>
>
> To reproduce, create a table similar to the following:
> {code}
> CREATE EXTERNAL TABLE
> (raw_value STRING)
> ROW FORMAT DELIMITED
> LINES TERMINATED BY '\n'
> STORED BY 'org.apache.hadoop.hive.kafka.KafkaStorageHandler'
> TBLPROPERTIES(
> "kafka.topic"="",
> "kafka.bootstrap.servers"="",
> "kafka.consumer.security.protocol"="PLAINTEXT",
> "kafka.serde.class"="org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe");
> {code}
> Note the SerDe isn't the default SerDe. Additionally, this error occurs when
> vectorization is enabled.
> Basic queries work fine:
> {code}
> SELECT * FROM LIMIT 1;
> {code}
> Doing a CTAS to bring it into a managed table fails:
> {code}
> CREATE TABLE AS
> SELECT * FROM ;
> {code}
> The exception is:
> {code}
> Caused by: java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.lazy.LazyString cannot be cast to
> org.apache.hadoop.io.TextCaused by: java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.lazy.LazyString cannot be cast to
> org.apache.hadoop.io.Text at
> org.apache.hadoop.hive.ql.exec.vector.VectorAssignRow.assignRowColumn(VectorAssignRow.java:471)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorAssignRow.assignRowColumn(VectorAssignRow.java:350)
> at
> org.apache.hadoop.hive.kafka.VectorizedKafkaRecordReader.readNextBatch(VectorizedKafkaRecordReader.java:159)
> at
> org.apache.hadoop.hive.kafka.VectorizedKafkaRecordReader.next(VectorizedKafkaRecordReader.java:113)
> at
> org.apache.hadoop.hive.kafka.VectorizedKafkaRecordReader.next(VectorizedKafkaRecordReader.java:47)
> at
> org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:360)
> ... 24 more
> {code}
> A workaround to this is to disable vectorization via:
> {code}
> set hive.vectorized.execution.enabled = false;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21970) Avoid using RegistryUtils.currentUser()
[ https://issues.apache.org/jira/browse/HIVE-21970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-21970: - Attachment: HIVE-21970.2.patch > Avoid using RegistryUtils.currentUser() > --- > > Key: HIVE-21970 > URL: https://issues.apache.org/jira/browse/HIVE-21970 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0, 3.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-21970.1.patch, HIVE-21970.2.patch > > > RegistryUtils.currentUser() does replacement of '_' with '-' for DNS reasons. > This is used inconsistently in some places causing issues wrt. ZK (deletion > token secret manager, llap cluster membership for external clients). > > Replace RegistryUtils.currentUser() with > UserGroupInformation.getCurrentUser().getShortUserName() for consistency. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21970) Avoid using RegistryUtils.currentUser()
[ https://issues.apache.org/jira/browse/HIVE-21970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881588#comment-16881588 ] Prasanth Jayachandran commented on HIVE-21970: -- Failure seems unrelated. Retrying. > Avoid using RegistryUtils.currentUser() > --- > > Key: HIVE-21970 > URL: https://issues.apache.org/jira/browse/HIVE-21970 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0, 3.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-21970.1.patch, HIVE-21970.2.patch > > > RegistryUtils.currentUser() does replacement of '_' with '-' for DNS reasons. > This is used inconsistently in some places causing issues wrt. ZK (deletion > token secret manager, llap cluster membership for external clients). > > Replace RegistryUtils.currentUser() with > UserGroupInformation.getCurrentUser().getShortUserName() for consistency. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21973) "show locks" print the header twice.
[
https://issues.apache.org/jira/browse/HIVE-21973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881586#comment-16881586
]
Hive QA commented on HIVE-21973:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12974015/HIVE-21973.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16603 tests
executed
*Failed tests:*
{noformat}
TestReplAcrossInstancesWithJsonMessageFormat - did not produce a TEST-*.xml
file (likely timed out) (batchId=255)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17931/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17931/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17931/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12974015 - PreCommit-HIVE-Build
> "show locks" print the header twice.
>
>
> Key: HIVE-21973
> URL: https://issues.apache.org/jira/browse/HIVE-21973
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.1
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Minor
> Attachments: HIVE-21973.patch
>
>
> show locks;
> -- output
> {code:java}
> +--+---+++-+-++-+-+--+---+---+-+
> | lockid | database | table | partition | lock_state | blocked_by |
> lock_type | transaction_id | last_heartbeat | acquired_at | user |
> hostname | agent_info |
> +--+---+++-+-++-+-+--+---+---+-+
> | Lock ID | Database | Table | Partition | State | Blocked By |
> Type | Transaction ID | Last Heartbeat | Acquired At | User |
> Hostname | Agent Info |
> +--+---+++-+-++-+-+--+---+---+-+
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally
[
https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881575#comment-16881575
]
Vineet Garg commented on HIVE-21225:
Went over the patch once again and I am struggling to understand how this patch
solves the problem cited in the jira. This patch is still calling FS in
{{isValidBase}}, {{isCompactedBase}} and {{isRawFormated}}. May be I am
misunderstanding something here?
Currently file/dir listing is done to retrieve top level directories/files only
(and then a recursive call within original (non-acid) directories if
necessary). Multiple calls are done for {{isCompactedBase}} which makes a call
to FS to read metadata file and for {{isRawFormatted}} which beside reading
metadata file also makes FS call to look for files matching particular pattern.
This seems to be done multiple times for the same base. I believe the low
hanging fruit here is to cache this information and reuse it. This should
reduce unnecessary calls to FS.
Doing a recursive call instead of only looking for top level
subdirectories/files are causing issue where directory layout is different
(.e.g. CTAS with union). I am not sure how should the current logic be
tuned/fixed to cater to this case.
> ACID: getAcidState() should cache a recursive dir listing locally
> -
>
> Key: HIVE-21225
> URL: https://issues.apache.org/jira/browse/HIVE-21225
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Reporter: Gopal V
>Assignee: Vaibhav Gumashta
>Priority: Major
> Attachments: HIVE-21225.1.patch, HIVE-21225.2.patch,
> HIVE-21225.3.patch, HIVE-21225.4.patch, HIVE-21225.4.patch,
> HIVE-21225.5.patch, HIVE-21225.6.patch, HIVE-21225.7.patch,
> HIVE-21225.7.patch, HIVE-21225.8.patch, HIVE-21225.9.patch, async-pid-44-2.svg
>
>
> Currently getAcidState() makes 3 calls into the FS api which could be
> answered by making a single recursive listDir call and reusing the same data
> to check for isRawFormat() and isValidBase().
> All delta operations for a single partition can go against a single listed
> directory snapshot instead of interacting with the NameNode or ObjectStore
> within the inner loop.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21637) Synchronized metastore cache
[ https://issues.apache.org/jira/browse/HIVE-21637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-21637: -- Attachment: HIVE-21637.22.patch > Synchronized metastore cache > > > Key: HIVE-21637 > URL: https://issues.apache.org/jira/browse/HIVE-21637 > Project: Hive > Issue Type: New Feature >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-21637-1.patch, HIVE-21637.10.patch, > HIVE-21637.11.patch, HIVE-21637.12.patch, HIVE-21637.13.patch, > HIVE-21637.14.patch, HIVE-21637.15.patch, HIVE-21637.16.patch, > HIVE-21637.17.patch, HIVE-21637.18.patch, HIVE-21637.19.patch, > HIVE-21637.19.patch, HIVE-21637.2.patch, HIVE-21637.20.patch, > HIVE-21637.21.patch, HIVE-21637.22.patch, HIVE-21637.3.patch, > HIVE-21637.4.patch, HIVE-21637.5.patch, HIVE-21637.6.patch, > HIVE-21637.7.patch, HIVE-21637.8.patch, HIVE-21637.9.patch > > > Currently, HMS has a cache implemented by CachedStore. The cache is > asynchronized and in HMS HA setting, we can only get eventual consistency. In > this Jira, we try to make it synchronized. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (HIVE-21348) Execute the TIMESTAMP types roadmap
[ https://issues.apache.org/jira/browse/HIVE-21348?focusedWorklogId=274395=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-274395 ] ASF GitHub Bot logged work on HIVE-21348: - Author: ASF GitHub Bot Created on: 09/Jul/19 21:15 Start Date: 09/Jul/19 21:15 Worklog Time Spent: 10m Work Description: srdo commented on issue #711: HIVE-21348: Add currentTransactionBatch and staticPartitionValues to … URL: https://github.com/apache/hive/pull/711#issuecomment-509812159 Fixed here https://github.com/apache/hive/pull/712 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 274395) Time Spent: 40m (was: 0.5h) > Execute the TIMESTAMP types roadmap > --- > > Key: HIVE-21348 > URL: https://issues.apache.org/jira/browse/HIVE-21348 > Project: Hive > Issue Type: Task >Reporter: Zoltan Ivanfi >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > This is the top-level JIRA for tracking the addition and/or alteration of > different TIMESTAMP types in order to eventually reach the desired state as > specified in the [design doc for TIMESTAMP > types|https://cwiki.apache.org/confluence/display/Hive/Different+TIMESTAMP+types]. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21438) HiveStreamingConnection.toString doesn't print transaction batch
[ https://issues.apache.org/jira/browse/HIVE-21438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-21438: -- Labels: pull-request-available (was: ) > HiveStreamingConnection.toString doesn't print transaction batch > > > Key: HIVE-21438 > URL: https://issues.apache.org/jira/browse/HIVE-21438 > Project: Hive > Issue Type: Improvement > Components: Streaming >Affects Versions: 3.1.1 >Reporter: Stig Rohde Døssing >Assignee: Stig Rohde Døssing >Priority: Minor > Labels: pull-request-available > Attachments: > 0001-HIVE-21348-Add-currentTransactionBatch-and-staticPar.patch > > > HiveStreamingConnection.toString doesn't contain the current transaction > state. In hive-hcatalog-streaming, the transaction batch was exposed to the > user, which allowed the application to log e.g. transaction id and state when > errors occur. Some exceptions from TransactionBatch contain the current > transaction id, but many don't. > It would be nice if HiveStreamingConnection.toString also included the > currentTransactionBatch.toString. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (HIVE-21438) HiveStreamingConnection.toString doesn't print transaction batch
[ https://issues.apache.org/jira/browse/HIVE-21438?focusedWorklogId=274394=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-274394 ] ASF GitHub Bot logged work on HIVE-21438: - Author: ASF GitHub Bot Created on: 09/Jul/19 21:14 Start Date: 09/Jul/19 21:14 Worklog Time Spent: 10m Work Description: srdo commented on pull request #712: HIVE-21438: Add currentTransactionBatch and staticPartitionValues to … URL: https://github.com/apache/hive/pull/712 …HiveStreamingConnection toString output https://issues.apache.org/jira/browse/HIVE-21438 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 274394) Time Spent: 10m Remaining Estimate: 0h > HiveStreamingConnection.toString doesn't print transaction batch > > > Key: HIVE-21438 > URL: https://issues.apache.org/jira/browse/HIVE-21438 > Project: Hive > Issue Type: Improvement > Components: Streaming >Affects Versions: 3.1.1 >Reporter: Stig Rohde Døssing >Assignee: Stig Rohde Døssing >Priority: Minor > Labels: pull-request-available > Attachments: > 0001-HIVE-21348-Add-currentTransactionBatch-and-staticPar.patch > > Time Spent: 10m > Remaining Estimate: 0h > > HiveStreamingConnection.toString doesn't contain the current transaction > state. In hive-hcatalog-streaming, the transaction batch was exposed to the > user, which allowed the application to log e.g. transaction id and state when > errors occur. Some exceptions from TransactionBatch contain the current > transaction id, but many don't. > It would be nice if HiveStreamingConnection.toString also included the > currentTransactionBatch.toString. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21973) "show locks" print the header twice.
[
https://issues.apache.org/jira/browse/HIVE-21973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881556#comment-16881556
]
Hive QA commented on HIVE-21973:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
5s{color} | {color:blue} ql in master has 2252 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 36s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17931/dev-support/hive-personality.sh
|
| git revision | master / 33d2995 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17931/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> "show locks" print the header twice.
>
>
> Key: HIVE-21973
> URL: https://issues.apache.org/jira/browse/HIVE-21973
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.1
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Minor
> Attachments: HIVE-21973.patch
>
>
> show locks;
> -- output
> {code:java}
> +--+---+++-+-++-+-+--+---+---+-+
> | lockid | database | table | partition | lock_state | blocked_by |
> lock_type | transaction_id | last_heartbeat | acquired_at | user |
> hostname | agent_info |
> +--+---+++-+-++-+-+--+---+---+-+
> | Lock ID | Database | Table | Partition | State | Blocked By |
> Type | Transaction ID | Last Heartbeat | Acquired At | User |
> Hostname | Agent Info |
> +--+---+++-+-++-+-+--+---+---+-+
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (HIVE-21348) Execute the TIMESTAMP types roadmap
[ https://issues.apache.org/jira/browse/HIVE-21348?focusedWorklogId=274388=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-274388 ] ASF GitHub Bot logged work on HIVE-21348: - Author: ASF GitHub Bot Created on: 09/Jul/19 21:09 Start Date: 09/Jul/19 21:09 Worklog Time Spent: 10m Work Description: jcamachor commented on pull request #711: HIVE-21348: Add currentTransactionBatch and staticPartitionValues to … URL: https://github.com/apache/hive/pull/711 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 274388) Time Spent: 0.5h (was: 20m) > Execute the TIMESTAMP types roadmap > --- > > Key: HIVE-21348 > URL: https://issues.apache.org/jira/browse/HIVE-21348 > Project: Hive > Issue Type: Task >Reporter: Zoltan Ivanfi >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > This is the top-level JIRA for tracking the addition and/or alteration of > different TIMESTAMP types in order to eventually reach the desired state as > specified in the [design doc for TIMESTAMP > types|https://cwiki.apache.org/confluence/display/Hive/Different+TIMESTAMP+types]. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (HIVE-21348) Execute the TIMESTAMP types roadmap
[ https://issues.apache.org/jira/browse/HIVE-21348?focusedWorklogId=274387=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-274387 ] ASF GitHub Bot logged work on HIVE-21348: - Author: ASF GitHub Bot Created on: 09/Jul/19 21:09 Start Date: 09/Jul/19 21:09 Worklog Time Spent: 10m Work Description: jcamachor commented on issue #711: HIVE-21348: Add currentTransactionBatch and staticPartitionValues to … URL: https://github.com/apache/hive/pull/711#issuecomment-509810189 The title of this PR is referencing wrong issue. HIVE-21348 -> HIVE-21438 I am closing it, please reopen with new title so it is linked to the correct JIRA. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 274387) Time Spent: 20m (was: 10m) > Execute the TIMESTAMP types roadmap > --- > > Key: HIVE-21348 > URL: https://issues.apache.org/jira/browse/HIVE-21348 > Project: Hive > Issue Type: Task >Reporter: Zoltan Ivanfi >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > This is the top-level JIRA for tracking the addition and/or alteration of > different TIMESTAMP types in order to eventually reach the desired state as > specified in the [design doc for TIMESTAMP > types|https://cwiki.apache.org/confluence/display/Hive/Different+TIMESTAMP+types]. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21972) "show transactions" display the header twice
[
https://issues.apache.org/jira/browse/HIVE-21972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajkumar Singh updated HIVE-21972:
--
Status: Open (was: Patch Available)
> "show transactions" display the header twice
>
>
> Key: HIVE-21972
> URL: https://issues.apache.org/jira/browse/HIVE-21972
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.1
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-21972.01.patch, HIVE-21972.patch
>
>
> show transactions;
> {code:java}
> +-+++--+---+---+
> | txnid | state| startedtime | lastheartbeattime
> | user | host|
> +-+++--+---+---+
> | Transaction ID | Transaction State | Started Time | Last Heartbeat Time
> | User | Hostname |
> | 896 | ABORTED| 1560209607000 | 1560209607000
> | hive | hostname |
> +-+++--+---+---+
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21972) "show transactions" display the header twice
[
https://issues.apache.org/jira/browse/HIVE-21972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajkumar Singh updated HIVE-21972:
--
Attachment: HIVE-21972.01.patch
Status: Patch Available (was: Open)
> "show transactions" display the header twice
>
>
> Key: HIVE-21972
> URL: https://issues.apache.org/jira/browse/HIVE-21972
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.1
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-21972.01.patch, HIVE-21972.patch
>
>
> show transactions;
> {code:java}
> +-+++--+---+---+
> | txnid | state| startedtime | lastheartbeattime
> | user | host|
> +-+++--+---+---+
> | Transaction ID | Transaction State | Started Time | Last Heartbeat Time
> | User | Hostname |
> | 896 | ABORTED| 1560209607000 | 1560209607000
> | hive | hostname |
> +-+++--+---+---+
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21972) "show transactions" display the header twice
[
https://issues.apache.org/jira/browse/HIVE-21972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881544#comment-16881544
]
Rajkumar Singh commented on HIVE-21972:
---
test failure seems unrelated but attaching the patch again for a clean run.
Thanks
> "show transactions" display the header twice
>
>
> Key: HIVE-21972
> URL: https://issues.apache.org/jira/browse/HIVE-21972
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.1
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-21972.patch
>
>
> show transactions;
> {code:java}
> +-+++--+---+---+
> | txnid | state| startedtime | lastheartbeattime
> | user | host|
> +-+++--+---+---+
> | Transaction ID | Transaction State | Started Time | Last Heartbeat Time
> | User | Hostname |
> | 896 | ABORTED| 1560209607000 | 1560209607000
> | hive | hostname |
> +-+++--+---+---+
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21972) "show transactions" display the header twice
[
https://issues.apache.org/jira/browse/HIVE-21972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881527#comment-16881527
]
Hive QA commented on HIVE-21972:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12974014/HIVE-21972.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16632 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.exec.spark.TestLocalHiveSparkClient.testMultiSessionSparkContextReUse
(batchId=333)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17930/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17930/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17930/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12974014 - PreCommit-HIVE-Build
> "show transactions" display the header twice
>
>
> Key: HIVE-21972
> URL: https://issues.apache.org/jira/browse/HIVE-21972
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.1
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-21972.patch
>
>
> show transactions;
> {code:java}
> +-+++--+---+---+
> | txnid | state| startedtime | lastheartbeattime
> | user | host|
> +-+++--+---+---+
> | Transaction ID | Transaction State | Started Time | Last Heartbeat Time
> | User | Hostname |
> | 896 | ABORTED| 1560209607000 | 1560209607000
> | hive | hostname |
> +-+++--+---+---+
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21972) "show transactions" display the header twice
[
https://issues.apache.org/jira/browse/HIVE-21972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881496#comment-16881496
]
Hive QA commented on HIVE-21972:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
7s{color} | {color:blue} ql in master has 2252 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17930/dev-support/hive-personality.sh
|
| git revision | master / 33d2995 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17930/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> "show transactions" display the header twice
>
>
> Key: HIVE-21972
> URL: https://issues.apache.org/jira/browse/HIVE-21972
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.1
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-21972.patch
>
>
> show transactions;
> {code:java}
> +-+++--+---+---+
> | txnid | state| startedtime | lastheartbeattime
> | user | host|
> +-+++--+---+---+
> | Transaction ID | Transaction State | Started Time | Last Heartbeat Time
> | User | Hostname |
> | 896 | ABORTED| 1560209607000 | 1560209607000
> | hive | hostname |
> +-+++--+---+---+
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (HIVE-21348) Execute the TIMESTAMP types roadmap
[ https://issues.apache.org/jira/browse/HIVE-21348?focusedWorklogId=274311=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-274311 ] ASF GitHub Bot logged work on HIVE-21348: - Author: ASF GitHub Bot Created on: 09/Jul/19 19:27 Start Date: 09/Jul/19 19:27 Worklog Time Spent: 10m Work Description: srdo commented on pull request #711: HIVE-21348: Add currentTransactionBatch and staticPartitionValues to … URL: https://github.com/apache/hive/pull/711 …HiveStreamingConnection toString output https://issues.apache.org/jira/browse/HIVE-21438 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 274311) Time Spent: 10m Remaining Estimate: 0h > Execute the TIMESTAMP types roadmap > --- > > Key: HIVE-21348 > URL: https://issues.apache.org/jira/browse/HIVE-21348 > Project: Hive > Issue Type: Task >Reporter: Zoltan Ivanfi >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > This is the top-level JIRA for tracking the addition and/or alteration of > different TIMESTAMP types in order to eventually reach the desired state as > specified in the [design doc for TIMESTAMP > types|https://cwiki.apache.org/confluence/display/Hive/Different+TIMESTAMP+types]. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21348) Execute the TIMESTAMP types roadmap
[ https://issues.apache.org/jira/browse/HIVE-21348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-21348: -- Labels: pull-request-available (was: ) > Execute the TIMESTAMP types roadmap > --- > > Key: HIVE-21348 > URL: https://issues.apache.org/jira/browse/HIVE-21348 > Project: Hive > Issue Type: Task >Reporter: Zoltan Ivanfi >Priority: Major > Labels: pull-request-available > > This is the top-level JIRA for tracking the addition and/or alteration of > different TIMESTAMP types in order to eventually reach the desired state as > specified in the [design doc for TIMESTAMP > types|https://cwiki.apache.org/confluence/display/Hive/Different+TIMESTAMP+types]. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21958) The list of table expression in the inclusion and exclusion list should be separated by '|' instead of comma.
[
https://issues.apache.org/jira/browse/HIVE-21958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881479#comment-16881479
]
Hive QA commented on HIVE-21958:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12974001/HIVE-21958.05.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16632 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17929/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17929/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17929/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12974001 - PreCommit-HIVE-Build
> The list of table expression in the inclusion and exclusion list should be
> separated by '|' instead of comma.
> -
>
> Key: HIVE-21958
> URL: https://issues.apache.org/jira/browse/HIVE-21958
> Project: Hive
> Issue Type: Sub-task
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21958.01.patch, HIVE-21958.02.patch,
> HIVE-21958.03.patch, HIVE-21958.04.patch, HIVE-21958.05.patch
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> Java regex expression does not support comma. If user wants multiple
> expression to be present in the include or exclude list, then the expressions
> can be provided separated by pipe ('|') character. The policy will look
> something like db_name.'(t1*)|(t3)'.'t100'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21164) ACID: explore how we can avoid a move step during inserts/compaction
[ https://issues.apache.org/jira/browse/HIVE-21164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-21164: Attachment: HIVE-21164.4.patch > ACID: explore how we can avoid a move step during inserts/compaction > > > Key: HIVE-21164 > URL: https://issues.apache.org/jira/browse/HIVE-21164 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.1.1 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta >Priority: Major > Attachments: HIVE-21164.1.patch, HIVE-21164.2.patch, > HIVE-21164.3.patch, HIVE-21164.4.patch > > > Currently, we write compacted data to a temporary location and then move the > files to a final location, which is an expensive operation on some cloud file > systems. Since HIVE-20823 is already in, it can control the visibility of > compacted data for the readers. Therefore, we can perhaps avoid writing data > to a temporary location and directly write compacted data to the intended > final path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21958) The list of table expression in the inclusion and exclusion list should be separated by '|' instead of comma.
[
https://issues.apache.org/jira/browse/HIVE-21958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881463#comment-16881463
]
Hive QA commented on HIVE-21958:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
43s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
27s{color} | {color:blue} standalone-metastore/metastore-common in master has
31 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
2s{color} | {color:blue} ql in master has 2252 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
38s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
18s{color} | {color:red} ql generated 9 new + 2243 unchanged - 9 fixed = 2252
total (was 2252) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
38s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 19s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Should org.apache.hadoop.hive.ql.parse.HiveParser$DFA235 be a _static_
inner class? At HiveParser.java:inner class? At HiveParser.java:[lines
48087-48100] |
| | Dead store to LA29_128 in
org.apache.hadoop.hive.ql.parse.HiveParser$DFA29.specialStateTransition(int,
IntStream) At
HiveParser.java:org.apache.hadoop.hive.ql.parse.HiveParser$DFA29.specialStateTransition(int,
IntStream) At HiveParser.java:[line 47827] |
| | Dead store to LA29_130 in
org.apache.hadoop.hive.ql.parse.HiveParser$DFA29.specialStateTransition(int,
IntStream) At
HiveParser.java:org.apache.hadoop.hive.ql.parse.HiveParser$DFA29.specialStateTransition(int,
IntStream) At HiveParser.java:[line 47840] |
| | Dead store to LA29_132 in
org.apache.hadoop.hive.ql.parse.HiveParser$DFA29.specialStateTransition(int,
IntStream) At
HiveParser.java:org.apache.hadoop.hive.ql.parse.HiveParser$DFA29.specialStateTransition(int,
IntStream) At HiveParser.java:[line 47853] |
| | Dead store to LA29_134 in
org.apache.hadoop.hive.ql.parse.HiveParser$DFA29.specialStateTransition(int,
IntStream) At
HiveParser.java:org.apache.hadoop.hive.ql.parse.HiveParser$DFA29.specialStateTransition(int,
IntStream) At HiveParser.java:[line 47866] |
| | Dead store to LA29_136 in
org.apache.hadoop.hive.ql.parse.HiveParser$DFA29.specialStateTransition(int,
IntStream) At
HiveParser.java:org.apache.hadoop.hive.ql.parse.HiveParser$DFA29.specialStateTransition(int,
IntStream) At HiveParser.java:[line 47879] |
| | Dead store to LA29_138 in
org.apache.hadoop.hive.ql.parse.HiveParser$DFA29.specialStateTransition(int,
IntStream) At
[jira] [Comment Edited] (HIVE-21901) Join queries across different datasources (Druid and JDBC StorageHandler)
[
https://issues.apache.org/jira/browse/HIVE-21901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881402#comment-16881402
]
slim bouguerra edited comment on HIVE-21901 at 7/9/19 5:54 PM:
---
in theory this class
{code}
DruidSelectQueryRecordReader.java
{code}
should not be used but i see in practice still used, will look at this.
was (Author: bslim):
in theory this class
{code }DruidSelectQueryRecordReader.java {code}
should not be used but i see in practice still used, will look at this.
> Join queries across different datasources (Druid and JDBC StorageHandler)
> -
>
> Key: HIVE-21901
> URL: https://issues.apache.org/jira/browse/HIVE-21901
> Project: Hive
> Issue Type: Bug
> Components: Druid integration, StorageHandler
>Affects Versions: 3.1.1
>Reporter: Subramani Raju V
>Priority: Major
>
> We have a druid datasource and have external table created in hive for the
> same datasource.
> For example:
>
> {code:java}
> CREATE EXTERNAL TABLE druid_table_1
> STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler'
> TBLPROPERTIES ("druid.datasource" = "wikipedia");
> {code}
>
>
> We have another table in mysql database, which also has an external table
> created in hive in this fashion:
>
> {code:java}
> CREATE EXTERNAL TABLE sample_table_1
> (
> old_id int,
> city_name string,
> new_id int
> )
> STORED BY 'org.apache.hive.storage.jdbc.JdbcStorageHandler'
> TBLPROPERTIES (
> "hive.sql.database.type" = "MYSQL",
> "hive.sql.jdbc.driver" = "com.mysql.jdbc.Driver",
> "hive.sql.jdbc.url" = "jdbc:mysql://172.16.0.15:3307/test",
> "hive.sql.dbcp.username" = "hive_user",
> "hive.sql.dbcp.password" = "hive_pass",
> "hive.sql.table" = "city_mapping"
> );
> {code}
> So we are able to perform normal queries on the individual tables, but when
> we try to do join operation for both the above tables in this fashion:
>
>
> {code:java}
> SELECT *
> FROM druid_table_1 o
> JOIN sample_table_1 c
> ON (c.city_name = o.channel) limit 10;
> {code}
> Then we are getting the error as follows:
>
>
> {code:java}
> TaskAttempt 1 failed, info=[Error: Error while running task ( failure ) :
> attempt_1560945328057_0022_2_01_00_1:java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:80)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:419)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> ... 16 more
> Caused by: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderNextException(HiveIOExceptionHandlerChain.java:121)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderNextException(HiveIOExceptionHandlerUtil.java:77)
> at
> org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:365)
> at
>
[jira] [Updated] (HIVE-18842) CLUSTERED ON/DISTRIBUTED ON+SORTED ON support for materialized views
[
https://issues.apache.org/jira/browse/HIVE-18842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-18842:
---
Attachment: HIVE-18842.04.patch
> CLUSTERED ON/DISTRIBUTED ON+SORTED ON support for materialized views
>
>
> Key: HIVE-18842
> URL: https://issues.apache.org/jira/browse/HIVE-18842
> Project: Hive
> Issue Type: New Feature
> Components: Materialized views
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available, todoc4.0
> Attachments: HIVE-18842.01.patch, HIVE-18842.01.patch,
> HIVE-18842.02.patch, HIVE-18842.03.patch, HIVE-18842.03.patch,
> HIVE-18842.04.patch, HIVE-18842.04.patch, HIVE-18842.patch
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> We should support defining a {{CLUSTERED ON/DISTRIBUTED ON+SORTED ON}}
> specification for materialized views.
> The syntax should be extended as follows:
> {code:sql}
> CREATE MATERIALIZED VIEW [IF NOT EXISTS] [db_name.]materialized_view_name
> [COMMENT materialized_view_comment]
> [PARTITIONED ON (col_name, ...)]
> [CLUSTERED ON (col_name, ...) | DISTRIBUTED ON (col_name, ...) SORTED ON
> (col_name, ...)] -- NEW!
> [
>[ROW FORMAT row_format]
>[STORED AS file_format]
> | STORED BY 'storage.handler.class.name' [WITH SERDEPROPERTIES (...)]
> ]
> [LOCATION hdfs_path]
> [TBLPROPERTIES (property_name=property_value, ...)]
> AS select_statement;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18842) CLUSTERED ON/DISTRIBUTED ON+SORTED ON support for materialized views
[
https://issues.apache.org/jira/browse/HIVE-18842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881435#comment-16881435
]
Hive QA commented on HIVE-18842:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12973999/HIVE-18842.04.patch
{color:green}SUCCESS:{color} +1 due to 7 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16639 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.hcatalog.listener.TestDbNotificationListener.sqlInsertPartition
(batchId=281)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17928/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17928/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17928/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12973999 - PreCommit-HIVE-Build
> CLUSTERED ON/DISTRIBUTED ON+SORTED ON support for materialized views
>
>
> Key: HIVE-18842
> URL: https://issues.apache.org/jira/browse/HIVE-18842
> Project: Hive
> Issue Type: New Feature
> Components: Materialized views
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available, todoc4.0
> Attachments: HIVE-18842.01.patch, HIVE-18842.01.patch,
> HIVE-18842.02.patch, HIVE-18842.03.patch, HIVE-18842.03.patch,
> HIVE-18842.04.patch, HIVE-18842.patch
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> We should support defining a {{CLUSTERED ON/DISTRIBUTED ON+SORTED ON}}
> specification for materialized views.
> The syntax should be extended as follows:
> {code:sql}
> CREATE MATERIALIZED VIEW [IF NOT EXISTS] [db_name.]materialized_view_name
> [COMMENT materialized_view_comment]
> [PARTITIONED ON (col_name, ...)]
> [CLUSTERED ON (col_name, ...) | DISTRIBUTED ON (col_name, ...) SORTED ON
> (col_name, ...)] -- NEW!
> [
>[ROW FORMAT row_format]
>[STORED AS file_format]
> | STORED BY 'storage.handler.class.name' [WITH SERDEPROPERTIES (...)]
> ]
> [LOCATION hdfs_path]
> [TBLPROPERTIES (property_name=property_value, ...)]
> AS select_statement;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21224) Upgrade tests JUnit3 to JUnit4
[ https://issues.apache.org/jira/browse/HIVE-21224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Pusztahazi updated HIVE-21224: Status: Patch Available (was: Open) > Upgrade tests JUnit3 to JUnit4 > -- > > Key: HIVE-21224 > URL: https://issues.apache.org/jira/browse/HIVE-21224 > Project: Hive > Issue Type: Improvement >Reporter: Bruno Pusztahazi >Assignee: Bruno Pusztahazi >Priority: Major > Attachments: HIVE-21224.1.patch, HIVE-21224.10.patch, > HIVE-21224.11.patch, HIVE-21224.12.patch, HIVE-21224.13.patch, > HIVE-21224.14.patch, HIVE-21224.15.patch, HIVE-21224.2.patch, > HIVE-21224.3.patch, HIVE-21224.4.patch, HIVE-21224.5.patch, > HIVE-21224.6.patch, HIVE-21224.7.patch, HIVE-21224.8.patch, HIVE-21224.9.patch > > > Old JUnit3 tests should be upgraded to JUnit4 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21224) Upgrade tests JUnit3 to JUnit4
[ https://issues.apache.org/jira/browse/HIVE-21224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Pusztahazi updated HIVE-21224: Status: Open (was: Patch Available) > Upgrade tests JUnit3 to JUnit4 > -- > > Key: HIVE-21224 > URL: https://issues.apache.org/jira/browse/HIVE-21224 > Project: Hive > Issue Type: Improvement >Reporter: Bruno Pusztahazi >Assignee: Bruno Pusztahazi >Priority: Major > Attachments: HIVE-21224.1.patch, HIVE-21224.10.patch, > HIVE-21224.11.patch, HIVE-21224.12.patch, HIVE-21224.13.patch, > HIVE-21224.14.patch, HIVE-21224.15.patch, HIVE-21224.2.patch, > HIVE-21224.3.patch, HIVE-21224.4.patch, HIVE-21224.5.patch, > HIVE-21224.6.patch, HIVE-21224.7.patch, HIVE-21224.8.patch, HIVE-21224.9.patch > > > Old JUnit3 tests should be upgraded to JUnit4 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21224) Upgrade tests JUnit3 to JUnit4
[ https://issues.apache.org/jira/browse/HIVE-21224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Pusztahazi updated HIVE-21224: Attachment: HIVE-21224.15.patch > Upgrade tests JUnit3 to JUnit4 > -- > > Key: HIVE-21224 > URL: https://issues.apache.org/jira/browse/HIVE-21224 > Project: Hive > Issue Type: Improvement >Reporter: Bruno Pusztahazi >Assignee: Bruno Pusztahazi >Priority: Major > Attachments: HIVE-21224.1.patch, HIVE-21224.10.patch, > HIVE-21224.11.patch, HIVE-21224.12.patch, HIVE-21224.13.patch, > HIVE-21224.14.patch, HIVE-21224.15.patch, HIVE-21224.2.patch, > HIVE-21224.3.patch, HIVE-21224.4.patch, HIVE-21224.5.patch, > HIVE-21224.6.patch, HIVE-21224.7.patch, HIVE-21224.8.patch, HIVE-21224.9.patch > > > Old JUnit3 tests should be upgraded to JUnit4 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18842) CLUSTERED ON/DISTRIBUTED ON+SORTED ON support for materialized views
[
https://issues.apache.org/jira/browse/HIVE-18842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881404#comment-16881404
]
Hive QA commented on HIVE-18842:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
34s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
8s{color} | {color:blue} ql in master has 2252 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
46s{color} | {color:red} ql: The patch generated 11 new + 702 unchanged - 2
fixed = 713 total (was 704) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
13s{color} | {color:red} ql generated 2 new + 2251 unchanged - 1 fixed = 2253
total (was 2252) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 4s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Dead store to LA29_5 in
org.apache.hadoop.hive.ql.parse.HiveLexer$DFA29.specialStateTransition(int,
IntStream) At
HiveLexer.java:org.apache.hadoop.hive.ql.parse.HiveLexer$DFA29.specialStateTransition(int,
IntStream) At HiveLexer.java:[line 12954] |
| | Should org.apache.hadoop.hive.ql.parse.HiveParser$DFA240 be a _static_
inner class? At HiveParser.java:inner class? At HiveParser.java:[lines
48890-48903] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17928/dev-support/hive-personality.sh
|
| git revision | master / 33d2995 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17928/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17928/yetus/new-findbugs-ql.html
|
| modules | C: common ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17928/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> CLUSTERED ON/DISTRIBUTED ON+SORTED ON support for materialized views
>
>
> Key: HIVE-18842
> URL: https://issues.apache.org/jira/browse/HIVE-18842
> Project: Hive
> Issue Type: New Feature
> Components: Materialized views
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho
[jira] [Commented] (HIVE-21901) Join queries across different datasources (Druid and JDBC StorageHandler)
[
https://issues.apache.org/jira/browse/HIVE-21901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881402#comment-16881402
]
slim bouguerra commented on HIVE-21901:
---
in theory this class
{code }DruidSelectQueryRecordReader.java {code}
should not be used but i see in practice still used, will look at this.
> Join queries across different datasources (Druid and JDBC StorageHandler)
> -
>
> Key: HIVE-21901
> URL: https://issues.apache.org/jira/browse/HIVE-21901
> Project: Hive
> Issue Type: Bug
> Components: Druid integration, StorageHandler
>Affects Versions: 3.1.1
>Reporter: Subramani Raju V
>Priority: Major
>
> We have a druid datasource and have external table created in hive for the
> same datasource.
> For example:
>
> {code:java}
> CREATE EXTERNAL TABLE druid_table_1
> STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler'
> TBLPROPERTIES ("druid.datasource" = "wikipedia");
> {code}
>
>
> We have another table in mysql database, which also has an external table
> created in hive in this fashion:
>
> {code:java}
> CREATE EXTERNAL TABLE sample_table_1
> (
> old_id int,
> city_name string,
> new_id int
> )
> STORED BY 'org.apache.hive.storage.jdbc.JdbcStorageHandler'
> TBLPROPERTIES (
> "hive.sql.database.type" = "MYSQL",
> "hive.sql.jdbc.driver" = "com.mysql.jdbc.Driver",
> "hive.sql.jdbc.url" = "jdbc:mysql://172.16.0.15:3307/test",
> "hive.sql.dbcp.username" = "hive_user",
> "hive.sql.dbcp.password" = "hive_pass",
> "hive.sql.table" = "city_mapping"
> );
> {code}
> So we are able to perform normal queries on the individual tables, but when
> we try to do join operation for both the above tables in this fashion:
>
>
> {code:java}
> SELECT *
> FROM druid_table_1 o
> JOIN sample_table_1 c
> ON (c.city_name = o.channel) limit 10;
> {code}
> Then we are getting the error as follows:
>
>
> {code:java}
> TaskAttempt 1 failed, info=[Error: Error while running task ( failure ) :
> attempt_1560945328057_0022_2_01_00_1:java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:80)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:419)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> ... 16 more
> Caused by: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderNextException(HiveIOExceptionHandlerChain.java:121)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderNextException(HiveIOExceptionHandlerUtil.java:77)
> at
> org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:365)
> at
> org.apache.hadoop.hive.ql.io.HiveRecordReader.doNext(HiveRecordReader.java:79)
> at
> org.apache.hadoop.hive.ql.io.HiveRecordReader.doNext(HiveRecordReader.java:33)
> at
>
[jira] [Commented] (HIVE-21861) ClassCastException during CTAS over external table using KafkaStorageHandler
[
https://issues.apache.org/jira/browse/HIVE-21861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881401#comment-16881401
]
Jesus Camacho Rodriguez commented on HIVE-21861:
[~Rajkumar Singh], [~kgyrtkirk], I have not had time to check the rest of code
in detail yet, sorry. However, I am not sure the problem is where we are
implementing this fix, i.e., though I may be wrong, my understanding is that we
should not end up with a LazyString in that part of the code. Have we compared
the code in {{VectorizedKafkaRecordReader}} with other vectorized record
readers?
> ClassCastException during CTAS over external table using KafkaStorageHandler
>
>
> Key: HIVE-21861
> URL: https://issues.apache.org/jira/browse/HIVE-21861
> Project: Hive
> Issue Type: Bug
> Components: kafka integration
>Affects Versions: 4.0.0
>Reporter: Justin Leet
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-21861.patch
>
>
> To reproduce, create a table similar to the following:
> {code}
> CREATE EXTERNAL TABLE
> (raw_value STRING)
> ROW FORMAT DELIMITED
> LINES TERMINATED BY '\n'
> STORED BY 'org.apache.hadoop.hive.kafka.KafkaStorageHandler'
> TBLPROPERTIES(
> "kafka.topic"="",
> "kafka.bootstrap.servers"="",
> "kafka.consumer.security.protocol"="PLAINTEXT",
> "kafka.serde.class"="org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe");
> {code}
> Note the SerDe isn't the default SerDe. Additionally, this error occurs when
> vectorization is enabled.
> Basic queries work fine:
> {code}
> SELECT * FROM LIMIT 1;
> {code}
> Doing a CTAS to bring it into a managed table fails:
> {code}
> CREATE TABLE AS
> SELECT * FROM ;
> {code}
> The exception is:
> {code}
> Caused by: java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.lazy.LazyString cannot be cast to
> org.apache.hadoop.io.TextCaused by: java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.lazy.LazyString cannot be cast to
> org.apache.hadoop.io.Text at
> org.apache.hadoop.hive.ql.exec.vector.VectorAssignRow.assignRowColumn(VectorAssignRow.java:471)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorAssignRow.assignRowColumn(VectorAssignRow.java:350)
> at
> org.apache.hadoop.hive.kafka.VectorizedKafkaRecordReader.readNextBatch(VectorizedKafkaRecordReader.java:159)
> at
> org.apache.hadoop.hive.kafka.VectorizedKafkaRecordReader.next(VectorizedKafkaRecordReader.java:113)
> at
> org.apache.hadoop.hive.kafka.VectorizedKafkaRecordReader.next(VectorizedKafkaRecordReader.java:47)
> at
> org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:360)
> ... 24 more
> {code}
> A workaround to this is to disable vectorization via:
> {code}
> set hive.vectorized.execution.enabled = false;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21838) Hive Metastore Translation: Add API call to tell client why table has limited access
[ https://issues.apache.org/jira/browse/HIVE-21838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-21838: - Status: Open (was: Patch Available) > Hive Metastore Translation: Add API call to tell client why table has limited > access > > > Key: HIVE-21838 > URL: https://issues.apache.org/jira/browse/HIVE-21838 > Project: Hive > Issue Type: Sub-task >Reporter: Yongzhi Chen >Assignee: Naveen Gangam >Priority: Major > Attachments: HIVE-21838.2.patch, HIVE-21838.3.patch, > HIVE-21838.4.patch, HIVE-21838.5.patch, HIVE-21838.6.patch, HIVE-21838.patch > > > When a table access type is Read-only or None, we need a way to tell clients > why. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21838) Hive Metastore Translation: Add API call to tell client why table has limited access
[ https://issues.apache.org/jira/browse/HIVE-21838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-21838: - Status: Patch Available (was: Open) > Hive Metastore Translation: Add API call to tell client why table has limited > access > > > Key: HIVE-21838 > URL: https://issues.apache.org/jira/browse/HIVE-21838 > Project: Hive > Issue Type: Sub-task >Reporter: Yongzhi Chen >Assignee: Naveen Gangam >Priority: Major > Attachments: HIVE-21838.2.patch, HIVE-21838.3.patch, > HIVE-21838.4.patch, HIVE-21838.5.patch, HIVE-21838.6.patch, HIVE-21838.patch > > > When a table access type is Read-only or None, we need a way to tell clients > why. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21838) Hive Metastore Translation: Add API call to tell client why table has limited access
[ https://issues.apache.org/jira/browse/HIVE-21838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-21838: - Attachment: HIVE-21838.6.patch > Hive Metastore Translation: Add API call to tell client why table has limited > access > > > Key: HIVE-21838 > URL: https://issues.apache.org/jira/browse/HIVE-21838 > Project: Hive > Issue Type: Sub-task >Reporter: Yongzhi Chen >Assignee: Naveen Gangam >Priority: Major > Attachments: HIVE-21838.2.patch, HIVE-21838.3.patch, > HIVE-21838.4.patch, HIVE-21838.5.patch, HIVE-21838.6.patch, HIVE-21838.patch > > > When a table access type is Read-only or None, we need a way to tell clients > why. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21637) Synchronized metastore cache
[
https://issues.apache.org/jira/browse/HIVE-21637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881368#comment-16881368
]
Hive QA commented on HIVE-21637:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
44s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
18s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
27s{color} | {color:blue} storage-api in master has 48 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
30s{color} | {color:blue} standalone-metastore/metastore-common in master has
31 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
8s{color} | {color:blue} standalone-metastore/metastore-server in master has
179 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
8s{color} | {color:blue} ql in master has 2252 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
29s{color} | {color:blue} beeline in master has 44 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} hcatalog/server-extensions in master has 3 extant
Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
25s{color} | {color:blue}
standalone-metastore/metastore-tools/metastore-benchmarks in master has 3
extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
52s{color} | {color:blue} itests/util in master has 44 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
31s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
11s{color} | {color:red} storage-api: The patch generated 1 new + 5 unchanged -
0 fixed = 6 total (was 5) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
15s{color} | {color:red} standalone-metastore/metastore-common: The patch
generated 9 new + 498 unchanged - 2 fixed = 507 total (was 500) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 164 new + 2193 unchanged - 65 fixed = 2357 total (was 2258) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
50s{color} | {color:red} ql: The patch generated 27 new + 1040 unchanged - 10
fixed = 1067 total (was 1050) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} standalone-metastore/metastore-tools/tools-common: The
patch generated 5 new + 31 unchanged - 0 fixed = 36 total (was 31) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
11s{color} | {color:red} itests/hcatalog-unit: The patch generated 2 new + 24
unchanged - 3 fixed = 26 total (was 27) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} itests/hive-unit: The patch generated 3 new + 163
unchanged - 1 fixed = 166 total (was 164) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} |
[jira] [Commented] (HIVE-21637) Synchronized metastore cache
[
https://issues.apache.org/jira/browse/HIVE-21637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881369#comment-16881369
]
Hive QA commented on HIVE-21637:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12974028/HIVE-21637.21.patch
{color:green}SUCCESS:{color} +1 due to 10 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 513 failed/errored test(s), 15789 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.org.apache.hadoop.hive.cli.TestAccumuloCliDriver
(batchId=279)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_insert_overwrite]
(batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_nullscan]
(batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_stats2] (batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_stats3] (batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_stats4] (batchId=67)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_subquery]
(batchId=43)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_table_stats]
(batchId=58)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[allow_change_col_type_par]
(batchId=78)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_partition_update_status]
(batchId=99)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_rename_partition_authorization]
(batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_update_status]
(batchId=87)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_update_status_disable_bitvector]
(batchId=87)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_1]
(batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_3]
(batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_4]
(batchId=70)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_6]
(batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_create_temp_table]
(batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_view_1]
(batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_view_2]
(batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_view_3]
(batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_view_4]
(batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_view_disable_cbo_1]
(batchId=78)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_view_disable_cbo_2]
(batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_view_disable_cbo_3]
(batchId=10)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_view_disable_cbo_4]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_4]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_7]
(batchId=70)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_quoting]
(batchId=98)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constprog3] (batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_transactional_full_acid]
(batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cte_mat_4] (batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[delete_tmp_table]
(batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[delete_where_partitioned]
(batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[delete_whole_partition]
(batchId=10)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[explain_locks]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[foldts] (batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_values_tmp_table]
(batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[lock1] (batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[lock2] (batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[lock3] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_10] (batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_9] (batchId=87)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_acid_no_masking]
(batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mm_all] (batchId=74)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mm_cttas] (batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mm_iow_temp] (batchId=11)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_wide_table]
(batchId=96)
[jira] [Commented] (HIVE-21923) Vectorized MapJoin may miss results when only the join key is selected
[ https://issues.apache.org/jira/browse/HIVE-21923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881354#comment-16881354 ] Zoltan Haindrich commented on HIVE-21923: - of course! opened rb: https://reviews.apache.org/r/71040/ > Vectorized MapJoin may miss results when only the join key is selected > -- > > Key: HIVE-21923 > URL: https://issues.apache.org/jira/browse/HIVE-21923 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21923.01.patch, HIVE-21923.02.patch, > HIVE-21923.03.patch, HIVE-21923.04.patch, HIVE-21923.05.patch, > HIVE-21923.06.patch > > > HIVE-21189 have introduced some resultset changes > in ql/src/test/results/clientpositive/llap/hybridgrace_hashjoin_2.q.out > https://github.com/apache/hive/commit/5799398450c17d06e8ef144ce835a8524f5abec9#diff-56b3ab96b6c90fdbebe2c4f84e8595afL500 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21923) Vectorized MapJoin may miss results when only the join key is selected
[ https://issues.apache.org/jira/browse/HIVE-21923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21923: Attachment: HIVE-21923.06.patch > Vectorized MapJoin may miss results when only the join key is selected > -- > > Key: HIVE-21923 > URL: https://issues.apache.org/jira/browse/HIVE-21923 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21923.01.patch, HIVE-21923.02.patch, > HIVE-21923.03.patch, HIVE-21923.04.patch, HIVE-21923.05.patch, > HIVE-21923.06.patch > > > HIVE-21189 have introduced some resultset changes > in ql/src/test/results/clientpositive/llap/hybridgrace_hashjoin_2.q.out > https://github.com/apache/hive/commit/5799398450c17d06e8ef144ce835a8524f5abec9#diff-56b3ab96b6c90fdbebe2c4f84e8595afL500 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21938) Add database and table filter options to PreUpgradeTool
[ https://issues.apache.org/jira/browse/HIVE-21938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21938: Resolution: Fixed Status: Resolved (was: Patch Available) pushed to master. Thank you [~kkasa]! > Add database and table filter options to PreUpgradeTool > --- > > Key: HIVE-21938 > URL: https://issues.apache.org/jira/browse/HIVE-21938 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.1.0 >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Blocker > Fix For: 4.0.0 > > Attachments: HIVE-21938.1.patch, HIVE-21938.2.patch, > HIVE-21938.3.patch, HIVE-21938.4.patch, HIVE-21938.5.patch, > HIVE-21938.6.patch, HIVE-21938.7.patch > > > By default pre upgrade tool scans all databases and tables in the warehouse. > Add database and table filter options to run the tool for a specific subset > of databases and tables only. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21933) Remove unused methods from Utilities
[ https://issues.apache.org/jira/browse/HIVE-21933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21933: Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) pushed to master. Thank you [~isuller]! > Remove unused methods from Utilities > > > Key: HIVE-21933 > URL: https://issues.apache.org/jira/browse/HIVE-21933 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Trivial > Fix For: 4.0.0 > > Attachments: HIVE-21933.1.patch > > > Over the years it seems org.apache.hadoop.hive.ql.exec.Utilities collected > many methods which are not used anymore. Removing them is the right thing to > do. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21547) Temp Tables: Use stORC format for temporary tables
[ https://issues.apache.org/jira/browse/HIVE-21547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21547: Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) pushed to master. Thank you [~kkasa]! > Temp Tables: Use stORC format for temporary tables > -- > > Key: HIVE-21547 > URL: https://issues.apache.org/jira/browse/HIVE-21547 > Project: Hive > Issue Type: Improvement > Components: ORC >Affects Versions: 4.0.0 >Reporter: Gopal V >Assignee: Krisztian Kasa >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21547.1.patch, HIVE-21547.2.patch, > HIVE-21547.3.patch > > > Using st(reaming)ORC > (hive.exec.orc.delta.streaming.optimizations.enabled=true) format has massive > performance advantages when creating data-sets which will not be stored for > long-term. > The format is compatible with ORC for vectorization and other features, while > being cheaper to write out to filesystem. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21967) Clean up CreateTableLikeOperation
[ https://issues.apache.org/jira/browse/HIVE-21967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21967: Resolution: Fixed Status: Resolved (was: Patch Available) pushed to master. Thank you [~mgergely]! > Clean up CreateTableLikeOperation > - > > Key: HIVE-21967 > URL: https://issues.apache.org/jira/browse/HIVE-21967 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-21967.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > > CreateTableLikeOperation has two sub types, creating from view or table. A > lot of their codes is common, they should be reused. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21861) ClassCastException during CTAS over external table using KafkaStorageHandler
[
https://issues.apache.org/jira/browse/HIVE-21861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881325#comment-16881325
]
Zoltan Haindrich commented on HIVE-21861:
-
could we add the described case as a qtest? it would be great to have a test
for this
> ClassCastException during CTAS over external table using KafkaStorageHandler
>
>
> Key: HIVE-21861
> URL: https://issues.apache.org/jira/browse/HIVE-21861
> Project: Hive
> Issue Type: Bug
> Components: kafka integration
>Affects Versions: 4.0.0
>Reporter: Justin Leet
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-21861.patch
>
>
> To reproduce, create a table similar to the following:
> {code}
> CREATE EXTERNAL TABLE
> (raw_value STRING)
> ROW FORMAT DELIMITED
> LINES TERMINATED BY '\n'
> STORED BY 'org.apache.hadoop.hive.kafka.KafkaStorageHandler'
> TBLPROPERTIES(
> "kafka.topic"="",
> "kafka.bootstrap.servers"="",
> "kafka.consumer.security.protocol"="PLAINTEXT",
> "kafka.serde.class"="org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe");
> {code}
> Note the SerDe isn't the default SerDe. Additionally, this error occurs when
> vectorization is enabled.
> Basic queries work fine:
> {code}
> SELECT * FROM LIMIT 1;
> {code}
> Doing a CTAS to bring it into a managed table fails:
> {code}
> CREATE TABLE AS
> SELECT * FROM ;
> {code}
> The exception is:
> {code}
> Caused by: java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.lazy.LazyString cannot be cast to
> org.apache.hadoop.io.TextCaused by: java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.lazy.LazyString cannot be cast to
> org.apache.hadoop.io.Text at
> org.apache.hadoop.hive.ql.exec.vector.VectorAssignRow.assignRowColumn(VectorAssignRow.java:471)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorAssignRow.assignRowColumn(VectorAssignRow.java:350)
> at
> org.apache.hadoop.hive.kafka.VectorizedKafkaRecordReader.readNextBatch(VectorizedKafkaRecordReader.java:159)
> at
> org.apache.hadoop.hive.kafka.VectorizedKafkaRecordReader.next(VectorizedKafkaRecordReader.java:113)
> at
> org.apache.hadoop.hive.kafka.VectorizedKafkaRecordReader.next(VectorizedKafkaRecordReader.java:47)
> at
> org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:360)
> ... 24 more
> {code}
> A workaround to this is to disable vectorization via:
> {code}
> set hive.vectorized.execution.enabled = false;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-21967) Clean up CreateTableLikeOperation
[ https://issues.apache.org/jira/browse/HIVE-21967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881319#comment-16881319 ] Zoltan Haindrich edited comment on HIVE-21967 at 7/9/19 3:29 PM: - +1 was (Author: kgyrtkirk): +! > Clean up CreateTableLikeOperation > - > > Key: HIVE-21967 > URL: https://issues.apache.org/jira/browse/HIVE-21967 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-21967.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > > CreateTableLikeOperation has two sub types, creating from view or table. A > lot of their codes is common, they should be reused. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21967) Clean up CreateTableLikeOperation
[ https://issues.apache.org/jira/browse/HIVE-21967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881319#comment-16881319 ] Zoltan Haindrich commented on HIVE-21967: - +! > Clean up CreateTableLikeOperation > - > > Key: HIVE-21967 > URL: https://issues.apache.org/jira/browse/HIVE-21967 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-21967.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > > CreateTableLikeOperation has two sub types, creating from view or table. A > lot of their codes is common, they should be reused. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-21962) Replacing ArrayList params with List in and around PlanUtils and MapWork
[
https://issues.apache.org/jira/browse/HIVE-21962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881310#comment-16881310
]
Zoltan Haindrich edited comment on HIVE-21962 at 7/9/19 3:25 PM:
-
+1 pending tests
we have an incremental compilation issue around SA ; mostly because of "?
extends Serializable"
can be reproduced with:
{code}
git clean -dfx
mvn install -pl ql -am -DskipTests
touch `find . -name Sema*A*java` `find . -name Task*Factory.java`
mvn install -pl ql -DskipTests
{code}
I've opened: HIVE-21975
was (Author: kgyrtkirk):
+1 pending tests
we also have an incremental compilation issue around SA ; mostly because of "?
extends Serializable"
can be reproduced with:
{code}
git clean -dfx
mvn install -pl ql -am -DskipTests
touch `find . -name Sema*A*java` `find . -name Task*Factory.java`
mvn install -pl ql -DskipTests
{code}
> Replacing ArrayList params with List in and around PlanUtils and MapWork
>
>
> Key: HIVE-21962
> URL: https://issues.apache.org/jira/browse/HIVE-21962
> Project: Hive
> Issue Type: Improvement
>Reporter: Ivan Suller
>Assignee: Ivan Suller
>Priority: Minor
> Attachments: HIVE-21962.1.patch, HIVE-21962.1.patch
>
>
> Using the implementing class is usually a bad practice. OO suggest to use the
> least restrictive interface instead. ArrayList is used in many-many methods
> as a parameter - this is just a tiny part of this work.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21923) Vectorized MapJoin may miss results when only the join key is selected
[ https://issues.apache.org/jira/browse/HIVE-21923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881312#comment-16881312 ] Jesus Camacho Rodriguez commented on HIVE-21923: [~kgyrtkirk], would you mind to create a PR / rb link? Thanks > Vectorized MapJoin may miss results when only the join key is selected > -- > > Key: HIVE-21923 > URL: https://issues.apache.org/jira/browse/HIVE-21923 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21923.01.patch, HIVE-21923.02.patch, > HIVE-21923.03.patch, HIVE-21923.04.patch, HIVE-21923.05.patch > > > HIVE-21189 have introduced some resultset changes > in ql/src/test/results/clientpositive/llap/hybridgrace_hashjoin_2.q.out > https://github.com/apache/hive/commit/5799398450c17d06e8ef144ce835a8524f5abec9#diff-56b3ab96b6c90fdbebe2c4f84e8595afL500 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21962) Replacing ArrayList params with List in and around PlanUtils and MapWork
[
https://issues.apache.org/jira/browse/HIVE-21962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881310#comment-16881310
]
Zoltan Haindrich commented on HIVE-21962:
-
+1 pending tests
we also have an incremental compilation issue around SA ; mostly because of "?
extends Serializable"
can be reproduced with:
{code}
git clean -dfx
mvn install -pl ql -am -DskipTests
touch `find . -name Sema*A*java` `find . -name Task*Factory.java`
mvn install -pl ql -DskipTests
{code}
> Replacing ArrayList params with List in and around PlanUtils and MapWork
>
>
> Key: HIVE-21962
> URL: https://issues.apache.org/jira/browse/HIVE-21962
> Project: Hive
> Issue Type: Improvement
>Reporter: Ivan Suller
>Assignee: Ivan Suller
>Priority: Minor
> Attachments: HIVE-21962.1.patch, HIVE-21962.1.patch
>
>
> Using the implementing class is usually a bad practice. OO suggest to use the
> least restrictive interface instead. ArrayList is used in many-many methods
> as a parameter - this is just a tiny part of this work.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21959) Clean up Concatenate and Msck DDL commands
[ https://issues.apache.org/jira/browse/HIVE-21959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-21959: --- Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master, thanks [~mgergely] and [~kgyrtkirk]. > Clean up Concatenate and Msck DDL commands > -- > > Key: HIVE-21959 > URL: https://issues.apache.org/jira/browse/HIVE-21959 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-21959.01.patch, HIVE-21959.02.patch, > HIVE-21959.03.patch, HIVE-21959.04.patch > > Time Spent: 2h > Remaining Estimate: 0h > > Concatenate and Msck DDL use basic data structure implementations instead of > their interfaces for variable declaration, like ArrayList instead of List, > LinkedHashMap instead of Map, etc. This is due to a lot of similar issues in > the code, which needs to be cleaned. > Concatenate also had a non-immutable Desc class, that needs to be transformed > into an immutable one. Concatenate operation code should be cut into smaller > functions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (HIVE-21928) Fix for statistics annotation in nested AND expressions
[
https://issues.apache.org/jira/browse/HIVE-21928?focusedWorklogId=274128=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-274128
]
ASF GitHub Bot logged work on HIVE-21928:
-
Author: ASF GitHub Bot
Created on: 09/Jul/19 15:21
Start Date: 09/Jul/19 15:21
Worklog Time Spent: 10m
Work Description: asfgit commented on pull request #700: HIVE-21928
URL: https://github.com/apache/hive/pull/700
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 274128)
Time Spent: 20m (was: 10m)
> Fix for statistics annotation in nested AND expressions
> ---
>
> Key: HIVE-21928
> URL: https://issues.apache.org/jira/browse/HIVE-21928
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21928.01.patch, HIVE-21928.01.patch,
> HIVE-21928.02.patch, HIVE-21928.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Discovered while working on HIVE-21867. Having predicates with nested AND
> expressions may result in different stats, even if predicates are basically
> similar (from stats estimation standpoint).
> For instance, stats for {{AND(x=5, true, true)}} are different from {{x=5}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21928) Fix for statistics annotation in nested AND expressions
[
https://issues.apache.org/jira/browse/HIVE-21928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-21928:
---
Resolution: Fixed
Fix Version/s: 4.0.0
Status: Resolved (was: Patch Available)
[~kgyrtkirk], no problem, thanks.
Pushed to master.
> Fix for statistics annotation in nested AND expressions
> ---
>
> Key: HIVE-21928
> URL: https://issues.apache.org/jira/browse/HIVE-21928
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21928.01.patch, HIVE-21928.01.patch,
> HIVE-21928.02.patch, HIVE-21928.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Discovered while working on HIVE-21867. Having predicates with nested AND
> expressions may result in different stats, even if predicates are basically
> similar (from stats estimation standpoint).
> For instance, stats for {{AND(x=5, true, true)}} are different from {{x=5}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (HIVE-21959) Clean up Concatenate and Msck DDL commands
[ https://issues.apache.org/jira/browse/HIVE-21959?focusedWorklogId=274129=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-274129 ] ASF GitHub Bot logged work on HIVE-21959: - Author: ASF GitHub Bot Created on: 09/Jul/19 15:21 Start Date: 09/Jul/19 15:21 Worklog Time Spent: 10m Work Description: asfgit commented on pull request #703: HIVE-21959 Clean up Concatenate and Msck DDL commands URL: https://github.com/apache/hive/pull/703 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 274129) Time Spent: 2h (was: 1h 50m) > Clean up Concatenate and Msck DDL commands > -- > > Key: HIVE-21959 > URL: https://issues.apache.org/jira/browse/HIVE-21959 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-21959.01.patch, HIVE-21959.02.patch, > HIVE-21959.03.patch, HIVE-21959.04.patch > > Time Spent: 2h > Remaining Estimate: 0h > > Concatenate and Msck DDL use basic data structure implementations instead of > their interfaces for variable declaration, like ArrayList instead of List, > LinkedHashMap instead of Map, etc. This is due to a lot of similar issues in > the code, which needs to be cleaned. > Concatenate also had a non-immutable Desc class, that needs to be transformed > into an immutable one. Concatenate operation code should be cut into smaller > functions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21923) Vectorized MapJoin may miss results when only the join key is selected
[ https://issues.apache.org/jira/browse/HIVE-21923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881299#comment-16881299 ] Zoltan Haindrich commented on HIVE-21923: - [~jcamachorodriguez] Could you please take a look? > Vectorized MapJoin may miss results when only the join key is selected > -- > > Key: HIVE-21923 > URL: https://issues.apache.org/jira/browse/HIVE-21923 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21923.01.patch, HIVE-21923.02.patch, > HIVE-21923.03.patch, HIVE-21923.04.patch, HIVE-21923.05.patch > > > HIVE-21189 have introduced some resultset changes > in ql/src/test/results/clientpositive/llap/hybridgrace_hashjoin_2.q.out > https://github.com/apache/hive/commit/5799398450c17d06e8ef144ce835a8524f5abec9#diff-56b3ab96b6c90fdbebe2c4f84e8595afL500 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally
[
https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881291#comment-16881291
]
Hive QA commented on HIVE-21225:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12973993/HIVE-21225.9.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 45 failed/errored test(s), 16630 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_nullscan]
(batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_stats5] (batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_4]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mm_all] (batchId=74)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats_nonpart]
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats_part2] (batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats_part] (batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats_sizebug]
(batchId=89)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[acid_bucket_pruning]
(batchId=155)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[mm_all]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mm_exim]
(batchId=186)
org.apache.hadoop.hive.ql.TestTxnCommands.testMmExim (batchId=341)
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1
(batchId=322)
org.apache.hadoop.hive.ql.TestTxnCommands2.testOriginalFileReaderWhenNonAcidConvertedToAcid
(batchId=322)
org.apache.hadoop.hive.ql.TestTxnCommands2.updateDeletePartitioned (batchId=322)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testNonAcidToAcidConversion1
(batchId=336)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testOriginalFileReaderWhenNonAcidConvertedToAcid
(batchId=336)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.updateDeletePartitioned
(batchId=336)
org.apache.hadoop.hive.ql.TestTxnCommandsWithSplitUpdateAndVectorization.testMmExim
(batchId=324)
org.apache.hadoop.hive.ql.TestTxnExIm.testImport (batchId=322)
org.apache.hadoop.hive.ql.TestTxnExIm.testImportNoTarget (batchId=322)
org.apache.hadoop.hive.ql.TestTxnExIm.testMM (batchId=322)
org.apache.hadoop.hive.ql.TestTxnExIm.testMMCreate (batchId=322)
org.apache.hadoop.hive.ql.TestTxnLoadData.loadData (batchId=298)
org.apache.hadoop.hive.ql.TestTxnLoadData.loadDataNonAcid2AcidConversion
(batchId=298)
org.apache.hadoop.hive.ql.TestTxnLoadData.loadDataUpdate (batchId=298)
org.apache.hadoop.hive.ql.TestTxnLoadData.testMultiStatement (batchId=298)
org.apache.hadoop.hive.ql.TestTxnNoBuckets.testCompactStatsGather (batchId=322)
org.apache.hadoop.hive.ql.TestTxnNoBuckets.testEmptyCompactionResult
(batchId=322)
org.apache.hadoop.hive.ql.TestTxnNoBuckets.testToAcidConversionMultiBucket
(batchId=322)
org.apache.hadoop.hive.ql.TestTxnNoBucketsVectorized.testCompactStatsGather
(batchId=324)
org.apache.hadoop.hive.ql.TestTxnNoBucketsVectorized.testEmptyCompactionResult
(batchId=324)
org.apache.hadoop.hive.ql.TestTxnNoBucketsVectorized.testToAcidConversionMultiBucket
(batchId=324)
org.apache.hadoop.hive.ql.io.TestAcidUtils.testObsoleteOriginals (batchId=310)
org.apache.hadoop.hive.ql.parse.TestReplAcidTablesWithJsonMessage.testDumpAcidTableWithPartitionDirMissing
(batchId=260)
org.apache.hadoop.hive.ql.parse.TestReplAcidTablesWithJsonMessage.testDumpAcidTableWithTableDirMissing
(batchId=260)
org.apache.hadoop.hive.ql.parse.TestReplAcidTablesWithJsonMessage.testMultiDBTxn
(batchId=260)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testDumpAcidTableWithPartitionDirMissing
(batchId=262)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testDumpAcidTableWithTableDirMissing
(batchId=262)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testMultiDBTxn
(batchId=262)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosIncrementalLoadAcidTables.testAcidTableIncrementalReplication
(batchId=263)
org.apache.hadoop.hive.ql.txn.compactor.TestCompactor.testStatsAfterCompactionPartTbl
(batchId=245)
org.apache.hadoop.hive.ql.txn.compactor.TestCompactor.testStatsAfterCompactionPartTblNew
(batchId=245)
org.apache.hadoop.hive.ql.util.TestUpgradeTool.testPostUpgrade (batchId=298)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17926/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17926/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17926/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing
[jira] [Commented] (HIVE-21225) ACID: getAcidState() should cache a recursive dir listing locally
[
https://issues.apache.org/jira/browse/HIVE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881253#comment-16881253
]
Hive QA commented on HIVE-21225:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
48s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
10s{color} | {color:blue} ql in master has 2252 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
45s{color} | {color:red} ql: The patch generated 21 new + 903 unchanged - 1
fixed = 924 total (was 904) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 27 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
11s{color} | {color:red} ql generated 1 new + 2252 unchanged - 0 fixed = 2253
total (was 2252) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 21s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Unread field:AcidUtils.java:[line 1409] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17926/dev-support/hive-personality.sh
|
| git revision | master / f629501 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17926/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17926/yetus/whitespace-eol.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17926/yetus/new-findbugs-ql.html
|
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17926/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ACID: getAcidState() should cache a recursive dir listing locally
> -
>
> Key: HIVE-21225
> URL: https://issues.apache.org/jira/browse/HIVE-21225
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Reporter: Gopal V
>Assignee: Vaibhav Gumashta
>Priority: Major
> Attachments: HIVE-21225.1.patch, HIVE-21225.2.patch,
> HIVE-21225.3.patch, HIVE-21225.4.patch, HIVE-21225.4.patch,
> HIVE-21225.5.patch, HIVE-21225.6.patch, HIVE-21225.7.patch,
> HIVE-21225.7.patch,
