[jira] [Updated] (HIVE-22384) Hive MetaStore CachedStore Funcion Cache Dev
[ https://issues.apache.org/jira/browse/HIVE-22384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-22384: -- Fix Version/s: (was: 2.3.7) > Hive MetaStore CachedStore Funcion Cache Dev > > > Key: HIVE-22384 > URL: https://issues.apache.org/jira/browse/HIVE-22384 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Affects Versions: 2.3.7, 3.1.3 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: cachedstore > Fix For: 3.1.3 > > Attachments: HIVE-22384.1.patch, HIVE-22384.2.patch, > HIVE-22384.3.patch, HIVE-22384.4.patch, HIVE-22384.5.patch, > HIVE-22384.6.patch, HIVE-22384.7.patch, HIVE-22384.8.patch, HIVE-22384.9.patch > > > CachedStore is a very good new feature to alleviate the pressure on the > background database. > The Function Cache is a function that the CachedStore feature has not been > completed. > This patch completes the Function Cache. > The functional points involved are as follows: > 1) Function Cache prewarm > 2) Function Cache Update > 3) CreateFunction、AlterFunction、DropFunction、GetFunction . > 4) NotificationEvents (CREATE_FUNCTION_EVENT、DROP_FUNCTION_EVENT) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22372) Hive MetaStore Cachedstore Feature
[ https://issues.apache.org/jira/browse/HIVE-22372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-22372: -- Fix Version/s: (was: 2.3.7) > Hive MetaStore Cachedstore Feature > -- > > Key: HIVE-22372 > URL: https://issues.apache.org/jira/browse/HIVE-22372 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 2.3.5 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > > hive 3.x has cachedstore, this feature can reduce database pressure，hope this > feature in 2.3.X version -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22579) ACID v1: covered delta-only splits (without base) should be marked as covered (branch-2)
[
https://issues.apache.org/jira/browse/HIVE-22579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates updated HIVE-22579:
--
Fix Version/s: (was: 2.3.7)
2.4.0
> ACID v1: covered delta-only splits (without base) should be marked as covered
> (branch-2)
>
>
> Key: HIVE-22579
> URL: https://issues.apache.org/jira/browse/HIVE-22579
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Fix For: 2.4.0
>
> Attachments: HIVE-22579.01.branch-2.patch,
> HIVE-22579.01.branch-2.patch
>
>
> There is a scenario when different SplitGenerator instances try to cover the
> delta-only buckets (having no base file) more than once, so there could be
> multiple OrcSplit instances generated for the same delta file, causing more
> tasks to read the same delta file more than once, causing duplicate records
> in a simple select star query.
> File structure for a 256 bucket table
> {code}
> drwxrwxrwx - hive hadoop 0 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/base_013
> -rw-r--r-- 3 hive hadoop353 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/base_013/bucket_00012
> -rw-r--r-- 3 hive hadoop 1642 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/base_013/bucket_00140
> drwxrwxrwx - hive hadoop 0 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/delta_014_014_
> -rwxrwxrwx 3 hive hadoop348 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/delta_014_014_/bucket_00012
> -rwxrwxrwx 3 hive hadoop 1635 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/delta_014_014_/bucket_00140
> drwxrwxrwx - hive hadoop 0 2019-11-29 16:04
> /apps/hive/warehouse/naresh.db/test1/delta_015_015_
> -rwxrwxrwx 3 hive hadoop348 2019-11-29 16:04
> /apps/hive/warehouse/naresh.db/test1/delta_015_015_/bucket_00012
> -rwxrwxrwx 3 hive hadoop 1808 2019-11-29 16:04
> /apps/hive/warehouse/naresh.db/test1/delta_015_015_/bucket_00140
> drwxrwxrwx - hive hadoop 0 2019-11-29 16:06
> /apps/hive/warehouse/naresh.db/test1/delta_016_016_
> -rwxrwxrwx 3 hive hadoop348 2019-11-29 16:06
> /apps/hive/warehouse/naresh.db/test1/delta_016_016_/bucket_00043
> -rwxrwxrwx 3 hive hadoop 1633 2019-11-29 16:06
> /apps/hive/warehouse/naresh.db/test1/delta_016_016_/bucket_00171
> {code}
> in this case, when bucket_00171 file has a record, and there is no base file
> for that, a select (*) with ETL split strategy can generate 2 splits for the
> same delta bucket...
> the scenario of the issue:
> 1. ETLSplitStrategy contains a [covered[]
> array|https://github.com/apache/hive/blob/298f749ec7be04abb797fb119f3f0b923c8a1b27/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L763]
> which is [shared between the SplitInfo
> instances|https://github.com/apache/hive/blob/298f749ec7be04abb797fb119f3f0b923c8a1b27/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L824]
> to be created
> 2. a SplitInfo instance is created for [every base file (2 in this
> case)|https://github.com/apache/hive/blob/298f749ec7be04abb797fb119f3f0b923c8a1b27/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L809]
> 3. for every SplitInfo, [a SplitGenerator is
> created|https://github.com/apache/hive/blob/298f749ec7be04abb797fb119f3f0b923c8a1b27/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L925-L926],
> and in the constructor, [parent's getSplit is
> called|https://github.com/apache/hive/blob/298f749ec7be04abb797fb119f3f0b923c8a1b27/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L1251],
> which tries to take care of the deltas
> I'm not sure at the moment what's the intention of this, but this way,
> duplicated delta split can be generated, which can cause duplicated read
> later (note that both tasks read the same delta file: bucket_00171)
> {code}
> 2019-12-01T16:24:53,669 INFO [TezTR-127843_16_30_0_171_0
> (1575040127843_0016_30_00_000171_0)] orc.ReaderImpl: Reading ORC rows from
> hdfs://c3351-node2.squadron.support.hortonworks.com:8020/apps/hive/warehouse/naresh.db/test1/delta_016_016_/bucket_00171
> with {include: [true, true, true, true, true, true, true, true, true, true,
> true, true], offset: 0, length: 9223372036854775807, schema:
> struct}
> 2019-12-01T16:24:53,672 INFO [TezTR-127843_16_30_0_171_0
> (1575040127843_0016_30_00_000171_0)] lib.MRReaderMapred: Processing split:
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit
> [hdfs://c3351-node2.squadron.support.hor
[jira] [Updated] (HIVE-22224) Support Parquet-Avro Timestamp Type
[
https://issues.apache.org/jira/browse/HIVE-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates updated HIVE-4:
--
Fix Version/s: (was: 2.3.7)
> Support Parquet-Avro Timestamp Type
> ---
>
> Key: HIVE-4
> URL: https://issues.apache.org/jira/browse/HIVE-4
> Project: Hive
> Issue Type: Bug
> Components: Database/Schema
>Affects Versions: 2.3.5, 2.3.6
>Reporter: cdmikechen
>Assignee: cdmikechen
>Priority: Major
> Labels: parquet, pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When user create an external table and import a parquet-avro data with 1.8.2
> version which supported logical_type in Hive2.3 or before version, Hive can
> not read timestamp type column data correctly.
> Hive will read it as LongWritable which it actually stores as
> long(logical_type=timestamp-millis).So we may add some codes in
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableTimestampObjectInspector.java
> to let Hive cast long type to timestamp type.
> Some code like below:
>
> public Timestamp getPrimitiveJavaObject(Object o) {
> if (o instanceof LongWritable) {
> return new Timestamp(((LongWritable) o).get());
> }
> return o == null ? null : ((TimestampWritable) o).getTimestamp();
> }
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22410) CachedStore Prewarm Failure NPE
[ https://issues.apache.org/jira/browse/HIVE-22410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-22410: -- Fix Version/s: (was: 2.3.7) > CachedStore Prewarm Failure NPE > --- > > Key: HIVE-22410 > URL: https://issues.apache.org/jira/browse/HIVE-22410 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Affects Versions: 3.1.2, 2.3.7 > Environment: JDK 1.7.0_40 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: cachedstore > Fix For: 3.1.3 > > Attachments: HIVE-22410.1.patch, image-2019-11-01-11-14-42-114.png > > > java.lang.NullPointerException > at > org.apache.hadoop.hive.metastore.utils.IncrementalObjectSizeEstimator$ObjectEstimator.estimateArrayElements(IncrementalObjectSizeEstimator.java:501) > ~[hive-metastore-2.3.5.jar:2.3.5] > at > org.apache.hadoop.hive.metastore.utils.IncrementalObjectSizeEstimator$ObjectEstimator.estimate(IncrementalObjectSizeEstimator.java:443) > ~[hive-metastore-2.3.5.jar:2.3.5] > at > org.apache.hadoop.hive.metastore.utils.IncrementalObjectSizeEstimator$ObjectEstimator.estimate(IncrementalObjectSizeEstimator.java:433) > ~[hive-metastore-2.3.5.jar:2.3.5] > at > org.apache.hadoop.hive.metastore.utils.IncrementalObjectSizeEstimator$ObjectEstimator.estimate(IncrementalObjectSizeEstimator.java:369) > ~[hive-metastore-2.3.5.jar:2.3.5] > at > org.apache.hadoop.hive.metastore.cached.SharedCache.populateTableInCache(SharedCache.java:1028) > ~[hive-metastore-2.3.5.jar:2.3.5] > at > org.apache.hadoop.hive.metastore.cached.CachedStore.prewarm(CachedStore.java:314) > ~[hive-metastore-2.3.5.jar:2.3.5] > at > org.apache.hadoop.hive.metastore.cached.CachedStore$CacheUpdateMasterWork.run(CachedStore.java:492) > [hive-metastore-2.3.5.jar:2.3.5] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > [?:1.7.0_40] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > [?:1.7.0_40] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > [?:1.7.0_40] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [?:1.7.0_40] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [?:1.7.0_40] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [?:1.7.0_40] > at java.lang.Thread.run(Thread.java:724) [?:1.7.0_40] > *reason:* > Methods estimateArrayElements throw NPE > I found that the last class does not appear in the parent (HashMap, > Object Estimator> ), and the parent does not contain the information of the > ConcurrentHashMap Class. > The solution is to initialize the class information before using it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22384) Hive MetaStore CachedStore Funcion Cache Dev
[ https://issues.apache.org/jira/browse/HIVE-22384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-22384: -- Target Version/s: 3.1.3 (was: 2.3.7, 3.1.3) > Hive MetaStore CachedStore Funcion Cache Dev > > > Key: HIVE-22384 > URL: https://issues.apache.org/jira/browse/HIVE-22384 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Affects Versions: 2.3.7, 3.1.3 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: cachedstore > Fix For: 2.3.7, 3.1.3 > > Attachments: HIVE-22384.1.patch, HIVE-22384.2.patch, > HIVE-22384.3.patch, HIVE-22384.4.patch, HIVE-22384.5.patch, > HIVE-22384.6.patch, HIVE-22384.7.patch, HIVE-22384.8.patch, HIVE-22384.9.patch > > > CachedStore is a very good new feature to alleviate the pressure on the > background database. > The Function Cache is a function that the CachedStore feature has not been > completed. > This patch completes the Function Cache. > The functional points involved are as follows: > 1) Function Cache prewarm > 2) Function Cache Update > 3) CreateFunction、AlterFunction、DropFunction、GetFunction . > 4) NotificationEvents (CREATE_FUNCTION_EVENT、DROP_FUNCTION_EVENT) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22413) Avoid dirty read when reading the ACID table while compaction is running
[ https://issues.apache.org/jira/browse/HIVE-22413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-22413: -- Target Version/s: (was: 2.3.7) > Avoid dirty read when reading the ACID table while compaction is running > > > Key: HIVE-22413 > URL: https://issues.apache.org/jira/browse/HIVE-22413 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Hocheol Park >Priority: Major > Attachments: HIVE-22413.1.patch > > > There is a problem that dirty read occurs when reading the ACID table while > base or delta directories are being created by the compactor. Especially it > is highly likely to occur in the S3 storage because the “move” logic of S3 is > “copy and delete”, and it takes a long time to copy if the size of files are > large or bucketing count is large. > So here’s the logic to avoid this problem. If “_tmp” prefixed directories are > existed in the partition directory on the process of listing the child > directories when reading the ACID table, compare the names of the directory > in the “_tmp” one and skip it in case of the same. Then it will read the > files before merging, no difference on the results. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22839) Fix build failure of branch-2 against hbase-1.5
[
https://issues.apache.org/jira/browse/HIVE-22839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates updated HIVE-22839:
--
Target Version/s: (was: 2.3.7)
> Fix build failure of branch-2 against hbase-1.5
> ---
>
> Key: HIVE-22839
> URL: https://issues.apache.org/jira/browse/HIVE-22839
> Project: Hive
> Issue Type: Bug
> Components: Test
>Affects Versions: 2.3.6
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
> Attachments: HIVE-22839.001.branch-2.patch
>
>
> branch-2 build with {{-Dhbase.version=1.5.0}} failed due to the change of
> org.apache.hadoop.hbase.client.ResultScanner introduced in hbase-1.4.0.
>
> [https://archive.apache.org/dist/hbase/1.4.0/hbase-1.3.1-1.4.0RC1_compatibility_report.html]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23086) Two tests fail on branch-2.3
[
https://issues.apache.org/jira/browse/HIVE-23086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates updated HIVE-23086:
--
Fix Version/s: 2.3.7
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Two tests fail on branch-2.3
>
>
> Key: HIVE-23086
> URL: https://issues.apache.org/jira/browse/HIVE-23086
> Project: Hive
> Issue Type: Bug
> Components: Test
>Affects Versions: 2.3.6
>Reporter: Alan Gates
>Assignee: Alan Gates
>Priority: Major
> Fix For: 2.3.7
>
> Attachments: HIVE-23086.patch
>
>
> TestPerfCli query88.q and TestMiniLlapLocalCliDriver union_fast_stats.q fail
> on the 2.3 branch.
> The TestMiniLlapLocalCliDriver failure is fixed in HIVE-14977 where
> union_fast_stats.q is removed from the list of tests run with
> TestMiniLlapLocalCliDriver.
> The TestPerfCli failure is fixed HIVE-16602 where one line is added to
> query88.q to allow cartesian products:
> {{set hive.strict.checks.cartesian.product=false;}}
> I propose to backport these two changes to branch-2.3
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23086) Two tests fail on branch-2.3
[
https://issues.apache.org/jira/browse/HIVE-23086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072301#comment-17072301
]
Alan Gates commented on HIVE-23086:
---
Committed to branch-2.3. Thanks Jésus for the review.
> Two tests fail on branch-2.3
>
>
> Key: HIVE-23086
> URL: https://issues.apache.org/jira/browse/HIVE-23086
> Project: Hive
> Issue Type: Bug
> Components: Test
>Affects Versions: 2.3.6
>Reporter: Alan Gates
>Assignee: Alan Gates
>Priority: Major
> Attachments: HIVE-23086.patch
>
>
> TestPerfCli query88.q and TestMiniLlapLocalCliDriver union_fast_stats.q fail
> on the 2.3 branch.
> The TestMiniLlapLocalCliDriver failure is fixed in HIVE-14977 where
> union_fast_stats.q is removed from the list of tests run with
> TestMiniLlapLocalCliDriver.
> The TestPerfCli failure is fixed HIVE-16602 where one line is added to
> query88.q to allow cartesian products:
> {{set hive.strict.checks.cartesian.product=false;}}
> I propose to backport these two changes to branch-2.3
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23086) Two tests fail on branch-2.3
[
https://issues.apache.org/jira/browse/HIVE-23086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates updated HIVE-23086:
--
Attachment: HIVE-23086.patch
> Two tests fail on branch-2.3
>
>
> Key: HIVE-23086
> URL: https://issues.apache.org/jira/browse/HIVE-23086
> Project: Hive
> Issue Type: Bug
> Components: Test
>Affects Versions: 2.3.6
>Reporter: Alan Gates
>Assignee: Alan Gates
>Priority: Major
> Attachments: HIVE-23086.patch
>
>
> TestPerfCli query88.q and TestMiniLlapLocalCliDriver union_fast_stats.q fail
> on the 2.3 branch.
> The TestMiniLlapLocalCliDriver failure is fixed in HIVE-14977 where
> union_fast_stats.q is removed from the list of tests run with
> TestMiniLlapLocalCliDriver.
> The TestPerfCli failure is fixed HIVE-16602 where one line is added to
> query88.q to allow cartesian products:
> {{set hive.strict.checks.cartesian.product=false;}}
> I propose to backport these two changes to branch-2.3
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23086) Two tests fail on branch-2.3
[
https://issues.apache.org/jira/browse/HIVE-23086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates updated HIVE-23086:
--
Status: Patch Available (was: Open)
Attached patch to backport select changes from HIVE-14977 and HIVE-16602
> Two tests fail on branch-2.3
>
>
> Key: HIVE-23086
> URL: https://issues.apache.org/jira/browse/HIVE-23086
> Project: Hive
> Issue Type: Bug
> Components: Test
>Affects Versions: 2.3.6
>Reporter: Alan Gates
>Assignee: Alan Gates
>Priority: Major
> Attachments: HIVE-23086.patch
>
>
> TestPerfCli query88.q and TestMiniLlapLocalCliDriver union_fast_stats.q fail
> on the 2.3 branch.
> The TestMiniLlapLocalCliDriver failure is fixed in HIVE-14977 where
> union_fast_stats.q is removed from the list of tests run with
> TestMiniLlapLocalCliDriver.
> The TestPerfCli failure is fixed HIVE-16602 where one line is added to
> query88.q to allow cartesian products:
> {{set hive.strict.checks.cartesian.product=false;}}
> I propose to backport these two changes to branch-2.3
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-23086) Two tests fail on branch-2.3
[
https://issues.apache.org/jira/browse/HIVE-23086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates reassigned HIVE-23086:
-
> Two tests fail on branch-2.3
>
>
> Key: HIVE-23086
> URL: https://issues.apache.org/jira/browse/HIVE-23086
> Project: Hive
> Issue Type: Bug
> Components: Test
>Affects Versions: 2.3.6
>Reporter: Alan Gates
>Assignee: Alan Gates
>Priority: Major
>
> TestPerfCli query88.q and TestMiniLlapLocalCliDriver union_fast_stats.q fail
> on the 2.3 branch.
> The TestMiniLlapLocalCliDriver failure is fixed in HIVE-14977 where
> union_fast_stats.q is removed from the list of tests run with
> TestMiniLlapLocalCliDriver.
> The TestPerfCli failure is fixed HIVE-16602 where one line is added to
> query88.q to allow cartesian products:
> {{set hive.strict.checks.cartesian.product=false;}}
> I propose to backport these two changes to branch-2.3
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22306) Use nonblocking thrift server for metastore
[ https://issues.apache.org/jira/browse/HIVE-22306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16947915#comment-16947915 ] Alan Gates commented on HIVE-22306: --- This is a good idea, but will a lot of testing. HMS depends on thread local variables in a number of places. So we will need to make sure any changes don't result in issues there. > Use nonblocking thrift server for metastore > --- > > Key: HIVE-22306 > URL: https://issues.apache.org/jira/browse/HIVE-22306 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Qinghui Xu >Priority: Major > > Currently hive metastore's threads are blocking for network io (it's using > `TThreadPoolServer` behind the scene), which means with increasing use cases > (in our tech stack there are different services relying on it, hiveserver2, > spark, presto, and more, all with a significant number of users), to handle > all connections it needs either a big thread pool or many instances with > smaller thread pools. And often, those metastores will see their thread pool > saturated, while the cpu usage is still quite low, just because most > connections stay idle and only run a query from time to time. This is thus a > great misuse of the computation resources. > Thus I propose to use a non blocking threading model, and run computation > asynchronously. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22150) HS2 allows setting system properties
[
https://issues.apache.org/jira/browse/HIVE-22150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924495#comment-16924495
]
Alan Gates commented on HIVE-22150:
---
Looks good to me. Any feedback [~thejas], [~vaibhgup] or others who have
worked more on HS2 than me?
> HS2 allows setting system properties
>
>
> Key: HIVE-22150
> URL: https://issues.apache.org/jira/browse/HIVE-22150
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 4.0.0, 3.1.1
>Reporter: Craig Condit
>Assignee: Hui An
>Priority: Major
> Attachments: HIVE-22150.patch.1, HIVE-22150.patch.2
>
>
> HiveServer2 currently allows setting system properties, which is a problem
> when used in a multi-user environment.
> Connecting via beeline and executing the following demonstrates the issue:
> {noformat}
> 0: jdbc:hive2://serv1000.example.com:2181,serv> SET system:java.io.tmpdir;
> +-+
> | set |
> +-+
> | system:java.io.tmpdir=/tmp |
> +-+
> 1 row selected (0.018 seconds)
> 0: jdbc:hive2://serv1000.example.com:2181,serv> SET
> system:java.io.tmpdir=/tmp/attacker-dir;
> No rows affected (0.013 seconds)
> 0: jdbc:hive2://serv1000.example.com:2181,serv> SET system:java.io.tmpdir;
> +--+
> | set|
> +--+
> | system:java.io.tmpdir=/tmp/attacker-dir |
> +--+
> 1 row selected (0.019 seconds)
> {noformat}
> Any changes persist until HS2 is restarted, and affect all connected users.
> At the very least, this is a denial-of-service vector (verified by setting
> line.separator to a random string).
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (HIVE-22102) Reduce HMS call when creating HiveSession
[
https://issues.apache.org/jira/browse/HIVE-22102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates reassigned HIVE-22102:
-
Assignee: Rajesh Balamohan
> Reduce HMS call when creating HiveSession
> -
>
> Key: HIVE-22102
> URL: https://issues.apache.org/jira/browse/HIVE-22102
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Trivial
> Fix For: 4.0.0
>
> Attachments: HIVE-22102.1.patch
>
>
> When establishing HiveSession, it ends up configuring session
> variables/settings.
> As part of it, it ends up checking the database details.
> [https://github.com/apache/hive/blob/master/service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java#L314]
> Even if it is `default` DB, it ends up making this check. In corner cases,
> these calls turn out to be expensive.
> {noformat}
> 2019-08-13T03:16:57,189 INFO [b42ba57f-1740-4174-855d-4e3f08319ca5
> HiveServer2-Handler-Pool: Thread-1552313] metadata.Hive: Total time spent in
> this metastore function was greater than 1000ms : getDatabase_(String, )=13265
> {noformat}
> We can just skip this check if its `DEFAULT_DATABASE_NAME` (default) DB. This
> may not be an issue for CachedStore.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22102) Reduce HMS call when creating HiveSession
[
https://issues.apache.org/jira/browse/HIVE-22102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates updated HIVE-22102:
--
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
Patch committed to master. Thanks Rajesh.
> Reduce HMS call when creating HiveSession
> -
>
> Key: HIVE-22102
> URL: https://issues.apache.org/jira/browse/HIVE-22102
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Reporter: Rajesh Balamohan
>Priority: Trivial
> Fix For: 4.0.0
>
> Attachments: HIVE-22102.1.patch
>
>
> When establishing HiveSession, it ends up configuring session
> variables/settings.
> As part of it, it ends up checking the database details.
> [https://github.com/apache/hive/blob/master/service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java#L314]
> Even if it is `default` DB, it ends up making this check. In corner cases,
> these calls turn out to be expensive.
> {noformat}
> 2019-08-13T03:16:57,189 INFO [b42ba57f-1740-4174-855d-4e3f08319ca5
> HiveServer2-Handler-Pool: Thread-1552313] metadata.Hive: Total time spent in
> this metastore function was greater than 1000ms : getDatabase_(String, )=13265
> {noformat}
> We can just skip this check if its `DEFAULT_DATABASE_NAME` (default) DB. This
> may not be an issue for CachedStore.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22105) Update ORC to 1.5.6.
[ https://issues.apache.org/jira/browse/HIVE-22105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-22105: -- Fix Version/s: 3.2.0 > Update ORC to 1.5.6. > > > Key: HIVE-22105 > URL: https://issues.apache.org/jira/browse/HIVE-22105 > Project: Hive > Issue Type: Bug >Reporter: Owen O'Malley >Assignee: Owen O'Malley >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0, 3.1.2 > > Time Spent: 10m > Remaining Estimate: 0h > > ORC has had some important fixes in the 1.5 branch and they should be picked > up by Hive. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22105) Update ORC to 1.5.6.
[ https://issues.apache.org/jira/browse/HIVE-22105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916966#comment-16916966 ] Alan Gates commented on HIVE-22105: --- Also pushed to branch-3. > Update ORC to 1.5.6. > > > Key: HIVE-22105 > URL: https://issues.apache.org/jira/browse/HIVE-22105 > Project: Hive > Issue Type: Bug >Reporter: Owen O'Malley >Assignee: Owen O'Malley >Priority: Major > Labels: pull-request-available > Fix For: 3.1.2 > > Time Spent: 10m > Remaining Estimate: 0h > > ORC has had some important fixes in the 1.5 branch and they should be picked > up by Hive. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-19667) Remove distribution management tag from pom.xml
[ https://issues.apache.org/jira/browse/HIVE-19667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913710#comment-16913710 ] Alan Gates commented on HIVE-19667: --- Ported to branch-3 and branch-3.1 as well so we can make releases on those branches. > Remove distribution management tag from pom.xml > --- > > Key: HIVE-19667 > URL: https://issues.apache.org/jira/browse/HIVE-19667 > Project: Hive > Issue Type: Task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 4.0.0, 3.2.0, 3.1.2 > > Attachments: HIVE-19667.1.patch > > > This tag overrides apaches configuration in maven settings file and makes it > impossible to publish maven artifacts. There is no way around to it either. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-19667) Remove distribution management tag from pom.xml
[ https://issues.apache.org/jira/browse/HIVE-19667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-19667: -- Fix Version/s: 3.1.2 3.2.0 > Remove distribution management tag from pom.xml > --- > > Key: HIVE-19667 > URL: https://issues.apache.org/jira/browse/HIVE-19667 > Project: Hive > Issue Type: Task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 4.0.0, 3.2.0, 3.1.2 > > Attachments: HIVE-19667.1.patch > > > This tag overrides apaches configuration in maven settings file and makes it > impossible to publish maven artifacts. There is no way around to it either. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-21980) Parsing time can be high in case of deeply nested subqueries
[
https://issues.apache.org/jira/browse/HIVE-21980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates updated HIVE-21980:
--
Fix Version/s: 3.1.2
3.2.0
2.4.0
2.3.7
> Parsing time can be high in case of deeply nested subqueries
>
>
> Key: HIVE-21980
> URL: https://issues.apache.org/jira/browse/HIVE-21980
> Project: Hive
> Issue Type: Improvement
> Components: Parser
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Fix For: 2.4.0, 4.0.0, 3.2.0, 3.1.2, 2.3.7
>
> Attachments: HIVE-21980.01.patch, HIVE-21980.01.patch,
> HIVE-21980.02.patch, HIVE-21980.03.patch
>
>
> for queries which are recursively doing:
> {code}
> SELECT ...
> FROM (SELECT ...
> FROM ( [...]
> ) JOIN
> (SELECT ...
> FROM ( [...] )
> JOIN ( [...]
> )
> {code}
> when the issue is present hs2 stops right after starting compiling and the
> following stack is there
> {code}
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.subQuerySource(HiveParser_FromClauseParser.java:4745)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.synpred1_FromClauseParser_fragment(HiveParser_FromClauseParser.java:6513)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.synpred1_FromClauseParser(HiveParser_FromClauseParser.java:6660)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser$DFA10.specialStateTransition(HiveParser_FromClauseParser.java:6942)
> at org.antlr.runtime.DFA.predict(DFA.java:80)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.atomjoinSource(HiveParser_FromClauseParser.java:1596)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.joinSource(HiveParser_FromClauseParser.java:1859)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromSource(HiveParser_FromClauseParser.java:1532)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromClause(HiveParser_FromClauseParser.java:1375)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.fromClause(HiveParser.java:45755)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.atomSelectStatement(HiveParser.java:40337)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.selectStatement(HiveParser.java:40589)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.regularBody(HiveParser.java:40235)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpressionBody(HiveParser.java:39445)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:39333)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.subQuerySource(HiveParser_FromClauseParser.java:4745)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.atomjoinSource(HiveParser_FromClauseParser.java:1699)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.joinSource(HiveParser_FromClauseParser.java:1859)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromSource(HiveParser_FromClauseParser.java:1532)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromClause(HiveParser_FromClauseParser.java:1375)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.fromClause(HiveParser.java:45755)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.atomSelectStatement(HiveParser.java:40337)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.selectStatement(HiveParser.java:40589)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.regularBody(HiveParser.java:40235)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpressionBody(HiveParser.java:39445)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:39333)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.subQuerySource(HiveParser_FromClauseParser.java:4745)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.synpred1_FromClauseParser_fragment(HiveParser_FromClauseParser.java:6513)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.synpred1_FromClauseParser(HiveParser_FromClauseParser.java:6660)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser$DFA10.specialStateTransition(HiveParser_FromClauseParser.java:6942)
> at org.antlr.runtime.DFA.predict(DFA.java:80)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.atomjoinSource(HiveParser_FromClauseParser.java:1596)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.joinSource(HiveParser_FromClau
[jira] [Commented] (HIVE-19207) Support avro record writer for streaming ingest
[ https://issues.apache.org/jira/browse/HIVE-19207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16912660#comment-16912660 ] Alan Gates commented on HIVE-19207: --- cc [~prasanth_j] > Support avro record writer for streaming ingest > --- > > Key: HIVE-19207 > URL: https://issues.apache.org/jira/browse/HIVE-19207 > Project: Hive > Issue Type: Sub-task > Components: Streaming >Affects Versions: 3.0.0, 3.1.0 >Reporter: Prasanth Jayachandran >Assignee: Alan Gates >Priority: Major > Attachments: HIVE-19207.patch > > > Add support for Avro record writer in streaming ingest. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-19207) Support avro record writer for streaming ingest
[ https://issues.apache.org/jira/browse/HIVE-19207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-19207: -- Target Version/s: 3.1.0, 3.0.0 (was: 3.0.0, 3.1.0) Assignee: Alan Gates (was: Prasanth Jayachandran) Status: Patch Available (was: Open) Here is an initial pass at adding Avro support in streaming. I modified the RecordWriter interface to be parameterized by the type of records that the user is writing. Previously it seemed to assume that all records could be turned into byte[] and handled as such. This is not convient when you already have structured data like Avro. I tried to do it in a way that did not break backward compatibility with existing RecordWriter implementations. I've added two RecordWriter implementations, a StrictAvroWriter and a MappingAvroWriter. StrictAvroWriter assumes that the Hive table and Avro records exactly match in schema (or at least close enough that the type conversion can be done). It also assumes that the Avro schema passed to it exactly matches every Avro record in the stream. MappingAvroWriter takes a map of Hive column names to Avro paths. The avro path can be a simple column name, or a path through an Avro complex type. So the Hive column 'zipcode' could be mapped to an Avro column 'zipcode' or to an Avro record with a zipcode field (address.zipcode) or to an Avro map with a zipcode key (address[zipcode]). Again the system assumes the types are close enough that Hive can do type conversion if necessary. In this case the Avro schema passed to the writer does not have to exactly match every record in the stream, but it must be usuable to decode the referenced Avro columns for every record in the stream. Both writers support all Avro types except Null as a top level object. Avro unions created just to allow a null value are "read through" to the non-null type and that type is used. For example, an Avro nullableString will become a String in Hive. For both writers I did not use the existing AvroSerDe because it assumes that every Avro record has a schema encoded with it. In general this is not how I assume users generally stream their data. I did try to follow the same type conversions as the AvroSerDe. > Support avro record writer for streaming ingest > --- > > Key: HIVE-19207 > URL: https://issues.apache.org/jira/browse/HIVE-19207 > Project: Hive > Issue Type: Sub-task > Components: Streaming >Affects Versions: 3.1.0, 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Alan Gates >Priority: Major > Attachments: HIVE-19207.patch > > > Add support for Avro record writer in streaming ingest. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-19207) Support avro record writer for streaming ingest
[ https://issues.apache.org/jira/browse/HIVE-19207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-19207: -- Attachment: HIVE-19207.patch > Support avro record writer for streaming ingest > --- > > Key: HIVE-19207 > URL: https://issues.apache.org/jira/browse/HIVE-19207 > Project: Hive > Issue Type: Sub-task > Components: Streaming >Affects Versions: 3.0.0, 3.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-19207.patch > > > Add support for Avro record writer in streaming ingest. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-20057) For ALTER TABLE t SET TBLPROPERTIES ('EXTERNAL'='TRUE'); `TBL_TYPE` attribute change not reflecting for non-CAPS
[
https://issues.apache.org/jira/browse/HIVE-20057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates updated HIVE-20057:
--
Fix Version/s: (was: 3.1.0)
3.1.2
> For ALTER TABLE t SET TBLPROPERTIES ('EXTERNAL'='TRUE'); `TBL_TYPE` attribute
> change not reflecting for non-CAPS
>
>
> Key: HIVE-20057
> URL: https://issues.apache.org/jira/browse/HIVE-20057
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Affects Versions: All Versions
>Reporter: Anirudh
>Assignee: Anirudh
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.1.2
>
> Attachments: hive20057.patch
>
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> Hive EXTERNAL table shown as MANAGED after conversion using
> {code} ALTER TABLE t SET TBLPROPERTIES ('EXTERNAL'='True')
> {code}
>
> The DESCRIBE FORMATTED shows:
> {code}
> Table Type: MANAGED_TABLE
> Table Parameters:
> EXTERNAL True
> {code}
>
> This is actually a External table but is shown wrongly, as 'True' was used in
> place of 'TRUE' in the ALTER statement.
> Issue explained here:
> [StakOverflow - Hive Table is MANAGED or
> EXTERNAL|https://stackoverflow.com/questions/51103317/hive-table-is-managed-or-external/51142873#51142873]
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Resolved] (HIVE-22105) Update ORC to 1.5.6.
[ https://issues.apache.org/jira/browse/HIVE-22105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates resolved HIVE-22105. --- Fix Version/s: 3.1.2 Resolution: Fixed Tests pass when run locally. Committed to branch-3.1 > Update ORC to 1.5.6. > > > Key: HIVE-22105 > URL: https://issues.apache.org/jira/browse/HIVE-22105 > Project: Hive > Issue Type: Bug >Reporter: Owen O'Malley >Assignee: Owen O'Malley >Priority: Major > Labels: pull-request-available > Fix For: 3.1.2 > > Time Spent: 10m > Remaining Estimate: 0h > > ORC has had some important fixes in the 1.5 branch and they should be picked > up by Hive. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22105) Update ORC to 1.5.6.
[ https://issues.apache.org/jira/browse/HIVE-22105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16908426#comment-16908426 ] Alan Gates commented on HIVE-22105: --- Regenerating the results doesn't help. The test still fails after regeneration, which I take to mean it produces inconsistent results. > Update ORC to 1.5.6. > > > Key: HIVE-22105 > URL: https://issues.apache.org/jira/browse/HIVE-22105 > Project: Hive > Issue Type: Bug >Reporter: Owen O'Malley >Assignee: Owen O'Malley >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > ORC has had some important fixes in the 1.5 branch and they should be picked > up by Hive. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HIVE-22105) Update ORC to 1.5.6.
[ https://issues.apache.org/jira/browse/HIVE-22105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16908304#comment-16908304 ] Alan Gates commented on HIVE-22105: --- After the latest version of the patch I still see failures in TestCliDriver and TestMiniLlapLocalCliDriver with orc_merge.9. The only diff is: 181c181 < Found 1 items --- > Found 2 items If that looks reasonable I'll just regen the expected results when I apply the patch. > Update ORC to 1.5.6. > > > Key: HIVE-22105 > URL: https://issues.apache.org/jira/browse/HIVE-22105 > Project: Hive > Issue Type: Bug >Reporter: Owen O'Malley >Assignee: Owen O'Malley >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > ORC has had some important fixes in the 1.5 branch and they should be picked > up by Hive. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HIVE-20057) For ALTER TABLE t SET TBLPROPERTIES ('EXTERNAL'='TRUE'); `TBL_TYPE` attribute change not reflecting for non-CAPS
[
https://issues.apache.org/jira/browse/HIVE-20057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907778#comment-16907778
]
Alan Gates commented on HIVE-20057:
---
Patch also committed to branch-3.
> For ALTER TABLE t SET TBLPROPERTIES ('EXTERNAL'='TRUE'); `TBL_TYPE` attribute
> change not reflecting for non-CAPS
>
>
> Key: HIVE-20057
> URL: https://issues.apache.org/jira/browse/HIVE-20057
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Affects Versions: All Versions
>Reporter: Anirudh
>Assignee: Anirudh
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.1.0
>
> Attachments: hive20057.patch
>
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> Hive EXTERNAL table shown as MANAGED after conversion using
> {code} ALTER TABLE t SET TBLPROPERTIES ('EXTERNAL'='True')
> {code}
>
> The DESCRIBE FORMATTED shows:
> {code}
> Table Type: MANAGED_TABLE
> Table Parameters:
> EXTERNAL True
> {code}
>
> This is actually a External table but is shown wrongly, as 'True' was used in
> place of 'TRUE' in the ALTER statement.
> Issue explained here:
> [StakOverflow - Hive Table is MANAGED or
> EXTERNAL|https://stackoverflow.com/questions/51103317/hive-table-is-managed-or-external/51142873#51142873]
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HIVE-22105) Update ORC to 1.5.6.
[ https://issues.apache.org/jira/browse/HIVE-22105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907658#comment-16907658 ] Alan Gates commented on HIVE-22105: --- I get errors for the following qfile tests with TestMiniLlapLocalCliDriver: acid_vectorization_original.q, change_allowincompatible_vectorization_false_date.q, default_constraint.q, deleteAnalyze.q, enforce_constraint_notnull.q, extrapolate_part_stats_partial_ndv.q, materialized_view_create.q, materialized_view_create_rewrite.q, materialized_view_create_rewrite_4.q, materialized_view_create_rewrite_5.q, materialized_view_create_rewrite_dummy.q, materialized_view_create_rewrite_multi_db.q, materialized_view_create_rewrite_time_window.q, materialized_view_describe.q, orc_merge11.q, orc_merge9.q > Update ORC to 1.5.6. > > > Key: HIVE-22105 > URL: https://issues.apache.org/jira/browse/HIVE-22105 > Project: Hive > Issue Type: Bug >Reporter: Owen O'Malley >Assignee: Owen O'Malley >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > ORC has had some important fixes in the 1.5 branch and they should be picked > up by Hive. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HIVE-22096) Backport HIVE-21584 to branch-2.3
[ https://issues.apache.org/jira/browse/HIVE-22096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907517#comment-16907517 ] Alan Gates commented on HIVE-22096: --- Tested on branch-2 and committed there as well. > Backport HIVE-21584 to branch-2.3 > - > > Key: HIVE-22096 > URL: https://issues.apache.org/jira/browse/HIVE-22096 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Yuming Wang >Assignee: Yuming Wang >Priority: Major > Fix For: 2.3.6 > > Attachments: HIVE-22096.branch-2.3.patch > > > Backport HIVE-21584 to make Spark support JDK 11. > https://www.mail-archive.com/dev@hive.apache.org/msg137001.html -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HIVE-20057) For ALTER TABLE t SET TBLPROPERTIES ('EXTERNAL'='TRUE'); `TBL_TYPE` attribute change not reflecting for non-CAPS
[
https://issues.apache.org/jira/browse/HIVE-20057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates updated HIVE-20057:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
Patch committed to branch-3.1. Thanks Anirudh.
> For ALTER TABLE t SET TBLPROPERTIES ('EXTERNAL'='TRUE'); `TBL_TYPE` attribute
> change not reflecting for non-CAPS
>
>
> Key: HIVE-20057
> URL: https://issues.apache.org/jira/browse/HIVE-20057
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Affects Versions: All Versions
>Reporter: Anirudh
>Assignee: Anirudh
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.1.0
>
> Attachments: hive20057.patch
>
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> Hive EXTERNAL table shown as MANAGED after conversion using
> {code} ALTER TABLE t SET TBLPROPERTIES ('EXTERNAL'='True')
> {code}
>
> The DESCRIBE FORMATTED shows:
> {code}
> Table Type: MANAGED_TABLE
> Table Parameters:
> EXTERNAL True
> {code}
>
> This is actually a External table but is shown wrongly, as 'True' was used in
> place of 'TRUE' in the ALTER statement.
> Issue explained here:
> [StakOverflow - Hive Table is MANAGED or
> EXTERNAL|https://stackoverflow.com/questions/51103317/hive-table-is-managed-or-external/51142873#51142873]
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-22096) Backport HIVE-21584 to branch-2.3
[ https://issues.apache.org/jira/browse/HIVE-22096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-22096: -- Resolution: Fixed Status: Resolved (was: Patch Available) In local run all tests passed against 2.3 branch. Committed to branch-2.3. > Backport HIVE-21584 to branch-2.3 > - > > Key: HIVE-22096 > URL: https://issues.apache.org/jira/browse/HIVE-22096 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Yuming Wang >Assignee: Yuming Wang >Priority: Major > Fix For: 2.3.6 > > Attachments: HIVE-22096.branch-2.3.patch > > > Backport HIVE-21584 to make Spark support JDK 11. > https://www.mail-archive.com/dev@hive.apache.org/msg137001.html -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HIVE-21174) hive.stats.ndv.error parameter documentation issue
[ https://issues.apache.org/jira/browse/HIVE-21174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21174: -- Fix Version/s: (was: 3.1.2) > hive.stats.ndv.error parameter documentation issue > -- > > Key: HIVE-21174 > URL: https://issues.apache.org/jira/browse/HIVE-21174 > Project: Hive > Issue Type: Improvement > Components: Documentation >Affects Versions: 2.0.0, 2.0.1, 2.0.2, 2.1.0, 2.1.1, 2.1.2, 2.2.0, 2.3.0, > 3.0.0, 2.4.0, 2.2.1, 2.3.1, 2.3.2, 2.3.3, 3.1.0, 3.0.1, 3.10, 3.2.0, 3.1.1, > 2.3.4, 3.1.2 >Reporter: Pablo Junge >Assignee: Nita Dembla >Priority: Major > Fix For: 2.0.0, 2.0.1, 2.0.2, 2.1.0, 2.1.1, 2.1.2, 2.2.0, 2.3.0, > 3.0.0, 2.4.0, 2.2.1, 2.3.1, 2.3.2, 2.3.3, 3.1.0, 3.0.1, 3.10, 3.2.0, 3.1.1, > 2.3.4 > > > Hive documentation for hive.stats.ndv.error does not specify that > hive.stats.ndv.error will only affect FM Sketch and not HLL. > > https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HIVE-22096) Backport HIVE-21584 to branch-2.3
[ https://issues.apache.org/jira/browse/HIVE-22096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-22096: -- Fix Version/s: 2.3.6 > Backport HIVE-21584 to branch-2.3 > - > > Key: HIVE-22096 > URL: https://issues.apache.org/jira/browse/HIVE-22096 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Yuming Wang >Assignee: Yuming Wang >Priority: Major > Fix For: 2.3.6 > > Attachments: HIVE-22096.branch-2.3.patch > > > Backport HIVE-21584 to make Spark support JDK 11. > https://www.mail-archive.com/dev@hive.apache.org/msg137001.html -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HIVE-21616) Implement JSON_VALUE, JSON_QUERY, and IS [NOT] JSON
[ https://issues.apache.org/jira/browse/HIVE-21616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21616: -- Attachment: HIVE-21616.2.patch > Implement JSON_VALUE, JSON_QUERY, and IS [NOT] JSON > --- > > Key: HIVE-21616 > URL: https://issues.apache.org/jira/browse/HIVE-21616 > Project: Hive > Issue Type: Sub-task > Components: UDF >Affects Versions: 4.0.0 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Attachments: HIVE-21615.patch, HIVE-21616.2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21224) Upgrade tests JUnit3 to JUnit4
[ https://issues.apache.org/jira/browse/HIVE-21224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873463#comment-16873463 ] Alan Gates commented on HIVE-21224: --- Before committing this please make sure the ITests under standalone-metastore execute properly as well. Those are not run as part of the standard nightly build but are needed whenever the data scripts are updated. > Upgrade tests JUnit3 to JUnit4 > -- > > Key: HIVE-21224 > URL: https://issues.apache.org/jira/browse/HIVE-21224 > Project: Hive > Issue Type: Improvement >Reporter: Bruno Pusztahazi >Assignee: Bruno Pusztahazi >Priority: Major > Attachments: HIVE-21224.1.patch, HIVE-21224.2.patch, > HIVE-21224.3.patch, HIVE-21224.4.patch, HIVE-21224.5.patch, > HIVE-21224.6.patch, HIVE-21224.7.patch, HIVE-21224.8.patch, HIVE-21224.9.patch > > > Old JUnit3 tests should be upgraded to JUnit4 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21752) Thread Safety and Memory Leaks in HCatRecordObjectInspectorFactory
[ https://issues.apache.org/jira/browse/HIVE-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21752: -- Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) Patch committed to master. Thanks Jalpan for the patch and sorry it took so long to get it committed. > Thread Safety and Memory Leaks in HCatRecordObjectInspectorFactory > -- > > Key: HIVE-21752 > URL: https://issues.apache.org/jira/browse/HIVE-21752 > Project: Hive > Issue Type: Bug > Components: HCatalog >Reporter: Jalpan Randeri >Assignee: Jalpan Randeri >Priority: Minor > Labels: newbie, patch > Fix For: 4.0.0 > > Attachments: HIVE-21752.patch, HIVE-21752.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > h3. Summary > There are a couple of issues in HCatRecordObjectInspectorFactory[1] because > it uses a static Java HashMap to cache objects: > # Java HashMap is not thread safe. This can lead to data corruptions and > race conditions in multithreaded servers when two threads update the > ObjectInspector. > # There is no eviction policy and as a result, this can result in memory > leaks. If user reads a lot of different schemas, Hive server will start > seeing memory pressure, once it start going to have a lot of cached record > and object inspectors. > This patch propose to replace the cache using a Guava cache which enables > cache evictions and thread safety. Guava cache is already used in Hive > ObjectInspectorFactory [2], so this change is consistent with the rest of > Hive. > Attached is a patch that fixes this issue. > h3. References: > # > [https://github.com/apache/hive/blob/b58d50cb73a1f79a5d079e0a2c5ac33d2efc33a0/hcatalog/core/src/main/java/org/apache/hive/hcatalog/data/HCatRecordObjectInspectorFactory.java#L44-L47] > # > [https://github.com/apache/hive/blob/b58d50cb73a1f79a5d079e0a2c5ac33d2efc33a0/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java#L68-L87] > > h4. Review Board Link: > * [https://reviews.apache.org/r/70674/] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20854) Sensible Defaults: Hive's Zookeeper heartbeat interval is 20 minutes, change to 2
[
https://issues.apache.org/jira/browse/HIVE-20854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates updated HIVE-20854:
--
Resolution: Fixed
Fix Version/s: 4.0.0
Status: Resolved (was: Patch Available)
Committed patch 2 to master.
> Sensible Defaults: Hive's Zookeeper heartbeat interval is 20 minutes, change
> to 2
> -
>
> Key: HIVE-20854
> URL: https://issues.apache.org/jira/browse/HIVE-20854
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-20854.1.patch, HIVE-20854.2.patch,
> HIVE-20854.2.patch, HIVE-20854.2.patch
>
>
> {code}
> HIVE_ZOOKEEPER_SESSION_TIMEOUT("hive.zookeeper.session.timeout",
> "120ms",
> new TimeValidator(TimeUnit.MILLISECONDS),
> "ZooKeeper client's session timeout (in milliseconds). The client is
> disconnected, and as a result, all locks released, \n" +
> "if a heartbeat is not sent in the timeout."),
> {code}
> That's 1,200,000ms which is too long for all practical purposes - a 20 minute
> outage in case a node has a failure is too long.
> That is too long for the JDBC load-balancing, LLAP failure tolerance and the
> lock manager expiry.
> Change to 2 minutes, as a sensible default
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21198) Introduce a database object reference class
[
https://issues.apache.org/jira/browse/HIVE-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates updated HIVE-21198:
--
Status: Open (was: Patch Available)
This patch looks good, but it's very out of date. If you refresh it I'll take
another look.
> Introduce a database object reference class
> ---
>
> Key: HIVE-21198
> URL: https://issues.apache.org/jira/browse/HIVE-21198
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21198.1.patch, HIVE-21198.2.patch,
> HIVE-21198.3.patch, HIVE-21198.4.patch, HIVE-21198.5.patch,
> HIVE-21198.6.patch, HIVE-21198.7.patch
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> There are many places in which "{databasename}.{tablename}" is passed as a
> single string; there are some places where the they travel as 2 separate
> arguments.
> Idea would be to introduce a simple immutable class with 2 fields ; and pass
> these informations together. Making this better is required if we would be
> wanting to enable dot in tablenames
> HIVE-16907, HIVE-21151
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21869) Clean up the Kafka storage handler readme and examples
[ https://issues.apache.org/jira/browse/HIVE-21869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21869: -- Resolution: Fixed Status: Resolved (was: Patch Available) Patch 3 committed to master. Thanks Kristopher. > Clean up the Kafka storage handler readme and examples > -- > > Key: HIVE-21869 > URL: https://issues.apache.org/jira/browse/HIVE-21869 > Project: Hive > Issue Type: Improvement > Components: kafka integration >Affects Versions: 4.0.0 >Reporter: Kristopher Kane >Assignee: Kristopher Kane >Priority: Minor > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-21869.1.patch, HIVE-21869.2.patch, > HIVE-21869.3.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21821) Backport HIVE-21739 to branch-3.1
[ https://issues.apache.org/jira/browse/HIVE-21821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16872714#comment-16872714 ] Alan Gates commented on HIVE-21821: --- Committed patch to branch-3. > Backport HIVE-21739 to branch-3.1 > - > > Key: HIVE-21821 > URL: https://issues.apache.org/jira/browse/HIVE-21821 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.1 >Reporter: Aditya Shah >Assignee: Aditya Shah >Priority: Major > Fix For: 3.1.2 > > Attachments: HIVE-21821.branch-3.1.1.patch, > HIVE-21821.branch-3.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21756) Backport HIVE-21404 to branch-3: MSSQL upgrade script alters the wrong column
[ https://issues.apache.org/jira/browse/HIVE-21756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21756: -- Resolution: Fixed Status: Resolved (was: Patch Available) Patch committed to branch-3. > Backport HIVE-21404 to branch-3: MSSQL upgrade script alters the wrong column > - > > Key: HIVE-21756 > URL: https://issues.apache.org/jira/browse/HIVE-21756 > Project: Hive > Issue Type: Bug > Components: Metastore, Standalone Metastore >Affects Versions: 3.1.1 >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0, 3.1.2 > > Attachments: HIVE-21756.01.branch-3.patch, > HIVE-21756.branch-3.01.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > Backport of HIVE-21404to branch-3. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21786) Update repo URLs in poms
[ https://issues.apache.org/jira/browse/HIVE-21786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21786: -- Fix Version/s: 2.4.0 > Update repo URLs in poms > > > Key: HIVE-21786 > URL: https://issues.apache.org/jira/browse/HIVE-21786 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Affects Versions: 4.0.0, 3.1.1, 2.3.5 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Fix For: 2.4.0, 4.0.0 > > Attachments: HIVE-21786.patch > > > Need to update repo URLs in the poms. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21786) Update repo URLs in poms
[ https://issues.apache.org/jira/browse/HIVE-21786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16871819#comment-16871819 ] Alan Gates commented on HIVE-21786: --- Patch committed to branch-2 > Update repo URLs in poms > > > Key: HIVE-21786 > URL: https://issues.apache.org/jira/browse/HIVE-21786 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Affects Versions: 4.0.0, 3.1.1, 2.3.5 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Fix For: 2.4.0, 4.0.0 > > Attachments: HIVE-21786.patch > > > Need to update repo URLs in the poms. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21680) Backport HIVE-17644 to branch-2 and branch-2.3
[
https://issues.apache.org/jira/browse/HIVE-21680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates updated HIVE-21680:
--
Fix Version/s: 2.4.0
> Backport HIVE-17644 to branch-2 and branch-2.3
> --
>
> Key: HIVE-21680
> URL: https://issues.apache.org/jira/browse/HIVE-21680
> Project: Hive
> Issue Type: Bug
>Reporter: Yuming Wang
>Assignee: Yuming Wang
>Priority: Major
> Fix For: 2.4.0, 2.3.5
>
> Attachments: HIVE-21680.branch-2.3.patch, HIVE-21680.branch-2.patch
>
>
> Backport HIVE-17644 to fix the warning in {{get statistics when not analyzed
> in Hive or Spark}}:
> {code:scala}
> test("get statistics when not analyzed in Hive or Spark") {
> val tabName = "tab1"
> withTable(tabName) {
> createNonPartitionedTable(tabName, analyzedByHive = false,
> analyzedBySpark = false)
> checkTableStats(tabName, hasSizeInBytes = true, expectedRowCounts =
> None)
> // ALTER TABLE SET TBLPROPERTIES invalidates some contents of Hive
> specific statistics
> // This is triggered by the Hive alterTable API
> val describeResult = hiveClient.runSqlHive(s"DESCRIBE FORMATTED
> $tabName")
> val rawDataSize = extractStatsPropValues(describeResult, "rawDataSize")
> val numRows = extractStatsPropValues(describeResult, "numRows")
> val totalSize = extractStatsPropValues(describeResult, "totalSize")
> assert(rawDataSize.isEmpty, "rawDataSize should not be shown without
> table analysis")
> assert(numRows.isEmpty, "numRows should not be shown without table
> analysis")
> assert(totalSize.isDefined && totalSize.get > 0, "totalSize is lost")
> }
> }
> //
> https://github.com/apache/spark/blob/43dcb91a4cb25aa7e1cc5967194f098029a0361e/sql/hive/src/test/scala/org/apache/spark/sql/hive/StatisticsSuite.scala#L789-L806
> {code}
> {noformat}
> 06:23:46.103 WARN org.apache.hadoop.hive.metastore.MetaStoreDirectSql: Failed
> to execute [SELECT "DBS"."NAME", "TBLS"."TBL_NAME",
> "COLUMNS_V2"."COLUMN_NAME","KEY_CONSTRAINTS"."POSITION",
> "KEY_CONSTRAINTS"."CONSTRAINT_NAME", "KEY_CONSTRAINTS"."ENABLE_VALIDATE_RELY"
> FROM "TBLS" INNER JOIN "KEY_CONSTRAINTS" ON "TBLS"."TBL_ID" =
> "KEY_CONSTRAINTS"."PARENT_TBL_ID" INNER JOIN "DBS" ON "TBLS"."DB_ID" =
> "DBS"."DB_ID" INNER JOIN "COLUMNS_V2" ON "COLUMNS_V2"."CD_ID" =
> "KEY_CONSTRAINTS"."PARENT_CD_ID" AND "COLUMNS_V2"."INTEGER_IDX" =
> "KEY_CONSTRAINTS"."PARENT_INTEGER_IDX" WHERE
> "KEY_CONSTRAINTS"."CONSTRAINT_TYPE" = 0 AND "DBS"."NAME" = ? AND
> "TBLS"."TBL_NAME" = ?] with parameters [default, tab1]
> javax.jdo.JDODataStoreException: Error executing SQL query "SELECT
> "DBS"."NAME", "TBLS"."TBL_NAME",
> "COLUMNS_V2"."COLUMN_NAME","KEY_CONSTRAINTS"."POSITION",
> "KEY_CONSTRAINTS"."CONSTRAINT_NAME", "KEY_CONSTRAINTS"."ENABLE_VALIDATE_RELY"
> FROM "TBLS" INNER JOIN "KEY_CONSTRAINTS" ON "TBLS"."TBL_ID" =
> "KEY_CONSTRAINTS"."PARENT_TBL_ID" INNER JOIN "DBS" ON "TBLS"."DB_ID" =
> "DBS"."DB_ID" INNER JOIN "COLUMNS_V2" ON "COLUMNS_V2"."CD_ID" =
> "KEY_CONSTRAINTS"."PARENT_CD_ID" AND "COLUMNS_V2"."INTEGER_IDX" =
> "KEY_CONSTRAINTS"."PARENT_INTEGER_IDX" WHERE
> "KEY_CONSTRAINTS"."CONSTRAINT_TYPE" = 0 AND "DBS"."NAME" = ? AND
> "TBLS"."TBL_NAME" = ?".
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
> at org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:391)
> at org.datanucleus.api.jdo.JDOQuery.executeWithArray(JDOQuery.java:267)
> at
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql.executeWithArray(MetaStoreDirectSql.java:1750)
> at
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql.getPrimaryKeys(MetaStoreDirectSql.java:1939)
> at
> org.apache.hadoop.hive.metastore.ObjectStore$11.getSqlResult(ObjectStore.java:8213)
> at
> org.apache.hadoop.hive.metastore.ObjectStore$11.getSqlResult(ObjectStore.java:8209)
> at
> org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:2719)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getPrimaryKeysInternal(ObjectStore.java:8221)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getPrimaryKeys(ObjectStore.java:8199)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:101)
> at com.sun.proxy.$Proxy24.getPrimaryKeys(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.
[jira] [Commented] (HIVE-21680) Backport HIVE-17644 to branch-2 and branch-2.3
[
https://issues.apache.org/jira/browse/HIVE-21680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16871795#comment-16871795
]
Alan Gates commented on HIVE-21680:
---
Committed patch to branch-2, which I should have done initially.
> Backport HIVE-17644 to branch-2 and branch-2.3
> --
>
> Key: HIVE-21680
> URL: https://issues.apache.org/jira/browse/HIVE-21680
> Project: Hive
> Issue Type: Bug
>Reporter: Yuming Wang
>Assignee: Yuming Wang
>Priority: Major
> Fix For: 2.4.0, 2.3.5
>
> Attachments: HIVE-21680.branch-2.3.patch, HIVE-21680.branch-2.patch
>
>
> Backport HIVE-17644 to fix the warning in {{get statistics when not analyzed
> in Hive or Spark}}:
> {code:scala}
> test("get statistics when not analyzed in Hive or Spark") {
> val tabName = "tab1"
> withTable(tabName) {
> createNonPartitionedTable(tabName, analyzedByHive = false,
> analyzedBySpark = false)
> checkTableStats(tabName, hasSizeInBytes = true, expectedRowCounts =
> None)
> // ALTER TABLE SET TBLPROPERTIES invalidates some contents of Hive
> specific statistics
> // This is triggered by the Hive alterTable API
> val describeResult = hiveClient.runSqlHive(s"DESCRIBE FORMATTED
> $tabName")
> val rawDataSize = extractStatsPropValues(describeResult, "rawDataSize")
> val numRows = extractStatsPropValues(describeResult, "numRows")
> val totalSize = extractStatsPropValues(describeResult, "totalSize")
> assert(rawDataSize.isEmpty, "rawDataSize should not be shown without
> table analysis")
> assert(numRows.isEmpty, "numRows should not be shown without table
> analysis")
> assert(totalSize.isDefined && totalSize.get > 0, "totalSize is lost")
> }
> }
> //
> https://github.com/apache/spark/blob/43dcb91a4cb25aa7e1cc5967194f098029a0361e/sql/hive/src/test/scala/org/apache/spark/sql/hive/StatisticsSuite.scala#L789-L806
> {code}
> {noformat}
> 06:23:46.103 WARN org.apache.hadoop.hive.metastore.MetaStoreDirectSql: Failed
> to execute [SELECT "DBS"."NAME", "TBLS"."TBL_NAME",
> "COLUMNS_V2"."COLUMN_NAME","KEY_CONSTRAINTS"."POSITION",
> "KEY_CONSTRAINTS"."CONSTRAINT_NAME", "KEY_CONSTRAINTS"."ENABLE_VALIDATE_RELY"
> FROM "TBLS" INNER JOIN "KEY_CONSTRAINTS" ON "TBLS"."TBL_ID" =
> "KEY_CONSTRAINTS"."PARENT_TBL_ID" INNER JOIN "DBS" ON "TBLS"."DB_ID" =
> "DBS"."DB_ID" INNER JOIN "COLUMNS_V2" ON "COLUMNS_V2"."CD_ID" =
> "KEY_CONSTRAINTS"."PARENT_CD_ID" AND "COLUMNS_V2"."INTEGER_IDX" =
> "KEY_CONSTRAINTS"."PARENT_INTEGER_IDX" WHERE
> "KEY_CONSTRAINTS"."CONSTRAINT_TYPE" = 0 AND "DBS"."NAME" = ? AND
> "TBLS"."TBL_NAME" = ?] with parameters [default, tab1]
> javax.jdo.JDODataStoreException: Error executing SQL query "SELECT
> "DBS"."NAME", "TBLS"."TBL_NAME",
> "COLUMNS_V2"."COLUMN_NAME","KEY_CONSTRAINTS"."POSITION",
> "KEY_CONSTRAINTS"."CONSTRAINT_NAME", "KEY_CONSTRAINTS"."ENABLE_VALIDATE_RELY"
> FROM "TBLS" INNER JOIN "KEY_CONSTRAINTS" ON "TBLS"."TBL_ID" =
> "KEY_CONSTRAINTS"."PARENT_TBL_ID" INNER JOIN "DBS" ON "TBLS"."DB_ID" =
> "DBS"."DB_ID" INNER JOIN "COLUMNS_V2" ON "COLUMNS_V2"."CD_ID" =
> "KEY_CONSTRAINTS"."PARENT_CD_ID" AND "COLUMNS_V2"."INTEGER_IDX" =
> "KEY_CONSTRAINTS"."PARENT_INTEGER_IDX" WHERE
> "KEY_CONSTRAINTS"."CONSTRAINT_TYPE" = 0 AND "DBS"."NAME" = ? AND
> "TBLS"."TBL_NAME" = ?".
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
> at org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:391)
> at org.datanucleus.api.jdo.JDOQuery.executeWithArray(JDOQuery.java:267)
> at
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql.executeWithArray(MetaStoreDirectSql.java:1750)
> at
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql.getPrimaryKeys(MetaStoreDirectSql.java:1939)
> at
> org.apache.hadoop.hive.metastore.ObjectStore$11.getSqlResult(ObjectStore.java:8213)
> at
> org.apache.hadoop.hive.metastore.ObjectStore$11.getSqlResult(ObjectStore.java:8209)
> at
> org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:2719)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getPrimaryKeysInternal(ObjectStore.java:8221)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getPrimaryKeys(ObjectStore.java:8199)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:101)
> at com.sun.proxy.$Proxy24.getPr
[jira] [Commented] (HIVE-21639) Spark test failed since HIVE-10632
[
https://issues.apache.org/jira/browse/HIVE-21639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16871781#comment-16871781
]
Alan Gates commented on HIVE-21639:
---
Patch committed to branch-2.
> Spark test failed since HIVE-10632
> --
>
> Key: HIVE-21639
> URL: https://issues.apache.org/jira/browse/HIVE-21639
> Project: Hive
> Issue Type: Bug
>Reporter: Yuming Wang
>Assignee: Yuming Wang
>Priority: Major
> Fix For: 2.4.0, 2.3.5
>
> Attachments: HIVE-21639.branch-2.3.patch, HIVE-21639.branch-2.patch
>
>
> We hint the following exception when [upgrading Spark‘s built-in Hive to
> 2.3.4|https://issues.apache.org/jira/browse/SPARK-23710]:
> {noformat}
> ..
> [info] Cause: java.sql.SQLException: Failed to start database 'metastore_db'
> with class loader
> org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@2439ab23, see
> the next exception for details.
> [info] at
> org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
> [info] at
> org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
> [info] at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
> [info] at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown
> Source)
> [info] at org.apache.derby.impl.jdbc.EmbedConnection.(Unknown Source)
> [info] at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
> ...
> {noformat}
> This issue Introduced by HIVE-10632 and fixed by HIVE-17561, I'd like to
> backport part of HIVE-17561 to fix this issue for branch-2.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21639) Spark test failed since HIVE-10632
[
https://issues.apache.org/jira/browse/HIVE-21639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates updated HIVE-21639:
--
Fix Version/s: 2.4.0
> Spark test failed since HIVE-10632
> --
>
> Key: HIVE-21639
> URL: https://issues.apache.org/jira/browse/HIVE-21639
> Project: Hive
> Issue Type: Bug
>Reporter: Yuming Wang
>Assignee: Yuming Wang
>Priority: Major
> Fix For: 2.4.0, 2.3.5
>
> Attachments: HIVE-21639.branch-2.3.patch, HIVE-21639.branch-2.patch
>
>
> We hint the following exception when [upgrading Spark‘s built-in Hive to
> 2.3.4|https://issues.apache.org/jira/browse/SPARK-23710]:
> {noformat}
> ..
> [info] Cause: java.sql.SQLException: Failed to start database 'metastore_db'
> with class loader
> org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@2439ab23, see
> the next exception for details.
> [info] at
> org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
> [info] at
> org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
> [info] at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
> [info] at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown
> Source)
> [info] at org.apache.derby.impl.jdbc.EmbedConnection.(Unknown Source)
> [info] at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
> ...
> {noformat}
> This issue Introduced by HIVE-10632 and fixed by HIVE-17561, I'd like to
> backport part of HIVE-17561 to fix this issue for branch-2.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17429) Hive JDBC doesn't return rows when querying Impala
[ https://issues.apache.org/jira/browse/HIVE-17429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16871728#comment-16871728 ] Alan Gates commented on HIVE-17429: --- Patch committed to branch-2 > Hive JDBC doesn't return rows when querying Impala > -- > > Key: HIVE-17429 > URL: https://issues.apache.org/jira/browse/HIVE-17429 > Project: Hive > Issue Type: Bug > Components: JDBC >Affects Versions: 2.1.0, 2.2.0, 2.3.0, 2.3.1, 2.3.2 >Reporter: Zach Amsden >Assignee: Zach Amsden >Priority: Major > Fix For: 2.1.0, 2.1.1, 3.0.0, 2.4.0, 2.2.1, 2.3.4 > > Attachments: HIVE-17429.1.patch, HIVE-17429.2.patch > > > The Hive JDBC driver used to return a result set when querying Impala. Now, > instead, it gets data back but interprets the data as query logs instead of a > resultSet. This causes many issues (we see complaints about beeline as well > as test failures). > This appears to be a regression introduced with asynchronous operation > against Hive. > Ideally, we could make both behaviors work. I have a simple patch that > should fix the problem. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17429) Hive JDBC doesn't return rows when querying Impala
[ https://issues.apache.org/jira/browse/HIVE-17429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-17429: -- Fix Version/s: 2.3.4 2.4.0 > Hive JDBC doesn't return rows when querying Impala > -- > > Key: HIVE-17429 > URL: https://issues.apache.org/jira/browse/HIVE-17429 > Project: Hive > Issue Type: Bug > Components: JDBC >Affects Versions: 2.1.0, 2.2.0, 2.3.0, 2.3.1, 2.3.2 >Reporter: Zach Amsden >Assignee: Zach Amsden >Priority: Major > Fix For: 2.1.0, 2.1.1, 3.0.0, 2.4.0, 2.2.1, 2.3.4 > > Attachments: HIVE-17429.1.patch, HIVE-17429.2.patch > > > The Hive JDBC driver used to return a result set when querying Impala. Now, > instead, it gets data back but interprets the data as query logs instead of a > resultSet. This causes many issues (we see complaints about beeline as well > as test failures). > This appears to be a regression introduced with asynchronous operation > against Hive. > Ideally, we could make both behaviors work. I have a simple patch that > should fix the problem. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-16312) Flaky test: TestHCatClient.testTransportFailure
[
https://issues.apache.org/jira/browse/HIVE-16312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates updated HIVE-16312:
--
Fix Version/s: 2.4.0
> Flaky test: TestHCatClient.testTransportFailure
> ---
>
> Key: HIVE-16312
> URL: https://issues.apache.org/jira/browse/HIVE-16312
> Project: Hive
> Issue Type: Sub-task
>Reporter: Barna Zsombor Klara
>Assignee: Barna Zsombor Klara
>Priority: Major
> Fix For: 3.0.0, 2.4.0, 2.3.2
>
> Attachments: HIVE-16312.01-branch-2.3.patch, HIVE-16312.01.patch
>
>
> The test has been failing consistently for 10+ builds.
> Error message:
> {code}
> Error Message
> The expected exception was never thrown.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16312) Flaky test: TestHCatClient.testTransportFailure
[
https://issues.apache.org/jira/browse/HIVE-16312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16871670#comment-16871670
]
Alan Gates commented on HIVE-16312:
---
Committed to branch-2.
> Flaky test: TestHCatClient.testTransportFailure
> ---
>
> Key: HIVE-16312
> URL: https://issues.apache.org/jira/browse/HIVE-16312
> Project: Hive
> Issue Type: Sub-task
>Reporter: Barna Zsombor Klara
>Assignee: Barna Zsombor Klara
>Priority: Major
> Fix For: 3.0.0, 2.4.0, 2.3.2
>
> Attachments: HIVE-16312.01-branch-2.3.patch, HIVE-16312.01.patch
>
>
> The test has been failing consistently for 10+ builds.
> Error message:
> {code}
> Error Message
> The expected exception was never thrown.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17096) Fix test failures in 2.3 branch
[ https://issues.apache.org/jira/browse/HIVE-17096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16871647#comment-16871647 ] Alan Gates commented on HIVE-17096: --- Committed patch to branch-2. > Fix test failures in 2.3 branch > --- > > Key: HIVE-17096 > URL: https://issues.apache.org/jira/browse/HIVE-17096 > Project: Hive > Issue Type: Sub-task >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong >Priority: Major > Fix For: 2.3.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-15144) JSON.org license is now CatX
[
https://issues.apache.org/jira/browse/HIVE-15144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16871638#comment-16871638
]
Alan Gates commented on HIVE-15144:
---
Committed to branch-2
> JSON.org license is now CatX
>
>
> Key: HIVE-15144
> URL: https://issues.apache.org/jira/browse/HIVE-15144
> Project: Hive
> Issue Type: Bug
>Reporter: Robert Kanter
>Assignee: Owen O'Malley
>Priority: Blocker
> Fix For: 2.2.0, 2.3.0, 3.0.0
>
> Attachments: HIVE-15144.patch, HIVE-15144.patch, HIVE-15144.patch,
> HIVE-15144.patch
>
>
> per [update resolved legal|http://www.apache.org/legal/resolved.html#json]:
> {quote}
> CAN APACHE PRODUCTS INCLUDE WORKS LICENSED UNDER THE JSON LICENSE?
> No. As of 2016-11-03 this has been moved to the 'Category X' license list.
> Prior to this, use of the JSON Java library was allowed. See Debian's page
> for a list of alternatives.
> {quote}
> I'm not sure when this dependency was first introduced, but it looks like
> it's currently used in a few places:
> https://github.com/apache/hive/search?p=1&q=%22org.json%22&utf8=%E2%9C%93
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16640) The ASF Headers have some errors in some class
[ https://issues.apache.org/jira/browse/HIVE-16640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16871550#comment-16871550 ] Alan Gates commented on HIVE-16640: --- Commit to branch-2, which should have been done initially when I committed it to branch-2.3 > The ASF Headers have some errors in some class > -- > > Key: HIVE-16640 > URL: https://issues.apache.org/jira/browse/HIVE-16640 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: ZhangBing Lin >Assignee: ZhangBing Lin >Priority: Minor > Fix For: 2.3.0, 3.0.0 > > Attachments: HIVE-16640.1.patch > > > I found some class license hive placed in an incorrect location, some classes > missing license -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21890) Fix alter_partition_change_col.q qtest inclusion in minillaplocal.query.files
[
https://issues.apache.org/jira/browse/HIVE-21890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates updated HIVE-21890:
--
Resolution: Fixed
Fix Version/s: 4.0.0
Status: Resolved (was: Patch Available)
Patch committed to master.
> Fix alter_partition_change_col.q qtest inclusion in minillaplocal.query.files
> -
>
> Key: HIVE-21890
> URL: https://issues.apache.org/jira/browse/HIVE-21890
> Project: Hive
> Issue Type: Bug
>Reporter: David Lavati
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21890.01.patch, HIVE-21890.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> HIVE-20833 introduced alter_partition_change_col.q under
> minillaplocal.query.files, however it was named only
> {{alter_partition_change_col}} without the postfix.
> Looking at the recent precommit tests, it looks like this test never gets
> called.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21741) Backport HIVE-20221 & related fix HIVE-20833 to branch-3: Increase column width for partition_params
[ https://issues.apache.org/jira/browse/HIVE-21741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21741: -- Resolution: Fixed Fix Version/s: (was: 3.1.2) Status: Resolved (was: Patch Available) Patch applied to branch-3. > Backport HIVE-20221 & related fix HIVE-20833 to branch-3: Increase column > width for partition_params > > > Key: HIVE-21741 > URL: https://issues.apache.org/jira/browse/HIVE-21741 > Project: Hive > Issue Type: Bug > Components: Metastore, Standalone Metastore >Affects Versions: 3.1.1 >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0 > > Attachments: HIVE-21741.01.branch-3.patch, > HIVE-21741.01.branch-3.patch, HIVE-21741.02.branch-3.patch, > HIVE-21741.branch-3.patch > > Time Spent: 10m > Remaining Estimate: 0h > > This is an umbrella for backporting HIVE-20221 & the related fix of > HIVE-20833 to branch-3. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21758) DBInstall tests broken on master and branch-3.1
[ https://issues.apache.org/jira/browse/HIVE-21758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16867046#comment-16867046 ] Alan Gates commented on HIVE-21758: --- Cherry picked patch from 3.1 to branch-3. > DBInstall tests broken on master and branch-3.1 > --- > > Key: HIVE-21758 > URL: https://issues.apache.org/jira/browse/HIVE-21758 > Project: Hive > Issue Type: Bug > Components: Metastore, Tests >Affects Versions: 3.1.1 >Reporter: Alan Gates >Assignee: Aditya Shah >Priority: Major > Fix For: 4.0.0, 3.1.2 > > Attachments: HIVE-21758.patch > > > The Oracle and SqlServer install and upgrade tests in standalone-metastore > fail in master and branch-3.1. In the Oracle case it appears the docker > container that was used no longer exists. For SqlServer the cause of the > failures is not immediately clear. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21859) Backport HIVE-17466 (get_partition_values) to branch-2.3
[
https://issues.apache.org/jira/browse/HIVE-21859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates updated HIVE-21859:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
Patch committed to branch-2.3
> Backport HIVE-17466 (get_partition_values) to branch-2.3
>
>
> Key: HIVE-21859
> URL: https://issues.apache.org/jira/browse/HIVE-21859
> Project: Hive
> Issue Type: Improvement
>Reporter: Piotr Findeisen
>Assignee: Piotr Findeisen
>Priority: Major
> Fix For: 2.3.6
>
> Attachments: HIVE-21859-branch-2.3.patch
>
>
> Please backport HIVE-17466 that adds {{get_partition_values}} to
> {{branch-2.3}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21887) Multiple implementations of PersistenceManager are on the classpath
[ https://issues.apache.org/jira/browse/HIVE-21887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16866807#comment-16866807 ] Alan Gates commented on HIVE-21887: --- +1, makes sense to me. > Multiple implementations of PersistenceManager are on the classpath > --- > > Key: HIVE-21887 > URL: https://issues.apache.org/jira/browse/HIVE-21887 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21887.01.patch > > > While writing some test I've just bumped into that PersistenceManager is not > always AutoCloseable ; however when I was using it from the metastore it was. > it turned out that: > * 'standalone-metastore' uses: org.datanucleus:javax.jdo:3.2.0-m3 > * 'metastore' uses: javax.jdo:jdo-api:3.0.1 > the problem is that both dependency contains the jdo api; and sometimes > jdo-api is earlier on the classpath -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21884) Scheduled query support
[ https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16866774#comment-16866774 ] Alan Gates commented on HIVE-21884: --- Given this is a large patch and a big feature is there a design doc or write up somewhere of how this will work? > Scheduled query support > --- > > Key: HIVE-21884 > URL: https://issues.apache.org/jira/browse/HIVE-21884 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21884.01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21859) Backport HIVE-17466 (get_partition_values) to branch-2.3
[
https://issues.apache.org/jira/browse/HIVE-21859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16866037#comment-16866037
]
Alan Gates commented on HIVE-21859:
---
When I run the tests locally all the same tests pass as on branch-2.3.
> Backport HIVE-17466 (get_partition_values) to branch-2.3
>
>
> Key: HIVE-21859
> URL: https://issues.apache.org/jira/browse/HIVE-21859
> Project: Hive
> Issue Type: Improvement
>Reporter: Piotr Findeisen
>Assignee: Piotr Findeisen
>Priority: Major
> Fix For: 2.3.6
>
> Attachments: HIVE-21859-branch-2.3.patch
>
>
> Please backport HIVE-17466 that adds {{get_partition_values}} to
> {{branch-2.3}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21859) Backport HIVE-17466 (get_partition_values) to branch-2.3
[
https://issues.apache.org/jira/browse/HIVE-21859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16865795#comment-16865795
]
Alan Gates commented on HIVE-21859:
---
I'm a little concerned about backporting this as it's adding new functionality,
which we're not supposed to do in a patch release. However, branch-2 is a ways
out of date with branch-2.3 and I'm not aware of anyone working on a 2.4 and
this is just adding a new Thrift call not changing anything else, so I propose
to bend the rules and backport it unless anyone complains.
> Backport HIVE-17466 (get_partition_values) to branch-2.3
>
>
> Key: HIVE-21859
> URL: https://issues.apache.org/jira/browse/HIVE-21859
> Project: Hive
> Issue Type: Improvement
>Reporter: Piotr Findeisen
>Assignee: Piotr Findeisen
>Priority: Major
> Fix For: 2.3.6
>
> Attachments: HIVE-21859-branch-2.3.patch
>
>
> Please backport HIVE-17466 that adds {{get_partition_values}} to
> {{branch-2.3}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21741) Backport HIVE-20221 & related fix HIVE-20833 to branch-3: Increase column width for partition_params
[ https://issues.apache.org/jira/browse/HIVE-21741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16865765#comment-16865765 ] Alan Gates commented on HIVE-21741: --- The change in itests/src/test/resources/testconfiguration.properties is wrong, the filename should have a .q after it, otherwise it causes a failure in the tests. > Backport HIVE-20221 & related fix HIVE-20833 to branch-3: Increase column > width for partition_params > > > Key: HIVE-21741 > URL: https://issues.apache.org/jira/browse/HIVE-21741 > Project: Hive > Issue Type: Bug > Components: Metastore, Standalone Metastore >Affects Versions: 3.1.1 >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0, 3.1.2 > > Attachments: HIVE-21741.01.branch-3.patch, > HIVE-21741.01.branch-3.patch, HIVE-21741.branch-3.patch > > Time Spent: 10m > Remaining Estimate: 0h > > This is an umbrella for backporting HIVE-20221 & the related fix of > HIVE-20833 to branch-3. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21741) Backport HIVE-20221 & related fix HIVE-20833 to branch-3: Increase column width for partition_params
[ https://issues.apache.org/jira/browse/HIVE-21741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21741: -- Status: Open (was: Patch Available) > Backport HIVE-20221 & related fix HIVE-20833 to branch-3: Increase column > width for partition_params > > > Key: HIVE-21741 > URL: https://issues.apache.org/jira/browse/HIVE-21741 > Project: Hive > Issue Type: Bug > Components: Metastore, Standalone Metastore >Affects Versions: 3.1.1 >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0, 3.1.2 > > Attachments: HIVE-21741.01.branch-3.patch, > HIVE-21741.01.branch-3.patch, HIVE-21741.branch-3.patch > > Time Spent: 10m > Remaining Estimate: 0h > > This is an umbrella for backporting HIVE-20221 & the related fix of > HIVE-20833 to branch-3. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19661) switch Hive UDFs to use Re2J regex engine
[ https://issues.apache.org/jira/browse/HIVE-19661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16864218#comment-16864218 ] Alan Gates commented on HIVE-19661: --- This is the BSD 3 clause license which is permissible per [https://apache.org/legal/resolved.html] > switch Hive UDFs to use Re2J regex engine > - > > Key: HIVE-19661 > URL: https://issues.apache.org/jira/browse/HIVE-19661 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Rajkumar Singh >Assignee: Rajkumar Singh >Priority: Major > Labels: pull-request-available > Attachments: HIVE-19661.01.patch, HIVE-19661.02.patch, > HIVE-19661.03.patch, HIVE-19661.patch > > > Java regex engine can be very slow in some cases e.g. > https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8203458 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21850) branch-3 metastore installation installs wrong version
[ https://issues.apache.org/jira/browse/HIVE-21850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21850: -- Resolution: Fixed Status: Resolved (was: Patch Available) Patch committed to branch-3. Thanks Owen for the review. > branch-3 metastore installation installs wrong version > -- > > Key: HIVE-21850 > URL: https://issues.apache.org/jira/browse/HIVE-21850 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 3.2.0 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-21850-branch-3.patch > > > hive.version.shortname in standalone-metastore/pom.xml was not properly > updated in branch-3. It is still set to 3.1.0, which causes the > MetastoreSchemaTool to install the wrong version. Part of this Jira should > include updating the HowToRelease doc to include updating this value. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21859) Backport HIVE-17466 (get_partition_values) to branch-2.3
[
https://issues.apache.org/jira/browse/HIVE-21859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16861360#comment-16861360
]
Alan Gates commented on HIVE-21859:
---
[https://cwiki.apache.org/confluence/display/Hive/Hive+PreCommit+Patch+Testing]
covers how to name your patches. You'll also need to mark the Jira Patch
Available.
> Backport HIVE-17466 (get_partition_values) to branch-2.3
>
>
> Key: HIVE-21859
> URL: https://issues.apache.org/jira/browse/HIVE-21859
> Project: Hive
> Issue Type: Improvement
>Reporter: Piotr Findeisen
>Priority: Major
> Attachments: HIVE-21859-branch-2.3.patch
>
>
> Please backport HIVE-17466 that adds {{get_partition_values}} to
> {{branch-2.3}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21854) JDBC: Standalone Jar Breaks Spring Boot
[ https://issues.apache.org/jira/browse/HIVE-21854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860367#comment-16860367 ] Alan Gates commented on HIVE-21854: --- Alright, makes sense. I just saw the pom.xml and assumed it was patch with the changes. > JDBC: Standalone Jar Breaks Spring Boot > --- > > Key: HIVE-21854 > URL: https://issues.apache.org/jira/browse/HIVE-21854 > Project: Hive > Issue Type: Bug > Components: JDBC >Affects Versions: 3.0.0, 3.1.0, 3.1.1 >Reporter: Shawn Weeks >Priority: Major > Attachments: pom.xml > > > Much like HIVE-21417 since Hive 3.x the JDBC Standalone Jar includes > dependencies incompatible with the version of Tomcat included with Spring > Boot. Attached is an example Pom that shows the issue. Just place it in a > folder and run "mvn spring-boot:run" -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21854) JDBC: Standalone Jar Breaks Spring Boot
[ https://issues.apache.org/jira/browse/HIVE-21854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860308#comment-16860308 ] Alan Gates commented on HIVE-21854: --- [~Absolutesantaja], if you want the CI tests to run against this mark the Jira patch available (using the Submit Patch button above). > JDBC: Standalone Jar Breaks Spring Boot > --- > > Key: HIVE-21854 > URL: https://issues.apache.org/jira/browse/HIVE-21854 > Project: Hive > Issue Type: Bug > Components: JDBC >Affects Versions: 3.0.0, 3.1.0, 3.1.1 >Reporter: Shawn Weeks >Priority: Major > Attachments: pom.xml > > > Much like HIVE-21417 since Hive 3.x the JDBC Standalone Jar includes > dependencies incompatible with the version of Tomcat included with Spring > Boot. Attached is an example Pom that shows the issue. Just place it in a > folder and run "mvn spring-boot:run" -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21850) branch-3 metastore installation installs wrong version
[ https://issues.apache.org/jira/browse/HIVE-21850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21850: -- Status: Patch Available (was: Open) Patch to update hive.version.shortname. I ran the tests locally and saw the same pass rate as against branch-3. I also updated the HowToRelease doc to add changing these values to avoid having this issue in the future. > branch-3 metastore installation installs wrong version > -- > > Key: HIVE-21850 > URL: https://issues.apache.org/jira/browse/HIVE-21850 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 3.2.0 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-21850-branch-3.patch > > > hive.version.shortname in standalone-metastore/pom.xml was not properly > updated in branch-3. It is still set to 3.1.0, which causes the > MetastoreSchemaTool to install the wrong version. Part of this Jira should > include updating the HowToRelease doc to include updating this value. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21850) branch-3 metastore installation installs wrong version
[ https://issues.apache.org/jira/browse/HIVE-21850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21850: -- Attachment: HIVE-21850-branch-3.patch > branch-3 metastore installation installs wrong version > -- > > Key: HIVE-21850 > URL: https://issues.apache.org/jira/browse/HIVE-21850 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 3.2.0 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-21850-branch-3.patch > > > hive.version.shortname in standalone-metastore/pom.xml was not properly > updated in branch-3. It is still set to 3.1.0, which causes the > MetastoreSchemaTool to install the wrong version. Part of this Jira should > include updating the HowToRelease doc to include updating this value. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21850) branch-3 metastore installation installs wrong version
[ https://issues.apache.org/jira/browse/HIVE-21850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21850: -- Attachment: (was: HIVE-21850-branch-3.patch) > branch-3 metastore installation installs wrong version > -- > > Key: HIVE-21850 > URL: https://issues.apache.org/jira/browse/HIVE-21850 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 3.2.0 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Fix For: 3.2.0 > > > hive.version.shortname in standalone-metastore/pom.xml was not properly > updated in branch-3. It is still set to 3.1.0, which causes the > MetastoreSchemaTool to install the wrong version. Part of this Jira should > include updating the HowToRelease doc to include updating this value. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21850) branch-3 metastore installation installs wrong version
[ https://issues.apache.org/jira/browse/HIVE-21850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21850: -- Attachment: HIVE-21850-branch-3.patch > branch-3 metastore installation installs wrong version > -- > > Key: HIVE-21850 > URL: https://issues.apache.org/jira/browse/HIVE-21850 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 3.2.0 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-21850-branch-3.patch > > > hive.version.shortname in standalone-metastore/pom.xml was not properly > updated in branch-3. It is still set to 3.1.0, which causes the > MetastoreSchemaTool to install the wrong version. Part of this Jira should > include updating the HowToRelease doc to include updating this value. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21850) branch-3 metastore installation installs wrong version
[ https://issues.apache.org/jira/browse/HIVE-21850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates reassigned HIVE-21850: - > branch-3 metastore installation installs wrong version > -- > > Key: HIVE-21850 > URL: https://issues.apache.org/jira/browse/HIVE-21850 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 3.2.0 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Fix For: 3.2.0 > > > hive.version.shortname in standalone-metastore/pom.xml was not properly > updated in branch-3. It is still set to 3.1.0, which causes the > MetastoreSchemaTool to install the wrong version. Part of this Jira should > include updating the HowToRelease doc to include updating this value. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21826) Backport HIVE-21786 to branch-3
[ https://issues.apache.org/jira/browse/HIVE-21826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21826: -- Resolution: Fixed Fix Version/s: 3.2.0 Status: Resolved (was: Patch Available) Patch committed to branch-3 > Backport HIVE-21786 to branch-3 > --- > > Key: HIVE-21826 > URL: https://issues.apache.org/jira/browse/HIVE-21826 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Affects Versions: 3.2.0 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-21826.patch > > > Missed branch-3 in the original fix. Need to apply the patch here as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21821) Backport HIVE-21739 to branch-3.1
[ https://issues.apache.org/jira/browse/HIVE-21821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21821: -- Resolution: Fixed Fix Version/s: 3.1.2 Status: Resolved (was: Patch Available) Patch committed to branch-3.1 > Backport HIVE-21739 to branch-3.1 > - > > Key: HIVE-21821 > URL: https://issues.apache.org/jira/browse/HIVE-21821 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.1 >Reporter: Aditya Shah >Assignee: Aditya Shah >Priority: Major > Fix For: 3.1.2 > > Attachments: HIVE-21821.branch-3.1.1.patch, > HIVE-21821.branch-3.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21826) Backport HIVE-21786 to branch-3
[ https://issues.apache.org/jira/browse/HIVE-21826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21826: -- Status: Patch Available (was: Open) > Backport HIVE-21786 to branch-3 > --- > > Key: HIVE-21826 > URL: https://issues.apache.org/jira/browse/HIVE-21826 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Affects Versions: 3.2.0 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Attachments: HIVE-21826.patch > > > Missed branch-3 in the original fix. Need to apply the patch here as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21826) Backport HIVE-21786 to branch-3
[ https://issues.apache.org/jira/browse/HIVE-21826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21826: -- Attachment: HIVE-21826.patch > Backport HIVE-21786 to branch-3 > --- > > Key: HIVE-21826 > URL: https://issues.apache.org/jira/browse/HIVE-21826 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Affects Versions: 3.2.0 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Attachments: HIVE-21826.patch > > > Missed branch-3 in the original fix. Need to apply the patch here as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21826) Backport HIVE-21786 to branch-3
[ https://issues.apache.org/jira/browse/HIVE-21826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21826: -- Affects Version/s: 3.2.0 > Backport HIVE-21786 to branch-3 > --- > > Key: HIVE-21826 > URL: https://issues.apache.org/jira/browse/HIVE-21826 > Project: Hive > Issue Type: Bug >Affects Versions: 3.2.0 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > > Missed branch-3 in the original fix. Need to apply the patch here as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21826) Backport HIVE-21786 to branch-3
[ https://issues.apache.org/jira/browse/HIVE-21826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates reassigned HIVE-21826: - > Backport HIVE-21786 to branch-3 > --- > > Key: HIVE-21826 > URL: https://issues.apache.org/jira/browse/HIVE-21826 > Project: Hive > Issue Type: Bug >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > > Missed branch-3 in the original fix. Need to apply the patch here as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21826) Backport HIVE-21786 to branch-3
[ https://issues.apache.org/jira/browse/HIVE-21826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21826: -- Component/s: Build Infrastructure > Backport HIVE-21786 to branch-3 > --- > > Key: HIVE-21826 > URL: https://issues.apache.org/jira/browse/HIVE-21826 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Affects Versions: 3.2.0 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > > Missed branch-3 in the original fix. Need to apply the patch here as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21821) Backport HIVE-21739 to branch-3.1
[ https://issues.apache.org/jira/browse/HIVE-21821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21821: -- Status: Open (was: Patch Available) > Backport HIVE-21739 to branch-3.1 > - > > Key: HIVE-21821 > URL: https://issues.apache.org/jira/browse/HIVE-21821 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.1 >Reporter: Aditya Shah >Assignee: Aditya Shah >Priority: Major > Attachments: HIVE-21821.branch-3.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21821) Backport HIVE-21739 to branch-3.1
[
https://issues.apache.org/jira/browse/HIVE-21821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16855083#comment-16855083
]
Alan Gates commented on HIVE-21821:
---
When I run apply this patch and run TestSchemaToolForMetastore test I see:
{{[INFO] Running
org.apache.hadoop.hive.metastore.tools.TestSchemaToolForMetastore}}
{{[ERROR] Tests run: 11, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
58.965 s <<< FAILURE! - in
org.apache.hadoop.hive.metastore.tools.TestSchemaToolForMetastore}}
{{[ERROR]
testValidateLocations(org.apache.hadoop.hive.metastore.tools.TestSchemaToolForMetastore)
Time elapsed: 5.398 s <<< ERROR!}}
{{java.io.IOException: Schema script failed, errorcode OTHER}}
{{ at
org.apache.hadoop.hive.metastore.tools.TestSchemaToolForMetastore.testValidateLocations(TestSchemaToolForMetastore.java:358)}}
> Backport HIVE-21739 to branch-3.1
> -
>
> Key: HIVE-21821
> URL: https://issues.apache.org/jira/browse/HIVE-21821
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.1
>Reporter: Aditya Shah
>Assignee: Aditya Shah
>Priority: Major
> Attachments: HIVE-21821.branch-3.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21809) Backport HIVE-21786 to branch-2.3
[ https://issues.apache.org/jira/browse/HIVE-21809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21809: -- Resolution: Fixed Status: Resolved (was: Patch Available) Patch committed to branch-2.3 > Backport HIVE-21786 to branch-2.3 > - > > Key: HIVE-21809 > URL: https://issues.apache.org/jira/browse/HIVE-21809 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Affects Versions: 2.3.5 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Fix For: 2.3.6 > > Attachments: HIVE-21786-branch-2.3.patch > > > Need to update the URLs in the pom on branch-2.3 as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21808) Backport HIVE-21786 to branch-3.1
[ https://issues.apache.org/jira/browse/HIVE-21808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21808: -- Resolution: Fixed Status: Resolved (was: Patch Available) Patch committed to branch-3.1 > Backport HIVE-21786 to branch-3.1 > - > > Key: HIVE-21808 > URL: https://issues.apache.org/jira/browse/HIVE-21808 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Affects Versions: 3.1.1 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Fix For: 3.1.2 > > Attachments: HIVE-21786-branch-3.1.patch > > > Need to update the URLs in the poms for branch 3 as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21809) Backport HIVE-21786 to branch-2.3
[ https://issues.apache.org/jira/browse/HIVE-21809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21809: -- Status: Patch Available (was: Open) Ran the test locally and see same test results and when I build branch-2.3 without the patch. > Backport HIVE-21786 to branch-2.3 > - > > Key: HIVE-21809 > URL: https://issues.apache.org/jira/browse/HIVE-21809 > Project: Hive > Issue Type: Bug >Affects Versions: 2.3.5 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Fix For: 2.3.6 > > Attachments: HIVE-21786-branch-2.3.patch > > > Need to update the URLs in the pom on branch-2.3 as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21809) Backport HIVE-21786 to branch-2.3
[ https://issues.apache.org/jira/browse/HIVE-21809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21809: -- Component/s: Build Infrastructure > Backport HIVE-21786 to branch-2.3 > - > > Key: HIVE-21809 > URL: https://issues.apache.org/jira/browse/HIVE-21809 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Affects Versions: 2.3.5 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Fix For: 2.3.6 > > Attachments: HIVE-21786-branch-2.3.patch > > > Need to update the URLs in the pom on branch-2.3 as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21809) Backport HIVE-21786 to branch-2.3
[ https://issues.apache.org/jira/browse/HIVE-21809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21809: -- Attachment: HIVE-21786-branch-2.3.patch > Backport HIVE-21786 to branch-2.3 > - > > Key: HIVE-21809 > URL: https://issues.apache.org/jira/browse/HIVE-21809 > Project: Hive > Issue Type: Bug >Affects Versions: 2.3.5 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Fix For: 2.3.6 > > Attachments: HIVE-21786-branch-2.3.patch > > > Need to update the URLs in the pom on branch-2.3 as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21808) Backport HIVE-21786 to branch-3.1
[ https://issues.apache.org/jira/browse/HIVE-21808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21808: -- Status: Patch Available (was: Open) Have run the tests locally and I see the same results as running on branch-3.1 without the patch. > Backport HIVE-21786 to branch-3.1 > - > > Key: HIVE-21808 > URL: https://issues.apache.org/jira/browse/HIVE-21808 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Affects Versions: 3.1.1 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Fix For: 3.1.2 > > Attachments: HIVE-21786-branch-3.1.patch > > > Need to update the URLs in the poms for branch 3 as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21808) Backport HIVE-21786 to branch-3.1
[ https://issues.apache.org/jira/browse/HIVE-21808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21808: -- Component/s: Build Infrastructure > Backport HIVE-21786 to branch-3.1 > - > > Key: HIVE-21808 > URL: https://issues.apache.org/jira/browse/HIVE-21808 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Affects Versions: 3.1.1 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Fix For: 3.1.2 > > Attachments: HIVE-21786-branch-3.1.patch > > > Need to update the URLs in the poms for branch 3 as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21808) Backport HIVE-21786 to branch-3.1
[ https://issues.apache.org/jira/browse/HIVE-21808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21808: -- Affects Version/s: 3.1.1 > Backport HIVE-21786 to branch-3.1 > - > > Key: HIVE-21808 > URL: https://issues.apache.org/jira/browse/HIVE-21808 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.1 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Fix For: 3.1.2 > > Attachments: HIVE-21786-branch-3.1.patch > > > Need to update the URLs in the poms for branch 3 as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21809) Backport HIVE-21786 to branch-2.3
[ https://issues.apache.org/jira/browse/HIVE-21809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates reassigned HIVE-21809: - > Backport HIVE-21786 to branch-2.3 > - > > Key: HIVE-21809 > URL: https://issues.apache.org/jira/browse/HIVE-21809 > Project: Hive > Issue Type: Bug >Affects Versions: 2.3.5 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Fix For: 2.3.6 > > > Need to update the URLs in the pom on branch-2.3 as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21808) Backport HIVE-21786 to branch-3.1
[ https://issues.apache.org/jira/browse/HIVE-21808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21808: -- Attachment: HIVE-21786-branch-3.1.patch > Backport HIVE-21786 to branch-3.1 > - > > Key: HIVE-21808 > URL: https://issues.apache.org/jira/browse/HIVE-21808 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Affects Versions: 3.1.1 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Fix For: 3.1.2 > > Attachments: HIVE-21786-branch-3.1.patch > > > Need to update the URLs in the poms for branch 3 as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21808) Backport HIVE-21786 to branch-3.1
[ https://issues.apache.org/jira/browse/HIVE-21808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates reassigned HIVE-21808: - > Backport HIVE-21786 to branch-3.1 > - > > Key: HIVE-21808 > URL: https://issues.apache.org/jira/browse/HIVE-21808 > Project: Hive > Issue Type: Bug >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Fix For: 3.1.2 > > > Need to update the URLs in the poms for branch 3 as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21786) Update repo URLs in poms
[ https://issues.apache.org/jira/browse/HIVE-21786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-21786: -- Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) > Update repo URLs in poms > > > Key: HIVE-21786 > URL: https://issues.apache.org/jira/browse/HIVE-21786 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Affects Versions: 4.0.0, 3.1.1, 2.3.5 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21786.patch > > > Need to update repo URLs in the poms. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21786) Update repo URLs in poms
[ https://issues.apache.org/jira/browse/HIVE-21786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16852261#comment-16852261 ] Alan Gates commented on HIVE-21786: --- [~pvary], yes I ran it with a clean repo (though using -Dmaven.repo.local rather than cleaning .m2, as that screws up all your builds). Thanks for the reviews Gopal and Peter. I've committed the patch to master. > Update repo URLs in poms > > > Key: HIVE-21786 > URL: https://issues.apache.org/jira/browse/HIVE-21786 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Affects Versions: 4.0.0, 3.1.1, 2.3.5 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Attachments: HIVE-21786.patch > > > Need to update repo URLs in the poms. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21739) Make metastore DB backward compatible with pre-catalog versions of hive.
[
https://issues.apache.org/jira/browse/HIVE-21739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates updated HIVE-21739:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
Patch 4 committed to master.
> Make metastore DB backward compatible with pre-catalog versions of hive.
>
>
> Key: HIVE-21739
> URL: https://issues.apache.org/jira/browse/HIVE-21739
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 1.2.0, 2.1.1
>Reporter: Aditya Shah
>Assignee: Aditya Shah
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21739.1.patch, HIVE-21739.2.patch,
> HIVE-21739.3.patch, HIVE-21739.4.patch, HIVE-21739.patch
>
>
> Since the addition of foreign key constraint between Database ('DBS') table
> and catalogs ('CTLGS') table in HIVE-18755 we are unable to run a simple
> create database command with an older version of Metastore Server. This is
> due to older versions having JDO schema as per older schema of 'DBS' which
> did not have an additional 'CTLG_NAME' column.
> The error is as follows:
> {code:java}
> org.apache.hadoop.hive.ql.metadata.HiveException:
> MetaException(message:Exception thrown flushing changes to datastore)
>
> java.sql.BatchUpdateException: Cannot add or update a child row: a foreign
> key constraint fails ("metastore_1238"."DBS", CONSTRAINT "CTLG_FK1" FOREIGN
> KEY ("CTLG_NAME") REFERENCES "CTLGS" ("NAME"))
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
