[jira] [Updated] (HIVE-7145) Remove dependence on apache commons-lang
[ https://issues.apache.org/jira/browse/HIVE-7145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-7145: --- Fix Version/s: 2.3.10 > Remove dependence on apache commons-lang > > > Key: HIVE-7145 > URL: https://issues.apache.org/jira/browse/HIVE-7145 > Project: Hive > Issue Type: Bug >Reporter: Owen O'Malley >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > Fix For: 2.3.10, 4.0.0-alpha-1 > > Attachments: HIVE-7145.patch > > Time Spent: 20m > Remaining Estimate: 0h > > We currently depend on both Apache commons-lang and commons-lang3. They are > the same project, just at version 2.x vs 3.x. I propose that we move all of > the references in Hive to commons-lang3 and remove the v2 usage. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-20016) Investigate TestJdbcWithMiniHS2.testParallelCompilation3 random failure
[ https://issues.apache.org/jira/browse/HIVE-20016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-20016: Fix Version/s: 2.3.10 > Investigate TestJdbcWithMiniHS2.testParallelCompilation3 random failure > --- > > Key: HIVE-20016 > URL: https://issues.apache.org/jira/browse/HIVE-20016 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 4.0.0 >Reporter: Yongzhi Chen >Assignee: Yongzhi Chen >Priority: Major > Labels: pull-request-available > Fix For: 2.3.10 > > Attachments: HIVE-20016.1.patch > > > org.apache.hive.jdbc.TestJdbcWithMiniHS2.testParallelCompilation3 failed with: > java.lang.AssertionError: Concurrent Statement failed: > org.apache.hive.service.cli.HiveSQLException: java.lang.AssertionError: > Authorization plugins not initialized! > at org.junit.Assert.fail(Assert.java:88) > at > org.apache.hive.jdbc.TestJdbcWithMiniHS2.finishTasks(TestJdbcWithMiniHS2.java:374) > at > org.apache.hive.jdbc.TestJdbcWithMiniHS2.testParallelCompilation3(TestJdbcWithMiniHS2.java:304) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-28121) Use direct SQL for transactional altering table parameter
[ https://issues.apache.org/jira/browse/HIVE-28121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-28121: Fix Version/s: 2.3.10 > Use direct SQL for transactional altering table parameter > - > > Key: HIVE-28121 > URL: https://issues.apache.org/jira/browse/HIVE-28121 > Project: Hive > Issue Type: Bug > Components: Iceberg integration >Affects Versions: 4.0.0 >Reporter: Rui Li >Assignee: Rui Li >Priority: Major > Labels: hive-4.0.1-must, pull-request-available > Fix For: 2.3.10, 4.1.0 > > > Follow up of HIVE-26882, where more details can be found in the discussions > there. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-25665) Checkstyle LGPL files must not be in the release sources/binaries
[ https://issues.apache.org/jira/browse/HIVE-25665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-25665: Fix Version/s: 2.3.10 > Checkstyle LGPL files must not be in the release sources/binaries > - > > Key: HIVE-25665 > URL: https://issues.apache.org/jira/browse/HIVE-25665 > Project: Hive > Issue Type: Task > Components: Build Infrastructure >Affects Versions: 0.6.0 >Reporter: Stamatis Zampetakis >Assignee: Peter Vary >Priority: Blocker > Labels: pull-request-available > Fix For: 3.2.0, 2.3.10, 4.0.0-alpha-1 > > Time Spent: 0.5h > Remaining Estimate: 0h > > As discussed in the [dev > list|https://lists.apache.org/thread/r13e3236aa72a070b3267ed95f7cb3b45d3c4783fd4ca35f5376b1a35@%3cdev.hive.apache.org%3e] > LGPL files must not be present in the Apache released sources/binaries. > The following files must not be present in the release: > https://github.com/apache/hive/blob/6e152aa28bc5116bf9210f9deb0f95d2d73183f7/checkstyle/checkstyle-noframes-sorted.xsl > https://github.com/apache/hive/blob/6e152aa28bc5116bf9210f9deb0f95d2d73183f7/storage-api/checkstyle/checkstyle-noframes-sorted.xsl > https://github.com/apache/hive/blob/6e152aa28bc5116bf9210f9deb0f95d2d73183f7/standalone-metastore/checkstyle/checkstyle-noframes-sorted.xsl > There may be other checkstyle LGPL files in the repo. All these should either > be removed entirely from the repository or selectively excluded from the > release. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-25054) Upgrade jodd-core due to CVE-2018-21234
[ https://issues.apache.org/jira/browse/HIVE-25054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-25054: Fix Version/s: 2.3.10 > Upgrade jodd-core due to CVE-2018-21234 > --- > > Key: HIVE-25054 > URL: https://issues.apache.org/jira/browse/HIVE-25054 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Reporter: Abhay >Assignee: Abhay >Priority: Major > Labels: pull-request-available > Fix For: 2.3.10, 4.0.0-alpha-1 > > > Hive makes use of 3.5.2 version of the `jodd-core` library which is > susceptible to CVE-2018-21234. Below is a description of that vulnerability. > CVE-2018-21234 suppress > Jodd before 5.0.4 performs Deserialization of Untrusted JSON Data when > setClassMetadataName is set. > CWE-502 Deserialization of Untrusted Data > CVSSv2: > Base Score: HIGH (7.5) > Vector: /AV:N/AC:L/Au:N/C:P/I:P/A:P > CVSSv3: > Base Score: CRITICAL (9.8) > Vector: CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H > References: > MISC - > https://github.com/oblac/jodd/commit/9bffc3913aeb8472c11bb543243004b4b4376f16MISC > - https://github.com/oblac/jodd/compare/v5.0.3...v5.0.4MISC - > https://github.com/oblac/jodd/issues/628Vulnerable Software & Versions: > cpe:2.3:a:jodd:jodd:*:*:*:*:*:*:*:* versions up to (excluding) 5.0.4 > > This library needs to be upgraded. We use a couple of classes > `JDateTime`([https://github.infra.cloudera.com/CDH/hive/blob/cdpd-master/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/timestamp/NanoTimeUtils.java] > ) and `HtmlEncoder`, which have either been deprecated and/or have been > moved to a different package called jodd-util. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27478) Remove re2j, SQLLine, ClassNameCompleter entries from LICENSE file
[ https://issues.apache.org/jira/browse/HIVE-27478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-27478: Fix Version/s: 2.3.10 > Remove re2j, SQLLine, ClassNameCompleter entries from LICENSE file > -- > > Key: HIVE-27478 > URL: https://issues.apache.org/jira/browse/HIVE-27478 > Project: Hive > Issue Type: Task >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Blocker > Labels: pull-request-available > Fix For: 2.3.10, 4.0.0-beta-1 > > > com.google.re2j:re2j is not bundled in source packages (only in the binary). > Copy license (from https://github.com/google/re2j/blob/re2j-1.2/LICENSE) to > binary-package-licenses. > SQLLine is not bundled in source packages (only in binary). Copy license > (from https://github.com/julianhyde/sqlline/blob/sqlline-1.9.0/LICENSE) to > binary-package-licenses. > The class ClassNameCompleter is no longer present anywhere in the Hive repo. > As a reminder anything that is not bundled should not be in the LICENSE: > https://infra.apache.org/licensing-howto.html#bundled-vs-non-bundled -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27468) Restore original license in PriorityBlockingDeque
[ https://issues.apache.org/jira/browse/HIVE-27468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HIVE-27468. - Fix Version/s: 2.3.10 Resolution: Fixed > Restore original license in PriorityBlockingDeque > - > > Key: HIVE-27468 > URL: https://issues.apache.org/jira/browse/HIVE-27468 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Major > Labels: pull-request-available > Fix For: 2.3.10 > > > The org.apache.hadoop.hive.llap.daemon.impl.PriorityBlockingDeque was copied > from https://code.google.com/archive/p/infomancers-collections/ as part of > HIVE-10662. > However, the file was modified with most notable modification the the license > entry that was removed and replaced by the ASF header. > This class being a 3-party work should adhere to the following guidelines: > https://www.apache.org/legal/src-headers.html#3party > To comply with the guidelines above at the bare minimum we should remove the > AL2 header and restore the original copyright in the file. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-28104) Move HTTP related methods from Utils to HttpUtils in shims
[ https://issues.apache.org/jira/browse/HIVE-28104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HIVE-28104. - Fix Version/s: 2.3.10 Assignee: Cheng Pan Resolution: Fixed > Move HTTP related methods from Utils to HttpUtils in shims > -- > > Key: HIVE-28104 > URL: https://issues.apache.org/jira/browse/HIVE-28104 > Project: Hive > Issue Type: Improvement >Affects Versions: 2.3.9 >Reporter: Cheng Pan >Assignee: Cheng Pan >Priority: Major > Labels: pull-request-available > Fix For: 2.3.10 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27467) NOTICE files use incorrect wording "developed by" instead of "developed at"
[ https://issues.apache.org/jira/browse/HIVE-27467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-27467: Fix Version/s: 2.3.10 > NOTICE files use incorrect wording "developed by" instead of "developed at" > --- > > Key: HIVE-27467 > URL: https://issues.apache.org/jira/browse/HIVE-27467 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Blocker > Labels: pull-request-available > Fix For: 2.3.10, 4.0.0-beta-1 > > > The NOTICE files currently use the incorrect wording "developed by The Apache > Software Foundation". > As it can be shown in the following links (and also from other JIRA tickets) > the correct wording is "developed at". > https://www.apache.org/legal/src-headers.html#notice > https://www.apache.org/legal/release-policy.html#notice-required -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27172) Add the HMS client connection timeout config
[
https://issues.apache.org/jira/browse/HIVE-27172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated HIVE-27172:
Fix Version/s: 2.3.10
> Add the HMS client connection timeout config
>
>
> Key: HIVE-27172
> URL: https://issues.apache.org/jira/browse/HIVE-27172
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.3.10, 4.0.0-beta-1
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> Currently {{HiveMetastoreClient}} use {{CLIENT_SOCKET_TIMEOUT}} as both
> socket timeout and connection timeout, it's not convenient for users to set a
> smaller connection timeout.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-18586) Upgrade Derby to 10.14.1.0
[ https://issues.apache.org/jira/browse/HIVE-18586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-18586: Fix Version/s: 2.3.10 > Upgrade Derby to 10.14.1.0 > -- > > Key: HIVE-18586 > URL: https://issues.apache.org/jira/browse/HIVE-18586 > Project: Hive > Issue Type: Improvement >Reporter: Janaki Lahorani >Assignee: Janaki Lahorani >Priority: Major > Labels: pull-request-available > Fix For: 3.0.0, 2.3.10 > > Attachments: HIVE-18586.1.patch, HIVE-18586.2.patch, > HIVE-18586.3.patch, HIVE-18586.4.patch, HIVE-18586.5.patch > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27817) Disable ssl hostname verification for 127.0.0.1
[
https://issues.apache.org/jira/browse/HIVE-27817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun resolved HIVE-27817.
-
Fix Version/s: 2.3.10
Assignee: Yuming Wang
Resolution: Fixed
> Disable ssl hostname verification for 127.0.0.1
> ---
>
> Key: HIVE-27817
> URL: https://issues.apache.org/jira/browse/HIVE-27817
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 2.3.0
>Reporter: Yuming Wang
>Assignee: Yuming Wang
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.3.10
>
>
> {code:java}
> diff --git
> a/common/src/java/org/apache/hadoop/hive/common/auth/HiveAuthUtils.java
> b/common/src/java/org/apache/hadoop/hive/common/auth/HiveAuthUtils.java
> index e12f245871..632980e7cd 100644
> --- a/common/src/java/org/apache/hadoop/hive/common/auth/HiveAuthUtils.java
> +++ b/common/src/java/org/apache/hadoop/hive/common/auth/HiveAuthUtils.java
> @@ -71,7 +71,11 @@ public static TTransport getSSLSocket(String host, int
> port, int loginTimeout,
>private static TSocket getSSLSocketWithHttps(TSocket tSSLSocket) throws
> TTransportException {
> SSLSocket sslSocket = (SSLSocket) tSSLSocket.getSocket();
> SSLParameters sslParams = sslSocket.getSSLParameters();
> -sslParams.setEndpointIdentificationAlgorithm("HTTPS");
> +if (sslSocket.getLocalAddress().getHostAddress().equals("127.0.0.1")) {
> + sslParams.setEndpointIdentificationAlgorithm(null);
> +} else {
> + sslParams.setEndpointIdentificationAlgorithm("HTTPS");
> +}
> sslSocket.setSSLParameters(sslParams);
> return new TSocket(sslSocket);
>}
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-26173) Upgrade derby to 10.14.2.0
[ https://issues.apache.org/jira/browse/HIVE-26173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-26173: Fix Version/s: 2.3.10 > Upgrade derby to 10.14.2.0 > -- > > Key: HIVE-26173 > URL: https://issues.apache.org/jira/browse/HIVE-26173 > Project: Hive > Issue Type: Improvement >Reporter: Hemanth Boyina >Assignee: Hemanth Boyina >Priority: Major > Labels: hive-3.2.0-must, pull-request-available > Fix For: 2.3.10, 4.0.0-alpha-2 > > Time Spent: 40m > Remaining Estimate: 0h > > upgrade derby from 10.14.1.0 to 10.14.2.0, to fix the vulnerability > CVE-2018-1313 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27815) Support collect numModifiedRows
[ https://issues.apache.org/jira/browse/HIVE-27815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HIVE-27815. - Fix Version/s: 2.3.10 Assignee: Yuming Wang Resolution: Fixed > Support collect numModifiedRows > --- > > Key: HIVE-27815 > URL: https://issues.apache.org/jira/browse/HIVE-27815 > Project: Hive > Issue Type: Improvement >Affects Versions: 2.3.8 >Reporter: Yuming Wang >Assignee: Yuming Wang >Priority: Major > Labels: pull-request-available > Fix For: 2.3.10 > > > Backport part of HIVE-14388. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27580) Backport HIVE-20071: Migrate to jackson 2.x and prevent usage
[ https://issues.apache.org/jira/browse/HIVE-27580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HIVE-27580. - Fix Version/s: 2.3.10 Assignee: Yuming Wang Resolution: Fixed > Backport HIVE-20071: Migrate to jackson 2.x and prevent usage > - > > Key: HIVE-27580 > URL: https://issues.apache.org/jira/browse/HIVE-27580 > Project: Hive > Issue Type: Task >Reporter: Yuming Wang >Assignee: Yuming Wang >Priority: Major > Labels: pull-request-available > Fix For: 2.3.10 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27560) Enhancing compatibility with Guava
[ https://issues.apache.org/jira/browse/HIVE-27560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HIVE-27560. - Fix Version/s: 2.3.10 Assignee: Yang Jie Resolution: Fixed > Enhancing compatibility with Guava > -- > > Key: HIVE-27560 > URL: https://issues.apache.org/jira/browse/HIVE-27560 > Project: Hive > Issue Type: Improvement > Components: Build Infrastructure >Affects Versions: 2.3.10 >Reporter: Yang Jie >Assignee: Yang Jie >Priority: Major > Labels: pull-request-available > Fix For: 2.3.10 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-27922) Backport HMS client changes from HIVE-18755
Chao Sun created HIVE-27922: --- Summary: Backport HMS client changes from HIVE-18755 Key: HIVE-27922 URL: https://issues.apache.org/jira/browse/HIVE-27922 Project: Hive Issue Type: Improvement Reporter: Chao Sun This is a follow-up on HIVE-27868 to backport HMS client changes related to the catalog support. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27868) Backport a subset of HIVE-18755 to branch-2.3 to support reading from catalog in HMS 3+
[ https://issues.apache.org/jira/browse/HIVE-27868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HIVE-27868. - Fix Version/s: 2.3.10 Assignee: Chao Sun Resolution: Fixed > Backport a subset of HIVE-18755 to branch-2.3 to support reading from catalog > in HMS 3+ > --- > > Key: HIVE-27868 > URL: https://issues.apache.org/jira/browse/HIVE-27868 > Project: Hive > Issue Type: Improvement >Reporter: Chao Sun >Assignee: Chao Sun >Priority: Major > Labels: pull-request-available > Fix For: 2.3.10 > > > HIVE-18755 introduced the concept of catalog which adds another level of > namespace on top of tables and databases. Given HMS using Hive 3.x already > has this feature and Hive 2.3 client is commonly used to talk to these > metastores through frameworks such as Spark, this JIRA proposes to backport a > subset of the features to allow Hive 2.3 client to specify catalog to read > from the 3.x metastores. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27868) Backport a subset of HIVE-18755 to branch-2.3 to support reading from catalog in HMS 3+
[ https://issues.apache.org/jira/browse/HIVE-27868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-27868: Description: HIVE-18755 introduced the concept of catalog which adds another level of namespace on top of tables and databases. Given HMS using Hive 3.x already has this feature and Hive 2.3 client is commonly used to talk to these metastores through frameworks such as Spark, this JIRA proposes to backport a subset of the features to allow Hive 2.3 client to specify catalog to read from the 3.x metastores. > Backport a subset of HIVE-18755 to branch-2.3 to support reading from catalog > in HMS 3+ > --- > > Key: HIVE-27868 > URL: https://issues.apache.org/jira/browse/HIVE-27868 > Project: Hive > Issue Type: Improvement >Reporter: Chao Sun >Priority: Major > > HIVE-18755 introduced the concept of catalog which adds another level of > namespace on top of tables and databases. Given HMS using Hive 3.x already > has this feature and Hive 2.3 client is commonly used to talk to these > metastores through frameworks such as Spark, this JIRA proposes to backport a > subset of the features to allow Hive 2.3 client to specify catalog to read > from the 3.x metastores. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-27868) Backport a subset of HIVE-18755 to branch-2.3 to support reading from catalog in HMS 3+
Chao Sun created HIVE-27868: --- Summary: Backport a subset of HIVE-18755 to branch-2.3 to support reading from catalog in HMS 3+ Key: HIVE-27868 URL: https://issues.apache.org/jira/browse/HIVE-27868 Project: Hive Issue Type: Improvement Reporter: Chao Sun -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-26522) Test for HIVE-22033 and backport to 3.1 and 2.3
[ https://issues.apache.org/jira/browse/HIVE-26522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-26522: Fix Version/s: 2.3.10 > Test for HIVE-22033 and backport to 3.1 and 2.3 > --- > > Key: HIVE-26522 > URL: https://issues.apache.org/jira/browse/HIVE-26522 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Affects Versions: 2.3.8, 3.1.3 >Reporter: Pavan Lanka >Assignee: Pavan Lanka >Priority: Major > Labels: pull-request-available > Fix For: 2.3.10, 4.0.0-beta-1 > > Time Spent: 2h 40m > Remaining Estimate: 0h > > HIVE-22033 fixes the issue with Hive Delegation tokens so that the renewal > time is effective. > This looks at adding a test for HIVE-22033 and backporting this fix to 3.1 > and 2.3 branches in Hive. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-26882) Allow transactional check of Table parameter before altering the Table
[ https://issues.apache.org/jira/browse/HIVE-26882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-26882: Fix Version/s: 2.3.10 > Allow transactional check of Table parameter before altering the Table > -- > > Key: HIVE-26882 > URL: https://issues.apache.org/jira/browse/HIVE-26882 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Major > Labels: pull-request-available > Fix For: 2.3.10, 4.0.0-beta-1 > > Time Spent: 4h 40m > Remaining Estimate: 0h > > We should add the possibility to transactionally check if a Table parameter > is changed before altering the table in the HMS. > This would provide an alternative, less error-prone and faster way to commit > an Iceberg table, as the Iceberg table currently needs to: > - Create an exclusive lock > - Get the table metadata to check if the current snapshot is not changed > - Update the table metadata > - Release the lock > After the change these 4 HMS calls could be substituted with a single alter > table call. > Also we could avoid cases where the locks are left hanging by failed processes -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-25616) Backport HIVE-24741 to Hive 2, 3
[
https://issues.apache.org/jira/browse/HIVE-25616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated HIVE-25616:
Fix Version/s: 2.3.10
> Backport HIVE-24741 to Hive 2, 3
> -
>
> Key: HIVE-25616
> URL: https://issues.apache.org/jira/browse/HIVE-25616
> Project: Hive
> Issue Type: Improvement
>Reporter: Neelesh Srinivas Salian
>Assignee: Neelesh Srinivas Salian
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.3.10
>
> Time Spent: 4h
> Remaining Estimate: 0h
>
> HIVE-24741 adds a major improvement to the `{{get_partitions_ps_with_auth}}`
> API that is used by Spark to retrieve all partitions of the table.
> This has caused problems in Spark 3 - running on a Hive 2.3.x metastore.
> Patched this in my org and it helped get past large partitioned tables having
> problems reading metadata through Spark.
> [~vihangk1], thoughts?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-26890) Disable TestSSL#testConnectionWrongCertCN (Done as part of HIVE-22621 in master)
[
https://issues.apache.org/jira/browse/HIVE-26890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated HIVE-26890:
Fix Version/s: 2.3.10
> Disable TestSSL#testConnectionWrongCertCN (Done as part of HIVE-22621 in
> master)
>
>
> Key: HIVE-26890
> URL: https://issues.apache.org/jira/browse/HIVE-26890
> Project: Hive
> Issue Type: Sub-task
>Reporter: Aman Raj
>Assignee: Aman Raj
>Priority: Critical
> Labels: pull-request-available
> Fix For: 3.2.0, 2.3.10
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> TestSSL fails with the following error (this happens in the Hive-3.1.3
> release also, so disabling this test) :
> {code:java}
> [ERROR] Tests run: 10, Failures: 1, Errors: 0, Skipped: 2, Time elapsed:
> 23.143 s <<< FAILURE! - in org.apache.hive.jdbc.TestSSL
> [ERROR] testConnectionWrongCertCN(org.apache.hive.jdbc.TestSSL) Time
> elapsed: 0.64 s <<< FAILURE!
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN(TestSSL.java:408)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-26880) Upgrade Apache Directory Server to 1.5.7 for release 3.2.
[ https://issues.apache.org/jira/browse/HIVE-26880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-26880: Fix Version/s: 2.3.10 > Upgrade Apache Directory Server to 1.5.7 for release 3.2. > - > > Key: HIVE-26880 > URL: https://issues.apache.org/jira/browse/HIVE-26880 > Project: Hive > Issue Type: Improvement > Components: Test >Reporter: Chris Nauroth >Assignee: Chris Nauroth >Priority: Minor > Labels: hive-3.2.0-must, pull-request-available > Fix For: 3.2.0, 2.3.10 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > branch-3 uses Apache Directory Server in some tests. It currently uses > version 1.5.6. This version has a transitive dependency to a SNAPSHOT, making > it awkward to build and release. We can upgrade to 1.5.7 to remove the > SNAPSHOT dependency. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-25173) Fix build failure of hive-pre-upgrade due to missing dependency on pentaho-aggdesigner-algorithm
[
https://issues.apache.org/jira/browse/HIVE-25173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated HIVE-25173:
Fix Version/s: 2.3.10
> Fix build failure of hive-pre-upgrade due to missing dependency on
> pentaho-aggdesigner-algorithm
>
>
> Key: HIVE-25173
> URL: https://issues.apache.org/jira/browse/HIVE-25173
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 3.1.2
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.3.10, 4.0.0-alpha-1
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> {noformat}
> [ERROR] Failed to execute goal on project hive-pre-upgrade: Could not resolve
> dependencies for project org.apache.hive:hive-pre-upgrade:jar:4.0.0-SNAPSHOT:
> Failure to find org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.5-jhyde in
> https://repo.maven.apache.org/maven2 was cached in the local repository,
> resolution will not be reattempted until the update interval of central has
> elapsed or updates are forced
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HIVE-27518) 2.3 - Upgrade log4j2 from 2.17.0 to 2.17.2
[ https://issues.apache.org/jira/browse/HIVE-27518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HIVE-27518. - Fix Version/s: 2.3.10 Hadoop Flags: Reviewed Resolution: Fixed > 2.3 - Upgrade log4j2 from 2.17.0 to 2.17.2 > -- > > Key: HIVE-27518 > URL: https://issues.apache.org/jira/browse/HIVE-27518 > Project: Hive > Issue Type: Improvement >Affects Versions: 2.3.9 >Reporter: Cheng Pan >Assignee: Cheng Pan >Priority: Major > Labels: pull-request-available > Fix For: 2.3.10 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-27518) 2.3 - Upgrade log4j2 from 2.17.0 to 2.17.2
[ https://issues.apache.org/jira/browse/HIVE-27518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned HIVE-27518: --- Assignee: Cheng Pan > 2.3 - Upgrade log4j2 from 2.17.0 to 2.17.2 > -- > > Key: HIVE-27518 > URL: https://issues.apache.org/jira/browse/HIVE-27518 > Project: Hive > Issue Type: Improvement >Affects Versions: 2.3.9 >Reporter: Cheng Pan >Assignee: Cheng Pan >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-27581) Backport jackson upgrade related patch to branch-2.3
[ https://issues.apache.org/jira/browse/HIVE-27581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned HIVE-27581: --- Assignee: Yuming Wang > Backport jackson upgrade related patch to branch-2.3 > > > Key: HIVE-27581 > URL: https://issues.apache.org/jira/browse/HIVE-27581 > Project: Hive > Issue Type: Task >Reporter: Yuming Wang >Assignee: Yuming Wang >Priority: Major > Labels: pull-request-available > > 2.9.4 -> 2.9.5: > https://github.com/apache/hive/commit/33e208c0709fac5bd6380aacfba49448412d112b > 2.9.5 -> 2.9.8: > https://github.com/apache/hive/commit/2fa22bf360898dc8fd1408bfcc96e1c6aeaf9a53 > 2.9.8 -> 2.9.9: > https://github.com/apache/hive/commit/7fc5a88a149cf0767a5846cbb6ace22d8e99a63c > 2.9.9 -> 2.10.0: > https://github.com/apache/hive/commit/31935896a78f95ae0792ae7f29960d1b604fbe9d > 2.10.0 -> 2.10.5: > https://github.com/apache/hive/commit/aa5b6b7968d90d027c5336bf430719acbff70f68 > 2.10.5 -> 2.12.0: > https://github.com/apache/hive/commit/1e8cc12f2d60973b7674813ae82c8f3372423d54 > --- > 2.12.0 -> 2.12.7: > https://github.com/apache/hive/commit/568ded4b22a020f4d2d3567f15b287b25a3f2b71 > 2.12.7 -> 2.13.5: > https://github.com/apache/hive/commit/8236426ed7aa87430e82d47effe946e38fa1f7f2 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27581) Backport jackson upgrade related patch to branch-2.3
[ https://issues.apache.org/jira/browse/HIVE-27581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HIVE-27581. - Fix Version/s: 2.3.10 Hadoop Flags: Reviewed Resolution: Fixed > Backport jackson upgrade related patch to branch-2.3 > > > Key: HIVE-27581 > URL: https://issues.apache.org/jira/browse/HIVE-27581 > Project: Hive > Issue Type: Task >Reporter: Yuming Wang >Assignee: Yuming Wang >Priority: Major > Labels: pull-request-available > Fix For: 2.3.10 > > > 2.9.4 -> 2.9.5: > https://github.com/apache/hive/commit/33e208c0709fac5bd6380aacfba49448412d112b > 2.9.5 -> 2.9.8: > https://github.com/apache/hive/commit/2fa22bf360898dc8fd1408bfcc96e1c6aeaf9a53 > 2.9.8 -> 2.9.9: > https://github.com/apache/hive/commit/7fc5a88a149cf0767a5846cbb6ace22d8e99a63c > 2.9.9 -> 2.10.0: > https://github.com/apache/hive/commit/31935896a78f95ae0792ae7f29960d1b604fbe9d > 2.10.0 -> 2.10.5: > https://github.com/apache/hive/commit/aa5b6b7968d90d027c5336bf430719acbff70f68 > 2.10.5 -> 2.12.0: > https://github.com/apache/hive/commit/1e8cc12f2d60973b7674813ae82c8f3372423d54 > --- > 2.12.0 -> 2.12.7: > https://github.com/apache/hive/commit/568ded4b22a020f4d2d3567f15b287b25a3f2b71 > 2.12.7 -> 2.13.5: > https://github.com/apache/hive/commit/8236426ed7aa87430e82d47effe946e38fa1f7f2 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27200) Backport HIVE-24928 to branch-3
[ https://issues.apache.org/jira/browse/HIVE-27200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HIVE-27200. - Fix Version/s: 3.2.0 Resolution: Fixed > Backport HIVE-24928 to branch-3 > --- > > Key: HIVE-27200 > URL: https://issues.apache.org/jira/browse/HIVE-27200 > Project: Hive > Issue Type: Improvement > Components: StorageHandler >Reporter: Yi Zhang >Assignee: Yi Zhang >Priority: Critical > Labels: pull-request-available > Fix For: 3.2.0 > > Time Spent: 40m > Remaining Estimate: 0h > > This is to backport HIVE-24928 so that for HiveStorageHandler table 'ANALYZE > TABLE ... COMPUTE STATISTICS' can use storagehandler to provide basic stats > with BasicStatsNoJobTask -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27017) option to use createTable DDLTask in CTAS for StorgeHandler
[ https://issues.apache.org/jira/browse/HIVE-27017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HIVE-27017. - Fix Version/s: 3.2.0 Resolution: Fixed > option to use createTable DDLTask in CTAS for StorgeHandler > --- > > Key: HIVE-27017 > URL: https://issues.apache.org/jira/browse/HIVE-27017 > Project: Hive > Issue Type: Improvement > Components: StorageHandler >Affects Versions: 3.1.3 >Reporter: Yi Zhang >Assignee: Yi Zhang >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > This is to add a directInsert option for StorageHandler and advance DDLTask > for CTAS when it is storagehandler directInsert mode. This is partial > backport of HIVE-26771 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27016) Invoke optional output committer in TezProcessor
[ https://issues.apache.org/jira/browse/HIVE-27016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HIVE-27016. - Fix Version/s: 3.2.0 Resolution: Fixed > Invoke optional output committer in TezProcessor > > > Key: HIVE-27016 > URL: https://issues.apache.org/jira/browse/HIVE-27016 > Project: Hive > Issue Type: Improvement > Components: Query Processor, StorageHandler >Affects Versions: 3.1.3 >Reporter: Yi Zhang >Assignee: Yi Zhang >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > This is backport HIVE-24629 and HIVE-24867, so StorageHandler with their own > OutputCommitter run in tez. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-26819) Vectorization: wrong results when filter on repeating map key orc table
[
https://issues.apache.org/jira/browse/HIVE-26819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun resolved HIVE-26819.
-
Fix Version/s: 3.2.0
Hadoop Flags: Reviewed
Resolution: Fixed
> Vectorization: wrong results when filter on repeating map key orc table
> ---
>
> Key: HIVE-26819
> URL: https://issues.apache.org/jira/browse/HIVE-26819
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.1.3
>Reporter: Yi Zhang
>Assignee: Yi Zhang
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.2.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Same issue is fixed by HIVE-26447, this is to fix in 3.2.0.
> Example reproducible case:
> set hive.vectorized.execution.enabled=true;
> set hive.fetch.task.conversion=none;
> create temporary table foo (id int, x map) stored as orc;
> insert into foo values(1, map('ABC', 9)), (2, map('ABC', 7)), (3, map('ABC',
> 8)), (4, map('ABC', 9));
> select id from foo where x['ABC']=9;
> this only gives 1, when correct result should be 1,4
> For every VectorizedRowBatch, only the first row is checked.
> This seems to be a corner case of ORC table have repeating string type key
> for map field in the MapColumnVector.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HIVE-26815) Backport HIVE-26758 (Allow use scratchdir for staging final job)
[ https://issues.apache.org/jira/browse/HIVE-26815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HIVE-26815. - Fix Version/s: 3.2.0 Hadoop Flags: Reviewed Resolution: Fixed > Backport HIVE-26758 (Allow use scratchdir for staging final job) > > > Key: HIVE-26815 > URL: https://issues.apache.org/jira/browse/HIVE-26815 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.1.3 >Reporter: Yi Zhang >Assignee: Yi Zhang >Priority: Minor > Labels: pull-request-available > Fix For: 3.2.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > HIVE-26758 add an option to allow choose set final job staging with > hive.exec.scratchdir. This is to backport this into 3.2.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-26758) Allow use scratchdir for staging final job
[ https://issues.apache.org/jira/browse/HIVE-26758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HIVE-26758. - Fix Version/s: 4.0.0 Hadoop Flags: Reviewed Resolution: Fixed > Allow use scratchdir for staging final job > -- > > Key: HIVE-26758 > URL: https://issues.apache.org/jira/browse/HIVE-26758 > Project: Hive > Issue Type: New Feature > Components: Query Planning >Affects Versions: 4.0.0-alpha-2 >Reporter: Yi Zhang >Assignee: Yi Zhang >Priority: Minor > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 4h 10m > Remaining Estimate: 0h > > The query results are staged in stagingdir that is relative to the > destination path // > during blobstorage optimzation HIVE-17620 final job is set to use stagingdir. > HIVE-15215 mentioned the possibility of using scratch for staging when write > to S3 but it was long time ago and no activity. > > This is to allow final job to use hive.exec.scratchdir as the interim jobs, > with a configuration > hive.use.scratchdir.for.staging > This is useful for cross Filesystem, user can use local source filesystem > instead of remote filesystem for the staging. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-26478) Explicitly set Content-Type in QueryProfileServlet
[ https://issues.apache.org/jira/browse/HIVE-26478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HIVE-26478. - Fix Version/s: 4.0.0 Resolution: Fixed > Explicitly set Content-Type in QueryProfileServlet > -- > > Key: HIVE-26478 > URL: https://issues.apache.org/jira/browse/HIVE-26478 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 3.1.3, 4.0.0 >Reporter: Yi Zhang >Assignee: Yi Zhang >Priority: Minor > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > QueryProfileServlet does not set Content-type, though browser may detect it > correctly but for some application that checks Content-type, it would be > helpful to set it -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-26447) Vectorization: wrong results when filter on repeating map key orc table
[
https://issues.apache.org/jira/browse/HIVE-26447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun resolved HIVE-26447.
-
Fix Version/s: 4.0.0
Hadoop Flags: Reviewed
Resolution: Fixed
> Vectorization: wrong results when filter on repeating map key orc table
> ---
>
> Key: HIVE-26447
> URL: https://issues.apache.org/jira/browse/HIVE-26447
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.1.3, 4.0.0
>Reporter: Yi Zhang
>Assignee: Yi Zhang
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Example reproducible case:
>
> set hive.vectorized.execution.enabled=true;
> set hive.fetch.task.conversion=none;
> create temporary table foo (id int, x map) stored as orc;
> insert into foo values(1, map('ABC', 9)), (2, map('ABC', 7)), (3, map('ABC',
> 8)), (4, map('ABC', 9));
> select id from foo where x['ABC']=9;
> this only gives 1, when correct result should be 1,4
> For every VectorizedRowBatch, only the first row is checked.
> This seems to be a corner case of ORC table have repeating string type key
> for map field in the MapColumnVector.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HIVE-26448) Improve UnsignedInt128.setV*()
[ https://issues.apache.org/jira/browse/HIVE-26448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HIVE-26448. - Fix Version/s: 4.0.0 (was: 0.13.0) Resolution: Fixed > Improve UnsignedInt128.setV*() > -- > > Key: HIVE-26448 > URL: https://issues.apache.org/jira/browse/HIVE-26448 > Project: Hive > Issue Type: Sub-task > Components: SQL >Reporter: jiaan.geng >Assignee: jiaan.geng >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 50m > Remaining Estimate: 0h > > Currently, these method setV0, setV1, setV2 of UnsignedInt128 call > updateCount directly. > updateCount have a lot if else logic. > In fact, we can improve the code by judge count field. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-26448) Improve UnsignedInt128.setV*()
[ https://issues.apache.org/jira/browse/HIVE-26448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned HIVE-26448: --- Assignee: jiaan.geng > Improve UnsignedInt128.setV*() > -- > > Key: HIVE-26448 > URL: https://issues.apache.org/jira/browse/HIVE-26448 > Project: Hive > Issue Type: Sub-task > Components: SQL >Reporter: jiaan.geng >Assignee: jiaan.geng >Priority: Major > Labels: pull-request-available > Fix For: 0.13.0 > > Time Spent: 50m > Remaining Estimate: 0h > > Currently, these method setV0, setV1, setV2 of UnsignedInt128 call > updateCount directly. > updateCount have a lot if else logic. > In fact, we can improve the code by judge count field. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-26220) Shade & relocate dependencies in hive-exec to avoid conflicting with downstream projects
[
https://issues.apache.org/jira/browse/HIVE-26220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17575353#comment-17575353
]
Chao Sun commented on HIVE-26220:
-
I don't have a full list here but there were some previous attempt by
Liang-Chi: https://github.com/apache/hive/pull/2459
> I think most of the shaded dependencies can be relocated, I don't think that
> would be a problem, excluding couple of them like calcite.
Yes, it'd be great if we can shade most of the dependencies.
> Chao Sun have you tried using the current hive-exec from master?
[~kgyrtkirk] we haven't tried it yet, as Spark is still currently using Hive
2.3.9 and there's not strong motivation to move to a newer version of Hive. I
feel this doesn't have to be a blocker for the 4.0.0 release, since it'll be a
while for Spark to adopt the new version and we can solve the shading issue in
a next release.
> Shade & relocate dependencies in hive-exec to avoid conflicting with
> downstream projects
>
>
> Key: HIVE-26220
> URL: https://issues.apache.org/jira/browse/HIVE-26220
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0, 4.0.0-alpha-1
>Reporter: Chao Sun
>Priority: Blocker
>
> Currently projects like Spark, Trino/Presto, Iceberg, etc, are depending on
> {{hive-exec:core}} which was removed in HIVE-25531. The reason these projects
> use {{hive-exec:core}} is because they have the flexibility to exclude, shade
> & relocate dependencies in {{hive-exec}} that conflict with the ones they
> brought in by themselves. However, with {{hive-exec}} this is no longer
> possible, since it is a fat jar that shade those dependencies but do not
> relocate many of them.
> In order for the downstream projects to consume {{hive-exec}}, we will need
> to make sure all the dependencies in {{hive-exec}} are properly shaded and
> relocated, so they won't cause conflicts with those from the downstream.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HIVE-26434) HMS thrift method recv_get_table_objects_by_name exception list is broken
[ https://issues.apache.org/jira/browse/HIVE-26434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HIVE-26434. - Fix Version/s: 2.3.10 Assignee: Cheng Pan Resolution: Fixed > HMS thrift method recv_get_table_objects_by_name exception list is broken > - > > Key: HIVE-26434 > URL: https://issues.apache.org/jira/browse/HIVE-26434 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 2.3.9 >Reporter: Cheng Pan >Assignee: Cheng Pan >Priority: Major > Labels: pull-request-available > Fix For: 2.3.10 > > Time Spent: 50m > Remaining Estimate: 0h > > HIVE-15062 accidentally removed the HMS thrift method > recv_get_table_objects_by_name exception list, after HIVE-24608, some IT > consistently failed because of it. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-24893) Download data from Thriftserver through JDBC
[ https://issues.apache.org/jira/browse/HIVE-24893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HIVE-24893. - Fix Version/s: 4.0.0 Hadoop Flags: Reviewed Resolution: Fixed > Download data from Thriftserver through JDBC > > > Key: HIVE-24893 > URL: https://issues.apache.org/jira/browse/HIVE-24893 > Project: Hive > Issue Type: New Feature > Components: HiveServer2, JDBC >Affects Versions: 4.0.0 >Reporter: Yuming Wang >Assignee: Yuming Wang >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 2h 10m > Remaining Estimate: 0h > > It is very useful to support downloading large amounts of data (such as more > than 50GB) through JDBC. > Snowflake has similar support : > https://docs.snowflake.com/en/user-guide/jdbc-using.html#label-jdbc-download-from-stage-to-stream > https://github.com/snowflakedb/snowflake-jdbc/blob/95a7d8a03316093430dc3960df6635643208b6fd/src/main/java/net/snowflake/client/jdbc/SnowflakeConnectionV1.java#L886 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (HIVE-24893) Download data from Thriftserver through JDBC
[ https://issues.apache.org/jira/browse/HIVE-24893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned HIVE-24893: --- Assignee: Yuming Wang > Download data from Thriftserver through JDBC > > > Key: HIVE-24893 > URL: https://issues.apache.org/jira/browse/HIVE-24893 > Project: Hive > Issue Type: New Feature > Components: HiveServer2, JDBC >Affects Versions: 4.0.0 >Reporter: Yuming Wang >Assignee: Yuming Wang >Priority: Major > Labels: pull-request-available > Time Spent: 2h > Remaining Estimate: 0h > > It is very useful to support downloading large amounts of data (such as more > than 50GB) through JDBC. > Snowflake has similar support : > https://docs.snowflake.com/en/user-guide/jdbc-using.html#label-jdbc-download-from-stage-to-stream > https://github.com/snowflakedb/snowflake-jdbc/blob/95a7d8a03316093430dc3960df6635643208b6fd/src/main/java/net/snowflake/client/jdbc/SnowflakeConnectionV1.java#L886 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (HIVE-25824) Upgrade branch-2.3 to log4j 2.17.0
[ https://issues.apache.org/jira/browse/HIVE-25824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HIVE-25824. - Fix Version/s: 2.3.10 Hadoop Flags: Reviewed Resolution: Fixed > Upgrade branch-2.3 to log4j 2.17.0 > -- > > Key: HIVE-25824 > URL: https://issues.apache.org/jira/browse/HIVE-25824 > Project: Hive > Issue Type: Improvement >Affects Versions: 2.3.8 >Reporter: Luca Toscano >Assignee: Luca Toscano >Priority: Major > Labels: pull-request-available > Fix For: 2.3.10 > > Time Spent: 1h 40m > Remaining Estimate: 0h > > Hi everybody, > I am wondering if there are any plans to upgrade branch-2.3 to log4j 2.17.0 > as it was done in HIVE-25795 (and related). > In Apache Bigtop we created https://github.com/apache/bigtop/pull/844, since > the one before the last release (Bigtop 1.5.0) shipped Hive 2.3.6. > I can try to file a pull request for branch-2.3 adapting what was done for > Bigtop (if the branch is still maintained), but I am currently experiencing > some mvn package failures (that seem unrelated to log4j) so I'd need some > help for you in case :) -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (HIVE-25824) Upgrade branch-2.3 to log4j 2.17.0
[ https://issues.apache.org/jira/browse/HIVE-25824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned HIVE-25824: --- Assignee: Luca Toscano > Upgrade branch-2.3 to log4j 2.17.0 > -- > > Key: HIVE-25824 > URL: https://issues.apache.org/jira/browse/HIVE-25824 > Project: Hive > Issue Type: Improvement >Affects Versions: 2.3.8 >Reporter: Luca Toscano >Assignee: Luca Toscano >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > Hi everybody, > I am wondering if there are any plans to upgrade branch-2.3 to log4j 2.17.0 > as it was done in HIVE-25795 (and related). > In Apache Bigtop we created https://github.com/apache/bigtop/pull/844, since > the one before the last release (Bigtop 1.5.0) shipped Hive 2.3.6. > I can try to file a pull request for branch-2.3 adapting what was done for > Bigtop (if the branch is still maintained), but I am currently experiencing > some mvn package failures (that seem unrelated to log4j) so I'd need some > help for you in case :) -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HIVE-25824) Upgrade branch-2.3 to log4j 2.17.0
[ https://issues.apache.org/jira/browse/HIVE-25824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17463980#comment-17463980 ] Chao Sun commented on HIVE-25824: - [~elukey] Feel free to create a PR for this and I can help review & merge: what man package failures did you encounter? BTW the latest version in branch-2.3 is 2.3.10. > Upgrade branch-2.3 to log4j 2.17.0 > -- > > Key: HIVE-25824 > URL: https://issues.apache.org/jira/browse/HIVE-25824 > Project: Hive > Issue Type: Improvement >Affects Versions: 2.3.8 >Reporter: Luca Toscano >Priority: Major > > Hi everybody, > I am wondering if there are any plans to upgrade branch-2.3 to log4j 2.17.0 > as it was done in HIVE-25795 (and related). > In Apache Bigtop we created https://github.com/apache/bigtop/pull/844, since > the one before the last release (Bigtop 1.5.0) shipped Hive 2.3.6. > I can try to file a pull request for branch-2.3 adapting what was done for > Bigtop (if the branch is still maintained), but I am currently experiencing > some mvn package failures (that seem unrelated to log4j) so I'd need some > help for you in case :) -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (HIVE-25729) ThriftUnionObjectInspector should be notified when fully inited
[
https://issues.apache.org/jira/browse/HIVE-25729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun resolved HIVE-25729.
-
Fix Version/s: 4.0.0
Resolution: Fixed
> ThriftUnionObjectInspector should be notified when fully inited
> ---
>
> Key: HIVE-25729
> URL: https://issues.apache.org/jira/browse/HIVE-25729
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
>Reporter: Zhihua Deng
>Assignee: Zhihua Deng
>Priority: Minor
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> For thread safe purpose, a ReflectionStructObjectInspector instance would
> wait for 3 seconds to ensure the returning ObjectInspector is fully inited,
> {code:java}
> synchronized (soi) {
> while (!soi.isFullyInited(checkedTypes)) {
> //
> soi.wait(3000);
> }
> } {code}
> It seems that we are missing to notify ThriftUnionObjectInspector when it has
> been inited.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HIVE-25500) Switch back to alter_partition(s) in HMS client for Hive 2.3.x
[
https://issues.apache.org/jira/browse/HIVE-25500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated HIVE-25500:
Fix Version/s: (was: 2.3.9)
2.3.10
> Switch back to alter_partition(s) in HMS client for Hive 2.3.x
> --
>
> Key: HIVE-25500
> URL: https://issues.apache.org/jira/browse/HIVE-25500
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Affects Versions: 2.3.9
>Reporter: dzcxzl
>Assignee: dzcxzl
>Priority: Minor
> Labels: pull-request-available
> Fix For: 2.3.10
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> HIVE-12730 uses alter_partition_with_environment_context and
> alter_partitions_with_environment_context instead of alter_partition and
> alter_partitions when the EnvironmentContext is null.
> This causes the client version to be greater than 2.1 to connect to
> server<2.1, and this error will occur.
> {code:java}
> Caused by: org.apache.thrift.TApplicationException: Invalid method name:
> 'alter_partitions_with_environment_context'
> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_alter_partitions_with_environment_context(ThriftHiveMetastore.java:2843)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.alter_partitions_with_environment_context(ThriftHiveMetastore.java:2827)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.alter_partitions(HiveMetaStoreClient.java:1524)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HIVE-25500) Switch back to alter_partition(s) in HMS client for Hive 2.3.x
[
https://issues.apache.org/jira/browse/HIVE-25500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun resolved HIVE-25500.
-
Fix Version/s: 2.3.9
Resolution: Fixed
> Switch back to alter_partition(s) in HMS client for Hive 2.3.x
> --
>
> Key: HIVE-25500
> URL: https://issues.apache.org/jira/browse/HIVE-25500
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Affects Versions: 2.3.9
>Reporter: dzcxzl
>Assignee: dzcxzl
>Priority: Minor
> Labels: pull-request-available
> Fix For: 2.3.9
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> HIVE-12730 uses alter_partition_with_environment_context and
> alter_partitions_with_environment_context instead of alter_partition and
> alter_partitions when the EnvironmentContext is null.
> This causes the client version to be greater than 2.1 to connect to
> server<2.1, and this error will occur.
> {code:java}
> Caused by: org.apache.thrift.TApplicationException: Invalid method name:
> 'alter_partitions_with_environment_context'
> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_alter_partitions_with_environment_context(ThriftHiveMetastore.java:2843)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.alter_partitions_with_environment_context(ThriftHiveMetastore.java:2827)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.alter_partitions(HiveMetaStoreClient.java:1524)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HIVE-25277) Slow Hive partition deletion for Cloud object stores with expensive ListFiles
[ https://issues.apache.org/jira/browse/HIVE-25277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HIVE-25277. - Fix Version/s: 4.0.0 Hadoop Flags: Reviewed Resolution: Fixed > Slow Hive partition deletion for Cloud object stores with expensive ListFiles > - > > Key: HIVE-25277 > URL: https://issues.apache.org/jira/browse/HIVE-25277 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Affects Versions: All Versions >Reporter: Zhou Fang >Assignee: Zhou Fang >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 4.5h > Remaining Estimate: 0h > > Deleting a Hive partition is slow when use a Cloud object store as the > warehouse for which ListFiles is expensive. A root cause is that the > recursive parent dir deletion is very inefficient: there are many duplicated > calls to isEmpty (ListFiles is called at the end). This fix sorts the parents > to delete according to the path size, and always processes the longest one > (e.g., a/b/c is always before a/b). As a result, each parent path is only > needed to be checked once. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23896) hiveserver2 not listening on any port, am i miss some configurations?
[
https://issues.apache.org/jira/browse/HIVE-23896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17407703#comment-17407703
]
Chao Sun commented on HIVE-23896:
-
[~brahmareddy] I don't think there is PR for this.
> hiveserver2 not listening on any port, am i miss some configurations?
> -
>
> Key: HIVE-23896
> URL: https://issues.apache.org/jira/browse/HIVE-23896
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.1.2
> Environment: hive: 3.1.2
> hadoop: 3.2.1, standalone, url: hdfs://namenode.hadoop.svc.cluster.local:9000
> {quote}$ $HADOOP_HOME/bin/hadoop fs -mkdir /tmp
> $ $HADOOP_HOME/bin/hadoop fs -mkdir /user/hive/warehouse
> {quote}
> hadoop commands are workable in the hiveserver node(POD).
>
>Reporter: alanwake
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
>
>
> i try deply hive 3.1.2 on k8s. it was worked on version 2.3.2.
> metastore node and postgres node are ok, but hiveserver look like i miss some
> important configuration properties?
> {code:java}
> {code}
>
>
>
> {code:java}
> [root@master hive]# ./get.sh
> NAME READY STATUSRESTARTS AGE IP
> NODE NOMINATED NODE READINESS GATES
> hive-7bd48747d4-5zjmh1/1 Running 0 56s 10.244.3.110
> node03.51.local
> metastore-66b58f9f76-6wsxj 1/1 Running 0 56s 10.244.3.109
> node03.51.local
> postgres-57794b99b7-pqxwm1/1 Running 0 56s 10.244.2.241
> node02.51.local NAMETYPECLUSTER-IP
> EXTERNAL-IP PORT(S) AGE SELECTOR
> hiveNodePort10.108.40.17
> 10002:30626/TCP,1:31845/TCP 56s app=hive
> metastore ClusterIP 10.106.159.220 9083/TCP
>56s app=metastore
> postgresClusterIP 10.108.85.47 5432/TCP
>56s app=postgres
> {code}
>
>
> {code:java}
> [root@master hive]# kubectl logs hive-7bd48747d4-5zjmh -n=hive
> Configuring core
> - Setting hadoop.proxyuser.hue.hosts=*
> - Setting fs.defaultFS=hdfs://namenode.hadoop.svc.cluster.local:9000
> - Setting hadoop.http.staticuser.user=root
> - Setting hadoop.proxyuser.hue.groups=*
> Configuring hdfs
> - Setting dfs.namenode.datanode.registration.ip-hostname-check=false
> - Setting dfs.webhdfs.enabled=true
> - Setting dfs.permissions.enabled=false
> Configuring yarn
> - Setting yarn.timeline-service.enabled=true
> - Setting yarn.resourcemanager.system-metrics-publisher.enabled=true
> - Setting
> yarn.resourcemanager.store.class=org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore
> - Setting
> yarn.log.server.url=http://historyserver.hadoop.svc.cluster.local:8188/applicationhistory/logs/
> - Setting yarn.resourcemanager.fs.state-store.uri=/rmstate
> - Setting yarn.timeline-service.generic-application-history.enabled=true
> - Setting yarn.log-aggregation-enable=true
> - Setting
> yarn.resourcemanager.hostname=resourcemanager.hadoop.svc.cluster.local
> - Setting
> yarn.resourcemanager.resource.tracker.address=resourcemanager.hadoop.svc.cluster.local:8031
> - Setting
> yarn.timeline-service.hostname=historyserver.hadoop.svc.cluster.local
> - Setting
> yarn.resourcemanager.scheduler.address=resourcemanager.hadoop.svc.cluster.local:8030
> - Setting
> yarn.resourcemanager.address=resourcemanager.hadoop.svc.cluster.local:8032
> - Setting yarn.nodemanager.remote-app-log-dir=/app-logs
> - Setting yarn.resourcemanager.recovery.enabled=true
> Configuring httpfs
> Configuring kms
> Configuring mapred
> Configuring hive
> - Setting datanucleus.autoCreateSchema=false
> - Setting javax.jdo.option.ConnectionPassword=hive
> - Setting hive.metastore.uris=thrift://metastore:9083
> - Setting
> javax.jdo.option.ConnectionURL=jdbc:postgresql://metastore/metastore
> - Setting javax.jdo.option.ConnectionUserName=hive
> - Setting javax.jdo.option.ConnectionDriverName=org.postgresql.Driver
> Configuring for multihomed network
> [1/100] check for metastore:9083...
> [1/100] metastore:9083 is not available yet
> [1/100] try in 5s once again ...
> [2/100] check for metastore:9083...
> [2/100] metastore:9083 is not available yet
> [2/100] try in 5s once again ...
> [3/100] check for metastore:9083...
> [3/100] metastore:9083 is not available yet
> [3/100] try in 5s once again ...
> [4/100] check for metastore:9083...
> [4/100] metastore:9083 is not available yet
> [4/100] try in 5s once again ...
> [5/100] metastore:9083 is available.
> mkdir: `/tmp': File exists
> 2020-07-22 07:15:33:
[jira] [Updated] (HIVE-23896) hiveserver2 not listening on any port, am i miss some configurations?
[
https://issues.apache.org/jira/browse/HIVE-23896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated HIVE-23896:
Fix Version/s: (was: 4.0.0)
> hiveserver2 not listening on any port, am i miss some configurations?
> -
>
> Key: HIVE-23896
> URL: https://issues.apache.org/jira/browse/HIVE-23896
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.1.2
> Environment: hive: 3.1.2
> hadoop: 3.2.1, standalone, url: hdfs://namenode.hadoop.svc.cluster.local:9000
> {quote}$ $HADOOP_HOME/bin/hadoop fs -mkdir /tmp
> $ $HADOOP_HOME/bin/hadoop fs -mkdir /user/hive/warehouse
> {quote}
> hadoop commands are workable in the hiveserver node(POD).
>
>Reporter: alanwake
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
>
>
> i try deply hive 3.1.2 on k8s. it was worked on version 2.3.2.
> metastore node and postgres node are ok, but hiveserver look like i miss some
> important configuration properties?
> {code:java}
> {code}
>
>
>
> {code:java}
> [root@master hive]# ./get.sh
> NAME READY STATUSRESTARTS AGE IP
> NODE NOMINATED NODE READINESS GATES
> hive-7bd48747d4-5zjmh1/1 Running 0 56s 10.244.3.110
> node03.51.local
> metastore-66b58f9f76-6wsxj 1/1 Running 0 56s 10.244.3.109
> node03.51.local
> postgres-57794b99b7-pqxwm1/1 Running 0 56s 10.244.2.241
> node02.51.local NAMETYPECLUSTER-IP
> EXTERNAL-IP PORT(S) AGE SELECTOR
> hiveNodePort10.108.40.17
> 10002:30626/TCP,1:31845/TCP 56s app=hive
> metastore ClusterIP 10.106.159.220 9083/TCP
>56s app=metastore
> postgresClusterIP 10.108.85.47 5432/TCP
>56s app=postgres
> {code}
>
>
> {code:java}
> [root@master hive]# kubectl logs hive-7bd48747d4-5zjmh -n=hive
> Configuring core
> - Setting hadoop.proxyuser.hue.hosts=*
> - Setting fs.defaultFS=hdfs://namenode.hadoop.svc.cluster.local:9000
> - Setting hadoop.http.staticuser.user=root
> - Setting hadoop.proxyuser.hue.groups=*
> Configuring hdfs
> - Setting dfs.namenode.datanode.registration.ip-hostname-check=false
> - Setting dfs.webhdfs.enabled=true
> - Setting dfs.permissions.enabled=false
> Configuring yarn
> - Setting yarn.timeline-service.enabled=true
> - Setting yarn.resourcemanager.system-metrics-publisher.enabled=true
> - Setting
> yarn.resourcemanager.store.class=org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore
> - Setting
> yarn.log.server.url=http://historyserver.hadoop.svc.cluster.local:8188/applicationhistory/logs/
> - Setting yarn.resourcemanager.fs.state-store.uri=/rmstate
> - Setting yarn.timeline-service.generic-application-history.enabled=true
> - Setting yarn.log-aggregation-enable=true
> - Setting
> yarn.resourcemanager.hostname=resourcemanager.hadoop.svc.cluster.local
> - Setting
> yarn.resourcemanager.resource.tracker.address=resourcemanager.hadoop.svc.cluster.local:8031
> - Setting
> yarn.timeline-service.hostname=historyserver.hadoop.svc.cluster.local
> - Setting
> yarn.resourcemanager.scheduler.address=resourcemanager.hadoop.svc.cluster.local:8030
> - Setting
> yarn.resourcemanager.address=resourcemanager.hadoop.svc.cluster.local:8032
> - Setting yarn.nodemanager.remote-app-log-dir=/app-logs
> - Setting yarn.resourcemanager.recovery.enabled=true
> Configuring httpfs
> Configuring kms
> Configuring mapred
> Configuring hive
> - Setting datanucleus.autoCreateSchema=false
> - Setting javax.jdo.option.ConnectionPassword=hive
> - Setting hive.metastore.uris=thrift://metastore:9083
> - Setting
> javax.jdo.option.ConnectionURL=jdbc:postgresql://metastore/metastore
> - Setting javax.jdo.option.ConnectionUserName=hive
> - Setting javax.jdo.option.ConnectionDriverName=org.postgresql.Driver
> Configuring for multihomed network
> [1/100] check for metastore:9083...
> [1/100] metastore:9083 is not available yet
> [1/100] try in 5s once again ...
> [2/100] check for metastore:9083...
> [2/100] metastore:9083 is not available yet
> [2/100] try in 5s once again ...
> [3/100] check for metastore:9083...
> [3/100] metastore:9083 is not available yet
> [3/100] try in 5s once again ...
> [4/100] check for metastore:9083...
> [4/100] metastore:9083 is not available yet
> [4/100] try in 5s once again ...
> [5/100] metastore:9083 is available.
> mkdir: `/tmp': File exists
> 2020-07-22 07:15:33: Starting HiveServer2
> SLF4J: Class path contains multiple
[jira] [Reopened] (HIVE-23896) hiveserver2 not listening on any port, am i miss some configurations?
[
https://issues.apache.org/jira/browse/HIVE-23896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun reopened HIVE-23896:
-
Assignee: (was: alanwake)
> hiveserver2 not listening on any port, am i miss some configurations?
> -
>
> Key: HIVE-23896
> URL: https://issues.apache.org/jira/browse/HIVE-23896
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.1.2
> Environment: hive: 3.1.2
> hadoop: 3.2.1, standalone, url: hdfs://namenode.hadoop.svc.cluster.local:9000
> {quote}$ $HADOOP_HOME/bin/hadoop fs -mkdir /tmp
> $ $HADOOP_HOME/bin/hadoop fs -mkdir /user/hive/warehouse
> {quote}
> hadoop commands are workable in the hiveserver node(POD).
>
>Reporter: alanwake
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
>
>
> i try deply hive 3.1.2 on k8s. it was worked on version 2.3.2.
> metastore node and postgres node are ok, but hiveserver look like i miss some
> important configuration properties?
> {code:java}
> {code}
>
>
>
> {code:java}
> [root@master hive]# ./get.sh
> NAME READY STATUSRESTARTS AGE IP
> NODE NOMINATED NODE READINESS GATES
> hive-7bd48747d4-5zjmh1/1 Running 0 56s 10.244.3.110
> node03.51.local
> metastore-66b58f9f76-6wsxj 1/1 Running 0 56s 10.244.3.109
> node03.51.local
> postgres-57794b99b7-pqxwm1/1 Running 0 56s 10.244.2.241
> node02.51.local NAMETYPECLUSTER-IP
> EXTERNAL-IP PORT(S) AGE SELECTOR
> hiveNodePort10.108.40.17
> 10002:30626/TCP,1:31845/TCP 56s app=hive
> metastore ClusterIP 10.106.159.220 9083/TCP
>56s app=metastore
> postgresClusterIP 10.108.85.47 5432/TCP
>56s app=postgres
> {code}
>
>
> {code:java}
> [root@master hive]# kubectl logs hive-7bd48747d4-5zjmh -n=hive
> Configuring core
> - Setting hadoop.proxyuser.hue.hosts=*
> - Setting fs.defaultFS=hdfs://namenode.hadoop.svc.cluster.local:9000
> - Setting hadoop.http.staticuser.user=root
> - Setting hadoop.proxyuser.hue.groups=*
> Configuring hdfs
> - Setting dfs.namenode.datanode.registration.ip-hostname-check=false
> - Setting dfs.webhdfs.enabled=true
> - Setting dfs.permissions.enabled=false
> Configuring yarn
> - Setting yarn.timeline-service.enabled=true
> - Setting yarn.resourcemanager.system-metrics-publisher.enabled=true
> - Setting
> yarn.resourcemanager.store.class=org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore
> - Setting
> yarn.log.server.url=http://historyserver.hadoop.svc.cluster.local:8188/applicationhistory/logs/
> - Setting yarn.resourcemanager.fs.state-store.uri=/rmstate
> - Setting yarn.timeline-service.generic-application-history.enabled=true
> - Setting yarn.log-aggregation-enable=true
> - Setting
> yarn.resourcemanager.hostname=resourcemanager.hadoop.svc.cluster.local
> - Setting
> yarn.resourcemanager.resource.tracker.address=resourcemanager.hadoop.svc.cluster.local:8031
> - Setting
> yarn.timeline-service.hostname=historyserver.hadoop.svc.cluster.local
> - Setting
> yarn.resourcemanager.scheduler.address=resourcemanager.hadoop.svc.cluster.local:8030
> - Setting
> yarn.resourcemanager.address=resourcemanager.hadoop.svc.cluster.local:8032
> - Setting yarn.nodemanager.remote-app-log-dir=/app-logs
> - Setting yarn.resourcemanager.recovery.enabled=true
> Configuring httpfs
> Configuring kms
> Configuring mapred
> Configuring hive
> - Setting datanucleus.autoCreateSchema=false
> - Setting javax.jdo.option.ConnectionPassword=hive
> - Setting hive.metastore.uris=thrift://metastore:9083
> - Setting
> javax.jdo.option.ConnectionURL=jdbc:postgresql://metastore/metastore
> - Setting javax.jdo.option.ConnectionUserName=hive
> - Setting javax.jdo.option.ConnectionDriverName=org.postgresql.Driver
> Configuring for multihomed network
> [1/100] check for metastore:9083...
> [1/100] metastore:9083 is not available yet
> [1/100] try in 5s once again ...
> [2/100] check for metastore:9083...
> [2/100] metastore:9083 is not available yet
> [2/100] try in 5s once again ...
> [3/100] check for metastore:9083...
> [3/100] metastore:9083 is not available yet
> [3/100] try in 5s once again ...
> [4/100] check for metastore:9083...
> [4/100] metastore:9083 is not available yet
> [4/100] try in 5s once again ...
> [5/100] metastore:9083 is available.
> mkdir: `/tmp': File exists
> 2020-07-22 07:15:33: Starting HiveServer2
>
[jira] [Resolved] (HIVE-23896) hiveserver2 not listening on any port, am i miss some configurations?
[
https://issues.apache.org/jira/browse/HIVE-23896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun resolved HIVE-23896.
-
Fix Version/s: 4.0.0
Hadoop Flags: Reviewed
Assignee: alanwake
Resolution: Fixed
> hiveserver2 not listening on any port, am i miss some configurations?
> -
>
> Key: HIVE-23896
> URL: https://issues.apache.org/jira/browse/HIVE-23896
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.1.2
> Environment: hive: 3.1.2
> hadoop: 3.2.1, standalone, url: hdfs://namenode.hadoop.svc.cluster.local:9000
> {quote}$ $HADOOP_HOME/bin/hadoop fs -mkdir /tmp
> $ $HADOOP_HOME/bin/hadoop fs -mkdir /user/hive/warehouse
> {quote}
> hadoop commands are workable in the hiveserver node(POD).
>
>Reporter: alanwake
>Assignee: alanwake
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
>
>
> i try deply hive 3.1.2 on k8s. it was worked on version 2.3.2.
> metastore node and postgres node are ok, but hiveserver look like i miss some
> important configuration properties?
> {code:java}
> {code}
>
>
>
> {code:java}
> [root@master hive]# ./get.sh
> NAME READY STATUSRESTARTS AGE IP
> NODE NOMINATED NODE READINESS GATES
> hive-7bd48747d4-5zjmh1/1 Running 0 56s 10.244.3.110
> node03.51.local
> metastore-66b58f9f76-6wsxj 1/1 Running 0 56s 10.244.3.109
> node03.51.local
> postgres-57794b99b7-pqxwm1/1 Running 0 56s 10.244.2.241
> node02.51.local NAMETYPECLUSTER-IP
> EXTERNAL-IP PORT(S) AGE SELECTOR
> hiveNodePort10.108.40.17
> 10002:30626/TCP,1:31845/TCP 56s app=hive
> metastore ClusterIP 10.106.159.220 9083/TCP
>56s app=metastore
> postgresClusterIP 10.108.85.47 5432/TCP
>56s app=postgres
> {code}
>
>
> {code:java}
> [root@master hive]# kubectl logs hive-7bd48747d4-5zjmh -n=hive
> Configuring core
> - Setting hadoop.proxyuser.hue.hosts=*
> - Setting fs.defaultFS=hdfs://namenode.hadoop.svc.cluster.local:9000
> - Setting hadoop.http.staticuser.user=root
> - Setting hadoop.proxyuser.hue.groups=*
> Configuring hdfs
> - Setting dfs.namenode.datanode.registration.ip-hostname-check=false
> - Setting dfs.webhdfs.enabled=true
> - Setting dfs.permissions.enabled=false
> Configuring yarn
> - Setting yarn.timeline-service.enabled=true
> - Setting yarn.resourcemanager.system-metrics-publisher.enabled=true
> - Setting
> yarn.resourcemanager.store.class=org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore
> - Setting
> yarn.log.server.url=http://historyserver.hadoop.svc.cluster.local:8188/applicationhistory/logs/
> - Setting yarn.resourcemanager.fs.state-store.uri=/rmstate
> - Setting yarn.timeline-service.generic-application-history.enabled=true
> - Setting yarn.log-aggregation-enable=true
> - Setting
> yarn.resourcemanager.hostname=resourcemanager.hadoop.svc.cluster.local
> - Setting
> yarn.resourcemanager.resource.tracker.address=resourcemanager.hadoop.svc.cluster.local:8031
> - Setting
> yarn.timeline-service.hostname=historyserver.hadoop.svc.cluster.local
> - Setting
> yarn.resourcemanager.scheduler.address=resourcemanager.hadoop.svc.cluster.local:8030
> - Setting
> yarn.resourcemanager.address=resourcemanager.hadoop.svc.cluster.local:8032
> - Setting yarn.nodemanager.remote-app-log-dir=/app-logs
> - Setting yarn.resourcemanager.recovery.enabled=true
> Configuring httpfs
> Configuring kms
> Configuring mapred
> Configuring hive
> - Setting datanucleus.autoCreateSchema=false
> - Setting javax.jdo.option.ConnectionPassword=hive
> - Setting hive.metastore.uris=thrift://metastore:9083
> - Setting
> javax.jdo.option.ConnectionURL=jdbc:postgresql://metastore/metastore
> - Setting javax.jdo.option.ConnectionUserName=hive
> - Setting javax.jdo.option.ConnectionDriverName=org.postgresql.Driver
> Configuring for multihomed network
> [1/100] check for metastore:9083...
> [1/100] metastore:9083 is not available yet
> [1/100] try in 5s once again ...
> [2/100] check for metastore:9083...
> [2/100] metastore:9083 is not available yet
> [2/100] try in 5s once again ...
> [3/100] check for metastore:9083...
> [3/100] metastore:9083 is not available yet
> [3/100] try in 5s once again ...
> [4/100] check for metastore:9083...
> [4/100] metastore:9083 is not available yet
> [4/100] try in 5s once again ...
> [5/100]
[jira] [Updated] (HIVE-24324) Remove deprecated API usage from Avro
[
https://issues.apache.org/jira/browse/HIVE-24324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated HIVE-24324:
Fix Version/s: 3.1.3
> Remove deprecated API usage from Avro
> -
>
> Key: HIVE-24324
> URL: https://issues.apache.org/jira/browse/HIVE-24324
> Project: Hive
> Issue Type: Improvement
> Components: Avro
>Reporter: Chao Sun
>Assignee: Chao Sun
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.3.8, 3.1.3, 4.0.0
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> {{JsonProperties#getJsonProp}} has been marked as deprecated in Avro 1.8 and
> removed since Avro 1.9. This replaces the API usage for this with
> {{getObjectProp}} which doesn't leak Json node from jackson. This will help
> downstream apps to depend on Hive while using higher version of Avro, and
> also help Hive to upgrade Avro version itself.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-19228) Remove commons-httpclient 3.x usage
[ https://issues.apache.org/jira/browse/HIVE-19228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-19228: Fix Version/s: 2.3.9 > Remove commons-httpclient 3.x usage > --- > > Key: HIVE-19228 > URL: https://issues.apache.org/jira/browse/HIVE-19228 > Project: Hive > Issue Type: Improvement >Reporter: Janaki Lahorani >Assignee: Janaki Lahorani >Priority: Major > Labels: pull-request-available > Fix For: 3.1.0, 2.3.9, 4.0.0 > > Attachments: HIVE-19228.1.patch, HIVE-19228.2.patch, > HIVE-19228.3.patch, HIVE-19228.4.patch, HIVE-19228.4.patch, > HIVE-19228.4.patch, HIVE-19228.5.patch, HIVE-19228.6.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Commons-httpclient is not supported well anymore. Remove dependency and move > to Apache HTTP client. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-24608) Switch back to get_table in HMS client for Hive 2.3.x
[
https://issues.apache.org/jira/browse/HIVE-24608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun resolved HIVE-24608.
-
Fix Version/s: 2.3.9
Hadoop Flags: Reviewed
Assignee: Chao Sun
Resolution: Fixed
> Switch back to get_table in HMS client for Hive 2.3.x
> -
>
> Key: HIVE-24608
> URL: https://issues.apache.org/jira/browse/HIVE-24608
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 2.3.7
>Reporter: Chao Sun
>Assignee: Chao Sun
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.3.9
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> HIVE-15062 introduced a backward-incompatible change by replacing
> {{get_table}} with {{get_table_req}}. As consequence, when HMS client w/
> version > 2.3 talks to a HMS w/ version < 2.3, it will get error similar to
> the following:
> {code}
> AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: Unable
> to fetch table testpartitiondata. Invalid method name: 'get_table_req';
> {code}
> Looking at HIVE-15062, the {{get_table_req}} is to introduce client-side
> check for capabilities. However in branch-2.3 the check is a no-op since
> there is no capability yet (it is assigned to null). Therefore, this JIRA
> proposes to switch back to {{get_table}} in branch-2.3 to fix the
> compatibility issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-18147) Tests can fail with java.net.BindException: Address already in use
[ https://issues.apache.org/jira/browse/HIVE-18147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-18147: Fix Version/s: 2.3.9 > Tests can fail with java.net.BindException: Address already in use > -- > > Key: HIVE-18147 > URL: https://issues.apache.org/jira/browse/HIVE-18147 > Project: Hive > Issue Type: Bug >Reporter: Janaki Lahorani >Assignee: Janaki Lahorani >Priority: Major > Labels: pull-request-available > Fix For: 2.3.9, 2.4.0, 3.0.0 > > Attachments: HIVE-18147-branch-2.01.patch, HIVE-18147.1.patch, > HIVE-18147.2.patch, HIVE-18147.3.patch, HIVE-18147.4.patch, > HIVE-18147.5-branch-2.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Between the time a new port is picked up to start HMS or HS2, and the time > when HMS or HS2 is started, the port can get used by some other service > resulting in test failure. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21563) Improve Table#getEmptyTable performance by disable registerAllFunctionsOnce
[
https://issues.apache.org/jira/browse/HIVE-21563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated HIVE-21563:
Fix Version/s: 3.2.0
3.1.3
> Improve Table#getEmptyTable performance by disable registerAllFunctionsOnce
> ---
>
> Key: HIVE-21563
> URL: https://issues.apache.org/jira/browse/HIVE-21563
> Project: Hive
> Issue Type: Improvement
>Reporter: Yuming Wang
>Assignee: Yuming Wang
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.3.9, 3.1.3, 3.2.0, 4.0.0
>
> Attachments: HIVE-21563.001.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> We do not need registerAllFunctionsOnce when {{Table#getEmptyTable}}. The
> stack trace:
> {noformat}
> at
> org.apache.hadoop.hive.ql.exec.Registry.registerGenericUDF(Registry.java:177)
> at
> org.apache.hadoop.hive.ql.exec.Registry.registerGenericUDF(Registry.java:170)
> at
> org.apache.hadoop.hive.ql.exec.FunctionRegistry.(FunctionRegistry.java:209)
> at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:247)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:231)
> at org.apache.hadoop.hive.ql.metadata.Hive.(Hive.java:388)
> at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:332)
> at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:312)
> at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:288)
> at
> org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:913)
> at
> org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:877)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getAuthenticator(SessionState.java:1479)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getUserFromAuthenticator(SessionState.java:1150)
> at org.apache.hadoop.hive.ql.metadata.Table.getEmptyTable(Table.java:180)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-21563) Improve Table#getEmptyTable performance by disable registerAllFunctionsOnce
[
https://issues.apache.org/jira/browse/HIVE-21563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated HIVE-21563:
Fix Version/s: 2.3.9
> Improve Table#getEmptyTable performance by disable registerAllFunctionsOnce
> ---
>
> Key: HIVE-21563
> URL: https://issues.apache.org/jira/browse/HIVE-21563
> Project: Hive
> Issue Type: Improvement
>Reporter: Yuming Wang
>Assignee: Yuming Wang
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.3.9, 4.0.0
>
> Attachments: HIVE-21563.001.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> We do not need registerAllFunctionsOnce when {{Table#getEmptyTable}}. The
> stack trace:
> {noformat}
> at
> org.apache.hadoop.hive.ql.exec.Registry.registerGenericUDF(Registry.java:177)
> at
> org.apache.hadoop.hive.ql.exec.Registry.registerGenericUDF(Registry.java:170)
> at
> org.apache.hadoop.hive.ql.exec.FunctionRegistry.(FunctionRegistry.java:209)
> at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:247)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:231)
> at org.apache.hadoop.hive.ql.metadata.Hive.(Hive.java:388)
> at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:332)
> at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:312)
> at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:288)
> at
> org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:913)
> at
> org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:877)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getAuthenticator(SessionState.java:1479)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getUserFromAuthenticator(SessionState.java:1150)
> at org.apache.hadoop.hive.ql.metadata.Table.getEmptyTable(Table.java:180)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-21563) Improve Table#getEmptyTable performance by disable registerAllFunctionsOnce
[
https://issues.apache.org/jira/browse/HIVE-21563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated HIVE-21563:
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Improve Table#getEmptyTable performance by disable registerAllFunctionsOnce
> ---
>
> Key: HIVE-21563
> URL: https://issues.apache.org/jira/browse/HIVE-21563
> Project: Hive
> Issue Type: Improvement
>Reporter: Yuming Wang
>Assignee: Yuming Wang
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21563.001.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> We do not need registerAllFunctionsOnce when {{Table#getEmptyTable}}. The
> stack trace:
> {noformat}
> at
> org.apache.hadoop.hive.ql.exec.Registry.registerGenericUDF(Registry.java:177)
> at
> org.apache.hadoop.hive.ql.exec.Registry.registerGenericUDF(Registry.java:170)
> at
> org.apache.hadoop.hive.ql.exec.FunctionRegistry.(FunctionRegistry.java:209)
> at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:247)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:231)
> at org.apache.hadoop.hive.ql.metadata.Hive.(Hive.java:388)
> at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:332)
> at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:312)
> at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:288)
> at
> org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:913)
> at
> org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:877)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getAuthenticator(SessionState.java:1479)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getUserFromAuthenticator(SessionState.java:1150)
> at org.apache.hadoop.hive.ql.metadata.Table.getEmptyTable(Table.java:180)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-24922) Remove redundant code and improve readability
[ https://issues.apache.org/jira/browse/HIVE-24922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-24922: Priority: Trivial (was: Minor) > Remove redundant code and improve readability > - > > Key: HIVE-24922 > URL: https://issues.apache.org/jira/browse/HIVE-24922 > Project: Hive > Issue Type: Improvement > Components: SQL >Affects Versions: 3.1.2 >Reporter: hao >Assignee: hao >Priority: Trivial > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 2h > Remaining Estimate: 0h > > Remove redundant code and improve readability -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-24922) Remove redundant code and improve readability
[ https://issues.apache.org/jira/browse/HIVE-24922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned HIVE-24922: --- Assignee: hao > Remove redundant code and improve readability > - > > Key: HIVE-24922 > URL: https://issues.apache.org/jira/browse/HIVE-24922 > Project: Hive > Issue Type: Improvement > Components: SQL >Affects Versions: 3.1.2 >Reporter: hao >Assignee: hao >Priority: Minor > Labels: pull-request-available > Fix For: 3.1.2 > > Time Spent: 2h > Remaining Estimate: 0h > > Remove redundant code and improve readability -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-24922) Remove redundant code and improve readability
[ https://issues.apache.org/jira/browse/HIVE-24922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HIVE-24922. - Fix Version/s: (was: 3.1.2) 4.0.0 Resolution: Fixed > Remove redundant code and improve readability > - > > Key: HIVE-24922 > URL: https://issues.apache.org/jira/browse/HIVE-24922 > Project: Hive > Issue Type: Improvement > Components: SQL >Affects Versions: 3.1.2 >Reporter: hao >Assignee: hao >Priority: Minor > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 2h > Remaining Estimate: 0h > > Remove redundant code and improve readability -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24608) Switch back to get_table in HMS client for Hive 2.3.x
[
https://issues.apache.org/jira/browse/HIVE-24608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated HIVE-24608:
Target Version/s: 2.3.9
> Switch back to get_table in HMS client for Hive 2.3.x
> -
>
> Key: HIVE-24608
> URL: https://issues.apache.org/jira/browse/HIVE-24608
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 2.3.7
>Reporter: Chao Sun
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> HIVE-15062 introduced a backward-incompatible change by replacing
> {{get_table}} with {{get_table_req}}. As consequence, when HMS client w/
> version > 2.3 talks to a HMS w/ version < 2.3, it will get error similar to
> the following:
> {code}
> AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: Unable
> to fetch table testpartitiondata. Invalid method name: 'get_table_req';
> {code}
> Looking at HIVE-15062, the {{get_table_req}} is to introduce client-side
> check for capabilities. However in branch-2.3 the check is a no-op since
> there is no capability yet (it is assigned to null). Therefore, this JIRA
> proposes to switch back to {{get_table}} in branch-2.3 to fix the
> compatibility issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-24797) Disable validate default values when parsing Avro schemas
[
https://issues.apache.org/jira/browse/HIVE-24797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated HIVE-24797:
Fix Version/s: 2.3.9
> Disable validate default values when parsing Avro schemas
> -
>
> Key: HIVE-24797
> URL: https://issues.apache.org/jira/browse/HIVE-24797
> Project: Hive
> Issue Type: Bug
>Reporter: Yuming Wang
>Assignee: Yuming Wang
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.3.9, 4.0.0
>
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> It will throw exceptions when upgrading Avro to 1.10.1 for this schema:
> {code:json}
> {
> "type": "record",
> "name": "EventData",
> "doc": "event data",
> "fields": [
> {"name": "ARRAY_WITH_DEFAULT", "type": {"type": "array", "items":
> "string"}, "default": null }
> ]
> }
> {code}
> {noformat}
> org.apache.avro.AvroTypeException: Invalid default for field
> ARRAY_WITH_DEFAULT: null not a {"type":"array","items":"string"}
> at org.apache.avro.Schema.validateDefault(Schema.java:1571)
> at org.apache.avro.Schema.access$500(Schema.java:87)
> at org.apache.avro.Schema$Field.(Schema.java:544)
> at org.apache.avro.Schema.parse(Schema.java:1678)
> at org.apache.avro.Schema$Parser.parse(Schema.java:1425)
> at org.apache.avro.Schema$Parser.parse(Schema.java:1396)
> at
> org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils.getSchemaFor(AvroSerdeUtils.java:287)
> at
> org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils.getSchemaFromFS(AvroSerdeUtils.java:170)
> at
> org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils.determineSchemaOrThrowException(AvroSerdeUtils.java:139)
> at
> org.apache.hadoop.hive.serde2.avro.AvroSerDe.determineSchemaOrReturnErrorSchema(AvroSerDe.java:187)
> at
> org.apache.hadoop.hive.serde2.avro.AvroSerDe.initialize(AvroSerDe.java:107)
> at
> org.apache.hadoop.hive.serde2.avro.AvroSerDe.initialize(AvroSerDe.java:83)
> at
> org.apache.hadoop.hive.serde2.SerDeUtils.initializeSerDe(SerDeUtils.java:533)
> at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:493)
> at
> org.apache.hadoop.hive.ql.metadata.Partition.getDeserializer(Partition.java:225)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HIVE-24797) Disable validate default values when parsing Avro schemas
[
https://issues.apache.org/jira/browse/HIVE-24797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun resolved HIVE-24797.
-
Fix Version/s: 4.0.0
Hadoop Flags: Reviewed
Assignee: Yuming Wang
Resolution: Fixed
> Disable validate default values when parsing Avro schemas
> -
>
> Key: HIVE-24797
> URL: https://issues.apache.org/jira/browse/HIVE-24797
> Project: Hive
> Issue Type: Bug
>Reporter: Yuming Wang
>Assignee: Yuming Wang
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> It will throw exceptions when upgrading Avro to 1.10.1 for this schema:
> {code:json}
> {
> "type": "record",
> "name": "EventData",
> "doc": "event data",
> "fields": [
> {"name": "ARRAY_WITH_DEFAULT", "type": {"type": "array", "items":
> "string"}, "default": null }
> ]
> }
> {code}
> {noformat}
> org.apache.avro.AvroTypeException: Invalid default for field
> ARRAY_WITH_DEFAULT: null not a {"type":"array","items":"string"}
> at org.apache.avro.Schema.validateDefault(Schema.java:1571)
> at org.apache.avro.Schema.access$500(Schema.java:87)
> at org.apache.avro.Schema$Field.(Schema.java:544)
> at org.apache.avro.Schema.parse(Schema.java:1678)
> at org.apache.avro.Schema$Parser.parse(Schema.java:1425)
> at org.apache.avro.Schema$Parser.parse(Schema.java:1396)
> at
> org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils.getSchemaFor(AvroSerdeUtils.java:287)
> at
> org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils.getSchemaFromFS(AvroSerdeUtils.java:170)
> at
> org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils.determineSchemaOrThrowException(AvroSerdeUtils.java:139)
> at
> org.apache.hadoop.hive.serde2.avro.AvroSerDe.determineSchemaOrReturnErrorSchema(AvroSerDe.java:187)
> at
> org.apache.hadoop.hive.serde2.avro.AvroSerDe.initialize(AvroSerDe.java:107)
> at
> org.apache.hadoop.hive.serde2.avro.AvroSerDe.initialize(AvroSerDe.java:83)
> at
> org.apache.hadoop.hive.serde2.SerDeUtils.initializeSerDe(SerDeUtils.java:533)
> at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:493)
> at
> org.apache.hadoop.hive.ql.metadata.Partition.getDeserializer(Partition.java:225)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HIVE-24559) Fix some spelling issues
[ https://issues.apache.org/jira/browse/HIVE-24559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HIVE-24559. - Fix Version/s: 4.0.0 2.3.8 Resolution: Fixed > Fix some spelling issues > > > Key: HIVE-24559 > URL: https://issues.apache.org/jira/browse/HIVE-24559 > Project: Hive > Issue Type: Improvement >Reporter: RickyMa >Assignee: RickyMa >Priority: Trivial > Labels: pull-request-available > Fix For: 2.3.8, 4.0.0 > > Time Spent: 1h 40m > Remaining Estimate: 0h > > There are some minor typos: > [https://github.com/apache/hive/pull/1805/fileshttps://github.com/apache/hive/pull/1805/fileshttps://github.com/apache/hive/blob/branch-2.3/metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L858|https://github.com/apache/hive/blob/branch-2.3/metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L858] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-24559) Fix some spelling issues
[ https://issues.apache.org/jira/browse/HIVE-24559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned HIVE-24559: --- Assignee: RickyMa > Fix some spelling issues > > > Key: HIVE-24559 > URL: https://issues.apache.org/jira/browse/HIVE-24559 > Project: Hive > Issue Type: Improvement >Reporter: RickyMa >Assignee: RickyMa >Priority: Trivial > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > > There are some minor typos: > [https://github.com/apache/hive/pull/1805/fileshttps://github.com/apache/hive/pull/1805/fileshttps://github.com/apache/hive/blob/branch-2.3/metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L858|https://github.com/apache/hive/blob/branch-2.3/metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L858] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24608) Switch back to get_table in HMS client for Hive 2.3.x
[
https://issues.apache.org/jira/browse/HIVE-24608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated HIVE-24608:
Summary: Switch back to get_table in HMS client for Hive 2.3.x (was:
Switch back to get_table in HMS client)
> Switch back to get_table in HMS client for Hive 2.3.x
> -
>
> Key: HIVE-24608
> URL: https://issues.apache.org/jira/browse/HIVE-24608
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 2.3.7
>Reporter: Chao Sun
>Priority: Major
>
> HIVE-15062 introduced a backward-incompatible change by replacing
> {{get_table}} with {{get_table_req}}. As consequence, when HMS client w/
> version > 2.3 talks to a HMS w/ version < 2.3, it will get error similar to
> the following:
> {code}
> AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: Unable
> to fetch table testpartitiondata. Invalid method name: 'get_table_req';
> {code}
> Looking at HIVE-15062, the {{get_table_req}} is to introduce client-side
> check for capabilities. However in branch-2.3 the check is a no-op since
> there is no capability yet (it is assigned to null). Therefore, this JIRA
> proposes to switch back to {{get_table}} in branch-2.3 to fix the
> compatibility issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-24608) Switch back to get_table in HMS client
[
https://issues.apache.org/jira/browse/HIVE-24608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17261527#comment-17261527
]
Chao Sun commented on HIVE-24608:
-
cc [~sershe], [~thejas]
> Switch back to get_table in HMS client
> --
>
> Key: HIVE-24608
> URL: https://issues.apache.org/jira/browse/HIVE-24608
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 2.3.7
>Reporter: Chao Sun
>Priority: Major
>
> HIVE-15062 introduced a backward-incompatible change by replacing
> {{get_table}} with {{get_table_req}}. As consequence, when HMS client w/
> version > 2.3 talks to a HMS w/ version < 2.3, it will get error similar to
> the following:
> {code}
> AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: Unable
> to fetch table testpartitiondata. Invalid method name: 'get_table_req';
> {code}
> Looking at HIVE-15062, the {{get_table_req}} is to introduce client-side
> check for capabilities. However in branch-2.3 the check is a no-op since
> there is no capability yet (it is assigned to null). Therefore, this JIRA
> proposes to switch back to {{get_table}} in branch-2.3 to fix the
> compatibility issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HIVE-21961) Update jetty version to 9.4.x
[ https://issues.apache.org/jira/browse/HIVE-21961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256699#comment-17256699 ] Chao Sun edited comment on HIVE-21961 at 12/30/20, 8:07 PM: HADOOP-16152 is also in Hadoop 3.1.4 so can we upgrade to that instead [~abstractdog]? it could be easier since Hive is now on Hadoop 3.1.0. was (Author: csun): HADOOP-16152 is also in Hadoop 3.1.4 so can we upgrade to that instead? it could be easier since Hive is now on Hadoop 3.1.0. > Update jetty version to 9.4.x > - > > Key: HIVE-21961 > URL: https://issues.apache.org/jira/browse/HIVE-21961 > Project: Hive > Issue Type: Task >Reporter: Oleksiy Sayankin >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21961.02.patch, HIVE-21961.03.patch, > HIVE-21961.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Update jetty version to 9.4.x -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-21961) Update jetty version to 9.4.x
[ https://issues.apache.org/jira/browse/HIVE-21961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256699#comment-17256699 ] Chao Sun commented on HIVE-21961: - HADOOP-16152 is also in Hadoop 3.1.4 so can we upgrade to that instead? it could be easier since Hive is now on Hadoop 3.1.0. > Update jetty version to 9.4.x > - > > Key: HIVE-21961 > URL: https://issues.apache.org/jira/browse/HIVE-21961 > Project: Hive > Issue Type: Task >Reporter: Oleksiy Sayankin >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21961.02.patch, HIVE-21961.03.patch, > HIVE-21961.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Update jetty version to 9.4.x -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-21961) Update jetty version to 9.4.x
[ https://issues.apache.org/jira/browse/HIVE-21961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256696#comment-17256696 ] Chao Sun commented on HIVE-21961: - I think the guava upgrade in 3.2.2 may cause a bigger headache than this since Hive depends on Spark which uses Guava 14.x as for now. cc [~ayushtkn] > Update jetty version to 9.4.x > - > > Key: HIVE-21961 > URL: https://issues.apache.org/jira/browse/HIVE-21961 > Project: Hive > Issue Type: Task >Reporter: Oleksiy Sayankin >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21961.02.patch, HIVE-21961.03.patch, > HIVE-21961.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Update jetty version to 9.4.x -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-24551) Hive should include transitive dependencies from calcite after shading it
[
https://issues.apache.org/jira/browse/HIVE-24551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun resolved HIVE-24551.
-
Fix Version/s: 2.3.8
Hadoop Flags: Reviewed
Resolution: Fixed
> Hive should include transitive dependencies from calcite after shading it
> -
>
> Key: HIVE-24551
> URL: https://issues.apache.org/jira/browse/HIVE-24551
> Project: Hive
> Issue Type: Bug
> Components: SQL
>Affects Versions: 2.3.8
>Reporter: Chao Sun
>Assignee: Chao Sun
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 2.3.8
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Currently as part of effort of shading Guava from Hive, we shade Calcite and
> exclude all its artifacts from the binary distribution. However, this also
> removes all its transitive dependencies which will still needed at runtime.
> Without these, Hive queries will fail with error like:
> {code}
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/eigenbase/util/property/BooleanProperty
> at
> org.apache.calcite.util.SaffronProperties.(SaffronProperties.java:66)
> at
> org.apache.calcite.util.SaffronProperties.instance(SaffronProperties.java:134)
> at org.apache.calcite.util.Util.getDefaultCharset(Util.java:769)
> at
> org.apache.calcite.rel.type.RelDataTypeFactoryImpl.getDefaultCharset(RelDataTypeFactoryImpl.java:565)
> at
> org.apache.calcite.sql.type.SqlTypeUtil.addCharsetAndCollation(SqlTypeUtil.java:1070)
> at
> org.apache.calcite.sql.type.SqlTypeFactoryImpl.createSqlType(SqlTypeFactoryImpl.java:65)
> at org.apache.calcite.rex.RexBuilder.(RexBuilder.java:114)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:991)
> at
> org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:149)
> at
> org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:106)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.logicalPlan(CalcitePlanner.java:1069)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.getOptimizedAST(CalcitePlanner.java:1085)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:364)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:11138)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:286)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:258)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:512)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:234)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
> Caused by: java.lang.ClassNotFoundException:
> org.eigenbase.util.property.BooleanProperty
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> ... 33 more
> {code}
> This happens in branch-2.3 but the same thing might happen in master as well
> - although it uses calcite 1.21 which has changed a lot from 1.10 that
> branch-2.3 is using. For instance, in calcite 1.21 it no longer depends on
> org.eigenbase
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HIVE-24553) Exclude calcite from test-jar dependency of hive-exec
[
https://issues.apache.org/jira/browse/HIVE-24553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun resolved HIVE-24553.
-
Fix Version/s: 2.3.8
Hadoop Flags: Reviewed
Assignee: L. C. Hsieh
Resolution: Fixed
> Exclude calcite from test-jar dependency of hive-exec
> -
>
> Key: HIVE-24553
> URL: https://issues.apache.org/jira/browse/HIVE-24553
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.3.8
>Reporter: L. C. Hsieh
>Assignee: L. C. Hsieh
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 2.3.8
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The test "org.apache.hive.beeline.cli.TestHiveCli - testSetHeaderValue" we
> found the exception:
> {code}
> Caused by: java.lang.NoSuchMethodError:
> org.apache.calcite.rel.RelCollationImpl.(Lorg/apache/hive/com/google/common/collect/ImmutableList;)V
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelCollation.(HiveRelCollation.java:29)
> ~[hive-exec-2.3.8.jar:2.3.8]
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.RelOptHiveTable.getCollationList(RelOptHiveTable.java:181)
> ~[hive-exec-2.3.8.jar:2.3.8]
> at
> org.apache.calcite.rel.metadata.RelMdCollation.table(RelMdCollation.java:175)
> ~[calcite-core-1.10.0.jar:1.10.0]
> at
> org.apache.calcite.rel.metadata.RelMdCollation.collations(RelMdCollation.java:122)
> ~[calcite-core-1.10.0.jar:1.10.0]
> at GeneratedMetadataHandler_Collation.collations_$(Unknown Source)
> ~[?:?]
> at GeneratedMetadataHandler_Collation.collations(Unknown Source) ~[?:?]
> at
> org.apache.calcite.rel.metadata.RelMetadataQuery.collations(RelMetadataQuery.java:482)
> ~[calcite-core-1.10.0.jar:1.10.0]
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:189)
> ~[calcite-core-1.10.0.jar:1.10.0]
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:374)
> ~[calcite-core-1.10.0.jar:1.10.0]
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(HiveRelFieldTrimmer.java:273)
> ~[hive-exec-2.3.8.jar:2.3.8]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_262]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_262]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_262]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_262]
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
> ~[calcite-core-1.10.0.jar:1.10.0]
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:270)
> ~[calcite-core-1.10.0.jar:1.10.0]
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trim(RelFieldTrimmer.java:160)
> ~[calcite-core-1.10.0.jar:1.10.0]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1331)
> ~[hive-exec-2.3.8.jar:2.3.8]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1261)
> ~[hive-exec-2.3.8.jar:2.3.8]
> at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:113)
> ~[calcite-core-1.10.0.jar:1.10.0]
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:997)
> ~[calcite-core-1.10.0.jar:1.10.0]
> at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:149)
> ~[calcite-core-1.10.0.jar:1.10.0]
> at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:106)
> ~[calcite-core-1.10.0.jar:1.10.0]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.logicalPlan(CalcitePlanner.java:1069)
> ~[hive-exec-2.3.8.jar:2.3.8]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.getOptimizedAST(CalcitePlanner.java:1085)
> ~[hive-exec-2.3.8.jar:2.3.8]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:364)
> ~[hive-exec-2.3.8.jar:2.3.8]
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:11138)
> ~[hive-exec-2.3.8.jar:2.3.8]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:286)
> ~[hive-exec-2.3.8.jar:2.3.8]
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:258)
> ~[hive-exec-2.3.8.jar:2.3.8]
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:512)
> ~[hive-exec-2.3.8.jar:2.3.8]
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
> ~[hive-exec-2.3.8.jar:2.3.8]
> at
[jira] [Updated] (HIVE-24551) Hive should include transitive dependencies from calcite after shading it
[
https://issues.apache.org/jira/browse/HIVE-24551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated HIVE-24551:
Description:
Currently as part of effort of shading Guava from Hive, we shade Calcite and
exclude all its artifacts from the binary distribution. However, this also
removes all its transitive dependencies which will still needed at runtime.
Without these, Hive queries will fail with error like:
{code}
Exception in thread "main" java.lang.NoClassDefFoundError:
org/eigenbase/util/property/BooleanProperty
at
org.apache.calcite.util.SaffronProperties.(SaffronProperties.java:66)
at
org.apache.calcite.util.SaffronProperties.instance(SaffronProperties.java:134)
at org.apache.calcite.util.Util.getDefaultCharset(Util.java:769)
at
org.apache.calcite.rel.type.RelDataTypeFactoryImpl.getDefaultCharset(RelDataTypeFactoryImpl.java:565)
at
org.apache.calcite.sql.type.SqlTypeUtil.addCharsetAndCollation(SqlTypeUtil.java:1070)
at
org.apache.calcite.sql.type.SqlTypeFactoryImpl.createSqlType(SqlTypeFactoryImpl.java:65)
at org.apache.calcite.rex.RexBuilder.(RexBuilder.java:114)
at
org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:991)
at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:149)
at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:106)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.logicalPlan(CalcitePlanner.java:1069)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.getOptimizedAST(CalcitePlanner.java:1085)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:364)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:11138)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:286)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:258)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:512)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:234)
at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
Caused by: java.lang.ClassNotFoundException:
org.eigenbase.util.property.BooleanProperty
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 33 more
{code}
This happens in branch-2.3 but the same thing might happen in master as well -
although it uses calcite 1.21 which has changed a lot from 1.10 that branch-2.3
is using. For instance, in calcite 1.21 it no longer depends on org.eigenbase
was:
Currently as part of effort of shading Guava from Hive, we shade Calcite and
exclude all its artifacts from the binary distribution. However, this also
removes all its transitive dependencies which will still needed at runtime.
Without these, Hive queries will fail with error like:
{code}
Exception in thread "main" java.lang.NoClassDefFoundError:
org/eigenbase/util/property/BooleanProperty
at
org.apache.calcite.util.SaffronProperties.(SaffronProperties.java:66)
at
org.apache.calcite.util.SaffronProperties.instance(SaffronProperties.java:134)
at org.apache.calcite.util.Util.getDefaultCharset(Util.java:769)
at
org.apache.calcite.rel.type.RelDataTypeFactoryImpl.getDefaultCharset(RelDataTypeFactoryImpl.java:565)
at
org.apache.calcite.sql.type.SqlTypeUtil.addCharsetAndCollation(SqlTypeUtil.java:1070)
at
org.apache.calcite.sql.type.SqlTypeFactoryImpl.createSqlType(SqlTypeFactoryImpl.java:65)
at
[jira] [Updated] (HIVE-24551) Hive should include transitive dependencies from calcite after shading it
[
https://issues.apache.org/jira/browse/HIVE-24551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated HIVE-24551:
Affects Version/s: 2.3.8
> Hive should include transitive dependencies from calcite after shading it
> -
>
> Key: HIVE-24551
> URL: https://issues.apache.org/jira/browse/HIVE-24551
> Project: Hive
> Issue Type: Bug
> Components: SQL
>Affects Versions: 2.3.8
>Reporter: Chao Sun
>Assignee: Chao Sun
>Priority: Blocker
>
> Currently as part of effort of shading Guava from Hive, we shade Calcite and
> exclude all its artifacts from the binary distribution. However, this also
> removes all its transitive dependencies which will still needed at runtime.
> Without these, Hive queries will fail with error like:
> {code}
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/eigenbase/util/property/BooleanProperty
> at
> org.apache.calcite.util.SaffronProperties.(SaffronProperties.java:66)
> at
> org.apache.calcite.util.SaffronProperties.instance(SaffronProperties.java:134)
> at org.apache.calcite.util.Util.getDefaultCharset(Util.java:769)
> at
> org.apache.calcite.rel.type.RelDataTypeFactoryImpl.getDefaultCharset(RelDataTypeFactoryImpl.java:565)
> at
> org.apache.calcite.sql.type.SqlTypeUtil.addCharsetAndCollation(SqlTypeUtil.java:1070)
> at
> org.apache.calcite.sql.type.SqlTypeFactoryImpl.createSqlType(SqlTypeFactoryImpl.java:65)
> at org.apache.calcite.rex.RexBuilder.(RexBuilder.java:114)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:991)
> at
> org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:149)
> at
> org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:106)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.logicalPlan(CalcitePlanner.java:1069)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.getOptimizedAST(CalcitePlanner.java:1085)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:364)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:11138)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:286)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:258)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:512)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:234)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
> Caused by: java.lang.ClassNotFoundException:
> org.eigenbase.util.property.BooleanProperty
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> ... 33 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-24551) Hive should include transitive dependencies from calcite after shading it
[
https://issues.apache.org/jira/browse/HIVE-24551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun reassigned HIVE-24551:
---
> Hive should include transitive dependencies from calcite after shading it
> -
>
> Key: HIVE-24551
> URL: https://issues.apache.org/jira/browse/HIVE-24551
> Project: Hive
> Issue Type: Bug
> Components: SQL
>Reporter: Chao Sun
>Assignee: Chao Sun
>Priority: Blocker
>
> Currently as part of effort of shading Guava from Hive, we shade Calcite and
> exclude all its artifacts from the binary distribution. However, this also
> removes all its transitive dependencies which will still needed at runtime.
> Without these, Hive queries will fail with error like:
> {code}
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/eigenbase/util/property/BooleanProperty
> at
> org.apache.calcite.util.SaffronProperties.(SaffronProperties.java:66)
> at
> org.apache.calcite.util.SaffronProperties.instance(SaffronProperties.java:134)
> at org.apache.calcite.util.Util.getDefaultCharset(Util.java:769)
> at
> org.apache.calcite.rel.type.RelDataTypeFactoryImpl.getDefaultCharset(RelDataTypeFactoryImpl.java:565)
> at
> org.apache.calcite.sql.type.SqlTypeUtil.addCharsetAndCollation(SqlTypeUtil.java:1070)
> at
> org.apache.calcite.sql.type.SqlTypeFactoryImpl.createSqlType(SqlTypeFactoryImpl.java:65)
> at org.apache.calcite.rex.RexBuilder.(RexBuilder.java:114)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:991)
> at
> org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:149)
> at
> org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:106)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.logicalPlan(CalcitePlanner.java:1069)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.getOptimizedAST(CalcitePlanner.java:1085)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:364)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:11138)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:286)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:258)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:512)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:234)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
> Caused by: java.lang.ClassNotFoundException:
> org.eigenbase.util.property.BooleanProperty
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> ... 33 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22708) Test fix for http transport
[ https://issues.apache.org/jira/browse/HIVE-22708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-22708: Fix Version/s: 2.3.8 > Test fix for http transport > --- > > Key: HIVE-22708 > URL: https://issues.apache.org/jira/browse/HIVE-22708 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > Labels: pull-request-available > Fix For: 2.3.8, 4.0.0 > > Attachments: HIVE-22708.patch > > Time Spent: 2h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-24512) Exclude calcite in packaging Hive
[ https://issues.apache.org/jira/browse/HIVE-24512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HIVE-24512. - Fix Version/s: 2.3.8 Hadoop Flags: Reviewed Assignee: L. C. Hsieh Resolution: Fixed > Exclude calcite in packaging Hive > - > > Key: HIVE-24512 > URL: https://issues.apache.org/jira/browse/HIVE-24512 > Project: Hive > Issue Type: Bug >Affects Versions: 2.3.8 >Reporter: L. C. Hsieh >Assignee: L. C. Hsieh >Priority: Major > Labels: pull-request-available > Fix For: 2.3.8 > > Time Spent: 40m > Remaining Estimate: 0h > > The issue is similar to HIVE-23593. In 2.3.8 RC, ql already has a shaded > calcite, but we see such error: > Caused by: java.lang.NoSuchMethodError: > org.apache.calcite.rel.RelCollationImpl.(Lorg/apache/hive/com/google/common/collect/ImmutableList;)V > at > org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelCollation.(HiveRelCollation.java:29) > at > org.apache.hadoop.hive.ql.optimizer.calcite.RelOptHiveTable.getCollationList(RelOptHiveTable.java:181) > at > org.apache.calcite.rel.metadata.RelMdCollation.table(RelMdCollation.java:175) > We find in 2.3.8 binary distribution, there are calcite jars: > calcite-core-1.10.0.jar > calcite-druid-1.10.0.jar > calcite-linq4j-1.10.0.jar -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-24436) Fix Avro NULL_DEFAULT_VALUE compatibility issue
[
https://issues.apache.org/jira/browse/HIVE-24436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun resolved HIVE-24436.
-
Fix Version/s: 4.0.0
3.1.3
2.3.8
Hadoop Flags: Reviewed
Assignee: Yuming Wang
Resolution: Fixed
> Fix Avro NULL_DEFAULT_VALUE compatibility issue
> ---
>
> Key: HIVE-24436
> URL: https://issues.apache.org/jira/browse/HIVE-24436
> Project: Hive
> Issue Type: Improvement
> Components: Avro
>Affects Versions: 2.3.8
>Reporter: Yuming Wang
>Assignee: Yuming Wang
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.3.8, 3.1.3, 4.0.0
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Exception1:
> {noformat}
> - create hive serde table with Catalog
> *** RUN ABORTED ***
> java.lang.NoSuchMethodError: 'void
> org.apache.avro.Schema$Field.(java.lang.String, org.apache.avro.Schema,
> java.lang.String, org.codehaus.jackson.JsonNode)'
> at
> org.apache.hadoop.hive.serde2.avro.TypeInfoToSchema.createAvroField(TypeInfoToSchema.java:76)
> at
> org.apache.hadoop.hive.serde2.avro.TypeInfoToSchema.convert(TypeInfoToSchema.java:61)
> at
> org.apache.hadoop.hive.serde2.avro.AvroSerDe.getSchemaFromCols(AvroSerDe.java:170)
> at
> org.apache.hadoop.hive.serde2.avro.AvroSerDe.initialize(AvroSerDe.java:114)
> at
> org.apache.hadoop.hive.serde2.avro.AvroSerDe.initialize(AvroSerDe.java:83)
> at
> org.apache.hadoop.hive.serde2.SerDeUtils.initializeSerDe(SerDeUtils.java:533)
> at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:450)
> at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:437)
> at
> org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:281)
> at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:263)
> {noformat}
> Exception2:
> {noformat}
> - alter hive serde table add columns -- partitioned - AVRO *** FAILED ***
> org.apache.spark.sql.AnalysisException:
> org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.avro.AvroRuntimeException: Unknown datum class: class
> org.codehaus.jackson.node.NullNode;
> at
> org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:112)
> at
> org.apache.spark.sql.hive.HiveExternalCatalog.createTable(HiveExternalCatalog.scala:245)
> at
> org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.createTable(ExternalCatalogWithListener.scala:94)
> at
> org.apache.spark.sql.catalyst.catalog.SessionCatalog.createTable(SessionCatalog.scala:346)
> at
> org.apache.spark.sql.execution.command.CreateTableCommand.run(tables.scala:166)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:79)
> at org.apache.spark.sql.Dataset.$anonfun$logicalPlan$1(Dataset.scala:228)
> at org.apache.spark.sql.Dataset.$anonfun$withAction$1(Dataset.scala:3680)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HIVE-23980) Shade guava from existing Hive versions
[
https://issues.apache.org/jira/browse/HIVE-23980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun resolved HIVE-23980.
-
Resolution: Fixed
> Shade guava from existing Hive versions
> ---
>
> Key: HIVE-23980
> URL: https://issues.apache.org/jira/browse/HIVE-23980
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.3.7
>Reporter: L. C. Hsieh
>Assignee: L. C. Hsieh
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.3.8
>
> Attachments: HIVE-23980.01.branch-2.3.patch
>
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
> I'm trying to upgrade Guava version in Spark. The JIRA ticket is SPARK-32502.
> Running test hits an error:
> {code}
> sbt.ForkMain$ForkError: sbt.ForkMain$ForkError: java.lang.IllegalAccessError:
> tried to access method
> com.google.common.collect.Iterators.emptyIterator()Lcom/google/common/collect/UnmodifiableIterator;
> from class org.apache.hadoop.hive.ql.exec.FetchOperator
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.(FetchOperator.java:108)
> at
> org.apache.hadoop.hive.ql.exec.FetchTask.initialize(FetchTask.java:87)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:541)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
> {code}
> I know that hive-exec doesn't shade Guava until HIVE-22126 but that work
> targets 4.0.0. I'm wondering if there is a solution for current Hive
> versions, e.g. Hive 2.3.7? Any ideas?
> Thanks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23980) Shade guava from existing Hive versions
[
https://issues.apache.org/jira/browse/HIVE-23980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated HIVE-23980:
Fix Version/s: 2.3.8
> Shade guava from existing Hive versions
> ---
>
> Key: HIVE-23980
> URL: https://issues.apache.org/jira/browse/HIVE-23980
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.3.7
>Reporter: L. C. Hsieh
>Assignee: L. C. Hsieh
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.3.8
>
> Attachments: HIVE-23980.01.branch-2.3.patch
>
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
> I'm trying to upgrade Guava version in Spark. The JIRA ticket is SPARK-32502.
> Running test hits an error:
> {code}
> sbt.ForkMain$ForkError: sbt.ForkMain$ForkError: java.lang.IllegalAccessError:
> tried to access method
> com.google.common.collect.Iterators.emptyIterator()Lcom/google/common/collect/UnmodifiableIterator;
> from class org.apache.hadoop.hive.ql.exec.FetchOperator
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.(FetchOperator.java:108)
> at
> org.apache.hadoop.hive.ql.exec.FetchTask.initialize(FetchTask.java:87)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:541)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
> {code}
> I know that hive-exec doesn't shade Guava until HIVE-22126 but that work
> targets 4.0.0. I'm wondering if there is a solution for current Hive
> versions, e.g. Hive 2.3.7? Any ideas?
> Thanks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-24414) Backport HIVE-19662 to branch-3.1
[ https://issues.apache.org/jira/browse/HIVE-24414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-24414: Fix Version/s: 3.1.3 > Backport HIVE-19662 to branch-3.1 > - > > Key: HIVE-24414 > URL: https://issues.apache.org/jira/browse/HIVE-24414 > Project: Hive > Issue Type: Improvement >Reporter: Chao Sun >Assignee: Chao Sun >Priority: Major > Labels: pull-request-available > Fix For: 3.1.3 > > Time Spent: 50m > Remaining Estimate: 0h > > This JIRA proposes to backport HIVE-19662 to branch-3.1 and upgrade Avro to > 1.8.2. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-24414) Backport HIVE-19662 to branch-3.1
[ https://issues.apache.org/jira/browse/HIVE-24414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HIVE-24414. - Hadoop Flags: Reviewed Resolution: Fixed > Backport HIVE-19662 to branch-3.1 > - > > Key: HIVE-24414 > URL: https://issues.apache.org/jira/browse/HIVE-24414 > Project: Hive > Issue Type: Improvement >Reporter: Chao Sun >Assignee: Chao Sun >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > This JIRA proposes to backport HIVE-19662 to branch-3.1 and upgrade Avro to > 1.8.2. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23980) Shade guava from existing Hive versions
[
https://issues.apache.org/jira/browse/HIVE-23980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17238436#comment-17238436
]
Chao Sun commented on HIVE-23980:
-
Re-open this because there are some test failures related. The change was
reverted from branch-2 and branch-2.3. [~viirya] is working on a new version in
the PR #1356.
> Shade guava from existing Hive versions
> ---
>
> Key: HIVE-23980
> URL: https://issues.apache.org/jira/browse/HIVE-23980
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.3.7
>Reporter: L. C. Hsieh
>Assignee: L. C. Hsieh
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23980.01.branch-2.3.patch
>
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> I'm trying to upgrade Guava version in Spark. The JIRA ticket is SPARK-32502.
> Running test hits an error:
> {code}
> sbt.ForkMain$ForkError: sbt.ForkMain$ForkError: java.lang.IllegalAccessError:
> tried to access method
> com.google.common.collect.Iterators.emptyIterator()Lcom/google/common/collect/UnmodifiableIterator;
> from class org.apache.hadoop.hive.ql.exec.FetchOperator
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.(FetchOperator.java:108)
> at
> org.apache.hadoop.hive.ql.exec.FetchTask.initialize(FetchTask.java:87)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:541)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
> {code}
> I know that hive-exec doesn't shade Guava until HIVE-22126 but that work
> targets 4.0.0. I'm wondering if there is a solution for current Hive
> versions, e.g. Hive 2.3.7? Any ideas?
> Thanks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23980) Shade guava from existing Hive versions
[
https://issues.apache.org/jira/browse/HIVE-23980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated HIVE-23980:
Fix Version/s: (was: 2.4.0)
> Shade guava from existing Hive versions
> ---
>
> Key: HIVE-23980
> URL: https://issues.apache.org/jira/browse/HIVE-23980
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.3.7
>Reporter: L. C. Hsieh
>Assignee: L. C. Hsieh
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23980.01.branch-2.3.patch
>
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> I'm trying to upgrade Guava version in Spark. The JIRA ticket is SPARK-32502.
> Running test hits an error:
> {code}
> sbt.ForkMain$ForkError: sbt.ForkMain$ForkError: java.lang.IllegalAccessError:
> tried to access method
> com.google.common.collect.Iterators.emptyIterator()Lcom/google/common/collect/UnmodifiableIterator;
> from class org.apache.hadoop.hive.ql.exec.FetchOperator
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.(FetchOperator.java:108)
> at
> org.apache.hadoop.hive.ql.exec.FetchTask.initialize(FetchTask.java:87)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:541)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
> {code}
> I know that hive-exec doesn't shade Guava until HIVE-22126 but that work
> targets 4.0.0. I'm wondering if there is a solution for current Hive
> versions, e.g. Hive 2.3.7? Any ideas?
> Thanks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Reopened] (HIVE-23980) Shade guava from existing Hive versions
[
https://issues.apache.org/jira/browse/HIVE-23980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun reopened HIVE-23980:
-
> Shade guava from existing Hive versions
> ---
>
> Key: HIVE-23980
> URL: https://issues.apache.org/jira/browse/HIVE-23980
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.3.7
>Reporter: L. C. Hsieh
>Assignee: L. C. Hsieh
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.4.0
>
> Attachments: HIVE-23980.01.branch-2.3.patch
>
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> I'm trying to upgrade Guava version in Spark. The JIRA ticket is SPARK-32502.
> Running test hits an error:
> {code}
> sbt.ForkMain$ForkError: sbt.ForkMain$ForkError: java.lang.IllegalAccessError:
> tried to access method
> com.google.common.collect.Iterators.emptyIterator()Lcom/google/common/collect/UnmodifiableIterator;
> from class org.apache.hadoop.hive.ql.exec.FetchOperator
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.(FetchOperator.java:108)
> at
> org.apache.hadoop.hive.ql.exec.FetchTask.initialize(FetchTask.java:87)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:541)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
> {code}
> I know that hive-exec doesn't shade Guava until HIVE-22126 but that work
> targets 4.0.0. I'm wondering if there is a solution for current Hive
> versions, e.g. Hive 2.3.7? Any ideas?
> Thanks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-24324) Remove deprecated API usage from Avro
[
https://issues.apache.org/jira/browse/HIVE-24324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated HIVE-24324:
Fix Version/s: 2.3.8
> Remove deprecated API usage from Avro
> -
>
> Key: HIVE-24324
> URL: https://issues.apache.org/jira/browse/HIVE-24324
> Project: Hive
> Issue Type: Improvement
> Components: Avro
>Reporter: Chao Sun
>Assignee: Chao Sun
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.3.8, 4.0.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> {{JsonProperties#getJsonProp}} has been marked as deprecated in Avro 1.8 and
> removed since Avro 1.9. This replaces the API usage for this with
> {{getObjectProp}} which doesn't leak Json node from jackson. This will help
> downstream apps to depend on Hive while using higher version of Avro, and
> also help Hive to upgrade Avro version itself.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-24379) Backport HIVE-19662 to branch-2.3
[ https://issues.apache.org/jira/browse/HIVE-24379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-24379: Fix Version/s: 2.3.8 > Backport HIVE-19662 to branch-2.3 > - > > Key: HIVE-24379 > URL: https://issues.apache.org/jira/browse/HIVE-24379 > Project: Hive > Issue Type: Improvement >Reporter: Chao Sun >Assignee: Chao Sun >Priority: Major > Labels: pull-request-available > Fix For: 2.3.8 > > Time Spent: 40m > Remaining Estimate: 0h > > In order to backport HIVE-24324 to branch-2.3 to remove deprecated Avro API > so that downstream applications who depend on Hive 2.3.x can upgrade their > Avro version, we'll need to first backport HIVE-19662 and bump Avro version > in branch-2.3 first as it is currently on 1.7.x, while HIVE-24324 requires > Avro 1.8.x. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23323) Add qsplits profile
[ https://issues.apache.org/jira/browse/HIVE-23323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-23323: Fix Version/s: 2.3.8 > Add qsplits profile > --- > > Key: HIVE-23323 > URL: https://issues.apache.org/jira/browse/HIVE-23323 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Labels: pull-request-available > Fix For: 2.3.8, 4.0.0 > > Attachments: HIVE-23323.01.patch, HIVE-23323.02.patch, > HIVE-23323.02.patch, HIVE-23323.02.patch, HIVE-23323.03.patch > > Time Spent: 50m > Remaining Estimate: 0h > > provide an in-maven way to split-up big testclasses into smaller ones -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-24035) Add Jenkinsfile for branch-2.3
[ https://issues.apache.org/jira/browse/HIVE-24035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HIVE-24035. - Fix Version/s: 2.3.8 Hadoop Flags: Reviewed Resolution: Fixed > Add Jenkinsfile for branch-2.3 > -- > > Key: HIVE-24035 > URL: https://issues.apache.org/jira/browse/HIVE-24035 > Project: Hive > Issue Type: Test >Reporter: Chao Sun >Assignee: Chao Sun >Priority: Major > Labels: pull-request-available > Fix For: 2.3.8 > > Time Spent: 5h 50m > Remaining Estimate: 0h > > To enable precommit tests for github PR, we need to have a Jenkinsfile in the > repo. This is already done for master and branch-2. This adds the same for > branch-2.3 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24051) Hive lineage information exposed in ExecuteWithHookContext
[ https://issues.apache.org/jira/browse/HIVE-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-24051: Fix Version/s: 4.0.0 Hadoop Flags: Reviewed Resolution: Fixed Status: Resolved (was: Patch Available) > Hive lineage information exposed in ExecuteWithHookContext > -- > > Key: HIVE-24051 > URL: https://issues.apache.org/jira/browse/HIVE-24051 > Project: Hive > Issue Type: Improvement > Components: Configuration >Reporter: Szehon Ho >Assignee: Szehon Ho >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-24051.patch > > Time Spent: 1h 40m > Remaining Estimate: 0h > > The lineage information is not populated unless certain hooks are enabled. > However, this is a bit fragile, and no way for another hook that we write to > get this information. This proposes a flag to enable this instead. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HIVE-24324) Remove deprecated API usage from Avro
[
https://issues.apache.org/jira/browse/HIVE-24324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17238282#comment-17238282
]
Chao Sun edited comment on HIVE-24324 at 11/24/20, 5:37 PM:
[~pgaref], [~aihuaxu] in order to backport this to 3.1 we'll first need to
backport HIVE-19662 to 3.1. I've opened [this
PR|https://github.com/apache/hive/pull/1698]. Could you review it? Thanks.
was (Author: csun):
[~pgaref] in order to backport this to 3.1 we'll first need to backport
HIVE-19662 to 3.1. I've opened [this
PR|https://github.com/apache/hive/pull/1698]. Could you review it? Thanks.
> Remove deprecated API usage from Avro
> -
>
> Key: HIVE-24324
> URL: https://issues.apache.org/jira/browse/HIVE-24324
> Project: Hive
> Issue Type: Improvement
> Components: Avro
>Reporter: Chao Sun
>Assignee: Chao Sun
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> {{JsonProperties#getJsonProp}} has been marked as deprecated in Avro 1.8 and
> removed since Avro 1.9. This replaces the API usage for this with
> {{getObjectProp}} which doesn't leak Json node from jackson. This will help
> downstream apps to depend on Hive while using higher version of Avro, and
> also help Hive to upgrade Avro version itself.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-24324) Remove deprecated API usage from Avro
[
https://issues.apache.org/jira/browse/HIVE-24324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17238282#comment-17238282
]
Chao Sun commented on HIVE-24324:
-
[~pgaref] in order to backport this to 3.1 we'll first need to backport
HIVE-19662 to 3.1. I've opened [this
PR|https://github.com/apache/hive/pull/1698]. Could you review it? Thanks.
> Remove deprecated API usage from Avro
> -
>
> Key: HIVE-24324
> URL: https://issues.apache.org/jira/browse/HIVE-24324
> Project: Hive
> Issue Type: Improvement
> Components: Avro
>Reporter: Chao Sun
>Assignee: Chao Sun
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> {{JsonProperties#getJsonProp}} has been marked as deprecated in Avro 1.8 and
> removed since Avro 1.9. This replaces the API usage for this with
> {{getObjectProp}} which doesn't leak Json node from jackson. This will help
> downstream apps to depend on Hive while using higher version of Avro, and
> also help Hive to upgrade Avro version itself.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-24414) Backport HIVE-19662 to branch-3.1
[ https://issues.apache.org/jira/browse/HIVE-24414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned HIVE-24414: --- > Backport HIVE-19662 to branch-3.1 > - > > Key: HIVE-24414 > URL: https://issues.apache.org/jira/browse/HIVE-24414 > Project: Hive > Issue Type: Improvement >Reporter: Chao Sun >Assignee: Chao Sun >Priority: Major > > This JIRA proposes to backport HIVE-19662 to branch-3.1 and upgrade Avro to > 1.8.2. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-24379) Backport HIVE-19662 to branch-2.3
[ https://issues.apache.org/jira/browse/HIVE-24379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HIVE-24379. - Resolution: Fixed > Backport HIVE-19662 to branch-2.3 > - > > Key: HIVE-24379 > URL: https://issues.apache.org/jira/browse/HIVE-24379 > Project: Hive > Issue Type: Improvement >Reporter: Chao Sun >Assignee: Chao Sun >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > In order to backport HIVE-24324 to branch-2.3 to remove deprecated Avro API > so that downstream applications who depend on Hive 2.3.x can upgrade their > Avro version, we'll need to first backport HIVE-19662 and bump Avro version > in branch-2.3 first as it is currently on 1.7.x, while HIVE-24324 requires > Avro 1.8.x. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24324) Remove deprecated API usage from Avro
[
https://issues.apache.org/jira/browse/HIVE-24324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17236837#comment-17236837
]
Chao Sun commented on HIVE-24324:
-
Yes good point [~pgaref]. I'll create another PR to backport this to 3.1.
> Remove deprecated API usage from Avro
> -
>
> Key: HIVE-24324
> URL: https://issues.apache.org/jira/browse/HIVE-24324
> Project: Hive
> Issue Type: Improvement
> Components: Avro
>Reporter: Chao Sun
>Assignee: Chao Sun
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> {{JsonProperties#getJsonProp}} has been marked as deprecated in Avro 1.8 and
> removed since Avro 1.9. This replaces the API usage for this with
> {{getObjectProp}} which doesn't leak Json node from jackson. This will help
> downstream apps to depend on Hive while using higher version of Avro, and
> also help Hive to upgrade Avro version itself.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
