[jira] [Commented] (HIVE-12812) Enable mapred.input.dir.recursive by default to support union with aggregate function
[ https://issues.apache.org/jira/browse/HIVE-12812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514191#comment-16514191 ] Wang Haihua commented on HIVE-12812: [~kgyrtkirk] Thanks for your reply. For compability, we cannot idnetified if we set this parameter in hive-site.xml, which influence of tez execution is, on the condition of tez also use FileInputformat for retriving data. So i saw this would be a good solution which seperated and protected tez engine in our production. For hive version, we are glad to update to 3.0, but for upgrading hive 2.1.1 which i have done before just be a long period work.. you can see this work in my blog(Sorry there has only chinese version) [https://ericsahit.github.io/2017/09/10/Hive%E5%8D%87%E7%BA%A7%E5%85%A8%E5%A7%BF%E5%8A%BF/] > Enable mapred.input.dir.recursive by default to support union with aggregate > function > - > > Key: HIVE-12812 > URL: https://issues.apache.org/jira/browse/HIVE-12812 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 1.2.1, 2.1.0 >Reporter: Chaoyu Tang >Priority: Major > Attachments: HIVE-12812.patch, HIVE-12812.patch, HIVE-12812.patch > > > When union remove optimization is enabled, union query with aggregate > function writes its subquery intermediate results to subdirs which needs > mapred.input.dir.recursive to be enabled in order to be fetched. This > property is not defined by default in Hive and often ignored by user, which > causes the query failure and is hard to be debugged. > So we need set mapred.input.dir.recursive to true whenever union remove > optimization is enabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19912) Schema evolution checks prints a log line in INFO mode for each vectorized rowbatch, impacts performance
[
https://issues.apache.org/jira/browse/HIVE-19912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514213#comment-16514213
]
Prasanth Jayachandran commented on HIVE-19912:
--
Since this is logging only change, I don't think this needs a precommit run. I
will address the above comment and commit it.
> Schema evolution checks prints a log line in INFO mode for each vectorized
> rowbatch, impacts performance
>
>
> Key: HIVE-19912
> URL: https://issues.apache.org/jira/browse/HIVE-19912
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Nita Dembla
>Assignee: Nita Dembla
>Priority: Major
> Fix For: 3.0.1
>
> Attachments: HIVE-19912.1.patch
>
>
> While benchmarking query96, noticed 17K log lines printed for each vector
> rowbactch
>
> In file ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java
>
> {code:java}
> @@ -2554,8 +2554,8 @@ public static TypeDescription
> getDesiredRowTypeDescr(Configuration conf,
> }
> if (haveSchemaEvolutionProperties) {
> - if (LOG.isInfoEnabled()) {
> - LOG.info("Using schema evolution configuration variables
> schema.evolution.columns " +
> + if (LOG.isDebugEnabled()) {
> + LOG.debug("Using schema evolution configuration variables
> schema.evolution.columns " +
> schemaEvolutionColumnNames.toString() +
> " / schema.evolution.columns.types " +
> schemaEvolutionTypeDescrs.toString() +{code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19904) Load data rewrite into Tez job fails for ACID
[ https://issues.apache.org/jira/browse/HIVE-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514218#comment-16514218 ] Eugene Koifman commented on HIVE-19904: --- Table names are normalized when created by the metastore. Why is this not happening here? > Load data rewrite into Tez job fails for ACID > - > > Key: HIVE-19904 > URL: https://issues.apache.org/jira/browse/HIVE-19904 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-19904.1.patch > > > Load data rewrite into IAS fails for ACID as there is some code which does > not take into account the table name could be in upper case, specifically > ValidTxnWriteIdList -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19912) Schema evolution checks prints a log line in INFO mode for each vectorized rowbatch, impacts performance
[
https://issues.apache.org/jira/browse/HIVE-19912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran reassigned HIVE-19912:
Assignee: Prasanth Jayachandran (was: Nita Dembla)
> Schema evolution checks prints a log line in INFO mode for each vectorized
> rowbatch, impacts performance
>
>
> Key: HIVE-19912
> URL: https://issues.apache.org/jira/browse/HIVE-19912
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Nita Dembla
>Assignee: Prasanth Jayachandran
>Priority: Major
> Fix For: 3.1.0, 3.0.1, 4.0.0
>
> Attachments: HIVE-19912.1.patch, HIVE-19912.2.patch
>
>
> While benchmarking query96, noticed 17K log lines printed for each vector
> rowbactch
>
> In file ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java
>
> {code:java}
> @@ -2554,8 +2554,8 @@ public static TypeDescription
> getDesiredRowTypeDescr(Configuration conf,
> }
> if (haveSchemaEvolutionProperties) {
> - if (LOG.isInfoEnabled()) {
> - LOG.info("Using schema evolution configuration variables
> schema.evolution.columns " +
> + if (LOG.isDebugEnabled()) {
> + LOG.debug("Using schema evolution configuration variables
> schema.evolution.columns " +
> schemaEvolutionColumnNames.toString() +
> " / schema.evolution.columns.types " +
> schemaEvolutionTypeDescrs.toString() +{code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19892) Disable query results cache for for HiveServer2 doAs=true
[ https://issues.apache.org/jira/browse/HIVE-19892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514235#comment-16514235 ] Sergey Shelukhin commented on HIVE-19892: - In my original case, the get didn't fail I assume (it just didn't find the results in cache) but put failed... So I don't quite understand what you mean by "disabled due to this fix" - if get doesn't fail then it won't be disabled, right? However if I'm missing smth and it won't happen then there's no need to fix it. > Disable query results cache for for HiveServer2 doAs=true > - > > Key: HIVE-19892 > URL: https://issues.apache.org/jira/browse/HIVE-19892 > Project: Hive > Issue Type: Sub-task >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19892.1.patch, HIVE-19892.2.patch > > > If running HS2 with doAs=true, the temp query results directory will have > ownership/permissions based on the doAs user. A subsequent query running as a > different user may not be able to access this query results directory. > Results caching will have to be disabled in this case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19898) Disable TransactionalValidationListener when the table is not in the Hive catalog
[ https://issues.apache.org/jira/browse/HIVE-19898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514254#comment-16514254 ] Dongjoon Hyun commented on HIVE-19898: -- Ping, [~jdere] and [~ekoifman] . > Disable TransactionalValidationListener when the table is not in the Hive > catalog > - > > Key: HIVE-19898 > URL: https://issues.apache.org/jira/browse/HIVE-19898 > Project: Hive > Issue Type: Bug > Components: Metastore, Standalone Metastore, Transactions >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > Attachments: HIVE-19898.1.patch > > > The TransactionalValidationListener does validation of tables specified as > transactional tables, as well as enforcing create.as.acid. While this can be > useful to Hive, this may not be useful to other catalogs which do not support > transactional tables, and would not benefit from being automatically tagged > as a transactional table. This should be changed so the > TransactionalValidationListener does not run for non-hive catalogs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19889) Wrong results due to PPD of non deterministic functions with CBO
[
https://issues.apache.org/jira/browse/HIVE-19889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514258#comment-16514258
]
Hive QA commented on HIVE-19889:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
0s{color} | {color:blue} ql in master has 2276 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11807/dev-support/hive-personality.sh
|
| git revision | master / 5a9a328 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11807/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Wrong results due to PPD of non deterministic functions with CBO
>
>
> Key: HIVE-19889
> URL: https://issues.apache.org/jira/browse/HIVE-19889
> Project: Hive
> Issue Type: Bug
>Reporter: Janaki Lahorani
>Assignee: Janaki Lahorani
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-19889.1.patch
>
>
> The following query can give wrong results when CBO is on:
> {code}
> select * from (
> select part1,randum123
> from (SELECT *, cast(rand() as double) AS randum123 FROM testA where
> part1='CA' and part2 = 'ABC') a
> where randum123 <= 0.5) s where s.randum123 > 0.25 limit 20;
> The plan of the query is as follows:
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Map Operator Tree:
> TableScan
> alias: testa
> Statistics: Num rows: 2 Data size: 4580 Basic stats: COMPLETE
> Column stats: NONE
> Filter Operator
> predicate: ((rand() <= 0.5D) and (rand() > 0.25D)) (type:
> boolean)
> Statistics: Num rows: 1 Data size: 2290 Basic stats: COMPLETE
> Column stats: NONE
> Select Operator
> expressions: 'CA' (type: string), rand() (type: double)
> outputColumnNames: _col0, _col1
> Statistics: Num rows: 1 Data size: 2290 Basic stats: COMPLETE
> Column stats: NONE
> Limit
> Number of rows: 20
> Statistics: Num rows: 1 Data size: 2290 Basic stats:
> COMPLETE Column stats: NONE
> File Output Operator
>
[jira] [Updated] (HIVE-19904) Load data rewrite into Tez job fails for ACID
[ https://issues.apache.org/jira/browse/HIVE-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-19904: -- Component/s: Transactions > Load data rewrite into Tez job fails for ACID > - > > Key: HIVE-19904 > URL: https://issues.apache.org/jira/browse/HIVE-19904 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-19904.1.patch > > > Load data rewrite into IAS fails for ACID as there is some code which does > not take into account the table name could be in upper case, specifically > ValidTxnWriteIdList -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18725) Improve error handling for subqueries if there is wrong column reference
[
https://issues.apache.org/jira/browse/HIVE-18725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Kryvenko updated HIVE-18725:
-
Attachment: HIVE-18725.06.patch
> Improve error handling for subqueries if there is wrong column reference
>
>
> Key: HIVE-18725
> URL: https://issues.apache.org/jira/browse/HIVE-18725
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-18725.01.patch, HIVE-18725.02.patch,
> HIVE-18725.03.patch, HIVE-18725.04.patch, HIVE-18725.05.patch,
> HIVE-18725.06.patch
>
>
> If there is a column reference within subquery which doesn't exist Hive
> throws misleading error message.
> e.g.
> {code:sql}
> select * from table1 where table1.col1 IN (select col2 from table2 where
> table2.col1=table1.non_existing_column) and table1.col1 IN (select 4);
> {code}
> The above query, assuming table1 doesn't have non_existing_column, will throw
> following misleading error:
> {noformat}
> FAILED: SemanticException Line 0:-1 Unsupported SubQuery Expression 'col1':
> Only 1 SubQuery expression is supported.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19884) Invalidation cache may throw NPE when there is no data in table used by materialized view
[ https://issues.apache.org/jira/browse/HIVE-19884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514291#comment-16514291 ] Ashutosh Chauhan commented on HIVE-19884: - +1 > Invalidation cache may throw NPE when there is no data in table used by > materialized view > - > > Key: HIVE-19884 > URL: https://issues.apache.org/jira/browse/HIVE-19884 > Project: Hive > Issue Type: Bug > Components: Materialized views >Affects Versions: 3.1.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-19884.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19892) Disable query results cache for for HiveServer2 doAs=true
[ https://issues.apache.org/jira/browse/HIVE-19892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514373#comment-16514373 ] Jason Dere commented on HIVE-19892: --- The stack trace above was from the logs I recovered from your case .. the put didn't actually fail because there were no results to actually put into the cache, so the put basically succeeded as a no-op. The fetch of results (now "moved" to the results cache) failed due to permissions issues on the results cache directory when it tried to get the non-existent result path. Since the results cache (including saving to the cache) is disabled in this patch, this situation should not occur. > Disable query results cache for for HiveServer2 doAs=true > - > > Key: HIVE-19892 > URL: https://issues.apache.org/jira/browse/HIVE-19892 > Project: Hive > Issue Type: Sub-task >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19892.1.patch, HIVE-19892.2.patch > > > If running HS2 with doAs=true, the temp query results directory will have > ownership/permissions based on the doAs user. A subsequent query running as a > different user may not be able to access this query results directory. > Results caching will have to be disabled in this case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19837) Setting to have different default location for external tables
[
https://issues.apache.org/jira/browse/HIVE-19837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514377#comment-16514377
]
Hive QA commented on HIVE-19837:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
40s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
35s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
52s{color} | {color:blue} ql in master has 2276 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
58s{color} | {color:blue} standalone-metastore in master has 227 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
26s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
34s{color} | {color:red} hive-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
48s{color} | {color:red} ql in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
32s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
15s{color} | {color:red} itests/hive-unit: The patch generated 9 new + 10
unchanged - 3 fixed = 19 total (was 13) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
45s{color} | {color:red} ql: The patch generated 5 new + 875 unchanged - 6
fixed = 880 total (was 881) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
5s{color} | {color:red} ql generated 1 new + 2276 unchanged - 0 fixed = 2277
total (was 2276) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
54s{color} | {color:red} standalone-metastore generated 1 new + 53 unchanged -
0 fixed = 54 total (was 53) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 48s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Dead store to tableName in
org.apache.hadoop.hive.ql.util.HiveStrictManagedMigration.shouldModifyPartitionLocation(Database,
Table, Partition, Map) At
HiveStrictManagedMigration.java:org.apache.hadoop.hive.ql.util.HiveStrictManagedMigration.shouldModifyPartitionLocation(Database,
Table, Partition, Map) At HiveStrictManagedMigration.java:[line 471] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11809/dev-support/hive-personality.sh
|
| git revision | master / ccfca8e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11809/yetus/patch-mvninstall-itests_hive-unit.txt
|
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11809/yetus/patch-mvninstall-ql.txt
|
|
[jira] [Commented] (HIVE-19892) Disable query results cache for for HiveServer2 doAs=true
[ https://issues.apache.org/jira/browse/HIVE-19892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514386#comment-16514386 ] Sergey Shelukhin commented on HIVE-19892: - Thnx for the details! > Disable query results cache for for HiveServer2 doAs=true > - > > Key: HIVE-19892 > URL: https://issues.apache.org/jira/browse/HIVE-19892 > Project: Hive > Issue Type: Sub-task >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19892.1.patch, HIVE-19892.2.patch > > > If running HS2 with doAs=true, the temp query results directory will have > ownership/permissions based on the doAs user. A subsequent query running as a > different user may not be able to access this query results directory. > Results caching will have to be disabled in this case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19892) Disable query results cache for for HiveServer2 doAs=true
[ https://issues.apache.org/jira/browse/HIVE-19892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514385#comment-16514385 ] Sergey Shelukhin commented on HIVE-19892: - +1 > Disable query results cache for for HiveServer2 doAs=true > - > > Key: HIVE-19892 > URL: https://issues.apache.org/jira/browse/HIVE-19892 > Project: Hive > Issue Type: Sub-task >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19892.1.patch, HIVE-19892.2.patch > > > If running HS2 with doAs=true, the temp query results directory will have > ownership/permissions based on the doAs user. A subsequent query running as a > different user may not be able to access this query results directory. > Results caching will have to be disabled in this case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19861) Fix temp table path generation for acid table export
[ https://issues.apache.org/jira/browse/HIVE-19861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-19861: -- Resolution: Fixed Fix Version/s: 4.0.0 3.1.0 Status: Resolved (was: Patch Available) Committed to master/branch-3 > Fix temp table path generation for acid table export > > > Key: HIVE-19861 > URL: https://issues.apache.org/jira/browse/HIVE-19861 > Project: Hive > Issue Type: Bug > Components: Import/Export, Transactions >Affects Versions: 3.0.0 >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-19861.1.patch, HIVE-19861.2.patch > > > Temp tables that are analyzed by the SemanticAnalyzer get their default > location set to a location in the session directory. Export of Acid tables > also creates temp tables, but this is done via a plan transformation, and the > temp table creation never goes through the SemanticAnalyzer, meaning the > location is not set. There is some other logic in DDLTask (which I am > changing in HIV-19837) which ends up automatically setting this path to the > default table location in the warehouse directory. This should be fixed so > that the path defaults to a location in the session directory, like with > normal temp tables. > cc [~ekoifman] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19904) Load data rewrite into Tez job fails for ACID
[ https://issues.apache.org/jira/browse/HIVE-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Jaiswal updated HIVE-19904: -- Attachment: HIVE-19904.2.patch > Load data rewrite into Tez job fails for ACID > - > > Key: HIVE-19904 > URL: https://issues.apache.org/jira/browse/HIVE-19904 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-19904.1.patch, HIVE-19904.2.patch > > > Load data rewrite into IAS fails for ACID as there is some code which does > not take into account the table name could be in upper case, specifically > ValidTxnWriteIdList -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19853) Arrow serializer needs to create a TimeStampMicroTZVector instead of TimeStampMicroVector
[ https://issues.apache.org/jira/browse/HIVE-19853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514396#comment-16514396 ] Eric Wohlstadter commented on HIVE-19853: - lgtm [~mmccline], can you merge to master and branch-3? Thanks! > Arrow serializer needs to create a TimeStampMicroTZVector instead of > TimeStampMicroVector > - > > Key: HIVE-19853 > URL: https://issues.apache.org/jira/browse/HIVE-19853 > Project: Hive > Issue Type: Bug >Reporter: Teddy Choi >Assignee: Teddy Choi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-19853.1.patch, HIVE-19853.2.patch > > > HIVE-19723 changed nanosecond to microsecond in Arrow serialization. However, > it needs to be microsecond with time zone. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19837) Setting to have different default location for external tables
[
https://issues.apache.org/jira/browse/HIVE-19837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514398#comment-16514398
]
Hive QA commented on HIVE-19837:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927760/HIVE-19837.3.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14536 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11809/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11809/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11809/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927760 - PreCommit-HIVE-Build
> Setting to have different default location for external tables
> --
>
> Key: HIVE-19837
> URL: https://issues.apache.org/jira/browse/HIVE-19837
> Project: Hive
> Issue Type: Bug
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-19837.1.patch, HIVE-19837.3.patch
>
>
> Allow external tables to have a different default location than managed tables
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19786) RpcServer cancelTask log message is incorrect
[
https://issues.apache.org/jira/browse/HIVE-19786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bharathkrishna Guruvayoor Murali updated HIVE-19786:
Attachment: HIVE-19786.2.patch
> RpcServer cancelTask log message is incorrect
> -
>
> Key: HIVE-19786
> URL: https://issues.apache.org/jira/browse/HIVE-19786
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Bharathkrishna Guruvayoor Murali
>Priority: Major
> Attachments: HIVE-19786.1.patch, HIVE-19786.2.patch
>
>
> The log message inside the {{cancelTask}} of the {{RpcServer}}
> {{ChannelInitializer}} is incorrect. It states its measuring the timeout for
> the "test" message to be sent (basically a "hello" message to test the
> connection works). However, the {{cancelTask}} is actually used to timeout
> the SASL negotiation between the client and the server.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18545) Add UDF to parse complex types from json
[ https://issues.apache.org/jira/browse/HIVE-18545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-18545: - Affects Version/s: 4.0.0 > Add UDF to parse complex types from json > > > Key: HIVE-18545 > URL: https://issues.apache.org/jira/browse/HIVE-18545 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-18545.02.patch, HIVE-18545.03.patch, > HIVE-18545.04.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19915) master-txnstats branch - support CachedStore
[ https://issues.apache.org/jira/browse/HIVE-19915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-19915: --- > master-txnstats branch - support CachedStore > > > Key: HIVE-19915 > URL: https://issues.apache.org/jira/browse/HIVE-19915 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Steve Yeom >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19530) Vectorization: Fix JDBCSerde and re-enable vectorization
[ https://issues.apache.org/jira/browse/HIVE-19530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-19530: Attachment: HIVE-19530.01.patch > Vectorization: Fix JDBCSerde and re-enable vectorization > > > Key: HIVE-19530 > URL: https://issues.apache.org/jira/browse/HIVE-19530 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Attachments: HIVE-19530.01.patch > > > According to [~jcamachorodriguez] there is a big switch statement in the code > that has might have missing types. This can lead to the string types seen. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19530) Vectorization: Fix JDBCSerde and re-enable vectorization
[ https://issues.apache.org/jira/browse/HIVE-19530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-19530: Status: Patch Available (was: Open) > Vectorization: Fix JDBCSerde and re-enable vectorization > > > Key: HIVE-19530 > URL: https://issues.apache.org/jira/browse/HIVE-19530 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > Attachments: HIVE-19530.01.patch > > > According to [~jcamachorodriguez] there is a big switch statement in the code > that has might have missing types. This can lead to the string types seen. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19916) master-txnstats branch - integrate with HIVE-19382
[ https://issues.apache.org/jira/browse/HIVE-19916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-19916: --- > master-txnstats branch - integrate with HIVE-19382 > -- > > Key: HIVE-19916 > URL: https://issues.apache.org/jira/browse/HIVE-19916 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Steve Yeom >Priority: Major > > There's some code in original txn stats patch that may go to metastore to get > write Ids. This code should not go to metastore, it should fail instead. > HIVE-19382 should ensure that we have correct IDs already present during > optimizer - they are using by e.g. materialized view optimizer, so they > should be there; if they are not present, some integration might be needed so > that txn stats optimizations also have access to those write Ids. > cc [~jcamachorodriguez] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19532) fix tests for master-txnstats branch
[ https://issues.apache.org/jira/browse/HIVE-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514317#comment-16514317 ] Sergey Shelukhin commented on HIVE-19532: - [~steveyeom2017] Can you update RB with 03 patch? Just to finish the review and perhaps file some more follow up jiras. Also can you make a list of things that are missing? There are some JIRAs remaining as subtasks, I'm not sure if they are addressed as part of the epic patch or not. I filed some known issue JIRAs, let me know if there's more. Or is it just enough to fix the tests and it's good? > fix tests for master-txnstats branch > > > Key: HIVE-19532 > URL: https://issues.apache.org/jira/browse/HIVE-19532 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19532.01.patch, HIVE-19532.01.prepatch, > HIVE-19532.02.patch, HIVE-19532.02.prepatch, HIVE-19532.03.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19867) Test and verify Concurrent INSERTS
[ https://issues.apache.org/jira/browse/HIVE-19867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514328#comment-16514328 ] Steve Yeom commented on HIVE-19867: --- The current code change of patch 03 for 19532 is supposed to work on this case. But we need to test. > Test and verify Concurrent INSERTS > > > Key: HIVE-19867 > URL: https://issues.apache.org/jira/browse/HIVE-19867 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 4.0.0 >Reporter: Steve Yeom >Priority: Major > Fix For: 4.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19867) Test and verify Concurrent INSERTS
[ https://issues.apache.org/jira/browse/HIVE-19867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Yeom reassigned HIVE-19867: - Assignee: Steve Yeom > Test and verify Concurrent INSERTS > > > Key: HIVE-19867 > URL: https://issues.apache.org/jira/browse/HIVE-19867 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 4.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 4.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HIVE-19533) Modify Hive to support transactional-stats-using aggregation queries with all other than COUNT
[ https://issues.apache.org/jira/browse/HIVE-19533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Yeom resolved HIVE-19533. --- Resolution: Duplicate HIVE-19532 patch has the fix for this jira. > Modify Hive to support transactional-stats-using aggregation queries with all > other than COUNT > -- > > Key: HIVE-19533 > URL: https://issues.apache.org/jira/browse/HIVE-19533 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 3.1.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19861) Fix temp table path generation for acid table export
[
https://issues.apache.org/jira/browse/HIVE-19861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514342#comment-16514342
]
Hive QA commented on HIVE-19861:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927748/HIVE-19861.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11808/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11808/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11808/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12927748/HIVE-19861.2.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927748 - PreCommit-HIVE-Build
> Fix temp table path generation for acid table export
>
>
> Key: HIVE-19861
> URL: https://issues.apache.org/jira/browse/HIVE-19861
> Project: Hive
> Issue Type: Bug
> Components: Import/Export, Transactions
>Affects Versions: 3.0.0
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-19861.1.patch, HIVE-19861.2.patch
>
>
> Temp tables that are analyzed by the SemanticAnalyzer get their default
> location set to a location in the session directory. Export of Acid tables
> also creates temp tables, but this is done via a plan transformation, and the
> temp table creation never goes through the SemanticAnalyzer, meaning the
> location is not set. There is some other logic in DDLTask (which I am
> changing in HIV-19837) which ends up automatically setting this path to the
> default table location in the warehouse directory. This should be fixed so
> that the path defaults to a location in the session directory, like with
> normal temp tables.
> cc [~ekoifman]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19889) Wrong results due to PPD of non deterministic functions with CBO
[
https://issues.apache.org/jira/browse/HIVE-19889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514340#comment-16514340
]
Hive QA commented on HIVE-19889:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927746/HIVE-19889.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 14533 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_udf_col] (batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_offcbo]
(batchId=47)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11807/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11807/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11807/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927746 - PreCommit-HIVE-Build
> Wrong results due to PPD of non deterministic functions with CBO
>
>
> Key: HIVE-19889
> URL: https://issues.apache.org/jira/browse/HIVE-19889
> Project: Hive
> Issue Type: Bug
>Reporter: Janaki Lahorani
>Assignee: Janaki Lahorani
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-19889.1.patch
>
>
> The following query can give wrong results when CBO is on:
> {code}
> select * from (
> select part1,randum123
> from (SELECT *, cast(rand() as double) AS randum123 FROM testA where
> part1='CA' and part2 = 'ABC') a
> where randum123 <= 0.5) s where s.randum123 > 0.25 limit 20;
> The plan of the query is as follows:
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Map Operator Tree:
> TableScan
> alias: testa
> Statistics: Num rows: 2 Data size: 4580 Basic stats: COMPLETE
> Column stats: NONE
> Filter Operator
> predicate: ((rand() <= 0.5D) and (rand() > 0.25D)) (type:
> boolean)
> Statistics: Num rows: 1 Data size: 2290 Basic stats: COMPLETE
> Column stats: NONE
> Select Operator
> expressions: 'CA' (type: string), rand() (type: double)
> outputColumnNames: _col0, _col1
> Statistics: Num rows: 1 Data size: 2290 Basic stats: COMPLETE
> Column stats: NONE
> Limit
> Number of rows: 20
> Statistics: Num rows: 1 Data size: 2290 Basic stats:
> COMPLETE Column stats: NONE
> File Output Operator
> compressed: false
> Statistics: Num rows: 1 Data size: 2290 Basic stats:
> COMPLETE Column stats: NONE
> table:
> input format:
> org.apache.hadoop.mapred.SequenceFileInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
> serde:
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> Stage: Stage-0
> Fetch Operator
> limit: 20
> Processor Tree:
> ListSink
> {code}
> The relevant part in the plan is the filter:
> {code}
> Filter Operator
> predicate: ((rand() <= 0.5D) and (rand() > 0.25D)) (type:
> boolean)
> {code}
> The predicates s.randum123 > 0.25 and s.randum123 > 0.25 were pushed down.
> And randum123 was resolved to rand(). This is bad because it will result in
> invocation of rand() two times and rand() UDF is non-deterministic. Both the
> rand calls can generate values that can satisfy the predicates independently,
> but not together, whereas the original intention of the query is to give
> results when rand falls between 0.25 and 0.5.
> A sample result:
> {code}
> CA0.9191984370369802
> CA0.397933021566812
> {code}
> where the condition was not satisfied.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19875) increase LLAP IO queue size for perf
[ https://issues.apache.org/jira/browse/HIVE-19875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-19875: - Resolution: Fixed Fix Version/s: 4.0.0 3.0.1 3.1.0 Status: Resolved (was: Patch Available) Committed to branch-3, branch-3.0 and master. Thanks for the review! > increase LLAP IO queue size for perf > > > Key: HIVE-19875 > URL: https://issues.apache.org/jira/browse/HIVE-19875 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0, 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Fix For: 3.1.0, 3.0.1, 4.0.0 > > Attachments: HIVE-19875.1.patch, HIVE-19875.2.patch > > > According to [~gopalv] queue limit has perf impact, esp. during hashtable > load for mapjoin where in the past IO used to queue up more data for > processing to process. > 1) Overall the default limit could be adjusted higher. > 2) Depending on Decimal64 availability, the weight for decimal columns could > be reduced. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19898) Disable TransactionalValidationListener when the table is not in the Hive catalog
[ https://issues.apache.org/jira/browse/HIVE-19898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514361#comment-16514361 ] Jason Dere commented on HIVE-19898: --- Still waiting for the precommit tests to run for this one. > Disable TransactionalValidationListener when the table is not in the Hive > catalog > - > > Key: HIVE-19898 > URL: https://issues.apache.org/jira/browse/HIVE-19898 > Project: Hive > Issue Type: Bug > Components: Metastore, Standalone Metastore, Transactions >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > Attachments: HIVE-19898.1.patch > > > The TransactionalValidationListener does validation of tables specified as > transactional tables, as well as enforcing create.as.acid. While this can be > useful to Hive, this may not be useful to other catalogs which do not support > transactional tables, and would not benefit from being automatically tagged > as a transactional table. This should be changed so the > TransactionalValidationListener does not run for non-hive catalogs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19904) Load data rewrite into Tez job fails for ACID
[ https://issues.apache.org/jira/browse/HIVE-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514362#comment-16514362 ] Deepak Jaiswal commented on HIVE-19904: --- So you are suggesting I use all lower case for tempTableObj? Since this code path does not go thru normal path which normalizes the names, I think either I use lower case name or patch every place. I will take the earlier one as there might be new code paths in future which might break this. > Load data rewrite into Tez job fails for ACID > - > > Key: HIVE-19904 > URL: https://issues.apache.org/jira/browse/HIVE-19904 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-19904.1.patch > > > Load data rewrite into IAS fails for ACID as there is some code which does > not take into account the table name could be in upper case, specifically > ValidTxnWriteIdList -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19884) Invalidation cache may throw NPE when there is no data in table used by materialized view
[ https://issues.apache.org/jira/browse/HIVE-19884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-19884: --- Attachment: HIVE-19884.addendum.patch > Invalidation cache may throw NPE when there is no data in table used by > materialized view > - > > Key: HIVE-19884 > URL: https://issues.apache.org/jira/browse/HIVE-19884 > Project: Hive > Issue Type: Bug > Components: Materialized views >Affects Versions: 3.1.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19884.addendum.patch, HIVE-19884.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19532) Modify Hive Driver/Executor to support transactional-stats-using COUNT aggregation queries
[ https://issues.apache.org/jira/browse/HIVE-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514226#comment-16514226 ] Sergey Shelukhin commented on HIVE-19532: - Looks like it needs to be rebased again. [~steveyeom2017] do you want to clean up or expand TODOs (to be understandable by others while looking at code) and commit to branch off master? > Modify Hive Driver/Executor to support transactional-stats-using COUNT > aggregation queries > --- > > Key: HIVE-19532 > URL: https://issues.apache.org/jira/browse/HIVE-19532 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19532.01.patch, HIVE-19532.01.prepatch, > HIVE-19532.02.patch, HIVE-19532.02.prepatch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19904) Load data rewrite into Tez job fails for ACID
[ https://issues.apache.org/jira/browse/HIVE-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514228#comment-16514228 ] Deepak Jaiswal commented on HIVE-19904: --- In this case, the temp table object is created for rewrite purpose. It is in-memory only. Please see LoadSemanticAnalyzer.reparseAndAnalyze() for details. > Load data rewrite into Tez job fails for ACID > - > > Key: HIVE-19904 > URL: https://issues.apache.org/jira/browse/HIVE-19904 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-19904.1.patch > > > Load data rewrite into IAS fails for ACID as there is some code which does > not take into account the table name could be in upper case, specifically > ValidTxnWriteIdList -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19905) By default the configuration property hive.metastore.schema.verification is false
[ https://issues.apache.org/jira/browse/HIVE-19905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates reassigned HIVE-19905: - Assignee: Alan Gates > By default the configuration property hive.metastore.schema.verification is > false > - > > Key: HIVE-19905 > URL: https://issues.apache.org/jira/browse/HIVE-19905 > Project: Hive > Issue Type: Improvement >Reporter: 奔跑的蜗牛 >Assignee: Alan Gates >Priority: Major > > The default value is true in apache-hive-2.3.3-bin.tar. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19785) Race condition when timeout task is invoked during SASL negotation
[
https://issues.apache.org/jira/browse/HIVE-19785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-19785:
Status: In Progress (was: Patch Available)
> Race condition when timeout task is invoked during SASL negotation
> --
>
> Key: HIVE-19785
> URL: https://issues.apache.org/jira/browse/HIVE-19785
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Aihua Xu
>Priority: Major
>
> There is a race condition that leads to some extraneous exception messages
> when the timeout task is invoked in {{RpcServer}}.
> If a timeout is triggered by {{RpcServer#registerClient}} the method will
> remove the {{clientId}} from {{pendingClients}}. However, if the SASL
> negotiation is in progress when the timeout task is invoked, then
> {{SaslServerHandler#update}} will throw an {{IllegalArgumentException}}
> complaining that it can't find the {{clientId}} in the map of
> {{pendingClients}}.
> The timeout still succeeds, but the logging is confusing and multiple
> exceptions make this difficult to debug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19861) Fix temp table path generation for acid table export
[
https://issues.apache.org/jira/browse/HIVE-19861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514243#comment-16514243
]
Hive QA commented on HIVE-19861:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927748/HIVE-19861.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14532 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11806/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11806/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11806/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927748 - PreCommit-HIVE-Build
> Fix temp table path generation for acid table export
>
>
> Key: HIVE-19861
> URL: https://issues.apache.org/jira/browse/HIVE-19861
> Project: Hive
> Issue Type: Bug
> Components: Import/Export, Transactions
>Affects Versions: 3.0.0
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-19861.1.patch, HIVE-19861.2.patch
>
>
> Temp tables that are analyzed by the SemanticAnalyzer get their default
> location set to a location in the session directory. Export of Acid tables
> also creates temp tables, but this is done via a plan transformation, and the
> temp table creation never goes through the SemanticAnalyzer, meaning the
> location is not set. There is some other logic in DDLTask (which I am
> changing in HIV-19837) which ends up automatically setting this path to the
> default table location in the warehouse directory. This should be fixed so
> that the path defaults to a location in the session directory, like with
> normal temp tables.
> cc [~ekoifman]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19785) Race condition when timeout task is invoked during SASL negotation
[
https://issues.apache.org/jira/browse/HIVE-19785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-19785:
Attachment: (was: HIVE-19785.1.patch)
> Race condition when timeout task is invoked during SASL negotation
> --
>
> Key: HIVE-19785
> URL: https://issues.apache.org/jira/browse/HIVE-19785
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Aihua Xu
>Priority: Major
>
> There is a race condition that leads to some extraneous exception messages
> when the timeout task is invoked in {{RpcServer}}.
> If a timeout is triggered by {{RpcServer#registerClient}} the method will
> remove the {{clientId}} from {{pendingClients}}. However, if the SASL
> negotiation is in progress when the timeout task is invoked, then
> {{SaslServerHandler#update}} will throw an {{IllegalArgumentException}}
> complaining that it can't find the {{clientId}} in the map of
> {{pendingClients}}.
> The timeout still succeeds, but the logging is confusing and multiple
> exceptions make this difficult to debug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19800) Create separate submodules for pre and post upgrade and add rename file logic
[ https://issues.apache.org/jira/browse/HIVE-19800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-19800: -- Status: Open (was: Patch Available) > Create separate submodules for pre and post upgrade and add rename file logic > - > > Key: HIVE-19800 > URL: https://issues.apache.org/jira/browse/HIVE-19800 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Blocker > Attachments: HIVE-19800.01.patch, HIVE-19800.02.patch, > HIVE-19800.03.patch > > > this is a followup to HIVE-19751 which includes HIVE-19751 since it hasn't > landed yet > this includes file rename logic and HIVE-19750 since it hasn't landed yet > either > > cc [~jdere] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-19904) Load data rewrite into Tez job fails for ACID
[
https://issues.apache.org/jira/browse/HIVE-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514253#comment-16514253
]
Eugene Koifman edited comment on HIVE-19904 at 6/15/18 7:25 PM:
it seems to me that the name should be normalized when table is created rather
than trying to patch every place that may be doing a look up. Lower case is
the expectation throughout, so conforming to that is safer.
This is not related to this patch, but how does the parser know anything about
{{tempTableObj}} in {{reparseAndSuperAnalyze}}. It seems to be local to this
method.
was (Author: ekoifman):
it seems to me that the name should be normalized when table is created rather
than trying to patch every place that may be doing a look up. Lower case is
the expectation throughout, so conforming to that is safer.
> Load data rewrite into Tez job fails for ACID
> -
>
> Key: HIVE-19904
> URL: https://issues.apache.org/jira/browse/HIVE-19904
> Project: Hive
> Issue Type: Bug
>Reporter: Deepak Jaiswal
>Assignee: Deepak Jaiswal
>Priority: Major
> Attachments: HIVE-19904.1.patch
>
>
> Load data rewrite into IAS fails for ACID as there is some code which does
> not take into account the table name could be in upper case, specifically
> ValidTxnWriteIdList
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18725) Improve error handling for subqueries if there is wrong column reference
[
https://issues.apache.org/jira/browse/HIVE-18725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Kryvenko updated HIVE-18725:
-
Status: Patch Available (was: Open)
> Improve error handling for subqueries if there is wrong column reference
>
>
> Key: HIVE-18725
> URL: https://issues.apache.org/jira/browse/HIVE-18725
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-18725.01.patch, HIVE-18725.02.patch,
> HIVE-18725.03.patch, HIVE-18725.04.patch, HIVE-18725.05.patch
>
>
> If there is a column reference within subquery which doesn't exist Hive
> throws misleading error message.
> e.g.
> {code:sql}
> select * from table1 where table1.col1 IN (select col2 from table2 where
> table2.col1=table1.non_existing_column) and table1.col1 IN (select 4);
> {code}
> The above query, assuming table1 doesn't have non_existing_column, will throw
> following misleading error:
> {noformat}
> FAILED: SemanticException Line 0:-1 Unsupported SubQuery Expression 'col1':
> Only 1 SubQuery expression is supported.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19532) Modify Hive Driver/Executor to support transactional-stats-using COUNT aggregation queries
[ https://issues.apache.org/jira/browse/HIVE-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Yeom updated HIVE-19532: -- Attachment: HIVE-19532.03.patch > Modify Hive Driver/Executor to support transactional-stats-using COUNT > aggregation queries > --- > > Key: HIVE-19532 > URL: https://issues.apache.org/jira/browse/HIVE-19532 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19532.01.patch, HIVE-19532.01.prepatch, > HIVE-19532.02.patch, HIVE-19532.02.prepatch, HIVE-19532.03.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19532) Modify Hive Driver/Executor to support transactional-stats-using COUNT aggregation queries
[ https://issues.apache.org/jira/browse/HIVE-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514280#comment-16514280 ] Steve Yeom commented on HIVE-19532: --- Pre-commit tests on patch 02 did not run but failed to compile due to a conflict. I just rebased and submitted patch 03 which is equal to patch 01 except test failure related fixes. I.e., patch 03 does not have changes according to review comments and some more. > Modify Hive Driver/Executor to support transactional-stats-using COUNT > aggregation queries > --- > > Key: HIVE-19532 > URL: https://issues.apache.org/jira/browse/HIVE-19532 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19532.01.patch, HIVE-19532.01.prepatch, > HIVE-19532.02.patch, HIVE-19532.02.prepatch, HIVE-19532.03.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19532) legacy jira to run the tests
[ https://issues.apache.org/jira/browse/HIVE-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19532: Summary: legacy jira to run the tests (was: Modify Hive Driver/Executor to support transactional-stats-using COUNT aggregation queries ) > legacy jira to run the tests > > > Key: HIVE-19532 > URL: https://issues.apache.org/jira/browse/HIVE-19532 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19532.01.patch, HIVE-19532.01.prepatch, > HIVE-19532.02.patch, HIVE-19532.02.prepatch, HIVE-19532.03.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19884) Invalidation cache may throw NPE when there is no data in table used by materialized view
[ https://issues.apache.org/jira/browse/HIVE-19884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-19884: Resolution: Fixed Fix Version/s: 3.1.0 Status: Resolved (was: Patch Available) Pushed to master and branch-3. Thanks, Jesus! > Invalidation cache may throw NPE when there is no data in table used by > materialized view > - > > Key: HIVE-19884 > URL: https://issues.apache.org/jira/browse/HIVE-19884 > Project: Hive > Issue Type: Bug > Components: Materialized views >Affects Versions: 3.1.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19884.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19914) master-txnstats branch - make sure SQL changes are in correct upgrade scripts
[
https://issues.apache.org/jira/browse/HIVE-19914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin reassigned HIVE-19914:
---
Assignee: Steve Yeom
> master-txnstats branch - make sure SQL changes are in correct upgrade scripts
> -
>
> Key: HIVE-19914
> URL: https://issues.apache.org/jira/browse/HIVE-19914
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Steve Yeom
>Priority: Major
>
> The initial commit changed multiple files e.g.
> {noformat}
> standalone-metastore/src/main/sql/mysql/hive-schema-3.0.0.mysql.sql
> standalone-metastore/src/main/sql/mysql/hive-schema-4.0.0.mysql.sql
> standalone-metastore/src/main/sql/mysql/upgrade-2.3.0-to-3.0.0.mysql.sql
> standalone-metastore/src/main/sql/mysql/upgrade-3.1.0-to-4.0.0.mysql.sql
> {noformat}
> The target version is currently 4.0 (or 3.1? cc [~hagleitn]), so all the
> changes should be in the scripts upgrading to 4.0
> cc [~vgarg]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19533) Modify Hive to support transactional-stats-using aggregation queries with all other than COUNT
[ https://issues.apache.org/jira/browse/HIVE-19533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514308#comment-16514308 ] Sergey Shelukhin commented on HIVE-19533: - What is the status of this issue? > Modify Hive to support transactional-stats-using aggregation queries with all > other than COUNT > -- > > Key: HIVE-19533 > URL: https://issues.apache.org/jira/browse/HIVE-19533 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 3.1.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19867) Test and verify Concurrent INSERTS
[ https://issues.apache.org/jira/browse/HIVE-19867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514305#comment-16514305 ] Sergey Shelukhin commented on HIVE-19867: - What is the status of this issue? > Test and verify Concurrent INSERTS > > > Key: HIVE-19867 > URL: https://issues.apache.org/jira/browse/HIVE-19867 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 4.0.0 >Reporter: Steve Yeom >Priority: Major > Fix For: 4.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19879) Remove unused calcite sql operator.
[ https://issues.apache.org/jira/browse/HIVE-19879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-19879: Resolution: Fixed Fix Version/s: 3.1.0 Status: Resolved (was: Patch Available) Pushed to master and branch-3. Thanks, Slim! > Remove unused calcite sql operator. > --- > > Key: HIVE-19879 > URL: https://issues.apache.org/jira/browse/HIVE-19879 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Minor > Fix For: 3.1.0 > > Attachments: HIVE-19879.patch, HIVE-19879.patch > > > HIVE-19796 introduced by mistake an unused sql operator. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19865) Full ACID table stats has wrong rawDataSize
[ https://issues.apache.org/jira/browse/HIVE-19865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514306#comment-16514306 ] Sergey Shelukhin commented on HIVE-19865: - What is the status of this issue? > Full ACID table stats has wrong rawDataSize > --- > > Key: HIVE-19865 > URL: https://issues.apache.org/jira/browse/HIVE-19865 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 4.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 4.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19912) Schema evolution checks prints a log line in INFO mode for each vectorized rowbatch, impacts performance
[
https://issues.apache.org/jira/browse/HIVE-19912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran updated HIVE-19912:
-
Attachment: HIVE-19912.2.patch
> Schema evolution checks prints a log line in INFO mode for each vectorized
> rowbatch, impacts performance
>
>
> Key: HIVE-19912
> URL: https://issues.apache.org/jira/browse/HIVE-19912
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Nita Dembla
>Assignee: Prasanth Jayachandran
>Priority: Major
> Fix For: 3.1.0, 3.0.1, 4.0.0
>
> Attachments: HIVE-19912.1.patch, HIVE-19912.2.patch
>
>
> While benchmarking query96, noticed 17K log lines printed for each vector
> rowbactch
>
> In file ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java
>
> {code:java}
> @@ -2554,8 +2554,8 @@ public static TypeDescription
> getDesiredRowTypeDescr(Configuration conf,
> }
> if (haveSchemaEvolutionProperties) {
> - if (LOG.isInfoEnabled()) {
> - LOG.info("Using schema evolution configuration variables
> schema.evolution.columns " +
> + if (LOG.isDebugEnabled()) {
> + LOG.debug("Using schema evolution configuration variables
> schema.evolution.columns " +
> schemaEvolutionColumnNames.toString() +
> " / schema.evolution.columns.types " +
> schemaEvolutionTypeDescrs.toString() +{code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19912) Schema evolution checks prints a log line in INFO mode for each vectorized rowbatch, impacts performance
[
https://issues.apache.org/jira/browse/HIVE-19912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran updated HIVE-19912:
-
Resolution: Fixed
Fix Version/s: 4.0.0
3.1.0
Status: Resolved (was: Patch Available)
Committed to branch-3, branch-3.0 and master. Thanks for the patch!
> Schema evolution checks prints a log line in INFO mode for each vectorized
> rowbatch, impacts performance
>
>
> Key: HIVE-19912
> URL: https://issues.apache.org/jira/browse/HIVE-19912
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Nita Dembla
>Assignee: Nita Dembla
>Priority: Major
> Fix For: 3.1.0, 3.0.1, 4.0.0
>
> Attachments: HIVE-19912.1.patch, HIVE-19912.2.patch
>
>
> While benchmarking query96, noticed 17K log lines printed for each vector
> rowbactch
>
> In file ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java
>
> {code:java}
> @@ -2554,8 +2554,8 @@ public static TypeDescription
> getDesiredRowTypeDescr(Configuration conf,
> }
> if (haveSchemaEvolutionProperties) {
> - if (LOG.isInfoEnabled()) {
> - LOG.info("Using schema evolution configuration variables
> schema.evolution.columns " +
> + if (LOG.isDebugEnabled()) {
> + LOG.debug("Using schema evolution configuration variables
> schema.evolution.columns " +
> schemaEvolutionColumnNames.toString() +
> " / schema.evolution.columns.types " +
> schemaEvolutionTypeDescrs.toString() +{code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-19912) Schema evolution checks prints a log line in INFO mode for each vectorized rowbatch, impacts performance
[
https://issues.apache.org/jira/browse/HIVE-19912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran reassigned HIVE-19912:
Assignee: Nita Dembla (was: Prasanth Jayachandran)
> Schema evolution checks prints a log line in INFO mode for each vectorized
> rowbatch, impacts performance
>
>
> Key: HIVE-19912
> URL: https://issues.apache.org/jira/browse/HIVE-19912
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Nita Dembla
>Assignee: Nita Dembla
>Priority: Major
> Fix For: 3.1.0, 3.0.1, 4.0.0
>
> Attachments: HIVE-19912.1.patch, HIVE-19912.2.patch
>
>
> While benchmarking query96, noticed 17K log lines printed for each vector
> rowbactch
>
> In file ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java
>
> {code:java}
> @@ -2554,8 +2554,8 @@ public static TypeDescription
> getDesiredRowTypeDescr(Configuration conf,
> }
> if (haveSchemaEvolutionProperties) {
> - if (LOG.isInfoEnabled()) {
> - LOG.info("Using schema evolution configuration variables
> schema.evolution.columns " +
> + if (LOG.isDebugEnabled()) {
> + LOG.debug("Using schema evolution configuration variables
> schema.evolution.columns " +
> schemaEvolutionColumnNames.toString() +
> " / schema.evolution.columns.types " +
> schemaEvolutionTypeDescrs.toString() +{code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19810) StorageHandler fail to ship jars in Tez intermittently
[ https://issues.apache.org/jira/browse/HIVE-19810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514227#comment-16514227 ] Sergey Shelukhin commented on HIVE-19810: - +1. Did you test the patch on some cluster, since there's no unit test? > StorageHandler fail to ship jars in Tez intermittently > -- > > Key: HIVE-19810 > URL: https://issues.apache.org/jira/browse/HIVE-19810 > Project: Hive > Issue Type: Bug > Components: Tez >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-19810.1.patch, HIVE-19810.2.patch, testcase.patch > > > Hive relies on StorageHandler to ship jars to backend automatically in > several cases: JdbcStorageHandler, HBaseStorageHandler, > AccumuloStorageHandler. This does not work reliably, in particular, the first > dag in the session will have those jars, the second will not unless container > is reused. In the later case, the containers allocated to first dag will be > reused in the second dag so the container will have additional resources. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-19904) Load data rewrite into Tez job fails for ACID
[ https://issues.apache.org/jira/browse/HIVE-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514228#comment-16514228 ] Deepak Jaiswal edited comment on HIVE-19904 at 6/15/18 6:44 PM: In this case, the temp table object is created for rewrite purpose. It is in-memory only and never goes to metastore. Please see LoadSemanticAnalyzer.reparseAndAnalyze() for details. was (Author: djaiswal): In this case, the temp table object is created for rewrite purpose. It is in-memory only. Please see LoadSemanticAnalyzer.reparseAndAnalyze() for details. > Load data rewrite into Tez job fails for ACID > - > > Key: HIVE-19904 > URL: https://issues.apache.org/jira/browse/HIVE-19904 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-19904.1.patch > > > Load data rewrite into IAS fails for ACID as there is some code which does > not take into account the table name could be in upper case, specifically > ValidTxnWriteIdList -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19905) By default the configuration property hive.metastore.schema.verification is false
[ https://issues.apache.org/jira/browse/HIVE-19905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates reassigned HIVE-19905: - Assignee: (was: Alan Gates) > By default the configuration property hive.metastore.schema.verification is > false > - > > Key: HIVE-19905 > URL: https://issues.apache.org/jira/browse/HIVE-19905 > Project: Hive > Issue Type: Improvement >Reporter: 奔跑的蜗牛 >Priority: Major > > The default value is true in apache-hive-2.3.3-bin.tar. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19904) Load data rewrite into Tez job fails for ACID
[ https://issues.apache.org/jira/browse/HIVE-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514253#comment-16514253 ] Eugene Koifman commented on HIVE-19904: --- it seems to me that the name should be normalized when table is created rather than trying to patch every place that may be doing a look up. Lower case is the expectation throughout, so conforming to that is safer. > Load data rewrite into Tez job fails for ACID > - > > Key: HIVE-19904 > URL: https://issues.apache.org/jira/browse/HIVE-19904 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-19904.1.patch > > > Load data rewrite into IAS fails for ACID as there is some code which does > not take into account the table name could be in upper case, specifically > ValidTxnWriteIdList -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19532) legacy jira to run the tests
[ https://issues.apache.org/jira/browse/HIVE-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514299#comment-16514299 ] Sergey Shelukhin commented on HIVE-19532: - I've committed the 03 patch to master-txnstats branch for now. I will keep the JIRA open to 1) Run the tests. 2) Address the feedback on 03 patch. Please do all future work for transactional stats ON TOP OF master-txnstats branch, in separate patches. The epic patch to run the tests here should be made by diffing master with master-txnstats. I will periodically merge master to make sure it stays up to date; one can also merge locally just to produce a patch for tests. Given that the branch was just created, the 03 patch is good for now for a test run. I'm going to clean up existing jiras and file smaller scope items. cc [~steveyeom2017] [~ekoifman] [~hagleitn] > legacy jira to run the tests > > > Key: HIVE-19532 > URL: https://issues.apache.org/jira/browse/HIVE-19532 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19532.01.patch, HIVE-19532.01.prepatch, > HIVE-19532.02.patch, HIVE-19532.02.prepatch, HIVE-19532.03.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19879) Remove unused calcite sql operator.
[ https://issues.apache.org/jira/browse/HIVE-19879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514297#comment-16514297 ] Ashutosh Chauhan commented on HIVE-19879: - +1 > Remove unused calcite sql operator. > --- > > Key: HIVE-19879 > URL: https://issues.apache.org/jira/browse/HIVE-19879 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Minor > Attachments: HIVE-19879.patch, HIVE-19879.patch > > > HIVE-19796 introduced by mistake an unused sql operator. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19532) fix tests for master-txnstats branch
[ https://issues.apache.org/jira/browse/HIVE-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19532: Summary: fix tests for master-txnstats branch (was: legacy jira to run the tests) > fix tests for master-txnstats branch > > > Key: HIVE-19532 > URL: https://issues.apache.org/jira/browse/HIVE-19532 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19532.01.patch, HIVE-19532.01.prepatch, > HIVE-19532.02.patch, HIVE-19532.02.prepatch, HIVE-19532.03.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19417) Modify metastore to have/access persistent tables for stats
[ https://issues.apache.org/jira/browse/HIVE-19417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19417: Resolution: Fixed Fix Version/s: txnstats Status: Resolved (was: Patch Available) Committed to master-txnstats as part of another patch. > Modify metastore to have/access persistent tables for stats > --- > > Key: HIVE-19417 > URL: https://issues.apache.org/jira/browse/HIVE-19417 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: txnstats > > Attachments: HIVE-19417.01.patch, HIVE-19417.02.patch, > HIVE-19417.03.patch, HIVE-19417.04.patch, HIVE-19417.05.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HIVE-19470) Modify metastore to have application logic to retrieve/update transactional table stats
[ https://issues.apache.org/jira/browse/HIVE-19470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin resolved HIVE-19470. - Resolution: Fixed Fix Version/s: (was: 3.1.0) txnstats > Modify metastore to have application logic to retrieve/update transactional > table stats > > > Key: HIVE-19470 > URL: https://issues.apache.org/jira/browse/HIVE-19470 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: txnstats > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19533) Modify Hive to support transactional-stats-using aggregation queries with all other than COUNT
[ https://issues.apache.org/jira/browse/HIVE-19533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514310#comment-16514310 ] Steve Yeom commented on HIVE-19533: --- HIVE-19532 patch covers this jira. > Modify Hive to support transactional-stats-using aggregation queries with all > other than COUNT > -- > > Key: HIVE-19533 > URL: https://issues.apache.org/jira/browse/HIVE-19533 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 3.1.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-19532) fix tests for master-txnstats branch
[ https://issues.apache.org/jira/browse/HIVE-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514299#comment-16514299 ] Sergey Shelukhin edited comment on HIVE-19532 at 6/15/18 8:19 PM: -- I've committed the 03 patch to master-txnstats branch for now. I will keep the JIRA open to 1) Run the tests. 2) Address the feedback on 03 patch. Please do all future work for transactional stats ON TOP OF master-txnstats branch, in separate patches. The epic patch to run the tests here should be made by diffing master with master-txnstats. I will periodically merge master to make sure it stays up to date; one can also merge locally just to produce a patch for tests. Given that the branch was just created, the 03 patch is good for now for a test run. The fix version for commits to the branch should be txnstats. We will bulk change it to 4.0/3.1 when we merge the branch. I'm going to clean up existing jiras and file smaller scope items. cc [~steveyeom2017] [~ekoifman] [~hagleitn] was (Author: sershe): I've committed the 03 patch to master-txnstats branch for now. I will keep the JIRA open to 1) Run the tests. 2) Address the feedback on 03 patch. Please do all future work for transactional stats ON TOP OF master-txnstats branch, in separate patches. The epic patch to run the tests here should be made by diffing master with master-txnstats. I will periodically merge master to make sure it stays up to date; one can also merge locally just to produce a patch for tests. Given that the branch was just created, the 03 patch is good for now for a test run. I'm going to clean up existing jiras and file smaller scope items. cc [~steveyeom2017] [~ekoifman] [~hagleitn] > fix tests for master-txnstats branch > > > Key: HIVE-19532 > URL: https://issues.apache.org/jira/browse/HIVE-19532 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19532.01.patch, HIVE-19532.01.prepatch, > HIVE-19532.02.patch, HIVE-19532.02.prepatch, HIVE-19532.03.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19853) Arrow serializer needs to create a TimeStampMicroTZVector instead of TimeStampMicroVector
[ https://issues.apache.org/jira/browse/HIVE-19853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514414#comment-16514414 ] Matt McCline commented on HIVE-19853: - (I code reviewed patch #2 via the Pull Request) > Arrow serializer needs to create a TimeStampMicroTZVector instead of > TimeStampMicroVector > - > > Key: HIVE-19853 > URL: https://issues.apache.org/jira/browse/HIVE-19853 > Project: Hive > Issue Type: Bug >Reporter: Teddy Choi >Assignee: Teddy Choi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-19853.1.patch, HIVE-19853.2.patch > > > HIVE-19723 changed nanosecond to microsecond in Arrow serialization. However, > it needs to be microsecond with time zone. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19853) Arrow serializer needs to create a TimeStampMicroTZVector instead of TimeStampMicroVector
[ https://issues.apache.org/jira/browse/HIVE-19853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514411#comment-16514411 ] Matt McCline commented on HIVE-19853: - [~ewohlstadter] No I can't – patch #2 doesn't have a Hive QA run. > Arrow serializer needs to create a TimeStampMicroTZVector instead of > TimeStampMicroVector > - > > Key: HIVE-19853 > URL: https://issues.apache.org/jira/browse/HIVE-19853 > Project: Hive > Issue Type: Bug >Reporter: Teddy Choi >Assignee: Teddy Choi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-19853.1.patch, HIVE-19853.2.patch > > > HIVE-19723 changed nanosecond to microsecond in Arrow serialization. However, > it needs to be microsecond with time zone. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19917) Export of full CRUD transactional table fails if table is not in default database
[
https://issues.apache.org/jira/browse/HIVE-19917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-19917:
--
Summary: Export of full CRUD transactional table fails if table is not in
default database (was: Import of full CRUD transactional table fails if table
is not in default database)
> Export of full CRUD transactional table fails if table is not in default
> database
> -
>
> Key: HIVE-19917
> URL: https://issues.apache.org/jira/browse/HIVE-19917
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Major
>
> The actual issues is fixed by HIVE-19861.
> This is a follow up to add a test case.
> Issue:
> {noformat}
> org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.IllegalArgumentException: Can not create a Path from a null string
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:940)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:945)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.exec.DDLTask.createTableLike(DDLTask.java:5099)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:433)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.parse.UpdateDeleteSemanticAnalyzer.analyzeAcidExport(UpdateDeleteSemanticAnalyzer.java:195)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.parse.UpdateDeleteSemanticAnalyzer.analyzeInternal(UpdateDeleteSemanticAnalyzer.java:106)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:288)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:658)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1813)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1760)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1755)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:194)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:257)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.operation.Operation.run(Operation.java:243)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:541)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:527)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:312)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:562)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1557)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1542)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:647)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
>
[jira] [Assigned] (HIVE-19917) Import of full CRUD transactional table fails if table is not in default database
[
https://issues.apache.org/jira/browse/HIVE-19917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman reassigned HIVE-19917:
-
> Import of full CRUD transactional table fails if table is not in default
> database
> -
>
> Key: HIVE-19917
> URL: https://issues.apache.org/jira/browse/HIVE-19917
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Major
>
> The actual issues is fixed by HIVE-19861.
> This is a follow up to add a test case.
> Issue:
> {noformat}
> org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.IllegalArgumentException: Can not create a Path from a null string
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:940)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:945)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.exec.DDLTask.createTableLike(DDLTask.java:5099)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:433)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.parse.UpdateDeleteSemanticAnalyzer.analyzeAcidExport(UpdateDeleteSemanticAnalyzer.java:195)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.parse.UpdateDeleteSemanticAnalyzer.analyzeInternal(UpdateDeleteSemanticAnalyzer.java:106)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:288)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:658)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1813)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1760)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1755)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:194)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:257)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.operation.Operation.run(Operation.java:243)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:541)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:527)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:312)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:562)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1557)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1542)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:647)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[?:1.8.0_112]
> at
>
[jira] [Commented] (HIVE-19904) Load data rewrite into Tez job fails for ACID
[ https://issues.apache.org/jira/browse/HIVE-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514426#comment-16514426 ] Eugene Koifman commented on HIVE-19904: --- +1 pending tests > Load data rewrite into Tez job fails for ACID > - > > Key: HIVE-19904 > URL: https://issues.apache.org/jira/browse/HIVE-19904 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-19904.1.patch, HIVE-19904.2.patch > > > Load data rewrite into IAS fails for ACID as there is some code which does > not take into account the table name could be in upper case, specifically > ValidTxnWriteIdList -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19917) Export of full CRUD transactional table fails if table is not in default database
[
https://issues.apache.org/jira/browse/HIVE-19917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-19917:
--
Attachment: HIVE-19917.01.patch
> Export of full CRUD transactional table fails if table is not in default
> database
> -
>
> Key: HIVE-19917
> URL: https://issues.apache.org/jira/browse/HIVE-19917
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Major
> Attachments: HIVE-19917.01.patch
>
>
> The actual issues is fixed by HIVE-19861.
> This is a follow up to add a test case.
> Issue:
> {noformat}
> org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.IllegalArgumentException: Can not create a Path from a null string
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:940)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:945)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.exec.DDLTask.createTableLike(DDLTask.java:5099)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:433)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.parse.UpdateDeleteSemanticAnalyzer.analyzeAcidExport(UpdateDeleteSemanticAnalyzer.java:195)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.parse.UpdateDeleteSemanticAnalyzer.analyzeInternal(UpdateDeleteSemanticAnalyzer.java:106)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:288)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:658)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1813)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1760)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1755)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:194)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:257)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.operation.Operation.run(Operation.java:243)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:541)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:527)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:312)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:562)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1557)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1542)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:647)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
>
[jira] [Updated] (HIVE-19917) Export of full CRUD transactional table fails if table is not in default database
[
https://issues.apache.org/jira/browse/HIVE-19917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-19917:
--
Component/s: Transactions
> Export of full CRUD transactional table fails if table is not in default
> database
> -
>
> Key: HIVE-19917
> URL: https://issues.apache.org/jira/browse/HIVE-19917
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Major
> Attachments: HIVE-19917.01.patch
>
>
> The actual issues is fixed by HIVE-19861.
> This is a follow up to add a test case.
> Issue:
> {noformat}
> org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.IllegalArgumentException: Can not create a Path from a null string
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:940)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:945)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.exec.DDLTask.createTableLike(DDLTask.java:5099)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:433)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.parse.UpdateDeleteSemanticAnalyzer.analyzeAcidExport(UpdateDeleteSemanticAnalyzer.java:195)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.parse.UpdateDeleteSemanticAnalyzer.analyzeInternal(UpdateDeleteSemanticAnalyzer.java:106)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:288)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:658)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1813)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1760)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1755)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:194)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:257)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.operation.Operation.run(Operation.java:243)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:541)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:527)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:312)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:562)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1557)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1542)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:647)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
>
[jira] [Commented] (HIVE-19917) Export of full CRUD transactional table fails if table is not in default database
[
https://issues.apache.org/jira/browse/HIVE-19917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514429#comment-16514429
]
Eugene Koifman commented on HIVE-19917:
---
[~jdere] could you review please? this only adds a test - don't think we even
to take up a build queue spot
> Export of full CRUD transactional table fails if table is not in default
> database
> -
>
> Key: HIVE-19917
> URL: https://issues.apache.org/jira/browse/HIVE-19917
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Major
> Attachments: HIVE-19917.01.patch
>
>
> The actual issues is fixed by HIVE-19861.
> This is a follow up to add a test case.
> Issue:
> {noformat}
> org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.IllegalArgumentException: Can not create a Path from a null string
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:940)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:945)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.exec.DDLTask.createTableLike(DDLTask.java:5099)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:433)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.parse.UpdateDeleteSemanticAnalyzer.analyzeAcidExport(UpdateDeleteSemanticAnalyzer.java:195)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.parse.UpdateDeleteSemanticAnalyzer.analyzeInternal(UpdateDeleteSemanticAnalyzer.java:106)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:288)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:658)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1813)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1760)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1755)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:194)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:257)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.operation.Operation.run(Operation.java:243)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:541)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:527)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:312)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:562)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1557)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1542)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:647)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
>
[jira] [Assigned] (HIVE-19804) msck repair should hold locks
[
https://issues.apache.org/jira/browse/HIVE-19804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vihang Karajgaonkar reassigned HIVE-19804:
--
Assignee: Vihang Karajgaonkar
> msck repair should hold locks
> -
>
> Key: HIVE-19804
> URL: https://issues.apache.org/jira/browse/HIVE-19804
> Project: Hive
> Issue Type: Improvement
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
>
> {msck repair table }} does not hold locks. This can lead to weird
> race conditions when concurrent sessions are running on the same table. For
> example if two sessions run msck on the same table at the same time, they
> both try to add partitions and they might both end up with failures due to
> AlreadyExistsException. Another example would be if a query is running on a
> partitioned table while some other session issues msck repair which add/drops
> the partitions, it could trigger errors during query execution.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19917) Export of full CRUD transactional table fails if table is not in default database
[
https://issues.apache.org/jira/browse/HIVE-19917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514446#comment-16514446
]
Jason Dere commented on HIVE-19917:
---
Does the "create database foo" have to happen before "drop table if exists " +
tblName?
> Export of full CRUD transactional table fails if table is not in default
> database
> -
>
> Key: HIVE-19917
> URL: https://issues.apache.org/jira/browse/HIVE-19917
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Major
> Attachments: HIVE-19917.01.patch
>
>
> The actual issues is fixed by HIVE-19861.
> This is a follow up to add a test case.
> Issue:
> {noformat}
> org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.IllegalArgumentException: Can not create a Path from a null string
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:940)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:945)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.exec.DDLTask.createTableLike(DDLTask.java:5099)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:433)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.parse.UpdateDeleteSemanticAnalyzer.analyzeAcidExport(UpdateDeleteSemanticAnalyzer.java:195)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.parse.UpdateDeleteSemanticAnalyzer.analyzeInternal(UpdateDeleteSemanticAnalyzer.java:106)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:288)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:658)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1813)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1760)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1755)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:194)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:257)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.operation.Operation.run(Operation.java:243)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:541)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:527)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:312)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:562)
> ~[hive-service-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1557)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1542)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:647)
> ~[hive-exec-3.0.0.3.0.0.0-1485.jar:3.0.0.3.0.0.0-1485]
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
>
[jira] [Commented] (HIVE-19725) Add ability to dump non-native tables in replication metadata dump
[
https://issues.apache.org/jira/browse/HIVE-19725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514452#comment-16514452
]
Hive QA commented on HIVE-19725:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
0s{color} | {color:blue} ql in master has 2276 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
20s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11810/dev-support/hive-personality.sh
|
| git revision | master / bceb3dd |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: itests/hive-unit ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11810/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add ability to dump non-native tables in replication metadata dump
> --
>
> Key: HIVE-19725
> URL: https://issues.apache.org/jira/browse/HIVE-19725
> Project: Hive
> Issue Type: Task
> Components: repl
>Affects Versions: 3.0.0, 3.1.0, 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: Repl, pull-request-available
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19725.01.patch, HIVE-19725.02.patch,
> HIVE-19725.03.patch, HIVE-19725.04.patch, HIVE-19725.05.patch,
> HIVE-19725.06-branch-3.patch
>
>
> if hive.repl.dump.metadata.only is set to true, allow dumping non native
> tables also.
> Data dump for non-native tables should never be allowed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-15822) beeline ignore all sql under comment after semicolon
[
https://issues.apache.org/jira/browse/HIVE-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514462#comment-16514462
]
Vineet Garg commented on HIVE-15822:
This particular scenario is already fixed by HIVE-16935.
{code:sql}
-e "
show databases;--some comment here
show tables;"
{code}
Beeline executes both {{show databases}} and {{show tables}} after removing
comments.
> beeline ignore all sql under comment after semicolon
>
>
> Key: HIVE-15822
> URL: https://issues.apache.org/jira/browse/HIVE-15822
> Project: Hive
> Issue Type: Bug
> Components: Beeline, HiveServer2
>Affects Versions: 1.2.1, 2.1.1
>Reporter: muxin
>Assignee: muxin
>Priority: Major
> Attachments: HIVE-15822.patch
>
>
> way to reproduce this error:
> beeline -u jdbc:hive2://localhost:1 -n test -e "
> show databases;--some comment here
> show tables;"
> it will only execute 'show databases', and consider
> '--some comment here
> show tables;'
> as comment( all sql under the first comment appeared after semicolon).
> when the comment is also end with semicolon, the result will be right.
> the root cause is that beeline will only consider a entire command is inputed
> when a line is end with semicolon, otherwise if this line is not started with
> '--' or '#' beeline will combine it with next line until meet semicolon in
> the end. so actually the comment above is not removed(which causes the
> error). then beeline split the entire line by ';', so 'show databases' is
> recognized and executed,
> '--some comment here\n show tables' is considered a comment and discarded.
> my solution is to just remove comment before split by ';', the code can refer
> to solution 2 for similar issue
> :https://issues.apache.org/jira/browse/HIVE-15820
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-15822) beeline ignore all sql under comment after semicolon
[
https://issues.apache.org/jira/browse/HIVE-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514462#comment-16514462
]
Vineet Garg edited comment on HIVE-15822 at 6/15/18 10:38 PM:
--
This particular scenario is already fixed in Hive 3.0 by HIVE-16935.
{code:sql}
-e "
show databases;--some comment here
show tables;"
{code}
Beeline executes both {{show databases}} and {{show tables}} after removing
comments.
was (Author: vgarg):
This particular scenario is already fixed by HIVE-16935.
{code:sql}
-e "
show databases;--some comment here
show tables;"
{code}
Beeline executes both {{show databases}} and {{show tables}} after removing
comments.
> beeline ignore all sql under comment after semicolon
>
>
> Key: HIVE-15822
> URL: https://issues.apache.org/jira/browse/HIVE-15822
> Project: Hive
> Issue Type: Bug
> Components: Beeline, HiveServer2
>Affects Versions: 1.2.1, 2.1.1
>Reporter: muxin
>Assignee: muxin
>Priority: Major
> Attachments: HIVE-15822.patch
>
>
> way to reproduce this error:
> beeline -u jdbc:hive2://localhost:1 -n test -e "
> show databases;--some comment here
> show tables;"
> it will only execute 'show databases', and consider
> '--some comment here
> show tables;'
> as comment( all sql under the first comment appeared after semicolon).
> when the comment is also end with semicolon, the result will be right.
> the root cause is that beeline will only consider a entire command is inputed
> when a line is end with semicolon, otherwise if this line is not started with
> '--' or '#' beeline will combine it with next line until meet semicolon in
> the end. so actually the comment above is not removed(which causes the
> error). then beeline split the entire line by ';', so 'show databases' is
> recognized and executed,
> '--some comment here\n show tables' is considered a comment and discarded.
> my solution is to just remove comment before split by ';', the code can refer
> to solution 2 for similar issue
> :https://issues.apache.org/jira/browse/HIVE-15820
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-15820) comment at the head of beeline -e
[
https://issues.apache.org/jira/browse/HIVE-15820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514469#comment-16514469
]
Vineet Garg commented on HIVE-15820:
[~muxin] You don't need to write this logic. This logic already exists in
{{HiveStringUtils.removeComments(line)}}. You can updated {{Beeline::dispatch}}
to call this method.
> comment at the head of beeline -e
> -
>
> Key: HIVE-15820
> URL: https://issues.apache.org/jira/browse/HIVE-15820
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 1.2.1, 2.1.1
>Reporter: muxin
>Assignee: muxin
>Priority: Major
> Labels: patch
> Attachments: HIVE-15820.patch
>
>
> $ beeline -u jdbc:hive2://localhost:1 -n test -e "
> > --asdfasdfasdfasdf
> > select * from test_table;
> > "
> expected result of the above command should be all rows of test_table(same as
> run in beeline interactive mode),but it does not output anything.
> the cause is that -e option will read commands as one string, and in method
> dispatch(String line) it calls function isComment(String line) in the first,
> which using
> 'lineTrimmed.startsWith("#") || lineTrimmed.startsWith("--")'
> to regard commands as a comment.
> two ways can be considered to fix this problem:
> 1. in method initArgs(String[] args), split command by '\n' into command list
> before dispatch when cl.getOptionValues('e') != null
> 2. in method dispatch(String line), remove comments using this:
> static String removeComments(String line) {
> if (line == null || line.isEmpty()) {
> return line;
> }
> StringBuilder builder = new StringBuilder();
> int escape = -1;
> for (int index = 0; index < line.length(); index++) {
> if (index < line.length() - 1 && line.charAt(index) ==
> line.charAt(index + 1)) {
> if (escape == -1 && line.charAt(index) == '-') {
> //find \n as the end of comment
> index = line.indexOf('\n',index+1);
> //there is no sql after this comment,so just break out
> if (-1==index){
> break;
> }
> }
> }
> char letter = line.charAt(index);
> if (letter == escape) {
> escape = -1; // Turn escape off.
> } else if (escape == -1 && (letter == '\'' || letter == '"')) {
> escape = letter; // Turn escape on.
> }
> builder.append(letter);
> }
> return builder.toString();
> }
> the second way can be a general solution to remove all comments start with
> '--' in a sql
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-15820) comment at the head of beeline -e
[
https://issues.apache.org/jira/browse/HIVE-15820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514469#comment-16514469
]
Vineet Garg edited comment on HIVE-15820 at 6/15/18 10:55 PM:
--
[~muxin] You don't need to write this logic. This logic already exists in
{{HiveStringUtils.removeComments()}}. You can updated {{Beeline::dispatch}} to
call this method.
was (Author: vgarg):
[~muxin] You don't need to write this logic. This logic already exists in
{{HiveStringUtils.removeComments(line)}}. You can updated {{Beeline::dispatch}}
to call this method.
> comment at the head of beeline -e
> -
>
> Key: HIVE-15820
> URL: https://issues.apache.org/jira/browse/HIVE-15820
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 1.2.1, 2.1.1
>Reporter: muxin
>Assignee: muxin
>Priority: Major
> Labels: patch
> Attachments: HIVE-15820.patch
>
>
> $ beeline -u jdbc:hive2://localhost:1 -n test -e "
> > --asdfasdfasdfasdf
> > select * from test_table;
> > "
> expected result of the above command should be all rows of test_table(same as
> run in beeline interactive mode),but it does not output anything.
> the cause is that -e option will read commands as one string, and in method
> dispatch(String line) it calls function isComment(String line) in the first,
> which using
> 'lineTrimmed.startsWith("#") || lineTrimmed.startsWith("--")'
> to regard commands as a comment.
> two ways can be considered to fix this problem:
> 1. in method initArgs(String[] args), split command by '\n' into command list
> before dispatch when cl.getOptionValues('e') != null
> 2. in method dispatch(String line), remove comments using this:
> static String removeComments(String line) {
> if (line == null || line.isEmpty()) {
> return line;
> }
> StringBuilder builder = new StringBuilder();
> int escape = -1;
> for (int index = 0; index < line.length(); index++) {
> if (index < line.length() - 1 && line.charAt(index) ==
> line.charAt(index + 1)) {
> if (escape == -1 && line.charAt(index) == '-') {
> //find \n as the end of comment
> index = line.indexOf('\n',index+1);
> //there is no sql after this comment,so just break out
> if (-1==index){
> break;
> }
> }
> }
> char letter = line.charAt(index);
> if (letter == escape) {
> escape = -1; // Turn escape off.
> } else if (escape == -1 && (letter == '\'' || letter == '"')) {
> escape = letter; // Turn escape on.
> }
> builder.append(letter);
> }
> return builder.toString();
> }
> the second way can be a general solution to remove all comments start with
> '--' in a sql
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19914) master-txnstats branch - make sure SQL changes are in correct upgrade scripts
[
https://issues.apache.org/jira/browse/HIVE-19914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514473#comment-16514473
]
Vineet Garg commented on HIVE-19914:
[~sershe] Can you provide the commit which changed these files? Target version
currently is 4.0 for master and 3.1 for branch-3.
> master-txnstats branch - make sure SQL changes are in correct upgrade scripts
> -
>
> Key: HIVE-19914
> URL: https://issues.apache.org/jira/browse/HIVE-19914
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Steve Yeom
>Priority: Major
>
> The initial commit changed multiple files e.g.
> {noformat}
> standalone-metastore/src/main/sql/mysql/hive-schema-3.0.0.mysql.sql
> standalone-metastore/src/main/sql/mysql/hive-schema-4.0.0.mysql.sql
> standalone-metastore/src/main/sql/mysql/upgrade-2.3.0-to-3.0.0.mysql.sql
> standalone-metastore/src/main/sql/mysql/upgrade-3.1.0-to-4.0.0.mysql.sql
> {noformat}
> The target version is currently 4.0 (or 3.1? cc [~hagleitn]), so all the
> changes should be in the scripts upgrading to 4.0
> cc [~vgarg]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19918) IllegalArgumentException when getContext().getVertexTaskResource().getMemory() returns -1
[
https://issues.apache.org/jira/browse/HIVE-19918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514472#comment-16514472
]
Prasanth Jayachandran commented on HIVE-19918:
--
Missing hive.tez.container.size?
> IllegalArgumentException when
> getContext().getVertexTaskResource().getMemory() returns -1
> -
>
> Key: HIVE-19918
> URL: https://issues.apache.org/jira/browse/HIVE-19918
> Project: Hive
> Issue Type: Bug
>Reporter: Jaume M
>Priority: Major
>
> {code}
> Beeline version 4.0.0-SNAPSHOT by Apache Hive
> ;: jdbc:hive2://hs2.example.com:1/> CREATE TABLE pokes (foo INT, bar
> STRING)
> No rows affected (3.834 seconds)
> G) PARTITIONED BY (ds STRING);m:1/> CREATE TABLE invites (foo INT, bar
> STRIN
> No rows affected (0.175 seconds)
> 0: jdbc:hive2://hs2.example.com:1/> SELECT * FROM pokes, invites;
> Error: Error while processing statement: FAILED: Execution Error, return code
> 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed,
> vertexName=Map 1, vertexId=vertex_1529099908701_0003_1_01,
> diagnostics=[Vertex vertex_1529099908701_0003_1_01 [Map 1] killed/failed due
> to:ROOT_INPUT_INIT_FAILURE, Vertex Input: pokes initializer failed,
> vertex=vertex_1529099908701_0003_1_01 [Map 1],
> java.lang.IllegalArgumentException: Illegal Capacity: -10444
> at java.util.ArrayList.(ArrayList.java:157)
> at
> org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:339)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.addSplitsForGroup(HiveInputFormat.java:519)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getSplits(HiveInputFormat.java:768)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:211)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:269)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:253)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> ]DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
> (state=08S01,code=2)
> {code}
> This is happening because
> [here|https://github.com/apache/hive/blob/bceb3dd82cbeb0334c3c33da7a21a74ad631810b/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HiveSplitGenerator.java#L188]
> the return is -1 and afterwards we request a negative number of splits.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19416) Create single version transactional table metastore statistics for aggregation queries
[ https://issues.apache.org/jira/browse/HIVE-19416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514477#comment-16514477 ] Sergey Shelukhin commented on HIVE-19416: - c/p: Permalink Edit Delete sershe Sergey Shelukhin added a comment - 3 hours ago - edited I've committed the 03 patch to master-txnstats branch for now. I will keep the JIRA open to 1) Run the tests. 2) Address the feedback on 03 patch. Please do all future work for transactional stats ON TOP OF master-txnstats branch, in separate patches. The epic patch to run the tests here should be made by diffing master with master-txnstats. I will periodically merge master to make sure it stays up to date; one can also merge locally just to produce a patch for tests. Given that the branch was just created, the 03 patch is good for now for a test run. The fix version for commits to the branch should be txnstats. We will bulk change it to 4.0/3.1 when we merge the branch. I'm going to clean up existing jiras and file smaller scope items. cc Steve Yeom Eugene Koifman Gunther Hagleitner Permalink Edit Delete sershe Sergey Shelukhin added a comment - 3 hours ago Steve Yeom Can you update RB with 03 patch? Just to finish the review and perhaps file some more follow up jiras. Also can you make a list of things that are missing? There are some JIRAs remaining as subtasks, I'm not sure if they are addressed as part of the epic patch or not. I filed some known issue JIRAs, let me know if there's more. Or is it just enough to fix the tests and it's good? > Create single version transactional table metastore statistics for > aggregation queries > -- > > Key: HIVE-19416 > URL: https://issues.apache.org/jira/browse/HIVE-19416 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > > The system should use only statistics for aggregation queries like count on > transactional tables. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19725) Add ability to dump non-native tables in replication metadata dump
[
https://issues.apache.org/jira/browse/HIVE-19725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514485#comment-16514485
]
Hive QA commented on HIVE-19725:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927770/HIVE-19725.06-branch-3.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 46 failed/errored test(s), 14339 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=254)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=254)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=254)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=254)
TestTezPerfCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=254)
org.apache.hadoop.hive.ql.TestTxnCommands2.testACIDwithSchemaEvolutionAndCompaction
(batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testAcidWithSchemaEvolution
(batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testAlterTable (batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testBucketCodec (batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testBucketizedInputFormat
(batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testCleanerForTxnToWriteId
(batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testCompactWithDelete (batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testDeleteIn (batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testDynamicPartitionsMerge
(batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testDynamicPartitionsMerge2
(batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testETLSplitStrategyForACID
(batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testEmptyInTblproperties
(batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testFailHeartbeater (batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testFailureOnAlteringTransactionalProperties
(batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testFileSystemUnCaching (batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testInitiatorWithMultipleFailedCompactions
(batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testInsertOverwrite1 (batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testInsertOverwrite2 (batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testInsertOverwriteWithSelfJoin
(batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMerge (batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMerge2 (batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMerge3 (batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMergeWithPredicate (batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMmTableCompaction (batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMultiInsert (batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMultiInsertStatement
(batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testNoHistory (batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidInsert (batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion02
(batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1
(batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion2
(batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion3
(batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testOpenTxnsCounter (batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testOrcNoPPD (batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testOrcPPD (batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testOriginalFileReaderWhenNonAcidConvertedToAcid
(batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testUpdateMixedCase (batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.testValidTxnsBookkeeping
(batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.updateDeletePartitioned (batchId=287)
org.apache.hadoop.hive.ql.TestTxnCommands2.writeBetweenWorkerAndCleaner
(batchId=287)
org.apache.hive.spark.client.rpc.TestRpc.testServerPort (batchId=306)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11810/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11810/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11810/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 46 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT
[jira] [Commented] (HIVE-19267) Create/Replicate ACID Write event
[
https://issues.apache.org/jira/browse/HIVE-19267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514488#comment-16514488
]
Hive QA commented on HIVE-19267:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927765/HIVE-19267.15.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11811/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11811/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11811/
Messages:
{noformat}
This message was trimmed, see log for full details
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-06-15 23:17:51.991
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
bceb3dd..73ee8a1 master -> origin/master
+ git reset --hard HEAD
HEAD is now at 56fd278 HIVE-19861: Fix temp table path generation for acid
table export (Jason Dere, reviewed by Eugene Koifman)
+ git clean -f -d
+ git checkout master
Switched to branch 'master'
Your branch is behind 'origin/master' by 3 commits, and can be fast-forwarded.
(use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at 73ee8a1 HIVE-19837: Setting to have different default location
for external tables (Jason Dere, reviewed by Ashutosh Chauhan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-06-15 23:17:53.665
+ rm -rf ../yetus_PreCommit-HIVE-Build-11811
+ mkdir ../yetus_PreCommit-HIVE-Build-11811
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-11811
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-11811/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java: does not
exist in index
error:
a/hcatalog/server-extensions/src/main/java/org/apache/hive/hcatalog/listener/DbNotificationListener.java:
does not exist in index
error:
a/itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java:
does not exist in index
error:
a/itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/TestDbNotificationListener.java:
does not exist in index
error:
a/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationScenarios.java:
does not exist in index
error:
a/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationScenariosAcidTables.java:
does not exist in index
error:
a/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationScenariosAcrossInstances.java:
does not exist in index
error:
a/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/WarehouseInstance.java:
does not exist in index
error:
a/ql/src/java/org/apache/hadoop/hive/metastore/SynchronizedMetaStoreClient.java:
does not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java: does not
exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/ReplCopyTask.java: does not
exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/ReplTxnTask.java: does not
exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java:
does not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java: does not
exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java: does
not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/lockmgr/DbTxnManager.java: does
not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/lockmgr/DummyTxnManager.java:
does not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/lockmgr/HiveTxnManager.java:
does not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java: does not
exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveUtils.java: does
not exist in index
error:
a/ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java: does
not exist in index
error:
a/ql/src/java/org/apache/hadoop/hive/ql/parse/ReplicationSemanticAnalyzer.java:
does not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:
does not exist in index
error:
a/ql/src/java/org/apache/hadoop/hive/ql/parse/UpdateDeleteSemanticAnalyzer.java:
does not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/parse/repl/CopyUtils.java: does
not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/Utils.java: does
not exist in
[jira] [Commented] (HIVE-11573) PointLookupOptimizer can be pessimistic at a low nDV
[ https://issues.apache.org/jira/browse/HIVE-11573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514489#comment-16514489 ] Gopal V commented on HIVE-11573: [~kgyrtkirk]: this was written to prevent algorithmic complexity of the expression walkers because (a = 1 || a = 2) is actually UDFOr(UDFOpEq(a, 1), UDFOpEq(a, 2)) while the other one is GenericUDFIn(a, [1,2]) which is faster to walk through. The original bad case had 680 (a = 1 and b = 2) or (a=1 and b=3) ... etc, which got generated as (a=1) and (b=(2,3,...)) and ((a,b) IN ((1,2),(2,3)...) so that the disjunction could be pushed down to the ORC PPD. > PointLookupOptimizer can be pessimistic at a low nDV > > > Key: HIVE-11573 > URL: https://issues.apache.org/jira/browse/HIVE-11573 > Project: Hive > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Fix For: 2.0.0 > > Attachments: HIVE-11573.1.patch, HIVE-11573.2.patch, > HIVE-11573.3.patch, HIVE-11573.4.patch, HIVE-11573.5.patch, HIVE-11573.6.patch > > > The PointLookupOptimizer can turn off some of the optimizations due to its > use of tuple IN() clauses. > Limit the application of the optimizer for very low nDV cases and extract the > sub-clause as a pre-condition during runtime, to trigger the simple column > predicate index lookups. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-11573) PointLookupOptimizer can be pessimistic at a low nDV
[ https://issues.apache.org/jira/browse/HIVE-11573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514489#comment-16514489 ] Gopal V edited comment on HIVE-11573 at 6/15/18 11:18 PM: -- [~kgyrtkirk]: this was written to prevent algorithmic complexity of the expression walkers because (a = 1 || a = 2) is actually UDFOr(UDFOpEq(a, 1), UDFOpEq(a, 2)) while the other one is GenericUDFIn(a, [1,2]) which is faster to walk through. The original bad case had 680 x (a = 1 and b = 2) or (a=1 and b=3) ... etc, which got generated as (a=1) and (b=(2,3,...)) and ((a,b) IN ((1,2),(2,3)...) so that the disjunction could be pushed down to the ORC PPD. was (Author: gopalv): [~kgyrtkirk]: this was written to prevent algorithmic complexity of the expression walkers because (a = 1 || a = 2) is actually UDFOr(UDFOpEq(a, 1), UDFOpEq(a, 2)) while the other one is GenericUDFIn(a, [1,2]) which is faster to walk through. The original bad case had 680 (a = 1 and b = 2) or (a=1 and b=3) ... etc, which got generated as (a=1) and (b=(2,3,...)) and ((a,b) IN ((1,2),(2,3)...) so that the disjunction could be pushed down to the ORC PPD. > PointLookupOptimizer can be pessimistic at a low nDV > > > Key: HIVE-11573 > URL: https://issues.apache.org/jira/browse/HIVE-11573 > Project: Hive > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Fix For: 2.0.0 > > Attachments: HIVE-11573.1.patch, HIVE-11573.2.patch, > HIVE-11573.3.patch, HIVE-11573.4.patch, HIVE-11573.5.patch, HIVE-11573.6.patch > > > The PointLookupOptimizer can turn off some of the optimizations due to its > use of tuple IN() clauses. > Limit the application of the optimizer for very low nDV cases and extract the > sub-clause as a pre-condition during runtime, to trigger the simple column > predicate index lookups. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19837) Setting to have different default location for external tables
[ https://issues.apache.org/jira/browse/HIVE-19837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-19837: -- Resolution: Fixed Fix Version/s: 4.0.0 3.1.0 Status: Resolved (was: Patch Available) Committed to master/branch-3 > Setting to have different default location for external tables > -- > > Key: HIVE-19837 > URL: https://issues.apache.org/jira/browse/HIVE-19837 > Project: Hive > Issue Type: Bug >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-19837.1.patch, HIVE-19837.3.patch > > > Allow external tables to have a different default location than managed tables -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19914) master-txnstats branch - make sure SQL changes are in correct upgrade scripts
[
https://issues.apache.org/jira/browse/HIVE-19914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514508#comment-16514508
]
Sergey Shelukhin commented on HIVE-19914:
-
[~vgarg] head of master-txnstats branch.
We are actually going to recreate the branch because apparently the patch I
used was not complete. But in any case it's going to be on the branch for now.
> master-txnstats branch - make sure SQL changes are in correct upgrade scripts
> -
>
> Key: HIVE-19914
> URL: https://issues.apache.org/jira/browse/HIVE-19914
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Steve Yeom
>Priority: Major
>
> The initial commit changed multiple files e.g.
> {noformat}
> standalone-metastore/src/main/sql/mysql/hive-schema-3.0.0.mysql.sql
> standalone-metastore/src/main/sql/mysql/hive-schema-4.0.0.mysql.sql
> standalone-metastore/src/main/sql/mysql/upgrade-2.3.0-to-3.0.0.mysql.sql
> standalone-metastore/src/main/sql/mysql/upgrade-3.1.0-to-4.0.0.mysql.sql
> {noformat}
> The target version is currently 4.0 (or 3.1? cc [~hagleitn]), so all the
> changes should be in the scripts upgrading to 4.0
> cc [~vgarg]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HIVE-19918) IllegalArgumentException when getContext().getVertexTaskResource().getMemory() returns -1
[
https://issues.apache.org/jira/browse/HIVE-19918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jaume M resolved HIVE-19918.
Resolution: Not A Bug
That solved it, thanks!
> IllegalArgumentException when
> getContext().getVertexTaskResource().getMemory() returns -1
> -
>
> Key: HIVE-19918
> URL: https://issues.apache.org/jira/browse/HIVE-19918
> Project: Hive
> Issue Type: Bug
>Reporter: Jaume M
>Priority: Major
>
> {code}
> Beeline version 4.0.0-SNAPSHOT by Apache Hive
> ;: jdbc:hive2://hs2.example.com:1/> CREATE TABLE pokes (foo INT, bar
> STRING)
> No rows affected (3.834 seconds)
> G) PARTITIONED BY (ds STRING);m:1/> CREATE TABLE invites (foo INT, bar
> STRIN
> No rows affected (0.175 seconds)
> 0: jdbc:hive2://hs2.example.com:1/> SELECT * FROM pokes, invites;
> Error: Error while processing statement: FAILED: Execution Error, return code
> 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed,
> vertexName=Map 1, vertexId=vertex_1529099908701_0003_1_01,
> diagnostics=[Vertex vertex_1529099908701_0003_1_01 [Map 1] killed/failed due
> to:ROOT_INPUT_INIT_FAILURE, Vertex Input: pokes initializer failed,
> vertex=vertex_1529099908701_0003_1_01 [Map 1],
> java.lang.IllegalArgumentException: Illegal Capacity: -10444
> at java.util.ArrayList.(ArrayList.java:157)
> at
> org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:339)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.addSplitsForGroup(HiveInputFormat.java:519)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getSplits(HiveInputFormat.java:768)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:211)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:269)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:253)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> ]DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
> (state=08S01,code=2)
> {code}
> This is happening because
> [here|https://github.com/apache/hive/blob/bceb3dd82cbeb0334c3c33da7a21a74ad631810b/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HiveSplitGenerator.java#L188]
> the return is -1 and afterwards we request a negative number of splits.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-19416) Create single version transactional table metastore statistics for aggregation queries
[ https://issues.apache.org/jira/browse/HIVE-19416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514477#comment-16514477 ] Sergey Shelukhin edited comment on HIVE-19416 at 6/15/18 11:27 PM: --- c/p: Permalink Edit Delete sershe Sergey Shelukhin added a comment - 3 hours ago - edited I've committed the 03 patch to master-txnstats branch for now. I will keep the JIRA open to 1) Run the tests. 2) Address the feedback on 03 patch. Please do all future work for transactional stats ON TOP OF master-txnstats branch, in separate patches. The epic patch to run the tests here should be made by diffing master with master-txnstats. I will periodically merge master to make sure it stays up to date; one can also merge locally just to produce a patch for tests. Given that the branch was just created, the 03 patch is good for now for a test run. The fix version for commits to the branch should be txnstats. We will bulk change it to 4.0/3.1 when we merge the branch. I'm going to clean up existing jiras and file smaller scope items. cc Steve Yeom Eugene Koifman Gunther Hagleitner Permalink Edit Delete sershe Sergey Shelukhin added a comment - 3 hours ago Steve Yeom Can you update RB with 03 patch? Just to finish the review and perhaps file some more follow up jiras. Also can you make a list of things that are missing? There are some JIRAs remaining as subtasks, I'm not sure if they are addressed as part of the epic patch or not. I filed some known issue JIRAs, let me know if there's more. Or is it just enough to fix the tests and it's good? Update: apparently 03 patch is not good, we will create the branch again later based on the 04-patch to come. The rest will remain the same. was (Author: sershe): c/p: Permalink Edit Delete sershe Sergey Shelukhin added a comment - 3 hours ago - edited I've committed the 03 patch to master-txnstats branch for now. I will keep the JIRA open to 1) Run the tests. 2) Address the feedback on 03 patch. Please do all future work for transactional stats ON TOP OF master-txnstats branch, in separate patches. The epic patch to run the tests here should be made by diffing master with master-txnstats. I will periodically merge master to make sure it stays up to date; one can also merge locally just to produce a patch for tests. Given that the branch was just created, the 03 patch is good for now for a test run. The fix version for commits to the branch should be txnstats. We will bulk change it to 4.0/3.1 when we merge the branch. I'm going to clean up existing jiras and file smaller scope items. cc Steve Yeom Eugene Koifman Gunther Hagleitner Permalink Edit Delete sershe Sergey Shelukhin added a comment - 3 hours ago Steve Yeom Can you update RB with 03 patch? Just to finish the review and perhaps file some more follow up jiras. Also can you make a list of things that are missing? There are some JIRAs remaining as subtasks, I'm not sure if they are addressed as part of the epic patch or not. I filed some known issue JIRAs, let me know if there's more. Or is it just enough to fix the tests and it's good? > Create single version transactional table metastore statistics for > aggregation queries > -- > > Key: HIVE-19416 > URL: https://issues.apache.org/jira/browse/HIVE-19416 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > > The system should use only statistics for aggregation queries like count on > transactional tables. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19533) Modify Hive to support transactional-stats-using aggregation queries with all other than COUNT
[ https://issues.apache.org/jira/browse/HIVE-19533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19533: Fix Version/s: (was: 3.1.0) > Modify Hive to support transactional-stats-using aggregation queries with all > other than COUNT > -- > > Key: HIVE-19533 > URL: https://issues.apache.org/jira/browse/HIVE-19533 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19532) fix tests for master-txnstats branch
[ https://issues.apache.org/jira/browse/HIVE-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514509#comment-16514509 ] Sergey Shelukhin commented on HIVE-19532: - Update: apparently 03 patch is not good, we will create the branch again later based on the 04-patch to come. The rest will remain the same. > fix tests for master-txnstats branch > > > Key: HIVE-19532 > URL: https://issues.apache.org/jira/browse/HIVE-19532 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19532.01.patch, HIVE-19532.01.prepatch, > HIVE-19532.02.patch, HIVE-19532.02.prepatch, HIVE-19532.03.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18785) Make JSON SerDe First-Class SerDe
[
https://issues.apache.org/jira/browse/HIVE-18785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16513517#comment-16513517
]
Zoltan Haindrich commented on HIVE-18785:
-
I've a json udf related ticket; which is half-way between moving the json
handling codes away from hcat - so that it can be reused in ql... I think that
would help in this matter as well..
> Make JSON SerDe First-Class SerDe
> -
>
> Key: HIVE-18785
> URL: https://issues.apache.org/jira/browse/HIVE-18785
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Assignee: Vihang Karajgaonkar
>Priority: Major
>
> According to the [Hive SerDe
> Docs|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-RowFormats],
> there are some extra steps involved in getting the JSON SerDe to work:
> {quote}
> ROW FORMAT SERDE
> 'org.apache.hive.hcatalog.data.JsonSerDe'
> STORED AS TEXTFILE
> In some distributions, a reference to hive-hcatalog-core.jar is required.
> ADD JAR /usr/lib/hive-hcatalog/lib/hive-hcatalog-core.jar;
> {quote}
> I would like to propose that we move this SerDe into first-class status:
> {{STORED AS JSONFILE}}
> The user should have to perform no additional steps to use this SerDe.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19753) Strict managed tables mode in Hive
[
https://issues.apache.org/jira/browse/HIVE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16513521#comment-16513521
]
Hive QA commented on HIVE-19753:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927899/HIVE-19753.4.patch
{color:green}SUCCESS:{color} +1 due to 7 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 22 failed/errored test(s), 14539 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_rename_partition_authorization]
(batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_3]
(batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_4]
(batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_6]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_alter_table_exchange_partition]
(batchId=46)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_view_2]
(batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_view_3]
(batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_view_4]
(batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_view_disable_cbo_1]
(batchId=74)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_view_disable_cbo_2]
(batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_view_disable_cbo_4]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[init_file] (batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[strict_managed_tables2]
(batchId=73)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[authorization_2]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[authorization_view_8]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[retry_failure]
(batchId=166)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[retry_failure_oom]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[retry_failure_stat_changes]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[strict_managed_tables_sysdb]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_retry_failure]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_3]
(batchId=105)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11798/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11798/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11798/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 22 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927899 - PreCommit-HIVE-Build
> Strict managed tables mode in Hive
> --
>
> Key: HIVE-19753
> URL: https://issues.apache.org/jira/browse/HIVE-19753
> Project: Hive
> Issue Type: Bug
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-19753.1.patch, HIVE-19753.2.patch,
> HIVE-19753.3.patch, HIVE-19753.4.patch
>
>
> Create a mode in Hive which enforces that all managed tables are
> transactional (both full or insert-only tables allowed). Non-transactional
> tables, as well as non-native tables, must be created as external tables when
> this mode is enabled.
> The idea would be that in strict managed tables mode all of the data written
> to managed tables would have been done through Hive.
> The mode would be enabled using config setting hive.strict.managed.tables.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19870) HCatalog dynamic partition query can fail, if the table path is managed by Sentry
[ https://issues.apache.org/jira/browse/HIVE-19870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary updated HIVE-19870: -- Attachment: HIVE-19870.4.patch > HCatalog dynamic partition query can fail, if the table path is managed by > Sentry > - > > Key: HIVE-19870 > URL: https://issues.apache.org/jira/browse/HIVE-19870 > Project: Hive > Issue Type: Bug >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Major > Attachments: HIVE-19870.2.patch, HIVE-19870.3.patch, > HIVE-19870.4.patch, HIVE-19870.patch > > > The central issue is that HCatalog is assuming it needs to handle the storage > based authorization features. When a job completes, in HCatalog's file > committing phase it tries to manually set the permissions for the table for > authorization's sake, which makes it go against auto-authorization managment > features provided by Sentry. > The offending code is specifically at > [https://github.com/apache/hive/blob/master/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java#L373-L374] > and > [https://github.com/apache/hive/blob/master/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java#L380-L385] > and > [https://github.com/apache/hive/blob/master/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java#L952-L954] > where you can notice that _after_ it moves the files created by the job into > their respective partition directories under the final table destination, it > goes onto trying to perform chmod/chgrp operations which will fail out -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19097) related equals and in operators may cause inaccurate stats estimations
[
https://issues.apache.org/jira/browse/HIVE-19097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-19097:
Attachment: HIVE-19097.03.patch
> related equals and in operators may cause inaccurate stats estimations
> --
>
> Key: HIVE-19097
> URL: https://issues.apache.org/jira/browse/HIVE-19097
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-19097.01.patch, HIVE-19097.02.patch,
> HIVE-19097.03.patch, HIVE-19097.partial.patch
>
>
> tpcds#74 is optimized in a way that for date_dim the condition contains IN
> and = for the same column
> {code:java}
> | Map Operator Tree: |
> | TableScan |
> | alias: date_dim |
> | filterExpr: (((d_year) IN (2001, 2002) and (d_year =
> 2002) and d_date_sk is not null) or ((d_year) IN (2001, 2002) and (d_year =
> 2001) and d_date_sk is not null)) (type: boolean) |
> | Statistics: Num rows: 73049 Data size: 876588 Basic
> stats: COMPLETE Column stats: COMPLETE |
> | Filter Operator |
> | predicate: ((d_year) IN (2001, 2002) and (d_year =
> 2002) and d_date_sk is not null) (type: boolean) |
> | Statistics: Num rows: 4 Data size: 48 Basic stats:
> COMPLETE Column stats: COMPLETE |
> {code}
> the "real" row count will be 365
> for separate {{IN}} and {{=}} the estimation is very good; but if both are
> present it becomes (very) underestimated.
> {code:java}
> set hive.query.results.cache.enabled=false;
> drop table if exists t1;
> drop table if exists t8;
> create table t1 (a integer,b integer);
> create table t8 like t1;
> insert into t1 values (1,1),(2,2),(3,3),(4,4),(5,5);
> insert into t8
> select * from t1 union all select * from t1 union all select * from t1 union
> all select * from t1 union all
> select * from t1 union all select * from t1 union all select * from t1 union
> all select * from t1
> ;
> analyze table t1 compute statistics for columns;
> analyze table t8 compute statistics for columns;
> explain analyze select sum(a) from t8 where b in (2,3) group by b;
> explain analyze select sum(a) from t8 where b=2 group by b;
> explain analyze select sum(a) from t1 where b in (2,3) and b=2 group by b;
> explain analyze select sum(a) from t8 where b in (2,3) and b=2 group by b;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19753) Strict managed tables mode in Hive
[
https://issues.apache.org/jira/browse/HIVE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16513529#comment-16513529
]
Hive QA commented on HIVE-19753:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
32s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
28s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
28s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
41s{color} | {color:blue} itests/util in master has 55 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
53s{color} | {color:blue} ql in master has 2276 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
0s{color} | {color:blue} standalone-metastore in master has 227 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
7s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} The patch common passed checkstyle {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
3s{color} | {color:red} root: The patch generated 5 new + 806 unchanged - 3
fixed = 811 total (was 809) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} The patch util passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
8s{color} | {color:green} The patch metastore passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} ql: The patch generated 0 new + 211 unchanged - 3
fixed = 211 total (was 214) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} standalone-metastore: The patch generated 5 new + 94
unchanged - 0 fixed = 99 total (was 94) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
49s{color} | {color:red} itests/util generated 1 new + 55 unchanged - 0 fixed =
56 total (was 55) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 9m
1s{color} | {color:green} root in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
0s{color} | {color:green} util in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} metastore in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} ql generated 0 new + 0 unchanged - 100 fixed = 0
total (was 100) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
26s{color} | {color:green} standalone-metastore in the patch passed. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate
[jira] [Updated] (HIVE-19864) Address TestTriggersWorkloadManager flakiness
[ https://issues.apache.org/jira/browse/HIVE-19864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-19864: - Resolution: Fixed Fix Version/s: 4.0.0 3.1.0 Status: Resolved (was: Patch Available) Committed to master and branch-3. Thanks for the review! > Address TestTriggersWorkloadManager flakiness > - > > Key: HIVE-19864 > URL: https://issues.apache.org/jira/browse/HIVE-19864 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0, 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-19864.1.patch > > > TestTriggersWorkloadManager seems flaky and all test cases gets timed out at > times. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
