[jira] [Commented] (HIVE-18148) NPE in SparkDynamicPartitionPruningResolver
[
https://issues.apache.org/jira/browse/HIVE-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287210#comment-16287210
]
Rui Li commented on HIVE-18148:
---
Upload a patch to demonstrate the idea: only the upper most DPP is kept.
[~kellyzly], the patch contains a new test. If you run it w/o the fix, it
should fail.
> NPE in SparkDynamicPartitionPruningResolver
> ---
>
> Key: HIVE-18148
> URL: https://issues.apache.org/jira/browse/HIVE-18148
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Reporter: Rui Li
>Assignee: Rui Li
> Attachments: HIVE-18148.1.patch
>
>
> The stack trace is:
> {noformat}
> 2017-11-27T10:32:38,752 ERROR [e6c8aab5-ddd2-461d-b185-a7597c3e7519 main]
> ql.Driver: FAILED: NullPointerException null
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.optimizer.physical.SparkDynamicPartitionPruningResolver$SparkDynamicPartitionPruningDispatcher.dispatch(SparkDynamicPartitionPruningResolver.java:100)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.dispatch(TaskGraphWalker.java:111)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.walk(TaskGraphWalker.java:180)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.startWalking(TaskGraphWalker.java:125)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.SparkDynamicPartitionPruningResolver.resolve(SparkDynamicPartitionPruningResolver.java:74)
> at
> org.apache.hadoop.hive.ql.parse.spark.SparkCompiler.optimizeTaskPlan(SparkCompiler.java:568)
> {noformat}
> At this stage, there shouldn't be a DPP sink whose target map work is null.
> The root cause seems to be a malformed operator tree generated by
> SplitOpTreeForDPP.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18148) NPE in SparkDynamicPartitionPruningResolver
[
https://issues.apache.org/jira/browse/HIVE-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-18148:
--
Attachment: HIVE-18148.1.patch
> NPE in SparkDynamicPartitionPruningResolver
> ---
>
> Key: HIVE-18148
> URL: https://issues.apache.org/jira/browse/HIVE-18148
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Reporter: Rui Li
>Assignee: Rui Li
> Attachments: HIVE-18148.1.patch
>
>
> The stack trace is:
> {noformat}
> 2017-11-27T10:32:38,752 ERROR [e6c8aab5-ddd2-461d-b185-a7597c3e7519 main]
> ql.Driver: FAILED: NullPointerException null
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.optimizer.physical.SparkDynamicPartitionPruningResolver$SparkDynamicPartitionPruningDispatcher.dispatch(SparkDynamicPartitionPruningResolver.java:100)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.dispatch(TaskGraphWalker.java:111)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.walk(TaskGraphWalker.java:180)
> at
> org.apache.hadoop.hive.ql.lib.TaskGraphWalker.startWalking(TaskGraphWalker.java:125)
> at
> org.apache.hadoop.hive.ql.optimizer.physical.SparkDynamicPartitionPruningResolver.resolve(SparkDynamicPartitionPruningResolver.java:74)
> at
> org.apache.hadoop.hive.ql.parse.spark.SparkCompiler.optimizeTaskPlan(SparkCompiler.java:568)
> {noformat}
> At this stage, there shouldn't be a DPP sink whose target map work is null.
> The root cause seems to be a malformed operator tree generated by
> SplitOpTreeForDPP.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18241) Query with LEFT SEMI JOIN producing wrong result
[
https://issues.apache.org/jira/browse/HIVE-18241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287145#comment-16287145
]
Vineet Garg commented on HIVE-18241:
[~jcamachorodriguez] Actually this test helped us finding this bug, it wasn't
suppose to produce any data but due to bug was producing data. So I think this
test still has value. I'll see if I can add another test which produces data.
> Query with LEFT SEMI JOIN producing wrong result
>
>
> Key: HIVE-18241
> URL: https://issues.apache.org/jira/browse/HIVE-18241
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
> Attachments: HIVE-18241.1.patch, HIVE-18241.2.patch
>
>
> Following query produces wrong result
> {code:sql}
> select key, value from src outr left semi join (select a.key, b.value from
> src a join (select distinct value from src) b on a.value > b.value group by
> a.key, b.value) inr on outr.key=inr.key and outr.value=inr.value;
> {code}
> Expected result is empty set but it output bunch of rows.
> Schema for {{src}} table could be find in {{data/scripts/q_test_init.sql}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-12719) As a hive user, I am facing issues using permanent UDAF's.
[ https://issues.apache.org/jira/browse/HIVE-12719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ganesha Shreedhara updated HIVE-12719: -- Release Note: Currently function name for the permanent function is getting formed as dbName.WINDOW_FUNC_PREFIX+functionName where as function registry has function name as WINDOW_FUNC_PREFIX+dbName.functionName. This is leading to invalid function error when we use permanent function with window function. Fix has been done such that we form the permanent function name rightly as WINDOW_FUNC_PREFIX+dbName+.+functionName so that it matches with the name in function registry. The functionality for built-in/temporary function would remain the same. (was: We will get invalid function error when permanent function is used along with window function because the permanent function is stored as db_name.function_name in the function registry where as built-in/temporary function gets stored without any prefix in the function registry. The fix is done such that the we will first form the qualified name for the permanent function and then compare if it exists in function registry. ) > As a hive user, I am facing issues using permanent UDAF's. > -- > > Key: HIVE-12719 > URL: https://issues.apache.org/jira/browse/HIVE-12719 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 1.2.1 >Reporter: Surbhit >Assignee: Ganesha Shreedhara > Attachments: HIVE-12719.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18241) Query with LEFT SEMI JOIN producing wrong result
[
https://issues.apache.org/jira/browse/HIVE-18241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287116#comment-16287116
]
Vineet Garg commented on HIVE-18241:
Thanks for your comments [~jcamachorodriguez]. Just to confirm
{{GroupType.SIMPLE}} should indicate that grouping sets is not present?
> Query with LEFT SEMI JOIN producing wrong result
>
>
> Key: HIVE-18241
> URL: https://issues.apache.org/jira/browse/HIVE-18241
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
> Attachments: HIVE-18241.1.patch, HIVE-18241.2.patch
>
>
> Following query produces wrong result
> {code:sql}
> select key, value from src outr left semi join (select a.key, b.value from
> src a join (select distinct value from src) b on a.value > b.value group by
> a.key, b.value) inr on outr.key=inr.key and outr.value=inr.value;
> {code}
> Expected result is empty set but it output bunch of rows.
> Schema for {{src}} table could be find in {{data/scripts/q_test_init.sql}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18258) Vectorization: Reduce-Side GROUP BY MERGEPARTIAL with duplicate columns is broken
[
https://issues.apache.org/jira/browse/HIVE-18258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287102#comment-16287102
]
Hive QA commented on HIVE-18258:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12901545/HIVE-18258.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 27 failed/errored test(s), 11527 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_reduce_groupby_duplicate_cols]
(batchId=35)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_groupby_grouping_id1]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_grouping_sets]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_windowing]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_12]
(batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_13]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_14]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_short_regress]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_dynamic_semijoin_reduction]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_ptf]
(batchId=165)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part]
(batchId=93)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_12]
(batchId=115)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_13]
(batchId=128)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_14]
(batchId=121)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=113)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_12]
(batchId=108)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_13]
(batchId=127)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_14]
(batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_short_regress]
(batchId=127)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorized_ptf]
(batchId=133)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=209)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=226)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8192/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8192/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8192/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 27 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12901545 - PreCommit-HIVE-Build
> Vectorization: Reduce-Side GROUP BY MERGEPARTIAL with duplicate columns is
> broken
> -
>
> Key: HIVE-18258
> URL: https://issues.apache.org/jira/browse/HIVE-18258
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HIVE-18258.01.patch, HIVE-18258.02.patch
>
>
> See Q file. Duplicate columns in key are not handled correctly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18112) show create for view having special char in where clause is not showing properly
[ https://issues.apache.org/jira/browse/HIVE-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287101#comment-16287101 ] Naresh P R commented on HIVE-18112: --- I verified those failing testcases locally. This failures are not related to the patch. [~sankarh], Can you please review and merge this patch into branch-2.2 ? > show create for view having special char in where clause is not showing > properly > > > Key: HIVE-18112 > URL: https://issues.apache.org/jira/browse/HIVE-18112 > Project: Hive > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Naresh P R >Assignee: Naresh P R >Priority: Minor > Fix For: 2.2.0 > > Attachments: HIVE-18112-branch-2.2.patch, > HIVE-18112.1-branch-2.2.patch > > > e.g., > CREATE VIEW `v2` AS select `evil_byte1`.`a` from `default`.`EVIL_BYTE1` where > `evil_byte1`.`a` = 'abcÖdefÖgh'; > Output: > == > 0: jdbc:hive2://172.26.122.227:1> show create table v2; > ++--+ > | createtab_stmt >| > ++--+ > | CREATE VIEW `v2` AS select `evil_byte1`.`a` from `default`.`EVIL_BYTE1` > where `evil_byte1`.`a` = 'abc�def�gh' | > ++--+ > Only show create output is having invalid characters, actual source table > content is displayed properly in the console. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18208) SMB Join : Fix the unit tests to run SMB Joins.
[ https://issues.apache.org/jira/browse/HIVE-18208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287072#comment-16287072 ] Deepak Jaiswal commented on HIVE-18208: --- Thanks for the review [~jdere]. The test failures in the run are independent of changes in the patch. > SMB Join : Fix the unit tests to run SMB Joins. > --- > > Key: HIVE-18208 > URL: https://issues.apache.org/jira/browse/HIVE-18208 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal > Attachments: HIVE-18208.1.patch, HIVE-18208.2.patch > > > Most of the SMB Join tests are actually not creating SMB Joins. Need them to > test the intended join. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18258) Vectorization: Reduce-Side GROUP BY MERGEPARTIAL with duplicate columns is broken
[
https://issues.apache.org/jira/browse/HIVE-18258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287071#comment-16287071
]
Hive QA commented on HIVE-18258:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
29s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
35s{color} | {color:red} ql: The patch generated 7 new + 8 unchanged - 4 fixed
= 15 total (was 12) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / d6ce23d |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8192/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8192/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Vectorization: Reduce-Side GROUP BY MERGEPARTIAL with duplicate columns is
> broken
> -
>
> Key: HIVE-18258
> URL: https://issues.apache.org/jira/browse/HIVE-18258
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HIVE-18258.01.patch, HIVE-18258.02.patch
>
>
> See Q file. Duplicate columns in key are not handled correctly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18251) Loosen restriction for some checks
[ https://issues.apache.org/jira/browse/HIVE-18251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287053#comment-16287053 ] Lefty Leverenz commented on HIVE-18251: --- Doc note: This changes the default value of *hive.strict.checks.cartesian.product* to false. It isn't documented in the wiki yet -- see HIVE-12727. Added a TODOC3.0 label. (Please add your own TODOC labels and doc notes in the future.) > Loosen restriction for some checks > -- > > Key: HIVE-18251 > URL: https://issues.apache.org/jira/browse/HIVE-18251 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Labels: TODOC3.0 > Fix For: 3.0.0 > > Attachments: HIVE-18251.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18054) Make Lineage work with concurrent queries on a Session
[
https://issues.apache.org/jira/browse/HIVE-18054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287048#comment-16287048
]
Hive QA commented on HIVE-18054:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12901525/HIVE-18054.13.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 11527 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part]
(batchId=93)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_aggregator_error_1]
(batchId=93)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=113)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=226)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8191/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8191/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8191/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12901525 - PreCommit-HIVE-Build
> Make Lineage work with concurrent queries on a Session
> ---
>
> Key: HIVE-18054
> URL: https://issues.apache.org/jira/browse/HIVE-18054
> Project: Hive
> Issue Type: Bug
>Reporter: Andrew Sherman
>Assignee: Andrew Sherman
> Attachments: HIVE-18054.1.patch, HIVE-18054.10.patch,
> HIVE-18054.11.patch, HIVE-18054.12.patch, HIVE-18054.13.patch,
> HIVE-18054.2.patch, HIVE-18054.3.patch, HIVE-18054.4.patch,
> HIVE-18054.5.patch, HIVE-18054.6.patch, HIVE-18054.7.patch,
> HIVE-18054.8.patch, HIVE-18054.9.patch
>
>
> A Hive Session can contain multiple concurrent sql Operations.
> Lineage is currently tracked in SessionState and is cleared when a query
> completes. This results in Lineage for other running queries being lost.
> To fix this, move LineageState from SessionState to QueryState.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18251) Loosen restriction for some checks
[ https://issues.apache.org/jira/browse/HIVE-18251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-18251: -- Labels: TODOC3.0 (was: ) > Loosen restriction for some checks > -- > > Key: HIVE-18251 > URL: https://issues.apache.org/jira/browse/HIVE-18251 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Labels: TODOC3.0 > Fix For: 3.0.0 > > Attachments: HIVE-18251.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-14498) Freshness period for query rewriting using materialized views
[
https://issues.apache.org/jira/browse/HIVE-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286312#comment-16286312
]
Eugene Koifman commented on HIVE-14498:
---
could
{noformat}
s = "select ctc_database, ctc_table, ctc_id, ctc_timestamp from
COMPLETED_TXN_COMPONENTS where ctc_txnid = " + txnid;
840 // Update registry with modifications
841 LOG.debug("Going to register table modification in invalidation
cache <" + s + ">");
842 rs = stmt.executeQuery(s);
843 if (rs.next()) {
844
MaterializationsInvalidationCache.get().notifyTableModification(
845 rs.getString(1), rs.getString(2), rs.getLong(3),
846 rs.getTimestamp(4,
Calendar.getInstance(TimeZone.getTimeZone("UTC"))).getTime());
847 }
{noformat}
be done after dbConn.commit();?
I would prefer to keep the db transaction as short as possible. The info
produced by this query should be "stable" since it has
{noformat}
where ctc_txnid = " + txnid;.
{noformat}
> Freshness period for query rewriting using materialized views
> -
>
> Key: HIVE-14498
> URL: https://issues.apache.org/jira/browse/HIVE-14498
> Project: Hive
> Issue Type: Sub-task
> Components: Materialized views
>Affects Versions: 2.2.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-14498.patch
>
>
> Once we have query rewriting in place (HIVE-14496), one of the main issues is
> data freshness in the materialized views.
> Since we will not support view maintenance at first, we could include a
> HiveConf property to configure a max freshness period (_n timeunits_). If a
> query comes, and the materialized view has been populated (by create,
> refresh, etc.) for a longer period than _n_, then we should not use it for
> rewriting the query.
> Optionally, we could print a warning for the user indicating that the
> materialized was not used because it was not fresh.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18196) Druid Mini Cluster to run Qtests integrations tests.
[ https://issues.apache.org/jira/browse/HIVE-18196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287082#comment-16287082 ] Lefty Leverenz commented on HIVE-18196: --- Doc note: This adds "derby" to the possible values for *hive.druid.metadata.db.type*, which was introduced in release 2.2.0 by HIVE-15277 and is not documented in the wiki yet. Added a TODOC3.0 label. (Please add your own TODOC labels and doc notes in the future.) > Druid Mini Cluster to run Qtests integrations tests. > > > Key: HIVE-18196 > URL: https://issues.apache.org/jira/browse/HIVE-18196 > Project: Hive > Issue Type: Bug >Reporter: slim bouguerra >Assignee: Ashutosh Chauhan > Labels: TODOC3.0 > Fix For: 3.0.0 > > Attachments: HIVE-18196.10.patch, HIVE-18196.11.patch, > HIVE-18196.12.patch, HIVE-18196.2.patch, HIVE-18196.3.patch, > HIVE-18196.4.patch, HIVE-18196.5.patch, HIVE-18196.6.patch, > HIVE-18196.7.patch, HIVE-18196.8.patch, HIVE-18196.patch > > > The overall Goal of this is to add a new Module that can fork a druid cluster > to run integration testing as part of the Mini Clusters Qtest suite. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HIVE-15277) Teach Hive how to create/delete Druid segments
[
https://issues.apache.org/jira/browse/HIVE-15277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15753038#comment-15753038
]
Lefty Leverenz edited comment on HIVE-15277 at 12/12/17 4:14 AM:
-
The new table property should be documented here as well as in the Druid
Integration doc:
* [DDL -- Table Properties |
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-listTableProperties]
Also document the new configuration parameters:
* *hive.druid.indexer.segments.granularity*
* *hive.druid.indexer.partition.size.max*
* *hive.druid.indexer.memory.rownum.max*
* *hive.druid.basePersistDirectory*
* *hive.druid.storage.storageDirectory*
* *hive.druid.metadata.base*
* *hive.druid.metadata.db.type* (Edit: see HIVE-15809 for correct values)
(Edit 2: see HIVE-18196 for new value in 3.0.0)
* *hive.druid.metadata.username*
* *hive.druid.metadata.password*
* *hive.druid.metadata.uri*
* *hive.druid.working.directory*
At this point there are enough Druid configuration parameters for a separate
subsection in the Configuration Properties doc. (Also see HIVE-14217 and
HIVE-15273.)
* [Hive Configuration Properties |
https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-HiveConfigurationProperties]
Added a TODOC2.2 label.
was (Author: le...@hortonworks.com):
The new table property should be documented here as well as in the Druid
Integration doc:
* [DDL -- Table Properties |
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-listTableProperties]
Also document the new configuration parameters:
* *hive.druid.indexer.segments.granularity*
* *hive.druid.indexer.partition.size.max*
* *hive.druid.indexer.memory.rownum.max*
* *hive.druid.basePersistDirectory*
* *hive.druid.storage.storageDirectory*
* *hive.druid.metadata.base*
* *hive.druid.metadata.db.type* (Edit: see HIVE-15809 for correct values)
* *hive.druid.metadata.username*
* *hive.druid.metadata.password*
* *hive.druid.metadata.uri*
* *hive.druid.working.directory*
At this point there are enough Druid configuration parameters for a separate
subsection in the Configuration Properties doc. (Also see HIVE-14217 and
HIVE-15273.)
* [Hive Configuration Properties |
https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-HiveConfigurationProperties]
Added a TODOC2.2 label.
> Teach Hive how to create/delete Druid segments
> ---
>
> Key: HIVE-15277
> URL: https://issues.apache.org/jira/browse/HIVE-15277
> Project: Hive
> Issue Type: Sub-task
> Components: Druid integration
>Affects Versions: 2.2.0
>Reporter: slim bouguerra
>Assignee: slim bouguerra
> Labels: TODOC2.2
> Fix For: 2.2.0
>
> Attachments: HIVE-15277.2.patch, HIVE-15277.patch, HIVE-15277.patch,
> HIVE-15277.patch, HIVE-15277.patch, HIVE-15277.patch, HIVE-15277.patch,
> HIVE-15277.patch, file.patch
>
>
> We want to extend the DruidStorageHandler to support CTAS queries.
> In this implementation Hive will generate druid segment files and insert the
> metadata to signal the handoff to druid.
> The syntax will be as follows:
> {code:sql}
> CREATE TABLE druid_table_1
> STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler'
> TBLPROPERTIES ("druid.datasource" = "datasourcename")
> AS `metric2`>;
> {code}
> This statement stores the results of query in a Druid
> datasource named 'datasourcename'. One of the columns of the query needs to
> be the time dimension, which is mandatory in Druid. In particular, we use the
> same convention that it is used for Druid: there needs to be a the column
> named '__time' in the result of the executed query, which will act as the
> time dimension column in Druid. Currently, the time column dimension needs to
> be a 'timestamp' type column.
> metrics can be of type long, double and float while dimensions are strings.
> Keep in mind that druid has a clear separation between dimensions and
> metrics, therefore if you have a column in hive that contains number and need
> to be presented as dimension use the cast operator to cast as string.
> This initial implementation interacts with Druid Meta data storage to
> add/remove the table in druid, user need to supply the meta data config as
> --hiveconf hive.druid.metadata.password=XXX --hiveconf
> hive.druid.metadata.username=druid --hiveconf
> hive.druid.metadata.uri=jdbc:mysql://host/druid
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18196) Druid Mini Cluster to run Qtests integrations tests.
[ https://issues.apache.org/jira/browse/HIVE-18196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-18196: -- Labels: TODOC3.0 (was: ) > Druid Mini Cluster to run Qtests integrations tests. > > > Key: HIVE-18196 > URL: https://issues.apache.org/jira/browse/HIVE-18196 > Project: Hive > Issue Type: Bug >Reporter: slim bouguerra >Assignee: Ashutosh Chauhan > Labels: TODOC3.0 > Fix For: 3.0.0 > > Attachments: HIVE-18196.10.patch, HIVE-18196.11.patch, > HIVE-18196.12.patch, HIVE-18196.2.patch, HIVE-18196.3.patch, > HIVE-18196.4.patch, HIVE-18196.5.patch, HIVE-18196.6.patch, > HIVE-18196.7.patch, HIVE-18196.8.patch, HIVE-18196.patch > > > The overall Goal of this is to add a new Module that can fork a druid cluster > to run integration testing as part of the Mini Clusters Qtest suite. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HIVE-12727) refactor Hive strict checks to be more granular, allow order by no limit and no partition filter by default for now
[ https://issues.apache.org/jira/browse/HIVE-12727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15124535#comment-15124535 ] Lefty Leverenz edited comment on HIVE-12727 at 12/12/17 3:44 AM: - Doc note: This deprecates *hive.mapred.mode* in 2.0.0, changing its default value back to nonstrict after HIVE-12413 changed it to strict in the same release, and adds three new configuration parameters to replace *hive.mapred.mode* (*hive.strict.checks.large.query* with default false, *hive.strict.checks.type.safety* with default true, and *hive.strict.checks.cartesian.product* with default true), so I added a TODOC2.0 label. The parameter changes should be documented in the wiki here: * [Configuration Properties -- hive.mapred.mode | https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.mapred.mode] Edit 11/Dec/17: See doc updates in comment 22/Nov/16. was (Author: le...@hortonworks.com): Doc note: This deprecates *hive.mapred.mode* in 2.0.0, changing its default value back to nonstrict after HIVE-12413 changed it to strict in the same release, and adds three new configuration parameters to replace *hive.mapred.mode* (*hive.strict.checks.large.query* with default false, *hive.strict.checks.type.safety* with default true, and *hive.strict.checks.cartesian.product* with default true), so I added a TODOC2.0 label. The parameter changes should be documented in the wiki here: * [Configuration Properties -- hive.mapred.mode | https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.mapred.mode] > refactor Hive strict checks to be more granular, allow order by no limit and > no partition filter by default for now > --- > > Key: HIVE-12727 > URL: https://issues.apache.org/jira/browse/HIVE-12727 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Blocker > Labels: TODOC2.0 > Fix For: 2.0.0 > > Attachments: HIVE-12727.01.patch, HIVE-12727.02.patch, > HIVE-12727.03.patch, HIVE-12727.04.patch, HIVE-12727.05.patch, > HIVE-12727.06.patch, HIVE-12727.07.patch, HIVE-12727.patch > > > Making strict mode the default recently appears to have broken many normal > queries, such as some TPCDS benchmark queries, e.g. Q85: > Response message: org.apache.hive.service.cli.HiveSQLException: Error while > compiling statement: FAILED: SemanticException [Error 10041]: No partition > predicate found for Alias "web_sales" Table "web_returns" > We should remove this restriction from strict mode, or change the default > back to non-strict. Perhaps make a 3-value parameter, nonstrict, semistrict, > and strict, for backward compat for people who are relying on strict already. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HIVE-12727) refactor Hive strict checks to be more granular, allow order by no limit and no partition filter by default for now
[ https://issues.apache.org/jira/browse/HIVE-12727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15686092#comment-15686092 ] Lefty Leverenz edited comment on HIVE-12727 at 12/12/17 3:41 AM: - HIVE-15148 changes the description of *hive.strict.checks.cartesian.product* in release 2.2.0. Edit 11/Dec/17: HIVE-18251 changes the default value of *hive.strict.checks.cartesian.product* to false in release 3.0.0. was (Author: le...@hortonworks.com): HIVE-15148 changes the description of *hive.strict.checks.cartesian.product* in release 2.2.0. > refactor Hive strict checks to be more granular, allow order by no limit and > no partition filter by default for now > --- > > Key: HIVE-12727 > URL: https://issues.apache.org/jira/browse/HIVE-12727 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Blocker > Labels: TODOC2.0 > Fix For: 2.0.0 > > Attachments: HIVE-12727.01.patch, HIVE-12727.02.patch, > HIVE-12727.03.patch, HIVE-12727.04.patch, HIVE-12727.05.patch, > HIVE-12727.06.patch, HIVE-12727.07.patch, HIVE-12727.patch > > > Making strict mode the default recently appears to have broken many normal > queries, such as some TPCDS benchmark queries, e.g. Q85: > Response message: org.apache.hive.service.cli.HiveSQLException: Error while > compiling statement: FAILED: SemanticException [Error 10041]: No partition > predicate found for Alias "web_sales" Table "web_returns" > We should remove this restriction from strict mode, or change the default > back to non-strict. Perhaps make a 3-value parameter, nonstrict, semistrict, > and strict, for backward compat for people who are relying on strict already. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18209) Fix API call in VectorizedListColumnReader to get value from BytesColumnVector
[ https://issues.apache.org/jira/browse/HIVE-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287032#comment-16287032 ] Colin Ma commented on HIVE-18209: - @ferdinand, ok, the previous yetus report is unavailable, I'll re-run the patch and fix the style issue. > Fix API call in VectorizedListColumnReader to get value from BytesColumnVector > -- > > Key: HIVE-18209 > URL: https://issues.apache.org/jira/browse/HIVE-18209 > Project: Hive > Issue Type: Sub-task >Reporter: Colin Ma >Assignee: Colin Ma > Attachments: HIVE-18209.001.patch, HIVE-18209.002.patch, > HIVE-18209.003.patch > > > With the API BytesColumnVector.setVal(), the isRepeating attribute can't be > set correctly if ListColumnVector.child is BytesColumnVector. > BytesColumnVector.setRef() should be used to avoid this problem. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18250) CBO gets turned off with duplicates in RR error
[
https://issues.apache.org/jira/browse/HIVE-18250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-18250:
---
Status: Patch Available (was: In Progress)
> CBO gets turned off with duplicates in RR error
> ---
>
> Key: HIVE-18250
> URL: https://issues.apache.org/jira/browse/HIVE-18250
> Project: Hive
> Issue Type: Bug
> Components: CBO, Query Planning
>Affects Versions: 2.3.0, 2.2.0, 2.1.0, 2.0.0
>Reporter: Ashutosh Chauhan
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-18250.patch
>
>
> {code}
> create table t1 (a int);
> explain select t1.a as a1, min(t1.a) as a from t1 group by t1.a;
> {code}
> CBO gets turned off with:
> {code}
> WARN [2e80e34e-dc46-49cf-88bf-2c24c0262d41 main] parse.RowResolver: Found
> duplicate column alias in RR: null.a => {null, a1, _col0: int} adding null.a
> => {null, null, _col1: int}
> 2017-12-07T15:27:47,651 ERROR [2e80e34e-dc46-49cf-88bf-2c24c0262d41 main]
> parse.CalcitePlanner: CBO failed, skipping CBO.
> org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSemanticException: Cannot
> add column to RR: null.a => _col1: int due to duplication, see previous
> warnings
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genSelectLogicalPlan(CalcitePlanner.java:3985)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:4313)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1392)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1322)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> {code}
> After that non-CBO path completes the query.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Work started] (HIVE-18250) CBO gets turned off with duplicates in RR error
[
https://issues.apache.org/jira/browse/HIVE-18250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-18250 started by Jesus Camacho Rodriguez.
--
> CBO gets turned off with duplicates in RR error
> ---
>
> Key: HIVE-18250
> URL: https://issues.apache.org/jira/browse/HIVE-18250
> Project: Hive
> Issue Type: Bug
> Components: CBO, Query Planning
>Affects Versions: 2.0.0, 2.1.0, 2.2.0, 2.3.0
>Reporter: Ashutosh Chauhan
>Assignee: Jesus Camacho Rodriguez
>
> {code}
> create table t1 (a int);
> explain select t1.a as a1, min(t1.a) as a from t1 group by t1.a;
> {code}
> CBO gets turned off with:
> {code}
> WARN [2e80e34e-dc46-49cf-88bf-2c24c0262d41 main] parse.RowResolver: Found
> duplicate column alias in RR: null.a => {null, a1, _col0: int} adding null.a
> => {null, null, _col1: int}
> 2017-12-07T15:27:47,651 ERROR [2e80e34e-dc46-49cf-88bf-2c24c0262d41 main]
> parse.CalcitePlanner: CBO failed, skipping CBO.
> org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSemanticException: Cannot
> add column to RR: null.a => _col1: int due to duplication, see previous
> warnings
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genSelectLogicalPlan(CalcitePlanner.java:3985)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:4313)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1392)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1322)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> {code}
> After that non-CBO path completes the query.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HIVE-18095) add a global flag to triggers (applies to all WM pools & container based sessions)
[ https://issues.apache.org/jira/browse/HIVE-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-18095: --- Assignee: Sergey Shelukhin > add a global flag to triggers (applies to all WM pools & container based > sessions) > -- > > Key: HIVE-18095 > URL: https://issues.apache.org/jira/browse/HIVE-18095 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > > cc [~prasanth_j] > It should be impossible to attach global triggers for pools. Setting global > flag should probably automatically remove attachments to pools. > Global triggers would only support actions that Tez supports (for simplicity; > also, for now, move doesn't make a lot of sense because the trigger would > apply again after the move). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18054) Make Lineage work with concurrent queries on a Session
[
https://issues.apache.org/jira/browse/HIVE-18054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287013#comment-16287013
]
Hive QA commented on HIVE-18054:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
51s{color} | {color:red} ql: The patch generated 4 new + 2369 unchanged - 15
fixed = 2373 total (was 2384) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / d6ce23d |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8191/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8191/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Make Lineage work with concurrent queries on a Session
> ---
>
> Key: HIVE-18054
> URL: https://issues.apache.org/jira/browse/HIVE-18054
> Project: Hive
> Issue Type: Bug
>Reporter: Andrew Sherman
>Assignee: Andrew Sherman
> Attachments: HIVE-18054.1.patch, HIVE-18054.10.patch,
> HIVE-18054.11.patch, HIVE-18054.12.patch, HIVE-18054.13.patch,
> HIVE-18054.2.patch, HIVE-18054.3.patch, HIVE-18054.4.patch,
> HIVE-18054.5.patch, HIVE-18054.6.patch, HIVE-18054.7.patch,
> HIVE-18054.8.patch, HIVE-18054.9.patch
>
>
> A Hive Session can contain multiple concurrent sql Operations.
> Lineage is currently tracked in SessionState and is cleared when a query
> completes. This results in Lineage for other running queries being lost.
> To fix this, move LineageState from SessionState to QueryState.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18250) CBO gets turned off with duplicates in RR error
[
https://issues.apache.org/jira/browse/HIVE-18250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-18250:
---
Attachment: (was: HIVE-18250.patch)
> CBO gets turned off with duplicates in RR error
> ---
>
> Key: HIVE-18250
> URL: https://issues.apache.org/jira/browse/HIVE-18250
> Project: Hive
> Issue Type: Bug
> Components: CBO, Query Planning
>Affects Versions: 2.0.0, 2.1.0, 2.2.0, 2.3.0
>Reporter: Ashutosh Chauhan
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-18250.01.patch
>
>
> {code}
> create table t1 (a int);
> explain select t1.a as a1, min(t1.a) as a from t1 group by t1.a;
> {code}
> CBO gets turned off with:
> {code}
> WARN [2e80e34e-dc46-49cf-88bf-2c24c0262d41 main] parse.RowResolver: Found
> duplicate column alias in RR: null.a => {null, a1, _col0: int} adding null.a
> => {null, null, _col1: int}
> 2017-12-07T15:27:47,651 ERROR [2e80e34e-dc46-49cf-88bf-2c24c0262d41 main]
> parse.CalcitePlanner: CBO failed, skipping CBO.
> org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSemanticException: Cannot
> add column to RR: null.a => _col1: int due to duplication, see previous
> warnings
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genSelectLogicalPlan(CalcitePlanner.java:3985)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:4313)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1392)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1322)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> {code}
> After that non-CBO path completes the query.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18228) Azure credential properties should be added to the HiveConf hidden list
[
https://issues.apache.org/jira/browse/HIVE-18228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286993#comment-16286993
]
Hive QA commented on HIVE-18228:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12901526/HIVE-18228.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 11525 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_into_dynamic_partitions]
(batchId=249)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=48)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part]
(batchId=93)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_aggregator_error_1]
(batchId=93)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=113)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=226)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8190/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8190/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8190/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12901526 - PreCommit-HIVE-Build
> Azure credential properties should be added to the HiveConf hidden list
> ---
>
> Key: HIVE-18228
> URL: https://issues.apache.org/jira/browse/HIVE-18228
> Project: Hive
> Issue Type: Bug
>Reporter: Andrew Sherman
>Assignee: Andrew Sherman
> Attachments: HIVE-18228.1.patch, HIVE-18228.2.patch,
> HIVE-18228.3.patch
>
>
> The HIVE_CONF_HIDDEN_LIST("hive.conf.hidden.list") already contains keys
> contaiing aws credentials. The Azure properties to be added are:
> * dfs.adls.oauth2.credential
> * fs.adl.oauth2.credential
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18241) Query with LEFT SEMI JOIN producing wrong result
[
https://issues.apache.org/jira/browse/HIVE-18241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286992#comment-16286992
]
Jesus Camacho Rodriguez commented on HIVE-18241:
[~vgarg], another quick comment before pushing it: could we add some data to
the new test or modify the test so that it produces data (even if it is a few
rows)? I think that will be important so we do not regress in the future.
> Query with LEFT SEMI JOIN producing wrong result
>
>
> Key: HIVE-18241
> URL: https://issues.apache.org/jira/browse/HIVE-18241
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
> Attachments: HIVE-18241.1.patch, HIVE-18241.2.patch
>
>
> Following query produces wrong result
> {code:sql}
> select key, value from src outr left semi join (select a.key, b.value from
> src a join (select distinct value from src) b on a.value > b.value group by
> a.key, b.value) inr on outr.key=inr.key and outr.value=inr.value;
> {code}
> Expected result is empty set but it output bunch of rows.
> Schema for {{src}} table could be find in {{data/scripts/q_test_init.sql}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18209) Fix API call in VectorizedListColumnReader to get value from BytesColumnVector
[ https://issues.apache.org/jira/browse/HIVE-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286985#comment-16286985 ] Ferdinand Xu commented on HIVE-18209: - hi [~colinma], can you help check the failed code style issue? Thank you! > Fix API call in VectorizedListColumnReader to get value from BytesColumnVector > -- > > Key: HIVE-18209 > URL: https://issues.apache.org/jira/browse/HIVE-18209 > Project: Hive > Issue Type: Sub-task >Reporter: Colin Ma >Assignee: Colin Ma > Attachments: HIVE-18209.001.patch, HIVE-18209.002.patch, > HIVE-18209.003.patch > > > With the API BytesColumnVector.setVal(), the isRepeating attribute can't be > set correctly if ListColumnVector.child is BytesColumnVector. > BytesColumnVector.setRef() should be used to avoid this problem. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18208) SMB Join : Fix the unit tests to run SMB Joins.
[ https://issues.apache.org/jira/browse/HIVE-18208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286986#comment-16286986 ] Jason Dere commented on HIVE-18208: --- I think the changes to the tests/plans look good. +1 > SMB Join : Fix the unit tests to run SMB Joins. > --- > > Key: HIVE-18208 > URL: https://issues.apache.org/jira/browse/HIVE-18208 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal > Attachments: HIVE-18208.1.patch, HIVE-18208.2.patch > > > Most of the SMB Join tests are actually not creating SMB Joins. Need them to > test the intended join. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18241) Query with LEFT SEMI JOIN producing wrong result
[
https://issues.apache.org/jira/browse/HIVE-18241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286990#comment-16286990
]
Jesus Camacho Rodriguez commented on HIVE-18241:
[~vgarg], could you check {{rightAggregate.indicator}} check by
{{rightAggregate.getType == GroupType.SIMPLE}}? {{indicator}} was deprecated
recently.
Once that is solved, looks good. +1
> Query with LEFT SEMI JOIN producing wrong result
>
>
> Key: HIVE-18241
> URL: https://issues.apache.org/jira/browse/HIVE-18241
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
> Attachments: HIVE-18241.1.patch, HIVE-18241.2.patch
>
>
> Following query produces wrong result
> {code:sql}
> select key, value from src outr left semi join (select a.key, b.value from
> src a join (select distinct value from src) b on a.value > b.value group by
> a.key, b.value) inr on outr.key=inr.key and outr.value=inr.value;
> {code}
> Expected result is empty set but it output bunch of rows.
> Schema for {{src}} table could be find in {{data/scripts/q_test_init.sql}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18250) CBO gets turned off with duplicates in RR error
[
https://issues.apache.org/jira/browse/HIVE-18250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-18250:
---
Attachment: HIVE-18250.01.patch
> CBO gets turned off with duplicates in RR error
> ---
>
> Key: HIVE-18250
> URL: https://issues.apache.org/jira/browse/HIVE-18250
> Project: Hive
> Issue Type: Bug
> Components: CBO, Query Planning
>Affects Versions: 2.0.0, 2.1.0, 2.2.0, 2.3.0
>Reporter: Ashutosh Chauhan
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-18250.01.patch
>
>
> {code}
> create table t1 (a int);
> explain select t1.a as a1, min(t1.a) as a from t1 group by t1.a;
> {code}
> CBO gets turned off with:
> {code}
> WARN [2e80e34e-dc46-49cf-88bf-2c24c0262d41 main] parse.RowResolver: Found
> duplicate column alias in RR: null.a => {null, a1, _col0: int} adding null.a
> => {null, null, _col1: int}
> 2017-12-07T15:27:47,651 ERROR [2e80e34e-dc46-49cf-88bf-2c24c0262d41 main]
> parse.CalcitePlanner: CBO failed, skipping CBO.
> org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSemanticException: Cannot
> add column to RR: null.a => _col1: int due to duplication, see previous
> warnings
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genSelectLogicalPlan(CalcitePlanner.java:3985)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:4313)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1392)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1322)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> {code}
> After that non-CBO path completes the query.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18241) Query with LEFT SEMI JOIN producing wrong result
[
https://issues.apache.org/jira/browse/HIVE-18241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286810#comment-16286810
]
Hive QA commented on HIVE-18241:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12901424/HIVE-18241.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 11516 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=48)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=113)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testApplyPlanQpChanges
(batchId=285)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=226)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8187/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8187/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8187/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12901424 - PreCommit-HIVE-Build
> Query with LEFT SEMI JOIN producing wrong result
>
>
> Key: HIVE-18241
> URL: https://issues.apache.org/jira/browse/HIVE-18241
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
> Attachments: HIVE-18241.1.patch, HIVE-18241.2.patch
>
>
> Following query produces wrong result
> {code:sql}
> select key, value from src outr left semi join (select a.key, b.value from
> src a join (select distinct value from src) b on a.value > b.value group by
> a.key, b.value) inr on outr.key=inr.key and outr.value=inr.value;
> {code}
> Expected result is empty set but it output bunch of rows.
> Schema for {{src}} table could be find in {{data/scripts/q_test_init.sql}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18250) CBO gets turned off with duplicates in RR error
[
https://issues.apache.org/jira/browse/HIVE-18250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-18250:
---
Attachment: HIVE-18250.patch
> CBO gets turned off with duplicates in RR error
> ---
>
> Key: HIVE-18250
> URL: https://issues.apache.org/jira/browse/HIVE-18250
> Project: Hive
> Issue Type: Bug
> Components: CBO, Query Planning
>Affects Versions: 2.0.0, 2.1.0, 2.2.0, 2.3.0
>Reporter: Ashutosh Chauhan
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-18250.patch
>
>
> {code}
> create table t1 (a int);
> explain select t1.a as a1, min(t1.a) as a from t1 group by t1.a;
> {code}
> CBO gets turned off with:
> {code}
> WARN [2e80e34e-dc46-49cf-88bf-2c24c0262d41 main] parse.RowResolver: Found
> duplicate column alias in RR: null.a => {null, a1, _col0: int} adding null.a
> => {null, null, _col1: int}
> 2017-12-07T15:27:47,651 ERROR [2e80e34e-dc46-49cf-88bf-2c24c0262d41 main]
> parse.CalcitePlanner: CBO failed, skipping CBO.
> org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSemanticException: Cannot
> add column to RR: null.a => _col1: int due to duplication, see previous
> warnings
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genSelectLogicalPlan(CalcitePlanner.java:3985)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:4313)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1392)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1322)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> {code}
> After that non-CBO path completes the query.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-13567) Enable auto-gather column stats by default
[ https://issues.apache.org/jira/browse/HIVE-13567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-13567: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Pushed to master. Thanks, Zoltan! > Enable auto-gather column stats by default > -- > > Key: HIVE-13567 > URL: https://issues.apache.org/jira/browse/HIVE-13567 > Project: Hive > Issue Type: Sub-task >Reporter: Pengcheng Xiong >Assignee: Zoltan Haindrich > Fix For: 3.0.0 > > Attachments: HIVE-13567.01.patch, HIVE-13567.02.patch, > HIVE-13567.03.patch, HIVE-13567.04.patch, HIVE-13567.05.patch, > HIVE-13567.06.patch, HIVE-13567.07.patch, HIVE-13567.08.patch, > HIVE-13567.09.patch, HIVE-13567.10.patch, HIVE-13567.11.patch, > HIVE-13567.12.patch, HIVE-13567.13.patch, HIVE-13567.14.patch, > HIVE-13567.15.patch, HIVE-13567.16.patch, HIVE-13567.17.patch, > HIVE-13567.18.patch, HIVE-13567.19.patch, HIVE-13567.20.patch, > HIVE-13567.21.patch, HIVE-13567.22.patch, HIVE-13567.23.patch, > HIVE-13567.23wip01.patch, HIVE-13567.23wip02.patch, HIVE-13567.23wip03.patch, > HIVE-13567.23wip04.patch, HIVE-13567.23wip05.patch, HIVE-13567.23wip06.patch, > HIVE-13567.23wip07.patch, HIVE-13567.23wip08.patch, HIVE-13567.23wip09.patch, > HIVE-13567.23wip10.patch, HIVE-13567.24.patch > > > in phase 2, we are going to set auto-gather column on as default. This needs > to update golden files. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18048) Support Struct type with vectorization for Parquet file
[ https://issues.apache.org/jira/browse/HIVE-18048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Ma updated HIVE-18048: Description: Struct type is not supported in MapWork with vectorization, it should be supported to improve the performance. (was: Struct type is supported in vectorization, but there is no qtests to test such case.) > Support Struct type with vectorization for Parquet file > --- > > Key: HIVE-18048 > URL: https://issues.apache.org/jira/browse/HIVE-18048 > Project: Hive > Issue Type: Sub-task >Reporter: Colin Ma >Assignee: Colin Ma > > Struct type is not supported in MapWork with vectorization, it should be > supported to improve the performance. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18133) Parametrize TestTxnNoBuckets wrt Vectorization
[
https://issues.apache.org/jira/browse/HIVE-18133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286808#comment-16286808
]
Prasanth Jayachandran commented on HIVE-18133:
--
+1
> Parametrize TestTxnNoBuckets wrt Vectorization
> --
>
> Key: HIVE-18133
> URL: https://issues.apache.org/jira/browse/HIVE-18133
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Minor
> Attachments: HIVE-18133.01.patch, HIVE-18133.02.patch,
> HIVE-18133.03.patch
>
>
> it currently runs in Vector mode only
> {noformat}
> public void setUp() throws Exception {
> setUpInternal();
> hiveConf.setBoolVar(HiveConf.ConfVars.HIVE_VECTORIZATION_ENABLED, true);
> }
> {noformat}
> would be good to run both modes
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18252) Limit the size of the object inspector caches
[ https://issues.apache.org/jira/browse/HIVE-18252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286980#comment-16286980 ] Ashutosh Chauhan commented on HIVE-18252: - I wonder if there is really an advantage of this cache. Caching OI seems to be of very little. Perhaps, we shall just delete the cache and factory. > Limit the size of the object inspector caches > - > > Key: HIVE-18252 > URL: https://issues.apache.org/jira/browse/HIVE-18252 > Project: Hive > Issue Type: Bug > Components: Types >Reporter: Jason Dere >Assignee: Jason Dere > Attachments: HIVE-18252.1.patch > > > Was running some tests that had a lot of queries with constant values, and > noticed that ObjectInspectorFactory.cachedStandardStructObjectInspector > started using up a lot of memory. > It appears that StructObjectInspector caching does not work properly with > constant values. Constant ObjectInspectors are not cached, so each constant > expression creates a new constant ObjectInspector. And since object > inspectors do not override equals(), object inspector comparison relies on > object instance comparison. So even if the values are exactly the same as > what is already in the cache, the StructObjectInspector cache lookup would > fail, and Hive would create a new object inspector and add it to the cache, > creating another entry that would never be used. Plus, there is no max cache > size - it's just a map that is allowed to grow as long as values keep getting > added to it. > Some possible solutions I can think of: > 1. Limit the size of the object inspector caches, rather than growing without > bound. > 2. Try to fix the caching to work with constant values. This would require > implementing equals() on the constant object inspectors (which could be slow > in nested cases), or else we would have to start caching constant object > inspectors, which could be expensive in terms of memory usage. Could be used > in combination with (1). By itself this is not a great solution because this > still has the unbounded cache growth issue. > 3. Disable caching in the case of constant object inspectors since this > scenario currently doesn't work. This could be used in combination with (1). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18203) change the way WM is enabled and allow dropping the last resource plan
[ https://issues.apache.org/jira/browse/HIVE-18203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18203: Attachment: HIVE-18203.02.patch Again... > change the way WM is enabled and allow dropping the last resource plan > -- > > Key: HIVE-18203 > URL: https://issues.apache.org/jira/browse/HIVE-18203 > Project: Hive > Issue Type: Sub-task >Reporter: Aswathy Chellammal Sreekumar >Assignee: Sergey Shelukhin > Attachments: HIVE-18203.01.patch, HIVE-18203.02.patch, > HIVE-18203.patch > > > Currently it's impossible to drop the last active resource plan even if WM is > disabled. It should be possible to deactivate the last resource plan AND > disable WM in the same action. Activating a resource plan should enable WM in > this case. > This should interact with the WM queue config in a sensible manner. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18240) support getClientInfo/setClientInfo in JDBC
[ https://issues.apache.org/jira/browse/HIVE-18240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18240: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Committed to master. Thanks for the reviews! > support getClientInfo/setClientInfo in JDBC > --- > > Key: HIVE-18240 > URL: https://issues.apache.org/jira/browse/HIVE-18240 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 3.0.0 > > Attachments: HIVE-18240.01.patch, HIVE-18240.patch > > > These are JDBC APIs that allow the user of the driver to provide client info > to the server; the list of the fields supported by the driver is returned as > a result set by getClientInfoProperties API. > I've looked at IBM, Oracle, MS etc. docs and it seems like ApplicationName is > a common one; there's also ClientHostname, etc. that we don't need because > HS2 derives them already. > The client will then set these properties via setClientInfo if desired. > Whether it is desired by any BI tools of significance I've no idea. > The properties are sent to the server on connect (which is what Microsoft > seems to do, but in Hive model it's impossible because HiveConnection > connects in ctor), or on the next query (I don't recall where I've seen > this), or immediately (which is what I do in this patch). > The getClientInfo API on the driver side seems completely pointless, so I > cache clientinfo locally for it. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18068) Upgrade to Calcite 1.15
[ https://issues.apache.org/jira/browse/HIVE-18068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286977#comment-16286977 ] Ashutosh Chauhan commented on HIVE-18068: - +1 > Upgrade to Calcite 1.15 > --- > > Key: HIVE-18068 > URL: https://issues.apache.org/jira/browse/HIVE-18068 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-18068.03.patch, HIVE-18068.04.patch, > HIVE-18068.05.patch, HIVE-18068.06.patch, HIVE-18068.2.patch, HIVE-18068.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18153) refactor reopen and file management in TezTask
[ https://issues.apache.org/jira/browse/HIVE-18153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18153: Attachment: HIVE-18153.04.patch Fixed the tests. > refactor reopen and file management in TezTask > -- > > Key: HIVE-18153 > URL: https://issues.apache.org/jira/browse/HIVE-18153 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-18153.01.patch, HIVE-18153.02.patch, > HIVE-18153.03.patch, HIVE-18153.04.patch, HIVE-18153.patch > > > TezTask reopen relies on getting the same session object in terms of setup; > WM reopen returns a new session from the pool. > The former has the advantage of not having to reupload files and stuff... but > the object reuse results in a lot of ugly code, and also reopen might be > slower on average with the session pool than just getting a session from the > pool. Either WM needs to do the object-preserving reopen, or TezTask needs to > be refactored. It looks like DAG would have to be rebuilt to do the latter > because of some paths tied to a directory of the old session. Let me see if I > can get around that; if not we can do the former; and then if the former > results in too much ugly code in WM to account for object reuse for different > Tez client I'd do the latter anyway since it's a failure path :) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18003) add explicit jdbc connection string args for mappings
[ https://issues.apache.org/jira/browse/HIVE-18003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18003: Attachment: HIVE-18003.03.patch Again... > add explicit jdbc connection string args for mappings > - > > Key: HIVE-18003 > URL: https://issues.apache.org/jira/browse/HIVE-18003 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-18003.01.patch, HIVE-18003.02.patch, > HIVE-18003.03.patch, HIVE-18003.patch > > > 1) Force using unmanaged/containers execution. > 2) Optional - specify pool name (config setting to gate this, disabled by > default?). > In phase 2 (or 4?) we might allow #2 to be used by a user to choose between > multiple mappings if they have multiple pools they could be mapped to (i.e. > to change the ordering essentially). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18078) WM getSession needs some retry logic
[ https://issues.apache.org/jira/browse/HIVE-18078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18078: Attachment: HIVE-18078.02.patch A combined patch again, after rebasing. > WM getSession needs some retry logic > > > Key: HIVE-18078 > URL: https://issues.apache.org/jira/browse/HIVE-18078 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-18078.01.patch, HIVE-18078.01.patch, > HIVE-18078.02.patch, HIVE-18078.only.patch, HIVE-18078.patch > > > When we get a bad session (e.g. no registry info because AM has gone > catatonic), the failure by the timeout future fails the getSession call. > The retry model in TezTask is that it would get a session (which in original > model can be completely unusable, but we still get the object), and then > retry (reopen) if it's a lemon. If the reopen fails, we fail. > getSession is not covered by this retry scheme, and should thus do its own > retries (or the retry logic needs to be changed) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18075) verify commands on a cluster
[ https://issues.apache.org/jira/browse/HIVE-18075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18075: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Committed to master. Thanks for the patch! > verify commands on a cluster > > > Key: HIVE-18075 > URL: https://issues.apache.org/jira/browse/HIVE-18075 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Harish Jaiprakash > Fix For: 3.0.0 > > Attachments: HIVE-18075.01.patch > > > I was running the commands in the cluster, with potentially a slightly > outdated version of the DB on mysql (yet, the DB was created with a script > from master and I didn't hit any schema issues, so it's fairly recent); > however, recent master code + the pools commands patch. I've hit the > following issues. > # Cannot drop pool or RP with a mapping (see also 3). > # Cannot drop pool that is set as default (probably correct, but the error > message is bad). > # When I dropped an RP with a mapping, and then created it again with the > same name, the pool creation in that RP would fail with an error that a > unique query returned multiple results. In the DB, there were actually 2 RPs > with the same name. Not sure how exactly that happened, there might have been > intermediate states, but I didn't mess with mysql. I think the name > uniqueness is either missing from some script or doesn't work. > # -Setting RP default pool no longer works. I think I might have broken it > with one of the rebases in that area, but it could also be something else (or > like other things, it works in q tests but not on cluster for whatever > reason-). > # -Resource plan rename doesn't check the disable state. It probably should. > Also need to see for other commands-. > Need to figure out which are real issues and which aren't and fix. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18228) Azure credential properties should be added to the HiveConf hidden list
[
https://issues.apache.org/jira/browse/HIVE-18228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286939#comment-16286939
]
Hive QA commented on HIVE-18228:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 9m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / b38544f |
| Default Java | 1.8.0_111 |
| modules | C: common U: common |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8190/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Azure credential properties should be added to the HiveConf hidden list
> ---
>
> Key: HIVE-18228
> URL: https://issues.apache.org/jira/browse/HIVE-18228
> Project: Hive
> Issue Type: Bug
>Reporter: Andrew Sherman
>Assignee: Andrew Sherman
> Attachments: HIVE-18228.1.patch, HIVE-18228.2.patch,
> HIVE-18228.3.patch
>
>
> The HIVE_CONF_HIDDEN_LIST("hive.conf.hidden.list") already contains keys
> contaiing aws credentials. The Azure properties to be added are:
> * dfs.adls.oauth2.credential
> * fs.adl.oauth2.credential
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18124) clean up isAcidTable() API vs isInsertOnlyTable()
[
https://issues.apache.org/jira/browse/HIVE-18124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286920#comment-16286920
]
Hive QA commented on HIVE-18124:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12901515/HIVE-18124.04.patch
{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 16 failed/errored test(s), 11516 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join25] (batchId=72)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udaf_context_ngrams]
(batchId=73)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mergejoin]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part]
(batchId=93)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[delete_sorted]
(batchId=93)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[insert_sorted]
(batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[insert_values_sorted]
(batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[update_sorted]
(batchId=93)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=113)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=226)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8189/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8189/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8189/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 16 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12901515 - PreCommit-HIVE-Build
> clean up isAcidTable() API vs isInsertOnlyTable()
> --
>
> Key: HIVE-18124
> URL: https://issues.apache.org/jira/browse/HIVE-18124
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
> Attachments: HIVE-18124.01.patch, HIVE-18124.02.patch,
> HIVE-18124.03.patch, HIVE-18124.04.patch
>
>
> With the addition of MM tables (_AcidUtils.isInsertOnlyTable(table)_) the
> methods in AcidUtils and dependent places are very muddled.
> Need to clean it up so that there is a isTransactional(Table) that checks
> transactional=true setting and isAcid(Table) to mean full ACID and
> isInsertOnly(Table) to mean MM tables.
> This would accurately describe the semantics of the tables.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17794) HCatLoader breaks when a member is added to a struct-column of a table
[
https://issues.apache.org/jira/browse/HIVE-17794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mithun Radhakrishnan updated HIVE-17794:
Attachment: HIVE-17794.2-branch-2.patch
The tests for {{branch-2}} took some doing. Submitting for tests.
> HCatLoader breaks when a member is added to a struct-column of a table
> --
>
> Key: HIVE-17794
> URL: https://issues.apache.org/jira/browse/HIVE-17794
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 2.2.0, 3.0.0
>Reporter: Mithun Radhakrishnan
>Assignee: Mithun Radhakrishnan
> Attachments: HIVE-17794.1.patch, HIVE-17794.2-branch-2.patch,
> HIVE-17794.2.patch
>
>
> When a table's schema evolves to add a new member to a struct column, Hive
> queries work fine, but {{HCatLoader}} breaks with the following trace:
> {noformat}
> TaskAttempt 1 failed, info=
> Error: Failure while running
> task:org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: kite_composites_with_segments: Local Rearrange
> tuple
> {chararray}(false) - scope-555-> scope-974 Operator Key: scope-555):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: gup: New For Each(false,false)
> bag
> - scope-548 Operator Key: scope-548):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: gup_filtered: Filter
> bag
> - scope-522 Operator Key: scope-522):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
> converting read value to tuple
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:314)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNextTuple(POLocalRearrange.java:287)
> at
> org.apache.pig.backend.hadoop.executionengine.tez.plan.operator.POLocalRearrangeTez.getNextTuple(POLocalRearrangeTez.java:127)
> at
> org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor.runPipeline(PigProcessor.java:376)
> at
> org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor.run(PigProcessor.java:241)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:362)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1679)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> Exception while executing (Name: gup: New For Each(false,false)
> bag
> - scope-548 Operator Key: scope-548):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: gup_filtered: Filter
> bag
> - scope-522 Operator Key: scope-522):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
> converting read value to tuple
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:314)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:252)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:305)
> ... 17 more
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> Exception while executing (Name: gup_filtered: Filter
> bag
> - scope-522 Operator Key: scope-522):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
> converting read value to tuple
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:314)
> at
>
[jira] [Updated] (HIVE-17794) HCatLoader breaks when a member is added to a struct-column of a table
[
https://issues.apache.org/jira/browse/HIVE-17794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mithun Radhakrishnan updated HIVE-17794:
Status: Open (was: Patch Available)
Alas, I spoke too soon. Cancelling patch for {{branch-2}} to resolve conflicts
with {{TestExtendedAcls}}.
> HCatLoader breaks when a member is added to a struct-column of a table
> --
>
> Key: HIVE-17794
> URL: https://issues.apache.org/jira/browse/HIVE-17794
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 2.2.0, 3.0.0
>Reporter: Mithun Radhakrishnan
>Assignee: Mithun Radhakrishnan
> Attachments: HIVE-17794.1.patch, HIVE-17794.2-branch-2.patch,
> HIVE-17794.2.patch
>
>
> When a table's schema evolves to add a new member to a struct column, Hive
> queries work fine, but {{HCatLoader}} breaks with the following trace:
> {noformat}
> TaskAttempt 1 failed, info=
> Error: Failure while running
> task:org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: kite_composites_with_segments: Local Rearrange
> tuple
> {chararray}(false) - scope-555-> scope-974 Operator Key: scope-555):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: gup: New For Each(false,false)
> bag
> - scope-548 Operator Key: scope-548):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: gup_filtered: Filter
> bag
> - scope-522 Operator Key: scope-522):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
> converting read value to tuple
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:314)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNextTuple(POLocalRearrange.java:287)
> at
> org.apache.pig.backend.hadoop.executionengine.tez.plan.operator.POLocalRearrangeTez.getNextTuple(POLocalRearrangeTez.java:127)
> at
> org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor.runPipeline(PigProcessor.java:376)
> at
> org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor.run(PigProcessor.java:241)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:362)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1679)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> Exception while executing (Name: gup: New For Each(false,false)
> bag
> - scope-548 Operator Key: scope-548):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: gup_filtered: Filter
> bag
> - scope-522 Operator Key: scope-522):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
> converting read value to tuple
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:314)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:252)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:305)
> ... 17 more
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> Exception while executing (Name: gup_filtered: Filter
> bag
> - scope-522 Operator Key: scope-522):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
> converting read value to tuple
> at
>
[jira] [Commented] (HIVE-18124) clean up isAcidTable() API vs isInsertOnlyTable()
[
https://issues.apache.org/jira/browse/HIVE-18124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286862#comment-16286862
]
Hive QA commented on HIVE-18124:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
1s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
10s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} The patch common passed checkstyle {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} standalone-metastore: The patch generated 4 new + 63
unchanged - 1 fixed = 67 total (was 64) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
58s{color} | {color:red} ql: The patch generated 14 new + 3231 unchanged - 19
fixed = 3245 total (was 3250) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch llap-server passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} hcatalog/core: The patch generated 0 new + 40
unchanged - 1 fixed = 40 total (was 41) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} The patch streaming passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch hive-unit passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 1m
8s{color} | {color:red} standalone-metastore generated 5 new + 54 unchanged - 0
fixed = 59 total (was 54) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 3bbc24d |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8189/yetus/diff-checkstyle-standalone-metastore.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8189/yetus/diff-checkstyle-ql.txt
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8189/yetus/diff-javadoc-javadoc-standalone-metastore.txt
|
| modules | C: common standalone-metastore ql llap-server hcatalog/core
hcatalog/streaming itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8189/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> clean up isAcidTable() API vs
[jira] [Updated] (HIVE-17794) HCatLoader breaks when a member is added to a struct-column of a table
[
https://issues.apache.org/jira/browse/HIVE-17794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mithun Radhakrishnan updated HIVE-17794:
Attachment: HIVE-17794.2-branch-2.2.patch
> HCatLoader breaks when a member is added to a struct-column of a table
> --
>
> Key: HIVE-17794
> URL: https://issues.apache.org/jira/browse/HIVE-17794
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 2.2.0, 3.0.0
>Reporter: Mithun Radhakrishnan
>Assignee: Mithun Radhakrishnan
> Attachments: HIVE-17794.1.patch, HIVE-17794.2-branch-2.2.patch,
> HIVE-17794.2-branch-2.patch, HIVE-17794.2.patch
>
>
> When a table's schema evolves to add a new member to a struct column, Hive
> queries work fine, but {{HCatLoader}} breaks with the following trace:
> {noformat}
> TaskAttempt 1 failed, info=
> Error: Failure while running
> task:org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: kite_composites_with_segments: Local Rearrange
> tuple
> {chararray}(false) - scope-555-> scope-974 Operator Key: scope-555):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: gup: New For Each(false,false)
> bag
> - scope-548 Operator Key: scope-548):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: gup_filtered: Filter
> bag
> - scope-522 Operator Key: scope-522):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
> converting read value to tuple
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:314)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNextTuple(POLocalRearrange.java:287)
> at
> org.apache.pig.backend.hadoop.executionengine.tez.plan.operator.POLocalRearrangeTez.getNextTuple(POLocalRearrangeTez.java:127)
> at
> org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor.runPipeline(PigProcessor.java:376)
> at
> org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor.run(PigProcessor.java:241)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:362)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1679)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> Exception while executing (Name: gup: New For Each(false,false)
> bag
> - scope-548 Operator Key: scope-548):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: gup_filtered: Filter
> bag
> - scope-522 Operator Key: scope-522):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
> converting read value to tuple
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:314)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:252)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:305)
> ... 17 more
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> Exception while executing (Name: gup_filtered: Filter
> bag
> - scope-522 Operator Key: scope-522):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
> converting read value to tuple
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:314)
> at
>
[jira] [Updated] (HIVE-17794) HCatLoader breaks when a member is added to a struct-column of a table
[
https://issues.apache.org/jira/browse/HIVE-17794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mithun Radhakrishnan updated HIVE-17794:
Status: Patch Available (was: Open)
> HCatLoader breaks when a member is added to a struct-column of a table
> --
>
> Key: HIVE-17794
> URL: https://issues.apache.org/jira/browse/HIVE-17794
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 2.2.0, 3.0.0
>Reporter: Mithun Radhakrishnan
>Assignee: Mithun Radhakrishnan
> Attachments: HIVE-17794.1.patch, HIVE-17794.2-branch-2.patch,
> HIVE-17794.2.patch
>
>
> When a table's schema evolves to add a new member to a struct column, Hive
> queries work fine, but {{HCatLoader}} breaks with the following trace:
> {noformat}
> TaskAttempt 1 failed, info=
> Error: Failure while running
> task:org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: kite_composites_with_segments: Local Rearrange
> tuple
> {chararray}(false) - scope-555-> scope-974 Operator Key: scope-555):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: gup: New For Each(false,false)
> bag
> - scope-548 Operator Key: scope-548):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: gup_filtered: Filter
> bag
> - scope-522 Operator Key: scope-522):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
> converting read value to tuple
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:314)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNextTuple(POLocalRearrange.java:287)
> at
> org.apache.pig.backend.hadoop.executionengine.tez.plan.operator.POLocalRearrangeTez.getNextTuple(POLocalRearrangeTez.java:127)
> at
> org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor.runPipeline(PigProcessor.java:376)
> at
> org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor.run(PigProcessor.java:241)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:362)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1679)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> Exception while executing (Name: gup: New For Each(false,false)
> bag
> - scope-548 Operator Key: scope-548):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: gup_filtered: Filter
> bag
> - scope-522 Operator Key: scope-522):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
> converting read value to tuple
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:314)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:252)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:305)
> ... 17 more
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> Exception while executing (Name: gup_filtered: Filter
> bag
> - scope-522 Operator Key: scope-522):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
> converting read value to tuple
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:314)
> at
>
[jira] [Updated] (HIVE-17794) HCatLoader breaks when a member is added to a struct-column of a table
[
https://issues.apache.org/jira/browse/HIVE-17794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mithun Radhakrishnan updated HIVE-17794:
Attachment: HIVE-17794.2-branch-2.patch
Re-resubmitting for tests, for {{branch-2}}. I'll start with {{branch-2.2}}
next.
> HCatLoader breaks when a member is added to a struct-column of a table
> --
>
> Key: HIVE-17794
> URL: https://issues.apache.org/jira/browse/HIVE-17794
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 2.2.0, 3.0.0
>Reporter: Mithun Radhakrishnan
>Assignee: Mithun Radhakrishnan
> Attachments: HIVE-17794.1.patch, HIVE-17794.2-branch-2.patch,
> HIVE-17794.2.patch
>
>
> When a table's schema evolves to add a new member to a struct column, Hive
> queries work fine, but {{HCatLoader}} breaks with the following trace:
> {noformat}
> TaskAttempt 1 failed, info=
> Error: Failure while running
> task:org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: kite_composites_with_segments: Local Rearrange
> tuple
> {chararray}(false) - scope-555-> scope-974 Operator Key: scope-555):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: gup: New For Each(false,false)
> bag
> - scope-548 Operator Key: scope-548):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: gup_filtered: Filter
> bag
> - scope-522 Operator Key: scope-522):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
> converting read value to tuple
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:314)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNextTuple(POLocalRearrange.java:287)
> at
> org.apache.pig.backend.hadoop.executionengine.tez.plan.operator.POLocalRearrangeTez.getNextTuple(POLocalRearrangeTez.java:127)
> at
> org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor.runPipeline(PigProcessor.java:376)
> at
> org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor.run(PigProcessor.java:241)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:362)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1679)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> Exception while executing (Name: gup: New For Each(false,false)
> bag
> - scope-548 Operator Key: scope-548):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: gup_filtered: Filter
> bag
> - scope-522 Operator Key: scope-522):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
> converting read value to tuple
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:314)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:252)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:305)
> ... 17 more
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> Exception while executing (Name: gup_filtered: Filter
> bag
> - scope-522 Operator Key: scope-522):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
> converting read value to tuple
> at
>
[jira] [Updated] (HIVE-17981) Create a set of builders for Thrift classes
[
https://issues.apache.org/jira/browse/HIVE-17981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates updated HIVE-17981:
--
Attachment: HIVE-17981.copy.patch
Attaching a copy of the original patch to get ptest to pick it up.
> Create a set of builders for Thrift classes
> ---
>
> Key: HIVE-17981
> URL: https://issues.apache.org/jira/browse/HIVE-17981
> Project: Hive
> Issue Type: Sub-task
> Components: Standalone Metastore
>Reporter: Alan Gates
>Assignee: Alan Gates
> Labels: pull-request-available
> Attachments: HIVE-17981.copy.patch, HIVE-17981.patch
>
>
> Instantiating some of the Thrift classes is painful. Consider building a
> {{Table}} object, which requires a {{StorageDescriptor}}, which requires a
> {{SerDeInfo}} and a list of {{FieldInfo}}. All that is really necessary for
> a Table in the most simple case is a name, a database, and some columns. But
> currently creating even a simple Table requires 20+ lines of code. This is
> particularly painful in tests.
> I propose to add a set of builders. These will come with reasonable defaults
> to minimize the boilerplate code. They will also include simple methods for
> common operations (like adding columns, or a parameter) without requiring the
> user to create all the sub-objects (like {{StorageDescriptor}}).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17794) HCatLoader breaks when a member is added to a struct-column of a table
[
https://issues.apache.org/jira/browse/HIVE-17794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mithun Radhakrishnan updated HIVE-17794:
Attachment: (was: HIVE-17794.2-branch-2.patch)
> HCatLoader breaks when a member is added to a struct-column of a table
> --
>
> Key: HIVE-17794
> URL: https://issues.apache.org/jira/browse/HIVE-17794
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 2.2.0, 3.0.0
>Reporter: Mithun Radhakrishnan
>Assignee: Mithun Radhakrishnan
> Attachments: HIVE-17794.1.patch, HIVE-17794.2.patch
>
>
> When a table's schema evolves to add a new member to a struct column, Hive
> queries work fine, but {{HCatLoader}} breaks with the following trace:
> {noformat}
> TaskAttempt 1 failed, info=
> Error: Failure while running
> task:org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: kite_composites_with_segments: Local Rearrange
> tuple
> {chararray}(false) - scope-555-> scope-974 Operator Key: scope-555):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: gup: New For Each(false,false)
> bag
> - scope-548 Operator Key: scope-548):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: gup_filtered: Filter
> bag
> - scope-522 Operator Key: scope-522):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
> converting read value to tuple
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:314)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNextTuple(POLocalRearrange.java:287)
> at
> org.apache.pig.backend.hadoop.executionengine.tez.plan.operator.POLocalRearrangeTez.getNextTuple(POLocalRearrangeTez.java:127)
> at
> org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor.runPipeline(PigProcessor.java:376)
> at
> org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor.run(PigProcessor.java:241)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:362)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1679)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> Exception while executing (Name: gup: New For Each(false,false)
> bag
> - scope-548 Operator Key: scope-548):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: gup_filtered: Filter
> bag
> - scope-522 Operator Key: scope-522):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
> converting read value to tuple
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:314)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:252)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:305)
> ... 17 more
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> Exception while executing (Name: gup_filtered: Filter
> bag
> - scope-522 Operator Key: scope-522):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
> converting read value to tuple
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:314)
> at
>
[jira] [Resolved] (HIVE-18133) Parametrize TestTxnNoBuckets wrt Vectorization
[
https://issues.apache.org/jira/browse/HIVE-18133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman resolved HIVE-18133.
---
Resolution: Fixed
Fix Version/s: 3.0.0
committed to master
thanks Prasanth for the review
> Parametrize TestTxnNoBuckets wrt Vectorization
> --
>
> Key: HIVE-18133
> URL: https://issues.apache.org/jira/browse/HIVE-18133
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HIVE-18133.01.patch, HIVE-18133.02.patch,
> HIVE-18133.03.patch
>
>
> it currently runs in Vector mode only
> {noformat}
> public void setUp() throws Exception {
> setUpInternal();
> hiveConf.setBoolVar(HiveConf.ConfVars.HIVE_VECTORIZATION_ENABLED, true);
> }
> {noformat}
> would be good to run both modes
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-12719) As a hive user, I am facing issues using permanent UDAF's.
[
https://issues.apache.org/jira/browse/HIVE-12719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286812#comment-16286812
]
Hive QA commented on HIVE-12719:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12901512/HIVE-12719.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8188/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8188/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8188/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-12-12 00:02:21.199
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-8188/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-12-12 00:02:21.213
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
31c1b71..3bbc24d master -> origin/master
+ git reset --hard HEAD
HEAD is now at 31c1b71 HIVE-18245 - clean up acid_vectorization_original.q
(Eugene Koifman, reviewed by Jason Dere)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
(use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at 3bbc24d HIVE-13567 : Enable auto-gather column stats by default
(Zoltan Haindrich, Pengcheng Xiong via Ashutosh Chauhan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-12-12 00:02:27.316
+ rm -rf ../yetus
+ mkdir ../yetus
+ cp -R . ../yetus
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-8188/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Going to apply patch with: git apply -p0
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
protoc-jar: protoc version: 250, detected platform: linux/amd64
protoc-jar: executing: [/tmp/protoc2924753280834078335.exe,
-I/data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/protobuf/org/apache/hadoop/hive/metastore,
--java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/target/generated-sources,
/data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto]
ANTLR Parser Generator Version 3.5.2
Output file
/data/hiveptest/working/apache-github-source-source/standalone-metastore/target/generated-sources/org/apache/hadoop/hive/metastore/parser/FilterParser.java
does not exist: must build
/data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/parser/Filter.g
org/apache/hadoop/hive/metastore/parser/Filter.g
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process
(process-resource-bundles) on project hive-hcatalog: Failed to resolve
dependencies for one or more projects in the reactor. Reason: No versions are
present in the repository for the artifact with a range [1.3.1,2.3]
[ERROR] net.minidev:json-smart:jar:null
[ERROR]
[ERROR] from the specified remote repositories:
[ERROR] datanucleus (http://www.datanucleus.org/downloads/maven2,
releases=true, snapshots=false),
[ERROR] glassfish-repository
(http://maven.glassfish.org/content/groups/glassfish, releases=false,
snapshots=false),
[ERROR] glassfish-repo-archive
(http://maven.glassfish.org/content/groups/glassfish, releases=false,
snapshots=false),
[ERROR] sonatype-snapshot
[jira] [Commented] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286811#comment-16286811 ] Xuefu Zhang commented on HIVE-17486: I meant, if FIL[52] and FIL[53] is the same in your example, then we should break after the filter op for M-M split. Looking forward to your complete design doc for this. Thanks. > Enable SharedWorkOptimizer in tez on HOS > > > Key: HIVE-17486 > URL: https://issues.apache.org/jira/browse/HIVE-17486 > Project: Hive > Issue Type: Bug >Reporter: liyunzhang >Assignee: liyunzhang > Attachments: HIVE-17486.1.patch, explain.28.share.false, > explain.28.share.true, scanshare.after.svg, scanshare.before.svg > > > in HIVE-16602, Implement shared scans with Tez. > Given a query plan, the goal is to identify scans on input tables that can be > merged so the data is read only once. Optimization will be carried out at the > physical level. In Hive on Spark, it caches the result of spark work if the > spark work is used by more than 1 child spark work. After sharedWorkOptimizer > is enabled in physical plan in HoS, the identical table scans are merged to 1 > table scan. This result of table scan will be used by more 1 child spark > work. Thus we need not do the same computation because of cache mechanism. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18241) Query with LEFT SEMI JOIN producing wrong result
[
https://issues.apache.org/jira/browse/HIVE-18241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286714#comment-16286714
]
Hive QA commented on HIVE-18241:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
31s{color} | {color:green} ql: The patch generated 0 new + 1 unchanged - 30
fixed = 1 total (was 31) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 31c1b71 |
| Default Java | 1.8.0_111 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8187/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Query with LEFT SEMI JOIN producing wrong result
>
>
> Key: HIVE-18241
> URL: https://issues.apache.org/jira/browse/HIVE-18241
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
> Attachments: HIVE-18241.1.patch, HIVE-18241.2.patch
>
>
> Following query produces wrong result
> {code:sql}
> select key, value from src outr left semi join (select a.key, b.value from
> src a join (select distinct value from src) b on a.value > b.value group by
> a.key, b.value) inr on outr.key=inr.key and outr.value=inr.value;
> {code}
> Expected result is empty set but it output bunch of rows.
> Schema for {{src}} table could be find in {{data/scripts/q_test_init.sql}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18227) Tez parallel execution fail
[
https://issues.apache.org/jira/browse/HIVE-18227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated HIVE-18227:
--
Attachment: HIVE-18227.2.patch
Since the execution is sequential in nature, we can simply make TezTask
nonparallel. Thanks [~anishek] for suggestion and attach a new patch.
> Tez parallel execution fail
> ---
>
> Key: HIVE-18227
> URL: https://issues.apache.org/jira/browse/HIVE-18227
> Project: Hive
> Issue Type: Bug
> Components: Tez
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Attachments: HIVE-18227.1.patch, HIVE-18227.2.patch
>
>
> Running tez Dag in parallel within a session fail. Here is the test case:
> {code}
> set hive.exec.parallel=true;
> set hive.merge.tezfiles=true;
> set tez.grouping.max-size=10;
> set tez.grouping.min-size=1;
> from student
> insert overwrite table student4 select *
> insert overwrite table student5 select *
> insert overwrite table student6 select *;
> {code}
> The merge task run in parallel and result the exception:
> {code}
> org.apache.tez.dag.api.TezException: App master already running a DAG
> at
> org.apache.tez.dag.app.DAGAppMaster.submitDAGToAppMaster(DAGAppMaster.java:1255)
> at
> org.apache.tez.dag.api.client.DAGClientHandler.submitDAG(DAGClientHandler.java:118)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.submitDAG(DAGClientAMProtocolBlockingPBServerImpl.java:161)
> at
> org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7471)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2273)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2269)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2267)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18201) Disable XPROD_EDGE for sq_count_check() created for scalar subqueries
[ https://issues.apache.org/jira/browse/HIVE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286636#comment-16286636 ] Gunther Hagleitner commented on HIVE-18201: --- Hm. Actually I think it's more complicated than just total data transfer. Time taken to produce the xprod needs to be factored in as well. > Disable XPROD_EDGE for sq_count_check() created for scalar subqueries > -- > > Key: HIVE-18201 > URL: https://issues.apache.org/jira/browse/HIVE-18201 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Nita Dembla > Attachments: query6.explain2.out > > > sq_count_check() will either return an error at runtime or a single row. In > case of query6, the subquery has avg() function that should return a single > row. Attaching the explain. > This does not need an x-prod, because it is not useful to shuffle the big > table side for a cross-product against 1 row. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17495) CachedStore: prewarm improvement (avoid multiple sql calls to read partition column stats), refactoring and caching some aggregate stats
[
https://issues.apache.org/jira/browse/HIVE-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286692#comment-16286692
]
Hive QA commented on HIVE-17495:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12901164/HIVE-17495.5.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8186/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8186/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8186/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-12-11 22:48:58.681
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-8186/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-12-11 22:48:58.684
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
22e7bff..31c1b71 master -> origin/master
+ git reset --hard HEAD
HEAD is now at 22e7bff HIVE-18246 : Replace toString with getExprString in
AbstractOperatorDesc::getColumnExprMapForExplain (Vineet Garg, reviewed by
Ashutosh Chauhan)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
(use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at 31c1b71 HIVE-18245 - clean up acid_vectorization_original.q
(Eugene Koifman, reviewed by Jason Dere)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-12-11 22:49:06.486
+ rm -rf ../yetus
+ mkdir ../yetus
+ cp -R . ../yetus
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-8186/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error:
a/itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java:
does not exist in index
error:
a/itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java:
does not exist in index
error:
a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java:
does not exist in index
error:
a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java:
does not exist in index
error:
a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java:
does not exist in index
error:
a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java:
does not exist in index
error:
a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java:
does not exist in index
error:
a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java:
does not exist in index
error:
a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/columnstats/aggr/BinaryColumnStatsAggregator.java:
does not exist in index
error:
a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/columnstats/aggr/BooleanColumnStatsAggregator.java:
does not exist in index
error:
a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/columnstats/aggr/ColumnStatsAggregator.java:
does not exist in index
error:
a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/columnstats/aggr/DateColumnStatsAggregator.java:
does not exist in index
error:
a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/columnstats/aggr/DecimalColumnStatsAggregator.java:
does not exist in index
error:
a/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/columnstats/aggr/DoubleColumnStatsAggregator.java:
[jira] [Commented] (HIVE-18112) show create for view having special char in where clause is not showing properly
[
https://issues.apache.org/jira/browse/HIVE-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286688#comment-16286688
]
Hive QA commented on HIVE-18112:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12901457/HIVE-18112.1-branch-2.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 58 failed/errored test(s), 9943 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=244)
TestJdbcDriver2 - did not produce a TEST-*.xml file (likely timed out)
(batchId=225)
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=167)
[acid_globallimit.q,alter_merge_2_orc.q]
TestMiniSparkOnYarnCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=173)
[infer_bucket_sort_reducers_power_two.q,list_bucket_dml_10.q,orc_merge9.q,orc_merge6.q,leftsemijoin_mr.q,bucket6.q,bucketmapjoin7.q,uber_reduce.q,empty_dir_in_table.q,vector_outer_join3.q,index_bitmap_auto.q,vector_outer_join2.q,vector_outer_join1.q,orc_merge1.q,orc_merge_diff_fs.q,load_hdfs_file_with_space_in_the_name.q,scriptfile1_win.q,quotedid_smb.q,truncate_column_buckets.q,orc_merge3.q]
TestMiniSparkOnYarnCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=174)
[infer_bucket_sort_num_buckets.q,gen_udf_example_add10.q,insert_overwrite_directory2.q,orc_merge5.q,bucketmapjoin6.q,import_exported_table.q,vector_outer_join0.q,orc_merge4.q,temp_table_external.q,orc_merge_incompat1.q,root_dir_external_table.q,constprog_semijoin.q,auto_sortmerge_join_16.q,schemeAuthority.q,index_bitmap3.q,external_table_with_space_in_location_path.q,parallel_orderby.q,infer_bucket_sort_map_operators.q,bucketizedhiveinputformat.q,remote_script.q]
TestMiniSparkOnYarnCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=175)
[scriptfile1.q,vector_outer_join5.q,file_with_header_footer.q,bucket4.q,input16_cc.q,bucket5.q,infer_bucket_sort_merge.q,constprog_partitioner.q,orc_merge2.q,reduce_deduplicate.q,schemeAuthority2.q,load_fs2.q,orc_merge8.q,orc_merge_incompat2.q,infer_bucket_sort_bucketed_table.q,vector_outer_join4.q,disable_merge_for_bucketing.q,vector_inner_join.q,orc_merge7.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=118)
[bucketmapjoin4.q,bucket_map_join_spark4.q,union21.q,groupby2_noskew.q,timestamp_2.q,date_join1.q,mergejoins.q,smb_mapjoin_11.q,auto_sortmerge_join_3.q,mapjoin_test_outer.q,vectorization_9.q,merge2.q,groupby6_noskew.q,auto_join_without_localtask.q,multi_join_union.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=119)
[join_cond_pushdown_unqual4.q,union_remove_7.q,join13.q,join_vc.q,groupby_cube1.q,bucket_map_join_spark2.q,sample3.q,smb_mapjoin_19.q,stats16.q,union23.q,union.q,union31.q,cbo_udf_udaf.q,ptf_decimal.q,bucketmapjoin2.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=120)
[parallel_join1.q,union27.q,union12.q,groupby7_map_multi_single_reducer.q,varchar_join1.q,join7.q,join_reorder4.q,skewjoinopt2.q,bucketsortoptimize_insert_2.q,smb_mapjoin_17.q,script_env_var1.q,groupby7_map.q,groupby3.q,bucketsortoptimize_insert_8.q,union20.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=121)
[ptf_general_queries.q,auto_join_reordering_values.q,sample2.q,join1.q,decimal_join.q,mapjoin_subquery2.q,join32_lessSize.q,mapjoin1.q,order2.q,skewjoinopt18.q,union_remove_18.q,join25.q,groupby9.q,bucketsortoptimize_insert_6.q,ctas.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=122)
[groupby_map_ppr.q,nullgroup4_multi_distinct.q,join_rc.q,union14.q,smb_mapjoin_12.q,vector_cast_constant.q,union_remove_4.q,auto_join11.q,load_dyn_part7.q,udaf_collect_set.q,vectorization_12.q,groupby_sort_skew_1.q,groupby_sort_skew_1_23.q,smb_mapjoin_25.q,skewjoinopt12.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=123)
[skewjoinopt15.q,auto_join18.q,list_bucket_dml_2.q,input1_limit.q,load_dyn_part3.q,union_remove_14.q,auto_sortmerge_join_14.q,auto_sortmerge_join_15.q,union10.q,bucket_map_join_tez2.q,groupby5_map_skew.q,join_reorder.q,sample1.q,bucketmapjoin8.q,union34.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=124)
[avro_joins.q,skewjoinopt16.q,auto_join14.q,vectorization_14.q,auto_join26.q,stats1.q,cbo_stats.q,auto_sortmerge_join_6.q,union22.q,union_remove_24.q,union_view.q,smb_mapjoin_22.q,stats15.q,ptf_matchpath.q,transform_ppr1.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=125)
[jira] [Commented] (HIVE-18252) Limit the size of the object inspector caches
[ https://issues.apache.org/jira/browse/HIVE-18252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286676#comment-16286676 ] Jason Dere commented on HIVE-18252: --- [~ashutoshc] can you review? > Limit the size of the object inspector caches > - > > Key: HIVE-18252 > URL: https://issues.apache.org/jira/browse/HIVE-18252 > Project: Hive > Issue Type: Bug > Components: Types >Reporter: Jason Dere >Assignee: Jason Dere > Attachments: HIVE-18252.1.patch > > > Was running some tests that had a lot of queries with constant values, and > noticed that ObjectInspectorFactory.cachedStandardStructObjectInspector > started using up a lot of memory. > It appears that StructObjectInspector caching does not work properly with > constant values. Constant ObjectInspectors are not cached, so each constant > expression creates a new constant ObjectInspector. And since object > inspectors do not override equals(), object inspector comparison relies on > object instance comparison. So even if the values are exactly the same as > what is already in the cache, the StructObjectInspector cache lookup would > fail, and Hive would create a new object inspector and add it to the cache, > creating another entry that would never be used. Plus, there is no max cache > size - it's just a map that is allowed to grow as long as values keep getting > added to it. > Some possible solutions I can think of: > 1. Limit the size of the object inspector caches, rather than growing without > bound. > 2. Try to fix the caching to work with constant values. This would require > implementing equals() on the constant object inspectors (which could be slow > in nested cases), or else we would have to start caching constant object > inspectors, which could be expensive in terms of memory usage. Could be used > in combination with (1). By itself this is not a great solution because this > still has the unbounded cache growth issue. > 3. Disable caching in the case of constant object inspectors since this > scenario currently doesn't work. This could be used in combination with (1). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18201) Disable XPROD_EDGE for sq_count_check() created for scalar subqueries
[ https://issues.apache.org/jira/browse/HIVE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286566#comment-16286566 ] Gunther Hagleitner commented on HIVE-18201: --- [~ashutoshc]: https://github.com/apache/hive/blob/22e7bffa1c1a9be1e8503ebe0f7adf39060d1979/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java#L108 We explicitly turn it off. The problem is that this decision needs to be cost based (i.e. take a look at the total data transfer.). We ran into cases where the xprod is much faster than map join... > Disable XPROD_EDGE for sq_count_check() created for scalar subqueries > -- > > Key: HIVE-18201 > URL: https://issues.apache.org/jira/browse/HIVE-18201 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Nita Dembla > Attachments: query6.explain2.out > > > sq_count_check() will either return an error at runtime or a single row. In > case of query6, the subquery has avg() function that should return a single > row. Attaching the explain. > This does not need an x-prod, because it is not useful to shuffle the big > table side for a cross-product against 1 row. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18252) Limit the size of the object inspector caches
[ https://issues.apache.org/jira/browse/HIVE-18252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-18252: -- Attachment: HIVE-18252.1.patch Patch to limit size of the object inspector cache for complex types, and to not cache complex object inspectors that contain constants. > Limit the size of the object inspector caches > - > > Key: HIVE-18252 > URL: https://issues.apache.org/jira/browse/HIVE-18252 > Project: Hive > Issue Type: Bug > Components: Types >Reporter: Jason Dere >Assignee: Jason Dere > Attachments: HIVE-18252.1.patch > > > Was running some tests that had a lot of queries with constant values, and > noticed that ObjectInspectorFactory.cachedStandardStructObjectInspector > started using up a lot of memory. > It appears that StructObjectInspector caching does not work properly with > constant values. Constant ObjectInspectors are not cached, so each constant > expression creates a new constant ObjectInspector. And since object > inspectors do not override equals(), object inspector comparison relies on > object instance comparison. So even if the values are exactly the same as > what is already in the cache, the StructObjectInspector cache lookup would > fail, and Hive would create a new object inspector and add it to the cache, > creating another entry that would never be used. Plus, there is no max cache > size - it's just a map that is allowed to grow as long as values keep getting > added to it. > Some possible solutions I can think of: > 1. Limit the size of the object inspector caches, rather than growing without > bound. > 2. Try to fix the caching to work with constant values. This would require > implementing equals() on the constant object inspectors (which could be slow > in nested cases), or else we would have to start caching constant object > inspectors, which could be expensive in terms of memory usage. Could be used > in combination with (1). By itself this is not a great solution because this > still has the unbounded cache growth issue. > 3. Disable caching in the case of constant object inspectors since this > scenario currently doesn't work. This could be used in combination with (1). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18252) Limit the size of the object inspector caches
[ https://issues.apache.org/jira/browse/HIVE-18252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-18252: -- Status: Patch Available (was: Open) > Limit the size of the object inspector caches > - > > Key: HIVE-18252 > URL: https://issues.apache.org/jira/browse/HIVE-18252 > Project: Hive > Issue Type: Bug > Components: Types >Reporter: Jason Dere >Assignee: Jason Dere > Attachments: HIVE-18252.1.patch > > > Was running some tests that had a lot of queries with constant values, and > noticed that ObjectInspectorFactory.cachedStandardStructObjectInspector > started using up a lot of memory. > It appears that StructObjectInspector caching does not work properly with > constant values. Constant ObjectInspectors are not cached, so each constant > expression creates a new constant ObjectInspector. And since object > inspectors do not override equals(), object inspector comparison relies on > object instance comparison. So even if the values are exactly the same as > what is already in the cache, the StructObjectInspector cache lookup would > fail, and Hive would create a new object inspector and add it to the cache, > creating another entry that would never be used. Plus, there is no max cache > size - it's just a map that is allowed to grow as long as values keep getting > added to it. > Some possible solutions I can think of: > 1. Limit the size of the object inspector caches, rather than growing without > bound. > 2. Try to fix the caching to work with constant values. This would require > implementing equals() on the constant object inspectors (which could be slow > in nested cases), or else we would have to start caching constant object > inspectors, which could be expensive in terms of memory usage. Could be used > in combination with (1). By itself this is not a great solution because this > still has the unbounded cache growth issue. > 3. Disable caching in the case of constant object inspectors since this > scenario currently doesn't work. This could be used in combination with (1). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-13567) Enable auto-gather column stats by default
[
https://issues.apache.org/jira/browse/HIVE-13567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286576#comment-16286576
]
Hive QA commented on HIVE-13567:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
56s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
41s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
5s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 56m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile
xml |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 22e7bff |
| Default Java | 1.8.0_111 |
| modules | C: common ql accumulo-handler contrib hbase-handler .
itests/hive-blobstore U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8184/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Enable auto-gather column stats by default
> --
>
> Key: HIVE-13567
> URL: https://issues.apache.org/jira/browse/HIVE-13567
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Zoltan Haindrich
> Attachments: HIVE-13567.01.patch, HIVE-13567.02.patch,
> HIVE-13567.03.patch, HIVE-13567.04.patch, HIVE-13567.05.patch,
> HIVE-13567.06.patch, HIVE-13567.07.patch, HIVE-13567.08.patch,
> HIVE-13567.09.patch, HIVE-13567.10.patch, HIVE-13567.11.patch,
> HIVE-13567.12.patch, HIVE-13567.13.patch, HIVE-13567.14.patch,
> HIVE-13567.15.patch, HIVE-13567.16.patch, HIVE-13567.17.patch,
> HIVE-13567.18.patch, HIVE-13567.19.patch, HIVE-13567.20.patch,
> HIVE-13567.21.patch, HIVE-13567.22.patch, HIVE-13567.23.patch,
> HIVE-13567.23wip01.patch, HIVE-13567.23wip02.patch, HIVE-13567.23wip03.patch,
> HIVE-13567.23wip04.patch, HIVE-13567.23wip05.patch, HIVE-13567.23wip06.patch,
> HIVE-13567.23wip07.patch, HIVE-13567.23wip08.patch, HIVE-13567.23wip09.patch,
> HIVE-13567.23wip10.patch, HIVE-13567.24.patch
>
>
> in phase 2, we are going to set auto-gather column on as default. This needs
> to update golden files.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18075) verify commands on a cluster
[ https://issues.apache.org/jira/browse/HIVE-18075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286653#comment-16286653 ] Sergey Shelukhin commented on HIVE-18075: - +1 > verify commands on a cluster > > > Key: HIVE-18075 > URL: https://issues.apache.org/jira/browse/HIVE-18075 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Harish Jaiprakash > Attachments: HIVE-18075.01.patch > > > I was running the commands in the cluster, with potentially a slightly > outdated version of the DB on mysql (yet, the DB was created with a script > from master and I didn't hit any schema issues, so it's fairly recent); > however, recent master code + the pools commands patch. I've hit the > following issues. > # Cannot drop pool or RP with a mapping (see also 3). > # Cannot drop pool that is set as default (probably correct, but the error > message is bad). > # When I dropped an RP with a mapping, and then created it again with the > same name, the pool creation in that RP would fail with an error that a > unique query returned multiple results. In the DB, there were actually 2 RPs > with the same name. Not sure how exactly that happened, there might have been > intermediate states, but I didn't mess with mysql. I think the name > uniqueness is either missing from some script or doesn't work. > # -Setting RP default pool no longer works. I think I might have broken it > with one of the rebases in that area, but it could also be something else (or > like other things, it works in q tests but not on cluster for whatever > reason-). > # -Resource plan rename doesn't check the disable state. It probably should. > Also need to see for other commands-. > Need to figure out which are real issues and which aren't and fix. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HIVE-18250) CBO gets turned off with duplicates in RR error
[
https://issues.apache.org/jira/browse/HIVE-18250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-18250:
--
Assignee: Jesus Camacho Rodriguez
> CBO gets turned off with duplicates in RR error
> ---
>
> Key: HIVE-18250
> URL: https://issues.apache.org/jira/browse/HIVE-18250
> Project: Hive
> Issue Type: Bug
> Components: CBO, Query Planning
>Affects Versions: 2.0.0, 2.1.0, 2.2.0, 2.3.0
>Reporter: Ashutosh Chauhan
>Assignee: Jesus Camacho Rodriguez
>
> {code}
> create table t1 (a int);
> explain select t1.a as a1, min(t1.a) as a from t1 group by t1.a;
> {code}
> CBO gets turned off with:
> {code}
> WARN [2e80e34e-dc46-49cf-88bf-2c24c0262d41 main] parse.RowResolver: Found
> duplicate column alias in RR: null.a => {null, a1, _col0: int} adding null.a
> => {null, null, _col1: int}
> 2017-12-07T15:27:47,651 ERROR [2e80e34e-dc46-49cf-88bf-2c24c0262d41 main]
> parse.CalcitePlanner: CBO failed, skipping CBO.
> org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSemanticException: Cannot
> add column to RR: null.a => _col1: int due to duplication, see previous
> warnings
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genSelectLogicalPlan(CalcitePlanner.java:3985)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:4313)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1392)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1322)
> ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
> {code}
> After that non-CBO path completes the query.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-13567) Enable auto-gather column stats by default
[
https://issues.apache.org/jira/browse/HIVE-13567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286593#comment-16286593
]
Hive QA commented on HIVE-13567:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12901388/HIVE-13567.24.patch
{color:green}SUCCESS:{color} +1 due to 48 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 11120 tests
executed
*Failed tests:*
{noformat}
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[jira] [Updated] (HIVE-17794) HCatLoader breaks when a member is added to a struct-column of a table
[
https://issues.apache.org/jira/browse/HIVE-17794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mithun Radhakrishnan updated HIVE-17794:
Status: Patch Available (was: Open)
> HCatLoader breaks when a member is added to a struct-column of a table
> --
>
> Key: HIVE-17794
> URL: https://issues.apache.org/jira/browse/HIVE-17794
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 2.2.0, 3.0.0
>Reporter: Mithun Radhakrishnan
>Assignee: Mithun Radhakrishnan
> Attachments: HIVE-17794.1.patch, HIVE-17794.2.patch
>
>
> When a table's schema evolves to add a new member to a struct column, Hive
> queries work fine, but {{HCatLoader}} breaks with the following trace:
> {noformat}
> TaskAttempt 1 failed, info=
> Error: Failure while running
> task:org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: kite_composites_with_segments: Local Rearrange
> tuple
> {chararray}(false) - scope-555-> scope-974 Operator Key: scope-555):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: gup: New For Each(false,false)
> bag
> - scope-548 Operator Key: scope-548):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: gup_filtered: Filter
> bag
> - scope-522 Operator Key: scope-522):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
> converting read value to tuple
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:314)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNextTuple(POLocalRearrange.java:287)
> at
> org.apache.pig.backend.hadoop.executionengine.tez.plan.operator.POLocalRearrangeTez.getNextTuple(POLocalRearrangeTez.java:127)
> at
> org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor.runPipeline(PigProcessor.java:376)
> at
> org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor.run(PigProcessor.java:241)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:362)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1679)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> Exception while executing (Name: gup: New For Each(false,false)
> bag
> - scope-548 Operator Key: scope-548):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: gup_filtered: Filter
> bag
> - scope-522 Operator Key: scope-522):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
> converting read value to tuple
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:314)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:252)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:305)
> ... 17 more
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> Exception while executing (Name: gup_filtered: Filter
> bag
> - scope-522 Operator Key: scope-522):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
> converting read value to tuple
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:314)
> at
>
[jira] [Updated] (HIVE-17710) LockManager should only lock Managed tables
[ https://issues.apache.org/jira/browse/HIVE-17710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-17710: -- Attachment: HIVE-17710.04.patch patch 4 ensures that Managed_Table and Maetriazlied_view are locked - other types are ignored. > LockManager should only lock Managed tables > --- > > Key: HIVE-17710 > URL: https://issues.apache.org/jira/browse/HIVE-17710 > Project: Hive > Issue Type: New Feature > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-17710.01.patch, HIVE-17710.02.patch, > HIVE-17710.03.patch, HIVE-17710.04.patch > > > should the LM take locks on External tables? Out of the box Acid LM is being > conservative which can cause throughput issues. > A better strategy may be to exclude External tables but enable explicit "lock > table/partition " command (only on external tables?). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18068) Upgrade to Calcite 1.15
[ https://issues.apache.org/jira/browse/HIVE-18068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-18068: --- Attachment: HIVE-18068.06.patch > Upgrade to Calcite 1.15 > --- > > Key: HIVE-18068 > URL: https://issues.apache.org/jira/browse/HIVE-18068 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-18068.03.patch, HIVE-18068.04.patch, > HIVE-18068.05.patch, HIVE-18068.06.patch, HIVE-18068.2.patch, HIVE-18068.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17794) HCatLoader breaks when a member is added to a struct-column of a table
[
https://issues.apache.org/jira/browse/HIVE-17794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mithun Radhakrishnan updated HIVE-17794:
Status: Open (was: Patch Available)
> HCatLoader breaks when a member is added to a struct-column of a table
> --
>
> Key: HIVE-17794
> URL: https://issues.apache.org/jira/browse/HIVE-17794
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 2.2.0, 3.0.0
>Reporter: Mithun Radhakrishnan
>Assignee: Mithun Radhakrishnan
> Attachments: HIVE-17794.1.patch, HIVE-17794.2.patch
>
>
> When a table's schema evolves to add a new member to a struct column, Hive
> queries work fine, but {{HCatLoader}} breaks with the following trace:
> {noformat}
> TaskAttempt 1 failed, info=
> Error: Failure while running
> task:org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: kite_composites_with_segments: Local Rearrange
> tuple
> {chararray}(false) - scope-555-> scope-974 Operator Key: scope-555):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: gup: New For Each(false,false)
> bag
> - scope-548 Operator Key: scope-548):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: gup_filtered: Filter
> bag
> - scope-522 Operator Key: scope-522):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
> converting read value to tuple
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:314)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNextTuple(POLocalRearrange.java:287)
> at
> org.apache.pig.backend.hadoop.executionengine.tez.plan.operator.POLocalRearrangeTez.getNextTuple(POLocalRearrangeTez.java:127)
> at
> org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor.runPipeline(PigProcessor.java:376)
> at
> org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor.run(PigProcessor.java:241)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:362)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1679)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> Exception while executing (Name: gup: New For Each(false,false)
> bag
> - scope-548 Operator Key: scope-548):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception
> while executing (Name: gup_filtered: Filter
> bag
> - scope-522 Operator Key: scope-522):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
> converting read value to tuple
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:314)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:252)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:305)
> ... 17 more
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> Exception while executing (Name: gup_filtered: Filter
> bag
> - scope-522 Operator Key: scope-522):
> org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
> converting read value to tuple
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:314)
> at
>
[jira] [Updated] (HIVE-16850) Converting table to insert-only acid may open a txn in an inappropriate place
[
https://issues.apache.org/jira/browse/HIVE-16850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-16850:
--
Fix Version/s: (was: hive-14535)
> Converting table to insert-only acid may open a txn in an inappropriate place
> -
>
> Key: HIVE-16850
> URL: https://issues.apache.org/jira/browse/HIVE-16850
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Wei Zheng
>Assignee: Eugene Koifman
> Labels: mm-gap-2
>
> This would work for unit-testing, but would need to be fixed for production.
> {noformat}
> HiveTxnManager txnManager = SessionState.get().getTxnMgr();
> if (txnManager.isTxnOpen()) {
> mmWriteId = txnManager.getCurrentTxnId();
> } else {
> mmWriteId = txnManager.openTxn(new Context(conf), conf.getUser());
> txnManager.commitTxn();
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-14653) week value is different between weekofyear(date) and date_format(date, "ww")
[
https://issues.apache.org/jira/browse/HIVE-14653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286570#comment-16286570
]
Zachary Tyler Pruitt commented on HIVE-14653:
-
ISO 8601 specifies that the first day of the week is Monday.
ISO 8601 specifies that the first week of the year will be the week with the
first Thursday in it.
WEEKOFYEAR is following ISO 8601.
DATE_FORMAT is not.
{code}
SELECT Calendar_Date,
DATE_FORMAT(Calendar_Date, '') AS Day_Of_Week,
DATE_FORMAT(Calendar_Date, 'u') AS Day_Of_Week,
WEEKOFYEAR(Calendar_Date) AS Week_Of_Year_wof,
DATE_FORMAT(Calendar_Date, 'w') AS Week_Of_Year_df,
DATE_FORMAT(Calendar_Date, 'W') AS Week_Of_Month
FROM (SELECT EXPLODE(ARRAY('2016-12-31','2017-01-01','2017-01-02')) AS
Calendar_Date) c;
++--+--+---+--++--+
| calendar_date | day_of_week | day_of_week | week_of_year_wof |
week_of_year_df | week_of_month |
++--+--+---+--++--+
| 2016-12-31 | Saturday | 6| 52| 53
| 5 |
| 2017-01-01 | Sunday | 7| 52| 1
| 1 |
| 2017-01-02 | Monday | 1| 1 | 1
| 1 |
++--+--+---+--++--+
{code}
> week value is different between weekofyear(date) and date_format(date, "ww")
>
>
> Key: HIVE-14653
> URL: https://issues.apache.org/jira/browse/HIVE-14653
> Project: Hive
> Issue Type: Bug
>Reporter: Shawn Zhou
>Priority: Minor
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18245) clean up acid_vectorization_original.q
[ https://issues.apache.org/jira/browse/HIVE-18245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-18245: -- Resolution: Fixed Target Version/s: 3.0.0 Status: Resolved (was: Patch Available) committed to master thanks Jason for the review > clean up acid_vectorization_original.q > -- > > Key: HIVE-18245 > URL: https://issues.apache.org/jira/browse/HIVE-18245 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Fix For: 3.0.0 > > Attachments: HIVE-18245.01.patch, HIVE-18245.02.patch, > HIVE-18245.03.patch > > > now that HIVE-17923 is fixed, > acid_vectorization_original_tez.q/acid_vectorization_original.q can be > cleaned up -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18221) test acid default
[
https://issues.apache.org/jira/browse/HIVE-18221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286478#comment-16286478
]
Hive QA commented on HIVE-18221:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12901367/HIVE-18221.09.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 353 failed/errored test(s), 7079 tests
executed
*Failed tests:*
{noformat}
TestMetaStoreEventListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=218)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_custom_key2]
(batchId=238)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_custom_key]
(batchId=238)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_index]
(batchId=238)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_joins]
(batchId=238)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_predicate_pushdown]
(batchId=238)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=238)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_single_sourced_multi_insert]
(batchId=238)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[orc_format_part]
(batchId=249)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[orc_nonstd_partitions_loc]
(batchId=249)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[ptf_orcfile]
(batchId=249)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=10)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=11)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=19)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=29)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=31)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=33)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
[jira] [Assigned] (HIVE-18264) CachedStore: Store cached partitions within the table cache
[ https://issues.apache.org/jira/browse/HIVE-18264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta reassigned HIVE-18264: --- > CachedStore: Store cached partitions within the table cache > - > > Key: HIVE-18264 > URL: https://issues.apache.org/jira/browse/HIVE-18264 > Project: Hive > Issue Type: Bug >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > > Currently we have a separate cache for partitions and partition col stats > which results in some calls iterating through each of these for > retrieving/updating. We can get better performance by organizing > hierarchically. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18264) CachedStore: Store cached partitions/col stats within the table cache
[ https://issues.apache.org/jira/browse/HIVE-18264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-18264: Summary: CachedStore: Store cached partitions/col stats within the table cache(was: CachedStore: Store cached partitions within the table cache ) > CachedStore: Store cached partitions/col stats within the table cache > --- > > Key: HIVE-18264 > URL: https://issues.apache.org/jira/browse/HIVE-18264 > Project: Hive > Issue Type: Bug >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > > Currently we have a separate cache for partitions and partition col stats > which results in some calls iterating through each of these for > retrieving/updating. We can get better performance by organizing > hierarchically. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18244) CachedStore: Fix UT when CachedStore is enabled
[ https://issues.apache.org/jira/browse/HIVE-18244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-18244: Status: Patch Available (was: Open) > CachedStore: Fix UT when CachedStore is enabled > --- > > Key: HIVE-18244 > URL: https://issues.apache.org/jira/browse/HIVE-18244 > Project: Hive > Issue Type: Bug >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > Attachments: HIVE-18244.1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17495) CachedStore: prewarm improvement (avoid multiple sql calls to read partition column stats), refactoring and caching some aggregate stats
[ https://issues.apache.org/jira/browse/HIVE-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-17495: Status: Open (was: Patch Available) > CachedStore: prewarm improvement (avoid multiple sql calls to read partition > column stats), refactoring and caching some aggregate stats > > > Key: HIVE-17495 > URL: https://issues.apache.org/jira/browse/HIVE-17495 > Project: Hive > Issue Type: Bug > Components: Metastore >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > Attachments: HIVE-17495.1.patch, HIVE-17495.2.patch, > HIVE-17495.3.patch, HIVE-17495.4.patch, HIVE-17495.5.patch > > > 1. One sql call to retrieve column stats objects for a db > 2. Cache some aggregate stats for speedup -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17495) CachedStore: prewarm improvement (avoid multiple sql calls to read partition column stats), refactoring and caching some aggregate stats
[ https://issues.apache.org/jira/browse/HIVE-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-17495: Status: Patch Available (was: Open) > CachedStore: prewarm improvement (avoid multiple sql calls to read partition > column stats), refactoring and caching some aggregate stats > > > Key: HIVE-17495 > URL: https://issues.apache.org/jira/browse/HIVE-17495 > Project: Hive > Issue Type: Bug > Components: Metastore >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > Attachments: HIVE-17495.1.patch, HIVE-17495.2.patch, > HIVE-17495.3.patch, HIVE-17495.4.patch, HIVE-17495.5.patch > > > 1. One sql call to retrieve column stats objects for a db > 2. Cache some aggregate stats for speedup -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18244) CachedStore: Fix UT when CachedStore is enabled
[ https://issues.apache.org/jira/browse/HIVE-18244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-18244: Status: Open (was: Patch Available) > CachedStore: Fix UT when CachedStore is enabled > --- > > Key: HIVE-18244 > URL: https://issues.apache.org/jira/browse/HIVE-18244 > Project: Hive > Issue Type: Bug >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > Attachments: HIVE-18244.1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18191) Vectorization: Add validation of TableScanOperator (gather statistics) back
[ https://issues.apache.org/jira/browse/HIVE-18191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18191: Status: In Progress (was: Patch Available) > Vectorization: Add validation of TableScanOperator (gather statistics) back > --- > > Key: HIVE-18191 > URL: https://issues.apache.org/jira/browse/HIVE-18191 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-18191.01.patch, HIVE-18191.02.patch, > HIVE-18191.03.patch, HIVE-18191.04.patch, HIVE-18191.05.patch, > HIVE-18191.06.patch > > > HIVE-17433 accidentally removed call to validateTableScanOperator. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18191) Vectorization: Add validation of TableScanOperator (gather statistics) back
[ https://issues.apache.org/jira/browse/HIVE-18191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18191: Status: Patch Available (was: In Progress) > Vectorization: Add validation of TableScanOperator (gather statistics) back > --- > > Key: HIVE-18191 > URL: https://issues.apache.org/jira/browse/HIVE-18191 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-18191.01.patch, HIVE-18191.02.patch, > HIVE-18191.03.patch, HIVE-18191.04.patch, HIVE-18191.05.patch, > HIVE-18191.06.patch, HIVE-18191.07.patch > > > HIVE-17433 accidentally removed call to validateTableScanOperator. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18191) Vectorization: Add validation of TableScanOperator (gather statistics) back
[ https://issues.apache.org/jira/browse/HIVE-18191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18191: Attachment: HIVE-18191.07.patch > Vectorization: Add validation of TableScanOperator (gather statistics) back > --- > > Key: HIVE-18191 > URL: https://issues.apache.org/jira/browse/HIVE-18191 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-18191.01.patch, HIVE-18191.02.patch, > HIVE-18191.03.patch, HIVE-18191.04.patch, HIVE-18191.05.patch, > HIVE-18191.06.patch, HIVE-18191.07.patch > > > HIVE-17433 accidentally removed call to validateTableScanOperator. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18258) Vectorization: Reduce-Side GROUP BY MERGEPARTIAL with duplicate columns is broken
[ https://issues.apache.org/jira/browse/HIVE-18258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18258: Status: Patch Available (was: In Progress) > Vectorization: Reduce-Side GROUP BY MERGEPARTIAL with duplicate columns is > broken > - > > Key: HIVE-18258 > URL: https://issues.apache.org/jira/browse/HIVE-18258 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0 > > Attachments: HIVE-18258.01.patch, HIVE-18258.02.patch > > > See Q file. Duplicate columns in key are not handled correctly. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18245) clean up acid_vectorization_original.q
[ https://issues.apache.org/jira/browse/HIVE-18245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286460#comment-16286460 ] Jason Dere commented on HIVE-18245: --- +1 > clean up acid_vectorization_original.q > -- > > Key: HIVE-18245 > URL: https://issues.apache.org/jira/browse/HIVE-18245 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Fix For: 3.0.0 > > Attachments: HIVE-18245.01.patch, HIVE-18245.02.patch, > HIVE-18245.03.patch > > > now that HIVE-17923 is fixed, > acid_vectorization_original_tez.q/acid_vectorization_original.q can be > cleaned up -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18258) Vectorization: Reduce-Side GROUP BY MERGEPARTIAL with duplicate columns is broken
[ https://issues.apache.org/jira/browse/HIVE-18258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18258: Attachment: HIVE-18258.02.patch > Vectorization: Reduce-Side GROUP BY MERGEPARTIAL with duplicate columns is > broken > - > > Key: HIVE-18258 > URL: https://issues.apache.org/jira/browse/HIVE-18258 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0 > > Attachments: HIVE-18258.01.patch, HIVE-18258.02.patch > > > See Q file. Duplicate columns in key are not handled correctly. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18212) Make sure Yetus check always has a full log
[
https://issues.apache.org/jira/browse/HIVE-18212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286291#comment-16286291
]
Adam Szita commented on HIVE-18212:
---
Thanks for reviewing Peter!
> Make sure Yetus check always has a full log
> ---
>
> Key: HIVE-18212
> URL: https://issues.apache.org/jira/browse/HIVE-18212
> Project: Hive
> Issue Type: Sub-task
>Reporter: Adam Szita
>Assignee: Adam Szita
> Fix For: 3.0.0
>
> Attachments: HIVE-18212.0.patch
>
>
> Some yetus log files are left incomplete, and in these same runs {{tee}}
> subprocesses are left running and dangling on the ptest server.
> This is because of a bug in the yetus runner velocity template script where
> we make a redirection of stdout:
> {code}
> ./dev-support/test-patch.sh ${patchFile} . 2>&1 | tee ${logFile}
> {code}
> If the yetus output is big enough (>62K) tee will stop writing the log file
> and is left running even after {{test-patch.sh}} finished successfully. This
> because we don't make anything consume the stdout and most probably some
> buffers get full on Linux side.
> We should also make sure that yetus runs(since they are executed parallel to
> ptest test phase) are not interfering with each other in case they run very
> long and overlap.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18221) test acid default
[
https://issues.apache.org/jira/browse/HIVE-18221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286457#comment-16286457
]
Hive QA commented on HIVE-18221:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
6s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} standalone-metastore: The patch generated 8 new + 209
unchanged - 0 fixed = 217 total (was 209) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
36s{color} | {color:red} ql: The patch generated 1 new + 315 unchanged - 0
fixed = 316 total (was 315) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 22e7bff |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8183/yetus/diff-checkstyle-standalone-metastore.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8183/yetus/diff-checkstyle-ql.txt
|
| modules | C: common standalone-metastore ql hcatalog/hcatalog-pig-adapter U:
. |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8183/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> test acid default
> -
>
> Key: HIVE-18221
> URL: https://issues.apache.org/jira/browse/HIVE-18221
> Project: Hive
> Issue Type: Test
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
> Attachments: HIVE-18221.01.patch, HIVE-18221.02.patch,
> HIVE-18221.03.patch, HIVE-18221.04.patch, HIVE-18221.07.patch,
> HIVE-18221.08.patch, HIVE-18221.09.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18258) Vectorization: Reduce-Side GROUP BY MERGEPARTIAL with duplicate columns is broken
[ https://issues.apache.org/jira/browse/HIVE-18258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18258: Status: In Progress (was: Patch Available) > Vectorization: Reduce-Side GROUP BY MERGEPARTIAL with duplicate columns is > broken > - > > Key: HIVE-18258 > URL: https://issues.apache.org/jira/browse/HIVE-18258 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0 > > Attachments: HIVE-18258.01.patch > > > See Q file. Duplicate columns in key are not handled correctly. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18245) clean up acid_vectorization_original.q
[ https://issues.apache.org/jira/browse/HIVE-18245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286440#comment-16286440 ] Eugene Koifman commented on HIVE-18245: --- [~jdere] could you review please > clean up acid_vectorization_original.q > -- > > Key: HIVE-18245 > URL: https://issues.apache.org/jira/browse/HIVE-18245 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Fix For: 3.0.0 > > Attachments: HIVE-18245.01.patch, HIVE-18245.02.patch, > HIVE-18245.03.patch > > > now that HIVE-17923 is fixed, > acid_vectorization_original_tez.q/acid_vectorization_original.q can be > cleaned up -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18245) clean up acid_vectorization_original.q
[
https://issues.apache.org/jira/browse/HIVE-18245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286428#comment-16286428
]
Hive QA commented on HIVE-18245:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12901368/HIVE-18245.03.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 11516 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketizedhiveinputformat]
(batchId=178)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=113)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=226)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8182/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8182/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8182/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12901368 - PreCommit-HIVE-Build
> clean up acid_vectorization_original.q
> --
>
> Key: HIVE-18245
> URL: https://issues.apache.org/jira/browse/HIVE-18245
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
> Fix For: 3.0.0
>
> Attachments: HIVE-18245.01.patch, HIVE-18245.02.patch,
> HIVE-18245.03.patch
>
>
> now that HIVE-17923 is fixed,
> acid_vectorization_original_tez.q/acid_vectorization_original.q can be
> cleaned up
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18246) Replace toString with getExprString in AbstractOperatorDesc::getColumnExprMapForExplain
[ https://issues.apache.org/jira/browse/HIVE-18246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-18246: --- Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master, Thanks [~ashutoshc] > Replace toString with getExprString in > AbstractOperatorDesc::getColumnExprMapForExplain > --- > > Key: HIVE-18246 > URL: https://issues.apache.org/jira/browse/HIVE-18246 > Project: Hive > Issue Type: Improvement >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-18246.1.patch, HIVE-18246.2.patch > > > AbstractOperatorDesc::getColumnExprMapForExplain uses toString on ExprNode to > get the string representation of an expr. getExprString is better suited here > since each ExprNode class has suitable implementation for this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18237) missing results for insert_only table after DP insert
[
https://issues.apache.org/jira/browse/HIVE-18237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286399#comment-16286399
]
Steve Yeom commented on HIVE-18237:
---
Hi Zoltan,
Sorry I do not normally interrupt a jira.
This time it just happened to be that this jira is an issue to be cleared for
the work I am currently working on.
I hope you understand.
> missing results for insert_only table after DP insert
> -
>
> Key: HIVE-18237
> URL: https://issues.apache.org/jira/browse/HIVE-18237
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Zoltan Haindrich
> Attachments: HIVE-18237.01.patch
>
>
> {code}
> set hive.stats.column.autogather=false;
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.exec.max.dynamic.partitions.pernode=200;
> set hive.exec.max.dynamic.partitions=200;
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> create table i0 (p int,v int);
> insert into i0 values
> (0,0),
> (2,2),
> (3,3);
> create table p0 (v int) partitioned by (p int) stored as orc
> tblproperties ("transactional"="true",
> "transactional_properties"="insert_only");
> explain insert overwrite table p0 partition (p) select * from i0 where v < 3;
> insert overwrite table p0 partition (p) select * from i0 where v < 3;
> select count(*) from p0 where v!=1;
> {code}
> The table p0 should contain {{2}} rows at this point; but the result is {{0}}.
> * seems to be specific to insert_only tables
> * the existing data appears if an {{insert into}} is executed.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-15883) HBase mapped table in Hive insert fail for decimal
[
https://issues.apache.org/jira/browse/HIVE-15883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286368#comment-16286368
]
Aihua Xu commented on HIVE-15883:
-
Yeah. The test passes. I will use HIVE-18202 to find why you don't need to
specify hbase.mapreduce.hfileoutputformat.table.name.
But your change looks good. +1.
> HBase mapped table in Hive insert fail for decimal
> --
>
> Key: HIVE-15883
> URL: https://issues.apache.org/jira/browse/HIVE-15883
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.1.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
> Attachments: HIVE-15883.1.patch, HIVE-15883.1.patch, HIVE-15883.patch
>
>
> CREATE TABLE hbase_table (
> id int,
> balance decimal(15,2))
> ROW FORMAT DELIMITED
> COLLECTION ITEMS TERMINATED BY '~'
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES (
> "hbase.columns.mapping"=":key,cf:balance#b");
> insert into hbase_table values (1,1);
>
> Diagnostic Messages for this Task:
> Error: java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row {"tmp_values_col1":"1","tmp_values_col2":"1"}
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:179)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1783)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row {"tmp_values_col1":"1","tmp_values_col2":"1"}
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:507)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:170)
> ... 8 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.hadoop.hive.serde2.SerDeException: java.lang.RuntimeException:
> Hive internal error.
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:733)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:97)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:157)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:497)
> ... 9 more
> Caused by: org.apache.hadoop.hive.serde2.SerDeException:
> java.lang.RuntimeException: Hive internal error.
> at
> org.apache.hadoop.hive.hbase.HBaseSerDe.serialize(HBaseSerDe.java:286)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:668)
> ... 15 more
> Caused by: java.lang.RuntimeException: Hive internal error.
> at
> org.apache.hadoop.hive.serde2.lazy.LazyUtils.writePrimitive(LazyUtils.java:328)
> at
> org.apache.hadoop.hive.hbase.HBaseRowSerializer.serialize(HBaseRowSerializer.java:220)
> at
> org.apache.hadoop.hive.hbase.HBaseRowSerializer.serializeField(HBaseRowSerializer.java:194)
> at
> org.apache.hadoop.hive.hbase.HBaseRowSerializer.serialize(HBaseRowSerializer.java:118)
> at
> org.apache.hadoop.hive.hbase.HBaseSerDe.serialize(HBaseSerDe.java:282)
> ... 16 more
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18245) clean up acid_vectorization_original.q
[
https://issues.apache.org/jira/browse/HIVE-18245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286377#comment-16286377
]
Hive QA commented on HIVE-18245:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 2m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / f17c9b4 |
| modules | C: ql . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8182/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> clean up acid_vectorization_original.q
> --
>
> Key: HIVE-18245
> URL: https://issues.apache.org/jira/browse/HIVE-18245
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
> Fix For: 3.0.0
>
> Attachments: HIVE-18245.01.patch, HIVE-18245.02.patch,
> HIVE-18245.03.patch
>
>
> now that HIVE-17923 is fixed,
> acid_vectorization_original_tez.q/acid_vectorization_original.q can be
> cleaned up
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17710) LockManager should only lock Managed tables
[
https://issues.apache.org/jira/browse/HIVE-17710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286370#comment-16286370
]
Hive QA commented on HIVE-17710:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12901369/HIVE-17710.03.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 11501 tests
executed
*Failed tests:*
{noformat}
TestMiniTezCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=102)
[update_orig_table.q,explainanalyze_3.q,orc_vectorization_ppd.q,vectorization_limit.q,vector_non_string_partition.q,multi_count_distinct.q,explainanalyze_1.q,explainuser_3.q,hybridgrace_hashjoin_1.q,delete_orig_table.q,hybridgrace_hashjoin_2.q,tez-tag.q,orc_merge12.q,explainanalyze_5.q,vector_join_part_col_char.q]
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketizedhiveinputformat]
(batchId=178)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi]
(batchId=113)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=226)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8181/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8181/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8181/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12901369 - PreCommit-HIVE-Build
> LockManager should only lock Managed tables
> ---
>
> Key: HIVE-17710
> URL: https://issues.apache.org/jira/browse/HIVE-17710
> Project: Hive
> Issue Type: New Feature
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
> Attachments: HIVE-17710.01.patch, HIVE-17710.02.patch,
> HIVE-17710.03.patch
>
>
> should the LM take locks on External tables? Out of the box Acid LM is being
> conservative which can cause throughput issues.
> A better strategy may be to exclude External tables but enable explicit "lock
> table/partition " command (only on external tables?).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HIVE-18263) Ptest execution are multiple times slower sometimes due to dying executor slaves
[ https://issues.apache.org/jira/browse/HIVE-18263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Szita reassigned HIVE-18263: - > Ptest execution are multiple times slower sometimes due to dying executor > slaves > > > Key: HIVE-18263 > URL: https://issues.apache.org/jira/browse/HIVE-18263 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Adam Szita >Assignee: Adam Szita > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
