[jira] [Commented] (HIVE-19155) Day time saving cause Druid inserts to fail with org.apache.hive.druid.io.druid.java.util.common.UOE: Cannot add overlapping segments
[
https://issues.apache.org/jira/browse/HIVE-19155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434966#comment-16434966
]
Hive QA commented on HIVE-19155:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12918445/HIVE-19155.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 106 failed/errored test(s), 14069 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=253)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=253)
TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed
out) (batchId=217)
TestSequenceFileReadWrite - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestTezPerfCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[groupby_groupingset_bug]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mergejoin]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_main]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[update_access_time_non_current_db]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_dynamic_semijoin_reduction]
(batchId=155)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=105)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.org.apache.hadoop.hive.cli.TestNegativeCliDriver
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.org.apache.hadoop.hive.cli.TestNegativeCliDriver
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_notnull_constraint_violation]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[default_constraint_invalid_default_value_type]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[insert_into_acid_notnull]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[insert_into_notnull_constraint]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[insert_multi_into_notnull]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[insert_overwrite_notnull_constraint]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[smb_bucketmapjoin]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[smb_mapjoin_14]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[sortmerge_mapjoin_mismatch_1]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_aggregator_error_1]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_aggregator_error_2]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_publisher_error_1]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_publisher_error_2]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_corr_in_agg]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_in_implicit_gby]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_notin_implicit_gby]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[truncate_bucketed_column]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[truncate_column_list_bucketing]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[truncate_column_seqfile]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[truncate_nonexistant_column]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[truncate_partition_column2]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[truncate_partition_column]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_invalid]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_likeall_wrong1]
(batchId=96)
[jira] [Updated] (HIVE-19130) NPE is thrown when REPL LOAD applied drop partition event.
[
https://issues.apache.org/jira/browse/HIVE-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sankar Hariappan updated HIVE-19130:
Resolution: Fixed
Status: Resolved (was: Patch Available)
> NPE is thrown when REPL LOAD applied drop partition event.
> --
>
> Key: HIVE-19130
> URL: https://issues.apache.org/jira/browse/HIVE-19130
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, repl
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
> Labels: DR, Replication, pull-request-available
> Fix For: 3.1.0
>
> Attachments: HIVE-19130.01.patch
>
>
> During incremental replication, if we split the events batch as follows, then
> the REPL LOAD on second batch throws NPE.
> Batch-1: CREATE_TABLE(t1) -> ADD_PARTITION(t1.p1) -> DROP_PARTITION (t1.p1)
> Batch-2: DROP_TABLE(t1) -> CREATE_TABLE(t1) -> ADD_PARTITION(t1.p1) ->
> DROP_PARTITION (t1.p1)
> {code}
> 2018-04-05 16:20:36,531 ERROR [HiveServer2-Background-Pool: Thread-107044]:
> metadata.Hive (Hive.java:getTable(1219)) - Table catalog_sales_new not found:
> new5_tpcds_real_bin_partitioned_orc_1000.catalog_sales_new table not found
> 2018-04-05 16:20:36,538 ERROR [HiveServer2-Background-Pool: Thread-107044]:
> exec.DDLTask (DDLTask.java:failed(540)) -
> org.apache.hadoop.hive.ql.metadata.HiveException
> at
> org.apache.hadoop.hive.ql.exec.DDLTask.dropPartitions(DDLTask.java:4016)
> at
> org.apache.hadoop.hive.ql.exec.DDLTask.dropTableOrPartitions(DDLTask.java:3983)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:341)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:162)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:89)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1765)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1506)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1303)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1170)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1165)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:197)
> at
> org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:76)
> at
> org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
> at
> org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.metadata.Hive.getPartitionsByExpr(Hive.java:2613)
> at
> org.apache.hadoop.hive.ql.exec.DDLTask.dropPartitions(DDLTask.java:4008)
> ... 23 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19130) NPE is thrown when REPL LOAD applied drop partition event.
[
https://issues.apache.org/jira/browse/HIVE-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434929#comment-16434929
]
Sankar Hariappan commented on HIVE-19130:
-
Test failures are irrelevant to the patch.
Patch is committed to master.
Thanks [~maheshk114] and [~thejas] for the review!
> NPE is thrown when REPL LOAD applied drop partition event.
> --
>
> Key: HIVE-19130
> URL: https://issues.apache.org/jira/browse/HIVE-19130
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, repl
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
> Labels: DR, Replication, pull-request-available
> Fix For: 3.1.0
>
> Attachments: HIVE-19130.01.patch
>
>
> During incremental replication, if we split the events batch as follows, then
> the REPL LOAD on second batch throws NPE.
> Batch-1: CREATE_TABLE(t1) -> ADD_PARTITION(t1.p1) -> DROP_PARTITION (t1.p1)
> Batch-2: DROP_TABLE(t1) -> CREATE_TABLE(t1) -> ADD_PARTITION(t1.p1) ->
> DROP_PARTITION (t1.p1)
> {code}
> 2018-04-05 16:20:36,531 ERROR [HiveServer2-Background-Pool: Thread-107044]:
> metadata.Hive (Hive.java:getTable(1219)) - Table catalog_sales_new not found:
> new5_tpcds_real_bin_partitioned_orc_1000.catalog_sales_new table not found
> 2018-04-05 16:20:36,538 ERROR [HiveServer2-Background-Pool: Thread-107044]:
> exec.DDLTask (DDLTask.java:failed(540)) -
> org.apache.hadoop.hive.ql.metadata.HiveException
> at
> org.apache.hadoop.hive.ql.exec.DDLTask.dropPartitions(DDLTask.java:4016)
> at
> org.apache.hadoop.hive.ql.exec.DDLTask.dropTableOrPartitions(DDLTask.java:3983)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:341)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:162)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:89)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1765)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1506)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1303)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1170)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1165)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:197)
> at
> org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:76)
> at
> org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
> at
> org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.metadata.Hive.getPartitionsByExpr(Hive.java:2613)
> at
> org.apache.hadoop.hive.ql.exec.DDLTask.dropPartitions(DDLTask.java:4008)
> ... 23 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19175) TestMiniLlapLocalCliDriver.testCliDriver update_access_time_non_current_db failing
[ https://issues.apache.org/jira/browse/HIVE-19175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19175: --- Status: Open (was: Patch Available) > TestMiniLlapLocalCliDriver.testCliDriver update_access_time_non_current_db > failing > -- > > Key: HIVE-19175 > URL: https://issues.apache.org/jira/browse/HIVE-19175 > Project: Hive > Issue Type: Sub-task > Components: Test >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19175.1.patch, HIVE-19175.2.patch > > > Caused by HIVE-18060. Instead of generating golden file under > clientpositive/llap it is under clientpositive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19175) TestMiniLlapLocalCliDriver.testCliDriver update_access_time_non_current_db failing
[ https://issues.apache.org/jira/browse/HIVE-19175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19175: --- Attachment: HIVE-19175.2.patch > TestMiniLlapLocalCliDriver.testCliDriver update_access_time_non_current_db > failing > -- > > Key: HIVE-19175 > URL: https://issues.apache.org/jira/browse/HIVE-19175 > Project: Hive > Issue Type: Sub-task > Components: Test >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19175.1.patch, HIVE-19175.2.patch > > > Caused by HIVE-18060. Instead of generating golden file under > clientpositive/llap it is under clientpositive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19175) TestMiniLlapLocalCliDriver.testCliDriver update_access_time_non_current_db failing
[ https://issues.apache.org/jira/browse/HIVE-19175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19175: --- Status: Patch Available (was: Open) > TestMiniLlapLocalCliDriver.testCliDriver update_access_time_non_current_db > failing > -- > > Key: HIVE-19175 > URL: https://issues.apache.org/jira/browse/HIVE-19175 > Project: Hive > Issue Type: Sub-task > Components: Test >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19175.1.patch, HIVE-19175.2.patch > > > Caused by HIVE-18060. Instead of generating golden file under > clientpositive/llap it is under clientpositive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19155) Day time saving cause Druid inserts to fail with org.apache.hive.druid.io.druid.java.util.common.UOE: Cannot add overlapping segments
[
https://issues.apache.org/jira/browse/HIVE-19155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434912#comment-16434912
]
Hive QA commented on HIVE-19155:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
47s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} druid-handler: The patch generated 2 new + 6 unchanged
- 0 fixed = 8 total (was 6) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 45s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10148/dev-support/hive-personality.sh
|
| git revision | master / 244ca8e |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10148/yetus/diff-checkstyle-druid-handler.txt
|
| modules | C: druid-handler ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10148/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Day time saving cause Druid inserts to fail with
> org.apache.hive.druid.io.druid.java.util.common.UOE: Cannot add overlapping
> segments
> -
>
> Key: HIVE-19155
> URL: https://issues.apache.org/jira/browse/HIVE-19155
> Project: Hive
> Issue Type: Bug
> Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-19155.patch
>
>
> If you try to insert data around the daylight saving time hour the query
> fails with following exception
> {code}
> 2018-04-10T11:24:58,836 ERROR [065fdaa2-85f9-4e49-adaf-3dc14d51be90 main]
> exec.DDLTask: Failed
> org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.hive.druid.io.druid.java.util.common.UOE: Cannot add overlapping
> segments [2015-03-08T05:00:00.000Z/2015-03-09T05:00:00.000Z and
> 2015-03-09T04:00:00.000Z/2015-03-10T04:00:00.000Z] with the same version
> [2018-04-10T11:24:48.388-07:00]
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:914)
> ~[hive-exec-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:919)
>
[jira] [Commented] (HIVE-19089) Create/Replicate Allocate write-id event
[
https://issues.apache.org/jira/browse/HIVE-19089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434897#comment-16434897
]
Hive QA commented on HIVE-19089:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12918427/HIVE-19089.06.patch
{color:green}SUCCESS:{color} +1 due to 5 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 37 failed/errored test(s), 13246 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestCopyUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=231)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=253)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=253)
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[jira] [Commented] (HIVE-19175) TestMiniLlapLocalCliDriver.testCliDriver update_access_time_non_current_db failing
[ https://issues.apache.org/jira/browse/HIVE-19175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434874#comment-16434874 ] Ashutosh Chauhan commented on HIVE-19175: - We shall move this test to minillaplocal.shared.query.files I don't think there is any additional coverage by running it on CliDriver. > TestMiniLlapLocalCliDriver.testCliDriver update_access_time_non_current_db > failing > -- > > Key: HIVE-19175 > URL: https://issues.apache.org/jira/browse/HIVE-19175 > Project: Hive > Issue Type: Sub-task > Components: Test >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19175.1.patch > > > Caused by HIVE-18060. Instead of generating golden file under > clientpositive/llap it is under clientpositive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19147) Fix PerfCliDrivers: Tpcds30T missed CAT_NAME change
[ https://issues.apache.org/jira/browse/HIVE-19147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-19147: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Pushed to master. Thanks, Zoltan! > Fix PerfCliDrivers: Tpcds30T missed CAT_NAME change > --- > > Key: HIVE-19147 > URL: https://issues.apache.org/jira/browse/HIVE-19147 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19147.01.patch > > > it seems the baked metastore dump misses the CAT_NAME field added by some > recent metastore change -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (HIVE-18883) Add findbugs to yetus pre-commit checks
[ https://issues.apache.org/jira/browse/HIVE-18883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar reopened HIVE-18883: - hmm seems this isn't working, will take a look > Add findbugs to yetus pre-commit checks > --- > > Key: HIVE-18883 > URL: https://issues.apache.org/jira/browse/HIVE-18883 > Project: Hive > Issue Type: Sub-task > Components: Testing Infrastructure >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-18883.1.patch, HIVE-18883.2.patch > > > We should enable FindBugs for our YETUS pre-commit checks, this will help > overall code quality and should decrease the overall number of bugs in Hive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19089) Create/Replicate Allocate write-id event
[
https://issues.apache.org/jira/browse/HIVE-19089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434856#comment-16434856
]
Hive QA commented on HIVE-19089:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
39s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
52s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
20s{color} | {color:red} server-extensions in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
20s{color} | {color:red} hcatalog-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
37s{color} | {color:red} ql in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
47s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
51s{color} | {color:red} ql: The patch generated 4 new + 742 unchanged - 6
fixed = 746 total (was 748) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
33s{color} | {color:red} standalone-metastore: The patch generated 12 new +
1557 unchanged - 12 fixed = 1569 total (was 1569) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 8 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10146/dev-support/hive-personality.sh
|
| git revision | master / b3fe652 |
| Default Java | 1.8.0_111 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10146/yetus/patch-mvninstall-hcatalog_server-extensions.txt
|
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10146/yetus/patch-mvninstall-itests_hcatalog-unit.txt
|
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10146/yetus/patch-mvninstall-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10146/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10146/yetus/diff-checkstyle-standalone-metastore.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10146/yetus/whitespace-eol.txt
|
| modules | C: hcatalog/server-extensions itests/hcatalog-unit ql
standalone-metastore U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10146/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Create/Replicate Allocate write-id event
>
>
> Key: HIVE-19089
> URL: https://issues.apache.org/jira/browse/HIVE-19089
>
[jira] [Commented] (HIVE-17824) msck repair table should drop the missing partitions from metastore
[
https://issues.apache.org/jira/browse/HIVE-17824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434831#comment-16434831
]
Hive QA commented on HIVE-17824:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12918408/HIVE-17824.3.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 78 failed/errored test(s), 13677 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=253)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=253)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[jira] [Commented] (HIVE-17824) msck repair table should drop the missing partitions from metastore
[
https://issues.apache.org/jira/browse/HIVE-17824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434780#comment-16434780
]
Hive QA commented on HIVE-17824:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
49s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
24s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 2s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10145/dev-support/hive-personality.sh
|
| git revision | master / b3fe652 |
| Default Java | 1.8.0_111 |
| modules | C: itests/util ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10145/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> msck repair table should drop the missing partitions from metastore
> ---
>
> Key: HIVE-17824
> URL: https://issues.apache.org/jira/browse/HIVE-17824
> Project: Hive
> Issue Type: Improvement
>Reporter: Vihang Karajgaonkar
>Assignee: Janaki Lahorani
>Priority: Major
> Attachments: HIVE-17824.1.patch, HIVE-17824.2.patch,
> HIVE-17824.3.patch
>
>
> {{msck repair table }} is often used in environments where the new
> partitions are loaded as directories on HDFS or S3 and users want to create
> the missing partitions in bulk. However, currently it only supports addition
> of missing partitions. If there are any partitions which are present in
> metastore but not on the FileSystem, it should also delete them so that it
> truly repairs the table metadata.
> We should be careful not to break backwards compatibility so we should either
> introduce a new config or keyword to add support to delete unnecessary
> partitions from the metastore. This way users who want the old behavior can
> easily turn it off.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19187) Update Druid Storage Handler to Druid 0.12.0
[ https://issues.apache.org/jira/browse/HIVE-19187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-19187: -- Attachment: HIVE-19187.patch > Update Druid Storage Handler to Druid 0.12.0 > > > Key: HIVE-19187 > URL: https://issues.apache.org/jira/browse/HIVE-19187 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19187.patch > > > Current used Druid Version is 0.11.0 > This Patch updates the Druid version to the most recent version 0.12.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19187) Update Druid Storage Handler to Druid 0.12.0
[ https://issues.apache.org/jira/browse/HIVE-19187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-19187: -- Status: Patch Available (was: Open) > Update Druid Storage Handler to Druid 0.12.0 > > > Key: HIVE-19187 > URL: https://issues.apache.org/jira/browse/HIVE-19187 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Fix For: 3.1.0 > > > Current used Druid Version is 0.11.0 > This Patch updates the Druid version to the most recent version 0.12.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19187) Update Druid Storage Handler to Druid 0.12.0
[ https://issues.apache.org/jira/browse/HIVE-19187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra reassigned HIVE-19187: - > Update Druid Storage Handler to Druid 0.12.0 > > > Key: HIVE-19187 > URL: https://issues.apache.org/jira/browse/HIVE-19187 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Fix For: 3.1.0 > > > Current used Druid Version is 0.11.0 > This Patch updates the Druid version to the most recent version 0.12.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434762#comment-16434762
]
Hive QA commented on HIVE-17502:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12918499/HIVE-17502.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 87 failed/errored test(s), 13625 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestCopyUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=231)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=253)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=253)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=96)
[jira] [Updated] (HIVE-19186) Multi Table INSERT statements query has a flaw for partitioned table when INSERT INTO and INSERT OVERWRITE are used
[ https://issues.apache.org/jira/browse/HIVE-19186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Yeom updated HIVE-19186: -- Summary: Multi Table INSERT statements query has a flaw for partitioned table when INSERT INTO and INSERT OVERWRITE are used (was: Multi Table INSERT statements query have a flaw for partitioned table when INSERT INTO and INSERT OVERWRITE are used) > Multi Table INSERT statements query has a flaw for partitioned table when > INSERT INTO and INSERT OVERWRITE are used > --- > > Key: HIVE-19186 > URL: https://issues.apache.org/jira/browse/HIVE-19186 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Priority: Major > Fix For: 3.0.0 > > > One problem test case is: > create table intermediate(key int) partitioned by (p int) stored as orc; > insert into table intermediate partition(p='455') select distinct key from > src where key >= 0 order by key desc limit 2; > insert into table intermediate partition(p='456') select distinct key from > src where key is not null order by key asc limit 2; > insert into table intermediate partition(p='457') select distinct key from > src where key >= 100 order by key asc limit 2; > from intermediate > insert into table multi_partitioned partition(p=2) select p, key > insert overwrite table multi_partitioned partition(p=1) select key, p; -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19186) Multi Table INSERT statements query has a flaw for partitioned table when INSERT INTO and INSERT OVERWRITE are used
[ https://issues.apache.org/jira/browse/HIVE-19186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Yeom reassigned HIVE-19186: - Assignee: Steve Yeom > Multi Table INSERT statements query has a flaw for partitioned table when > INSERT INTO and INSERT OVERWRITE are used > --- > > Key: HIVE-19186 > URL: https://issues.apache.org/jira/browse/HIVE-19186 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 3.0.0 > > > One problem test case is: > create table intermediate(key int) partitioned by (p int) stored as orc; > insert into table intermediate partition(p='455') select distinct key from > src where key >= 0 order by key desc limit 2; > insert into table intermediate partition(p='456') select distinct key from > src where key is not null order by key asc limit 2; > insert into table intermediate partition(p='457') select distinct key from > src where key >= 100 order by key asc limit 2; > from intermediate > insert into table multi_partitioned partition(p=2) select p, key > insert overwrite table multi_partitioned partition(p=1) select key, p; -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19184) Hive 3.0.0 release branch preparation
[ https://issues.apache.org/jira/browse/HIVE-19184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19184: --- Summary: Hive 3.0.0 release branch preparation (was: Hive 3.0.0 release preparation) > Hive 3.0.0 release branch preparation > - > > Key: HIVE-19184 > URL: https://issues.apache.org/jira/browse/HIVE-19184 > Project: Hive > Issue Type: Task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19184-branch-3.patch > > > Need to do bunch of things to prepare branch-3 for release e.g. > * Update pom.xml to delete SNAPSHOT > * Update .reviewboardrc > * Remove storage-api module to build > * Change storage-api depdency etc -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19184) Hive 3.0.0 release preparation
[ https://issues.apache.org/jira/browse/HIVE-19184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19184: --- Attachment: HIVE-19184-branch-3.patch > Hive 3.0.0 release preparation > -- > > Key: HIVE-19184 > URL: https://issues.apache.org/jira/browse/HIVE-19184 > Project: Hive > Issue Type: Task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19184-branch-3.patch > > > Need to do bunch of things to prepare branch-3 for release e.g. > * Update pom.xml to delete SNAPSHOT > * Update .reviewboardrc > * Remove storage-api module to build > * Change storage-api depdency etc -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19184) Hive 3.0.0 release preparation
[ https://issues.apache.org/jira/browse/HIVE-19184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19184: --- Status: Patch Available (was: Open) > Hive 3.0.0 release preparation > -- > > Key: HIVE-19184 > URL: https://issues.apache.org/jira/browse/HIVE-19184 > Project: Hive > Issue Type: Task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19184-branch-3.patch > > > Need to do bunch of things to prepare branch-3 for release e.g. > * Update pom.xml to delete SNAPSHOT > * Update .reviewboardrc > * Remove storage-api module to build > * Change storage-api depdency etc -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19184) Hive 3.0.0 release preparation
[ https://issues.apache.org/jira/browse/HIVE-19184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg reassigned HIVE-19184: -- > Hive 3.0.0 release preparation > -- > > Key: HIVE-19184 > URL: https://issues.apache.org/jira/browse/HIVE-19184 > Project: Hive > Issue Type: Task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > > Need to do bunch of things to prepare branch-3 for release e.g. > * Update pom.xml to delete SNAPSHOT > * Update .reviewboardrc > * Remove storage-api module to build > * Change storage-api depdency etc -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18652) Print Spark metrics on console
[
https://issues.apache.org/jira/browse/HIVE-18652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-18652:
Status: Patch Available (was: Open)
> Print Spark metrics on console
> --
>
> Key: HIVE-18652
> URL: https://issues.apache.org/jira/browse/HIVE-18652
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18652.1.patch
>
>
> For Hive-on-MR, each MR job launched prints out some stats about the job:
> {code}
> INFO : 2018-02-07 17:51:11,218 Stage-1 map = 0%, reduce = 0%
> INFO : 2018-02-07 17:51:18,396 Stage-1 map = 100%, reduce = 0%, Cumulative
> CPU 1.87 sec
> INFO : 2018-02-07 17:51:25,742 Stage-1 map = 100%, reduce = 100%,
> Cumulative CPU 4.34 sec
> INFO : MapReduce Total cumulative CPU time: 4 seconds 340 msec
> INFO : Ended Job = job_1517865654989_0004
> INFO : MapReduce Jobs Launched:
> INFO : Stage-Stage-1: Map: 1 Reduce: 1 Cumulative CPU: 4.34 sec HDFS
> Read: 7353 HDFS Write: 151 SUCCESS
> INFO : Total MapReduce CPU Time Spent: 4 seconds 340 msec
> {code}
> We should do the same for Hive-on-Spark.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18652) Print Spark metrics on console
[
https://issues.apache.org/jira/browse/HIVE-18652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-18652:
Attachment: HIVE-18652.1.patch
> Print Spark metrics on console
> --
>
> Key: HIVE-18652
> URL: https://issues.apache.org/jira/browse/HIVE-18652
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18652.1.patch
>
>
> For Hive-on-MR, each MR job launched prints out some stats about the job:
> {code}
> INFO : 2018-02-07 17:51:11,218 Stage-1 map = 0%, reduce = 0%
> INFO : 2018-02-07 17:51:18,396 Stage-1 map = 100%, reduce = 0%, Cumulative
> CPU 1.87 sec
> INFO : 2018-02-07 17:51:25,742 Stage-1 map = 100%, reduce = 100%,
> Cumulative CPU 4.34 sec
> INFO : MapReduce Total cumulative CPU time: 4 seconds 340 msec
> INFO : Ended Job = job_1517865654989_0004
> INFO : MapReduce Jobs Launched:
> INFO : Stage-Stage-1: Map: 1 Reduce: 1 Cumulative CPU: 4.34 sec HDFS
> Read: 7353 HDFS Write: 151 SUCCESS
> INFO : Total MapReduce CPU Time Spent: 4 seconds 340 msec
> {code}
> We should do the same for Hive-on-Spark.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18652) Print Spark metrics on console
[
https://issues.apache.org/jira/browse/HIVE-18652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar reassigned HIVE-18652:
---
Assignee: Sahil Takiar
> Print Spark metrics on console
> --
>
> Key: HIVE-18652
> URL: https://issues.apache.org/jira/browse/HIVE-18652
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18652.1.patch
>
>
> For Hive-on-MR, each MR job launched prints out some stats about the job:
> {code}
> INFO : 2018-02-07 17:51:11,218 Stage-1 map = 0%, reduce = 0%
> INFO : 2018-02-07 17:51:18,396 Stage-1 map = 100%, reduce = 0%, Cumulative
> CPU 1.87 sec
> INFO : 2018-02-07 17:51:25,742 Stage-1 map = 100%, reduce = 100%,
> Cumulative CPU 4.34 sec
> INFO : MapReduce Total cumulative CPU time: 4 seconds 340 msec
> INFO : Ended Job = job_1517865654989_0004
> INFO : MapReduce Jobs Launched:
> INFO : Stage-Stage-1: Map: 1 Reduce: 1 Cumulative CPU: 4.34 sec HDFS
> Read: 7353 HDFS Write: 151 SUCCESS
> INFO : Total MapReduce CPU Time Spent: 4 seconds 340 msec
> {code}
> We should do the same for Hive-on-Spark.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434698#comment-16434698
]
Hive QA commented on HIVE-17502:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
50s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
22s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 19m 21s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10144/dev-support/hive-personality.sh
|
| git revision | master / b3fe652 |
| Default Java | 1.8.0_111 |
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10144/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-17502.2.patch, HIVE-17502.3.patch, HIVE-17502.patch
>
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
>
[jira] [Resolved] (HIVE-15310) CTAS fails when target location contains multiple directories levels that don't exist
[
https://issues.apache.org/jira/browse/HIVE-15310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar resolved HIVE-15310.
-
Resolution: Duplicate
> CTAS fails when target location contains multiple directories levels that
> don't exist
> -
>
> Key: HIVE-15310
> URL: https://issues.apache.org/jira/browse/HIVE-15310
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Sahil Takiar
>Assignee: Janaki Lahorani
>Priority: Major
>
> The second query below fails, if the {{/tmp/}} directory is empty:
> {code}
> create table test1 (id int) location '/tmp/test1/one/two/';
> create table test2 location '/tmp/test2/one/two' as select * from test1;
> {code}
> The stacktrace is:
> {code}
> Error: org.apache.hive.service.cli.HiveSQLException: Error while processing
> statement: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.MoveTask. Unable to move source
> file:/var/folders/jb/350gyf853s91hk5xk06jyw2wgp/T/stakiar/f6c5c246-3209-4d76-aaf4-1ac91436f59a/hive_2016-11-29_14-02-33_047_219358498634208469-1/-mr-10002
> to destination /tmp/test2/one/two
> at
> org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:393)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:250)
> at
> org.apache.hive.service.cli.operation.SQLOperation.access$800(SQLOperation.java:91)
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:340)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:353)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to move
> source
> file:/var/folders/jb/350gyf853s91hk5xk06jyw2wgp/T/stakiar/f6c5c246-3209-4d76-aaf4-1ac91436f59a/hive_2016-11-29_14-02-33_047_219358498634208469-1/-mr-10002
> to destination /tmp/test2/one/two
> at org.apache.hadoop.hive.ql.exec.MoveTask.moveFile(MoveTask.java:104)
> at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:263)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2166)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1822)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1510)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1221)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1216)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:248)
> ... 11 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to move
> source
> file:/var/folders/jb/350gyf853s91hk5xk06jyw2wgp/T/stakiar/f6c5c246-3209-4d76-aaf4-1ac91436f59a/hive_2016-11-29_14-02-33_047_219358498634208469-1/-mr-10002
> to destination /tmp/test2/one/two
> at org.apache.hadoop.hive.ql.metadata.Hive.moveFile(Hive.java:3119)
> at
> org.apache.hadoop.hive.ql.exec.MoveTask.moveFileInDfs(MoveTask.java:120)
> at org.apache.hadoop.hive.ql.exec.MoveTask.moveFile(MoveTask.java:97)
> ... 20 more
> Caused by: java.io.FileNotFoundException: File /tmp/test2/one does not exist
> at
> org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:609)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:822)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:599)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:421)
> at
> org.apache.hadoop.hive.io.HdfsUtils$HadoopFileStatus.(HdfsUtils.java:178)
> at org.apache.hadoop.hive.ql.metadata.Hive.moveFile(Hive.java:3029)
> ... 22 more (state=08S01,code=1)
> {code}
> The second query works if the target location is simply {{/tmp/test2/}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-15310) CTAS fails when target location contains multiple directories levels that don't exist
[
https://issues.apache.org/jira/browse/HIVE-15310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434692#comment-16434692
]
Sahil Takiar commented on HIVE-15310:
-
Yes, seems this has been fixed on master. Not sure which change fixed it, but
will close it as a Duplicate.
> CTAS fails when target location contains multiple directories levels that
> don't exist
> -
>
> Key: HIVE-15310
> URL: https://issues.apache.org/jira/browse/HIVE-15310
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Sahil Takiar
>Assignee: Janaki Lahorani
>Priority: Major
>
> The second query below fails, if the {{/tmp/}} directory is empty:
> {code}
> create table test1 (id int) location '/tmp/test1/one/two/';
> create table test2 location '/tmp/test2/one/two' as select * from test1;
> {code}
> The stacktrace is:
> {code}
> Error: org.apache.hive.service.cli.HiveSQLException: Error while processing
> statement: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.MoveTask. Unable to move source
> file:/var/folders/jb/350gyf853s91hk5xk06jyw2wgp/T/stakiar/f6c5c246-3209-4d76-aaf4-1ac91436f59a/hive_2016-11-29_14-02-33_047_219358498634208469-1/-mr-10002
> to destination /tmp/test2/one/two
> at
> org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:393)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:250)
> at
> org.apache.hive.service.cli.operation.SQLOperation.access$800(SQLOperation.java:91)
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:340)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:353)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to move
> source
> file:/var/folders/jb/350gyf853s91hk5xk06jyw2wgp/T/stakiar/f6c5c246-3209-4d76-aaf4-1ac91436f59a/hive_2016-11-29_14-02-33_047_219358498634208469-1/-mr-10002
> to destination /tmp/test2/one/two
> at org.apache.hadoop.hive.ql.exec.MoveTask.moveFile(MoveTask.java:104)
> at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:263)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2166)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1822)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1510)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1221)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1216)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:248)
> ... 11 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to move
> source
> file:/var/folders/jb/350gyf853s91hk5xk06jyw2wgp/T/stakiar/f6c5c246-3209-4d76-aaf4-1ac91436f59a/hive_2016-11-29_14-02-33_047_219358498634208469-1/-mr-10002
> to destination /tmp/test2/one/two
> at org.apache.hadoop.hive.ql.metadata.Hive.moveFile(Hive.java:3119)
> at
> org.apache.hadoop.hive.ql.exec.MoveTask.moveFileInDfs(MoveTask.java:120)
> at org.apache.hadoop.hive.ql.exec.MoveTask.moveFile(MoveTask.java:97)
> ... 20 more
> Caused by: java.io.FileNotFoundException: File /tmp/test2/one does not exist
> at
> org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:609)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:822)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:599)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:421)
> at
> org.apache.hadoop.hive.io.HdfsUtils$HadoopFileStatus.(HdfsUtils.java:178)
> at org.apache.hadoop.hive.ql.metadata.Hive.moveFile(Hive.java:3029)
> ... 22 more (state=08S01,code=1)
> {code}
> The second query works if the target location is simply
[jira] [Updated] (HIVE-19126) CachedStore: Use memory estimation to limit cache size during prewarm
[ https://issues.apache.org/jira/browse/HIVE-19126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-19126: Attachment: HIVE-19126.5.patch > CachedStore: Use memory estimation to limit cache size during prewarm > - > > Key: HIVE-19126 > URL: https://issues.apache.org/jira/browse/HIVE-19126 > Project: Hive > Issue Type: Sub-task > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta >Priority: Major > Attachments: HIVE-19126.1.patch, HIVE-19126.2.patch, > HIVE-19126.3.patch, HIVE-19126.4.patch, HIVE-19126.5.patch > > > We can rely on > https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/IncrementalObjectSizeEstimator.java > to estimate memory of SharedCache. This jira addresses the size estimation > during prewarm, so that we can stop when we hit the memory limit. In a > follow-up jira, we will work on estimation/eviction after prewarm is > complete, so that we can keep the frequently used tables and their partitions > in cache. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19126) CachedStore: Use memory estimation to limit cache size during prewarm
[ https://issues.apache.org/jira/browse/HIVE-19126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-19126: Attachment: (was: HIVE-19126.5.patch) > CachedStore: Use memory estimation to limit cache size during prewarm > - > > Key: HIVE-19126 > URL: https://issues.apache.org/jira/browse/HIVE-19126 > Project: Hive > Issue Type: Sub-task > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta >Priority: Major > Attachments: HIVE-19126.1.patch, HIVE-19126.2.patch, > HIVE-19126.3.patch, HIVE-19126.4.patch, HIVE-19126.5.patch > > > We can rely on > https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/IncrementalObjectSizeEstimator.java > to estimate memory of SharedCache. This jira addresses the size estimation > during prewarm, so that we can stop when we hit the memory limit. In a > follow-up jira, we will work on estimation/eviction after prewarm is > complete, so that we can keep the frequently used tables and their partitions > in cache. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19104) When test MetaStore is started with retry the instances should be independent
[
https://issues.apache.org/jira/browse/HIVE-19104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434669#comment-16434669
]
Hive QA commented on HIVE-19104:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12918382/HIVE-19104.3.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 90 failed/errored test(s), 13658 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=253)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=253)
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=96)
[jira] [Commented] (HIVE-15310) CTAS fails when target location contains multiple directories levels that don't exist
[
https://issues.apache.org/jira/browse/HIVE-15310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434670#comment-16434670
]
Janaki Lahorani commented on HIVE-15310:
[~stakiar]
The following test works in latest Master Branch:
create table test1 (id int) location '/tmp/test1/one/two/';
create table test2 location '/tmp/test2/one/two' as select * from test1;
Please let me know if I am missing something.
> CTAS fails when target location contains multiple directories levels that
> don't exist
> -
>
> Key: HIVE-15310
> URL: https://issues.apache.org/jira/browse/HIVE-15310
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Sahil Takiar
>Assignee: Janaki Lahorani
>Priority: Major
>
> The second query below fails, if the {{/tmp/}} directory is empty:
> {code}
> create table test1 (id int) location '/tmp/test1/one/two/';
> create table test2 location '/tmp/test2/one/two' as select * from test1;
> {code}
> The stacktrace is:
> {code}
> Error: org.apache.hive.service.cli.HiveSQLException: Error while processing
> statement: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.MoveTask. Unable to move source
> file:/var/folders/jb/350gyf853s91hk5xk06jyw2wgp/T/stakiar/f6c5c246-3209-4d76-aaf4-1ac91436f59a/hive_2016-11-29_14-02-33_047_219358498634208469-1/-mr-10002
> to destination /tmp/test2/one/two
> at
> org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:393)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:250)
> at
> org.apache.hive.service.cli.operation.SQLOperation.access$800(SQLOperation.java:91)
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:340)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:353)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to move
> source
> file:/var/folders/jb/350gyf853s91hk5xk06jyw2wgp/T/stakiar/f6c5c246-3209-4d76-aaf4-1ac91436f59a/hive_2016-11-29_14-02-33_047_219358498634208469-1/-mr-10002
> to destination /tmp/test2/one/two
> at org.apache.hadoop.hive.ql.exec.MoveTask.moveFile(MoveTask.java:104)
> at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:263)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2166)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1822)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1510)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1221)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1216)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:248)
> ... 11 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to move
> source
> file:/var/folders/jb/350gyf853s91hk5xk06jyw2wgp/T/stakiar/f6c5c246-3209-4d76-aaf4-1ac91436f59a/hive_2016-11-29_14-02-33_047_219358498634208469-1/-mr-10002
> to destination /tmp/test2/one/two
> at org.apache.hadoop.hive.ql.metadata.Hive.moveFile(Hive.java:3119)
> at
> org.apache.hadoop.hive.ql.exec.MoveTask.moveFileInDfs(MoveTask.java:120)
> at org.apache.hadoop.hive.ql.exec.MoveTask.moveFile(MoveTask.java:97)
> ... 20 more
> Caused by: java.io.FileNotFoundException: File /tmp/test2/one does not exist
> at
> org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:609)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:822)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:599)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:421)
> at
> org.apache.hadoop.hive.io.HdfsUtils$HadoopFileStatus.(HdfsUtils.java:178)
> at
[jira] [Updated] (HIVE-18840) CachedStore: Prioritize loading of recently accessed tables during prewarm
[ https://issues.apache.org/jira/browse/HIVE-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-18840: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.1.0 3.0.0 Status: Resolved (was: Patch Available) Committed to master and branch-3. Thanks [~daijy]. > CachedStore: Prioritize loading of recently accessed tables during prewarm > -- > > Key: HIVE-18840 > URL: https://issues.apache.org/jira/browse/HIVE-18840 > Project: Hive > Issue Type: Sub-task > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta >Priority: Major > Fix For: 3.0.0, 3.1.0 > > Attachments: HIVE-18840.1.patch, HIVE-18840.2.patch, > HIVE-18840.2.patch, HIVE-18840.2.patch, HIVE-18840.2.patch > > > On clusters with large metadata, prewarming the cache can take several hours. > Now that CachedStore does not block on prewarm anymore (after HIVE-18264), we > should prioritize loading of recently accessed tables during prewarm. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19126) CachedStore: Use memory estimation to limit cache size during prewarm
[ https://issues.apache.org/jira/browse/HIVE-19126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-19126: Attachment: HIVE-19126.5.patch > CachedStore: Use memory estimation to limit cache size during prewarm > - > > Key: HIVE-19126 > URL: https://issues.apache.org/jira/browse/HIVE-19126 > Project: Hive > Issue Type: Sub-task > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta >Priority: Major > Attachments: HIVE-19126.1.patch, HIVE-19126.2.patch, > HIVE-19126.3.patch, HIVE-19126.4.patch, HIVE-19126.5.patch > > > We can rely on > https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/IncrementalObjectSizeEstimator.java > to estimate memory of SharedCache. This jira addresses the size estimation > during prewarm, so that we can stop when we hit the memory limit. In a > follow-up jira, we will work on estimation/eviction after prewarm is > complete, so that we can keep the frequently used tables and their partitions > in cache. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19161) Add authorizations to information schema
[ https://issues.apache.org/jira/browse/HIVE-19161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434660#comment-16434660 ] Daniel Dai commented on HIVE-19161: --- Also include a flag which would not restrict information schema (old behavior). > Add authorizations to information schema > > > Key: HIVE-19161 > URL: https://issues.apache.org/jira/browse/HIVE-19161 > Project: Hive > Issue Type: Improvement >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-19161.1.patch, HIVE-19161.2.patch, > HIVE-19161.3.patch > > > We need to control the access of information schema so user can only query > the information authorized to. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19161) Add authorizations to information schema
[ https://issues.apache.org/jira/browse/HIVE-19161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-19161: -- Attachment: HIVE-19161.3.patch > Add authorizations to information schema > > > Key: HIVE-19161 > URL: https://issues.apache.org/jira/browse/HIVE-19161 > Project: Hive > Issue Type: Improvement >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-19161.1.patch, HIVE-19161.2.patch, > HIVE-19161.3.patch > > > We need to control the access of information schema so user can only query > the information authorized to. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19098) Hive: impossible to insert data in a parquet's table with "union all" in the select query
[
https://issues.apache.org/jira/browse/HIVE-19098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434616#comment-16434616
]
Janaki Lahorani commented on HIVE-19098:
[~SDAT/SACT/IET]
The following test works in master branch:
create table t1 (col1 string);
create table t2 (col1 string);
insert into t2 values ('2017');
insert into t1 values ('2017');
create table t3 (col1 string) STORED AS PARQUETFILE;
INSERT into t3 select col1 from t1 union all select col1 from t2;
select * from t3;
Please confirm the version you tried this on.
> Hive: impossible to insert data in a parquet's table with "union all" in the
> select query
> -
>
> Key: HIVE-19098
> URL: https://issues.apache.org/jira/browse/HIVE-19098
> Project: Hive
> Issue Type: Bug
> Components: File Formats, Hive
>Affects Versions: 2.3.2
>Reporter: ACOSS
>Assignee: Janaki Lahorani
>Priority: Minor
>
> Hello
> We have a parquet's table.
> We want to insert data in the table by a querie like this:
> "insert into my_table select * from my_select_table_1 union all select * from
> my_select_table_2"
> It's fail with the error:
> 2018-04-03 15:49:28,898 FATAL [IPC Server handler 2 on 38465]
> org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task:
> attempt_1522749003448_0028_m_00_0 - exited : java.io.IOException:
> java.lang.reflect.InvocationTargetException
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
> at
> org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:271)
> at
> org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.(HadoopShimsSecure.java:217)
> at
> org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getRecordReader(HadoopShimsSecure.java:345)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:695)
> at
> org.apache.hadoop.mapred.MapTask$TrackedRecordReader.(MapTask.java:169)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:257)
> ... 11 more
> Caused by: java.lang.NullPointerException
> at java.util.AbstractCollection.addAll(AbstractCollection.java:343)
> at
> org.apache.hadoop.hive.ql.io.parquet.ProjectionPusher.pushProjectionsAndFilters(ProjectionPusher.java:118)
> at
> org.apache.hadoop.hive.ql.io.parquet.ProjectionPusher.pushProjectionsAndFilters(ProjectionPusher.java:189)
> at
> org.apache.hadoop.hive.ql.io.parquet.ParquetRecordReaderBase.getSplit(ParquetRecordReaderBase.java:75)
> at
> org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.(ParquetRecordReaderWrapper.java:75)
> at
> org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.(ParquetRecordReaderWrapper.java:60)
> at
> org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat.getRecordReader(MapredParquetInputFormat.java:75)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.(CombineHiveRecordReader.java:99)
> ... 16 more
>
> Scenario:
> create table t1 (col1 string);
> create table t2 (col1 string);
> insert into t2 values ('2017');
> insert into t1 values ('2017');
> create table t3 (col1 string) STORED AS PARQUETFILE;
> INSERT into t3 select col1 from t1 union all select col1 from t2;
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19161) Add authorizations to information schema
[ https://issues.apache.org/jira/browse/HIVE-19161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434613#comment-16434613 ] Thejas M Nair commented on HIVE-19161: -- [~daijy] Can you please include a review board link or pull request ? > Add authorizations to information schema > > > Key: HIVE-19161 > URL: https://issues.apache.org/jira/browse/HIVE-19161 > Project: Hive > Issue Type: Improvement >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-19161.1.patch, HIVE-19161.2.patch > > > We need to control the access of information schema so user can only query > the information authorized to. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19182) Code clean up of HiveMetaStoreChecker.java
[ https://issues.apache.org/jira/browse/HIVE-19182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janaki Lahorani reassigned HIVE-19182: -- > Code clean up of HiveMetaStoreChecker.java > -- > > Key: HIVE-19182 > URL: https://issues.apache.org/jira/browse/HIVE-19182 > Project: Hive > Issue Type: Improvement >Reporter: Janaki Lahorani >Assignee: Janaki Lahorani >Priority: Major > > HiveMetaStoreChecker.java has code like: > result.getTablesNotOnFs().add(table.getTableName()); > result.getPartitionsNotOnFs().add(pr); > result.getPartitionsNotInMs().add(pr); > It is not a good practice to change internal states like this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19181) Remove BreakableService (unused class)
[ https://issues.apache.org/jira/browse/HIVE-19181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-19181: - Labels: newbie trivial (was: ) > Remove BreakableService (unused class) > -- > > Key: HIVE-19181 > URL: https://issues.apache.org/jira/browse/HIVE-19181 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0, 2.3.2 >Reporter: Thejas M Nair >Priority: Major > Labels: newbie, trivial > > BreakableService.java is not used anywhere -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19104) When test MetaStore is started with retry the instances should be independent
[
https://issues.apache.org/jira/browse/HIVE-19104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434599#comment-16434599
]
Hive QA commented on HIVE-19104:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
23s{color} | {color:red} standalone-metastore: The patch generated 2 new + 263
unchanged - 2 fixed = 265 total (was 265) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10143/dev-support/hive-personality.sh
|
| git revision | master / 42187fd |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10143/yetus/diff-checkstyle-standalone-metastore.txt
|
| modules | C: standalone-metastore U: standalone-metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10143/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> When test MetaStore is started with retry the instances should be independent
> -
>
> Key: HIVE-19104
> URL: https://issues.apache.org/jira/browse/HIVE-19104
> Project: Hive
> Issue Type: Improvement
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-19104.2.patch, HIVE-19104.3.patch, HIVE-19104.patch
>
>
> When multiple MetaStore instances are started with
> {{MetaStoreTestUtils.startMetaStoreWithRetry}} currently they use the same
> JDBC url, and warehouse directory. This can cause problem in the tests
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19130) NPE is thrown when REPL LOAD applied drop partition event.
[
https://issues.apache.org/jira/browse/HIVE-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434573#comment-16434573
]
Hive QA commented on HIVE-19130:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12918373/HIVE-19130.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 78 failed/errored test(s), 13659 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=253)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=253)
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[jira] [Updated] (HIVE-18845) SHOW COMAPCTIONS should show host name
[ https://issues.apache.org/jira/browse/HIVE-18845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Kryvenko updated HIVE-18845: - Attachment: HIVE-18845.01.patch > SHOW COMAPCTIONS should show host name > -- > > Key: HIVE-18845 > URL: https://issues.apache.org/jira/browse/HIVE-18845 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Igor Kryvenko >Priority: Minor > Attachments: HIVE-18845.01.patch > > > once the job starts, the WorkerId includes the hostname submitting the job > but before that there is no way to tell which of the Metastores in HA set up > has picked up a given item to compact. Should make it obvious to know which > log to look at. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18845) SHOW COMAPCTIONS should show host name
[ https://issues.apache.org/jira/browse/HIVE-18845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Kryvenko updated HIVE-18845: - Status: Patch Available (was: Open) > SHOW COMAPCTIONS should show host name > -- > > Key: HIVE-18845 > URL: https://issues.apache.org/jira/browse/HIVE-18845 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Igor Kryvenko >Priority: Minor > Attachments: HIVE-18845.01.patch > > > once the job starts, the WorkerId includes the hostname submitting the job > but before that there is no way to tell which of the Metastores in HA set up > has picked up a given item to compact. Should make it obvious to know which > log to look at. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18845) SHOW COMAPCTIONS should show host name
[ https://issues.apache.org/jira/browse/HIVE-18845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Kryvenko reassigned HIVE-18845: Assignee: Igor Kryvenko (was: Igor Kryvenko) > SHOW COMAPCTIONS should show host name > -- > > Key: HIVE-18845 > URL: https://issues.apache.org/jira/browse/HIVE-18845 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Igor Kryvenko >Priority: Minor > Attachments: HIVE-18845.01.patch > > > once the job starts, the WorkerId includes the hostname submitting the job > but before that there is no way to tell which of the Metastores in HA set up > has picked up a given item to compact. Should make it obvious to know which > log to look at. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18910) Migrate to Murmur hash for shuffle and bucketing
[ https://issues.apache.org/jira/browse/HIVE-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Jaiswal updated HIVE-18910: -- Attachment: HIVE-18910.29.patch > Migrate to Murmur hash for shuffle and bucketing > > > Key: HIVE-18910 > URL: https://issues.apache.org/jira/browse/HIVE-18910 > Project: Hive > Issue Type: Task >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-18910.1.patch, HIVE-18910.10.patch, > HIVE-18910.11.patch, HIVE-18910.12.patch, HIVE-18910.13.patch, > HIVE-18910.14.patch, HIVE-18910.15.patch, HIVE-18910.16.patch, > HIVE-18910.17.patch, HIVE-18910.18.patch, HIVE-18910.19.patch, > HIVE-18910.2.patch, HIVE-18910.20.patch, HIVE-18910.21.patch, > HIVE-18910.22.patch, HIVE-18910.23.patch, HIVE-18910.24.patch, > HIVE-18910.25.patch, HIVE-18910.26.patch, HIVE-18910.27.patch, > HIVE-18910.28.patch, HIVE-18910.29.patch, HIVE-18910.3.patch, > HIVE-18910.4.patch, HIVE-18910.5.patch, HIVE-18910.6.patch, > HIVE-18910.7.patch, HIVE-18910.8.patch, HIVE-18910.9.patch > > > Hive uses JAVA hash which is not as good as murmur for better distribution > and efficiency in bucketing a table. > Migrate to murmur hash but still keep backward compatibility for existing > users so that they dont have to reload the existing tables. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18845) SHOW COMAPCTIONS should show host name
[ https://issues.apache.org/jira/browse/HIVE-18845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434562#comment-16434562 ] Eugene Koifman commented on HIVE-18845: --- all yours > SHOW COMAPCTIONS should show host name > -- > > Key: HIVE-18845 > URL: https://issues.apache.org/jira/browse/HIVE-18845 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Igor Kryvenko >Priority: Minor > > once the job starts, the WorkerId includes the hostname submitting the job > but before that there is no way to tell which of the Metastores in HA set up > has picked up a given item to compact. Should make it obvious to know which > log to look at. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18845) SHOW COMAPCTIONS should show host name
[ https://issues.apache.org/jira/browse/HIVE-18845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman reassigned HIVE-18845: - Assignee: Igor Kryvenko (was: Eugene Koifman) > SHOW COMAPCTIONS should show host name > -- > > Key: HIVE-18845 > URL: https://issues.apache.org/jira/browse/HIVE-18845 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Igor Kryvenko >Priority: Minor > > once the job starts, the WorkerId includes the hostname submitting the job > but before that there is no way to tell which of the Metastores in HA set up > has picked up a given item to compact. Should make it obvious to know which > log to look at. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18845) SHOW COMAPCTIONS should show host name
[ https://issues.apache.org/jira/browse/HIVE-18845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434554#comment-16434554 ] Igor Kryvenko commented on HIVE-18845: -- [~ekoifman] Hi. May i take this JIRA? I have patch for it. > SHOW COMAPCTIONS should show host name > -- > > Key: HIVE-18845 > URL: https://issues.apache.org/jira/browse/HIVE-18845 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Minor > > once the job starts, the WorkerId includes the hostname submitting the job > but before that there is no way to tell which of the Metastores in HA set up > has picked up a given item to compact. Should make it obvious to know which > log to look at. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18807) Fix broken tests caused by HIVE-18493
[ https://issues.apache.org/jira/browse/HIVE-18807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434539#comment-16434539 ] Vineet Garg commented on HIVE-18807: [~mmccline] Are following failures due to HIVE-18493 as well? * TestBeeLineWithArgs.testEscapeCRLFOffInDSVOutput * TestBeeLineWithArgs.testEscapeCRLFInTSV2Output * TestBeeLineWithArgs.testEscapeCRLFInDSVOutpu https://builds.apache.org/job/PreCommit-HIVE-Build/10140/testReport/org.apache.hive.beeline/TestBeeLineWithArgs/testEscapeCRLFInDSVOutput/ https://builds.apache.org/job/PreCommit-HIVE-Build/10138/testReport/org.apache.hive.beeline/TestBeeLineWithArgs/testEscapeCRLFOffInDSVOutput/ https://builds.apache.org/job/PreCommit-HIVE-Build/10138/testReport/org.apache.hive.beeline/TestBeeLineWithArgs/testEscapeCRLFInTSV2Output/ > Fix broken tests caused by HIVE-18493 > - > > Key: HIVE-18807 > URL: https://issues.apache.org/jira/browse/HIVE-18807 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Major > > Broken tests = > org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB > org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress > org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17055) Flaky test: TestMiniLlapCliDriver.testCliDriver[llap_smb]
[ https://issues.apache.org/jira/browse/HIVE-17055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17055: --- Issue Type: Sub-task (was: Bug) Parent: HIVE-19142 > Flaky test: TestMiniLlapCliDriver.testCliDriver[llap_smb] > - > > Key: HIVE-17055 > URL: https://issues.apache.org/jira/browse/HIVE-17055 > Project: Hive > Issue Type: Sub-task >Reporter: Janaki Lahorani >Assignee: Deepak Jaiswal >Priority: Major > > Following tests are also failing with same symptoms: > * TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1] > * TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_main] > Client Execution succeeded but contained differences (error code = 1) after > executing llap_smb.q > 324,325c324,325 > < 2000 9 52 > < 2001 0 139630 > --- > > 2001 4 139630 > > 2001 6 52 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17055) Flaky test: TestMiniLlapCliDriver.testCliDriver[llap_smb]
[ https://issues.apache.org/jira/browse/HIVE-17055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17055: --- Issue Type: Bug (was: Sub-task) Parent: (was: HIVE-15058) > Flaky test: TestMiniLlapCliDriver.testCliDriver[llap_smb] > - > > Key: HIVE-17055 > URL: https://issues.apache.org/jira/browse/HIVE-17055 > Project: Hive > Issue Type: Bug >Reporter: Janaki Lahorani >Assignee: Deepak Jaiswal >Priority: Major > > Following tests are also failing with same symptoms: > * TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1] > * TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_main] > Client Execution succeeded but contained differences (error code = 1) after > executing llap_smb.q > 324,325c324,325 > < 2000 9 52 > < 2001 0 139630 > --- > > 2001 4 139630 > > 2001 6 52 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17055) Flaky test: TestMiniLlapCliDriver.testCliDriver[llap_smb]
[ https://issues.apache.org/jira/browse/HIVE-17055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434523#comment-16434523 ] Vineet Garg commented on HIVE-17055: There are three tests which are failing due to flaky stats as mentioned in description. Common theme among all of them are that they are SMB tests. > Flaky test: TestMiniLlapCliDriver.testCliDriver[llap_smb] > - > > Key: HIVE-17055 > URL: https://issues.apache.org/jira/browse/HIVE-17055 > Project: Hive > Issue Type: Sub-task >Reporter: Janaki Lahorani >Assignee: Deepak Jaiswal >Priority: Major > > Following tests are also failing with same symptoms: > * TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1] > * TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_main] > Client Execution succeeded but contained differences (error code = 1) after > executing llap_smb.q > 324,325c324,325 > < 2000 9 52 > < 2001 0 139630 > --- > > 2001 4 139630 > > 2001 6 52 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18888) Replace synchronizedMap with ConcurrentHashMap
[ https://issues.apache.org/jira/browse/HIVE-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434519#comment-16434519 ] Alexander Kolbasov commented on HIVE-1: --- Hmm, interesting. Looks like this is intentional - for maps with concurrent access there is no way to distinguish between the missing value and the NULL value. I am not sure whether this was an issue with the original use of wrapped maps. > Replace synchronizedMap with ConcurrentHashMap > -- > > Key: HIVE-1 > URL: https://issues.apache.org/jira/browse/HIVE-1 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0, 2.3.3 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-1.01.patch, HIVE-1.02.patch > > > There are a bunch of places that use Collections.synchronizedMap instead of > ConcurrentHashMap which are better. We should search/replace the uses. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17055) Flaky test: TestMiniLlapCliDriver.testCliDriver[llap_smb]
[ https://issues.apache.org/jira/browse/HIVE-17055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17055: --- Description: Following tests are also failing with same symptoms: * TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1] * TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_main] Client Execution succeeded but contained differences (error code = 1) after executing llap_smb.q 324,325c324,325 < 2000 9 52 < 2001 0 139630 --- > 2001 4 139630 > 2001 6 52 was: Client Execution succeeded but contained differences (error code = 1) after executing llap_smb.q 324,325c324,325 < 2000 9 52 < 2001 0 139630 --- > 2001 4 139630 > 2001 6 52 > Flaky test: TestMiniLlapCliDriver.testCliDriver[llap_smb] > - > > Key: HIVE-17055 > URL: https://issues.apache.org/jira/browse/HIVE-17055 > Project: Hive > Issue Type: Sub-task >Reporter: Janaki Lahorani >Assignee: Deepak Jaiswal >Priority: Major > > Following tests are also failing with same symptoms: > * TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1] > * TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_main] > Client Execution succeeded but contained differences (error code = 1) after > executing llap_smb.q > 324,325c324,325 > < 2000 9 52 > < 2001 0 139630 > --- > > 2001 4 139630 > > 2001 6 52 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19178) TestMiniTezCliDriver.testCliDriver[explainanalyze_5] failure
[
https://issues.apache.org/jira/browse/HIVE-19178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434504#comment-16434504
]
Vineet Garg commented on HIVE-19178:
Can you take a look [~jcamachorodriguez]?
> TestMiniTezCliDriver.testCliDriver[explainanalyze_5] failure
>
>
> Key: HIVE-19178
> URL: https://issues.apache.org/jira/browse/HIVE-19178
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Fix For: 3.0.0
>
>
> I have verified that this failure is due to HIVE-18825.
> Error stack:
> {code}
> java.lang.IllegalStateException: calling recordValidTxn() more than once in
> the same txnid:5
> at org.apache.hadoop.hive.ql.Driver.acquireLocks(Driver.java:1439)
> at org.apache.hadoop.hive.ql.Driver.lockAndRespond(Driver.java:1624)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1794)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1538)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1527)
> at
> org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:137)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:287)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:635)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1655)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1602)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1597)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:200)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:335)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1455)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:1429)
> at
> org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:177)
> at
> org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:104)
> at
> org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver(TestMiniTezCliDriver.java:59)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.apache.hadoop.hive.cli.control.CliAdapter$2$1.evaluate(CliAdapter.java:92)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at
> org.apache.hadoop.hive.cli.control.CliAdapter$1$1.evaluate(CliAdapter.java:73)
> at
[jira] [Assigned] (HIVE-19178) TestMiniTezCliDriver.testCliDriver[explainanalyze_5] failure
[
https://issues.apache.org/jira/browse/HIVE-19178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg reassigned HIVE-19178:
--
Assignee: Jesus Camacho Rodriguez (was: Vineet Garg)
> TestMiniTezCliDriver.testCliDriver[explainanalyze_5] failure
>
>
> Key: HIVE-19178
> URL: https://issues.apache.org/jira/browse/HIVE-19178
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Fix For: 3.0.0
>
>
> I have verified that this failure is due to HIVE-18825.
> Error stack:
> {code}
> java.lang.IllegalStateException: calling recordValidTxn() more than once in
> the same txnid:5
> at org.apache.hadoop.hive.ql.Driver.acquireLocks(Driver.java:1439)
> at org.apache.hadoop.hive.ql.Driver.lockAndRespond(Driver.java:1624)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1794)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1538)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1527)
> at
> org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:137)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:287)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:635)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1655)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1602)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1597)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:200)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:335)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1455)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:1429)
> at
> org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:177)
> at
> org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:104)
> at
> org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver(TestMiniTezCliDriver.java:59)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.apache.hadoop.hive.cli.control.CliAdapter$2$1.evaluate(CliAdapter.java:92)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at
> org.apache.hadoop.hive.cli.control.CliAdapter$1$1.evaluate(CliAdapter.java:73)
> at
[jira] [Assigned] (HIVE-19178) TestMiniTezCliDriver.testCliDriver[explainanalyze_5] failure
[
https://issues.apache.org/jira/browse/HIVE-19178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg reassigned HIVE-19178:
--
> TestMiniTezCliDriver.testCliDriver[explainanalyze_5] failure
>
>
> Key: HIVE-19178
> URL: https://issues.apache.org/jira/browse/HIVE-19178
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
>
> I have verified that this failure is due to HIVE-18825.
> Error stack:
> {code}
> java.lang.IllegalStateException: calling recordValidTxn() more than once in
> the same txnid:5
> at org.apache.hadoop.hive.ql.Driver.acquireLocks(Driver.java:1439)
> at org.apache.hadoop.hive.ql.Driver.lockAndRespond(Driver.java:1624)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1794)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1538)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1527)
> at
> org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:137)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:287)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:635)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1655)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1602)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1597)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:200)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:335)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1455)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:1429)
> at
> org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:177)
> at
> org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:104)
> at
> org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver(TestMiniTezCliDriver.java:59)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.apache.hadoop.hive.cli.control.CliAdapter$2$1.evaluate(CliAdapter.java:92)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at
> org.apache.hadoop.hive.cli.control.CliAdapter$1$1.evaluate(CliAdapter.java:73)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at
[jira] [Commented] (HIVE-19175) TestMiniLlapLocalCliDriver.testCliDriver update_access_time_non_current_db failing
[
https://issues.apache.org/jira/browse/HIVE-19175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434500#comment-16434500
]
Vineet Garg commented on HIVE-19175:
I think you are right that this file needs to in
{{q/src/test/results/clientpositive}} too since this is added under
{{minillaplocal.shared.query.files}}.
> TestMiniLlapLocalCliDriver.testCliDriver update_access_time_non_current_db
> failing
> --
>
> Key: HIVE-19175
> URL: https://issues.apache.org/jira/browse/HIVE-19175
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-19175.1.patch
>
>
> Caused by HIVE-18060. Instead of generating golden file under
> clientpositive/llap it is under clientpositive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19175) TestMiniLlapLocalCliDriver.testCliDriver update_access_time_non_current_db failing
[
https://issues.apache.org/jira/browse/HIVE-19175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434498#comment-16434498
]
Peter Vary commented on HIVE-19175:
---
[~vgarg]: Let's see, what ptest tells us. But if I understand correctly, this
test is added to {{minillaplocal.shared.query.files}} which means that
according to the {{CliConfigs}} this test will be run by:
{{MiniLlapLocalCliConfig}}, but also with {{CliConfig}}. So I think you are
right, that we need this file in {{ql/src/test/results/clientpositive/llap}},
but I think we still need it in {{ql/src/test/results/clientpositive}} too.
[~kgyrtkirk]: could you confirm this please?
> TestMiniLlapLocalCliDriver.testCliDriver update_access_time_non_current_db
> failing
> --
>
> Key: HIVE-19175
> URL: https://issues.apache.org/jira/browse/HIVE-19175
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-19175.1.patch
>
>
> Caused by HIVE-18060. Instead of generating golden file under
> clientpositive/llap it is under clientpositive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19142) Umbrella: branch-3 failing tests
[ https://issues.apache.org/jira/browse/HIVE-19142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434496#comment-16434496 ] Vineet Garg commented on HIVE-19142: testLlapInputFormatEndToEnd should be fixed by HIVE-17645 > Umbrella: branch-3 failing tests > > > Key: HIVE-19142 > URL: https://issues.apache.org/jira/browse/HIVE-19142 > Project: Hive > Issue Type: Test >Reporter: Vineet Garg >Priority: Major > > This is the list [~alangates] specified on HIVE-19135 which are non-oom test > failures: > *Errors*: > TestAcidOnTez.testGetSplitsLocks > TestJdbcWithLocalClusterSpark.testSparkQuery > TestJdbcWithLocalClusterSpark.testTempTable > TestJdbcWithMiniLlap.testLlapInputFormatEndToEnd > TestMTQueries.testMTQueries1 > TestMultiSessionsHS2WithLocalClusterSpark.testSparkQuery > TestNegativeCliDriver.alter_notnull_constraint_violation > TestNegativeCliDriver.insert_into_acid_notnull > TestNegativeCliDriver.insert_into_notnull_constraint > TestNegativeCliDriver.insert_multi_into_notnull > TestNegativeCliDriver.insert_overwrite_notnull_constraint > TestNegativeCliDriver.update_notnull_constraint > *Failures*: > TestBlobstoreCliDriver.insert_into_dynamic_partitions > TestBlobstoreCliDriver.insert_overwrite_directory > TestBlobstoreCliDriver.insert_overwrite_dynamic_partitions > -TestCliDriver.acid_table_stats- > TestCliDriver.auto_sortmerge_join_2 > TestCliDriver.avro_alter_table_update_columns > TestCliDriver.avrotblsjoin > TestCliDriver.dbtxnmgr_showlocks > TestCliDriver.orc_merge10 > TestCliDriver.orc_schema_evolution_float > TestCliDriver.parquet_ppd_multifiles > TestCliDriver.schema_evol_par_vec_table_dictionary_encoding > TestCliDriver.schema_evol_par_vec_table_non_dictionary_encoding > TestCliDriver.selectindate > -TestCliDriver.statsoptimizer- > TestCliDriver.vector_bround > TestCliDriver.vector_case_when_1 > TestCliDriver.vector_coalesce_2 > TestCliDriver.vector_coalesce_3 > TestCliDriver.vector_interval_1 > TestCliDriver.vectorized_parquet_types > TestMetastoreVersion.testMetastoreVersion > TestMetastoreVersion.testVersionMatching > TestMiniDruidCliDriver.druidkafkamini_basic > TestMiniLlapCliDriver.llap_smb > TestMiniLlapCliDriver.unionDistinct_1 > TestMiniTezCliDriver.explainanalyze_5 > -TestNegativeCliDriver.authorization_caseinsensitivity- > -TestNegativeCliDriver.authorization_fail_1- > -TestNegativeCliDriver.authorization_grant_table_dup- > -TestNegativeCliDriver.authorization_role_case- > -TestNegativeCliDriver.authorization_role_grant_nosuchrole- > -TestNegativeCliDriver.authorization_table_grant_nosuchrole- > TestNegativeCliDriver.subquery_subquery_chain > TestSessionState.testCreatePath > TestSessionState.testCreatePath > TestSparkStatistics.testSparkStatistics -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19154) Poll notification events to invalidate the results cache
[ https://issues.apache.org/jira/browse/HIVE-19154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434495#comment-16434495 ] Gopal V commented on HIVE-19154: LGTM - +1 Once this is tested with a large number of tables, we would probably add a filter in front of this. > Poll notification events to invalidate the results cache > > > Key: HIVE-19154 > URL: https://issues.apache.org/jira/browse/HIVE-19154 > Project: Hive > Issue Type: Sub-task >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > Attachments: HIVE-19154.1.patch, HIVE-19154.2.patch > > > Related to the work for HIVE-18609. HIVE-18609 will only invalidate entries > in the cache if that query looked up again, which could potentially leave a > lot of undetected invalid entries in the cache taking up space which could > cause other entries to be evicted. To remove these entries in a more timely > fashion, have a background thread to periodically check the notification > events for updates to the tables used in the results cache. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17645) MM tables patch conflicts with HIVE-17482 (Spark/Acid integration)
[
https://issues.apache.org/jira/browse/HIVE-17645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Dere updated HIVE-17645:
--
Attachment: HIVE-17645.3.patch
> MM tables patch conflicts with HIVE-17482 (Spark/Acid integration)
> --
>
> Key: HIVE-17645
> URL: https://issues.apache.org/jira/browse/HIVE-17645
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Jason Dere
>Priority: Major
> Labels: mm-gap-2
> Attachments: HIVE-17645.1.patch, HIVE-17645.2.patch,
> HIVE-17645.3.patch
>
>
> MM code introduces
> {noformat}
> HiveTxnManager txnManager = SessionState.get().getTxnMgr()
> {noformat}
> in a number of places (e.g _DDLTask.generateAddMmTasks(Table tbl)_).
> HIVE-17482 adds a mode where a TransactionManager not associated with the
> session should be used. This will need to be addressed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19130) NPE is thrown when REPL LOAD applied drop partition event.
[
https://issues.apache.org/jira/browse/HIVE-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434477#comment-16434477
]
Hive QA commented on HIVE-19130:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
45s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} The patch hive-unit passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} ql: The patch generated 0 new + 245 unchanged - 1
fixed = 245 total (was 246) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10142/dev-support/hive-personality.sh
|
| git revision | master / 42187fd |
| Default Java | 1.8.0_111 |
| modules | C: itests/hive-unit ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10142/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> NPE is thrown when REPL LOAD applied drop partition event.
> --
>
> Key: HIVE-19130
> URL: https://issues.apache.org/jira/browse/HIVE-19130
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, repl
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
> Labels: DR, Replication, pull-request-available
> Fix For: 3.1.0
>
> Attachments: HIVE-19130.01.patch
>
>
> During incremental replication, if we split the events batch as follows, then
> the REPL LOAD on second batch throws NPE.
> Batch-1: CREATE_TABLE(t1) -> ADD_PARTITION(t1.p1) -> DROP_PARTITION (t1.p1)
> Batch-2: DROP_TABLE(t1) -> CREATE_TABLE(t1) -> ADD_PARTITION(t1.p1) ->
> DROP_PARTITION (t1.p1)
> {code}
> 2018-04-05 16:20:36,531 ERROR [HiveServer2-Background-Pool: Thread-107044]:
> metadata.Hive (Hive.java:getTable(1219)) - Table catalog_sales_new not found:
> new5_tpcds_real_bin_partitioned_orc_1000.catalog_sales_new table not found
> 2018-04-05 16:20:36,538 ERROR [HiveServer2-Background-Pool: Thread-107044]:
> exec.DDLTask (DDLTask.java:failed(540)) -
> org.apache.hadoop.hive.ql.metadata.HiveException
> at
>
[jira] [Updated] (HIVE-19161) Add authorizations to information schema
[ https://issues.apache.org/jira/browse/HIVE-19161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-19161: -- Attachment: HIVE-19161.2.patch > Add authorizations to information schema > > > Key: HIVE-19161 > URL: https://issues.apache.org/jira/browse/HIVE-19161 > Project: Hive > Issue Type: Improvement >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-19161.1.patch, HIVE-19161.2.patch > > > We need to control the access of information schema so user can only query > the information authorized to. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19161) Add authorizations to information schema
[ https://issues.apache.org/jira/browse/HIVE-19161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434470#comment-16434470 ] Daniel Dai commented on HIVE-19161: --- HIVE-19161.2.patch uses atomic metastore call to refresh privileges. > Add authorizations to information schema > > > Key: HIVE-19161 > URL: https://issues.apache.org/jira/browse/HIVE-19161 > Project: Hive > Issue Type: Improvement >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-19161.1.patch, HIVE-19161.2.patch > > > We need to control the access of information schema so user can only query > the information authorized to. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18831) Differentiate errors that are thrown by Spark tasks
[ https://issues.apache.org/jira/browse/HIVE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18831: Attachment: HIVE-18831.92.patch > Differentiate errors that are thrown by Spark tasks > --- > > Key: HIVE-18831 > URL: https://issues.apache.org/jira/browse/HIVE-18831 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Attachments: HIVE-18831.1.patch, HIVE-18831.2.patch, > HIVE-18831.3.patch, HIVE-18831.4.patch, HIVE-18831.6.patch, > HIVE-18831.7.patch, HIVE-18831.8.WIP.patch, HIVE-18831.9.patch, > HIVE-18831.90.patch, HIVE-18831.91.patch, HIVE-18831.92.patch > > > We propagate exceptions from Spark task failures to the client well, but we > don't differentiate between errors from HS2 / RSC vs. errors thrown by > individual tasks. > Main motivation is that when the client sees a propagated Spark exception its > difficult to know what part of the excution threw the exception. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-12192) Hive should carry out timestamp computations in UTC
[
https://issues.apache.org/jira/browse/HIVE-12192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434443#comment-16434443
]
Hive QA commented on HIVE-12192:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12918368/HIVE-12192.02.patch
{color:green}SUCCESS:{color} +1 due to 51 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 521 failed/errored test(s), 13251 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=253)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=253)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[jira] [Commented] (HIVE-18906) Lower Logging for "Using direct SQL"
[
https://issues.apache.org/jira/browse/HIVE-18906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434438#comment-16434438
]
BELUGA BEHR commented on HIVE-18906:
I think what we want here is:
* An INFO message at startup which tells the administrator if the back-end
database is supported for the direct SQL optimization
* An INFO message at startup which tells the administrator if the direct SQL
optimization is enabled/disabled
* A WARN message if the SQL direct optimization is disabled for a database
that is supported
* A DEBUG message for each query if it is using (or not using) direct SQL
optimization
> Lower Logging for "Using direct SQL"
>
>
> Key: HIVE-18906
> URL: https://issues.apache.org/jira/browse/HIVE-18906
> Project: Hive
> Issue Type: Improvement
>Reporter: BELUGA BEHR
>Assignee: Antal Sinkovits
>Priority: Trivial
> Labels: noob
>
> https://github.com/apache/hive/blob/4047befe48c8f762c58d8854e058385c1df151c6/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java#L181
> {code}
> 2018-02-26 14:32:50,620 INFO
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql: [pool-4-thread-199]:
> Using direct SQL, underlying DB is MYSQL
> 2018-02-26 14:33:09,566 INFO
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql: [pool-4-thread-172]:
> Using direct SQL, underlying DB is MYSQL
> 2018-02-26 14:33:43,170 INFO
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql: [pool-4-thread-172]:
> Using direct SQL, underlying DB is MYSQL
> 2018-02-26 14:33:46,301 INFO
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql: [pool-4-thread-199]:
> Using direct SQL, underlying DB is MYSQL
> 2018-02-26 14:34:16,966 INFO
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql: [pool-4-thread-172]:
> Using direct SQL, underlying DB is MYSQL
> 2018-02-26 14:34:18,072 INFO
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql: [pool-4-thread-196]:
> Using direct SQL, underlying DB is MYSQL
> 2018-02-26 14:34:25,064 INFO
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql: [pool-4-thread-199]:
> Using direct SQL, underlying DB is MYSQL
> 2018-02-26 14:34:51,781 INFO
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql: [pool-4-thread-172]:
> Using direct SQL, underlying DB is MYSQL
> 2018-02-26 14:35:04,685 INFO
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql: [pool-4-thread-199]:
> Using direct SQL, underlying DB is MYSQL
> 2018-02-26 14:35:26,689 INFO
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql: [pool-4-thread-172]:
> Using direct SQL, underlying DB is MYSQL
> {code}
> I wouldn't mind knowing this information when the service starts up and
> detects that there is a Direct SQL compatible database, but I don't need to
> know about this every time.
> Please lower to _debug_ level logging and parameterize with SLF4J. Direct
> SQL is a performance thing and not something I care about during the normal
> course of operation (unless there is an error which would presumably be
> covered by an _error_ logging message).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19141) TestNegativeCliDriver insert_into_notnull_constraint, insert_into_acid_notnull failing
[ https://issues.apache.org/jira/browse/HIVE-19141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Kryvenko updated HIVE-19141: - Attachment: HIVE-19141.01.patch > TestNegativeCliDriver insert_into_notnull_constraint, > insert_into_acid_notnull failing > -- > > Key: HIVE-19141 > URL: https://issues.apache.org/jira/browse/HIVE-19141 > Project: Hive > Issue Type: Sub-task >Reporter: Vineet Garg >Assignee: Igor Kryvenko >Priority: Major > Fix For: 3.0.0, 3.1.0 > > Attachments: HIVE-19141.01.patch > > > These tests have been consistently failing for a while. I suspect HIVE-18727 > has caused these failures. HIVE-18727 changed the code to throw ERROR instead > of EXCEPTION if constraints are violated. I guess Negative cli driver doesn't > handle errors. > Following are full list of related failures: > TestNegativeCliDriver.alter_notnull_constraint_violation > TestNegativeCliDriver.insert_into_acid_notnull > TestNegativeCliDriver.insert_into_notnull_constraint > TestNegativeCliDriver.insert_multi_into_notnull > TestNegativeCliDriver.insert_overwrite_notnull_constraint > TestNegativeCliDriver.update_notnull_constraint -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19141) TestNegativeCliDriver insert_into_notnull_constraint, insert_into_acid_notnull failing
[ https://issues.apache.org/jira/browse/HIVE-19141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Kryvenko updated HIVE-19141: - Status: Patch Available (was: Open) > TestNegativeCliDriver insert_into_notnull_constraint, > insert_into_acid_notnull failing > -- > > Key: HIVE-19141 > URL: https://issues.apache.org/jira/browse/HIVE-19141 > Project: Hive > Issue Type: Sub-task >Reporter: Vineet Garg >Assignee: Igor Kryvenko >Priority: Major > Fix For: 3.0.0, 3.1.0 > > Attachments: HIVE-19141.01.patch > > > These tests have been consistently failing for a while. I suspect HIVE-18727 > has caused these failures. HIVE-18727 changed the code to throw ERROR instead > of EXCEPTION if constraints are violated. I guess Negative cli driver doesn't > handle errors. > Following are full list of related failures: > TestNegativeCliDriver.alter_notnull_constraint_violation > TestNegativeCliDriver.insert_into_acid_notnull > TestNegativeCliDriver.insert_into_notnull_constraint > TestNegativeCliDriver.insert_multi_into_notnull > TestNegativeCliDriver.insert_overwrite_notnull_constraint > TestNegativeCliDriver.update_notnull_constraint -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18883) Add findbugs to yetus pre-commit checks
[ https://issues.apache.org/jira/browse/HIVE-18883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18883: Resolution: Fixed Fix Version/s: 3.1.0 Status: Resolved (was: Patch Available) [~pvary] thanks for noticing that, removed the extra file. Pushed to master. Thanks for the review! > Add findbugs to yetus pre-commit checks > --- > > Key: HIVE-18883 > URL: https://issues.apache.org/jira/browse/HIVE-18883 > Project: Hive > Issue Type: Sub-task > Components: Testing Infrastructure >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-18883.1.patch, HIVE-18883.2.patch > > > We should enable FindBugs for our YETUS pre-commit checks, this will help > overall code quality and should decrease the overall number of bugs in Hive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19175) TestMiniLlapLocalCliDriver.testCliDriver update_access_time_non_current_db failing
[ https://issues.apache.org/jira/browse/HIVE-19175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434418#comment-16434418 ] Vineet Garg commented on HIVE-19175: [~pvary] Can you take a look at the patch? > TestMiniLlapLocalCliDriver.testCliDriver update_access_time_non_current_db > failing > -- > > Key: HIVE-19175 > URL: https://issues.apache.org/jira/browse/HIVE-19175 > Project: Hive > Issue Type: Sub-task > Components: Test >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19175.1.patch > > > Caused by HIVE-18060. Instead of generating golden file under > clientpositive/llap it is under clientpositive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19175) TestMiniLlapLocalCliDriver.testCliDriver update_access_time_non_current_db failing
[ https://issues.apache.org/jira/browse/HIVE-19175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19175: --- Status: Patch Available (was: Open) > TestMiniLlapLocalCliDriver.testCliDriver update_access_time_non_current_db > failing > -- > > Key: HIVE-19175 > URL: https://issues.apache.org/jira/browse/HIVE-19175 > Project: Hive > Issue Type: Sub-task > Components: Test >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19175.1.patch > > > Caused by HIVE-18060. Instead of generating golden file under > clientpositive/llap it is under clientpositive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19175) TestMiniLlapLocalCliDriver.testCliDriver update_access_time_non_current_db failing
[ https://issues.apache.org/jira/browse/HIVE-19175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19175: --- Attachment: HIVE-19175.1.patch > TestMiniLlapLocalCliDriver.testCliDriver update_access_time_non_current_db > failing > -- > > Key: HIVE-19175 > URL: https://issues.apache.org/jira/browse/HIVE-19175 > Project: Hive > Issue Type: Sub-task > Components: Test >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19175.1.patch > > > Caused by HIVE-18060. Instead of generating golden file under > clientpositive/llap it is under clientpositive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19175) TestMiniLlapLocalCliDriver.testCliDriver update_access_time_non_current_db failing
[ https://issues.apache.org/jira/browse/HIVE-19175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg reassigned HIVE-19175: -- > TestMiniLlapLocalCliDriver.testCliDriver update_access_time_non_current_db > failing > -- > > Key: HIVE-19175 > URL: https://issues.apache.org/jira/browse/HIVE-19175 > Project: Hive > Issue Type: Sub-task > Components: Test >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > > Caused by HIVE-18060. Instead of generating golden file under > clientpositive/llap it is under clientpositive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-12192) Hive should carry out timestamp computations in UTC
[
https://issues.apache.org/jira/browse/HIVE-12192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434374#comment-16434374
]
Hive QA commented on HIVE-12192:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
39s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
18s{color} | {color:red} druid-handler in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
18s{color} | {color:red} hbase-handler in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 1m
0s{color} | {color:red} ql in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
17s{color} | {color:red} druid-handler in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
19s{color} | {color:red} hbase-handler in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 17s{color}
| {color:red} druid-handler in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 19s{color}
| {color:red} hbase-handler in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
13s{color} | {color:red} storage-api: The patch generated 2 new + 44 unchanged
- 7 fixed = 46 total (was 51) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
15s{color} | {color:red} common: The patch generated 12 new + 17 unchanged - 8
fixed = 29 total (was 25) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} serde: The patch generated 0 new + 2597 unchanged -
19 fixed = 2597 total (was 2616) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} The patch accumulo-handler passed checkstyle {color}
|
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} The patch druid-handler passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch hbase-handler passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} The patch core passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch hive-jmh passed checkstyle {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
33s{color} | {color:red} ql: The patch generated 344 new + 5478 unchanged - 116
fixed = 5822 total (was 5594) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch vector-code-gen passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 1m
4s{color} | {color:red} ql generated 1 new + 99 unchanged - 1 fixed = 100 total
(was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF
[jira] [Commented] (HIVE-19166) TestMiniLlapLocalCliDriver sysdb failure
[ https://issues.apache.org/jira/browse/HIVE-19166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434372#comment-16434372 ] Vineet Garg commented on HIVE-19166: np [~kgyrtkirk]. I have pushed this to master and branch-3 > TestMiniLlapLocalCliDriver sysdb failure > > > Key: HIVE-19166 > URL: https://issues.apache.org/jira/browse/HIVE-19166 > Project: Hive > Issue Type: Sub-task > Components: Test >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19166.1.patch > > > Broken by HIVE-18715 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19166) TestMiniLlapLocalCliDriver sysdb failure
[ https://issues.apache.org/jira/browse/HIVE-19166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19166: --- Resolution: Fixed Status: Resolved (was: Patch Available) > TestMiniLlapLocalCliDriver sysdb failure > > > Key: HIVE-19166 > URL: https://issues.apache.org/jira/browse/HIVE-19166 > Project: Hive > Issue Type: Sub-task > Components: Test >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19166.1.patch > > > Broken by HIVE-18715 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19133) HS2 WebUI phase-wise performance metrics not showing correctly
[ https://issues.apache.org/jira/browse/HIVE-19133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434367#comment-16434367 ] Vihang Karajgaonkar commented on HIVE-19133: Thanks [~bharos92] for the patch. Passing Threadlocal objects across threads is an anti-pattern and cause weird race-conditions which are hard to debug. It should be noted that after this patch the Driver.run() perf metric value is split into two, one for compile phase and one for execution phase. This makes sense to me since the Driver.run() method spans across these two phases and the actual execution is done by different threads. I am not sure if there is value in keeping Driver.run metric anymore. [~thejas] [~kgyrtkirk] any thoughts on removing the driver.run() metric since after this patch it will become redundant. > HS2 WebUI phase-wise performance metrics not showing correctly > -- > > Key: HIVE-19133 > URL: https://issues.apache.org/jira/browse/HIVE-19133 > Project: Hive > Issue Type: Bug > Components: HiveServer2, Web UI >Reporter: Bharathkrishna Guruvayoor Murali >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-19133.1.patch, HIVE-19133.2.patch, WebUI-compile > time query metrics.png > > > The query specific WebUI metrics (go to drilldown -> performance logging) are > not showing up in the correct phase and are often mixed up. > Attaching screenshot. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19141) TestNegativeCliDriver insert_into_notnull_constraint, insert_into_acid_notnull failing
[ https://issues.apache.org/jira/browse/HIVE-19141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg reassigned HIVE-19141: -- Assignee: Igor Kryvenko (was: Vineet Garg) > TestNegativeCliDriver insert_into_notnull_constraint, > insert_into_acid_notnull failing > -- > > Key: HIVE-19141 > URL: https://issues.apache.org/jira/browse/HIVE-19141 > Project: Hive > Issue Type: Sub-task >Reporter: Vineet Garg >Assignee: Igor Kryvenko >Priority: Major > Fix For: 3.0.0, 3.1.0 > > > These tests have been consistently failing for a while. I suspect HIVE-18727 > has caused these failures. HIVE-18727 changed the code to throw ERROR instead > of EXCEPTION if constraints are violated. I guess Negative cli driver doesn't > handle errors. > Following are full list of related failures: > TestNegativeCliDriver.alter_notnull_constraint_violation > TestNegativeCliDriver.insert_into_acid_notnull > TestNegativeCliDriver.insert_into_notnull_constraint > TestNegativeCliDriver.insert_multi_into_notnull > TestNegativeCliDriver.insert_overwrite_notnull_constraint > TestNegativeCliDriver.update_notnull_constraint -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19141) TestNegativeCliDriver insert_into_notnull_constraint, insert_into_acid_notnull failing
[ https://issues.apache.org/jira/browse/HIVE-19141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434358#comment-16434358 ] Vineet Garg commented on HIVE-19141: [~vbeshka] You can try uploading it with jira instead of comment. > TestNegativeCliDriver insert_into_notnull_constraint, > insert_into_acid_notnull failing > -- > > Key: HIVE-19141 > URL: https://issues.apache.org/jira/browse/HIVE-19141 > Project: Hive > Issue Type: Sub-task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0, 3.1.0 > > > These tests have been consistently failing for a while. I suspect HIVE-18727 > has caused these failures. HIVE-18727 changed the code to throw ERROR instead > of EXCEPTION if constraints are violated. I guess Negative cli driver doesn't > handle errors. > Following are full list of related failures: > TestNegativeCliDriver.alter_notnull_constraint_violation > TestNegativeCliDriver.insert_into_acid_notnull > TestNegativeCliDriver.insert_into_notnull_constraint > TestNegativeCliDriver.insert_multi_into_notnull > TestNegativeCliDriver.insert_overwrite_notnull_constraint > TestNegativeCliDriver.update_notnull_constraint -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19104) When test MetaStore is started with retry the instances should be independent
[
https://issues.apache.org/jira/browse/HIVE-19104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434356#comment-16434356
]
Peter Vary commented on HIVE-19104:
---
{quote}Ok, then why do we just set {{javax.jdo.option.ConnectionURL}} inside
{{MetaStoreTestUtils}} rather than doing the string replacement
{quote}
Ok. Tomorrow I will create a new patch version where we set the warehouse dir,
and the jdbc url in the same way, than we do it with the thrift uri.
{quote}Can the following code just be moved into a {{BeforeClass}} block?
{quote}
Sure: Good idea! Thanks!
{quote}Why is {{MetastoreConf.newMetastoreConf(new Configuration(this.conf));}}
necessary? Can't it just be {{MetastoreConf.newMetastoreConf(this.conf);}}
{quote}
This particular test changes the conf. Since we initialize the conf once per
test class, either we revert the changes done by the test, or copy it so we do
not have a problem in the next tests
> When test MetaStore is started with retry the instances should be independent
> -
>
> Key: HIVE-19104
> URL: https://issues.apache.org/jira/browse/HIVE-19104
> Project: Hive
> Issue Type: Improvement
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-19104.2.patch, HIVE-19104.3.patch, HIVE-19104.patch
>
>
> When multiple MetaStore instances are started with
> {{MetaStoreTestUtils.startMetaStoreWithRetry}} currently they use the same
> JDBC url, and warehouse directory. This can cause problem in the tests
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17824) msck repair table should drop the missing partitions from metastore
[
https://issues.apache.org/jira/browse/HIVE-17824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434347#comment-16434347
]
Vihang Karajgaonkar commented on HIVE-17824:
Took a quick look on reviewboard. Looks good. Left a minor suggestion
(non-blocking) on RB. +1
> msck repair table should drop the missing partitions from metastore
> ---
>
> Key: HIVE-17824
> URL: https://issues.apache.org/jira/browse/HIVE-17824
> Project: Hive
> Issue Type: Improvement
>Reporter: Vihang Karajgaonkar
>Assignee: Janaki Lahorani
>Priority: Major
> Attachments: HIVE-17824.1.patch, HIVE-17824.2.patch,
> HIVE-17824.3.patch
>
>
> {{msck repair table }} is often used in environments where the new
> partitions are loaded as directories on HDFS or S3 and users want to create
> the missing partitions in bulk. However, currently it only supports addition
> of missing partitions. If there are any partitions which are present in
> metastore but not on the FileSystem, it should also delete them so that it
> truly repairs the table metadata.
> We should be careful not to break backwards compatibility so we should either
> introduce a new config or keyword to add support to delete unnecessary
> partitions from the metastore. This way users who want the old behavior can
> easily turn it off.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19171) Persist runtime statistics in metastore
[ https://issues.apache.org/jira/browse/HIVE-19171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434328#comment-16434328 ] Zoltan Haindrich commented on HIVE-19171: - [~alangates] runtime statistics are collected if the query fails (for example: map join problems; oom); the query is currently re-execute using the runtime stats by persisting this data in the metastore makes the "re-execution" happen only once...the statistics are saved in the metastore; and it will probably just work for the end user Its not ready yet...but my wip branch is at https://github.com/apache/hive/compare/master...kgyrtkirk:HIVE-19171-persist-metastore if you would like to take a look; it will make the following changes at the metastore side: * a new metastore table * 2 new thrift calls > Persist runtime statistics in metastore > --- > > Key: HIVE-19171 > URL: https://issues.apache.org/jira/browse/HIVE-19171 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19154) Poll notification events to invalidate the results cache
[ https://issues.apache.org/jira/browse/HIVE-19154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-19154: -- Attachment: HIVE-19154.2.patch > Poll notification events to invalidate the results cache > > > Key: HIVE-19154 > URL: https://issues.apache.org/jira/browse/HIVE-19154 > Project: Hive > Issue Type: Sub-task >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > Attachments: HIVE-19154.1.patch, HIVE-19154.2.patch > > > Related to the work for HIVE-18609. HIVE-18609 will only invalidate entries > in the cache if that query looked up again, which could potentially leave a > lot of undetected invalid entries in the cache taking up space which could > cause other entries to be evicted. To remove these entries in a more timely > fashion, have a background thread to periodically check the notification > events for updates to the tables used in the results cache. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18525) Add explain plan to Hive on Spark Web UI
[
https://issues.apache.org/jira/browse/HIVE-18525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434322#comment-16434322
]
Sahil Takiar commented on HIVE-18525:
-
[~aihuaxu] any other comments?
> Add explain plan to Hive on Spark Web UI
>
>
> Key: HIVE-18525
> URL: https://issues.apache.org/jira/browse/HIVE-18525
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18525.1.patch, HIVE-18525.2.patch,
> HIVE-18525.3.patch, HIVE-18525.4.patch, Job-Page-Collapsed.png,
> Job-Page-Expanded.png, Map-Explain-Plan.png, Reduce-Explain-Plan.png
>
>
> More of an investigation JIRA. The Spark UI has a "long description" of each
> stage in the Spark DAG. Typically one stage in the Spark DAG corresponds to
> either a {{MapWork}} or {{ReduceWork}} object. It would be useful if the long
> description contained the explain plan of the corresponding work object.
> I'm not sure how much additional overhead this would introduce. If not the
> full explain plan, then maybe a modified one that just lists out all the
> operator tree along with each operator name.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19104) When test MetaStore is started with retry the instances should be independent
[
https://issues.apache.org/jira/browse/HIVE-19104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434319#comment-16434319
]
Sahil Takiar commented on HIVE-19104:
-
{quote} The purpose of the MetaStoreTestUtils is to allow easy metastore
instance creation for tests and the user should not have to bother with
details, like making the jdbc url independent {quote} Ok, then why do we just
set {{javax.jdo.option.ConnectionURL}} inside {{MetaStoreTestUtils}} rather
than doing the string replacement? Are there any tests that require a custom
defined JDBC URL? If there are, can we just make separate helper methods for
them in {{MetaStoreTestUtils}}. Another approach to find a way to properly
parse the connection URL and extract out the database name, and then append the
port number. However, I don't think there is a standardized way to parse JDBC
URLs.
A few other comments:
Can the following code just be moved into a {{BeforeClass}} block?
{code}
if (conf == null) {
conf = MetastoreConf.newMetastoreConf();
}
{code}
Why is {{MetastoreConf.newMetastoreConf(new Configuration(this.conf));}}
necessary? Can't it just be {{MetastoreConf.newMetastoreConf(this.conf);}}
> When test MetaStore is started with retry the instances should be independent
> -
>
> Key: HIVE-19104
> URL: https://issues.apache.org/jira/browse/HIVE-19104
> Project: Hive
> Issue Type: Improvement
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-19104.2.patch, HIVE-19104.3.patch, HIVE-19104.patch
>
>
> When multiple MetaStore instances are started with
> {{MetaStoreTestUtils.startMetaStoreWithRetry}} currently they use the same
> JDBC url, and warehouse directory. This can cause problem in the tests
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19157) Assert that Insert into Druid Table fails if the publishing of metadata by HS2 fails
[
https://issues.apache.org/jira/browse/HIVE-19157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434318#comment-16434318
]
slim bouguerra commented on HIVE-19157:
---
I have attache a second patch that has a different approach.
[^HIVE-19157.2.patch] In this version will check for the existence of metadata
directory first.
If such Directory is present on the file system then it must have data to load
thus the second list should return valid file names.
if it is absent then that means there is nothing to load.
> Assert that Insert into Druid Table fails if the publishing of metadata by
> HS2 fails
>
>
> Key: HIVE-19157
> URL: https://issues.apache.org/jira/browse/HIVE-19157
> Project: Hive
> Issue Type: Bug
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-19157.2.patch, HIVE-19157.patch
>
>
> The usual work flow of loading Data into Druid relies on the fact that HS2 is
> able to load Segments metadata from HDFS that are produced by LLAP/TEZ works.
> In some cases where HS2 is not able to perform `ls` on the HDFS path the
> insert into query will return success and will not insert any data.
> This bug was introduced at function {code}
> org.apache.hadoop.hive.druid.DruidStorageHandlerUtils#getCreatedSegments{code}
>
> when we added feature to allow create empty tables.
> {code}
> try {
> fss = fs.listStatus(taskDir);
> } catch (FileNotFoundException e) {
> // This is a CREATE TABLE statement or query executed for CTAS/INSERT
> // did not produce any result. We do not need to do anything, this is
> // expected behavior.
> return publishedSegmentsBuilder.build();
> }
> {code}
> Am still looking for the way to fix this, [~jcamachorodriguez]/[~ashutoshc]
> any idea what is the best way to detect that it is an empty create table
> statement?
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19157) Assert that Insert into Druid Table fails if the publishing of metadata by HS2 fails
[
https://issues.apache.org/jira/browse/HIVE-19157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
slim bouguerra updated HIVE-19157:
--
Attachment: HIVE-19157.2.patch
> Assert that Insert into Druid Table fails if the publishing of metadata by
> HS2 fails
>
>
> Key: HIVE-19157
> URL: https://issues.apache.org/jira/browse/HIVE-19157
> Project: Hive
> Issue Type: Bug
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-19157.2.patch, HIVE-19157.patch
>
>
> The usual work flow of loading Data into Druid relies on the fact that HS2 is
> able to load Segments metadata from HDFS that are produced by LLAP/TEZ works.
> In some cases where HS2 is not able to perform `ls` on the HDFS path the
> insert into query will return success and will not insert any data.
> This bug was introduced at function {code}
> org.apache.hadoop.hive.druid.DruidStorageHandlerUtils#getCreatedSegments{code}
>
> when we added feature to allow create empty tables.
> {code}
> try {
> fss = fs.listStatus(taskDir);
> } catch (FileNotFoundException e) {
> // This is a CREATE TABLE statement or query executed for CTAS/INSERT
> // did not produce any result. We do not need to do anything, this is
> // expected behavior.
> return publishedSegmentsBuilder.build();
> }
> {code}
> Am still looking for the way to fix this, [~jcamachorodriguez]/[~ashutoshc]
> any idea what is the best way to detect that it is an empty create table
> statement?
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-16616) Javadoc generation is full of errors
[ https://issues.apache.org/jira/browse/HIVE-16616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich reassigned HIVE-16616: --- Assignee: (was: Janos Gub) > Javadoc generation is full of errors > > > Key: HIVE-16616 > URL: https://issues.apache.org/jira/browse/HIVE-16616 > Project: Hive > Issue Type: Bug >Reporter: Janos Gub >Priority: Major > > Trying to generate documentation from javadoc, full of errors. This ticket is > to track the process of cleaning the project of those. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-16716) Clean up javadoc from errors in module ql
[ https://issues.apache.org/jira/browse/HIVE-16716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434312#comment-16434312 ] Zoltan Haindrich commented on HIVE-16716: - [~gubjanos] I hope you don't mind; I've unassigned the ticket...in the hope that someone picks it up - sooner or later :) Thank you for all your efforst so far in making hive's javadoc closer to being useable :) > Clean up javadoc from errors in module ql > - > > Key: HIVE-16716 > URL: https://issues.apache.org/jira/browse/HIVE-16716 > Project: Hive > Issue Type: Sub-task >Reporter: Janos Gub >Priority: Major > Attachments: HIVE-16716-v2.patch, HIVE-16716.2.patch, HIVE-16716.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-16815) Clean up javadoc from error for the rest of modules
[ https://issues.apache.org/jira/browse/HIVE-16815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434310#comment-16434310 ] Zoltan Haindrich commented on HIVE-16815: - [~gubjanos] I hope you don't mind; I've unassigned the ticket...in the hope that someone picks it up - sooner or later :) > Clean up javadoc from error for the rest of modules > --- > > Key: HIVE-16815 > URL: https://issues.apache.org/jira/browse/HIVE-16815 > Project: Hive > Issue Type: Sub-task >Reporter: Janos Gub >Priority: Major > Attachments: HIVE-16815.2.patch, HIVE-16815.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-16815) Clean up javadoc from error for the rest of modules
[ https://issues.apache.org/jira/browse/HIVE-16815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich reassigned HIVE-16815: --- Assignee: (was: Janos Gub) > Clean up javadoc from error for the rest of modules > --- > > Key: HIVE-16815 > URL: https://issues.apache.org/jira/browse/HIVE-16815 > Project: Hive > Issue Type: Sub-task >Reporter: Janos Gub >Priority: Major > Attachments: HIVE-16815.2.patch, HIVE-16815.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-16716) Clean up javadoc from errors in module ql
[ https://issues.apache.org/jira/browse/HIVE-16716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich reassigned HIVE-16716: --- Assignee: (was: Janos Gub) > Clean up javadoc from errors in module ql > - > > Key: HIVE-16716 > URL: https://issues.apache.org/jira/browse/HIVE-16716 > Project: Hive > Issue Type: Sub-task >Reporter: Janos Gub >Priority: Major > Attachments: HIVE-16716-v2.patch, HIVE-16716.2.patch, HIVE-16716.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18469) HS2UI: Introduce separate option to show query on web ui
[
https://issues.apache.org/jira/browse/HIVE-18469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich reassigned HIVE-18469:
---
Assignee: Bharathkrishna Guruvayoor Murali
Of course! Thanks you picking it up! :)
> HS2UI: Introduce separate option to show query on web ui
>
>
> Key: HIVE-18469
> URL: https://issues.apache.org/jira/browse/HIVE-18469
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Bharathkrishna Guruvayoor Murali
>Priority: Major
>
> currently {{ConfVars.HIVE_LOG_EXPLAIN_OUTPUT}} enables 2 features:
> * log the query to the console (even thru beeline)
> * shows the query on the web ui
> I've enabled it...and ever since then my beeline is always flooded with an
> {{explain extended}} output...which is very verbose; even for simple queries.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19133) HS2 WebUI phase-wise performance metrics not showing correctly
[ https://issues.apache.org/jira/browse/HIVE-19133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434302#comment-16434302 ] Zoltan Haindrich commented on HIVE-19133: - +1 > HS2 WebUI phase-wise performance metrics not showing correctly > -- > > Key: HIVE-19133 > URL: https://issues.apache.org/jira/browse/HIVE-19133 > Project: Hive > Issue Type: Bug > Components: HiveServer2, Web UI >Reporter: Bharathkrishna Guruvayoor Murali >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-19133.1.patch, HIVE-19133.2.patch, WebUI-compile > time query metrics.png > > > The query specific WebUI metrics (go to drilldown -> performance logging) are > not showing up in the correct phase and are often mixed up. > Attaching screenshot. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18988) Support bootstrap replication of ACID tables
[
https://issues.apache.org/jira/browse/HIVE-18988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434295#comment-16434295
]
Hive QA commented on HIVE-18988:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12918356/HIVE-18988.03.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 71 failed/errored test(s), 13659 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=253)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=253)
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=96)
