[jira] [Created] (HIVE-24467) ConditionalTask remove other tasks exists thread safety problem
guojh created HIVE-24467: Summary: ConditionalTask remove other tasks exists thread safety problem Key: HIVE-24467 URL: https://issues.apache.org/jira/browse/HIVE-24467 Project: Hive Issue Type: Bug Components: Hive Affects Versions: 2.3.4 Reporter: guojh When hive execute jobs in parallel(control by “hive.exec.parallel” parameter), ConditionalTasks run parallel to remove the tasks that not pick to run, because there are thread safety issues, some task may not remove from the dependent task tree. This is a very serious bug, which causes some stage task not trigger execution. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24466) insert queries should not launch job when condition in the query would output 0 rows
Rajesh Balamohan created HIVE-24466:
---
Summary: insert queries should not launch job when condition in
the query would output 0 rows
Key: HIVE-24466
URL: https://issues.apache.org/jira/browse/HIVE-24466
Project: Hive
Issue Type: Improvement
Reporter: Rajesh Balamohan
{noformat}
-- This query would not generate any output and does not launch a job
select * from tpcds_bin_partitioned_orc_3.store_sales where 1 = 2;
-- This query generates a job (M -> R -> R) and runs for 30+ seconds in 2 node
cluster to generate 0 rows.
insert into table delete_orc_10.test_sales_1 select * from
tpcds_bin_partitioned_orc_3.store_sales where 1 = 2;
insert overwrite table delete_orc_10.test_sales_1 select * from
tpcds_bin_partitioned_orc_3.store_sales where ss_sold_date_sk >=2450816+300
and ss_sold_date_sk <= (2450816+100);
INFO : Status: Running (Executing on YARN cluster with App id
application_1606875286859_0001)
--
VERTICES MODESTATUS TOTAL COMPLETED RUNNING PENDING
FAILED KILLED
--
Map 1 .. llap SUCCEEDED 1 100
0 0
Reducer 2 .. llap SUCCEEDED 2 200
0 5
Reducer 3 .. llap SUCCEEDED 2 200
0 9
--
VERTICES: 03/03 [==>>] 100% ELAPSED TIME: 28.61 s
--
INFO : Status: DAG finished successfully in 18.72 seconds
INFO :
INFO : Query Execution Summary
INFO :
--
INFO : OPERATIONDURATION
INFO :
--
INFO : Compile Query 14.06s
INFO : Prepare Plan0.17s
INFO : Get Query Coordinator (AM) 0.14s
INFO : Submit Plan 0.03s
INFO : Start DAG 0.05s
INFO : Run DAG18.72s
INFO :
--
{noformat}
It would be good to stop launching the job, when the condition is not valid in
the query.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-24465) RuntimeException Invalid Stats number of null > no of tuples when fetch column stats is enabled
Aman Sinha created HIVE-24465:
-
Summary: RuntimeException Invalid Stats number of null > no of
tuples when fetch column stats is enabled
Key: HIVE-24465
URL: https://issues.apache.org/jira/browse/HIVE-24465
Project: Hive
Issue Type: Bug
Components: Query Planning
Reporter: Aman Sinha
Assignee: Aman Sinha
Seen in a downstream build. To reproduce, set hive.stats.fetch.column.stats to
true and execute TPC-DS q8. The FilterSelectivityEstimator.java hits the
following error:
{noformat}
RuntimeException: Invalid Stats number of null > no of tuples
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-24464) Evaluate the need to have directSQL implementation for data connectors
Naveen Gangam created HIVE-24464: Summary: Evaluate the need to have directSQL implementation for data connectors Key: HIVE-24464 URL: https://issues.apache.org/jira/browse/HIVE-24464 Project: Hive Issue Type: Sub-task Reporter: Naveen Gangam I expect that there will be just a handful of connectors not 100's of them like databases. But creating a placeholder item to evaluate at a future time. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24463) Add special case for Derby and MySQL in Get Next ID DbNotificationListener
David Mollitor created HIVE-24463:
-
Summary: Add special case for Derby and MySQL in Get Next ID
DbNotificationListener
Key: HIVE-24463
URL: https://issues.apache.org/jira/browse/HIVE-24463
Project: Hive
Issue Type: Improvement
Reporter: David Mollitor
Assignee: David Mollitor
* Derby does not support {{SELECT FOR UPDATE}} statements
* MySQL can be optimized to use {{LAST_INSERT_ID()}}
Debry tables are locked in other parts of the code already, but not in this
path.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-24462) JDBC: Support bearer token authentication
Gopal Vijayaraghavan created HIVE-24462: --- Summary: JDBC: Support bearer token authentication Key: HIVE-24462 URL: https://issues.apache.org/jira/browse/HIVE-24462 Project: Hive Issue Type: Improvement Components: JDBC Affects Versions: 4.0.0 Reporter: Gopal Vijayaraghavan SPENGO authentication (Negotiate) authentication is the only way to authenticate a user without providing a password in JDBC. The SPN model for that fails when load-balancing is used (see HIVE-20583). Add a native JDBC equivalent for the Knox flow, but for POST requests with appropriate Authorization bearer tokens. https://knox.apache.org/books/knox-1-1-0/knoxsso_integration.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24461) Provide CachedStore implementation for dataconnectors
Naveen Gangam created HIVE-24461: Summary: Provide CachedStore implementation for dataconnectors Key: HIVE-24461 URL: https://issues.apache.org/jira/browse/HIVE-24461 Project: Hive Issue Type: Sub-task Reporter: Naveen Gangam Currently, none of the connectors are cached. They are all delegated to the ObjectStore for every call. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24460) Refactor Get Next Event ID for DbNotificationListener
David Mollitor created HIVE-24460: - Summary: Refactor Get Next Event ID for DbNotificationListener Key: HIVE-24460 URL: https://issues.apache.org/jira/browse/HIVE-24460 Project: Hive Issue Type: Improvement Reporter: David Mollitor Assignee: David Mollitor Refactor event ID generation to match notification log ID generation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24459) Qtest to simulate query-based compaction
Karen Coppage created HIVE-24459: Summary: Qtest to simulate query-based compaction Key: HIVE-24459 URL: https://issues.apache.org/jira/browse/HIVE-24459 Project: Hive Issue Type: Test Reporter: Karen Coppage Assignee: Karen Coppage AFAIK all compaction tests run on a local filesystem, and none run on HDFS. Since HDFS and local filesystem behavior differs sometimes, it would be good to test query-based compaction on HDFS. Compaction threads don't run in the qtest environment so this qtest would simulate QB compaction by running the queries that QB compaction runs. -- This message was sent by Atlassian Jira (v8.3.4#803005)
