[jira] [Updated] (HIVE-17013) Delete request with a subquery based on select over a view
[
https://issues.apache.org/jira/browse/HIVE-17013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-17013:
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Delete request with a subquery based on select over a view
> --
>
> Key: HIVE-17013
> URL: https://issues.apache.org/jira/browse/HIVE-17013
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Frédéric ESCANDELL
>Assignee: Eugene Koifman
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: acid_view_bug.q, HIVE-17013.01.patch, HIVE-17013.01.patch
>
>
> Hi,
> I based my DDL on this exemple
> https://fr.hortonworks.com/tutorial/using-hive-acid-transactions-to-insert-update-and-delete-data/.
> In a delete request, the use of a view in a subquery throw an exception :
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
> {code:sql}
> drop table if exists mydim;
> create table mydim (key int, name string, zip string, is_current boolean)
> clustered by(key) into 3 buckets
> stored as orc tblproperties ('transactional'='true');
> insert into mydim values
> (1, 'bob', '95136', true),
> (2, 'joe', '70068', true),
> (3, 'steve', '22150', true);
> drop table if exists updates_staging_table;
> create table updates_staging_table (key int, newzip string);
> insert into updates_staging_table values (1, 87102), (3, 45220);
> drop view if exists updates_staging_view;
> create view updates_staging_view (key, newzip) as select key, newzip from
> updates_staging_table;
> delete from mydim
> where mydim.key in (select key from updates_staging_view);
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17013) Delete request with a subquery based on select over a view
[
https://issues.apache.org/jira/browse/HIVE-17013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169198#comment-16169198
]
Matt McCline commented on HIVE-17013:
-
Thank you [~ashutoshc] for your review.
Committed to master under Eugene's name.
> Delete request with a subquery based on select over a view
> --
>
> Key: HIVE-17013
> URL: https://issues.apache.org/jira/browse/HIVE-17013
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Frédéric ESCANDELL
>Assignee: Eugene Koifman
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: acid_view_bug.q, HIVE-17013.01.patch, HIVE-17013.01.patch
>
>
> Hi,
> I based my DDL on this exemple
> https://fr.hortonworks.com/tutorial/using-hive-acid-transactions-to-insert-update-and-delete-data/.
> In a delete request, the use of a view in a subquery throw an exception :
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
> {code:sql}
> drop table if exists mydim;
> create table mydim (key int, name string, zip string, is_current boolean)
> clustered by(key) into 3 buckets
> stored as orc tblproperties ('transactional'='true');
> insert into mydim values
> (1, 'bob', '95136', true),
> (2, 'joe', '70068', true),
> (3, 'steve', '22150', true);
> drop table if exists updates_staging_table;
> create table updates_staging_table (key int, newzip string);
> insert into updates_staging_table values (1, 87102), (3, 45220);
> drop view if exists updates_staging_view;
> create view updates_staging_view (key, newzip) as select key, newzip from
> updates_staging_table;
> delete from mydim
> where mydim.key in (select key from updates_staging_view);
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17013) Delete request with a subquery based on select over a view
[
https://issues.apache.org/jira/browse/HIVE-17013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-17013:
Fix Version/s: 3.0.0
> Delete request with a subquery based on select over a view
> --
>
> Key: HIVE-17013
> URL: https://issues.apache.org/jira/browse/HIVE-17013
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Frédéric ESCANDELL
>Assignee: Eugene Koifman
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: acid_view_bug.q, HIVE-17013.01.patch, HIVE-17013.01.patch
>
>
> Hi,
> I based my DDL on this exemple
> https://fr.hortonworks.com/tutorial/using-hive-acid-transactions-to-insert-update-and-delete-data/.
> In a delete request, the use of a view in a subquery throw an exception :
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
> {code:sql}
> drop table if exists mydim;
> create table mydim (key int, name string, zip string, is_current boolean)
> clustered by(key) into 3 buckets
> stored as orc tblproperties ('transactional'='true');
> insert into mydim values
> (1, 'bob', '95136', true),
> (2, 'joe', '70068', true),
> (3, 'steve', '22150', true);
> drop table if exists updates_staging_table;
> create table updates_staging_table (key int, newzip string);
> insert into updates_staging_table values (1, 87102), (3, 45220);
> drop view if exists updates_staging_view;
> create view updates_staging_view (key, newzip) as select key, newzip from
> updates_staging_table;
> delete from mydim
> where mydim.key in (select key from updates_staging_view);
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-13290) Support primary keys/foreign keys constraint as part of create table command in Hive
[
https://issues.apache.org/jira/browse/HIVE-13290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169196#comment-16169196
]
coffee commented on HIVE-13290:
---
Hive 2.1.1 still not support primary keys/foreign keys?
hive> select version();
OK
{color:red}2.1.1 {color}r1af77bbf8356e86cabbed92cfa8cc2e1470a1d5c
Time taken: 0.091 seconds, Fetched: 1 row(s)
hive> create table pk(id1 integer, id2 integer,primary key(id1, id2) disable
novalidate);
NoViableAltException(26@[])
at
org.apache.hadoop.hive.ql.parse.HiveParser.type(HiveParser.java:42781)
at
org.apache.hadoop.hive.ql.parse.HiveParser.colType(HiveParser.java:42538)
at
org.apache.hadoop.hive.ql.parse.HiveParser.columnNameType(HiveParser.java:42058)
at
org.apache.hadoop.hive.ql.parse.HiveParser.columnNameTypeOrPKOrFK(HiveParser.java:42308)
at
org.apache.hadoop.hive.ql.parse.HiveParser.columnNameTypeOrPKOrFKList(HiveParser.java:37938)
at
org.apache.hadoop.hive.ql.parse.HiveParser.createTableStatement(HiveParser.java:5259)
at
org.apache.hadoop.hive.ql.parse.HiveParser.ddlStatement(HiveParser.java:2763)
at
org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:1756)
at
org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1178)
at
org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:204)
at
org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:166)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:444)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1242)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1384)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1171)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1161)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:232)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:183)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:399)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:776)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:714)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
FAILED: ParseException line 1:20 cannot recognize input near 'integer' ','
'id2' in column type
hive> create table fk(id1 integer, id2 integer, constraint c1 foreign key(id1,
id2) references pk(id2, id1) disable novalidate);
NoViableAltException(26@[])
at
org.apache.hadoop.hive.ql.parse.HiveParser.type(HiveParser.java:42781)
at
org.apache.hadoop.hive.ql.parse.HiveParser.colType(HiveParser.java:42538)
at
org.apache.hadoop.hive.ql.parse.HiveParser.columnNameType(HiveParser.java:42058)
at
org.apache.hadoop.hive.ql.parse.HiveParser.columnNameTypeOrPKOrFK(HiveParser.java:42308)
at
org.apache.hadoop.hive.ql.parse.HiveParser.columnNameTypeOrPKOrFKList(HiveParser.java:37938)
at
org.apache.hadoop.hive.ql.parse.HiveParser.createTableStatement(HiveParser.java:5259)
at
org.apache.hadoop.hive.ql.parse.HiveParser.ddlStatement(HiveParser.java:2763)
at
org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:1756)
at
org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1178)
at
org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:204)
at
org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:166)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:444)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1242)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1384)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1171)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1161)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:232)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:183)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:399)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:776)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:714)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
[jira] [Commented] (HIVE-17535) Select 1 EXCEPT Select 1 fails with NPE
[
https://issues.apache.org/jira/browse/HIVE-17535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169195#comment-16169195
]
Hive QA commented on HIVE-17535:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12887523/HIVE-17535.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 25 failed/errored test(s), 11040 tests
executed
*Failed tests:*
{noformat}
TestAccumuloCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=230)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[mapjoin2]
(batchId=239)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[select_dummy_source]
(batchId=239)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_view] (batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[decimal_precision2]
(batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin2] (batchId=10)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[min_structvalue]
(batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[select_dummy_source]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz_1]
(batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_mask_hash]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udtf_stack] (batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_tablesample_rows]
(batchId=49)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[explainuser_1]
(batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mapjoin2]
(batchId=147)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[select_dummy_source]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1]
(batchId=170)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[drop_table_failure2]
(batchId=89)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_missing_from]
(batchId=89)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_select_no_source]
(batchId=89)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=234)
org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=215)
org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01
(batchId=215)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6844/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6844/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6844/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 25 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12887523 - PreCommit-HIVE-Build
> Select 1 EXCEPT Select 1 fails with NPE
> ---
>
> Key: HIVE-17535
> URL: https://issues.apache.org/jira/browse/HIVE-17535
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
> Attachments: HIVE-17535.1.patch, HIVE-17535.2.patch
>
>
> Since Hive CBO isn't able to handle queries with no table e.g. {{select 1}}
> queries with SET operators fail (intersect requires CBO).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17013) Delete request with a subquery based on select over a view
[
https://issues.apache.org/jira/browse/HIVE-17013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169189#comment-16169189
]
Ashutosh Chauhan commented on HIVE-17013:
-
+1
> Delete request with a subquery based on select over a view
> --
>
> Key: HIVE-17013
> URL: https://issues.apache.org/jira/browse/HIVE-17013
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Frédéric ESCANDELL
>Assignee: Eugene Koifman
>Priority: Blocker
> Attachments: acid_view_bug.q, HIVE-17013.01.patch, HIVE-17013.01.patch
>
>
> Hi,
> I based my DDL on this exemple
> https://fr.hortonworks.com/tutorial/using-hive-acid-transactions-to-insert-update-and-delete-data/.
> In a delete request, the use of a view in a subquery throw an exception :
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
> {code:sql}
> drop table if exists mydim;
> create table mydim (key int, name string, zip string, is_current boolean)
> clustered by(key) into 3 buckets
> stored as orc tblproperties ('transactional'='true');
> insert into mydim values
> (1, 'bob', '95136', true),
> (2, 'joe', '70068', true),
> (3, 'steve', '22150', true);
> drop table if exists updates_staging_table;
> create table updates_staging_table (key int, newzip string);
> insert into updates_staging_table values (1, 87102), (3, 45220);
> drop view if exists updates_staging_view;
> create view updates_staging_view (key, newzip) as select key, newzip from
> updates_staging_table;
> delete from mydim
> where mydim.key in (select key from updates_staging_view);
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-15899) check CTAS over acid table
[ https://issues.apache.org/jira/browse/HIVE-15899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-15899: -- Attachment: HIVE-15899.10.patch > check CTAS over acid table > --- > > Key: HIVE-15899 > URL: https://issues.apache.org/jira/browse/HIVE-15899 > Project: Hive > Issue Type: Sub-task >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-15899.01.patch, HIVE-15899.02.patch, > HIVE-15899.03.patch, HIVE-15899.04.patch, HIVE-15899.05.patch, > HIVE-15899.07.patch, HIVE-15899.08.patch, HIVE-15899.09.patch, > HIVE-15899.10.patch > > > need to add a test to check if create table as works correctly with acid > tables -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17465) Statistics: Drill-down filters don't reduce row-counts progressively
[
https://issues.apache.org/jira/browse/HIVE-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169185#comment-16169185
]
Hive QA commented on HIVE-17465:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12887516/HIVE-17465.6.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 16 failed/errored test(s), 11040 tests
executed
*Failed tests:*
{noformat}
TestAccumuloCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=230)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_view] (batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin47] (batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin_47]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_mask_hash]
(batchId=28)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=143)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_partitioned]
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1]
(batchId=170)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[drop_table_failure2]
(batchId=89)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=234)
org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=215)
org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01
(batchId=215)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testHttpRetryOnServerIdleTimeout
(batchId=227)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6843/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6843/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6843/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 16 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12887516 - PreCommit-HIVE-Build
> Statistics: Drill-down filters don't reduce row-counts progressively
>
>
> Key: HIVE-17465
> URL: https://issues.apache.org/jira/browse/HIVE-17465
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer, Statistics
>Reporter: Gopal V
>Assignee: Vineet Garg
> Attachments: HIVE-17465.1.patch, HIVE-17465.2.patch,
> HIVE-17465.3.patch, HIVE-17465.4.patch, HIVE-17465.5.patch, HIVE-17465.6.patch
>
>
> {code}
> explain select count(d_date_sk) from date_dim where d_year=2001 ;
> explain select count(d_date_sk) from date_dim where d_year=2001 and d_moy =
> 9;
> explain select count(d_date_sk) from date_dim where d_year=2001 and d_moy = 9
> and d_dom = 21;
> {code}
> All 3 queries end up with the same row-count estimates after the filter.
> {code}
> Map Operator Tree:
> TableScan
> alias: date_dim
> filterExpr: (d_year = 2001) (type: boolean)
> Statistics: Num rows: 73049 Data size: 82034027 Basic
> stats: COMPLETE Column stats: COMPLETE
> Filter Operator
> predicate: (d_year = 2001) (type: boolean)
> Statistics: Num rows: 363 Data size: 4356 Basic stats:
> COMPLETE Column stats: COMPLETE
>
> Map 1
> Map Operator Tree:
> TableScan
> alias: date_dim
> filterExpr: ((d_year = 2001) and (d_moy = 9)) (type:
> boolean)
> Statistics: Num rows: 73049 Data size: 82034027 Basic
> stats: COMPLETE Column stats: COMPLETE
> Filter Operator
> predicate: ((d_year = 2001) and (d_moy = 9)) (type:
> boolean)
> Statistics: Num rows: 363 Data size: 5808 Basic stats:
> COMPLETE Column stats: COMPLETE
> Map 1
> Map Operator Tree:
> TableScan
> alias: date_dim
> filterExpr: ((d_year = 2001) and (d_moy = 9) and (d_dom =
> 21)) (type: boolean)
> Statistics: Num rows: 73049 Data size: 82034027 Basic
> stats: COMPLETE Column stats: COMPLETE
> Filter Operator
> predicate:
[jira] [Comment Edited] (HIVE-17013) Delete request with a subquery based on select over a view
[
https://issues.apache.org/jira/browse/HIVE-17013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169164#comment-16169164
]
Matt McCline edited comment on HIVE-17013 at 9/17/17 2:37 AM:
--
I looked at the code. I'm not familiar with this area of the code.
[~ashutoshc] [~jcamachorodriguez] [~vgarg] can you review ASAP. Thanks.
was (Author: mmccline):
I looked at the code. I'm not familiar with this area of the code. Either
someone else should review it or I can do it as long as some else looks at it
later...
+1 provisional
> Delete request with a subquery based on select over a view
> --
>
> Key: HIVE-17013
> URL: https://issues.apache.org/jira/browse/HIVE-17013
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Frédéric ESCANDELL
>Assignee: Eugene Koifman
>Priority: Blocker
> Attachments: acid_view_bug.q, HIVE-17013.01.patch, HIVE-17013.01.patch
>
>
> Hi,
> I based my DDL on this exemple
> https://fr.hortonworks.com/tutorial/using-hive-acid-transactions-to-insert-update-and-delete-data/.
> In a delete request, the use of a view in a subquery throw an exception :
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
> {code:sql}
> drop table if exists mydim;
> create table mydim (key int, name string, zip string, is_current boolean)
> clustered by(key) into 3 buckets
> stored as orc tblproperties ('transactional'='true');
> insert into mydim values
> (1, 'bob', '95136', true),
> (2, 'joe', '70068', true),
> (3, 'steve', '22150', true);
> drop table if exists updates_staging_table;
> create table updates_staging_table (key int, newzip string);
> insert into updates_staging_table values (1, 87102), (3, 45220);
> drop view if exists updates_staging_view;
> create view updates_staging_view (key, newzip) as select key, newzip from
> updates_staging_table;
> delete from mydim
> where mydim.key in (select key from updates_staging_view);
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17538) Enhance estimation of stats to estimate even if only one column is missing stats
[
https://issues.apache.org/jira/browse/HIVE-17538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169172#comment-16169172
]
Hive QA commented on HIVE-17538:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12887217/HIVE-17538.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 116 failed/errored test(s), 11040 tests
executed
*Failed tests:*
{noformat}
TestAccumuloCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=230)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_groupby2]
(batchId=46)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_groupby]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_join_pkfk]
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_part]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_select]
(batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_table]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_12]
(batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_annotate_stats_groupby]
(batchId=81)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_view] (batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mergejoin] (batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[named_column_join]
(batchId=74)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_repeated_alias]
(batchId=10)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats_only_null]
(batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats_partial_size]
(batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats_ppr_all]
(batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_mask_hash]
(batchId=28)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[dynamic_partition_pruning_2]
(batchId=143)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[explainuser_2]
(batchId=144)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=145)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_stats]
(batchId=141)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_merge3]
(batchId=141)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_merge4]
(batchId=144)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[rcfile_createas1]
(batchId=143)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[rcfile_merge3]
(batchId=144)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[rcfile_merge4]
(batchId=141)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[reduce_deduplicate]
(batchId=144)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=143)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_11]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_12]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_1]
(batchId=155)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=155)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_3]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_4]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_7]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_8]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_groupby]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez2]
(batchId=148)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucketmapjoin1]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucketmapjoin2]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucketmapjoin3]
(batchId=155)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucketmapjoin7]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucketpruning1]
(batchId=163)
[jira] [Commented] (HIVE-17013) Delete request with a subquery based on select over a view
[
https://issues.apache.org/jira/browse/HIVE-17013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169164#comment-16169164
]
Matt McCline commented on HIVE-17013:
-
I looked at the code. I'm not familiar with this area of the code. Either
someone else should review it or I can do it as long as some else looks at it
later...
+1 provisional
> Delete request with a subquery based on select over a view
> --
>
> Key: HIVE-17013
> URL: https://issues.apache.org/jira/browse/HIVE-17013
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Frédéric ESCANDELL
>Assignee: Eugene Koifman
>Priority: Blocker
> Attachments: acid_view_bug.q, HIVE-17013.01.patch, HIVE-17013.01.patch
>
>
> Hi,
> I based my DDL on this exemple
> https://fr.hortonworks.com/tutorial/using-hive-acid-transactions-to-insert-update-and-delete-data/.
> In a delete request, the use of a view in a subquery throw an exception :
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
> {code:sql}
> drop table if exists mydim;
> create table mydim (key int, name string, zip string, is_current boolean)
> clustered by(key) into 3 buckets
> stored as orc tblproperties ('transactional'='true');
> insert into mydim values
> (1, 'bob', '95136', true),
> (2, 'joe', '70068', true),
> (3, 'steve', '22150', true);
> drop table if exists updates_staging_table;
> create table updates_staging_table (key int, newzip string);
> insert into updates_staging_table values (1, 87102), (3, 45220);
> drop view if exists updates_staging_view;
> create view updates_staging_view (key, newzip) as select key, newzip from
> updates_staging_table;
> delete from mydim
> where mydim.key in (select key from updates_staging_view);
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17261) Hive use deprecated ParquetInputSplit constructor which blocked parquet dictionary filter
[ https://issues.apache.org/jira/browse/HIVE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169162#comment-16169162 ] Ferdinand Xu commented on HIVE-17261: - Hi [~csun], Vectorization reader can't leverage the existing mechanism of predicate pushing down in ParquetRecordReader. So I just remove the blocking link. Thank you for pointing this out. > Hive use deprecated ParquetInputSplit constructor which blocked parquet > dictionary filter > - > > Key: HIVE-17261 > URL: https://issues.apache.org/jira/browse/HIVE-17261 > Project: Hive > Issue Type: Improvement > Components: Database/Schema >Affects Versions: 2.2.0 >Reporter: Junjie Chen >Assignee: Junjie Chen > Fix For: 3.0.0 > > Attachments: HIVE-17261.10.patch, HIVE-17261.11.patch, > HIVE-17261.2.patch, HIVE-17261.3.patch, HIVE-17261.4.patch, > HIVE-17261.5.patch, HIVE-17261.6.patch, HIVE-17261.7.patch, > HIVE-17261.8.patch, HIVE-17261.diff, HIVE-17261.patch > > > Hive use deprecated ParquetInputSplit in > [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/ParquetRecordReaderBase.java#L128] > Please see interface definition in > [https://github.com/apache/parquet-mr/blob/master/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetInputSplit.java#L80] > Old interface set rowgroupoffset values which will lead to skip dictionary > filter in parquet. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-14836) Test the predicate pushing down support for Parquet vectorization read path
[ https://issues.apache.org/jira/browse/HIVE-14836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169158#comment-16169158 ] Ferdinand Xu commented on HIVE-14836: - Hi [~xuefuz], after further investigation, I found that the vectorization reader supports predicate pushing down when constructing the Parquet file reader. So the work left is about enabling some unit tests covering predicate pushing down. > Test the predicate pushing down support for Parquet vectorization read path > --- > > Key: HIVE-14836 > URL: https://issues.apache.org/jira/browse/HIVE-14836 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.0.0 >Reporter: Ferdinand Xu >Assignee: Ferdinand Xu > Labels: pull-request-available > Attachments: HIVE-14836.patch > > > We should add more UT test for predicate pushing down support for Parquet > vectorization read path. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-14836) Test the predicate pushing down support for Parquet vectorization read path
[ https://issues.apache.org/jira/browse/HIVE-14836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferdinand Xu updated HIVE-14836: Description: We should add more UT test for predicate pushing down support for Parquet vectorization read path. (was: Currently we filter blocks using Predict pushing down. We should support it in page reader as well to improve its efficiency. ) > Test the predicate pushing down support for Parquet vectorization read path > --- > > Key: HIVE-14836 > URL: https://issues.apache.org/jira/browse/HIVE-14836 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.0.0 >Reporter: Ferdinand Xu >Assignee: Ferdinand Xu > Labels: pull-request-available > Attachments: HIVE-14836.patch > > > We should add more UT test for predicate pushing down support for Parquet > vectorization read path. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17493) Improve PKFK cardinality estimation in Physical planning
[
https://issues.apache.org/jira/browse/HIVE-17493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169149#comment-16169149
]
Hive QA commented on HIVE-17493:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12887518/HIVE-17493.4.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 11040 tests
executed
*Failed tests:*
{noformat}
TestAccumuloCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=230)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_view] (batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_mask_hash]
(batchId=28)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
(batchId=156)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[drop_table_failure2]
(batchId=89)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=234)
org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=215)
org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01
(batchId=215)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6841/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6841/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6841/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12887518 - PreCommit-HIVE-Build
> Improve PKFK cardinality estimation in Physical planning
>
>
> Key: HIVE-17493
> URL: https://issues.apache.org/jira/browse/HIVE-17493
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
> Attachments: HIVE-17493.1.patch, HIVE-17493.2.patch,
> HIVE-17493.3.patch, HIVE-17493.4.patch
>
>
> Cardinality estimation of a join, after PK-FK relation has been ascertained,
> could be improved if parent of the join operator is LEFT outer or RIGHT outer
> join.
> Currently estimation is done by estimating reduction of rows occurred on PK
> side, then multiplying the reduction to FK side row count. This estimation of
> reduction currently doesn't distinguish b/w INNER or OUTER joins. This could
> be improved to handle outer joins better.
> TPC-DS query45 is impacted by this.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17536) StatsUtil::getBasicStatForTable doesn't distinguish b/w absence of statistics or zero stats
[
https://issues.apache.org/jira/browse/HIVE-17536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169133#comment-16169133
]
Hive QA commented on HIVE-17536:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12887515/HIVE-17536.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 123 failed/errored test(s), 11040 tests
executed
*Failed tests:*
{noformat}
TestAccumuloCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=230)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_1]
(batchId=239)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_2]
(batchId=239)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_3]
(batchId=239)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_5]
(batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_5]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_6]
(batchId=80)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin5]
(batchId=81)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin_negative2]
(batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin_negative3]
(batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin_negative]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_join1]
(batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_udaf_percentile_approx_23]
(batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_tbllvl]
(batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[compute_stats_date]
(batchId=43)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlationoptimizer5]
(batchId=67)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_view] (batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[decimal_precision]
(batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[decimal_udf2]
(batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[decimal_udf] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[display_colstats_tbllvl]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[distinct_windowing]
(batchId=11)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[distinct_windowing_no_cbo]
(batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[drop_table_with_index]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[filter_cond_pushdown2]
(batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[gen_udf_example_add10]
(batchId=43)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby10] (batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_grouping_id3]
(batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_grouping_sets_grouping]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[index_serde] (batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input22] (batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input3_limit]
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input4] (batchId=78)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_star] (batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nullscript] (batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge5] (batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge6] (batchId=33)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge_incompat1]
(batchId=65)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge_incompat2]
(batchId=80)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ptf_matchpath]
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin_1]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin_25]
(batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin_2]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin_3]
(batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_in_having]
(batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udaf_percentile_approx_23]
(batchId=33)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_mask_hash]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_trunc_number]
(batchId=11)
[jira] [Commented] (HIVE-15899) check CTAS over acid table
[
https://issues.apache.org/jira/browse/HIVE-15899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169075#comment-16169075
]
Hive QA commented on HIVE-15899:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12887512/HIVE-15899.09.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 30 failed/errored test(s), 11047 tests
executed
*Failed tests:*
{noformat}
TestAccumuloCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=230)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join25] (batchId=70)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join_stats]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join_without_localtask]
(batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_14]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_15]
(batchId=11)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_1]
(batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_3]
(batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_4]
(batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_5]
(batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_7]
(batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketsortoptimize_insert_4]
(batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketsortoptimize_insert_5]
(batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketsortoptimize_insert_8]
(batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_view] (batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[infer_bucket_sort_convert_join]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[runtime_skewjoin_mapjoin_spark]
(batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[tez_join_hash]
(batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_mask_hash]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union22] (batchId=13)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=100)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[drop_table_failure2]
(batchId=89)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=234)
org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion02
(batchId=215)
org.apache.hadoop.hive.ql.security.TestMultiAuthorizationPreEventListener.org.apache.hadoop.hive.ql.security.TestMultiAuthorizationPreEventListener
(batchId=220)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6839/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6839/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6839/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 30 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12887512 - PreCommit-HIVE-Build
> check CTAS over acid table
> ---
>
> Key: HIVE-15899
> URL: https://issues.apache.org/jira/browse/HIVE-15899
> Project: Hive
> Issue Type: Sub-task
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
> Attachments: HIVE-15899.01.patch, HIVE-15899.02.patch,
> HIVE-15899.03.patch, HIVE-15899.04.patch, HIVE-15899.05.patch,
> HIVE-15899.07.patch, HIVE-15899.08.patch, HIVE-15899.09.patch
>
>
> need to add a test to check if create table as works correctly with acid
> tables
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17013) Delete request with a subquery based on select over a view
[
https://issues.apache.org/jira/browse/HIVE-17013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169072#comment-16169072
]
Eugene Koifman commented on HIVE-17013:
---
no related failures
(https://builds.apache.org/job/PreCommit-HIVE-Build/6836/testReport/ for
example includes all failures above)
[~mmccline] could you review
> Delete request with a subquery based on select over a view
> --
>
> Key: HIVE-17013
> URL: https://issues.apache.org/jira/browse/HIVE-17013
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Frédéric ESCANDELL
>Assignee: Eugene Koifman
>Priority: Blocker
> Attachments: acid_view_bug.q, HIVE-17013.01.patch, HIVE-17013.01.patch
>
>
> Hi,
> I based my DDL on this exemple
> https://fr.hortonworks.com/tutorial/using-hive-acid-transactions-to-insert-update-and-delete-data/.
> In a delete request, the use of a view in a subquery throw an exception :
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
> {code:sql}
> drop table if exists mydim;
> create table mydim (key int, name string, zip string, is_current boolean)
> clustered by(key) into 3 buckets
> stored as orc tblproperties ('transactional'='true');
> insert into mydim values
> (1, 'bob', '95136', true),
> (2, 'joe', '70068', true),
> (3, 'steve', '22150', true);
> drop table if exists updates_staging_table;
> create table updates_staging_table (key int, newzip string);
> insert into updates_staging_table values (1, 87102), (3, 45220);
> drop view if exists updates_staging_view;
> create view updates_staging_view (key, newzip) as select key, newzip from
> updates_staging_table;
> delete from mydim
> where mydim.key in (select key from updates_staging_view);
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17535) Select 1 EXCEPT Select 1 fails with NPE
[
https://issues.apache.org/jira/browse/HIVE-17535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-17535:
---
Attachment: HIVE-17535.2.patch
> Select 1 EXCEPT Select 1 fails with NPE
> ---
>
> Key: HIVE-17535
> URL: https://issues.apache.org/jira/browse/HIVE-17535
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
> Attachments: HIVE-17535.1.patch, HIVE-17535.2.patch
>
>
> Since Hive CBO isn't able to handle queries with no table e.g. {{select 1}}
> queries with SET operators fail (intersect requires CBO).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17535) Select 1 EXCEPT Select 1 fails with NPE
[
https://issues.apache.org/jira/browse/HIVE-17535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-17535:
---
Status: Open (was: Patch Available)
> Select 1 EXCEPT Select 1 fails with NPE
> ---
>
> Key: HIVE-17535
> URL: https://issues.apache.org/jira/browse/HIVE-17535
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
> Attachments: HIVE-17535.1.patch, HIVE-17535.2.patch
>
>
> Since Hive CBO isn't able to handle queries with no table e.g. {{select 1}}
> queries with SET operators fail (intersect requires CBO).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17535) Select 1 EXCEPT Select 1 fails with NPE
[
https://issues.apache.org/jira/browse/HIVE-17535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-17535:
---
Status: Patch Available (was: Open)
> Select 1 EXCEPT Select 1 fails with NPE
> ---
>
> Key: HIVE-17535
> URL: https://issues.apache.org/jira/browse/HIVE-17535
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
> Attachments: HIVE-17535.1.patch, HIVE-17535.2.patch
>
>
> Since Hive CBO isn't able to handle queries with no table e.g. {{select 1}}
> queries with SET operators fail (intersect requires CBO).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17013) Delete request with a subquery based on select over a view
[
https://issues.apache.org/jira/browse/HIVE-17013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169058#comment-16169058
]
Hive QA commented on HIVE-17013:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12887511/HIVE-17013.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 11041 tests
executed
*Failed tests:*
{noformat}
TestAccumuloCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=230)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_view] (batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_mask_hash]
(batchId=28)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
(batchId=156)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[drop_table_failure2]
(batchId=89)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=234)
org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=215)
org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01
(batchId=215)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6838/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6838/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6838/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12887511 - PreCommit-HIVE-Build
> Delete request with a subquery based on select over a view
> --
>
> Key: HIVE-17013
> URL: https://issues.apache.org/jira/browse/HIVE-17013
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Frédéric ESCANDELL
>Assignee: Eugene Koifman
>Priority: Blocker
> Attachments: acid_view_bug.q, HIVE-17013.01.patch, HIVE-17013.01.patch
>
>
> Hi,
> I based my DDL on this exemple
> https://fr.hortonworks.com/tutorial/using-hive-acid-transactions-to-insert-update-and-delete-data/.
> In a delete request, the use of a view in a subquery throw an exception :
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
> {code:sql}
> drop table if exists mydim;
> create table mydim (key int, name string, zip string, is_current boolean)
> clustered by(key) into 3 buckets
> stored as orc tblproperties ('transactional'='true');
> insert into mydim values
> (1, 'bob', '95136', true),
> (2, 'joe', '70068', true),
> (3, 'steve', '22150', true);
> drop table if exists updates_staging_table;
> create table updates_staging_table (key int, newzip string);
> insert into updates_staging_table values (1, 87102), (3, 45220);
> drop view if exists updates_staging_view;
> create view updates_staging_view (key, newzip) as select key, newzip from
> updates_staging_table;
> delete from mydim
> where mydim.key in (select key from updates_staging_view);
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17538) Enhance estimation of stats to estimate even if only one column is missing stats
[ https://issues.apache.org/jira/browse/HIVE-17538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17538: --- Status: Patch Available (was: Open) > Enhance estimation of stats to estimate even if only one column is missing > stats > > > Key: HIVE-17538 > URL: https://issues.apache.org/jira/browse/HIVE-17538 > Project: Hive > Issue Type: Improvement >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-17538.1.patch > > > HIVE-16811 provided support for estimating statistics in absence of stats. > But that estimation is done if and only if statistics are missing for all > columns. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17493) Improve PKFK cardinality estimation in Physical planning
[ https://issues.apache.org/jira/browse/HIVE-17493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17493: --- Attachment: HIVE-17493.4.patch Uploading rebased patch to trigger qtests. > Improve PKFK cardinality estimation in Physical planning > > > Key: HIVE-17493 > URL: https://issues.apache.org/jira/browse/HIVE-17493 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-17493.1.patch, HIVE-17493.2.patch, > HIVE-17493.3.patch, HIVE-17493.4.patch > > > Cardinality estimation of a join, after PK-FK relation has been ascertained, > could be improved if parent of the join operator is LEFT outer or RIGHT outer > join. > Currently estimation is done by estimating reduction of rows occurred on PK > side, then multiplying the reduction to FK side row count. This estimation of > reduction currently doesn't distinguish b/w INNER or OUTER joins. This could > be improved to handle outer joins better. > TPC-DS query45 is impacted by this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17493) Improve PKFK cardinality estimation in Physical planning
[ https://issues.apache.org/jira/browse/HIVE-17493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17493: --- Status: Patch Available (was: Open) > Improve PKFK cardinality estimation in Physical planning > > > Key: HIVE-17493 > URL: https://issues.apache.org/jira/browse/HIVE-17493 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-17493.1.patch, HIVE-17493.2.patch, > HIVE-17493.3.patch, HIVE-17493.4.patch > > > Cardinality estimation of a join, after PK-FK relation has been ascertained, > could be improved if parent of the join operator is LEFT outer or RIGHT outer > join. > Currently estimation is done by estimating reduction of rows occurred on PK > side, then multiplying the reduction to FK side row count. This estimation of > reduction currently doesn't distinguish b/w INNER or OUTER joins. This could > be improved to handle outer joins better. > TPC-DS query45 is impacted by this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17493) Improve PKFK cardinality estimation in Physical planning
[ https://issues.apache.org/jira/browse/HIVE-17493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17493: --- Status: Open (was: Patch Available) > Improve PKFK cardinality estimation in Physical planning > > > Key: HIVE-17493 > URL: https://issues.apache.org/jira/browse/HIVE-17493 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-17493.1.patch, HIVE-17493.2.patch, > HIVE-17493.3.patch > > > Cardinality estimation of a join, after PK-FK relation has been ascertained, > could be improved if parent of the join operator is LEFT outer or RIGHT outer > join. > Currently estimation is done by estimating reduction of rows occurred on PK > side, then multiplying the reduction to FK side row count. This estimation of > reduction currently doesn't distinguish b/w INNER or OUTER joins. This could > be improved to handle outer joins better. > TPC-DS query45 is impacted by this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17465) Statistics: Drill-down filters don't reduce row-counts progressively
[
https://issues.apache.org/jira/browse/HIVE-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-17465:
---
Attachment: HIVE-17465.6.patch
For some reason ptests are not being scheduled for this issue, so rebasing and
re-uploading new patch. Hopefully ptests will be triggered now.
> Statistics: Drill-down filters don't reduce row-counts progressively
>
>
> Key: HIVE-17465
> URL: https://issues.apache.org/jira/browse/HIVE-17465
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer, Statistics
>Reporter: Gopal V
>Assignee: Vineet Garg
> Attachments: HIVE-17465.1.patch, HIVE-17465.2.patch,
> HIVE-17465.3.patch, HIVE-17465.4.patch, HIVE-17465.5.patch, HIVE-17465.6.patch
>
>
> {code}
> explain select count(d_date_sk) from date_dim where d_year=2001 ;
> explain select count(d_date_sk) from date_dim where d_year=2001 and d_moy =
> 9;
> explain select count(d_date_sk) from date_dim where d_year=2001 and d_moy = 9
> and d_dom = 21;
> {code}
> All 3 queries end up with the same row-count estimates after the filter.
> {code}
> Map Operator Tree:
> TableScan
> alias: date_dim
> filterExpr: (d_year = 2001) (type: boolean)
> Statistics: Num rows: 73049 Data size: 82034027 Basic
> stats: COMPLETE Column stats: COMPLETE
> Filter Operator
> predicate: (d_year = 2001) (type: boolean)
> Statistics: Num rows: 363 Data size: 4356 Basic stats:
> COMPLETE Column stats: COMPLETE
>
> Map 1
> Map Operator Tree:
> TableScan
> alias: date_dim
> filterExpr: ((d_year = 2001) and (d_moy = 9)) (type:
> boolean)
> Statistics: Num rows: 73049 Data size: 82034027 Basic
> stats: COMPLETE Column stats: COMPLETE
> Filter Operator
> predicate: ((d_year = 2001) and (d_moy = 9)) (type:
> boolean)
> Statistics: Num rows: 363 Data size: 5808 Basic stats:
> COMPLETE Column stats: COMPLETE
> Map 1
> Map Operator Tree:
> TableScan
> alias: date_dim
> filterExpr: ((d_year = 2001) and (d_moy = 9) and (d_dom =
> 21)) (type: boolean)
> Statistics: Num rows: 73049 Data size: 82034027 Basic
> stats: COMPLETE Column stats: COMPLETE
> Filter Operator
> predicate: ((d_year = 2001) and (d_moy = 9) and (d_dom =
> 21)) (type: boolean)
> Statistics: Num rows: 363 Data size: 7260 Basic stats:
> COMPLETE Column stats: COMPLETE
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17465) Statistics: Drill-down filters don't reduce row-counts progressively
[
https://issues.apache.org/jira/browse/HIVE-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-17465:
---
Status: Patch Available (was: Open)
> Statistics: Drill-down filters don't reduce row-counts progressively
>
>
> Key: HIVE-17465
> URL: https://issues.apache.org/jira/browse/HIVE-17465
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer, Statistics
>Reporter: Gopal V
>Assignee: Vineet Garg
> Attachments: HIVE-17465.1.patch, HIVE-17465.2.patch,
> HIVE-17465.3.patch, HIVE-17465.4.patch, HIVE-17465.5.patch, HIVE-17465.6.patch
>
>
> {code}
> explain select count(d_date_sk) from date_dim where d_year=2001 ;
> explain select count(d_date_sk) from date_dim where d_year=2001 and d_moy =
> 9;
> explain select count(d_date_sk) from date_dim where d_year=2001 and d_moy = 9
> and d_dom = 21;
> {code}
> All 3 queries end up with the same row-count estimates after the filter.
> {code}
> Map Operator Tree:
> TableScan
> alias: date_dim
> filterExpr: (d_year = 2001) (type: boolean)
> Statistics: Num rows: 73049 Data size: 82034027 Basic
> stats: COMPLETE Column stats: COMPLETE
> Filter Operator
> predicate: (d_year = 2001) (type: boolean)
> Statistics: Num rows: 363 Data size: 4356 Basic stats:
> COMPLETE Column stats: COMPLETE
>
> Map 1
> Map Operator Tree:
> TableScan
> alias: date_dim
> filterExpr: ((d_year = 2001) and (d_moy = 9)) (type:
> boolean)
> Statistics: Num rows: 73049 Data size: 82034027 Basic
> stats: COMPLETE Column stats: COMPLETE
> Filter Operator
> predicate: ((d_year = 2001) and (d_moy = 9)) (type:
> boolean)
> Statistics: Num rows: 363 Data size: 5808 Basic stats:
> COMPLETE Column stats: COMPLETE
> Map 1
> Map Operator Tree:
> TableScan
> alias: date_dim
> filterExpr: ((d_year = 2001) and (d_moy = 9) and (d_dom =
> 21)) (type: boolean)
> Statistics: Num rows: 73049 Data size: 82034027 Basic
> stats: COMPLETE Column stats: COMPLETE
> Filter Operator
> predicate: ((d_year = 2001) and (d_moy = 9) and (d_dom =
> 21)) (type: boolean)
> Statistics: Num rows: 363 Data size: 7260 Basic stats:
> COMPLETE Column stats: COMPLETE
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17465) Statistics: Drill-down filters don't reduce row-counts progressively
[
https://issues.apache.org/jira/browse/HIVE-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-17465:
---
Status: Open (was: Patch Available)
> Statistics: Drill-down filters don't reduce row-counts progressively
>
>
> Key: HIVE-17465
> URL: https://issues.apache.org/jira/browse/HIVE-17465
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer, Statistics
>Reporter: Gopal V
>Assignee: Vineet Garg
> Attachments: HIVE-17465.1.patch, HIVE-17465.2.patch,
> HIVE-17465.3.patch, HIVE-17465.4.patch, HIVE-17465.5.patch
>
>
> {code}
> explain select count(d_date_sk) from date_dim where d_year=2001 ;
> explain select count(d_date_sk) from date_dim where d_year=2001 and d_moy =
> 9;
> explain select count(d_date_sk) from date_dim where d_year=2001 and d_moy = 9
> and d_dom = 21;
> {code}
> All 3 queries end up with the same row-count estimates after the filter.
> {code}
> Map Operator Tree:
> TableScan
> alias: date_dim
> filterExpr: (d_year = 2001) (type: boolean)
> Statistics: Num rows: 73049 Data size: 82034027 Basic
> stats: COMPLETE Column stats: COMPLETE
> Filter Operator
> predicate: (d_year = 2001) (type: boolean)
> Statistics: Num rows: 363 Data size: 4356 Basic stats:
> COMPLETE Column stats: COMPLETE
>
> Map 1
> Map Operator Tree:
> TableScan
> alias: date_dim
> filterExpr: ((d_year = 2001) and (d_moy = 9)) (type:
> boolean)
> Statistics: Num rows: 73049 Data size: 82034027 Basic
> stats: COMPLETE Column stats: COMPLETE
> Filter Operator
> predicate: ((d_year = 2001) and (d_moy = 9)) (type:
> boolean)
> Statistics: Num rows: 363 Data size: 5808 Basic stats:
> COMPLETE Column stats: COMPLETE
> Map 1
> Map Operator Tree:
> TableScan
> alias: date_dim
> filterExpr: ((d_year = 2001) and (d_moy = 9) and (d_dom =
> 21)) (type: boolean)
> Statistics: Num rows: 73049 Data size: 82034027 Basic
> stats: COMPLETE Column stats: COMPLETE
> Filter Operator
> predicate: ((d_year = 2001) and (d_moy = 9) and (d_dom =
> 21)) (type: boolean)
> Statistics: Num rows: 363 Data size: 7260 Basic stats:
> COMPLETE Column stats: COMPLETE
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17536) StatsUtil::getBasicStatForTable doesn't distinguish b/w absence of statistics or zero stats
[ https://issues.apache.org/jira/browse/HIVE-17536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17536: --- Attachment: HIVE-17536.3.patch > StatsUtil::getBasicStatForTable doesn't distinguish b/w absence of statistics > or zero stats > --- > > Key: HIVE-17536 > URL: https://issues.apache.org/jira/browse/HIVE-17536 > Project: Hive > Issue Type: Improvement > Components: Statistics >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-17536.1.patch, HIVE-17536.2.patch, > HIVE-17536.3.patch > > > This method returns zero for both of the following cases: > * Statistics are missing in metastore > * Actual stats e.g. number of rows are zero > It'll be good for this method to return e.g. -1 in absence of statistics > instead of assuming it to be zero. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17536) StatsUtil::getBasicStatForTable doesn't distinguish b/w absence of statistics or zero stats
[ https://issues.apache.org/jira/browse/HIVE-17536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17536: --- Status: Patch Available (was: Open) > StatsUtil::getBasicStatForTable doesn't distinguish b/w absence of statistics > or zero stats > --- > > Key: HIVE-17536 > URL: https://issues.apache.org/jira/browse/HIVE-17536 > Project: Hive > Issue Type: Improvement > Components: Statistics >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-17536.1.patch, HIVE-17536.2.patch, > HIVE-17536.3.patch > > > This method returns zero for both of the following cases: > * Statistics are missing in metastore > * Actual stats e.g. number of rows are zero > It'll be good for this method to return e.g. -1 in absence of statistics > instead of assuming it to be zero. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17536) StatsUtil::getBasicStatForTable doesn't distinguish b/w absence of statistics or zero stats
[ https://issues.apache.org/jira/browse/HIVE-17536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17536: --- Status: Open (was: Patch Available) > StatsUtil::getBasicStatForTable doesn't distinguish b/w absence of statistics > or zero stats > --- > > Key: HIVE-17536 > URL: https://issues.apache.org/jira/browse/HIVE-17536 > Project: Hive > Issue Type: Improvement > Components: Statistics >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-17536.1.patch, HIVE-17536.2.patch > > > This method returns zero for both of the following cases: > * Statistics are missing in metastore > * Actual stats e.g. number of rows are zero > It'll be good for this method to return e.g. -1 in absence of statistics > instead of assuming it to be zero. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-15899) check CTAS over acid table
[ https://issues.apache.org/jira/browse/HIVE-15899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-15899: -- Attachment: HIVE-15899.09.patch > check CTAS over acid table > --- > > Key: HIVE-15899 > URL: https://issues.apache.org/jira/browse/HIVE-15899 > Project: Hive > Issue Type: Sub-task >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-15899.01.patch, HIVE-15899.02.patch, > HIVE-15899.03.patch, HIVE-15899.04.patch, HIVE-15899.05.patch, > HIVE-15899.07.patch, HIVE-15899.08.patch, HIVE-15899.09.patch > > > need to add a test to check if create table as works correctly with acid > tables -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17013) Delete request with a subquery based on select over a view
[
https://issues.apache.org/jira/browse/HIVE-17013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-17013:
--
Attachment: HIVE-17013.01.patch
> Delete request with a subquery based on select over a view
> --
>
> Key: HIVE-17013
> URL: https://issues.apache.org/jira/browse/HIVE-17013
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Frédéric ESCANDELL
>Assignee: Eugene Koifman
>Priority: Blocker
> Attachments: acid_view_bug.q, HIVE-17013.01.patch, HIVE-17013.01.patch
>
>
> Hi,
> I based my DDL on this exemple
> https://fr.hortonworks.com/tutorial/using-hive-acid-transactions-to-insert-update-and-delete-data/.
> In a delete request, the use of a view in a subquery throw an exception :
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
> {code:sql}
> drop table if exists mydim;
> create table mydim (key int, name string, zip string, is_current boolean)
> clustered by(key) into 3 buckets
> stored as orc tblproperties ('transactional'='true');
> insert into mydim values
> (1, 'bob', '95136', true),
> (2, 'joe', '70068', true),
> (3, 'steve', '22150', true);
> drop table if exists updates_staging_table;
> create table updates_staging_table (key int, newzip string);
> insert into updates_staging_table values (1, 87102), (3, 45220);
> drop view if exists updates_staging_view;
> create view updates_staging_view (key, newzip) as select key, newzip from
> updates_staging_table;
> delete from mydim
> where mydim.key in (select key from updates_staging_view);
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17013) Delete request with a subquery based on select over a view
[
https://issues.apache.org/jira/browse/HIVE-17013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-17013:
--
Status: Patch Available (was: Open)
> Delete request with a subquery based on select over a view
> --
>
> Key: HIVE-17013
> URL: https://issues.apache.org/jira/browse/HIVE-17013
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Frédéric ESCANDELL
>Assignee: Eugene Koifman
>Priority: Blocker
> Attachments: acid_view_bug.q, HIVE-17013.01.patch
>
>
> Hi,
> I based my DDL on this exemple
> https://fr.hortonworks.com/tutorial/using-hive-acid-transactions-to-insert-update-and-delete-data/.
> In a delete request, the use of a view in a subquery throw an exception :
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
> {code:sql}
> drop table if exists mydim;
> create table mydim (key int, name string, zip string, is_current boolean)
> clustered by(key) into 3 buckets
> stored as orc tblproperties ('transactional'='true');
> insert into mydim values
> (1, 'bob', '95136', true),
> (2, 'joe', '70068', true),
> (3, 'steve', '22150', true);
> drop table if exists updates_staging_table;
> create table updates_staging_table (key int, newzip string);
> insert into updates_staging_table values (1, 87102), (3, 45220);
> drop view if exists updates_staging_view;
> create view updates_staging_view (key, newzip) as select key, newzip from
> updates_staging_table;
> delete from mydim
> where mydim.key in (select key from updates_staging_view);
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17013) Delete request with a subquery based on select over a view
[
https://issues.apache.org/jira/browse/HIVE-17013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-17013:
--
Attachment: HIVE-17013.01.patch
> Delete request with a subquery based on select over a view
> --
>
> Key: HIVE-17013
> URL: https://issues.apache.org/jira/browse/HIVE-17013
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Frédéric ESCANDELL
>Assignee: Eugene Koifman
>Priority: Blocker
> Attachments: acid_view_bug.q, HIVE-17013.01.patch
>
>
> Hi,
> I based my DDL on this exemple
> https://fr.hortonworks.com/tutorial/using-hive-acid-transactions-to-insert-update-and-delete-data/.
> In a delete request, the use of a view in a subquery throw an exception :
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
> {code:sql}
> drop table if exists mydim;
> create table mydim (key int, name string, zip string, is_current boolean)
> clustered by(key) into 3 buckets
> stored as orc tblproperties ('transactional'='true');
> insert into mydim values
> (1, 'bob', '95136', true),
> (2, 'joe', '70068', true),
> (3, 'steve', '22150', true);
> drop table if exists updates_staging_table;
> create table updates_staging_table (key int, newzip string);
> insert into updates_staging_table values (1, 87102), (3, 45220);
> drop view if exists updates_staging_view;
> create view updates_staging_view (key, newzip) as select key, newzip from
> updates_staging_table;
> delete from mydim
> where mydim.key in (select key from updates_staging_view);
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HIVE-15899) check CTAS over acid table
[ https://issues.apache.org/jira/browse/HIVE-15899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169019#comment-16169019 ] Eugene Koifman edited comment on HIVE-15899 at 9/16/17 6:49 PM: FAILED: IllegalArgumentException Unexpected Work object: class org.apache.hadoop.hive.ql.plan.StatsNoJobWork FAILED: IllegalArgumentException Unexpected Work object: class org.apache.hadoop.hive.ql.plan.DDLWork was (Author: ekoifman): FAILED: IllegalArgumentException Unexpected Work object: class org.apache.hadoop.hive.ql.plan.StatsNoJobWork > check CTAS over acid table > --- > > Key: HIVE-15899 > URL: https://issues.apache.org/jira/browse/HIVE-15899 > Project: Hive > Issue Type: Sub-task >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-15899.01.patch, HIVE-15899.02.patch, > HIVE-15899.03.patch, HIVE-15899.04.patch, HIVE-15899.05.patch, > HIVE-15899.07.patch, HIVE-15899.08.patch > > > need to add a test to check if create table as works correctly with acid > tables -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-15899) check CTAS over acid table
[ https://issues.apache.org/jira/browse/HIVE-15899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169019#comment-16169019 ] Eugene Koifman commented on HIVE-15899: --- FAILED: IllegalArgumentException Unexpected Work object: class org.apache.hadoop.hive.ql.plan.StatsNoJobWork > check CTAS over acid table > --- > > Key: HIVE-15899 > URL: https://issues.apache.org/jira/browse/HIVE-15899 > Project: Hive > Issue Type: Sub-task >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-15899.01.patch, HIVE-15899.02.patch, > HIVE-15899.03.patch, HIVE-15899.04.patch, HIVE-15899.05.patch, > HIVE-15899.07.patch, HIVE-15899.08.patch > > > need to add a test to check if create table as works correctly with acid > tables -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HIVE-17013) Delete request with a subquery based on select over a view
[
https://issues.apache.org/jira/browse/HIVE-17013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman reassigned HIVE-17013:
-
Assignee: Eugene Koifman (was: Matt McCline)
> Delete request with a subquery based on select over a view
> --
>
> Key: HIVE-17013
> URL: https://issues.apache.org/jira/browse/HIVE-17013
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Frédéric ESCANDELL
>Assignee: Eugene Koifman
>Priority: Blocker
> Attachments: acid_view_bug.q
>
>
> Hi,
> I based my DDL on this exemple
> https://fr.hortonworks.com/tutorial/using-hive-acid-transactions-to-insert-update-and-delete-data/.
> In a delete request, the use of a view in a subquery throw an exception :
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
> {code:sql}
> drop table if exists mydim;
> create table mydim (key int, name string, zip string, is_current boolean)
> clustered by(key) into 3 buckets
> stored as orc tblproperties ('transactional'='true');
> insert into mydim values
> (1, 'bob', '95136', true),
> (2, 'joe', '70068', true),
> (3, 'steve', '22150', true);
> drop table if exists updates_staging_table;
> create table updates_staging_table (key int, newzip string);
> insert into updates_staging_table values (1, 87102), (3, 45220);
> drop view if exists updates_staging_view;
> create view updates_staging_view (key, newzip) as select key, newzip from
> updates_staging_table;
> delete from mydim
> where mydim.key in (select key from updates_staging_view);
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-15899) check CTAS over acid table
[
https://issues.apache.org/jira/browse/HIVE-15899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168992#comment-16168992
]
Hive QA commented on HIVE-15899:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12887493/HIVE-15899.08.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 203 failed/errored test(s), 6763 tests
executed
*Failed tests:*
{noformat}
TestAccumuloCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDFSErrorHandling - did not produce a TEST-*.xml file (likely timed out)
(batchId=227)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=230)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=10)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=11)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=19)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=29)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=31)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=33)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=40)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=42)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=43)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=45)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=46)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=48)
[jira] [Updated] (HIVE-15899) check CTAS over acid table
[ https://issues.apache.org/jira/browse/HIVE-15899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-15899: -- Attachment: HIVE-15899.08.patch > check CTAS over acid table > --- > > Key: HIVE-15899 > URL: https://issues.apache.org/jira/browse/HIVE-15899 > Project: Hive > Issue Type: Sub-task >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-15899.01.patch, HIVE-15899.02.patch, > HIVE-15899.03.patch, HIVE-15899.04.patch, HIVE-15899.05.patch, > HIVE-15899.07.patch, HIVE-15899.08.patch > > > need to add a test to check if create table as works correctly with acid > tables -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-15899) check CTAS over acid table
[ https://issues.apache.org/jira/browse/HIVE-15899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-15899: -- Status: Patch Available (was: Open) > check CTAS over acid table > --- > > Key: HIVE-15899 > URL: https://issues.apache.org/jira/browse/HIVE-15899 > Project: Hive > Issue Type: Sub-task >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-15899.01.patch, HIVE-15899.02.patch, > HIVE-15899.03.patch, HIVE-15899.04.patch, HIVE-15899.05.patch, > HIVE-15899.07.patch, HIVE-15899.08.patch > > > need to add a test to check if create table as works correctly with acid > tables -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-15899) check CTAS over acid table
[ https://issues.apache.org/jira/browse/HIVE-15899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-15899: -- Status: Open (was: Patch Available) > check CTAS over acid table > --- > > Key: HIVE-15899 > URL: https://issues.apache.org/jira/browse/HIVE-15899 > Project: Hive > Issue Type: Sub-task >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-15899.01.patch, HIVE-15899.02.patch, > HIVE-15899.03.patch, HIVE-15899.04.patch, HIVE-15899.05.patch, > HIVE-15899.07.patch, HIVE-15899.08.patch > > > need to add a test to check if create table as works correctly with acid > tables -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17013) Delete request with a subquery based on select over a view
[
https://issues.apache.org/jira/browse/HIVE-17013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168841#comment-16168841
]
Matt McCline commented on HIVE-17013:
-
Thank you [~fescandell] for a great repro.
I easily made a Q file from it (acid_view_bug.q). FYI [~ekoifman]
> Delete request with a subquery based on select over a view
> --
>
> Key: HIVE-17013
> URL: https://issues.apache.org/jira/browse/HIVE-17013
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Frédéric ESCANDELL
>Assignee: Matt McCline
>Priority: Blocker
> Attachments: acid_view_bug.q
>
>
> Hi,
> I based my DDL on this exemple
> https://fr.hortonworks.com/tutorial/using-hive-acid-transactions-to-insert-update-and-delete-data/.
> In a delete request, the use of a view in a subquery throw an exception :
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
> {code:sql}
> drop table if exists mydim;
> create table mydim (key int, name string, zip string, is_current boolean)
> clustered by(key) into 3 buckets
> stored as orc tblproperties ('transactional'='true');
> insert into mydim values
> (1, 'bob', '95136', true),
> (2, 'joe', '70068', true),
> (3, 'steve', '22150', true);
> drop table if exists updates_staging_table;
> create table updates_staging_table (key int, newzip string);
> insert into updates_staging_table values (1, 87102), (3, 45220);
> drop view if exists updates_staging_view;
> create view updates_staging_view (key, newzip) as select key, newzip from
> updates_staging_table;
> delete from mydim
> where mydim.key in (select key from updates_staging_view);
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17013) Delete request with a subquery based on select over a view
[
https://issues.apache.org/jira/browse/HIVE-17013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-17013:
Attachment: acid_view_bug.q
> Delete request with a subquery based on select over a view
> --
>
> Key: HIVE-17013
> URL: https://issues.apache.org/jira/browse/HIVE-17013
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Frédéric ESCANDELL
>Assignee: Matt McCline
>Priority: Blocker
> Attachments: acid_view_bug.q
>
>
> Hi,
> I based my DDL on this exemple
> https://fr.hortonworks.com/tutorial/using-hive-acid-transactions-to-insert-update-and-delete-data/.
> In a delete request, the use of a view in a subquery throw an exception :
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
> {code:sql}
> drop table if exists mydim;
> create table mydim (key int, name string, zip string, is_current boolean)
> clustered by(key) into 3 buckets
> stored as orc tblproperties ('transactional'='true');
> insert into mydim values
> (1, 'bob', '95136', true),
> (2, 'joe', '70068', true),
> (3, 'steve', '22150', true);
> drop table if exists updates_staging_table;
> create table updates_staging_table (key int, newzip string);
> insert into updates_staging_table values (1, 87102), (3, 45220);
> drop view if exists updates_staging_view;
> create view updates_staging_view (key, newzip) as select key, newzip from
> updates_staging_table;
> delete from mydim
> where mydim.key in (select key from updates_staging_view);
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HIVE-17013) Delete request with a subquery based on select over a view
[
https://issues.apache.org/jira/browse/HIVE-17013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline reassigned HIVE-17013:
---
Assignee: Matt McCline
> Delete request with a subquery based on select over a view
> --
>
> Key: HIVE-17013
> URL: https://issues.apache.org/jira/browse/HIVE-17013
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Frédéric ESCANDELL
>Assignee: Matt McCline
>Priority: Blocker
>
> Hi,
> I based my DDL on this exemple
> https://fr.hortonworks.com/tutorial/using-hive-acid-transactions-to-insert-update-and-delete-data/.
> In a delete request, the use of a view in a subquery throw an exception :
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
> {code:sql}
> drop table if exists mydim;
> create table mydim (key int, name string, zip string, is_current boolean)
> clustered by(key) into 3 buckets
> stored as orc tblproperties ('transactional'='true');
> insert into mydim values
> (1, 'bob', '95136', true),
> (2, 'joe', '70068', true),
> (3, 'steve', '22150', true);
> drop table if exists updates_staging_table;
> create table updates_staging_table (key int, newzip string);
> insert into updates_staging_table values (1, 87102), (3, 45220);
> drop view if exists updates_staging_view;
> create view updates_staging_view (key, newzip) as select key, newzip from
> updates_staging_table;
> delete from mydim
> where mydim.key in (select key from updates_staging_view);
> FAILED: IllegalStateException Expected 'insert into table default.mydim
> select ROW__ID from default.mydim sort by ROW__ID' to be in sub-query or set
> operation.
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17537) Move Warehouse class to standalone metastore
[ https://issues.apache.org/jira/browse/HIVE-17537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168831#comment-16168831 ] Zoltan Haindrich commented on HIVE-17537: - I've left some comments on the github pr... +1 > Move Warehouse class to standalone metastore > > > Key: HIVE-17537 > URL: https://issues.apache.org/jira/browse/HIVE-17537 > Project: Hive > Issue Type: Sub-task > Components: Metastore >Reporter: Alan Gates >Assignee: Alan Gates > Labels: pull-request-available > Attachments: HIVE-17537.patch > > > Move the Warehouse class. This is done in its own JIRA as it was somewhat > more involved than some of the other classes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
