[
https://issues.apache.org/jira/browse/HIVE-11846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14791046#comment-14791046
]
Pengcheng Xiong commented on HIVE-11846:
----------------------------------------
[~xuefuz]. thanks for your attention. The problem can be better understood if
you can take a look at my patch. So, when CliDriver tries to shutdown, it tries
to drop all the tables that are created during q test. In this case, it
iterates through all the tables in db.getAllTables() in QTestUtil.java and try
to drop every one of them. Let's assume there are two tables, A, an original
table, and index_A, which is an index table created based on A. If index_A
comes before A in the iteration, there is no problem, because L674 in
QTestUtil.java will skip it and later when A is dropped, index_A is dropped as
well. However, If A comes before index_A in the iteration, A will be dropped
and index_A is dropped as well, later, it will not find index_A and throw
InvalidTableException. That is the symptom of the problem and why it is related
to CliDriver shutdown. Thanks.
> CliDriver shutdown tries to drop index table again which was already dropped
> when dropping the original table
> --------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-11846
> URL: https://issues.apache.org/jira/browse/HIVE-11846
> Project: Hive
> Issue Type: Sub-task
> Reporter: Pengcheng Xiong
> Assignee: Pengcheng Xiong
> Priority: Critical
> Attachments: HIVE-11846.01.patch
>
>
> Steps to repro:
> {code}
> set hive.stats.dbclass=fs;
> set hive.stats.autogather=true;
> set hive.cbo.enable=true;
> DROP TABLE IF EXISTS aa;
> CREATE TABLE aa (L_ORDERKEY INT,
> L_PARTKEY INT,
> L_SUPPKEY INT,
> L_LINENUMBER INT,
> L_QUANTITY DOUBLE,
> L_EXTENDEDPRICE DOUBLE,
> L_DISCOUNT DOUBLE,
> L_TAX DOUBLE,
> L_RETURNFLAG STRING,
> L_LINESTATUS STRING,
> l_shipdate STRING,
> L_COMMITDATE STRING,
> L_RECEIPTDATE STRING,
> L_SHIPINSTRUCT STRING,
> L_SHIPMODE STRING,
> L_COMMENT STRING)
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY '|';
> LOAD DATA LOCAL INPATH '../../data/files/lineitem.txt' OVERWRITE INTO TABLE
> aa;
> CREATE INDEX aa_lshipdate_idx ON TABLE aa(l_shipdate) AS
> 'org.apache.hadoop.hive.ql.index.AggregateIndexHandler' WITH DEFERRED REBUILD
> IDXPROPERTIES("AGGREGATES"="count(l_shipdate)");
> ALTER INDEX aa_lshipdate_idx ON aa REBUILD;
> show tables;
> explain select l_shipdate, count(l_shipdate)
> from aa
> group by l_shipdate;
> {code}
> The problem is that, we create an index table default_aa_lshipdate_idx,
> (default is the database name) and it comes after the table aa. Then, it
> first drop aa, which will drop default_aa_lshipdate_idx as well as it is
> related to aa. It will not find the table default_aa_lshipdate_idx when it
> tries to drop it again, which will throw an exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
