[
https://issues.apache.org/jira/browse/DRILL-5083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173529#comment-16173529
]
Roman Kulyk commented on DRILL-5083:
------------------------------------
[~khfaraaz],
Sorry for the late answer I missed your message.
To reproduce this issue I used the next env:
1 node cluster (16G RAM, 2 cores)
I used tpcds_sf100 2 query:
{code:sql}
WITH wscs AS (SELECT sold_date_sk, sales_price FROM (SELECT ws_sold_date_sk
sold_date_sk, ws_ext_sales_price sales_price FROM web_sales) UNION ALL (SELECT
cs_sold_date_sk sold_date_sk, cs_ext_sales_price sales_price FROM
catalog_sales)), wswscs AS (SELECT d_week_seq, Sum(CASE WHEN ( d_day_name =
'Sunday' ) THEN sales_price ELSE NULL END) sun_sales, Sum(CASE WHEN (
d_day_name = 'Monday' ) THEN sales_price ELSE NULL END) mon_sales, Sum(CASE
WHEN ( d_day_name = 'Tuesday' ) THEN sales_price ELSE NULL END) tue_sales,
Sum(CASE WHEN ( d_day_name = 'Wednesday' ) THEN sales_price ELSE NULL END)
wed_sales, Sum(CASE WHEN ( d_day_name = 'Thursday' ) THEN sales_price ELSE NULL
END) thu_sales, Sum(CASE WHEN ( d_day_name = 'Friday' ) THEN sales_price ELSE
NULL END) fri_sales, Sum(CASE WHEN ( d_day_name = 'Saturday' ) THEN sales_price
ELSE NULL END) sat_sales FROM wscs, date_dim WHERE d_date_sk = sold_date_sk
GROUP BY d_week_seq) SELECT d_week_seq1, Round(sun_sales1 / sun_sales2, 2),
Round(mon_sales1 / mon_sales2, 2), Round(tue_sales1 / tue_sales2, 2),
Round(wed_sales1 / wed_sales2, 2), Round(thu_sales1 / thu_sales2, 2),
Round(fri_sales1 / fri_sales2, 2), Round(sat_sales1 / sat_sales2, 2) FROM
(SELECT wswscs.d_week_seq d_week_seq1, sun_sales sun_sales1, mon_sales
mon_sales1, tue_sales tue_sales1, wed_sales wed_sales1, thu_sales thu_sales1,
fri_sales fri_sales1, sat_sales sat_sales1 FROM wswscs, date_dim WHERE
date_dim.d_week_seq = wswscs.d_week_seq AND d_year = 1998) y, (SELECT
wswscs.d_week_seq d_week_seq2, sun_sales sun_sales2, mon_sales mon_sales2,
tue_sales tue_sales2, wed_sales wed_sales2, thu_sales thu_sales2, fri_sales
fri_sales2, sat_sales sat_sales2 FROM wswscs, date_dim WHERE
date_dim.d_week_seq = wswscs.d_week_seq AND d_year = 1998 + 1) z WHERE
d_week_seq1 = d_week_seq2 - 53 ORDER BY d_week_seq1 limit 1;
{code}
on views which were created from the next dataset (all files were in parquet
format):
{code:xml}
hadoop fs -du -h /drill/testdata/
1.4 G /drill/testdata/catalog_sales
22.4 M /drill/testdata/date_dim
734.0 M /drill/testdata/web_sales
{code}
To get a reproduce you should change the next properties before you run the
query:
{code:sql}
alter session set `planner.enable_hashjoin` = false;
alter session set `planner.enable_hashagg` = false;
alter session set `planner.enable_mergejoin` = true;
alter session set `planner.memory.max_query_memory_per_node` = 1048576;
{code}
After that, you should run the query (tpcds_sf100 2) and interrupt it with
"Ctrl + c" at the right moment. There are 2 ways to find a moment to get the
reproduce:
1) In my case, if I will not interrupt the query, it will fail after 1 min 40
sec. To get hang in CANCELLATION_REQUESTED state I need to make "Ctrl + c"
after ~1 min 30 sec. So I think you can scale time for the situation on your
cluster.
2) You can open 2nd terminal window with "tail -f
/opt/mapr/drill/log/drillbit.log" and you should interrupt the query before the
first fragment will change state to FINISHED.
To make this you should run the query for the first time. The query should fail
and you will be able to see a number of operators which were running before
some fragment finishes (as an example you can see a required number of
operators in the red circles in Reproduce5083.jpg attachment). When you will
run the query in the second time, just wait for a required number in the 2nd
terminal and interrupt query with "Ctrl + c".
You can reproduce this issue in Drill before 368bc38b1 and you can verify it on
the latest apache master.
> RecordIterator can sometimes restart a query on close
> -----------------------------------------------------
>
> Key: DRILL-5083
> URL: https://issues.apache.org/jira/browse/DRILL-5083
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.8.0
> Reporter: Paul Rogers
> Assignee: Roman Kulyk
> Priority: Minor
> Labels: ready-to-commit
> Fix For: 1.11.0
>
> Attachments: DrillOperatorErrorHandlingRedesign.pdf, Reproduce5083.jpg
>
>
> This one is very confusing...
> In a test with a MergeJoin and external sort, operators are stacked something
> like this:
> {code}
> Screen
> - MergeJoin
> - - External Sort
> ...
> {code}
> Using the injector to force a OOM in spill, the external sort threw a
> UserException up the stack. This was handed by:
> {code}
> IteratorValidatorBatchIterator.next( )
> RecordIterator.clearInflightBatches( )
> RecordIterator.close( )
> MergeJoinBatch.close( )
> {code}
> Which does the following:
> {code}
> // Check whether next() should even have been called in current state.
> if (null != exceptionState) {
> throw new IllegalStateException(
> {code}
> But, the exceptionState is set, so we end up throwing an
> IllegalStateException during cleanup.
> Seems the code should agree: if {{next( )}} will be called during cleanup,
> then {{next( )}} should gracefully handle that case.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
