[
https://issues.apache.org/jira/browse/IMPALA-9992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17189681#comment-17189681
]
Joe McDonnell commented on IMPALA-9992:
---------------------------------------
We output a list of the files at the start of the tests after dataload. For a
run not impacted by this, logs/file-list-begin-1.log has these entries for
complextypestbl_medium (in the ORC format):
{noformat}
drwxr-xr-x - jenkins supergroup 0 2020-09-02 05:11
/test-warehouse/managed/complextypestbl_medium_orc_def
drwxr-xr-x - jenkins supergroup 0 2020-09-02 05:11
/test-warehouse/managed/complextypestbl_medium_orc_def/base_0000001
-rw-r--r-- 3 jenkins supergroup 1 2020-09-02 05:11
/test-warehouse/managed/complextypestbl_medium_orc_def/base_0000001/_orc_acid_version
-rw-r--r-- 3 jenkins supergroup 6513 2020-09-02 05:11
/test-warehouse/managed/complextypestbl_medium_orc_def/base_0000001/bucket_00000_1
-rw-r--r-- 3 jenkins supergroup 6600 2020-09-02 05:11
/test-warehouse/managed/complextypestbl_medium_orc_def/base_0000001/bucket_00001_0
-rw-r--r-- 3 jenkins supergroup 6671 2020-09-02 05:11
/test-warehouse/managed/complextypestbl_medium_orc_def/base_0000001/bucket_00002_1{noformat}
For a run impacted by this, it has this list in logs/file-list-begin-1.log:
{noformat}
drwxr-xr-x - jenkins supergroup 0 2020-09-01 03:05
/test-warehouse/managed/complextypestbl_medium_orc_def
drwxr-xr-x - jenkins supergroup 0 2020-09-01 03:05
/test-warehouse/managed/complextypestbl_medium_orc_def/base_0000001
-rw-r--r-- 3 jenkins supergroup 1 2020-09-01 03:05
/test-warehouse/managed/complextypestbl_medium_orc_def/base_0000001/_orc_acid_version
-rw-r--r-- 3 jenkins supergroup 6513 2020-09-01 03:05
/test-warehouse/managed/complextypestbl_medium_orc_def/base_0000001/bucket_00000_1
-rw-r--r-- 3 jenkins supergroup 6600 2020-09-01 03:05
/test-warehouse/managed/complextypestbl_medium_orc_def/base_0000001/bucket_00001_0
-rw-r--r-- 3 jenkins supergroup 6671 2020-09-01 03:05
/test-warehouse/managed/complextypestbl_medium_orc_def/base_0000001/bucket_00002_0
-rw-r--r-- 3 jenkins supergroup 6671 2020-09-01 03:05
/test-warehouse/managed/complextypestbl_medium_orc_def/base_0000001/bucket_00002_1{noformat}
It looks like there is an extra file (bucket_00002_0 and bucket_00002_1 have
the same size). This table is written by Hive during dataload.
>From the symptoms that I know about, this seems to only happen on ORC (but, of
>course, the file list would have the other formats if we have ever seen it
>elsewhere).
> test_scanner_position seems flaky
> ---------------------------------
>
> Key: IMPALA-9992
> URL: https://issues.apache.org/jira/browse/IMPALA-9992
> Project: IMPALA
> Issue Type: Bug
> Components: Backend, Frontend
> Reporter: Fang-Yu Rao
> Assignee: Bikramjeet Vig
> Priority: Critical
> Labels: broken-build, flaky
>
> [test_scanner_position|https://github.com/apache/impala/blob/master/tests/query_test/test_nested_types.py#L72-L76]
> failed in a recent build when executing the following query at
> [https://github.com/apache/impala/blob/master/testdata/workloads/functional-query/queries/QueryTest/nested-types-scanner-position.test#L646-L666]
> {code:java}
> select pos, item, count(*)
> from complextypestbl_medium.int_array
> group by 1, 2
> {code}
> The error message is as follows.
> {code:java}
> ERROR:test_configuration:Comparing QueryTestResults (expected vs actual):
> 0,-1,7300 != 0,-1,9856
> 0,1,7300 != 0,1,9524
> 0,NULL,7300 != 0,NULL,9700
> 1,1,7300 != 1,1,9700
> 1,2,7300 != 1,2,9524
> 2,2,7300 != 2,2,9700
> 2,3,7300 != 2,3,9524
> 3,NULL,7300 != 3,NULL,9700
> 4,3,7300 != 4,3,9700
> 5,NULL,7300 != 5,NULL,9700
> {code}
> Maybe [~tarmstrong], [~bikram], and [~csringhofer] could offer some insight
> into the issue since you were working on/reviewing the corresponding patch.
> Assign the JIRA to [~tarmstrong] for now but please feel free to assign to
> other as you find appropriate. Thanks!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
