[
https://issues.apache.org/jira/browse/IMPALA-13778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17929404#comment-17929404
]
ASF subversion and git services commented on IMPALA-13778:
----------------------------------------------------------
Commit a9db5f1d72d637b499b670bb5442973f8ae3b116 in impala's branch
refs/heads/master from Csaba Ringhofer
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=a9db5f1d7 ]
IMPALA-13778: Update ignored_dir_prefix_list
There are some missing prefixes that are not yet ignored during
recursive listing:
- -tmp. (Hive changed to this new tmp dir from _tmp. in HIVE-27536)
- _impala_insert_staging
Impala may bump to these often so it is better to ignore them
by default.
Change-Id: I023244525dd333af1c5bfbad06708f3ec86aeacf
Reviewed-on: http://gerrit.cloudera.org:8080/22518
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Recursive file listing should ignore more directories
> -----------------------------------------------------
>
> Key: IMPALA-13778
> URL: https://issues.apache.org/jira/browse/IMPALA-13778
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Reporter: Csaba Ringhofer
> Assignee: Csaba Ringhofer
> Priority: Critical
>
> The paths to ignore is governed by ignored_dir_prefix_list (default=
> ".,_tmp.,_spark_metadata").
> Hive recently started writing tmp files to folders starting with _tmp instead
> of -tmp, leading to errors if Impala loads the table when these temporary
> files still exist.
> Another prefix missing in ignored_dir_prefix_list is Impala's own tmp dir,
> _impala_insert_staging. This can only cause issues in non-partitioned tables
> as Impala writes this dir to the root folder of the table.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
