[
https://issues.apache.org/jira/browse/IMPALA-11469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17574628#comment-17574628
]
Quanlong Huang commented on IMPALA-11469:
-----------------------------------------
I'm recently fixing a bug in this area (IMPALA-11464) so I take this BTW.
Uploaded a patch for review: [https://gerrit.cloudera.org/c/18811/]
CC [~kdeschle]
> Ignore _spark_metadata folder in table location
> -----------------------------------------------
>
> Key: IMPALA-11469
> URL: https://issues.apache.org/jira/browse/IMPALA-11469
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Matthias Wies
> Assignee: Quanlong Huang
> Priority: Major
>
> When spark streaming is used to write parquet files out to an external table
> a folder _spark_metadata is created within the directory of the table. Hive
> is capable of dealing with this directory, but Impala trips on it.
> So REFRESH TABLE won't work as it sees a directory with data Impala cannot
> cope with. A SELECT will also not work as it trips on the _spark_metadata __
> folder _._
> Issue was found in CDP 7.1.7 SP1 but I suspect it is in all versions
> Regards Matthias
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
