[
https://issues.apache.org/jira/browse/SPARK-26407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bao Yunz updated SPARK-26407:
-----------------------------
Summary: For an external non-partitioned table, if add a directory named
with k=v to the table path, select result will be wrong (was: For an external
non-partition table, if add a directory named with k=v to the table path,
select result will be wrong)
> For an external non-partitioned table, if add a directory named with k=v to
> the table path, select result will be wrong
> -----------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-26407
> URL: https://issues.apache.org/jira/browse/SPARK-26407
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.3.2, 2.4.0
> Reporter: Bao Yunz
> Priority: Major
> Labels: usability
>
> Scene 1
> Create a external non-partition table, in which location directory has a
> directory named with "part=1", for example. Then desc the table, we will find
> the string "part" is showed in table column. when insert the table with data
> which has same column with target table , will throw a exception that target
> table has different column number with the inserted data.
> Scene 2
> Create a external non-partition table, which location path is empty. After
> several times insert operation, we add a directory named with "part=1" in the
> table location directory. Then do insert and select operation, we will find
> the scan path is changed to "tablePath/part=1"，so that we will get a wrong
> result.
>
> It seems that the existing logic of spark will process this kind of table
> like a partition table. But when we do show partitions operation, it will
> throw the exception that the table is not partitioned, which is confusing。We
> believe that the normal logic should be that if a table is a non-partitioned
> table, the folder under tablePath should not change its basic properties.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
