[
https://issues.apache.org/jira/browse/SPARK-19120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li updated SPARK-19120:
----------------------------
Description:
{noformat}
sql(
"""
|CREATE TABLE test_added_partitions (a STRING)
|PARTITIONED BY (b INT)
|STORED AS PARQUET
""".stripMargin)
// Create partition without data files and check whether it can be read
sql(s"ALTER TABLE test_added_partitions ADD PARTITION (b='1')")
spark.table("test_added_partitions").show()
sql(
s"""
|LOAD DATA LOCAL INPATH '$newPartitionDir' OVERWRITE
|INTO TABLE test_added_partitions PARTITION(b='1')
""".stripMargin)
spark.table("test_added_partitions").show()
{noformat}
The returned result is empty after table loading. We should do metadata refresh
after loading the data to the table.
This only happens if users manually add the partition by {{ALTER TABLE ADD
PARTITION}}
was:
{noformat}
sql(
"""
|CREATE TABLE test_added_partitions (a STRING)
|PARTITIONED BY (b INT)
|STORED AS PARQUET
""".stripMargin)
// Create partition without data files and check whether it can be read
sql(s"ALTER TABLE test_added_partitions ADD PARTITION (b='1')")
spark.table("test_added_partitions").show()
sql(
s"""
|LOAD DATA LOCAL INPATH '$newPartitionDir' OVERWRITE
|INTO TABLE test_added_partitions PARTITION(b='1')
""".stripMargin)
spark.table("test_added_partitions").show()
{noformat}
The returned result is empty after table loading. We should do metadata refresh
after loading the data to the table.
> Returned an Empty Result after Loading a Hive Partitioned Table
> ---------------------------------------------------------------
>
> Key: SPARK-19120
> URL: https://issues.apache.org/jira/browse/SPARK-19120
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.1.0
> Reporter: Xiao Li
> Assignee: Xiao Li
> Priority: Critical
>
> {noformat}
> sql(
> """
> |CREATE TABLE test_added_partitions (a STRING)
> |PARTITIONED BY (b INT)
> |STORED AS PARQUET
> """.stripMargin)
> // Create partition without data files and check whether it can be
> read
> sql(s"ALTER TABLE test_added_partitions ADD PARTITION (b='1')")
> spark.table("test_added_partitions").show()
> sql(
> s"""
> |LOAD DATA LOCAL INPATH '$newPartitionDir' OVERWRITE
> |INTO TABLE test_added_partitions PARTITION(b='1')
> """.stripMargin)
> spark.table("test_added_partitions").show()
> {noformat}
> The returned result is empty after table loading. We should do metadata
> refresh after loading the data to the table.
> This only happens if users manually add the partition by {{ALTER TABLE ADD
> PARTITION}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
