zzzzming95 created SPARK-46211:
----------------------------------
Summary: Directly writing the parquet table location will cause
the query to be null
Key: SPARK-46211
URL: https://issues.apache.org/jira/browse/SPARK-46211
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.5.0
Reporter: zzzzming95
reproduce code
{code:java}
spark.sql("create table data_table (column string) using parquet")
val identifier = TableIdentifier("data_table")
val location = spark.sessionState.catalog.getTableMetadata(identifier).location
val df = spark.range(2000).map(f => {
BigInt(100, scala.util.Random).toString(36)
}).toDF()
df.repartition(1).write.mode("overwrite").parquet(location.getPath)
spark.sql("select * from data_table").show() {code}
it will get the null result
{code:java}
+------+
|column|
+------+
| NULL|
| NULL|
| NULL|
| NULL|
| NULL|
| NULL|
| NULL|
| NULL|
| NULL|
| NULL|
| NULL|
| NULL|
| NULL|
| NULL|
| NULL|
| NULL|
| NULL|
| NULL|
| NULL|
| NULL|
+------+
only showing top 20 rows {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
