wzx140 commented on code in PR #3476:
URL: https://github.com/apache/fluss/pull/3476#discussion_r3402183516
##########
fluss-spark/fluss-spark-common/src/main/scala/org/apache/fluss/spark/SparkTable.scala:
##########
@@ -68,6 +68,10 @@ class SparkTable(
flussConfig.get(SparkFlussConf.SCAN_START_UP_MODE))
.toUpperCase
val isFullMode = startupMode == SparkFlussConf.StartUpMode.FULL.toString
+ if (isDataLakeEnabled &&
LakeTableUtil.hasCustomLakePath(tableInfo.getProperties)) {
+ throw new UnsupportedOperationException(
+ "Custom lake table path is not supported in Spark connector yet.")
+ }
Review Comment:
Good point. I narrowed this check to Spark lake-read paths only.
For log/append tables, Spark now rejects custom lake table path only when
`datalake.enabled=true`, scan mode is `FULL`, and custom mapping is configured.
For primary-key tables, it still rejects when custom mapping is configured
because the current Spark scan path uses lake upsert scan.
I also updated the error message to: `Custom lake table path is not
supported for Spark lake reads yet.`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
