gnailJC commented on code in PR #5616:
URL: https://github.com/apache/hudi/pull/5616#discussion_r875489218
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/HoodieSqlCommonUtils.scala:
##########
@@ -250,7 +250,7 @@ object HoodieSqlCommonUtils extends SparkAdapterSupport {
(baseConfig: Map[String, String] = Map.empty): Map[String,
String] = {
baseConfig ++ DFSPropertiesConfiguration.getGlobalProps.asScala ++ //
Table options has the highest priority
(spark.sessionState.conf.getAllConfs ++
HoodieOptionConfig.mappingSqlOptionToHoodieParam(options))
- .filterKeys(_.startsWith("hoodie."))
+ .filterKeys(DataSourceOptionsHelper.isHoodieConfigKey)
Review Comment:
> The logic has changed ?
Yes, if only use `startWith(''hoodie.'')` to determine whether it is the
hoodie config key, then "as.of.instant" will be misjudged and cannot achieve
the expected purpose.
for example:
create a hoodie_tbl table by sql:
`
CREATE external TABLE hoodie_tbl
using hudi
options (
hoodie.datasource.query.type='snapshot',
as.of.instant='20220512165111404'
) LOCATION 'xxx';
`
hoodie_tbl will always query the latest snapshot, not historical
snapshots(20220512165111404), because `as.of.instant `will be filtered and will
not take effect.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
