[
https://issues.apache.org/jira/browse/HUDI-8577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17903485#comment-17903485
]
Y Ethan Guo commented on HUDI-8577:
-----------------------------------
Related to `HUDI-8576`. Could you check if removing the following for the old
logic of clustering works in MultipleSparkJobExecutionStrategy (make sure you
test this particular code path because the latest is using file group
reader-based clustering)?
{code:java}
if (hasLogFiles) {
params.put("hoodie.datasource.query.type", "snapshot");
} else {
params.put("hoodie.datasource.query.type", "read_optimized");
} {code}
> Use snapshot query as default in DefaultSource
> ----------------------------------------------
>
> Key: HUDI-8577
> URL: https://issues.apache.org/jira/browse/HUDI-8577
> Project: Apache Hudi
> Issue Type: Improvement
> Reporter: Y Ethan Guo
> Assignee: Jonathan Vexler
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.0.1
>
>
> This does not take the default query type from the config definition.
>
> {code:java}
> def createRelation(sqlContext: SQLContext,
> metaClient: HoodieTableMetaClient,
> schema: StructType,
> globPaths: Seq[StoragePath],
> parameters: Map[String, String]): BaseRelation = {
> val tableType = metaClient.getTableType
> val isBootstrappedTable =
> metaClient.getTableConfig.getBootstrapBasePath.isPresent
> val queryType = parameters(QUERY_TYPE.key) {code}
> {code:java}
> val QUERY_TYPE: ConfigProperty[String] = ConfigProperty
> .key("hoodie.datasource.query.type")
> .defaultValue(QUERY_TYPE_SNAPSHOT_OPT_VAL)
> .withAlternatives("hoodie.datasource.view.type")
> .withValidValues(QUERY_TYPE_SNAPSHOT_OPT_VAL,
> QUERY_TYPE_READ_OPTIMIZED_OPT_VAL, QUERY_TYPE_INCREMENTAL_OPT_VAL)
> .withDocumentation("Whether data needs to be read, in `" +
> QUERY_TYPE_INCREMENTAL_OPT_VAL + "` mode (new data since an instantTime) " +
> "(or) `" + QUERY_TYPE_READ_OPTIMIZED_OPT_VAL + "` mode (obtain latest
> view, based on base files) (or) `" + QUERY_TYPE_SNAPSHOT_OPT_VAL + "` mode " +
> "(obtain latest view, by merging base and (if any) log files)") {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
