yihua commented on code in PR #13650:
URL: https://github.com/apache/hudi/pull/13650#discussion_r2357197737
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/SparkBaseIndexSupport.scala:
##########
@@ -174,13 +173,14 @@ abstract class SparkBaseIndexSupport(spark: SparkSession,
var recordKeyQueries: List[Expression] = List.empty
var compositeRecordKeys: List[String] = List.empty
val recordKeyOpt = getRecordKeyConfig
- val isComplexRecordKey = {
- val fieldCount = recordKeyOpt.map(recordKeys =>
recordKeys.length).getOrElse(0)
- // Consider as complex if:
- // 1. Multiple fields (> 1), OR
- // 2. Using complex key generator with single record key field
- fieldCount > 1 ||
isComplexKeyGeneratorWithSingleRecordKeyField(metaClient.getTableConfig)
+
+ if
(KeyGenUtils.mayUseNewEncodingForComplexKeyGen(metaClient.getTableConfig)) {
+ // Avoid using the index for this case
+ return (List.empty, List.empty)
}
+ // for other cases though we should be able to handle complex key
+ // a complex key can be a single field or multiple fields based on hudi
docs, regardless of encoding
+ val isComplexRecordKey = recordKeyOpt.map(recordKeys =>
recordKeys.length).getOrElse(0) >= 1
Review Comment:
This should be `>1` instead of `>=1`?
```suggestion
val isComplexRecordKey = recordKeyOpt.map(recordKeys =>
recordKeys.length).getOrElse(0) > 1
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
