nsivabalan commented on code in PR #8107:
URL: https://github.com/apache/hudi/pull/8107#discussion_r1156418648
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/keygen/ComplexAvroKeyGenerator.java:
##########
@@ -44,6 +48,9 @@ public ComplexAvroKeyGenerator(TypedProperties props) {
@Override
public String getRecordKey(GenericRecord record) {
+ if (autoGenerateRecordKeys()) {
+ return StringUtils.EMPTY_STRING;
+ }
Review Comment:
this is kind of unavoidable as of current structure. For eg, even to fetch
partition path, our KeyGenerator interface, only exposes
```
HoodieKey getKey(GenericRecord record)
```
So, to fetch partition path, we have to call getKey(genRec).getPartitionPath
and hence I had to return empty string here. we don't want to add a new api to
the interface just for this purpose.
Incase of auto key gen flows, we generate the record keys explicitly (not
via key gen class) and add it to HoodieKey that we materialize in memory for
all records.
I can sync up w/ you f2f to clarify this.
Ideally, we need to have 2 different interfaces. one to generate partition
path and one to generate record key. and so some of these workarounds may not
be required. but w/ current structure, we use a single key gen class to
generate both record keys and partition paths as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
