nsivabalan commented on a change in pull request #5088:
URL: https://github.com/apache/hudi/pull/5088#discussion_r833747469
##########
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieMergeHandle.java
##########
@@ -294,13 +292,11 @@ protected boolean writeRecord(HoodieRecord<T>
hoodieRecord, Option<IndexedRecord
try {
if (indexedRecord.isPresent() && !isDelete) {
// Convert GenericRecord to GenericRecord with hoodie commit metadata
in schema
- IndexedRecord recordWithMetadataInSchema =
rewriteRecord((GenericRecord) indexedRecord.get(), preserveMetadata, oldRecord);
- if (preserveMetadata && useWriterSchema) { // useWriteSchema will be
true only incase of compaction.
- // do not preserve FILENAME_METADATA_FIELD
- recordWithMetadataInSchema.put(FILENAME_METADATA_FIELD_POS,
newFilePath.getName());
- fileWriter.writeAvro(hoodieRecord.getRecordKey(),
recordWithMetadataInSchema);
+ if (preserveMetadata && useWriterSchema) { // useWriteSchema will be
true only in case of compaction.
Review comment:
Specifically I am talking about this code snippet in HoodieMergeHandle.
combinedAvroRecord below does not contain any meta fields. So, if you remove
rewriting w/ meta columns, not sure how that would pan out
```
public void write(GenericRecord oldRecord) {
String key = KeyGenUtils.getRecordKeyFromGenericRecord(oldRecord,
keyGeneratorOpt);
boolean copyOldRecord = true;
if (keyToNewRecords.containsKey(key)) {
// If we have duplicate records that we are updating, then the hoodie
record will be deflated after
// writing the first record. So make a copy of the record to be merged
HoodieRecord<T> hoodieRecord = keyToNewRecords.get(key).newInstance();
try {
Option<IndexedRecord> combinedAvroRecord =
hoodieRecord.getData().combineAndGetUpdateValue(oldRecord,
useWriterSchema ? tableSchemaWithMetaFields : tableSchema,
config.getPayloadConfig().getProps());
.
.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
