manojpec commented on a change in pull request #4067:
URL: https://github.com/apache/hudi/pull/4067#discussion_r768489330
##########
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieMergeHandle.java
##########
@@ -182,9 +185,14 @@ private void init(String fileId, String partitionPath,
HoodieBaseFile baseFileTo
// Create Marker file
createMarkerFile(partitionPath, newFileName);
+ Option<Schema.Field> keySchemaFieldID = Option.empty();
Review comment:
write handle doesn't track these details today. its all done in derived
classes.
##########
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/storage/HoodieFileWriterFactory.java
##########
@@ -48,12 +48,18 @@
public static <T extends HoodieRecordPayload, R extends IndexedRecord, I, K,
O> HoodieFileWriter<R> getFileWriter(
String instantTime, Path path, HoodieTable<T, I, K, O> hoodieTable,
HoodieWriteConfig config, Schema schema,
TaskContextSupplier taskContextSupplier) throws IOException {
+ return getFileWriter(instantTime, path, hoodieTable, config, schema,
Option.empty(), taskContextSupplier);
+ }
+
+ public static <T extends HoodieRecordPayload, R extends IndexedRecord, I, K,
O> HoodieFileWriter<R> getFileWriter(
+ String instantTime, Path path, HoodieTable<T, I, K, O> hoodieTable,
HoodieWriteConfig config, Schema schema,
+ Option<Schema.Field> schemaKeyFieldID, TaskContextSupplier
taskContextSupplier) throws IOException {
Review comment:
fixed.
##########
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/storage/HoodieHFileWriter.java
##########
@@ -122,7 +131,14 @@ public boolean canWrite() {
@Override
public void writeAvro(String recordKey, IndexedRecord object) throws
IOException {
- byte[] value = HoodieAvroUtils.avroToBytes((GenericRecord)object);
+ byte[] value = HoodieAvroUtils.avroToBytes((GenericRecord) object);
+ if (excludeKeyFromPayload) {
+ ValidationUtils.checkArgument(schemaKeyFieldID.isPresent(),
+ "Failed to exclude key from payload. Unknown key field for the
record.");
+ GenericRecord tmp = HoodieAvroUtils.bytesToAvro(value, schema);
Review comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
