alexeykudinkin commented on code in PR #6132:
URL: https://github.com/apache/hudi/pull/6132#discussion_r938180595
##########
rfc/rfc-46/rfc-46.md:
##########
@@ -128,21 +173,88 @@ Following major components will be refactored:
1. `HoodieWriteHandle`s will be
1. Accepting `HoodieRecord` instead of raw Avro payload (avoiding Avro
conversion)
- 2. Using Combining API engine to merge records (when necessary)
+ 2. Using Record Merge API to merge records (when necessary)
3. Passes `HoodieRecord` as is to `FileWriter`
2. `HoodieFileWriter`s will be
1. Accepting `HoodieRecord`
2. Will be engine-specific (so that they're able to handle internal record
representation)
3. `HoodieRealtimeRecordReader`s
1. API will be returning opaque `HoodieRecord` instead of raw Avro payload
+### Config for Record Merge
+The MERGE_CLASS_NAME config is engine-aware. If you are not specified the
MERGE_CLASS_NAME, MERGE_CLASS_NAME will be specified default according to your
engine type.
+
+### Public Api in HoodieRecord
+Because we implement different types of records, we need to implement
functionality similar to AvroUtils in HoodieRecord for different data(avro,
InternalRow, RowData).
+Its public API will look like following:
+
+```java
+import java.util.Properties;
+
+class HoodieRecord {
+
+ /**
+ * Get column in record to support RDDCustomColumnsSortPartitioner
+ */
+ Object getRecordColumnValues(Schema recordSchema, String[] columns,
+ boolean consistentLogicalTimestampEnabled);
+
+ /**
+ * Support bootstrap.
+ */
+ HoodieRecord mergeWith(HoodieRecord other, Schema targetSchema) throws
IOException;
+
+ /**
+ * Rewrite record into new schema(add meta columns)
+ */
+ HoodieRecord rewriteRecord(Schema recordSchema, Properties props, Schema
targetSchema)
+ throws IOException;
+
+ /**
+ * Support schema evolution.
+ */
+ HoodieRecord rewriteRecordWithNewSchema(Schema recordSchema, Properties
props, Schema newSchema,
+ Map<String, String> renameCols) throws IOException;
+
+ HoodieRecord updateValues(Schema recordSchema, Properties props,
+ Map<String, String> metadataValues) throws IOException;
+
+ boolean isDelete(Schema recordSchema, Properties props) throws IOException;
+
+ /**
+ * Is EmptyRecord. Generated by ExpressionPayload.
+ */
+ boolean shouldIgnore(Schema recordSchema, Properties props) throws
IOException;
+
+ /**
+ * This method used to extract HoodieKey through parameters.
+ */
+ HoodieRecord getKeyWithParams(
Review Comment:
Sorry, i think i previously was under wrong impression that this method
returns `HoodieKey`, but now i see that it returns the same `HoodieRecord` and
the only thing that is changed essentially is just the name of the method.
I [left my
comment](https://github.com/apache/hudi/pull/5629#discussion_r938253077)
elaborating why i think we should have no need for this method, PTAL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
