prashantwason commented on a change in pull request #1687:
URL: https://github.com/apache/hudi/pull/1687#discussion_r435494902
##########
File path: hudi-client/src/main/java/org/apache/hudi/io/HoodieMergeHandle.java
##########
@@ -214,6 +223,36 @@ private boolean writeRecord(HoodieRecord<T> hoodieRecord,
Option<IndexedRecord>
*/
public void write(GenericRecord oldRecord) {
String key =
oldRecord.get(HoodieRecord.RECORD_KEY_METADATA_FIELD).toString();
+
+ if (hoodieTable.requireSortedRecords()) {
Review comment:
In my changes:
1. Spark based sorting is used (see WriteHandle.java)
2. Sorting is ONLY done if format requires it (so there is no change to
Parquet format)
3. Merge style sort is implemented
>> The advantage is more modularity as well as low memory overhead to merge..
Yes, that was my vision too.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
