linliu-code commented on code in PR #12866:
URL: https://github.com/apache/hudi/pull/12866#discussion_r2105850608
##########
hudi-io/src/main/java/org/apache/hudi/io/hfile/HFileDataBlock.java:
##########
@@ -149,4 +155,74 @@ public boolean next(HFileCursor cursor, int
blockStartOffsetInFile) {
private boolean isAtFirstKey(int relativeOffset) {
return relativeOffset == HFILEBLOCK_HEADER_SIZE;
}
+
+ // ================ Below are for Write ================
+ protected final List<KeyValueEntry> entries = new ArrayList<>();
+
+ public HFileDataBlock(HFileContext context) {
+ this(context,-1L);
+ }
+
+ public HFileDataBlock(HFileContext context, long previousBlockOffset) {
+ super(context, HFileBlockType.DATA, previousBlockOffset);
+ // This is not used for write.
+ uncompressedContentEndRelativeOffset = -1;
+ }
+
+ public List<KeyValueEntry> getEntries() {
+ return entries;
+ }
+
+ public boolean isEmpty() {
+ return entries.isEmpty();
+ }
+
+ public void add(byte[] key, byte[] value) {
+ KeyValueEntry kv = new KeyValueEntry(key, value);
+ // Assume all entries are sorted before write.
+ add(kv, false);
+ }
+
+ public int getNumOfEntries() {
+ return entries.size();
+ }
+
+ protected void add(KeyValueEntry kv, boolean sorted) {
+ entries.add(kv);
+ if (sorted) {
+ entries.sort(KeyValueEntry::compareTo);
+ }
Review Comment:
As discussed offline, we could keep appending kv to a ByteBuffer instead of
accumulating entries in the list first. Ideally it should be more efficient.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
