yihua commented on code in PR #12866:
URL: https://github.com/apache/hudi/pull/12866#discussion_r2105494826
##########
hudi-io/src/main/java/org/apache/hudi/io/hfile/HFileDataBlock.java:
##########
@@ -149,4 +155,74 @@ public boolean next(HFileCursor cursor, int
blockStartOffsetInFile) {
private boolean isAtFirstKey(int relativeOffset) {
return relativeOffset == HFILEBLOCK_HEADER_SIZE;
}
+
+ // ================ Below are for Write ================
+ protected final List<KeyValueEntry> entries = new ArrayList<>();
+
+ public HFileDataBlock(HFileContext context) {
+ this(context,-1L);
+ }
+
Review Comment:
Could we remove this constructor?
##########
hudi-io/src/main/java/org/apache/hudi/io/hfile/HFileContext.java:
##########
@@ -29,37 +29,49 @@
public class HFileContext {
private final CompressionCodec compressionCodec;
private final HoodieDecompressor decompressor;
+ private final int blockSize;
- private HFileContext(CompressionCodec compressionCodec) {
+ private HFileContext(CompressionCodec compressionCodec, int blockSize) {
this.compressionCodec = compressionCodec;
this.decompressor =
HoodieDecompressorFactory.getDecompressor(compressionCodec);
+ this.blockSize = blockSize;
}
- CompressionCodec getCompressionCodec() {
+ public CompressionCodec getCompressionCodec() {
return compressionCodec;
}
- HoodieDecompressor getDecompressor() {
+ public HoodieDecompressor getDecompressor() {
Review Comment:
There is no need to change the visibility?
##########
hudi-io/src/main/java/org/apache/hudi/io/hfile/HFileDataBlock.java:
##########
@@ -149,4 +155,74 @@ public boolean next(HFileCursor cursor, int
blockStartOffsetInFile) {
private boolean isAtFirstKey(int relativeOffset) {
return relativeOffset == HFILEBLOCK_HEADER_SIZE;
}
+
+ // ================ Below are for Write ================
+ protected final List<KeyValueEntry> entries = new ArrayList<>();
+
+ public HFileDataBlock(HFileContext context) {
+ this(context,-1L);
+ }
+
+ public HFileDataBlock(HFileContext context, long previousBlockOffset) {
Review Comment:
It might be better to make the constructor private and create a static
method `HFileDataBlock.createOnWrite` to indicate that it should be used by
writer only.
##########
hudi-io/src/main/java/org/apache/hudi/io/hfile/HFileDataBlock.java:
##########
@@ -149,4 +155,74 @@ public boolean next(HFileCursor cursor, int
blockStartOffsetInFile) {
private boolean isAtFirstKey(int relativeOffset) {
return relativeOffset == HFILEBLOCK_HEADER_SIZE;
}
+
+ // ================ Below are for Write ================
+ protected final List<KeyValueEntry> entries = new ArrayList<>();
+
+ public HFileDataBlock(HFileContext context) {
+ this(context,-1L);
+ }
+
+ public HFileDataBlock(HFileContext context, long previousBlockOffset) {
+ super(context, HFileBlockType.DATA, previousBlockOffset);
+ // This is not used for write.
+ uncompressedContentEndRelativeOffset = -1;
+ }
+
+ public List<KeyValueEntry> getEntries() {
+ return entries;
+ }
+
+ public boolean isEmpty() {
+ return entries.isEmpty();
+ }
+
+ public void add(byte[] key, byte[] value) {
+ KeyValueEntry kv = new KeyValueEntry(key, value);
+ // Assume all entries are sorted before write.
+ add(kv, false);
+ }
+
+ public int getNumOfEntries() {
+ return entries.size();
+ }
+
+ protected void add(KeyValueEntry kv, boolean sorted) {
+ entries.add(kv);
+ if (sorted) {
+ entries.sort(KeyValueEntry::compareTo);
+ }
+ }
+
+ public byte[] getFirstKey() {
+ return entries.get(0).key;
+ }
+
+ public byte[] getLastKeyContent() {
+ if (entries.isEmpty()) {
+ return new byte[0];
+ }
+ return entries.get(entries.size() - 1).key;
+ }
Review Comment:
Revisit the visibility of these methods; if not intended to be called
outside `HFileWriterImpl`, make these default visibility instead of public.
##########
hudi-io/src/main/java/org/apache/hudi/io/hfile/HFileDataBlock.java:
##########
@@ -149,4 +155,74 @@ public boolean next(HFileCursor cursor, int
blockStartOffsetInFile) {
private boolean isAtFirstKey(int relativeOffset) {
return relativeOffset == HFILEBLOCK_HEADER_SIZE;
}
+
+ // ================ Below are for Write ================
+ protected final List<KeyValueEntry> entries = new ArrayList<>();
+
+ public HFileDataBlock(HFileContext context) {
+ this(context,-1L);
+ }
+
+ public HFileDataBlock(HFileContext context, long previousBlockOffset) {
+ super(context, HFileBlockType.DATA, previousBlockOffset);
+ // This is not used for write.
+ uncompressedContentEndRelativeOffset = -1;
+ }
Review Comment:
Move the constructor the beginning of the class
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
