pitrou commented on code in PR #1291:
URL: https://github.com/apache/parquet-mr/pull/1291#discussion_r1519548335
##########
parquet-column/src/main/java/org/apache/parquet/column/values/bytestreamsplit/ByteStreamSplitValuesWriter.java:
##########
@@ -140,4 +141,61 @@ public String memUsageString(String prefix) {
return String.format("%s DoubleByteStreamSplitWriter %d bytes", prefix,
getAllocatedSize());
}
}
+
+ public static class IntegerByteStreamSplitValuesWriter extends
ByteStreamSplitValuesWriter {
+ public IntegerByteStreamSplitValuesWriter(int initialCapacity, int
pageSize, ByteBufferAllocator allocator) {
+ super(4, initialCapacity, pageSize, allocator);
+ }
+
+ @Override
+ public void writeInteger(int v) {
+ super.scatterBytes(BytesUtils.intToBytes(v));
+ }
+
+ @Override
+ public String memUsageString(String prefix) {
+ return String.format("%s IntegerByteStreamSplitWriter %d bytes", prefix,
getAllocatedSize());
+ }
+ }
+
+ public static class LongByteStreamSplitValuesWriter extends
ByteStreamSplitValuesWriter {
+ public LongByteStreamSplitValuesWriter(int initialCapacity, int pageSize,
ByteBufferAllocator allocator) {
+ super(8, initialCapacity, pageSize, allocator);
+ }
+
+ @Override
+ public void writeLong(long v) {
+ super.scatterBytes(BytesUtils.longToBytes(v));
+ }
+
+ @Override
+ public String memUsageString(String prefix) {
+ return String.format("%s LongByteStreamSplitWriter %d bytes", prefix,
getAllocatedSize());
+ }
+ }
+
+ public static class FixedLenByteArrayByteStreamSplitValuesWriter extends
ByteStreamSplitValuesWriter {
+ private final int length;
+
+ public FixedLenByteArrayByteStreamSplitValuesWriter(
+ int length, int initialCapacity, int pageSize, ByteBufferAllocator
allocator) {
+ super(length, initialCapacity, pageSize, allocator);
+ this.length = length;
+ }
+
+ @Override
+ public final void writeBytes(Binary v) {
+ if (v.length() != length) {
Review Comment:
Good point. I copied this from `FixedLenByteArrayPlainValuesWriter` which
does the same runtime check, but other `writeBytes` implementation don't seem
to bother.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
