the-other-tim-brown commented on code in PR #13726:
URL: https://github.com/apache/hudi/pull/13726#discussion_r2294933750
##########
hudi-common/src/test/java/org/apache/hudi/common/table/read/TestHoodieFileGroupReaderBase.java:
##########
@@ -969,4 +964,30 @@ private static String removeHiveStylePartition(String
partitionPath) {
return partitionPath;
}
+ private static IndexedRecord resetByteBufferPosition(IndexedRecord record) {
+ for (Schema.Field field : record.getSchema().getFields()) {
+ Object value = record.get(field.pos());
+ resetByteBufferField(value, field.schema());
+ }
+ return record;
+ }
+
+ private static void resetByteBufferField(Object value, Schema fieldSchema) {
+ if (value == null) {
+ return;
+ }
+ Schema.Type fieldType =
HoodieAvroUtils.unwrapNullable(fieldSchema).getType();
+ if (fieldType == Schema.Type.BYTES || fieldType == Schema.Type.FIXED) {
+ // Reset position of ByteBuffer or Fixed type fields
+ if (value instanceof ByteBuffer) {
+ ((ByteBuffer) value).rewind();
+ }
+ } else if (fieldType == Schema.Type.RECORD) {
+ resetByteBufferPosition((IndexedRecord) value);
+ } else if (fieldType == Schema.Type.ARRAY) {
+ ((List<Object>) value).forEach(element -> resetByteBufferField(element,
fieldSchema.getElementType()));
+ } else if (fieldType == Schema.Type.MAP) {
+ ((Map<Object, Object>) value).values().forEach(element ->
resetByteBufferField(element, fieldSchema.getValueType()));
+ }
+ }
Review Comment:
The writer will move the position in the byte buffer for Avro Orc (not sure
about others) and therefore you cannot read the value again. We can update that
handling though if required and then remove this to validate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
