yihua commented on code in PR #11163:
URL: https://github.com/apache/hudi/pull/11163#discussion_r1594782992
##########
hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieParquetDataBlock.java:
##########
@@ -109,7 +112,7 @@ protected byte[] serializeRecords(List<HoodieRecord>
records) throws IOException
Schema writerSchema = new
Schema.Parser().parse(super.getLogBlockHeader().get(HeaderMetadataType.SCHEMA));
ByteArrayOutputStream baos = new ByteArrayOutputStream();
- try (FSDataOutputStream outputStream = new FSDataOutputStream(baos, null))
{
+ try (SeekableDataOutputStream outputStream =
storage.makeOutputSeekable(baos)) {
Review Comment:
Should we add a new `HoodieFileWriterFactory#getFileWriter` that takes
`ByteArrayOutputStream`, instead of relying on `SeekableDataOutputStream`, to
avoid adding `storage.makeOutputSeekable(baos)`?
##########
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieArchivedTimeline.java:
##########
@@ -266,7 +266,7 @@ public static void loadInstants(
.filter(fileName -> filter == null ||
LSMTimeline.isFileInRange(filter, fileName))
.parallel().forEach(fileName -> {
// Read the archived file
- try (HoodieAvroParquetReader reader = (HoodieAvroParquetReader)
HoodieFileReaderFactory.getReaderFactory(HoodieRecordType.AVRO)
+ try (HoodieAvroFileReaderBase reader = (HoodieAvroFileReaderBase)
HoodieFileReaderFactory.getReaderFactory(HoodieRecordType.AVRO)
Review Comment:
A side topic, what is the difference between `HoodieAvroFileReader` and
`HoodieAvroFileReaderBase`? I think we can rename `HoodieAvroFileReaderBase`
to `HoodieAvroFileReader` if there is no difference and remove the interface
`HoodieAvroFileReader`.
##########
hudi-common/src/main/java/org/apache/hudi/common/table/TableSchemaResolver.java:
##########
@@ -356,8 +355,8 @@ private MessageType readSchemaFromHFileBaseFile(Path
hFilePath) throws IOExcepti
private MessageType readSchemaFromORCBaseFile(StoragePath orcFilePath)
throws IOException {
LOG.info("Reading schema from {}", orcFilePath);
-
- HoodieAvroOrcReader orcReader = new
HoodieAvroOrcReader(metaClient.getRawHoodieStorage().getConf(), orcFilePath);
+ HoodieFileReader orcReader =
HoodieFileReaderFactory.getReaderFactory(HoodieRecord.HoodieRecordType.AVRO)
+ .getFileReader(null, metaClient.getRawHoodieStorage().getConf(),
orcFilePath, HoodieFileFormat.ORC, Option.empty());
Review Comment:
What is the first argument which is `null`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
