rangareddy commented on issue #17383: URL: https://github.com/apache/hudi/issues/17383#issuecomment-5351242048
This issue was reviewed as part of the JIRA-migrated backlog triage (HUDI-9094). **Findings: not done - the exact code in the description is unchanged on `master`.** `hudi-common/src/main/java/org/apache/hudi/common/schema/internal/io/FileBasedInternalSchemaStorageManager.java:87` still materialises the full string as a `byte[]` and wraps it: ```java byte[] writeContent = getUTF8Bytes(historySchemaStr); ... Option.of(HoodieInstantWriter.convertByteArrayToWriter(writeContent))); ``` Worth scoping the ticket a little wider than the single site you quoted, because `convertByteArrayToWriter` is the same pattern everywhere it appears: - `hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/timeline/LSMTimelineWriter.java:209`, `:219` - `hudi-client/hudi-client-common/src/main/java/org/apache/hudi/index/bucket/ConsistentBucketIndexUtils.java:201` - `hudi-common/src/main/java/org/apache/hudi/common/model/HoodiePartitionMetadata.java:116` - `hudi-common/src/main/java/org/apache/hudi/common/model/PartitionBucketIndexHashingConfig.java:158` - `hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableMetaClient.java:336` Fixing only `persistHistorySchemaStr` would leave the helper - and the buffer-the-whole-payload behaviour - in place for the rest. If the intent is for `HoodieInstantWriter` implementations to stream to the output stream instead, that is a change to the family, and `convertByteArrayToWriter` should end up with no production callers. Keeping this open. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
