Davis-Zhang-Onehouse opened a new issue, #20691: URL: https://github.com/apache/hudi/issues/20691
Part of #20683 (RFC-118, #20684). The first RFC-118 stage: `UnstructuredDataSource` over the S3/GCS events table, a source contract in Hudi Streamer (a source declares the transformer chain, record key, Parquet injector and marker settings it requires; conflicting settings are refused), an instant-aware transformer, the blob table contract (schema, keys, config) and the blob flow (one row per object version, inline bytes up to a size threshold). In this PR an object above the inline threshold is still copied into the table under `.hoodie/.blob`; follow-up sub-issue 37 of the plan replaces that with an in-place reference to the source object (RFC-118 ยง2). A failed round can leave unreferenced copies until the blob-rollback PR later in the chain adds marker-based cleanup. PR: (linked once opened) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
