Davis-Zhang-Onehouse opened a new issue, #20691:
URL: https://github.com/apache/hudi/issues/20691

   Part of #20683 (RFC-118, #20684).
   
   The first RFC-118 stage: `UnstructuredDataSource` over the S3/GCS events 
table, a source contract in Hudi Streamer (a source declares the transformer 
chain, record key, Parquet injector and marker settings it requires; 
conflicting settings are refused), an instant-aware transformer, the blob table 
contract (schema, keys, config) and the blob flow (one row per object version, 
inline bytes up to a size threshold).
   
   In this PR an object above the inline threshold is still copied into the 
table under `.hoodie/.blob`; follow-up sub-issue 37 of the plan replaces that 
with an in-place reference to the source object (RFC-118 ยง2). A failed round 
can leave unreferenced copies until the blob-rollback PR later in the chain 
adds marker-based cleanup.
   
   PR: (linked once opened)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to