Chowdhury-Anik opened a new pull request, #1416: URL: https://github.com/apache/hamilton/pull/1416
This commit adds four data sink methods for Polars LazyFrame: - sink_parquet: Write LazyFrame to Parquet format - sink_csv: Write LazyFrame to CSV format - sink_ipc: Write LazyFrame to IPC/Feather format - sink_ndjson: Write LazyFrame to NDJSON format These sinks allow users to write LazyFrames directly without needing to call .collect() first, improving performance for large datasets. Fixes #791 --- PR TEMPLATE INSTRUCTIONS (1) --- Looking to submit a Apache Hamilton Dataflow to the sf-hamilton-contrib module? If so go the the `Preview` tab and select the appropriate sub-template: * [sf-hamilton-contrib template](?expand=1&template=HAMILTON_CONTRIB_PR_TEMPLATE.md) Else, if not, please remove this block of text. --- PR TEMPLATE INSTRUCTIONS (2) --- [Short description explaining the high-level reason for the pull request] ## Changes ## How I tested this ## Notes ## Checklist - [ ] PR has an informative and human-readable title (this will be pulled into the release notes) - [ ] Changes are limited to a single goal (no scope creep) - [ ] Code passed the pre-commit check & code is left cleaner/nicer than when first encountered. - [ ] Any _change_ in functionality is tested - [ ] New functions are documented (with a description, list of inputs, and expected output) - [ ] Placeholder code is flagged / future TODOs are captured in comments - [ ] Project documentation has been updated if adding/changing functionality. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
