Yang Jie created SPARK-56177:
--------------------------------
Summary: V2 file bucketing write support
Key: SPARK-56177
URL: https://issues.apache.org/jira/browse/SPARK-56177
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 4.2.0
Reporter: Yang Jie
Enable bucketed writes for V2 file tables. `FileWrite.bucketSpec` uses
`V1WritesUtils.getWriterBucketSpec()` replacing hardcoded `None`.
`FileTable.createFileWriteBuilder` passes `catalogTable.bucketSpec` to the
write pipeline. `FileDataSourceV2.getTable` uses `collect` to skip
`BucketTransform` (handled via `catalogTable.bucketSpec` instead). All 6 format
Write/Table classes updated with `BucketSpec` parameter. Note: bucket pruning
and bucket join optimization (read-path) are not included.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
