Shekharrajak opened a new pull request, #6915: URL: https://github.com/apache/paimon/pull/6915
[spark][filesystems][core] Introduce archive ability for table partitions ### Purpose Linked issue: close #5510 Implements archive functionality for Paimon table partitions to optimize storage costs by moving partition files to Archive/ColdArchive storage tiers in S3 and OSS. Supports archive, restore, and unarchive operations via Spark SQL DDL. ### Tests - Unit tests: `ArchivePartitionActionTest` (9 tests) - Integration tests: `ArchivePartitionActionITCase` (3 test templates) - SQL tests: `ArchivePartitionSQLTest` (8 tests) ### API and Format **New APIs**: - `StorageType` enum (Standard, Archive, ColdArchive) - `FileIO.archive()`, `FileIO.restoreArchive()`, `FileIO.unarchive()` **SQL Syntax**: ALTER TABLE table PARTITION (dt='2024-01-01') ARCHIVE; ALTER TABLE table PARTITION (dt='2024-01-01') COLD ARCHIVE; ALTER TABLE table PARTITION (dt='2024-01-01') RESTORE ARCHIVE; ALTER TABLE table PARTITION (dt='2024-01-01') UNARCHIVE;**Storage Format**: No changes. Original paths preserved in metadata (in-place archiving). ### Documentation - Added `docs/content/concepts/archive.md` - Updated `docs/content/spark/sql-alter.md` with archive syntax -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
