laskoviymishka opened a new issue, #287: URL: https://github.com/apache/iceberg-go/issues/287
### Feature Request / Improvement ### **Feature Request: Implement Append Operation for Iceberg Tables** #### **Description** Currently, the `iceberg-go` library lacks support for appending data files to an Iceberg table. I would like to have such feature to implement iceberg target for https://github.com/doublecloud/transfer/issues/199 #### **Expected Behavior** - Implement an **Append API** that allows adding new data files to an Iceberg table. - Ensure atomic transactions using the **Iceberg commit protocol**. - Generate **DataFile metadata** for newly appended files. - Support appending Parquet files (initially) with potential expansion to other formats like Avro and ORC. #### **Implementation details** - Follow the **commit logic** similar to what’s implemented in [[Iceberg Java](https://github.com/apache/iceberg/tree/main/api)](https://github.com/apache/iceberg/tree/main/api) and [[Iceberg Rust](https://github.com/apache/iceberg-rust/issues/329)](https://github.com/apache/iceberg-rust/issues/329). - Modify the table metadata to include newly added files while preserving the **snapshot isolation model** of Iceberg. - Ensure compatibility with different **catalog implementations** (e.g., Hadoop, REST). - Consider implementing tests using **temporary test tables and mock filesystems**. #### **References** - Iceberg Spec: [[Append Transactions](https://iceberg.apache.org/spec/#append-transactions)](https://iceberg.apache.org/spec/#append-transactions) - Iceberg Java Append API: [[Java Implementation](https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/AppendFiles.java)](https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/AppendFiles.java) - Related discussion in Iceberg Rust: [[Issue #329](https://github.com/apache/iceberg-rust/issues/329)](https://github.com/apache/iceberg-rust/issues/329) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
