Smotrov opened a new issue, #10878:
URL: https://github.com/apache/iceberg/issues/10878
### Apache Iceberg version
1.6.0 (latest release)
### Query engine
Athena
### Please describe the bug 🐞
I have Athena Iceberg table.
The table has 2 partitions.
Each hour I update it with `MERGE`
I also do `DELETE` command daily to clean up everything inserted earlier
than 7 days.
```
SELECT count(*) FROM "my_table$files"
```
now **gives 16. Meanwhile data folder contains 158 files**.
None of
```
VACUUM my_table
```
or
```
OPTIMIZE my_table REWRITE DATA USING BIN_PACK
```
Is not helping to remove unnecessary files.
Table has following `TBLPROPERTIES`
```
TBLPROPERTIES (
'table_type'='iceberg',
'vacuum_max_snapshot_age_seconds'='60',
'write_compression'='ZSTD',
'format'='parquet',
'vacuum_max_metadata_files_to_keep'='2',
'optimize_rewrite_delete_file_threshold'='2',
'optimize_rewrite_data_file_threshold'='2'
)
```
It is that aggressive because I do not need any history of changes. I'm
interested in the latest state only.
Number of files is keep growing and never decrease, the spite the fact that
the number of rows in the table is almost constant.
What I'm doing wrong, and how to stop files inflation.
### Willingness to contribute
- [ ] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from
the Iceberg community
- [X] I cannot contribute a fix for this bug at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
