RussellSpitzer commented on issue #15307:
URL: https://github.com/apache/iceberg/issues/15307#issuecomment-3893654353
We have a few tests for this internally,
In TestTableMetadata see testAddPreviousMetadataRemove*****()
But i realized none of those touched Spark so I had cursor add a quick test
for this (do 110 alter table operations, make sure that you only have 100
entries"
```java
// I Live in TestMetadataTables for convience
@TestTemplate
public void testMetadataLogEntriesLimitedByPreviousVersionsMax() throws
Exception {
sql(
"CREATE TABLE %s (id bigint, data string) USING iceberg
TBLPROPERTIES"
+ "('format-version'='%s',
'write.metadata.previous-versions-max'='100')",
tableName, formatVersion);
// Perform 110 commits by changing table properties
for (int i = 0; i < 110; i++) {
sql("ALTER TABLE %s SET TBLPROPERTIES('test.property.%d' '%d')",
tableName, i, i);
}
Table table = Spark3Util.loadIcebergTable(spark, tableName);
table.refresh();
TableMetadata tableMetadata = ((HasTableOperations)
table).operations().current();
// Verify that previousFiles (metadata log) has at most 100 entries
// The metadata log tracks up to write.metadata.previous-versions-max
previous metadata files
List<TableMetadata.MetadataLogEntry> metadataLogEntries =
Lists.newArrayList(tableMetadata.previousFiles());
assertThat(metadataLogEntries)
.as("Metadata log should be limited to 100 entries by
write.metadata.previous-versions-max")
.hasSizeLessThanOrEqualTo(100);
// Also verify using the metadata_log_entries table (which includes the
current metadata file)
// So it should have at most 101 entries (100 previous + 1 current)
List<Object[]> metadataLogs = sql("SELECT * FROM
%s.metadata_log_entries", tableName);
assertThat(metadataLogs)
.as("Metadata log entries table should have at most 101 entries (100
previous + 1 current)")
.hasSizeLessThanOrEqualTo(101);
}
```
This also passed, so I think we'll need more specific details to your
circumstance to repo
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
