aiborodin opened a new pull request, #18539: URL: https://github.com/apache/iceberg/pull/18539
During a storage outage, `JdbcTableOperations` and `JdbcViewOperations` retry reading the metadata file 20 times, for up to 10 minutes, before a table or view load fails. Clients stall waiting for these retries, and in a REST catalog server backed by `JdbcCatalog` each stalled load also holds a request thread. This adds a `metadata-refresh.max-retries` catalog property, similar to `iceberg.hive.metadata-refresh-max-retries` added for Hive in #3379. The default stays at 20, so behavior doesn't change. Changes: - Add `metadata-refresh.max-retries` to `CatalogProperties`, so other catalogs can use the same property. - Read it in `JdbcTableOperations` and `JdbcViewOperations`, reject negative values, and pass it to the existing `refreshFromMetadataLocation` overloads. - Document the property for the JDBC catalog. Each retry is a full metadata read, so the total time also depends on the FileIO's own timeouts and retries, for example `http-client.apache.socket-timeout-ms` and `s3.retry.num-retries` for `S3FileIO`. Testing: new tests in `TestJdbcCatalog` and `TestJdbcViewCatalog` check that a refresh makes `max-retries + 1` reads and that negative values are rejected. I ran the JDBC test suites, Spotless, checkstyle and revapi for `iceberg-core` locally. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
