VadimSurpin opened a new pull request, #12491: URL: https://github.com/apache/gravitino/pull/12491
Title: [#12490] improvement: Create supporting index on iceberg_tables for PostgreSQL JDBC catalogs at scale ### What changes were proposed in this pull request? Patched JdbcCatalogWithMetadataLocationSupport (the JDBC catalog implementation the Iceberg REST server uses for the jdbc backend) now creates a supporting index on the shared iceberg_tables control table right after schema initialization, when the backend is PostgreSQL: CREATE INDEX IF NOT EXISTS gravitino_iceberg_tables_namespace_pattern ON iceberg_tables (catalog_name, table_namespace text_pattern_ops) This is gated by a new property, jdbc.create-namespace-index (gravitino.iceberg-rest.jdbc.create-namespace-index in the REST server config), default true. Detection of PostgreSQL is done at runtime via DatabaseMetaData#getDatabaseProductName(), so this is a no-op for MySQL, SQLite, H2, and any other JDBC backend - no behavior changes for non-PostgreSQL deployments. If index creation fails for any reason (e.g. the configured database role lacks CREATE INDEX privileges), a warning is logged and catalog initialization proceeds normally; this is a performance fix, not a correctness one, so it must never block startup. ### Why are the changes needed? JdbcCatalog's namespace/table existence checks query iceberg_tables with an OR(exact match, LIKE prefix-match) predicate over (catalog_name, table_namespace). The table's only index is its primary key (catalog_name, table_namespace, table_name), which cannot serve a LIKE range scan under PostgreSQL's default locale-aware b-tree operator class, so this degrades to a full sequential scan once the table holds a large number of rows - observed at ~19.5ms per check and 100-570x throughput loss on NAMESPACE_CREATE/DROP/READ, TABLE_LIST/CREATE/DROP at 100,000 rows, confirmed via EXPLAIN ANALYZE. JdbcCatalogWithMetadataLocationSupport.initialize() now creates CREATE INDEX IF NOT EXISTS gravitino_iceberg_tables_namespace_pattern ON iceberg_tables (catalog_name, table_namespace text_pattern_ops) once per catalog initialization, detected via DatabaseMetaData so it's a no-op for MySQL/SQLite/H2. Controlled by the new jdbc.create-namespace-index property (default enabled) for operators whose database role lacks CREATE INDEX privileges. Never fails catalog initialization on error - a missing index is a performance issue, not a correctness one. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
