cc5566 opened a new pull request, #9779: URL: https://github.com/apache/gravitino/pull/9779
## What changes were proposed in this pull request? This PR adds MySQL as an alternative storage backend for partition statistics, complementing the existing Lance storage implementation. ## Why are the changes needed? Currently, partition statistics can only be stored using Lance format. This PR adds MySQL as an option for users who: - Prefer relational database storage - Already have MySQL infrastructure - Need transactional guarantees for statistics updates - Want to use existing database backup/recovery tools ## Does this PR introduce any user-facing change? No breaking changes. The storage backend is pluggable via configuration: ```properties # Use MySQL storage (default is Lance) gravitino.stats.partition.storageFactoryClass = org.apache.gravitino.stats.storage.MysqlPartitionStatisticStorageFactory # MySQL connection settings gravitino.stats.partition.storageOption.jdbc-url = jdbc:mysql://localhost:3306/gravitino gravitino.stats.partition.storageOption.jdbc-user = root gravitino.stats.partition.storageOption.jdbc-password = gravitino gravitino.stats.partition.storageOption.jdbc-driver = com.mysql.cj.jdbc.Driver ``` ## Implementation Details ### Architecture - Implements `PartitionStatisticStorage` interface - Uses Apache Commons DBCP2 for connection pooling - Supports all CRUD operations with transactions - Partition range queries with inclusive/exclusive bounds - Batch operations for efficiency ### Database Schema New table `partition_statistic_meta`: - Primary key: (table_id, partition_name, statistic_name) - Stores statistics as JSON - Includes audit information ### Features - ✅ Insert/Update (upsert) - ✅ List all/by name/by range - ✅ Delete operations - ✅ Transaction support - ✅ Connection pooling - ✅ JSON serialization ## How was this patch tested? **Unit Tests:** - `TestMysqlPartitionStatisticStorage` - Core storage operations - `TestMysqlPartitionStatisticStorageFactory` - Factory configuration **Integration Tests:** - `TestMysqlPartitionStatisticStorageIT` - End-to-end with real MySQL **Manual Testing:** - Verified with running Gravitino server - Tested MySQL container integration - Confirmed REST API compatibility ## Dependencies Zero new dependencies added. Uses existing: - Apache Commons DBCP2 - MySQL JDBC Driver - Jackson (JSON) ## Backward Compatibility ✅ Fully backward compatible - No changes to existing APIs - Default storage remains Lance - Pluggable via configuration only -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
