qzyu999 opened a new issue, #3672: URL: https://github.com/apache/fluss/issues/3672
## Summary The first tiering round for a newly created datalake-enabled table fails with a state machine validation error. Subsequent rounds succeed. ## Error ` ERROR org.apache.fluss.server.coordinator.LakeTableTieringManager - Fail to change state for table 1 from Scheduled to Tiered as it's not a valid state change. ERROR org.apache.fluss.server.coordinator.LakeTableTieringManager - Fail to change state for table 1 from Scheduled to Scheduled as it's not a valid state change. ` ## Expected Behavior The state machine should transition Scheduled Pending Tiering Tiered without errors on the first tiering round. ## Actual Behavior The tiering job completes its work (S3 write succeeds, \storeLakeTableOffsetsFile()\ succeeds) but the Coordinator rejects the state transition because the table is still in \Scheduled\ state when the tiering result arrives. The expected intermediate transitions (Scheduled Pending Tiering) haven't occurred yet. The second tiering round works fine by then the state machine has caught up. ## Steps to Reproduce 1. Create a table with \'table.datalake.enabled' = 'true', 'table.datalake.freshness' = '60s'\ 2. Insert a small amount of data (e.g., 3 rows) 3. Start the tiering service 4. Observe Coordinator logs within the first freshness interval ## Environment - Fluss: built from main (post-PR #3550) - Flink: 1.20.1 - Iceberg: 1.10.1 with Glue catalog - Infrastructure: ECS Fargate (single TabletServer, single bucket) ## Analysis Likely a timing issue: with a small dataset and fast S3 writes, the tiering job completes its first round before \LakeTableTieringManager\ has transitioned the table through the intermediate states. The state machine expects the sequence Scheduled Pending Tiering Tiered, but the tiering completion arrives while still in Scheduled. This may be more likely with: - Small datasets (fast tiering) - Fast network to S3 - Short freshness intervals ## Impact - First tiering round's snapshot is lost (not committed to Iceberg metadata) - Subsequent rounds work correctly - Data is eventually tiered, but there's a one-round delay on fresh tables -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
