fxbing commented on issue #3530: URL: https://github.com/apache/fluss/issues/3530#issuecomment-4817545658
I took a closer look at this flaky failure. The failure seems to be caused by a race between test cleanup and auto-partition creation. In `testCountPushDownForLogTable`, the first parameterized run creates a partitioned log table with auto-partition enabled. During cleanup, `DROP DATABASE defaultdb CASCADE` removes the database/table, but a scheduled auto-partition task may still run afterwards. When it reaches `ZooKeeperClient.registerPartitionAssignmentAndMetadata()`, the current code creates `PartitionsZNode.path(tablePath)` with `creatingParentsIfNeeded()`, so a stale partition creation can recreate the dropped database/table parent znodes as empty metadata paths. Then the next parameterized run fails at `CREATE DATABASE defaultdb` because the database path exists again. My proposed fix is: 1. In `ZooKeeperClient.registerPartitionAssignmentAndMetadata()`, create only the direct `/partitions` node for an existing table, and stop using `creatingParentsIfNeeded()` for the table metadata partition parent. If the table/database has already been dropped, partition creation should fail instead of recreating parent znodes. 2. In `CoordinatorEventProcessor.processCreatePartition()`, skip stale `CreatePartitionEvent`s when the table id is no longer present in the coordinator context. 3. In this count-pushdown IT, stop relying on auto-partition scheduling and add the required partitions explicitly, since this test is about count aggregate pushdown rather than auto-partition behavior. I also plan to add regression coverage for the ZooKeeper parent recreation case and the stale coordinator partition event case, then verify the affected server tests and the original Flink 2.2 IT. Does this direction sound reasonable? If there is any concern about changing the ZooKeeper parent creation behavior, I’d be happy to adjust the approach. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
