CodyPin opened a new issue, #3308: URL: https://github.com/apache/paimon/issues/3308
### Search before asking - [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version 0.7.0 ### Compute Engine Flink 1.17.2 ### Minimal reproduce step Not sure. The problem is inconsistent, on some MySQL server it won't happen, on some it will filter out most of the tables ### What doesn't meet your expectations? To load all the tables mentioned in the argument 'including_tables' ### Anything else? Some details: I have checked and tried most of the results online, check for database permission, table permission, etc. I have also tried loading all tables, but it will also says it 'filtered out of table capturing'. I have looked into Flink, Ververica, and Paimon's code but I still have no idea. And I don't seem to be able to find anyone else with this problem. Below I am trying to access some tables from the database 'littlemall', those tables are: - order - order_detail - payment_record - store - user - order_detail_product_item - order_detail_delivery_fee_item - order_audit_log - order_detail_promotion_item - sku_list - payment_refund_record - mall_category - store_category But the end result is that Flink was only able to capture two tables, namely sku_list and user  I have cropped some logs and screenshots to provide more information, please let me know if you need anything additional or if you have any idea at all! Args: ```2024-05-08 08:44:25,714 INFO org.apache.paimon.flink.action.ActionFactory [] - mysql_sync_database job args: --warehouse hdfs://hktv-cluster/core/data-stream/flink/data/paimon/warehouse --database littlemall --ignore_incompatible true --mysql_conf hostname=<hostname> --mysql_conf port=3306 --mysql_conf username=flink-cdc --mysql_conf password=**** --mysql_conf database-name=littlemall --mysql_conf start-server-id=7400 --mysql_conf server-time-zone=Asia/Hong_Kong --including_tables order|order_detail|payment_record|store|user|order_detail_product_item|order_detail_delivery_fee_item|order_audit_log|order_detail_promotion_item|sku_list|payment_refund_record|mall_category|store_category --catalog_conf metastore=filesystem --catalog_conf case-sensitive=false --table_conf bucket=4 --table_conf changelog-producer=input``` MySQL CLI to confirm user have permission to read and are base table:   Tables being filtered out: ```2024-05-08 08:44:32,440 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - list of available databases is: [littlemall] 2024-05-08 08:44:32,440 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - Read list of available tables in each database 2024-05-08 08:44:32,443 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.TASK_EXECUTION' is filtered out of table capturing 2024-05-08 08:44:32,443 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.TASK_EXECUTION_PARAMS' is filtered out of table capturing 2024-05-08 08:44:32,443 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.TASK_LOCK' is filtered out of table capturing 2024-05-08 08:44:32,443 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.TASK_SEQ' is filtered out of table capturing 2024-05-08 08:44:32,443 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.TASK_TASK_BATCH' is filtered out of table capturing 2024-05-08 08:44:32,443 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.address' is filtered out of table capturing 2024-05-08 08:44:32,443 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.address_area' is filtered out of table capturing 2024-05-08 08:44:32,443 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.address_district' is filtered out of table capturing 2024-05-08 08:44:32,443 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.address_estate' is filtered out of table capturing 2024-05-08 08:44:32,444 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.address_estate_flat' is filtered out of table capturing 2024-05-08 08:44:32,444 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.address_estate_floor' is filtered out of table capturing 2024-05-08 08:44:32,444 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.address_street' is filtered out of table capturing 2024-05-08 08:44:32,444 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.address_street_number' is filtered out of table capturing 2024-05-08 08:44:32,444 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.address_street_number_2_estate' is filtered out of table capturing 2024-05-08 08:44:32,444 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.delivery_provider' is filtered out of table capturing 2024-05-08 08:44:32,444 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.delivery_provider_store_delivery' is filtered out of table capturing 2024-05-08 08:44:32,444 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.delivery_provider_store_pickup_point' is filtered out of table capturing 2024-05-08 08:44:32,444 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.delivery_provider_third_party' is filtered out of table capturing 2024-05-08 08:44:32,444 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.mall_category' is filtered out of table capturing 2024-05-08 08:44:32,444 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.mall_landing_component' is filtered out of table capturing 2024-05-08 08:44:32,444 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.oauth2_authorization' is filtered out of table capturing 2024-05-08 08:44:32,444 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.oauth2_authorization_consent' is filtered out of table capturing 2024-05-08 08:44:32,444 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.oauth2_registered_client' is filtered out of table capturing 2024-05-08 08:44:32,444 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.order' is filtered out of table capturing 2024-05-08 08:44:32,444 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.order_audit_log' is filtered out of table capturing 2024-05-08 08:44:32,444 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Resolved ResourceManager address, beginning registration 2024-05-08 08:44:32,444 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.order_detail' is filtered out of table capturing 2024-05-08 08:44:32,444 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.order_detail_delivery_fee_item' is filtered out of table capturing 2024-05-08 08:44:32,444 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.order_detail_product_item' is filtered out of table capturing 2024-05-08 08:44:32,444 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.order_detail_promotion_item' is filtered out of table capturing 2024-05-08 08:44:32,444 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.payment_record' is filtered out of table capturing 2024-05-08 08:44:32,444 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.payment_refund_record' is filtered out of table capturing 2024-05-08 08:44:32,444 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.pending_refund_and_refill_stock' is filtered out of table capturing 2024-05-08 08:44:32,445 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.shipment' is filtered out of table capturing 2024-05-08 08:44:32,445 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.shipment_status_log' is filtered out of table capturing 2024-05-08 08:44:32,445 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.shipment_third_party' is filtered out of table capturing 2024-05-08 08:44:32,445 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - including table 'littlemall.sku_list' for further processing 2024-05-08 08:44:32,445 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.store' is filtered out of table capturing 2024-05-08 08:44:32,445 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.store_category' is filtered out of table capturing 2024-05-08 08:44:32,445 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.store_delivery_integration' is filtered out of table capturing 2024-05-08 08:44:32,445 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.store_landing_component' is filtered out of table capturing 2024-05-08 08:44:32,445 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.store_mall_category_mapping' is filtered out of table capturing 2024-05-08 08:44:32,445 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.store_static_page' is filtered out of table capturing 2024-05-08 08:44:32,445 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - including table 'littlemall.user' for further processing 2024-05-08 08:44:32,445 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - 'littlemall.user_membership' is filtered out of table capturing 2024-05-08 08:44:32,512 INFO io.debezium.jdbc.JdbcConnection [] - Connection gracefully closed``` ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
