terrymanu commented on issue #39090: URL: https://github.com/apache/shardingsphere/issues/39090#issuecomment-5004466458
Hi @MinLancer, thanks for the detailed report. This is a valid ShardingSphere-JDBC bug, but it is a recurrence of #21211 and should be classified as a duplicate. Both issues have the same root cause: Druid `proxyFilters` objects are captured as data-source properties, serialized with Java global tags, and then rejected when ShardingSphere reloads the YAML. This usage is supported. The [5.5.3 Java API documentation](https://shardingsphere.apache.org/document/5.5.3/en/user-manual/shardingsphere-jdbc/java-api/data-source/) states that ShardingSphere-JDBC supports all JDBC drivers and connection pools, while the [mode documentation](https://shardingsphere.apache.org/document/5.5.3/en/user-manual/shardingsphere-jdbc/java-api/mode/) explicitly supports Cluster mode with ZooKeeper. The correct fix is to persist only reconstructible data-source configuration values. We should not relax `ShardingSphereYamlConstructor` to instantiate arbitrary classes available on the classpath. No merged fixing PR or fixed release was identified for #21211, and the relevant serialization path remains in current master. I suggest reopening #21211 as the canonical bug, adding this 5.5.3 Cluster/ZooKeeper reproduction there, and then labeling #39090 with `type: duplicate` and `in: JDBC` and closing it as duplicate. The reply above is based on the analysis below; the detailed reasoning is kept here for reference and follow-up contributors. ### Problem Understanding - **Issue:** `OBS-1` — ShardingSphere-JDBC 5.5.3 starts successfully with Cluster mode, ZooKeeper, and a Druid data source containing filters, but restart fails on the persisted `!!com.alibaba.druid.filter.stat.StatFilter` tag. Source: #39090. - **Topology:** `OBS-1` — `JDBC + Cluster (ZooKeeper)`; the underlying database type is not reported, but the failure occurs while loading repository configuration before SQL execution or database access. - **Observed Evidence:** `OBS-2` — The [data-source documentation](https://shardingsphere.apache.org/document/5.5.3/en/user-manual/shardingsphere-jdbc/java-api/data-source/) supports all JDBC drivers and connection pools, and the [mode documentation](https://shardingsphere.apache.org/document/5.5.3/en/user-manual/shardingsphere-jdbc/java-api/mode/) demonstrates the same Java API with Cluster mode and ZooKeeper. ### Root Cause - **Observation:** `OBS-3` — `DataSourceProvidedDatabaseConfiguration` converts each supplied `DataSource` into `DataSourcePoolProperties` through `DataSourcePoolPropertiesCreator`: `infra/common/src/main/java/org/apache/shardingsphere/infra/config/database/impl/DataSourceProvidedDatabaseConfiguration.java:49` and `:79`. - **Observation:** `OBS-4` — `DataSourcePoolReflection` treats `List` and `Collection` getters as persistable properties and records their returned objects: `infra/data-source-pool/core/src/main/java/org/apache/shardingsphere/infra/datasource/pool/creator/DataSourcePoolReflection.java:67` and `:92`. `DataSourcePoolPropertiesCreator` retains all such properties when no pool-specific metadata marks them transient: `infra/data-source-pool/core/src/main/java/org/apache/shardingsphere/infra/datasource/pool/props/creator/DataSourcePoolPropertiesCreator.java:87`. - **Observation:** `OBS-5` — `YamlDataSourceConfigurationSwapper` places those properties into the persisted map, while `DataSourceUnitPersistService` marshals that map and later unmarshals it as `Map.class`: `infra/common/src/main/java/org/apache/shardingsphere/infra/yaml/config/swapper/resource/YamlDataSourceConfigurationSwapper.java:105`; `mode/core/src/main/java/org/apache/shardingsphere/mode/metadata/persist/config/database/DataSourceUnitPersistService.java:76` and `:95`. - **Observation:** `OBS-6` — The 5.5.3 YAML constructor accepts only the requested root class when resolving a class tag: `infra/util/src/main/java/org/apache/shardingsphere/infra/util/yaml/constructor/ShardingSphereYamlConstructor.java:115`. SnakeYAML therefore rejects Druid filter global tags while loading a `Map`. - **Inference:** `INF-1` (from `OBS-3`–`OBS-6`) — ShardingSphere is not literally serializing the complete `DruidDataSource` object. It reflectively snapshots getter values with selected declared types, but this still captures runtime state and object-valued collections such as `proxyFilters`. The persistence format then becomes impossible to load through the intentionally restricted YAML constructor. - **Inference:** `INF-2` (from `OBS-6`) — Allowing arbitrary classpath global tags would weaken the YAML class boundary and persist nonportable runtime instances. The fix belongs at the data-source property extraction/persistence boundary: retain reconstructible configuration values and exclude runtime object graphs. - **Confidence:** High. ### Problem Analysis - **Issue Type:** Duplicate; the underlying behavior remains a Bug. - **Evidence:** `OBS-7` — #21211 reports the same `DruidDataSource.setProxyFilters(...)` trigger and the same `StatFilter` global-tag rejection. Its discussion also records that #21048 did not resolve the Druid case. - **Prior-Fix Check:** `OBS-8` — #21211 has no linked fixing PR. Comparing tag `5.5.3` (`8d35894433416ef249ebb6ea21f8a8749648e9b6`) with current `apache/master` (`5c0e8aac13e35850b71bac61832de1e8881844cb`) shows that the relevant `DataSourcePoolReflection` and `DataSourceUnitPersistService` behavior remains unchanged. - **Negative Scenario:** `OBS-9` — #28480 is also Druid-related but concerns nondeterministic selection between overloaded setter methods. It has a different trigger and root-cause chain, so it is not the duplicate relationship used here. - **Fix Scope:** `INF-2` — Preserve the YAML class restriction and prevent object-valued runtime properties from entering persisted data-source configuration. Regression coverage should include a configuration round trip and a JDBC Cluster restart sentinel with an object-valued filter list. - **Label Recommendation:** `type: duplicate`, `in: JDBC`. ### Problem Conclusion - **Evidence Confidence:** High (`OBS-1`–`OBS-9`, `INF-1`, `INF-2`) - **Impact Scope:** ShardingSphere-JDBC Cluster persistence with Druid filters; potentially other reflection-adapted pools exposing object-valued collection getters (`OBS-1`, `OBS-4`, `INF-1`) - **Topology:** `JDBC + Cluster (ZooKeeper)` (`OBS-1`) - **Issue Type:** Duplicate; underlying defect is Bug (`OBS-7`, `INF-1`) - **Duplicate Of:** #21211 (`OBS-7`) - **Fix PR:** None identified; #21048 does not fix this root cause (`OBS-7`, `OBS-8`) - **Merged In:** N/A (`OBS-8`) - **Recommended Labels:** `type: duplicate`, `in: JDBC` (`OBS-1`, `OBS-7`) - **Next Action:** Reopen #21211 as the canonical unresolved bug, attach the current 5.5.3 Cluster/ZooKeeper evidence, track the fix and regression tests there, then close #39090 as duplicate (`OBS-7`, `OBS-8`, `INF-2`) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
