LuciferYang opened a new pull request, #11011:
URL: https://github.com/apache/paimon/pull/11011

   ### Purpose
   
   close #11010
   
   `SparkGenericCatalogWithHiveTest` fails in the Spark 4 CI job when a test 
class that creates a `HiveCatalog` with `hive-conf-dir` ran earlier in the same 
surefire fork. `HiveCatalog.createHiveConf` clears HiveConf's JVM-wide 
`hive-site.xml` location, so the next `TestHiveMetastore` falls back to the 
BoneCP connection pool, which is not on the Spark 4 test classpath. The issue 
has the stack trace.
   
   `TestHiveMetastore.initConf` now sets `datanucleus.connectionPoolingType` to 
`DBCP` with `setIfUnset`, like the other metastore options it sets there, so 
the pool no longer depends on which `hive-site.xml` the server reads. The 
`hive-site.xml` files of `paimon-spark-ut`, `paimon-spark-4.0` and 
`paimon-spark-4.1` already set DBCP. The metastores in `paimon-spark-3.x` and 
`paimon-hive-connector-common` tests, whose `hive-site.xml` sets no pool, 
switch from BoneCP to DBCP; `commons-dbcp` is already on their test classpath.
   
   `HiveCatalog.createHiveConf` is unchanged. I haven't found an effect of its 
statics outside tests.
   
   ### Tests
   
   `SparkHiveMetastoreAfterHiveConfDirITCase` builds a HiveConf from a 
`hive-conf-dir`, then starts a `TestHiveMetastore` and runs `CREATE DATABASE` 
and `DROP DATABASE` through Spark's built-in Hive catalog. Without the fix it 
fails on Spark 4 with the BoneCP `NoClassDefFoundError` and `There is no 
database named conf_dir_db`. It is an ITCase so that it gets its own JVM: the 
metastore's transaction handler creates its connection pool once per JVM, and a 
metastore that ran earlier in a shared fork would hide the problem. On Spark 3 
BoneCP is on the classpath, so the test passes there with or without the fix.
   
   ### API and Format
   
   no
   
   ### Documentation
   
   no
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to