Sigma-Ma opened a new pull request, #176: URL: https://github.com/apache/hbase-connectors/pull/176
### What changes were proposed in this pull request? Restore datasource option > SparkConf > default precedence for these Spark 4 read settings: - `hbase.spark.query.cacheblocks` - `hbase.spark.query.cachedrows` - `hbase.spark.bulkget.size` Resolve the settings on the driver and pass the immutable values to partition readers through the serialized reader factory. This PR depends on HBASE-30512 (#175) . Please merge #175 first. This change preserves its complete-row behavior: `hbase.spark.query.batchsize` remains ignored in both datasource options and SparkConf. ### Why are the changes needed? Spark 4 currently reads these settings only from datasource options. Applications that configure them globally through SparkConf can silently use different read settings after upgrading from Spark 3. Restoring the existing precedence preserves global settings while allowing per-read overrides. ### How was this patch tested? ```bash mvn -B -ntp -Pspark4 -pl spark4/hbase-spark4 -Dsuites=org.apache.hadoop.hbase.spark.datasources.HBaseReadOptionsSuite,org.apache.hadoop.hbase.spark.datasources.HBaseReadOptionsReaderSuite,org.apache.hadoop.hbase.spark.datasources.HBaseScanBuilderSuite,org.apache.hadoop.hbase.spark.datasources.HBaseTableProviderSuite -Dsurefire.rerunFailingTestsCount=0 test ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
