Sigma-Ma opened a new pull request, #172: URL: https://github.com/apache/hbase-connectors/pull/172
### What changes were proposed in this pull request? Normalize row-key read plans in both Spark connectors by deduplicating points, merging overlapping ranges, and removing points covered by scans. Honor exclusive scan starts and preserve finite empty upper bounds in Spark 3. ### Why are the changes needed? Predicates such as `key < 'm' OR key = 'a'` can return the same row through both a Scan and a Get. Repeated points and overlapping ranges can also produce duplicate results. ### How was this patch tested? ```bash export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk-17.0.2.jdk/Contents/Home mvn -B -ntp -Pspark4 -pl spark/hbase-spark,spark4/hbase-spark4 -am -DskipTests install mvn -B -ntp -pl spark/hbase-spark -Dtest=NoJavaTests -Dsurefire.failIfNoSpecifiedTests=false -Dsuites=org.apache.hadoop.hbase.spark.RowKeyFilterSuite,org.apache.hadoop.hbase.spark.DefaultSourceSuite,org.apache.hadoop.hbase.spark.PartitionFilterSuite -Dsurefire.rerunFailingTestsCount=0 test mvn -B -ntp -Pspark4 -pl spark4/hbase-spark4 -Dtest=NoJavaTests -Dsurefire.failIfNoSpecifiedTests=false -Dsuites=org.apache.hadoop.hbase.spark.datasources.RowKeyFilterSuite,org.apache.hadoop.hbase.spark.datasources.ScanRangeSuite,org.apache.hadoop.hbase.spark.datasources.HBaseScanBuilderSuite,org.apache.hadoop.hbase.spark.datasources.HBaseTableProviderSuite -Ds -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
