Sigma-Ma commented on code in PR #172:
URL: https://github.com/apache/hbase-connectors/pull/172#discussion_r4450610060
##########
spark4/hbase-spark4/src/main/scala/org/apache/hadoop/hbase/spark/datasources/ScanRange.scala:
##########
@@ -145,6 +145,31 @@ class RowKeyFilter(
if (currentRange != null) ranges += currentRange
if (currentPoint != null) points += currentPoint
+ // Normalize after the complete filter tree is built, before region pruning
and Get batching.
+ def normalize(): RowKeyFilter = {
+ val normalizedRanges = new ListBuffer[ScanRange]()
+ ranges
+ .sortWith((left, right) => Bytes.compareTo(left.lowerBound,
right.lowerBound) < 0)
Review Comment:
I checked the HBase implementation used here: Bytes.compareTo treats null as
a zero-length array, so this comparator does not throw for null inputs. The
built-in planner also uses an empty array for an unbounded lower bound,
including full scans. I don't think an additional null-handling comparator is
needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
