Daniel Roudnitsky created HBASE-29678:
-----------------------------------------
Summary: Filtering with BigDecimalComparator can fail
non-gracefully with NullPointerException
Key: HBASE-29678
URL: https://issues.apache.org/jira/browse/HBASE-29678
Project: HBase
Issue Type: Bug
Components: Filters
Reporter: Daniel Roudnitsky
Assignee: Daniel Roudnitsky
If a scan with a filter using BigDecimalComparator encounters a row/column/byte
array which was not encoded properly using Bytes.toBytes(BigDecimal) and is too
short to be a BigDecimal, the scan RPC will fail with a NullPointerException,
and the client will retry excessively until exhausting all retries, and the
operation will fail with RetriesExhausted and a trace that looks like:
{code:java}
RetriesExhausted Failed after attempts=13, exceptions:
2025-10-20T19:31:23.525Z, java.io.IOException: java.io.IOException: Cannot read
field "scale" because "val" is null
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:525)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:102)
at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82)
Caused by: java.lang.NullPointerException: Cannot read field "scale" because
"val" is null
at java.base/java.math.BigDecimal.compareTo(BigDecimal.java:3124)
at
org.apache.hadoop.hbase.filter.BigDecimalComparator.compareTo(BigDecimalComparator.java:72)
at
org.apache.hadoop.hbase.PrivateCellUtil.compareValue(PrivateCellUtil.java:1341)
at
org.apache.hadoop.hbase.filter.ColumnValueFilter.compareValue(ColumnValueFilter.java:128)
at
org.apache.hadoop.hbase.filter.ColumnValueFilter.filterCell(ColumnValueFilter.java:111)
at
org.apache.hadoop.hbase.regionserver.querymatcher.UserScanQueryMatcher.matchColumn(UserScanQueryMatcher.java:166)
at
org.apache.hadoop.hbase.regionserver.querymatcher.NormalUserScanQueryMatcher.match(NormalUserScanQueryMatcher.java:80)
at
org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:653)
at
org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:153)
at
org.apache.hadoop.hbase.regionserver.RegionScannerImpl.populateResult(RegionScannerImpl.java:348)
at
org.apache.hadoop.hbase.regionserver.RegionScannerImpl.nextInternal(RegionScannerImpl.java:520)
at
org.apache.hadoop.hbase.regionserver.RegionScannerImpl.nextRaw(RegionScannerImpl.java:285)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3430)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3703)
at
org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:45423)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:458)
... 3 more
2025-10-20T19:31:23.645Z, java.io.IOException: java.io.IOException: Cannot read
field "scale" because "val" is null
... {code}
+Root cause+
BigDecimalComparator#compareTo relies on ByteBufferUtils.toBigDecimal to
convert the input byte array to BigDecimal, and ByteBufferUtils.toBigDecimal
will return null if the byte array is too short. We don’t do a null check
before doing the comparison , which leads to a NullPointerException that
bubbles up the region server call stack which the client will treat as
retryable exception
+Proposed solution+
Add a null check before doing the comparison in BigDecimalComparator - if we
have null then the input byte array conversion to BigDecimal failed and we
should throw an exception that explains the cause clearly - input byte array is
too short to be a BigDecimal - and make sure the exception is handled as a
DoNotRetryIOException in the filter layer.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
