mazhengxuan created HBASE-30544:
-----------------------------------
Summary: HBaseContext releases connection references before lazy
mapPartitions iterators finish
Key: HBASE-30544
URL: https://issues.apache.org/jira/browse/HBASE-30544
Project: HBase
Issue Type: Sub-task
Components: hbase-connectors
Reporter: mazhengxuan
Assignee: mazhengxuan
h3. Impact
HBaseContext.mapPartitions releases its cached connection reference when the
callback returns an Iterator, although that iterator may still use the
connection.
Idle-cache cleanup can therefore close the connection while the task is still
consuming results, causing subsequent HBase operations to fail.
The same pattern exists in the Spark 3 and Spark 4 modules, including
delegating RDD/Java wrappers. Spark 3 streamMapPartitions also uses this
helper. Runtime reproduction was performed on Spark 4; Spark 3 was checked in
source only.
h3. Reproduction
Verified with Spark 4.0.0, Scala 2.13.14, and JDK 17, using a simulated cached
Connection:
# Set the internal cache test timeout to 10 ms.
# Run HBaseContext.mapPartitions on one partition, ensuring only the task's
connection reference remains.
# Return a lazy iterator. In next(), wait 100 ms, invoke normal cleanup with
performHousekeeping(false), and check connection.isClosed().
# Compare with an eager callback that performs the same wait and cleanup before
returning its iterator.
Observed:
{code}
eager_connection_closed=false
lazy_connection_closed=true
{code}
Expected: the connection remains open throughout iteration.
The reduced timeout only accelerates reproduction; the default cache close
delay is 10 minutes.
h3. Suggested fix
Retain the cached connection reference until task completion, using a
TaskCompletionListener.
Release it exactly once on success, failure, cancellation, or partial iterator
consumption. Coordinate this with the existing finally block to prevent double
release, and apply the fix to both connector modules.
Add regression tests covering lazy iteration across cache expiry and reference
cleanup after task completion.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)