zhangshenghang opened a new pull request, #12718:
URL: https://github.com/apache/seatunnel/pull/12718
### Purpose of this pull request
This PR fixes #12715.
`DorisSourceReader.pollNext()` replaced its single `valueReader` field for
every split **without closing the previous one**:
```java
valueReader = new DorisValueReader(partition, dorisSourceConfig,
dorisSourceTable); // previous reader silently dropped
while (valueReader.hasNext()) { ... }
```
Resource-release chain in the connector:
- `DorisValueReader.close()` is the only place that sends the `closeScanner`
Thrift RPC, which frees the scan context on the Doris BE.
- `BackendClient.close()`, the only place that closes the Thrift `TSocket`,
is `private` and only invoked at the end of `BackendClient.closeScanner()`.
- So a `DorisValueReader` that is never `close()`d leaks **both** the
BE-side scanner context (memory/tablet readers until BE timeout) and the
client-side TCP connection.
`DorisSourceReader.close()` runs only once at reader shutdown and therefore
only released the **last** split's reader. Any job whose table is split into N
`PartitionDefinition` splits (the normal case: partitioned/sharded tables,
multi-table sync; the split enumerator hands the whole split list to each
reader) leaked (N-1) scanners and sockets per reader task.
The fix closes the previous reader before starting the next split, so every
finished split releases its resources.
### Does this PR introduce _any_ user-facing change?
Yes, bug fix: Doris source jobs no longer accumulate one idle Thrift
connection plus one BE scan context per processed split; only the current
split's resources are held.
### How was this patch tested?
- `mvn -pl seatunnel-connectors-v2/connector-doris test` passes (138 tests).
- No reader-level unit test was added because `DorisValueReader` opens a
real Thrift `BackendClient` in its constructor and the connector has no
existing test seam for the source reader; the change is a straight-line
resource-management fix (close the previous reader before creating the next
one, shared by `pollNext()` and `close()`), reviewed against the release chain
above.
### Check list
* [ ] If any new Jar binary package adding in your PR, please add License
Notice according
[New License
Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md)
* [ ] If necessary, please update the documentation to describe the new
feature. https://github.com/apache/seatunnel/tree/dev/docs
* [ ] If necessary, please update `incompatible-changes.md` to describe the
incompatibility caused by this PR.
* [x] If you are contributing the connector code, please check that the
following files are updated:
1. No new connector is added, so `plugin-mapping.properties`,
`seatunnel-dist/pom.xml`, label-scope-conf and e2e updates do not apply
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
