cxzl25 commented on code in PR #46506:
URL: https://github.com/apache/spark/pull/46506#discussion_r1596362947
##########
common/network-common/src/main/java/org/apache/spark/network/client/TransportClientFactory.java:
##########
@@ -169,8 +169,10 @@ public TransportClient createClient(String remoteHost, int
remotePort, boolean f
// this code was able to update things.
TransportChannelHandler handler = cachedClient.getChannel().pipeline()
.get(TransportChannelHandler.class);
- synchronized (handler) {
- handler.getResponseHandler().updateTimeOfLastRequest();
+ if (handler != null) {
Review Comment:
Thanks everyone for the quick comments!
I observed a small number of cases in the production environment.
This is a modified version based on Spark3.2, so the number of lines is a
bit inconsistent.
However, when I checked the number of lines of code and decompiled the jar,
they all pointed to `synchronized (handler )` this line of code.
<img width="949" alt="image"
src="https://github.com/apache/spark/assets/3898450/2fa41d1b-9c6b-4cda-ac9c-052235aaf454";>
---
The NPE issue here is somewhat similar to
[SPARK-11865](https://issues.apache.org/jira/browse/SPARK-11865).
When another thread is closing the channel, `isActive` may return true, and
may be false when checked for the second time. There is an intermediate state,
resulting in NPE.
I tried to reproduce this issue in the latest version to match the number of
rows, however it's a bit hard to do.
```java
if (cachedClient != null && cachedClient.isActive() // true) {
TransportChannelHandler handler = cachedClient.getChannel().pipeline()
.get(TransportChannelHandler.class);
synchronized (handler) {
handler.getResponseHandler().updateTimeOfLastRequest();
}
if (cachedClient.isActive() // false) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
