JackieTien97 opened a new pull request, #19344:
URL: https://github.com/apache/iotdb/pull/19344

   ## Description
   
   When a downstream DataNode stops responding, a sink can keep retrying 
`onNewDataBlockEvent` or `onEndOfDataBlockEvent` after the query times out. 
With a 60-second node RPC timeout and three attempts, a notification for a 
4-second query can occupy a shared exchange worker for about three minutes. 
Other queries then queue behind it to deliver their results.
   
   ### Changes
   
   - Carry the coordinator's deadline in an optional fragment dispatch field. A 
receiver uses the earlier of that deadline and its local fragment deadline. 
Requests from older coordinators use the local deadline.
   - Recompute the remaining time when a notification leaves the queue, borrows 
a connection, connects, waits for a response, or retries. An expired queued 
notification sends zero RPCs. Cancelling the fragment interrupts in-flight 
notifications.
   - Set the socket timeout on each deadline-aware checkout and restore the 
pool configuration when returning a reusable connection.
   
   ### Example
   
   A 4-second query that has already used 3.2 seconds has 800 ms left for 
result exchange. In a loopback test, waiting for the pool took 301 ms. The 
borrowed socket received a 491 ms read timeout. With a silent peer, the call 
returned a timeout 813 ms after it started waiting for the pool, including 
exception handling.
   
   ### Verification
   
   - [x] Self-reviewed the notification, cancellation, and connection reuse 
paths.
   - [x] Ran 55 targeted unit tests covering exchange, client pooling, and 
fragment dispatch.
   - [x] Ran `IoTDBMPPDataExchangeLargeTsBlock128KIT` on one ConfigNode and 
three DataNodes.
   - [x] Verified mutual TLS with both standard and Snappy framing.
   - [x] Ran a full `mvn clean package -DskipTests` in both the default and 
`with-zh-locale` profiles.
   
   ### Key changed classes
   
   `SinkChannel`, `MPPDataExchangeClientManager`, 
`SyncDataNodeMPPDataExchangeServiceClient`, and 
`FragmentInstanceDispatcherImpl`.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to