Caideyipi opened a new pull request, #18704:
URL: https://github.com/apache/iotdb/pull/18704

   ## Description
   
   ### Problem
   
   Closing a subscription consumer can deadlock with a consumer-group metadata 
push on a DataNode:
   
   1. The metadata push holds the consumer-group metadata write lock while it 
closes consensus prefetch queues and waits for in-flight prefetch work to 
finish.
   2. Event materialization in the prefetch path calls 
`SubscriptionAgent.consumer().isTableModel(consumerGroupId)`, which tries to 
acquire the same metadata read lock.
   3. The metadata push and prefetch task then wait for each other. The 
ConfigNode `DropSubscriptionProcedure` times out and remains stuck, while the 
client blocks waiting for the close response.
   
   This was identified from the stable subscription regression represented by 
internal test case 1379/result 27213.
   
   ### Fix
   
   Derive the data model from the converter's immutable tree/table conversion 
pattern and cache it when `ConsensusPrefetchingQueue` is constructed. Event 
materialization now uses the cached value when obtaining the column filter 
matcher, so it no longer reads consumer-group metadata while queue shutdown is 
waiting for prefetch completion.
   
   This preserves synchronous queue shutdown and the existing subscription 
procedure consistency semantics while removing the lock cycle.
   
   ### Tests
   
   Added unit coverage for:
   
   - deriving table/tree model from the immutable converter pattern;
   - materializing a prefetched event while the real consumer metadata write 
lock is held, proving that prefetch no longer depends on the metadata read lock.
   
   Validated with:
   
   ```text
   mvn test -pl iotdb-core/datanode -am \
     
-Dtest=ConsensusLogToTabletConverterTest#testDataModelIsDerivedFromImmutableConversionPattern,ConsensusPrefetchingQueueTest#testPrefetchDoesNotDependOnConsumerMetaReadLock
 \
     -Dsurefire.failIfNoSpecifiedTests=false \
     -DfailIfNoTests=false
   ```
   
   Result: 2 tests run, 0 failures, 0 errors; full reactor build successful.
   
   <hr>
   
   This PR has:
   - [x] been self-reviewed.
       - [x] concurrent read and write
   - [x] added comments explaining the why and the intent of the code wherever 
it would not be obvious for an unfamiliar reader.
   - [x] added unit tests to cover the new code path.
   
   <hr>
   
   ##### Key changed/added classes
   
   - `ConsensusLogToTabletConverter`
   - `ConsensusPrefetchingQueue`
   - `ConsensusLogToTabletConverterTest`
   - `ConsensusPrefetchingQueueTest`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to