Caideyipi opened a new pull request, #18383: URL: https://github.com/apache/iotdb/pull/18383
## Description Backport of #18331 to dev/1.3. ### Problem IoTDBNonDataRegionSource assumed every non-minimum task progress was a MetaProgressIndex. Pipe task progress can also be merged into a HybridProgressIndex, and may be wrapped by a StateProgressIndex. This caused a ClassCastException while restoring a non-data-region source or calculating its remaining events during DataNode shutdown. ### Changes - Extract the requested progress-index component from direct, hybrid, and state-wrapped progress indexes. - Use the meta component for non-data-region source recovery, remaining-event accounting, and listening-queue cleanup. - Preserve aggregate and two-stage processor state when progress indexes are hybrid. - Handle time-window state nested in hybrid indexes when selecting historical TsFiles. - Conservatively fall back to snapshot recovery or full queue size when no meta component exists. - Add unit coverage for hybrid, state-wrapped hybrid, and hybrid-without-meta cases. ### Backport adaptations - Apply the historical-source fix to PipeHistoricalDataRegionTsFileSource, which is the implementation used by dev/1.3. - Keep the implementation compatible with the Java 8 source level and the logging structure on dev/1.3. ### Verification - mvn spotless:apply -pl iotdb-core/node-commons,iotdb-core/datanode,iotdb-core/confignode - mvn test -pl iotdb-core/node-commons -Dtest=ProgressIndexTest,IoTDBNonDataRegionSourceTest - mvn test -pl iotdb-core/datanode -Dtest=TwoStageCountProcessorTest,TsFileResourceProgressIndexTest - mvn test-compile -pl iotdb-core/confignode -DskipTests <hr> This PR has: - [x] been self-reviewed. - [x] added unit tests to cover new code paths. <hr> ##### Key changed/added classes - ProgressIndex and its implementations - IoTDBNonDataRegionSource - PipeHistoricalDataRegionTsFileSource - AggregateProcessor - TwoStageCountProcessor -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
