SteveYurongSu opened a new pull request, #15963: URL: https://github.com/apache/iotdb/pull/15963
This pull request refactors the `PipeHistoricalDataRegionTsFileExtractor` class to improve its handling of historical TsFile resources and introduces a new `ProgressReportEvent` for better progress tracking. The changes focus on separating filtered resources from the original list, ensuring accurate logging, and adding a mechanism to report progress when a resource is not part of the filtered set. ### Refactoring of resource handling: * Introduced a new `Set<TsFileResource>` named `filteredTsFileResources` to separate filtered TsFile resources from the original list, improving clarity and reducing potential errors. (`[[1]](diffhunk://#diff-cd3080430a195d28dd13a75aa4cbc39504bfdfbd13002e638e9244746d35df02R129)`, `[[2]](diffhunk://#diff-cd3080430a195d28dd13a75aa4cbc39504bfdfbd13002e638e9244746d35df02L400-R408)`, `[[3]](diffhunk://#diff-cd3080430a195d28dd13a75aa4cbc39504bfdfbd13002e638e9244746d35df02L424-R431)`) * Renamed `resourceList` to `originalResourceList` for better semantic clarity and updated its usage throughout the `start()` method. (`[[1]](diffhunk://#diff-cd3080430a195d28dd13a75aa4cbc39504bfdfbd13002e638e9244746d35df02L367-R370)`, `[[2]](diffhunk://#diff-cd3080430a195d28dd13a75aa4cbc39504bfdfbd13002e638e9244746d35df02L440-R450)`) ### Progress tracking: * Added a new `ProgressReportEvent` to track the progress of historical data extraction. This event is created and returned when a resource is not part of the `filteredTsFileResources` set. (`[iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/extractor/dataregion/historical/PipeHistoricalDataRegionTsFileExtractor.javaR575-R596](diffhunk://#diff-cd3080430a195d28dd13a75aa4cbc39504bfdfbd13002e638e9244746d35df02R575-R596)`) ### Logging improvements: * Updated log statements to reflect the separation of `filteredTsFileResources` and `originalResourceList`, ensuring accurate reporting of extracted file counts. (`[iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/extractor/dataregion/historical/PipeHistoricalDataRegionTsFileExtractor.javaL456-R461](diffhunk://#diff-cd3080430a195d28dd13a75aa4cbc39504bfdfbd13002e638e9244746d35df02L456-R461)`) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
