This is an automated email from the ASF dual-hosted git repository.
tanxinyu pushed a change to branch wal_compression
in repository https://gitbox.apache.org/repos/asf/iotdb.git
from 94f074f61e2 fix bug
add 5d552729ae8 Optimize node startup process & Normalize
system.properties management (#12752)
add b18412d62d5 [IOTDB-6340] Clear Coordinator.queryExecutionMap while
encountering MemoryNotEnough Exception
add 3d436b93802 [IOTDB-6342] Clear
FragmentInstanceManager.dataNodeQueryContextMap while encountering
MemoryNotEnough Exception (#12762)
add 612ecd4f75e Refactor node registration check (#12766)
add f22e3791694 Fix overlap between tsfile is not correctly marked (#12748)
add 6cbdf31bf6f Subscription: support topic loose range for path and time
(#12760)
add 36dd4890577 Add clean logic for FragmentInstance in case that callback
is not added. (#12768)
add 4e50e847378 SystemPropertiesHandler use Files::move instead of
File::renameTo (#12770)
add f196cf20900 Pipe: Filter devices by pattern before reading device
metadata from TsFile (#12765)
add 48028a15f3f Pipe: Adjusted some loggers of metrics to avoid
unnecessary warns & Include "lastEvent" into pipe's event count metrics &
Removed the "userConflict" judgment to data sync failure caused by
METADATA_ERROR (#12758)
add 881816265f6 Synchronize takeSnapshotAsync on RaftGroupID (#12767)
add 60b12d87351 SystemPropertiesHandler close reader and writer #12772
add 31cd61f771d Pipe Consensus: Fix events in connector are not in order
because of parameter `'realtime-first' = 'true'` (#12773)
add df68083d95a Pipe: Avoid event being continuously rate-limited even its
pipe is dropped (#12753)
add fe52d5de3e6 Load: Add some load metrics of time cost, write point and
disk throughput (#12735)
add 9c3d3ea8036 Update tsfile version (#12774)
add e58d427f2f6 Load: Avoid potential NPE in warn log when
LoadRuntimeOutOfMemoryException occurs & load-tsfile: Polish logs (#12777)
add 9e0b62181d0 Support editing cluster name (#12776)
add 500d4a911dd remove wal files when drop database (#12778)
add 4900407d8b1 load-tsfile: Add retry logic for load memory controls
(#12782)
add 3dd6c8c3213 update tsfile version
add bce9631e108 Add memory control for MergeReader
add 7ee30fb631a Classification properties (#12764)
add b1e9d96f49e bind cluster name to metrics (#12786)
add 0b8e31456a4 Load: change the metric type of LoadDiskIO metric from
RATE to COUNTER (#12787)
add a96349e298c Enhance leader quantity metric for load scenario (#12785)
add cbdade41d5a Use milliseconds to express TTL (#12757)
add a3f996cbbc9 Fix iotdb-system.properties cluster_name comments #12791
add bc4cf473b77 Try to fix set cluster name integration test (#12794)
add 07792b11254 Fix WAL resource leak when using IoT series consensus
#12796
add 79ea1ef23d2 Pipe: Enable `'sink.format'='tsfile'` to use tsFile as
tablet event batch (#12737)
add 4912d902a8b [IOTDB-6343] Fix the device path construction bug in
visitSingleDeviceViewNode of AggragationPushDown
add 1020258bdf8 PipeConsensus: fix concurrent problems regarding to
initiate receiver and progressIndex & optimize receiver logs & fix npe (#12790)
add 0afc1bea507 Pipe: Fix remaining time metrics is not dropped after
dropPipe(pipeName) & alter pipe does not take effect in non-data regions
(#12795)
add 86aa71177f1 Pipe: Added collect invocation count in the caculation of
data region extractor events for data node remaining time (#12799)
add 7f34bc03129 Specify error msg when DriverTask is aborted by
MemoryNotEnoughException
add 5c23a355579 Add flush cause metric (#12803)
add ada45de7d37 use template file start 1c1d (#12808)
add 3fa0320d81d Pipe/PipeConsensus: support progressIndex for
InsertRowsNode (#12806)
add 7106bca5020 Modify ttl (#12811)
add 691e1159fa1 Allow using python 3.6 and enable CI (#12792)
add c1df2bf8a65 Pipe: Avoid writing out-of-order data in tsfile when
`'sink.format' = 'file'` (#12810)
add 389ae8002e2 Fix region migration IT & Add daily IT (#12804)
add 28c970b1107 [IOTDB-6344] Display null in cli
add 282147773bd Pipe: Reduced the logs in ConfigNode caused by pipeMeta
sync and heartbeat report (#12813)
add 32bfeaf60cf Pipe/Load: Fix loaded files may be compacted before
listening (#12816)
add 3d1d28141b8 Pipe: Enable 'sink.format'='tablet' to force tsfiles to be
parsed into tablets (#12809)
add 4b805661eab bump ratis version to 3.1.0 (#12807)
add 9b03eb673e8 PipeConsensus: enhance receiver when the leader reboots
(#12815)
add 2e1ebf46e93 Pipe: Intoduce TsFileInsertionScanDataContainer to read
data from tsfile sequentially to improve pattern parse performance when filter
rate is high (#12781)
add c4828465214 Pipe: Fixed the bug that non-forwarding flag does not take
effect on insertRowsNode / insertRowsOfOneDevicesNode (#12824)
add 85dfa900e38 Pipe: Fix PipeTerminateEvent cannot report and mark the
pipe as completed & events with no generated events in pipe processor subtask
may not be reported (#12817)
add 1b992f29004 Pipe: Fixed some problems cause by reusing connectors
after exceptions occurred in connector stage (#12685)
add a7e0aa4f462 Merge remote-tracking branch 'origin/master' into
wal_compression_back
add a460acb28f0 Hot parameters recover (#12814)
add 747b7dc99d9 Merge remote-tracking branch 'origin/master' into
wal_compression_back
No new revisions were added by this update.
Summary of changes:
.../{cluster-it-1c3d.yml => daily-it.yml} | 33 +-
.github/workflows/multi-language-client.yml | 15 +-
distribution/src/assembly/all.xml | 9 -
distribution/src/assembly/confignode.xml | 9 -
distribution/src/assembly/datanode.xml | 9 -
integration-test/pom.xml | 14 +
.../iotdb/it/env/cluster/ClusterConstant.java | 3 -
.../iotdb/it/env/cluster/env/AbstractEnv.java | 13 +-
.../it/env/cluster/node/ConfigNodeWrapper.java | 4 +-
.../iotdb/it/env/cluster/node/DataNodeWrapper.java | 4 +-
.../iotdb/it/env/remote/env/RemoteServerEnv.java | 6 +
.../category/{RemoteIT.java => DailyIT.java} | 3 +-
.../java/org/apache/iotdb/itbase/env/BaseEnv.java | 3 +
.../apache/iotdb/cli/it/StartClientScriptIT.java | 65 ++
.../IoTDBRegionMigrateReliabilityITFramework.java | 55 +-
.../pass/IoTDBRegionMigrateClusterCrashIT.java | 3 +
.../pass/IoTDBRegionMigrateConfigNodeCrashIT.java | 3 +
.../pass/IoTDBRegionMigrateNormalIT.java | 3 +
...rateCoordinatorCrashWhenRemoveRemotePeerIT.java | 3 +
.../IoTDBRegionMigrateDataNodeCrashIT.java | 6 +
...nMigrateOriginalCrashWhenDeleteLocalPeerIT.java | 3 +
...MigrateOriginalCrashWhenRemoveRemotePeerIT.java | 3 +
.../org/apache/iotdb/db/it/IoTDBRecoverIT.java | 2 +-
.../org/apache/iotdb/db/it/IoTDBRestServiceIT.java | 15 +-
.../iotdb/db/it/IoTDBSetConfigurationIT.java | 90 +-
...oTDBAlignByDeviceWithTemplateAggregationIT.java | 29 +
.../iotdb/db/it/auth/IoTDBSeriesPermissionIT.java | 2 +-
.../org/apache/iotdb/db/it/utils/TestUtils.java | 36 +
.../pipe/it/autocreate/IoTDBPipeAutoDropIT.java | 6 +
.../pipe/it/autocreate/IoTDBPipeDataSinkIT.java | 100 +-
.../pipe/it/autocreate/IoTDBPipeExtractorIT.java | 17 +-
.../pipe/it/manual/IoTDBPipeMetaHistoricalIT.java | 6 +-
.../it/IoTDBSubscriptionITConstant.java | 2 +-
.../it/dual/IoTDBSubscriptionTopicIT.java | 135 +++
.../java/org/apache/iotdb/cli/AbstractCli.java | 27 +-
.../java/org/apache/iotdb/tool/ImportTsFile.java | 31 +-
.../client-py/iotdb/utils/IoTDBRpcDataSet.py | 2 +-
iotdb-client/client-py/requirements.txt | 4 +-
iotdb-client/client-py/requirements_dev.txt | 8 +-
iotdb-client/client-py/resources/setup.py | 6 +-
iotdb-client/client-py/tests/test_todf.py | 6 +-
.../org/apache/iotdb/jdbc/IoTDBJDBCResultSet.java | 23 +-
.../iotdb/rpc/subscription/config/TopicConfig.java | 34 +-
.../rpc/subscription/config/TopicConstant.java | 6 +
.../org/apache/iotdb/db/qp/sql/IoTDBSqlParser.g4 | 2 +-
.../handlers/ConfigNodeAsyncRequestContext.java | 6 +
.../iotdb/confignode/conf/ConfigNodeConfig.java | 12 -
.../confignode/conf/ConfigNodeDescriptor.java | 12 +-
.../confignode/conf/ConfigNodeStartupCheck.java | 3 +-
.../ConfigNodeSystemPropertiesHandler.java} | 44 +-
.../confignode/conf/SystemPropertiesUtils.java | 163 +--
.../response/datanode/DataNodeRegisterResp.java | 6 +-
.../iotdb/confignode/manager/ConfigManager.java | 26 +-
.../apache/iotdb/confignode/manager/IManager.java | 2 +
.../iotdb/confignode/manager/TTLManager.java | 12 -
.../manager/node/ClusterNodeStartUtils.java | 133 ++-
.../iotdb/confignode/manager/node/NodeManager.java | 26 +-
.../pipe/agent/task/PipeConfigNodeTaskAgent.java | 37 +-
.../protocol/IoTDBConfigRegionAirGapConnector.java | 7 +-
.../protocol/IoTDBConfigRegionConnector.java | 16 +-
.../runtime/heartbeat/PipeHeartbeat.java | 4 +
.../runtime/heartbeat/PipeHeartbeatParser.java | 39 +-
.../pipe/event/PipeConfigRegionSnapshotEvent.java | 9 +-
.../pipe/execution/PipeConfigNodeSubtask.java | 2 +
.../metric/PipeConfigNodeRemainingTimeMetrics.java | 10 +-
.../metric/PipeConfigRegionConnectorMetrics.java | 2 +-
.../resource/PipeConfigNodeResourceManager.java | 51 +
.../PipeConfigNodeSnapshotResourceManager.java | 11 +-
.../manager/schema/ClusterSchemaManager.java | 5 -
.../confignode/persistence/pipe/PipeTaskInfo.java | 16 +-
.../procedure/env/RegionMaintainHandler.java | 17 +-
.../impl/pipe/task/AlterPipeProcedureV2.java | 44 +-
.../thrift/ConfigNodeRPCServiceProcessor.java | 5 +
.../iotdb/confignode/persistence/NodeInfoTest.java | 3 +-
.../org/apache/iotdb/consensus/IConsensus.java | 8 +
.../apache/iotdb/consensus/iot/IoTConsensus.java | 6 +
.../apache/iotdb/consensus/pipe/PipeConsensus.java | 6 +
.../pipe/consensuspipe/ConsensusPipeManager.java | 2 +
.../iotdb/consensus/ratis/RatisConsensus.java | 32 +-
.../iotdb/consensus/simple/SimpleConsensus.java | 5 +
.../apache/iotdb/consensus/iot/StabilityTest.java | 4 +
.../iotdb/consensus/ratis/RatisConsensusTest.java | 7 +
.../consensus/simple/SimpleConsensusTest.java | 6 +-
iotdb-core/datanode/pom.xml | 4 +-
.../DataNodeSystemPropertiesHandler.java} | 40 +-
.../java/org/apache/iotdb/db/conf/IoTDBConfig.java | 14 +-
.../org/apache/iotdb/db/conf/IoTDBDescriptor.java | 292 +++---
.../org/apache/iotdb/db/conf/IoTDBStartCheck.java | 181 +---
.../db/consensus/DataRegionConsensusImpl.java | 9 +-
.../schemaregion/SchemaExecutionVisitor.java | 4 +-
.../schemaregion/SchemaRegionStateMachine.java | 16 +-
.../{PipeAgent.java => PipeDataNodeAgent.java} | 20 +-
.../db/pipe/agent/runtime/PipeAgentLauncher.java | 6 +-
.../agent/runtime/PipeDataNodeRuntimeAgent.java | 10 +-
.../db/pipe/agent/task/PipeDataNodeTaskAgent.java | 33 +-
.../IoTDBDataNodeCacheLeaderClientManager.java | 27 +-
.../evolvable/batch/PipeTabletEventBatch.java | 133 +++
.../PipeTabletEventPlainBatch.java} | 133 +--
.../batch/PipeTabletEventTsFileBatch.java | 424 ++++++++
.../PipeTransferBatchReqBuilder.java | 96 +-
.../request/PipeTransferTabletRawReq.java | 133 +--
.../airgap/IoTDBDataNodeAirGapConnector.java | 1 +
.../airgap/IoTDBDataRegionAirGapConnector.java | 19 +-
.../airgap/IoTDBSchemaRegionAirGapConnector.java | 6 +-
.../protocol/legacy/IoTDBLegacyPipeConnector.java | 14 +-
.../pipeconsensus/PipeConsensusAsyncConnector.java | 16 +-
.../PipeConsensusTransferBatchReqBuilder.java | 12 +-
.../async/IoTDBDataRegionAsyncConnector.java | 238 +++--
.../PipeTransferTabletBatchEventHandler.java | 40 +-
.../PipeTransferTabletInsertionEventHandler.java | 22 +-
...Handler.java => PipeTransferTsFileHandler.java} | 130 ++-
.../thrift/sync/IoTDBDataNodeSyncConnector.java | 1 +
.../thrift/sync/IoTDBDataRegionSyncConnector.java | 135 ++-
.../thrift/sync/IoTDBSchemaRegionConnector.java | 15 +-
.../protocol/writeback/WriteBackConnector.java | 7 +-
.../pipe/connector/util/PipeTabletEventSorter.java | 202 ++++
.../consensus/ConsensusPipeDataNodeDispatcher.java | 4 +-
.../ConsensusPipeDataNodeRuntimeAgentGuardian.java | 4 +-
.../consensus/ProgressIndexDataNodeManager.java | 24 +-
.../schema/PipeSchemaRegionSnapshotEvent.java | 4 +-
.../tablet/PipeInsertNodeTabletInsertionEvent.java | 24 +-
.../common/tablet/PipeRawTabletInsertionEvent.java | 4 +-
.../event/common/terminate/PipeTerminateEvent.java | 4 +-
.../common/tsfile/PipeTsFileInsertionEvent.java | 24 +-
.../container/TsFileInsertionDataContainer.java | 78 ++
.../TsFileInsertionDataContainerProvider.java | 123 +++
.../query/TsFileInsertionQueryDataContainer.java} | 179 ++--
.../TsFileInsertionQueryDataTabletIterator.java} | 22 +-
.../scan/AlignedSinglePageWholeChunkReader.java | 170 ++++
.../container/scan/SinglePageWholeChunkReader.java | 116 +++
.../scan/TsFileInsertionScanDataContainer.java | 388 ++++++++
.../PipeHistoricalDataRegionTsFileExtractor.java | 14 +-
.../realtime/PipeRealtimeDataRegionExtractor.java | 8 +-
.../PipeRealtimeDataRegionHybridExtractor.java | 14 +-
.../PipeRealtimeDataRegionLogExtractor.java | 11 +-
.../PipeRealtimeDataRegionTsFileExtractor.java | 8 +-
.../realtime/assigner/DisruptorQueue.java | 12 +-
.../listener/PipeInsertionDataNodeListener.java | 4 +-
.../schemaregion/IoTDBSchemaRegionExtractor.java | 13 +-
.../PipeDataNodeRemainingEventAndTimeMetrics.java | 25 +
.../PipeDataNodeRemainingEventAndTimeOperator.java | 41 +-
.../metric/PipeDataRegionConnectorMetrics.java | 4 +-
.../metric/PipeDataRegionExtractorMetrics.java | 8 +-
.../iotdb/db/pipe/metric/PipeProcessorMetrics.java | 48 +-
.../iotdb/db/pipe/metric/PipeResourceMetrics.java | 14 +-
.../metric/PipeSchemaRegionConnectorMetrics.java | 2 +-
.../processor/aggregate/AggregateProcessor.java | 4 +-
.../downsampling/PartialPathLastObjectCache.java | 20 +-
.../twostage/combiner/PipeCombineHandler.java | 4 +-
.../combiner/PipeCombineHandlerManager.java | 4 +-
.../protocol/airgap/IoTDBAirGapReceiver.java | 6 +-
.../pipeconsensus/PipeConsensusReceiver.java | 82 +-
.../pipeconsensus/PipeConsensusReceiverAgent.java | 53 +-
.../protocol/thrift/IoTDBDataNodeReceiver.java | 4 +-
.../visitor/PipeStatementTSStatusVisitor.java | 12 +-
...nager.java => PipeDataNodeResourceManager.java} | 8 +-
.../db/pipe/resource/memory/PipeMemoryBlock.java | 18 +-
.../db/pipe/resource/memory/PipeMemoryManager.java | 67 +-
...oryWeighUtil.java => PipeMemoryWeightUtil.java} | 63 +-
.../pipe/resource/tsfile/PipeTsFileResource.java | 102 +-
.../resource/tsfile/PipeTsFileResourceManager.java | 14 +-
.../pipe/resource/wal/PipeWALResourceManager.java | 38 +-
.../hardlink/PipeWALHardlinkResourceManager.java | 23 +-
.../selfhost/PipeWALSelfHostResourceManager.java | 8 +-
.../pipe/task/builder/PipeDataNodeTaskBuilder.java | 23 +-
.../pipe/task/connection/PipeEventCollector.java | 33 +-
.../db/pipe/task/stage/PipeTaskExtractorStage.java | 6 +-
.../db/pipe/task/stage/PipeTaskProcessorStage.java | 26 +-
.../subtask/connector/PipeConnectorSubtask.java | 84 +-
.../connector/PipeConnectorSubtaskManager.java | 8 +-
.../PipeRealtimePriorityBlockingQueue.java | 11 +-
.../subtask/processor/PipeProcessorSubtask.java | 42 +-
.../processor/PipeProcessorSubtaskWorker.java | 21 +-
.../iotdb/db/protocol/client/ConfigNodeClient.java | 9 +-
.../iotdb/db/protocol/client/ConfigNodeInfo.java | 66 +-
.../protocol/thrift/impl/ClientRPCServiceImpl.java | 27 +-
.../impl/DataNodeInternalRPCServiceImpl.java | 24 +-
.../thrift/impl/DataNodeRegionManager.java | 4 +-
.../db/queryengine/common/MPPQueryContext.java | 51 +-
.../common/header/ColumnHeaderConstant.java | 4 +-
.../execution/exchange/MPPDataExchangeManager.java | 15 +-
.../fragment/FragmentInstanceContext.java | 31 +-
.../fragment/FragmentInstanceExecution.java | 4 +-
.../fragment/FragmentInstanceManager.java | 10 +
.../execution/load/LoadTsFileManager.java | 40 +-
.../execution/load/LoadTsFileRateLimiter.java | 45 +-
.../queryengine/execution/memory/MemoryPool.java | 9 +-
.../operator/schema/source/DeviceSchemaSource.java | 2 +-
.../AbstractSeriesAggregationScanOperator.java | 2 +-
.../operator/source/AlignedSeriesScanOperator.java | 4 +-
.../operator/source/SeriesScanOperator.java | 3 +-
.../execution/operator/source/SeriesScanUtil.java | 3 +-
.../execution/schedule/AbstractDriverThread.java | 12 +-
.../schedule/DriverTaskAbortedException.java | 3 +
.../load/LoadTsFileAnalyzeSchemaMemoryBlock.java | 2 +-
.../metric/load/LoadTsFileCostMetricsSet.java | 131 +++
.../metric/{ => load}/LoadTsFileMemMetricSet.java | 2 +-
.../iotdb/db/queryengine/plan/Coordinator.java | 2 +-
.../queryengine/plan/analyze/AnalyzeVisitor.java | 8 +
.../plan/analyze/LoadTsfileAnalyzer.java | 4 +-
.../analyze/cache/schema/DataNodeTTLCache.java | 31 +-
.../queryengine/plan/execution/QueryExecution.java | 3 +-
.../config/executor/ClusterConfigTaskExecutor.java | 17 +-
.../plan/optimization/AggregationPushDown.java | 33 +-
.../plan/planner/LocalExecutionPlanner.java | 13 +-
...stimator.java => MemoryReservationManager.java} | 34 +-
.../NotThreadSafeMemoryReservationManager.java | 94 ++
...ava => ThreadSafeMemoryReservationManager.java} | 33 +-
.../plan/planner/plan/node/PlanGraphPrinter.java | 5 +
.../planner/plan/node/write/InsertRowsNode.java | 11 +
.../scheduler/load/LoadTsFileDispatcherImpl.java | 6 +-
.../plan/scheduler/load/LoadTsFileScheduler.java | 68 +-
.../statement/metadata/ShowDatabaseStatement.java | 37 +-
.../java/org/apache/iotdb/db/service/DataNode.java | 63 +-
.../db/service/metrics/DataNodeMetricsHelper.java | 6 +-
.../iotdb/db/service/metrics/WritingMetrics.java | 186 ++--
.../iotdb/db/storageengine/StorageEngine.java | 6 -
.../db/storageengine/dataregion/DataRegion.java | 80 +-
.../CompactionValidationFailedException.java | 14 +-
.../execute/task/AbstractCompactionTask.java | 17 +-
.../writer/RepairUnsortedFileCompactionWriter.java | 18 +-
.../dataregion/memtable/AbstractMemTable.java | 32 +-
.../dataregion/memtable/TsFileProcessor.java | 11 +-
.../read/reader/common/PriorityMergeReader.java | 34 +-
.../storageengine/dataregion/wal/WALManager.java | 7 +-
.../storageengine/dataregion/wal/node/WALNode.java | 2 +-
.../file/AbstractTsFileRecoverPerformer.java | 4 +-
.../file/UnsealedTsFileRecoverPerformer.java | 4 +-
.../dataregion/wal/utils/WALInsertNodeCache.java | 47 +-
.../broker/SubscriptionPrefetchingQueue.java | 6 +-
.../SubscriptionPrefetchingTabletsQueue.java | 6 +-
.../event/SubscriptionEventBinaryCache.java | 16 +-
.../SubscriptionConnectorSubtaskManager.java | 4 +-
.../pipe/connector/PipeTabletEventSorterTest.java | 217 ++++
.../event/TsFileInsertionDataContainerTest.java | 232 +++--
.../extractor/SchemaRegionListeningQueueTest.java | 22 +-
.../PipeDataNodeSnapshotResourceManagerTest.java | 20 +-
.../resource/PipeTsFileResourceManagerTest.java | 8 +-
.../execution/operator/OperatorMemoryTest.java | 19 +-
.../plan/planner/node/PlanGraphPrinterTest.java | 2 +-
.../repair/RepairUnsortedFileCompactionTest.java | 25 +
.../iotdb/metrics/core/type/IoTDBCounter.java | 4 +-
.../apache/iotdb/metrics/config/MetricConfig.java | 8 +-
iotdb-core/node-commons/pom.xml | 73 +-
.../resources/conf/generate_properties.bat | 65 --
.../assembly/resources/conf/generate_properties.sh | 59 --
...properties => iotdb-system.properties.template} | 1051 ++++++++++----------
.../apache/iotdb/commons/conf/CommonConfig.java | 20 +
.../iotdb/commons/conf/CommonDescriptor.java | 21 +-
.../iotdb/commons/conf/ConfigurationFileUtils.java | 124 ++-
.../apache/iotdb/commons/conf/IoTDBConstant.java | 1 -
.../consensus/index/impl/RecoverProgressIndex.java | 5 +-
.../iotdb/commons/file/SystemFileFactory.java | 4 -
.../commons/file/SystemPropertiesHandler.java | 200 ++++
.../commons/pipe/agent/task/PipeTaskAgent.java | 9 +
.../iotdb/commons/pipe/config/PipeConfig.java | 12 +
.../config/constant/PipeConnectorConstant.java | 12 +-
.../pipe/connector/limiter/GlobalRateLimiter.java | 43 +-
.../connector/limiter/PipeEndPointRateLimiter.java | 39 +-
.../connector/protocol/IoTDBAirGapConnector.java | 38 +-
.../pipe/connector/protocol/IoTDBConnector.java | 55 +-
.../connector/protocol/IoTDBSslSyncConnector.java | 18 +-
.../queue/ConcurrentIterableLinkedQueue.java | 27 +-
.../iotdb/commons/pipe/event/EnrichedEvent.java | 2 +-
.../commons/pipe/pattern/IoTDBPipePattern.java | 4 +
.../commons}/pipe/resource/log/PipeLogManager.java | 7 +-
.../commons}/pipe/resource/log/PipeLogStatus.java | 6 +-
.../pipe/task/connection/BlockingPendingQueue.java | 22 +-
.../commons/pipe/task/meta/PipeTemporaryMeta.java | 8 +-
.../task/subtask/PipeAbstractConnectorSubtask.java | 56 +-
.../apache/iotdb/commons/schema/ttl/TTLCache.java | 18 +-
.../iotdb/commons/service/metric/enums/Metric.java | 14 +-
.../commons/subscription/meta/topic/TopicMeta.java | 2 +
.../iotdb/commons/utils/CommonDateTimeUtils.java | 10 +
.../src/main/thrift/confignode.thrift | 6 +-
pom.xml | 4 +-
276 files changed, 7013 insertions(+), 3312 deletions(-)
copy .github/workflows/{cluster-it-1c3d.yml => daily-it.yml} (73%)
copy
integration-test/src/main/java/org/apache/iotdb/itbase/category/{RemoteIT.java
=> DailyIT.java} (96%)
copy
iotdb-core/confignode/src/main/java/org/apache/iotdb/confignode/{manager/pipe/resource/PipeConfigNodeCopiedFileDirStartupCleaner.java
=> conf/ConfigNodeSystemPropertiesHandler.java} (50%)
create mode 100644
iotdb-core/confignode/src/main/java/org/apache/iotdb/confignode/manager/pipe/resource/PipeConfigNodeResourceManager.java
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/{pipe/consensus/ConsensusPipeDataNodeRuntimeAgentGuardian.java
=> conf/DataNodeSystemPropertiesHandler.java} (50%)
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/agent/{PipeAgent.java
=> PipeDataNodeAgent.java} (81%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/connector/payload/evolvable/batch/PipeTabletEventBatch.java
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/connector/payload/evolvable/{builder/PipeEventBatch.java
=> batch/PipeTabletEventPlainBatch.java} (56%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/connector/payload/evolvable/batch/PipeTabletEventTsFileBatch.java
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/connector/payload/evolvable/{builder
=> batch}/PipeTransferBatchReqBuilder.java (55%)
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/connector/protocol/thrift/async/handler/{PipeTransferTsFileInsertionEventHandler.java
=> PipeTransferTsFileHandler.java} (66%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/connector/util/PipeTabletEventSorter.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/event/common/tsfile/container/TsFileInsertionDataContainer.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/event/common/tsfile/container/TsFileInsertionDataContainerProvider.java
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/event/common/tsfile/{TsFileInsertionDataContainer.java
=> container/query/TsFileInsertionQueryDataContainer.java} (64%)
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/event/common/tsfile/{TsFileInsertionDataTabletIterator.java
=> container/query/TsFileInsertionQueryDataTabletIterator.java} (89%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/event/common/tsfile/container/scan/AlignedSinglePageWholeChunkReader.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/event/common/tsfile/container/scan/SinglePageWholeChunkReader.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/event/common/tsfile/container/scan/TsFileInsertionScanDataContainer.java
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/resource/{PipeResourceManager.java
=> PipeDataNodeResourceManager.java} (93%)
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/resource/memory/{PipeMemoryWeighUtil.java
=> PipeMemoryWeightUtil.java} (56%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/metric/load/LoadTsFileCostMetricsSet.java
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/metric/{ =>
load}/LoadTsFileMemMetricSet.java (98%)
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/planner/memory/{ConsumeAllChildrenPipelineMemoryEstimator.java
=> MemoryReservationManager.java} (50%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/planner/memory/NotThreadSafeMemoryReservationManager.java
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/planner/memory/{ConsumeAllChildrenPipelineMemoryEstimator.java
=> ThreadSafeMemoryReservationManager.java} (53%)
create mode 100644
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/pipe/connector/PipeTabletEventSorterTest.java
delete mode 100644
iotdb-core/node-commons/src/assembly/resources/conf/generate_properties.bat
delete mode 100644
iotdb-core/node-commons/src/assembly/resources/conf/generate_properties.sh
copy
iotdb-core/node-commons/src/assembly/resources/conf/{iotdb-system.properties =>
iotdb-system.properties.template} (82%)
create mode 100644
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/file/SystemPropertiesHandler.java
rename iotdb-core/{datanode/src/main/java/org/apache/iotdb/db =>
node-commons/src/main/java/org/apache/iotdb/commons}/pipe/resource/log/PipeLogManager.java
(88%)
rename iotdb-core/{datanode/src/main/java/org/apache/iotdb/db =>
node-commons/src/main/java/org/apache/iotdb/commons}/pipe/resource/log/PipeLogStatus.java
(87%)
