This is an automated email from the ASF dual-hosted git repository.
Wei-hao-Li pushed a change to branch fix-issue19207
in repository https://gitbox.apache.org/repos/asf/iotdb.git
from ae02b3c312e fix
add a2d83f7abe0 Reduce WAL directory scans during subscription replay
refresh (#19129)
add 4b80fa56a99 Fix subscription prefetch retries under delivery
backpressure (#19137)
add 6a3e9d214a7 Fall back to a current replica when the cached region
leader is stale (#19135)
add c4086c5df64 Fix LOAD rate limiting to account for source TsFile bytes
(#19141)
add ad94d2b68cd Fix WAL roll file test flush wait (#17904)
add dbc4fb74f23 Fix cached nullable table attribute filtering (#19133)
add 51748ae1fde Fix subscription connection fencing during consumer
reconnect (#19130)
add 24e1203a4dc Fix table schema map registration for table-model TsFiles
(#19100)
add 219d2ea3bc7 [IT] Pin the DataRegions per DataNode in
IoTDBMultiTAGsWithAttributesTableIT (#19086)
add fc6f68c2132 Fix premature recovery from DiskFull read-only mode
(#19126)
add 4eebf87a79c Keep unrecoverable error status reasons on one line
(#19132)
add 4b447ff5c0b Reset DiskFull protection after successful configuration
reloads (#19151)
add 522c080122e Upgrade AINode security dependencies (#19153)
add 207503295e3 Add configurable Thrift buffer reuse thresholds to Java
clients (#19148)
add 1512190c043 Fix the 552 message of ALTER VIEW ADD COLUMN (#19158)
add 0ed7549b9b9 Use column header constants in pipe authentication
assertions (#19131)
add 3ab38d1f90f Retry inconsistent region metadata while waiting for
migrations (#19163)
add 23e1740ee07 Fix delayed operator memory release after fragment
termination (#18387)
add cf146aa8391 Fix timeout budget propagation in consensus subscription
polling (#19036)
add a61e5bdddc8 Honor auto-create-schema settings for tree and table LOAD
(#19128)
add 9a386abdc0e Fix MAD UDF exact calculation and silent approximate
result (#19162)
add 2691d4dac1e Fix NPE of approx_most_frequent and approx_count_distinct
on empty input (#19168)
add d8800f1cb38 Retry compression ratio baseline reads before region
migration (#19172)
add 73e3b6190ff Increase Edge metadata lease timeout to ten years (#19165)
add a48b879779e Fix race in SessionSeedNodeFallbackTest discovery buffer
checks (#19173)
add 5adec0b0961 [Parser] Skip error recovery in the SLL stage of the tree
and table SQL parsers (#19175)
add 30173fae954 [Core] Wake the procedure, WAL, disk-usage and periodical
workers on stop (#19177)
add c46fe4b8490 [Session] Write PLAIN-encoded tablet columns in bulk
instead of byte by byte (#19176)
add dd430bd2497 [Session] Fix tablet compression overflow with GZIP and
LZMA2 (#19179)
add 8189f42d16b Remove redundant access counters from TierManagerTest
(#19180)
add 29724cd62e8 Update LICENSE-binary and NOTICE-binary to match the
bundled dependencies (#19164)
add db0a9b4d756 Fix batch partial insert failure reporting (#17925)
add 3ec7ab105ff Fix WALNode outdated file deletion test (#17921)
add c6e229aef50 Prevent deleted values from reappearing after a local LOAD
crash (#19136)
add 133eadde54f [Bug] Recognize IoTConsensusV2 class name aliases on
ConfigNode (#18989)
add 21513cabf68 Preserve subscription progress during consensus leader
activation (#19170)
add c5f57e9305e Fix test helper JDBC cleanup and Pipe subtask stubbing
race (#19160)
add 947063c3783 Fix WALNodeTest compilation after explicit WAL rolling
(#19189)
add 36c35f63b65 [Pipe] Release processor worker threads when all subtasks
are deregistered (#19140)
add e6ef8949386 [Tools] Fix misspelled Alias in the import-schema error
message (#19138)
add cd31721c147 docs: require examples in feature PR descriptions (#19191)
add 6db3682ade5 Preserve accepted table attribute versions across schema
fetches (#19134)
add d10b69ee10f Fix tier folder recovery and DiskFull status after
configuration reload (#19161)
add 08cebbd76c1 Fix snapshots of deleted TsFile resources during DataNode
removal (#19192)
add b2929d321a3 Fix table-model UDFs on TAG/ATTRIBUTE predicates and
SHOW/COUNT DEVICES (#19195)
add 250f85387f5 Expose internal client sessions through multiplexed RPC
(#19125)
add 868f2c6e03f Fix LOAD splitting of sparse batched aligned chunks
(complete #19032) (#19184)
add 135c78461a5 Fix resetting maximum numeric table TTL to INF (#19183)
add 050fcb5aa98 Parallelize process of set semantic TVF when the conetent
of partition keys were same (#19155)
add e9267062671 Fix subscription timeout cleanup during DataNode removal
(#19197)
add 9e701208a88 Fix active load retries while table schema regions recover
(#19201)
add 330832abdc2 Fix Pipe memory allocation for five-part DataNode
proportions (#19202)
add 4b94bc62e29 [Test] Fix flaky LOAD restart and teardown ITs (#19103)
add b702764f564 Reject UDFs in table-model UPDATE statements (#19196)
add a4568b6e90c Reject RegionGroup policy hot reload on mixed-version
ConfigNodes (#19204)
add e3b60020bbb Fix schema file I/O failure handling and short writes
(#19190)
add 66209334eb7 [WAL] Flush a memtable instead of snapshotting it when the
snapshot exceeds the WAL entry limit (#19174)
add 4886d3c8bc4 fix that metadata high available frame could recognize the
timeout exception (#19206)
add 4f1483abc78 Fix customized cluster wrong config IT (#19213)
add fb0a0acbbda [Query] Cancel dispatched fragment instances right away
when the dispatch fails (#19085)
add 509c3954382 Remove unused Airline dependency (#19216)
add 76660832db6 [Pipe] Distribute parser admission across processor
workers (#19217)
add 6e7448d9beb Fix relational insert tablet redirect row count (#19221)
add 7b407ec0029 Upgrade Jetty to 12.0.41 and Jackson to 2.18.12 (#19226)
add 90515489a24 Support constant folding for insert values (#17850)
add d9122d83761 Support timestamp cast literals in table-model time
planning (#19223)
add 5aca93a2c2e Protect UDF classloader lifetime with query scopes (#19107)
add 54bff3d9a42 [Pipe] Allow retry heartbeats past realtime completion
barriers (#19219)
add 1d474709ba5 merge
No new revisions were added by this update.
Summary of changes:
.github/workflows/cluster-it-1c1d1a.yml | 29 +
AGENTS.md | 10 +
LICENSE-binary | 142 +++--
NOTICE-binary | 112 +++-
THREAT_MODEL.md | 20 +-
dependencies.json | 1 -
.../iotdb/it/env/cluster/config/MppJVMConfig.java | 26 +
.../it/env/cluster/node/AbstractNodeWrapper.java | 6 +
.../pass/commit/RegionMigrateFileAssertions.java | 3 +
.../IoTDBRegionMigrateNormalForIoTV2StreamIT.java | 3 +
.../IoTDBRegionMigrateWithCompressionRatioIT.java | 12 +-
.../IoTDBRegionMigrateWithLastEmptyDeletionIT.java | 3 +
.../iotdb/db/it/IoTDBCustomizedClusterIT.java | 49 +-
.../iotdb/db/it/IoTDBDiskFullFolderRecoveryIT.java | 158 ++++++
.../iotdb/db/it/IoTDBLoadTsFileClusterIT.java | 98 +++-
.../org/apache/iotdb/db/it/IoTDBLoadTsFileIT.java | 122 +++++
.../iotdb/db/it/IoTDBLoadTsFileInstallIT.java | 451 ++++++++++++++++
.../java/org/apache/iotdb/db/it/cq/IoTDBCQIT.java | 53 ++
.../apache/iotdb/db/it/utils/IoTDBTestUtilsIT.java | 126 +++++
.../org/apache/iotdb/db/it/utils/NodeDebugger.java | 232 ++++++++
.../org/apache/iotdb/db/it/utils/TestUtils.java | 56 +-
.../iotdb/it/utils/InternalSessionITUtils.java | 158 ++++++
.../treemodel/manual/IoTDBPipeInclusionIT.java | 15 +-
.../treemodel/manual/IoTDBPipePermissionIT.java | 5 +-
.../it/single/IoTDBPipeOperationDiagnosticsIT.java | 37 +-
.../relational/it/db/it/IoTDBInsertTableIT.java | 56 +-
.../it/IoTDBLoadStorageTsFileWithCompactionIT.java | 269 +++++++++
.../it/db/it/IoTDBLoadTableSchemaCompletionIT.java | 265 +++++++++
.../relational/it/db/it/IoTDBLoadTsFileIT.java | 122 +++++
.../db/it/IoTDBMultiTAGsWithAttributesTableIT.java | 6 +
.../relational/it/db/it/udf/IoTDBLocalIT.java | 92 ++++
.../it/db/it/udf/IoTDBSQLFunctionManagementIT.java | 233 ++++++++
.../it/query/recent/IoTDBComplexQueryIT.java | 109 ++++
.../it/query/recent/IoTDBGapFillTableIT.java | 40 ++
.../it/query/recent/IoTDBTableAggregationIT.java | 90 +++-
.../it/schema/IoTDBAttributeUpdateOrderingIT.java | 82 +++
.../it/schema/IoTDBNullableAttributeCacheIT.java | 261 +++++++++
.../iotdb/relational/it/schema/IoTDBTableIT.java | 41 +-
.../it/schema/IoTDBTableTagSchemaMigrationIT.java | 386 +++++++++++++
.../schema/TableMetadataPropagationTestBase.java | 450 ++++++++++++++++
.../it/schema/fault/MetadataFaultAgent.java | 600 +++++++++++++++++++++
.../it/schema/fault/MetadataFaultController.java | 244 +++++++++
.../it/session/InternalTableSessionIT.java | 174 ++++++
.../it/session/InternalTableSessionRedirectIT.java | 141 +++++
.../it/session/IoTDBSessionRelationalIT.java | 29 +-
.../apache/iotdb/session/it/InternalSessionIT.java | 225 ++++++++
.../session/it/InternalSessionRedirectIT.java | 140 +++++
.../tablemodel/InternalSubscriptionTableIT.java | 226 ++++++++
.../en/org/apache/iotdb/cli/i18n/CliMessages.java | 4 +-
.../zh/org/apache/iotdb/cli/i18n/CliMessages.java | 4 +-
.../apache/iotdb/tool/schema/ImportSchemaTree.java | 2 +-
.../org/apache/iotdb/isession/INodeSupplier.java | 18 +
.../org/apache/iotdb/rpc/AutoResizingBuffer.java | 7 +
.../iotdb/rpc/AutoScalingBufferReadTransport.java | 34 +-
.../iotdb/rpc/AutoScalingBufferWriteTransport.java | 51 +-
.../iotdb/rpc/DeepCopyRpcTransportFactory.java | 18 +-
...MemoryControl.java => RpcServiceConstants.java} | 9 +-
.../rpc/TCompressedElasticFramedTransport.java | 30 +-
.../apache/iotdb/rpc/TElasticFramedTransport.java | 19 +-
.../java/org/apache/iotdb/rpc/TSStatusCode.java | 1 +
.../iotdb/rpc/TSnappyElasticFramedTransport.java | 30 +-
.../rpc/TimeoutChangeableTFastFramedTransport.java | 29 +-
.../TimeoutChangeableTSnappyFramedTransport.java | 29 +-
.../iotdb/rpc/DeepCopyRpcTransportFactoryTest.java | 41 ++
.../rpc/TCompressedElasticFramedTransportTest.java | 295 ++++++++++
.../apache/iotdb/session/i18n/SessionMessages.java | 9 +
.../apache/iotdb/session/i18n/SessionMessages.java | 9 +
.../iotdb/session/AbstractSessionBuilder.java | 1 +
.../org/apache/iotdb/session/NodesSupplier.java | 57 +-
.../java/org/apache/iotdb/session/Session.java | 75 ++-
.../apache/iotdb/session/SessionConnection.java | 18 +-
.../apache/iotdb/session/TableSessionBuilder.java | 20 +-
.../org/apache/iotdb/session/ThriftConnection.java | 19 +-
.../session/pool/AbstractSessionPoolBuilder.java | 2 +
.../org/apache/iotdb/session/pool/SessionPool.java | 60 ++-
.../iotdb/session/pool/TableSessionPool.java | 2 +-
.../session/pool/TableSessionPoolBuilder.java | 22 +-
.../iotdb/session/rpccompress/TabletEncoder.java | 104 ++--
.../iotdb/session/util/SessionTypeServices.java | 141 +++++
.../apache/iotdb/session/util/SessionUtils.java | 17 +
.../iotdb/session/SessionSeedNodeFallbackTest.java | 149 +++++
.../rpccompress/TabletEncoderCompressionTest.java | 230 ++++++++
.../rpccompress/TabletEncoderPlainTest.java | 289 ++++++++++
.../subscription/i18n/SubscriptionMessages.java | 1 -
.../subscription/i18n/SubscriptionMessages.java | 1 -
.../rpc/subscription/config/ConsumerConstant.java | 7 +
.../subscription/SubscriptionSessionWrapper.java | 28 +-
.../subscription/SubscriptionTableSession.java | 4 +
.../SubscriptionTableSessionBuilder.java | 36 +-
.../SubscriptionTreeSessionBuilder.java | 30 ++
.../base/AbstractSubscriptionConsumer.java | 51 +-
.../base/AbstractSubscriptionConsumerBuilder.java | 37 ++
.../base/AbstractSubscriptionProvider.java | 24 +-
.../base/AbstractSubscriptionProviders.java | 2 +-
.../SubscriptionTablePullConsumerBuilder.java | 40 ++
.../SubscriptionTablePushConsumerBuilder.java | 36 ++
.../tree/SubscriptionTreePullConsumerBuilder.java | 36 ++
.../tree/SubscriptionTreePushConsumerBuilder.java | 36 ++
.../SubscriptionConsumerMultiProviderPollTest.java | 73 +++
.../SubscriptionProviderConnectionTimeoutTest.java | 25 +-
.../base/SubscriptionProviderStatusTest.java | 12 +
iotdb-core/ainode/README.md | 24 +-
iotdb-core/ainode/pyproject.toml | 17 +-
.../ainode/tests/test_dependency_security.py | 115 ++++
.../AbstractApproxMostFrequentAccumulator.java | 19 +-
.../ApproxCountDistinctAccumulator.java | 12 +-
.../BlobApproxMostFrequentAccumulator.java | 4 +
.../approximate/HyperLogLogStateFactory.java | 2 +-
.../aggregation/approximate/SpaceSaving.java | 4 +
...stractGroupedApproxMostFrequentAccumulator.java | 18 +-
.../BlobGroupedApproxMostFrequentAccumulator.java | 9 +-
.../GroupedApproxCountDistinctAccumulator.java | 12 +-
.../ir/GlobalTimePredicateExtractVisitor.java | 18 +-
.../iotdb/confignode/i18n/ConfigNodeMessages.java | 11 +
.../iotdb/confignode/i18n/ManagerMessages.java | 7 +-
.../iotdb/confignode/i18n/ConfigNodeMessages.java | 10 +
.../iotdb/confignode/i18n/ManagerMessages.java | 6 +-
.../handlers/rpc/DataNodeTSStatusRPCHandler.java | 8 +-
.../iotdb/confignode/conf/ConfigNodeConfig.java | 9 +-
.../confignode/conf/ConfigNodeDescriptor.java | 20 +-
.../confignode/conf/ConfigNodeStartupCheck.java | 25 +
.../iotdb/confignode/manager/ConfigManager.java | 32 +-
.../manager/load/balancer/RouteBalancer.java | 4 +-
.../iotdb/confignode/manager/node/NodeManager.java | 37 ++
.../manager/partition/PartitionManager.java | 11 +
.../pipe/coordinator/runtime/PipeMetaSyncer.java | 25 +-
.../pipe/coordinator/task/PipeTaskCoordinator.java | 4 +
.../manager/schema/ClusterSchemaManager.java | 6 +-
.../partition/DatabasePartitionTable.java | 16 +
.../persistence/partition/PartitionInfo.java | 17 +
.../persistence/partition/RegionGroup.java | 7 +
.../confignode/persistence/pipe/PipeTaskInfo.java | 19 +
.../confignode/procedure/ProcedureExecutor.java | 6 +-
.../procedure/TimeoutExecutorThread.java | 18 +-
.../procedure/env/RegionMaintainHandler.java | 37 +-
.../impl/schema/DataNodeRegionGroupUtil.java | 7 +-
.../confignode/service/ConfigNodeShutdownHook.java | 3 +-
.../confignode/conf/ConfigNodeConfigTest.java | 25 +
.../conf/ConfigNodeStartupCheckTest.java | 87 +++
.../manager/ClusterSchemaManagerTest.java | 16 +-
.../ConfigManagerRegionGroupPolicyVersionTest.java | 280 ++++++++++
.../manager/lease/ClusterCachePropagatorTest.java | 42 ++
.../manager/node/NodeManagerStatusReasonTest.java | 23 +-
.../node/NodeManagerVersionConsistencyTest.java | 178 ++++++
.../coordinator/runtime/PipeMetaSyncerTest.java | 79 +++
.../procedure/ProcedureExecutorStopTest.java | 57 ++
.../RegionMaintainHandlerConsensusPipeTest.java | 83 +++
.../impl/schema/DataNodeRegionGroupUtilTest.java | 54 ++
.../table/SetTablePropertiesProcedureTest.java | 70 +++
iotdb-core/datanode/pom.xml | 4 -
.../apache/iotdb/db/i18n/DataNodeMiscMessages.java | 13 +-
.../apache/iotdb/db/i18n/DataNodePipeMessages.java | 8 +
.../iotdb/db/i18n/DataNodeQueryMessages.java | 47 ++
.../iotdb/db/i18n/StorageEngineMessages.java | 2 +-
.../apache/iotdb/db/i18n/DataNodeMiscMessages.java | 13 +-
.../apache/iotdb/db/i18n/DataNodePipeMessages.java | 8 +
.../iotdb/db/i18n/DataNodeQueryMessages.java | 47 ++
.../iotdb/db/i18n/StorageEngineMessages.java | 2 +-
.../org/apache/iotdb/db/auth/AuthorityChecker.java | 2 +-
.../apache/iotdb/db/conf/DataNodeMemoryConfig.java | 6 +-
.../consensus/statemachine/BaseStateMachine.java | 26 +-
.../dataregion/DataRegionStateMachine.java | 6 +-
.../schemaregion/SchemaRegionStateMachine.java | 6 +-
...va => LoadAnalyzeMissingDatabaseException.java} | 4 +-
.../execution/PipeProcessorSubtaskExecutor.java | 10 +
.../subtask/processor/PipeProcessorSubtask.java | 16 +-
.../processor/PipeProcessorSubtaskWorker.java | 102 +++-
.../PipeProcessorSubtaskWorkerManager.java | 27 +-
.../sink/PipeRealtimePriorityBlockingQueue.java | 4 +-
.../common/tsfile/PipeTsFileInsertionEvent.java | 6 +-
.../db/pipe/resource/memory/PipeMemoryManager.java | 69 ++-
.../builder/PipeTableModelTsFileBuilderV2.java | 4 +
.../realtime/PipeRealtimeDataRegionSource.java | 1 +
.../PipeRealtimeDataRegionTsFileSource.java | 3 +-
.../iotdb/db/protocol/session/IClientSession.java | 13 +
.../iotdb/db/protocol/session/SessionManager.java | 134 +++--
.../thrift/InternalClientRPCServiceProcessor.java | 70 +++
.../handler/InternalServiceThriftHandler.java | 43 +-
.../protocol/thrift/impl/ClientRPCServiceImpl.java | 44 +-
.../impl/DataNodeInternalRPCServiceImpl.java | 87 ++-
...dler.java => InternalClientRPCServiceImpl.java} | 9 +-
.../db/queryengine/common/DataNodeEndPoints.java | 7 +
.../db/queryengine/common/MPPQueryContext.java | 19 +-
.../fragment/FragmentInstanceContext.java | 34 +-
.../fragment/FragmentInstanceManager.java | 43 +-
.../fragment/FragmentInstanceStateMachine.java | 74 ++-
.../operator/process/TransformOperator.java | 37 +-
.../iotdb/db/queryengine/plan/Coordinator.java | 11 +-
.../db/queryengine/plan/analyze/Analysis.java | 20 +-
.../db/queryengine/plan/analyze/AnalyzeUtils.java | 34 +-
.../db/queryengine/plan/analyze/Analyzer.java | 1 +
.../db/queryengine/plan/analyze/IAnalysis.java | 5 +-
.../plan/analyze/load/LoadTsFileAnalyzer.java | 109 ++--
.../analyze/load/LoadTsFileTableSchemaCache.java | 111 +++-
.../load/TreeSchemaAutoCreatorAndVerifier.java | 11 +-
.../plan/analyze/schema/NormalSchemaFetcher.java | 28 +-
.../plan/analyze/schema/SchemaValidator.java | 8 +-
.../plan/analyze/schema/TemplateSchemaFetcher.java | 13 +-
.../queryengine/plan/execution/QueryExecution.java | 92 +++-
.../config/executor/ClusterConfigTaskExecutor.java | 22 +-
.../config/metadata/ShowFunctionsTask.java | 16 +-
.../plan/parser/StatementGenerator.java | 4 +
.../planner/DataNodeTableOperatorGenerator.java | 6 +-
.../plan/planner/LocalExecutionPlanner.java | 73 ++-
.../plan/planner/plan/FragmentInstance.java | 28 +-
.../plan/node/load/LoadSingleTsFileNode.java | 37 +-
.../planner/plan/node/load/LoadTsFileNode.java | 6 +-
.../planner/plan/node/write/InsertRowNode.java | 3 +-
.../planner/plan/node/write/InsertRowsNode.java | 6 +-
.../plan/node/write/InsertRowsOfOneDeviceNode.java | 3 +-
.../planner/plan/node/write/InsertTabletNode.java | 6 +-
.../plan/node/write/RelationalInsertRowsNode.java | 18 +-
.../node/write/RelationalInsertTabletNode.java | 19 +-
.../plan/relational/analyzer/Analysis.java | 21 +-
.../plan/relational/analyzer/Analyzer.java | 1 +
.../relational/analyzer/ExpressionAnalyzer.java | 11 +
.../relational/analyzer/StatementAnalyzer.java | 46 ++
.../ConvertPredicateToTimeFilterVisitor.java | 12 +-
.../plan/relational/metadata/Metadata.java | 3 +-
.../relational/metadata/TableMetadataImpl.java | 4 +-
.../fetcher/DeviceInCacheFilterVisitor.java | 8 +-
.../fetcher/TableDeviceCacheAttributeGuard.java | 3 +-
.../metadata/fetcher/TableDeviceSchemaFetcher.java | 2 +-
.../fetcher/TableHeaderSchemaValidator.java | 8 +-
.../planner/IrExpressionInterpreter.java | 326 +++++++----
.../plan/relational/planner/QueryPlanner.java | 12 +-
.../plan/relational/planner/TranslationMap.java | 5 +-
.../ir/GapFillStartAndEndTimeExtractVisitor.java | 46 +-
.../security/TreeAccessCheckContext.java | 10 +-
.../security/TreeAccessCheckVisitor.java | 6 +-
.../plan/relational/sql/ast/InsertValues.java | 346 ++++++++++++
.../plan/relational/sql/parser/AstBuilder.java | 234 +-------
.../plan/relational/sql/parser/SqlParser.java | 13 +-
.../plan/relational/sql/util/AstUtil.java | 83 +++
.../plan/scheduler/ClusterScheduler.java | 12 +-
.../plan/scheduler/IQueryTerminator.java | 8 +
.../plan/scheduler/SimpleQueryTerminator.java | 12 +-
.../plan/scheduler/load/LoadFallbackHandler.java | 10 +-
.../crud/InsertMultiTabletsStatement.java | 46 +-
.../crud/InsertRowsOfOneDeviceStatement.java | 46 +-
.../plan/statement/crud/InsertRowsStatement.java | 49 +-
.../db/queryengine/plan/udf/TreeUDFUtils.java | 17 +-
.../transformation/dag/udf/UDTFContext.java | 2 -
.../mtree/impl/pbtree/schemafile/SchemaFile.java | 115 +++-
.../mtree/impl/pbtree/schemafile/SchemaPage.java | 3 +-
.../pbtree/schemafile/pagemgr/PageIOChannel.java | 79 ++-
.../pbtree/schemafile/pagemgr/PageManager.java | 27 +-
.../reader/impl/TimeseriesReaderWithViewFetch.java | 44 ++
.../db/service/DataNodeInternalRPCService.java | 40 +-
.../metrics/IoTDBInternalLocalReporter.java | 2 +-
.../db/storageengine/dataregion/DataRegion.java | 179 +++---
.../impl/ReadChunkCompactionPerformer.java | 4 +
.../compaction/io/CompactionTsFileWriter.java | 11 +
.../dataregion/memtable/TsFileProcessor.java | 17 +
.../dataregion/snapshot/SnapshotLoader.java | 17 +-
.../dataregion/snapshot/SnapshotTaker.java | 13 +-
.../dataregion/tsfile/TsFileResource.java | 24 +-
.../utils/TsFileTableSchemaRegistrationUtils.java | 110 ++++
.../tableDiskUsageIndex/TableDiskUsageIndex.java | 18 +
.../dataregion/wal/buffer/WALBuffer.java | 8 +-
.../storageengine/dataregion/wal/node/WALNode.java | 6 +-
.../file/UnsealedTsFileRecoverPerformer.java | 25 +-
.../db/storageengine/load/LoadStagingDirs.java | 16 +-
.../converter/LoadTsFileDataTypeConverter.java | 71 ++-
.../splitter/BatchedAlignedValueChunkData.java | 107 ++--
.../iotdb/db/storageengine/load/util/LoadUtil.java | 18 +-
.../db/storageengine/rescon/disk/TierManager.java | 198 ++++---
.../agent/SubscriptionReceiverAgent.java | 19 +-
.../broker/ConsensusSubscriptionBroker.java | 28 +-
.../consensus/ConsensusPrefetchingQueue.java | 154 ++++--
.../ConsensusSubscriptionCommitManager.java | 393 +++++++++++---
.../broker/consensus/ProgressWALIterator.java | 98 +++-
.../consensus/SubscriptionConsensusProgress.java | 29 +-
.../receiver/SubscriptionReceiver.java | 5 +
.../receiver/SubscriptionReceiverV1.java | 117 +++-
.../org/apache/iotdb/db/utils/CommonUtils.java | 62 ---
.../org/apache/iotdb/db/auth/TreeAccessTest.java | 2 +-
.../db/conf/DataNodeMemoryProportionTest.java | 106 ++++
.../iotdb/db/conf/DiskFullHotReloadTest.java | 143 +++++
.../dataregion/IoTConsensusShutdownTest.java | 10 +-
.../metadata/mtree/schemafile/SchemaFileTest.java | 261 +++++++++
.../metadata/mtree/schemafile/SchemaPageTest.java | 110 ++++
.../task/PipeProcessorSubtaskExecutorTest.java | 33 ++
.../pipe/agent/task/PipeSubtaskExecutorTest.java | 3 +-
.../task/connection/PipeEventCollectorTest.java | 31 ++
.../PipeProcessorSubtaskExecutionGuardTest.java | 7 +-
.../processor/PipeProcessorSubtaskWorkerTest.java | 271 +++++++++-
.../resource/memory/PipeMemoryManagerTest.java | 64 +++
.../db/pipe/source/PipeRealtimeExtractTest.java | 104 +++-
.../handler/InternalServiceThriftHandlerTest.java | 57 ++
.../DataNodeInternalRPCServiceImplDiskTest.java | 46 ++
.../fragment/FragmentInstanceStateMachineTest.java | 56 ++
.../load/LoadTsFileAnalyzerDiagnosticsTest.java | 417 +++++++++++++-
.../plan/analyze/load/LoadTsFileAnalyzerTest.java | 411 ++++++++++++++
.../load/LoadTsFileTableSchemaRecoveryTest.java | 185 +++++++
.../schema/TreeSchemaAutoCreationPolicyTest.java | 207 +++++++
.../plan/execution/QueryExecutionRetryTest.java | 68 ++-
.../plan/planner/FragmentInstanceSerdeTest.java | 41 +-
.../LocalExecutionPlannerOperatorsMemoryTest.java | 34 ++
.../plan/planner/node/load/LoadTsFileNodeTest.java | 38 +-
.../planner/node/write/WritePlanNodeSplitTest.java | 29 +
.../TableDeviceCacheAttributeGuardTest.java | 64 +++
.../relational/planner/TableModelPlannerTest.java | 13 +-
.../relational/sql/AstMemoryEstimatorTest.java | 29 +-
.../plan/relational/sql/ast/InsertValuesTest.java | 204 +++++++
.../plan/scheduler/SimpleQueryTerminatorTest.java | 72 +++
.../scheduler/load/LoadFallbackHandlerTest.java | 139 +++++
.../crud/InsertStatementPartialInsertTest.java | 117 ++++
.../storageengine/dataregion/DataRegionTest.java | 281 ++++++++++
.../dataregion/snapshot/IoTDBSnapshotTest.java | 30 ++
.../snapshot/SnapshotLoaderDiskFullReloadTest.java | 126 +++++
.../dataregion/tsfile/TsFileResourceTest.java | 5 +
.../TsFileTableSchemaRegistrationUtilsTest.java | 204 +++++++
.../dataregion/wal/node/WALNodeTest.java | 118 +++-
.../wal/node/WALNodeWaitForRollFileTest.java | 101 ++--
.../wal/recover/WALRecoverManagerTest.java | 6 +-
...ncryptedUnsealedTsFileRecoverPerformerTest.java | 10 +-
.../file/UnsealedTsFileRecoverPerformerTest.java | 13 +-
.../db/storageengine/load/TsFileSplitterTest.java | 209 +++++++
.../LoadTsFileDataTypeConverterSchemaTest.java | 349 ++++++++++++
.../disk/FolderManagerDiskFullRecoveryTest.java | 160 ++++++
.../rescon/disk/TierManagerDiskFullReloadTest.java | 120 +++++
.../storageengine/rescon/disk/TierManagerTest.java | 58 +-
.../agent/SubscriptionReceiverAgentTest.java | 24 +
...onsensusSubscriptionBrokerPayloadLimitTest.java | 37 +-
...ConsensusSubscriptionBrokerPollTimeoutTest.java | 198 +++++++
.../ConsensusPrefetchingQueuePollTimeoutTest.java | 180 +++++++
.../ConsensusPrefetchingQueueSeekTest.java | 242 +++++++++
...nsensusPrefetchingQueueWalBackpressureTest.java | 195 +++++++
.../ConsensusSubscriptionCommitStateTest.java | 358 +++++++++++-
.../broker/consensus/ProgressWALIteratorTest.java | 325 ++++++++++-
.../receiver/SubscriptionReceiverV1Test.java | 274 ++++++++++
.../db/utils/FileChannelWriteArchitectureTest.java | 91 ++++
iotdb-core/node-commons/pom.xml | 10 +
.../resources/conf/edge/iotdb-system.properties | 7 +
.../apache/iotdb/commons/i18n/QueryMessages.java | 9 +
.../apache/iotdb/commons/i18n/UtilMessages.java | 2 +
.../apache/iotdb/commons/i18n/QueryMessages.java | 9 +
.../apache/iotdb/commons/i18n/UtilMessages.java | 2 +
.../apache/iotdb/commons/audit/IAuditEntity.java | 8 +-
.../client/internal/session/InternalSession.java | 61 +++
.../internal/session/InternalSessionBuilder.java | 59 ++
.../internal/session/InternalSessionPool.java} | 18 +-
.../session/InternalSessionPoolBuilder.java | 55 ++
.../session/InternalTableSessionBuilder.java | 60 +++
.../session/InternalTableSessionPool.java} | 25 +-
.../session/InternalTableSessionPoolBuilder.java | 56 ++
.../InternalSubscriptionSessionWrapper.java | 78 +++
.../InternalSubscriptionTablePullConsumer.java | 21 +-
...ternalSubscriptionTablePullConsumerBuilder.java | 43 ++
.../InternalSubscriptionTableSession.java | 15 +-
.../InternalSubscriptionTableSessionBuilder.java | 55 ++
.../apache/iotdb/commons/conf/CommonConfig.java | 79 ++-
.../apache/iotdb/commons/disk/FolderManager.java | 37 +-
.../commons/disk/strategy/DirectoryStrategy.java | 22 +-
.../commons/enums/HandleSystemErrorStrategy.java | 31 +-
.../agent/task/execution/PipeSubtaskExecutor.java | 23 +-
.../commons/queryengine/common/SessionInfo.java | 70 ++-
.../queryengine/plan/udf/UDFManagementService.java | 129 +++--
.../AbstractPeriodicalServiceWithAdvance.java | 14 +-
.../iotdb/commons/service/ThriftService.java | 10 +-
.../org/apache/iotdb/commons/udf/UDFTable.java | 81 +--
.../iotdb/commons/udf/service/UDFClassLoader.java | 18 +-
.../commons/udf/service/UDFClassLoaderManager.java | 179 +++++-
.../org/apache/iotdb/commons/utils/IOUtils.java | 36 ++
.../apache/iotdb/commons/utils/JVMCommonUtils.java | 26 +-
.../session/InternalSessionProtocolTest.java | 167 ++++++
.../iotdb/commons/conf/CommonConfigTest.java | 205 ++++++-
.../plan/udf/UDFManagementServiceTest.java | 211 ++++++++
.../AbstractPeriodicalServiceWithAdvanceTest.java | 95 ++++
.../udf/service/UDFClassLoaderManagerTest.java | 95 ++++
.../apache/iotdb/commons/utils/IOUtilsTest.java | 152 ++++++
.../iotdb/commons/utils/JVMCommonUtilsTest.java | 35 ++
.../src/main/thrift/confignode.thrift | 2 +
.../iotdb/library/i18n/LibraryUdfMessages.java | 6 +
.../iotdb/library/i18n/LibraryUdfMessages.java | 5 +
.../org/apache/iotdb/library/dprofile/UDAFMad.java | 10 +-
.../dprofile/util/ExactOrderStatistics.java | 16 +-
.../apache/iotdb/library/dprofile/UDAFMadTest.java | 114 ++++
.../dprofile/util/ExactOrderStatisticsTest.java | 73 +++
licenses/CDDL License 1.1 | 90 ----
licenses/Creative Commons CC0 1.0 Universal | 121 +++++
.../Eclipse Distribution License 1.0 (EDL-1.0) | 30 ++
licenses/Eclipse Public License 1.0 (EPL-1.0) | 83 ---
...(Boost Software License 1.0, MIT, BSD 2-Clause) | 91 ++++
licenses/MIT No Attribution (Reactive Streams) | 7 +
.../The MIT License (Checker Framework) | 13 +-
... (progressbar) => The MIT License (ClassGraph)} | 4 +-
licenses/The MIT License (FastDoubleParser) | 61 +++
licenses/The MIT License (Java-WebSocket) | 24 +
.../The MIT License (Schubfach) | 12 +-
pom.xml | 17 +-
392 files changed, 23358 insertions(+), 2139 deletions(-)
create mode 100644
integration-test/src/test/java/org/apache/iotdb/db/it/IoTDBDiskFullFolderRecoveryIT.java
create mode 100644
integration-test/src/test/java/org/apache/iotdb/db/it/IoTDBLoadTsFileInstallIT.java
create mode 100644
integration-test/src/test/java/org/apache/iotdb/db/it/utils/NodeDebugger.java
create mode 100644
integration-test/src/test/java/org/apache/iotdb/it/utils/InternalSessionITUtils.java
create mode 100644
integration-test/src/test/java/org/apache/iotdb/relational/it/db/it/IoTDBLoadStorageTsFileWithCompactionIT.java
create mode 100644
integration-test/src/test/java/org/apache/iotdb/relational/it/db/it/IoTDBLoadTableSchemaCompletionIT.java
create mode 100644
integration-test/src/test/java/org/apache/iotdb/relational/it/schema/IoTDBAttributeUpdateOrderingIT.java
create mode 100644
integration-test/src/test/java/org/apache/iotdb/relational/it/schema/IoTDBNullableAttributeCacheIT.java
create mode 100644
integration-test/src/test/java/org/apache/iotdb/relational/it/schema/IoTDBTableTagSchemaMigrationIT.java
create mode 100644
integration-test/src/test/java/org/apache/iotdb/relational/it/schema/TableMetadataPropagationTestBase.java
create mode 100644
integration-test/src/test/java/org/apache/iotdb/relational/it/schema/fault/MetadataFaultAgent.java
create mode 100644
integration-test/src/test/java/org/apache/iotdb/relational/it/schema/fault/MetadataFaultController.java
create mode 100644
integration-test/src/test/java/org/apache/iotdb/relational/it/session/InternalTableSessionIT.java
create mode 100644
integration-test/src/test/java/org/apache/iotdb/relational/it/session/InternalTableSessionRedirectIT.java
create mode 100644
integration-test/src/test/java/org/apache/iotdb/session/it/InternalSessionIT.java
create mode 100644
integration-test/src/test/java/org/apache/iotdb/session/it/InternalSessionRedirectIT.java
create mode 100644
integration-test/src/test/java/org/apache/iotdb/subscription/it/consensus/local/tablemodel/InternalSubscriptionTableIT.java
copy
iotdb-client/service-rpc/src/main/java/org/apache/iotdb/rpc/{AutoResizingBufferMemoryControl.java
=> RpcServiceConstants.java} (73%)
create mode 100644
iotdb-client/service-rpc/src/test/java/org/apache/iotdb/rpc/TCompressedElasticFramedTransportTest.java
create mode 100644
iotdb-client/session/src/test/java/org/apache/iotdb/session/rpccompress/TabletEncoderCompressionTest.java
create mode 100644
iotdb-client/session/src/test/java/org/apache/iotdb/session/rpccompress/TabletEncoderPlainTest.java
create mode 100644 iotdb-core/ainode/tests/test_dependency_security.py
create mode 100644
iotdb-core/confignode/src/test/java/org/apache/iotdb/confignode/conf/ConfigNodeStartupCheckTest.java
create mode 100644
iotdb-core/confignode/src/test/java/org/apache/iotdb/confignode/manager/ConfigManagerRegionGroupPolicyVersionTest.java
create mode 100644
iotdb-core/confignode/src/test/java/org/apache/iotdb/confignode/manager/node/NodeManagerVersionConsistencyTest.java
create mode 100644
iotdb-core/confignode/src/test/java/org/apache/iotdb/confignode/manager/pipe/coordinator/runtime/PipeMetaSyncerTest.java
create mode 100644
iotdb-core/confignode/src/test/java/org/apache/iotdb/confignode/procedure/ProcedureExecutorStopTest.java
create mode 100644
iotdb-core/confignode/src/test/java/org/apache/iotdb/confignode/procedure/impl/schema/DataNodeRegionGroupUtilTest.java
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/exception/load/{LoadAnalyzeMissingSchemaException.java
=> LoadAnalyzeMissingDatabaseException.java} (84%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/protocol/thrift/InternalClientRPCServiceProcessor.java
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/protocol/thrift/impl/{IClientRPCServiceWithHandler.java
=> InternalClientRPCServiceImpl.java} (78%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/sql/ast/InsertValues.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/utils/TsFileTableSchemaRegistrationUtils.java
create mode 100644
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/conf/DataNodeMemoryProportionTest.java
create mode 100644
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/conf/DiskFullHotReloadTest.java
create mode 100644
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/protocol/thrift/handler/InternalServiceThriftHandlerTest.java
create mode 100644
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/queryengine/plan/analyze/load/LoadTsFileTableSchemaRecoveryTest.java
create mode 100644
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/queryengine/plan/analyze/schema/TreeSchemaAutoCreationPolicyTest.java
create mode 100644
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/queryengine/plan/relational/metadata/fetcher/TableDeviceCacheAttributeGuardTest.java
create mode 100644
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/queryengine/plan/relational/sql/ast/InsertValuesTest.java
create mode 100644
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/queryengine/plan/scheduler/SimpleQueryTerminatorTest.java
create mode 100644
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/storageengine/dataregion/snapshot/SnapshotLoaderDiskFullReloadTest.java
create mode 100644
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/storageengine/dataregion/utils/TsFileTableSchemaRegistrationUtilsTest.java
create mode 100644
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/storageengine/load/converter/LoadTsFileDataTypeConverterSchemaTest.java
create mode 100644
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/storageengine/rescon/disk/FolderManagerDiskFullRecoveryTest.java
create mode 100644
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/storageengine/rescon/disk/TierManagerDiskFullReloadTest.java
create mode 100644
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/subscription/broker/ConsensusSubscriptionBrokerPollTimeoutTest.java
create mode 100644
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/subscription/broker/consensus/ConsensusPrefetchingQueuePollTimeoutTest.java
create mode 100644
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/utils/FileChannelWriteArchitectureTest.java
create mode 100644
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/client/internal/session/InternalSession.java
create mode 100644
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/client/internal/session/InternalSessionBuilder.java
copy
iotdb-core/{datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/compaction/schedule/constant/CompactionType.java
=>
node-commons/src/main/java/org/apache/iotdb/commons/client/internal/session/InternalSessionPool.java}
(66%)
create mode 100644
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/client/internal/session/InternalSessionPoolBuilder.java
create mode 100644
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/client/internal/session/InternalTableSessionBuilder.java
copy
iotdb-core/{datanode/src/main/java/org/apache/iotdb/db/queryengine/transformation/dag/transformer/binary/ArithmeticAdditionTransformer.java
=>
node-commons/src/main/java/org/apache/iotdb/commons/client/internal/session/InternalTableSessionPool.java}
(53%)
create mode 100644
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/client/internal/session/InternalTableSessionPoolBuilder.java
create mode 100644
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/client/internal/subscription/InternalSubscriptionSessionWrapper.java
copy
integration-test/src/main/java/org/apache/iotdb/it/env/cluster/env/SimpleEnv.java
=>
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/client/internal/subscription/InternalSubscriptionTablePullConsumer.java
(58%)
create mode 100644
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/client/internal/subscription/InternalSubscriptionTablePullConsumerBuilder.java
copy
iotdb-client/session/src/main/java/org/apache/iotdb/session/pool/AbstractSessionPoolBuilder.java
=>
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/client/internal/subscription/InternalSubscriptionTableSession.java
(65%)
create mode 100644
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/client/internal/subscription/InternalSubscriptionTableSessionBuilder.java
create mode 100644
iotdb-core/node-commons/src/test/java/org/apache/iotdb/commons/client/internal/session/InternalSessionProtocolTest.java
create mode 100644
iotdb-core/node-commons/src/test/java/org/apache/iotdb/commons/queryengine/plan/udf/UDFManagementServiceTest.java
create mode 100644
iotdb-core/node-commons/src/test/java/org/apache/iotdb/commons/service/AbstractPeriodicalServiceWithAdvanceTest.java
create mode 100644
iotdb-core/node-commons/src/test/java/org/apache/iotdb/commons/udf/service/UDFClassLoaderManagerTest.java
create mode 100644
library-udf/src/test/java/org/apache/iotdb/library/dprofile/UDAFMadTest.java
create mode 100644
library-udf/src/test/java/org/apache/iotdb/library/dprofile/util/ExactOrderStatisticsTest.java
delete mode 100644 licenses/CDDL License 1.1
create mode 100644 licenses/Creative Commons CC0 1.0 Universal
create mode 100644 licenses/Eclipse Distribution License 1.0 (EDL-1.0)
delete mode 100644 licenses/Eclipse Public License 1.0 (EPL-1.0)
create mode 100644 licenses/FastDoubleParser Third-Party Licenses (Boost
Software License 1.0, MIT, BSD 2-Clause)
create mode 100644 licenses/MIT No Attribution (Reactive Streams)
copy iotdb-core/ainode/src/assembly/resources/licenses/LICENSE-GLIDE =>
licenses/The MIT License (Checker Framework) (83%)
rename licenses/{The MIT License (progressbar) => The MIT License
(ClassGraph)} (95%)
create mode 100644 licenses/The MIT License (FastDoubleParser)
create mode 100644 licenses/The MIT License (Java-WebSocket)
copy iotdb-core/ainode/src/assembly/resources/licenses/LICENSE-GLIDE =>
licenses/The MIT License (Schubfach) (88%)