This is an automated email from the ASF dual-hosted git repository.
haonan pushed a change to branch rc/1.3.3
in repository https://gitbox.apache.org/repos/asf/iotdb.git
from b2549039bab Fixed the bug that fetchSeriesSchema returns series in
black list when the given pattern tree is without wildcards
add 0fcd638a308 Pipe/Subscription: Update the way SubscriptionCoordinator
and PipeTaskCoordinator acquire locks to prevent procedure success without any
effect (#12909)
add ac110bce20b Use max bytes to allocate WAL compressed buffer (#12928)
add ed75066c119 PipeConsensus: Avoid pipe task being restarted frequently
by Pipe framework (#12931)
add 35ea5006a2d Optimized DeleteTimeSeriesProcedure not to set mods when
the deleted time series are all views
add de3adcd0363 Fix uncaught IllegalArgumentException when reading broken
WAL files (#12912)
add 10ae8196eaf Do not remove top-level load dir after load & add retry
for getting disk space (#12920)
add 361e577cbcf Fix python client dependency issue (#12934)
add 445226b7204 Support new dataType in select into clause and fill clause
add 8075822f4fe fix compression ratio warn log (#12936)
add 209d26329a4 PipeConsensus: Fix leader write points metric (#12933)
add c30ad41ecac Reduce default mod file threshold for settle compaction
(#12935)
add c6fe5c352b9 Subscription: improve end-to-end latency (#12925)
add e617586221a Subscription: avoid invalidating cache for null response &
improve exception message (#12939)
add 49c6deaaa67 Skip compiling cpp test code when using -DskipTests
(#12944)
add 065e887c5f8 Update dockerfile to use java 17 (#12948)
add 57aea073d78 Cherry pick Partite Graph Replication #12946
add 2eed096a330 Fix PointPriorityReader compare PointElement (#12945)
add ec63dc17220 Upgrade thrift plugin and optimize codegen for go client
(#12949)
add b2c42b79004 Fix flush error due to compression ratio (#12953)
add 8070bae8500 Pipe: Fixed the bug that historical alias/attributes/tags
may not be transferred in meta pipe when the time series already exists in
receiver & create timeseries non idempotent in PBTree
add d70b5b9ee3c Bump jline version to 3.26.2 (#12956)
add 71a4ecb642f Pipe: support alter pipe source (#12932)
add d9ff178aa08 Fixed the bug that some schema IT does not run in PBTree
mode
add 2f7069baf2f Fix sync stuck problem of IoTConsensus and WAL (#12955)
add a82e2793f4c Remove warning log when CheckPointManager find memtable id
= -1 #12959
add 5c10695f032 Pipe: filter out databases whose name starts with
`root.__system.` (#12957)
add e6d432a685c Modify the default parameters of DataNode to accelerated
startup #12941
add 83116396ae7 PipeConsensus: Fix circular replicate in kill -9/restart
cases (#12937)
add 4725e7e5978 Fixed issue where the wal version kept increasing after
the write load stopped (#12960)
add b68a906dd81 Enhance the automatic metadata creation logic of
root.__system prefixed devices #12962
add 3940892457e upgrade thrift maven plugin for cpp client (#12964)
add 6d513a75da0 Export Data And Schema: filter out databases whose name
starts with root.__system. (#12963)
add b527e066362 Wraps DateTimeUtilsTest in a session (#12971)
add d65fb09ee72 Remove verbose download log spam from CI builds (#12972)
add ce01d8627da Fixed potential NPE in local write execution & Refactored
queryTerminator
add 8cd57abda1d fix CompactionTaskManager ut (#12923)
add ea4fc975328 Optimize log for broken wal. (#12977)
add 9d73c8445c5 Clear storage engine snapshot after region migration
(#12975)
add 0f797e28cf9 Subscription: fix the issue with nodeUrls parameter being
ineffective & missing return for create topic (#12974)
add 76ba7333616 Fix concurrency problem in Memory Control (#12984)
add 8c319778781 Allows root.__system prefix time series creation #12991
add 392625f38bf Ensure the flush total points statistic function works
correctly when enable_auto_create_schema is false #12990
add f5f0a3401f8 Pipe/PipeConsensus: Fix invalid retry count in report &
enhance log in pipe consensus (#12989)
add cbabf876246 Replaces the internal maven property with an officially
supported alternative (#12982)
add 27c2f13324e Reduce error log when compaction interrupted (#12985)
add abb1f6ad12b Remove UNKNOWN from WALFileVersion which fails recovering
V1 WAL file (#12986)
add a45adbc8534 Pipe: Fix empty tablets generated by pattern parsing on
sender side may cause NPE on receiver side (#12994)
add 41213b097fb Bump tsfile version to 1.0.1-a6fb416-SNAPSHOT (#12999)
add 77b41310f55 Allows for unit tests to be run in parallel (#12980)
add 623fa5bac5f Pipe: add ops/latency metrics for different types of pipe
operations on receivers (DN / CN) (#12927)
add 3f112b9ad96 Change some error log to warn during region migration
(#12992)
add c866c0fb2cd Rectify the input parameters of LeaderBalancer (#12998)
add 7d94bba6ad6 PipePlugin: Enhance PipePlugin Jar Directory Compatibility
and Update Format During CN and DN Upgrade (#13006)
add b540636c58a Subscription: fix concurrent issues related to seal batch
& use hybrid realtime extractor for tsfile format topic (#12996)
add 8df0c79df6a Deprecate template APIs in Python SDK (#13009)
add ddd8d4d7533 Fix sonar bugs (#13003)
add cb0765836fd Pipe: Reported the progress of the non-forwarding events
(#13008)
add 957b0c3c87b Update c++ client dependency on windows (#13021)
add 97ce3e8a84a [Region Migration] Add retry when the read region does not
exist (#13001)
add d5e409d8d6f Optimize WALNode.PlanNodeIterator.hasNext (#13013)
add 25ed79191b8 [IoTConsensus] Support hot reload of IoTConsensusConfig
(#13025)
add 2e99b064507 [PY-CLIENT] Use pyproject.toml to configurate python
client project (#13027)
add 95cd06a0a78 Subscription: display the now value as the actual
timestamp when showing the topic (#13020)
add 872aca3138d [region migration] Fix exception capture processing logic
#13034
add ed9204e56b0 avoid start pipe by PipeConsensus (#13031)
add f0363a758cc fix md5 suffix too many (#13033)
add 2a06d48bdc9 PipeConsensus: Fix replication block when leader restart.
(#13028)
add 7f050df3e00 Fix python sdk release shell issue (#13038)
add 99769d1a6a0 Subscription: fix unexpected cancellation of workers
during consumer startup & optimize server-side subscription logs & add
synchronized modifier (#13032)
add 67af4ea13cb Pipe: Fixed the NPE in pipe schema pattern parsing (#13047)
add 376ed3c4957 Subscription: support payload size control fallback
strategy & fix issue where subscription events cannot be auto recycled & fix
issue where the reference count of tablet events for tsfile topic cannot
decrease to zero (#13053)
add 8af04248057 Pipe: Limit the frequency of progress report for
non-forwarding pipe requests to reduce the overhead when sync data between
clusters (#13041)
add e5ba36aba8a Pipe: Enhanced the "isEmpty" judgement for tablets to
prevent NPE (#13051)
add 137ec9aaff6 Fix WALInputStream read ByteBuffer issues (#13059)
add 04ba236ef64 Enhance remove-datanode function
add 36455413b93 fix WALInputStream not closed (#13050)
add 7bf6eea139c Subscription: improve deduplication logic for
PipeRawTabletInsertionEvent (#13061)
add 4c0a073a436 Fixed concurrency bug in IoTConsensus Region migration
#13070
add c60a5079eed Bump jetty.version from 9.4.53.v20231009 to
9.4.55.v20240627 (#13073)
add ef5b1b42e59 Bump org.codehaus.mojo:exec-maven-plugin from 3.1.0 to
3.1.1 (#13075)
add 43e562a2ac1 Pipe/Subscription: Add 'Create If Not Exists' and 'Drop If
Exists' Support for Pipes, Plugins, and Topics (#12969)
add cceaebd5919 Pipe/Subscription: Prevent NPE when some pipe SQL
parameter values are null (#13069)
add 994a3808422 Bump com.nimbusds:nimbus-jose-jwt from 9.37.2 to 9.37.3
(#13072)
add 0a611a8c7c4 [fix] Change IoTConsensusService and PipeConsensusService
from async to sync (#13077)
add b9ed555a3aa Refactor TsFileValidationTool with abstracting the
sequential scan process (#13066)
add 2c10213db2f Improve robustness of removing peer step of region
migration (#13078)
No new revisions were added by this update.
Summary of changes:
.github/workflows/cluster-it-1c1d.yml | 1 +
.github/workflows/cluster-it-1c3d.yml | 1 +
.github/workflows/daily-it.yml | 1 +
.github/workflows/multi-language-client.yml | 31 +-
.github/workflows/pipe-it-2cluster.yml | 1 +
.github/workflows/sonar-codecov.yml | 1 +
.github/workflows/unit-test.yml | 1 +
distribution/src/assembly/all.xml | 26 +-
distribution/src/assembly/cli.xml | 6 +-
distribution/src/assembly/client-cpp.xml | 2 +-
distribution/src/assembly/common-files.xml | 12 +-
distribution/src/assembly/confignode.xml | 10 +-
distribution/src/assembly/datanode.xml | 16 +-
distribution/src/assembly/library-udf.xml | 8 +-
docker/src/main/DockerCompose/do-docker-build.sh | 4 +-
docker/src/main/Dockerfile-0.10.1 | 39 --
docker/src/main/Dockerfile-0.11.4 | 41 --
docker/src/main/Dockerfile-0.12.6-cluster | 53 --
docker/src/main/Dockerfile-0.12.6-node | 45 --
docker/src/main/Dockerfile-0.13.0-cluster | 53 --
docker/src/main/Dockerfile-0.13.3-node | 45 --
.../src/main/Dockerfile-0.13.3-node-multi-platform | 45 --
docker/src/main/Dockerfile-0.8.1 | 40 --
docker/src/main/Dockerfile-0.9.1 | 41 --
docker/src/main/Dockerfile-0.9.1-jre8 | 41 --
docker/src/main/Dockerfile-1.0.0-confignode | 2 +-
docker/src/main/Dockerfile-1.0.0-datanode | 2 +-
docker/src/main/Dockerfile-1.0.0-standalone | 2 +-
docker/src/main/Dockerfile-1c1d | 2 +-
.../apache/iotdb/opcua/ClientExampleRunner.java | 2 +
.../apache/iotdb/SubscriptionSessionExample.java | 52 +-
integration-test/src/assembly/mpp-test.xml | 20 +-
.../it/env/cluster/config/MppCommonConfig.java | 6 +
.../env/cluster/config/MppSharedCommonConfig.java | 7 +
.../it/env/remote/config/RemoteCommonConfig.java | 5 +
.../org/apache/iotdb/itbase/env/CommonConfig.java | 2 +
.../db/it/fill/IoTDBFillWithNewDataTypeIT.java | 145 ++++++
.../apache/iotdb/db/it/schema/IoTDBTagAlterIT.java | 1 +
.../org/apache/iotdb/db/it/schema/IoTDBTagIT.java | 1 +
.../apache/iotdb/db/it/schema/IoTDBTagLimitIT.java | 15 +-
.../it/schema/quota/IoTDBClusterDeviceQuotaIT.java | 3 +-
.../db/it/schema/quota/IoTDBClusterQuotaIT.java | 1 +
.../schema/regionscan/IoTDBActiveRegionScanIT.java | 21 +-
.../regionscan/IoTDBActiveRegionScanIT2.java | 15 +-
.../IoTDBActiveRegionScanWithModsIT.java | 25 +-
.../IoTDBActiveRegionScanWithModsIT2.java | 15 +-
.../regionscan/IoTDBActiveRegionScanWithTTLIT.java | 45 +-
.../iotdb/db/it/selectinto/IoTDBSelectInto2IT.java | 3 +-
.../iotdb/db/it/selectinto/IoTDBSelectInto3IT.java | 3 +-
.../iotdb/db/it/selectinto/IoTDBSelectIntoIT.java | 56 ++-
.../iotdb/pipe/it/autocreate/IoTDBPipeAlterIT.java | 183 ++++++-
.../IoTDBPipeConditionalOperationsIT.java | 228 +++++++++
.../pipe/it/autocreate/IoTDBPipeIdempotentIT.java | 84 ++--
.../it/manual/IoTDBPipeMetaLeaderChangeIT.java | 22 +-
.../session/it/IoTDBSessionSchemaTemplateIT.java | 4 +-
.../it/local/IoTDBSubscriptionBasicIT.java | 63 +++
.../it/triple/AbstractSubscriptionTripleIT.java | 78 +++
.../it/triple/IoTDBSubscriptionSharingIT.java | 560 +++++++++++++++++++++
.../org/apache/iotdb/util/AbstractSchemaIT.java | 15 +-
.../api/customizer/parameter/PipeParameters.java | 25 +-
.../java/org/apache/iotdb/tool/ExportData.java | 136 ++---
.../java/org/apache/iotdb/tool/ExportSchema.java | 2 +-
.../java/org/apache/iotdb/tool/ImportData.java | 2 +-
.../java/org/apache/iotdb/tool/ImportTsFile.java | 5 +
.../org/apache/iotdb/tool/IoTDBDataBackTool.java | 19 +-
.../org/apache/iotdb/tool/WriteDataFileTest.java | 14 +
iotdb-client/client-cpp/pom.xml | 32 +-
iotdb-client/client-py/.gitignore | 5 +-
iotdb-client/client-py/README.md | 97 +---
iotdb-client/client-py/SessionExample.py | 28 --
iotdb-client/client-py/iotdb/Session.py | 67 +++
.../client-py/iotdb/template/InternalNode.py | 41 --
.../client-py/iotdb/template/MeasurementNode.py | 10 +
iotdb-client/client-py/iotdb/template/Template.py | 8 +
.../client-py/iotdb/template/TemplateNode.py | 8 +
iotdb-client/client-py/pom.xml | 15 +-
iotdb-client/client-py/release.sh | 3 +-
iotdb-client/client-py/requirements_dev.txt | 5 +-
.../client-py/{ => resources}/pyproject.toml | 43 +-
iotdb-client/client-py/resources/setup.py | 64 ---
.../{iotdb => tests/integration}/__init__.py | 0
.../tablet_performance_comparison.py | 0
.../{ => integration}/test_aligned_timeseries.py | 0
.../tests/{ => integration}/test_dataframe.py | 0
.../tests/{ => integration}/test_delete_data.py | 0
.../tests/{ => integration}/test_one_device.py | 0
.../tests/{ => integration}/test_session.py | 0
.../tests/{ => integration}/test_session_pool.py | 0
.../tests/{ => integration}/test_tablet.py | 0
.../client-py/tests/{ => integration}/test_todf.py | 0
iotdb-client/client-py/tests/test_template.py | 110 ----
.../client-py/{iotdb => tests/unit}/__init__.py | 0
.../tests/{ => unit}/test_numpy_tablet.py | 0
.../java/org/apache/iotdb/rpc/TSStatusCode.java | 6 +-
.../main/java/org/apache/iotdb/rpc/UrlUtils.java | 9 +-
.../rpc/subscription/config/ConsumerConstant.java | 3 +-
.../iotdb/rpc/subscription/config/TopicConfig.java | 47 +-
.../exception/SubscriptionConnectionException.java | 5 +
.../SubscriptionIdentifierSemanticException.java | 5 +
.../SubscriptionIncompatibleHandlerException.java | 5 +
.../SubscriptionParameterNotValidException.java | 5 +
.../SubscriptionRuntimeCriticalException.java | 5 +
.../exception/SubscriptionRuntimeException.java | 5 +
.../SubscriptionRuntimeNonCriticalException.java | 5 +
.../org/apache/iotdb/session/NodesSupplier.java | 1 +
.../java/org/apache/iotdb/session/Session.java | 1 +
.../apache/iotdb/session/SessionConnection.java | 1 +
.../org/apache/iotdb/session/pool/SessionPool.java | 1 +
.../session/subscription/SubscriptionSession.java | 1 +
.../consumer/SubscriptionConsumer.java | 68 +--
.../consumer/SubscriptionPullConsumer.java | 6 +-
.../consumer/SubscriptionPushConsumer.java | 16 +-
.../org/apache/iotdb/db/qp/sql/IoTDBSqlParser.g4 | 22 +-
.../antlr4/org/apache/iotdb/db/qp/sql/SqlLexer.g4 | 9 +
iotdb-core/confignode/src/assembly/confignode.xml | 4 +-
.../confignode/conf/ConfigNodeDescriptor.java | 2 +
.../iotdb/confignode/manager/ConfigManager.java | 15 +-
.../apache/iotdb/confignode/manager/IManager.java | 11 +-
.../iotdb/confignode/manager/ProcedureManager.java | 13 +-
.../manager/load/balancer/RegionBalancer.java | 7 +-
.../manager/load/balancer/RouteBalancer.java | 3 +-
...artiteGraphReplicationRegionGroupAllocator.java | 256 ++++++++++
.../router/leader/AbstractLeaderBalancer.java | 20 +
.../confignode/manager/load/cache/LoadCache.java | 18 +
.../manager/load/service/EventService.java | 1 +
.../ConsensusGroupStatisticsChangeEvent.java | 8 +
.../coordinator/plugin/PipePluginCoordinator.java | 12 +-
.../runtime/PipeLeaderChangeHandler.java | 5 +-
.../pipe/coordinator/task/PipeTaskCoordinator.java | 33 +-
.../coordinator/task/PipeTaskCoordinatorLock.java | 2 +-
.../pipe/extractor/ConfigRegionListeningQueue.java | 25 +-
.../manager/pipe/metric/PipeConfigNodeMetrics.java | 2 +
.../pipe/metric/PipeConfigNodeReceiverMetrics.java | 169 +++++++
.../metric/PipeConfigNodeRemainingTimeMetrics.java | 1 +
.../metric/PipeConfigRegionConnectorMetrics.java | 1 +
.../pipe/metric/PipeTemporaryMetaMetrics.java | 1 +
.../receiver/protocol/IoTDBConfigNodeReceiver.java | 44 +-
.../subscription/SubscriptionCoordinator.java | 36 +-
.../iotdb/confignode/persistence/ClusterInfo.java | 6 +
.../confignode/persistence/ProcedureInfo.java | 5 +
.../iotdb/confignode/persistence/TTLInfo.java | 6 +
.../confignode/persistence/pipe/PipeInfo.java | 1 +
.../persistence/pipe/PipePluginInfo.java | 53 +-
.../confignode/persistence/pipe/PipeTaskInfo.java | 41 +-
.../schema/CNPhysicalPlanGenerator.java | 1 +
.../persistence/subscription/SubscriptionInfo.java | 12 +-
.../impl/pipe/AbstractOperatePipeProcedureV2.java | 22 +-
.../pipe/plugin/CreatePipePluginProcedure.java | 28 +-
.../impl/pipe/plugin/DropPipePluginProcedure.java | 34 +-
.../runtime/PipeHandleLeaderChangeProcedure.java | 2 +-
.../runtime/PipeHandleMetaChangeProcedure.java | 14 +-
.../impl/pipe/runtime/PipeMetaSyncProcedure.java | 14 +-
.../impl/pipe/task/AlterPipeProcedureV2.java | 64 ++-
.../impl/pipe/task/CreatePipeProcedureV2.java | 19 +-
.../impl/pipe/task/DropPipeProcedureV2.java | 2 +-
.../impl/pipe/task/StartPipeProcedureV2.java | 4 +-
.../impl/pipe/task/StopPipeProcedureV2.java | 2 +-
.../impl/region/AddRegionPeerProcedure.java | 6 +
.../impl/region/RemoveRegionPeerProcedure.java | 6 +
.../impl/schema/DeleteTimeSeriesProcedure.java | 152 +++---
.../AbstractOperateSubscriptionProcedure.java | 31 +-
.../consumer/AlterConsumerGroupProcedure.java | 3 +-
.../runtime/ConsumerGroupMetaSyncProcedure.java | 15 +-
...bstractOperateSubscriptionAndPipeProcedure.java | 6 +-
.../subscription/CreateSubscriptionProcedure.java | 3 +-
.../subscription/DropSubscriptionProcedure.java | 3 +-
.../subscription/topic/AlterTopicProcedure.java | 4 +-
.../subscription/topic/CreateTopicProcedure.java | 27 +-
.../subscription/topic/DropTopicProcedure.java | 3 +-
.../topic/runtime/TopicMetaSyncProcedure.java | 15 +-
.../impl/testonly/NeverFinishProcedure.java | 6 +
.../procedure/store/ProcedureFactory.java | 5 +-
.../confignode/procedure/store/ProcedureType.java | 1 +
.../thrift/ConfigNodeRPCServiceHandler.java | 8 +-
.../thrift/ConfigNodeRPCServiceProcessor.java | 68 ++-
.../ConfigRegionListeningQueueTest.java | 13 +-
.../iotdb/confignode/persistence/PipeInfoTest.java | 2 +-
.../pipe/plugin/CreatePipePluginProcedureTest.java | 2 +-
.../pipe/plugin/DropPipePluginProcedureTest.java | 2 +-
.../impl/pipe/task/AlterPipeProcedureV2Test.java | 10 +-
...reV2Test.java => AlterPipeProcedureV3Test.java} | 19 +-
.../thrift/ConfigNodeRPCServiceProcessorTest.java | 164 ++++++
.../org/apache/iotdb/consensus/IStateMachine.java | 12 +-
.../org/apache/iotdb/consensus/common/Utils.java | 6 +-
.../exception/ConsensusGroupNotExistException.java | 5 +
.../apache/iotdb/consensus/iot/IoTConsensus.java | 36 +-
.../consensus/iot/IoTConsensusServerImpl.java | 86 ++--
.../consensus/iot/log/ConsensusReqReader.java | 2 +-
.../consensus/iot/logdispatcher/LogDispatcher.java | 2 +-
.../consensus/iot/logdispatcher/SyncStatus.java | 60 +--
.../iot/service/IoTConsensusRPCService.java | 14 +-
.../service/IoTConsensusRPCServiceProcessor.java | 192 +++----
.../apache/iotdb/consensus/pipe/PipeConsensus.java | 7 +-
.../consensus/pipe/PipeConsensusServerImpl.java | 5 +
.../pipe/service/PipeConsensusRPCService.java | 15 +-
.../service/PipeConsensusRPCServiceProcessor.java | 64 +--
.../iotdb/consensus/ratis/SnapshotStorage.java | 10 +-
.../apache/iotdb/consensus/ratis/utils/Utils.java | 2 +-
.../apache/iotdb/consensus/ratis/SnapshotTest.java | 5 +-
.../apache/iotdb/consensus/ratis/TestUtils.java | 11 +-
iotdb-core/datanode/pom.xml | 2 +-
iotdb-core/datanode/src/assembly/server.xml | 10 +-
.../java/org/apache/iotdb/db/conf/IoTDBConfig.java | 18 +-
.../org/apache/iotdb/db/conf/IoTDBDescriptor.java | 8 +
.../dataregion/DataExecutionVisitor.java | 6 +
.../dataregion/DataRegionStateMachine.java | 20 +-
.../schemaregion/SchemaExecutionVisitor.java | 168 ++++---
.../batch/PipeTabletEventTsFileBatch.java | 7 +-
.../request/PipeTransferTabletRawReq.java | 7 +
.../protocol/opcua/OpcUaServerBuilder.java | 139 ++---
.../pipeconsensus/PipeConsensusAsyncConnector.java | 11 +-
.../async/IoTDBDataRegionAsyncConnector.java | 1 +
.../common/tablet/PipeRawTabletInsertionEvent.java | 13 +-
.../common/tsfile/PipeTsFileInsertionEvent.java | 21 +-
.../event/realtime/PipeRealtimeEventFactory.java | 17 +-
.../PipeHistoricalDataRegionTsFileExtractor.java | 1 +
.../realtime/PipeRealtimeDataRegionExtractor.java | 16 +-
.../PipeRealtimeDataRegionHeartbeatExtractor.java | 3 +
.../PipeRealtimeDataRegionHybridExtractor.java | 8 +-
.../PipeRealtimeDataRegionLogExtractor.java | 8 +-
.../PipeRealtimeDataRegionTsFileExtractor.java | 8 +-
.../realtime/assigner/PipeDataRegionAssigner.java | 41 +-
.../schemaregion/PipePlanPatternParseVisitor.java | 16 +-
.../schemaregion/SchemaRegionListeningQueue.java | 22 +-
.../iotdb/db/pipe/metric/PipeDataNodeMetrics.java | 2 +
.../pipe/metric/PipeDataNodeReceiverMetrics.java | 333 ++++++++++++
.../PipeDataNodeRemainingEventAndTimeMetrics.java | 1 +
.../metric/PipeDataRegionConnectorMetrics.java | 1 +
.../metric/PipeDataRegionExtractorMetrics.java | 1 +
.../iotdb/db/pipe/metric/PipeProcessorMetrics.java | 1 +
.../metric/PipeSchemaRegionConnectorMetrics.java | 1 +
.../metric/PipeSchemaRegionExtractorMetrics.java | 1 +
.../metric/PipeSchemaRegionListenerMetrics.java | 1 +
.../pipe/metric/PipeWALInsertNodeCacheMetrics.java | 1 +
.../pipeconsensus/PipeConsensusProcessor.java | 7 +-
.../pipeconsensus/PipeConsensusReceiver.java | 146 +++---
.../protocol/thrift/IoTDBDataNodeReceiver.java | 112 +++--
.../visitor/PipeStatementPatternParseVisitor.java | 1 +
.../visitor/PipeStatementToBatchVisitor.java | 1 +
.../pipe/resource/wal/PipeWALResourceManager.java | 1 +
.../pipe/task/connection/PipeEventCollector.java | 26 +-
.../iotdb/db/protocol/client/ConfigNodeClient.java | 18 +-
.../protocol/thrift/impl/ClientRPCServiceImpl.java | 1 +
.../impl/DataNodeInternalRPCServiceImpl.java | 51 +-
.../execution/executor/RegionReadExecutor.java | 10 +
.../execution/executor/RegionWriteExecutor.java | 81 +--
.../execution/load/LoadTsFileManager.java | 26 +-
.../execution/load/LoadTsFileRateLimiter.java | 2 +-
.../operator/process/AbstractIntoOperator.java | 12 +
.../operator/process/FilterAndProjectOperator.java | 3 +-
.../analyze/cache/partition/PartitionCache.java | 23 +-
.../analyze/cache/schema/SchemaCacheEntry.java | 1 +
.../cache/schema/TimeSeriesSchemaCache.java | 3 +-
.../dualkeycache/impl/FIFOCacheEntryManager.java | 2 +
.../dualkeycache/impl/LRUCacheEntryManager.java | 2 +
.../config/executor/ClusterConfigTaskExecutor.java | 73 ++-
.../config/executor/IConfigTaskExecutor.java | 3 +-
.../config/metadata/DropPipePluginTask.java | 6 +-
.../db/queryengine/plan/parser/ASTVisitor.java | 33 +-
.../plan/parser/StatementGenerator.java | 2 +-
.../plan/planner/OperatorTreeGenerator.java | 20 +-
.../plan/planner/plan/node/PlanNodeType.java | 2 +
.../ContinuousSameSearchIndexSeparatorNode.java} | 27 +-
.../scheduler/FragmentInstanceDispatcherImpl.java | 129 +++--
.../plan/scheduler/SimpleQueryTerminator.java | 96 ++--
.../plan/statement/literal/BooleanLiteral.java | 4 +-
.../plan/statement/literal/DoubleLiteral.java | 3 +-
.../plan/statement/literal/Literal.java | 4 +
.../plan/statement/literal/LongLiteral.java | 26 +-
.../plan/statement/literal/StringLiteral.java | 21 +-
.../metadata/pipe/AlterPipeStatement.java | 27 +
.../metadata/pipe/CreatePipePluginStatement.java | 9 +-
.../metadata/pipe/CreatePipeStatement.java | 9 +
.../metadata/pipe/DropPipePluginStatement.java | 18 +-
.../statement/metadata/pipe/DropPipeStatement.java | 9 +
.../subscription/CreateTopicStatement.java | 10 +-
.../metadata/subscription/DropTopicStatement.java | 9 +
.../schemaengine/schemaregion/ISchemaRegion.java | 20 +-
.../schemaregion/impl/SchemaRegionMemoryImpl.java | 149 ++++--
.../schemaregion/impl/SchemaRegionPBTreeImpl.java | 158 ++++--
.../mtree/impl/mem/MTreeBelowSGMemoryImpl.java | 120 +++--
.../mtree/impl/pbtree/MTreeBelowSGCachedImpl.java | 131 +++--
.../mtree/impl/pbtree/mnode/ICachedMNode.java | 1 +
.../impl/pbtree/mnode/info/CacheMNodeInfo.java | 1 +
.../mtree/impl/pbtree/schemafile/SchemaFile.java | 2 +
.../mtree/loader/MNodeFactoryLoader.java | 4 +
.../write/req/SchemaRegionWritePlanFactory.java | 2 +-
.../req/impl/CreateAlignedTimeSeriesPlanImpl.java | 45 +-
.../write/req/impl/CreateTimeSeriesPlanImpl.java | 9 +
.../java/org/apache/iotdb/db/service/DataNode.java | 2 +-
.../db/service/DataNodeServerCommandLine.java | 145 +++---
.../metrics/IoTDBInternalLocalReporter.java | 101 +++-
.../iotdb/db/storageengine/StorageEngine.java | 5 +
.../db/storageengine/dataregion/DataRegion.java | 18 +
.../CompactionFileCountExceededException.java | 1 +
.../CompactionLastTimeCheckFailedException.java | 1 +
.../CompactionMemoryNotEnoughException.java | 1 +
.../CompactionValidationFailedException.java | 1 +
.../performer/impl/FastCompactionPerformer.java | 9 +-
.../execute/task/SettleCompactionTask.java | 10 +-
.../execute/utils/MultiTsFileDeviceIterator.java | 3 +-
.../execute/utils/reader/PointPriorityReader.java | 12 +-
.../compaction/io/CompactionTsFileReader.java | 1 +
.../repair/RepairTimePartitionScanTask.java | 1 +
.../repair/UnsortedFileRepairTaskScheduler.java | 1 +
.../schedule/CompactionScheduleTaskManager.java | 28 +-
.../schedule/CompactionScheduleTaskWorker.java | 1 +
.../compaction/schedule/CompactionTaskManager.java | 2 +
.../compaction/schedule/TTLScheduleTask.java | 1 +
.../dataregion/compaction/tool/PrintUtil.java | 2 +-
.../compaction/tool/TimePartitionProcessTask.java | 8 +-
.../tool/TimePartitionProcessWorker.java | 9 +-
.../dataregion/flush/CompressionRatio.java | 46 +-
.../memtable/AlignedWritableMemChunk.java | 13 -
.../dataregion/memtable/WritableMemChunk.java | 6 -
.../dataregion/modification/ModificationFile.java | 2 +-
.../io/LocalTextModificationAccessor.java | 1 +
.../dataregion/snapshot/SnapshotTaker.java | 28 ++
.../dataregion/tsfile/TsFileResource.java | 11 +
.../storageengine/dataregion/wal/WALManager.java | 2 +-
.../dataregion/wal/buffer/AbstractWALBuffer.java | 4 +-
.../dataregion/wal/buffer/IWALBuffer.java | 4 +-
.../dataregion/wal/buffer/WALBuffer.java | 25 +-
.../dataregion/wal/buffer/WALEntry.java | 3 +
.../dataregion/wal/buffer/WALEntryType.java | 1 +
.../dataregion/wal/buffer/WALInfoEntry.java | 1 +
.../wal/checkpoint/CheckpointManager.java | 3 +-
...rException.java => BrokenWALFileException.java} | 12 +-
.../storageengine/dataregion/wal/io/LogWriter.java | 18 +-
.../dataregion/wal/io/WALByteBufReader.java | 40 +-
.../dataregion/wal/io/WALFileVersion.java | 54 +-
.../dataregion/wal/io/WALInputStream.java | 57 ++-
.../dataregion/wal/io/WALMetaData.java | 70 +--
.../storageengine/dataregion/wal/io/WALWriter.java | 16 +-
.../dataregion/wal/node/IWALNode.java | 4 +
.../dataregion/wal/node/WALFakeNode.java | 7 +
.../storageengine/dataregion/wal/node/WALNode.java | 189 +++----
.../dataregion/wal/recover/WALNodeRecoverTask.java | 11 +
.../dataregion/wal/recover/WALRepairWriter.java | 30 +-
.../file/UnsealedTsFileRecoverPerformer.java | 4 +-
.../dataregion/wal/utils/WALInsertNodeCache.java | 5 +-
.../rescon/disk/DirectoryChecker.java | 2 +
.../rescon/disk/strategy/SequenceStrategy.java | 24 +-
.../db/storageengine/rescon/memory/SystemInfo.java | 20 +-
.../agent/SubscriptionBrokerAgent.java | 30 +-
.../agent/SubscriptionConsumerAgent.java | 12 +-
.../subscription/agent/SubscriptionTopicAgent.java | 10 +-
.../db/subscription/broker/SubscriptionBroker.java | 112 +++--
.../broker/SubscriptionPrefetchingQueue.java | 111 ++--
.../broker/SubscriptionPrefetchingTabletQueue.java | 60 +--
.../broker/SubscriptionPrefetchingTsFileQueue.java | 242 ++++-----
.../TsFileDeduplicationBlockingPendingQueue.java | 48 +-
.../db/subscription/event/SubscriptionEvent.java | 18 +-
.../batch/SubscriptionPipeTabletEventBatch.java | 87 +++-
.../batch/SubscriptionPipeTsFileEventBatch.java | 77 ++-
.../receiver/SubscriptionReceiverV1.java | 131 +++--
.../db/tools/schema/PBTreeFileSketchTool.java | 11 +-
.../iotdb/db/tools/utils/TsFileSequenceScan.java | 168 +++++++
.../iotdb/db/tools/utils/TsFileValidationScan.java | 397 +++++++++++++++
.../db/tools/validate/TsFileValidationTool.java | 422 ++--------------
.../org/apache/iotdb/db/utils/CommonUtils.java | 39 +-
.../iotdb/db/utils/constant/TestConstant.java | 8 +-
.../schemaRegion/AbstractSchemaRegionTest.java | 9 +-
.../schemaRegion/SchemaRegionAliasAndTagTest.java | 9 +-
.../schemaRegion/SchemaRegionBasicTest.java | 250 +++++++--
.../schemaRegion/SchemaRegionManagementTest.java | 4 +-
.../SchemaRegionSimpleRecoverTest.java | 135 ++++-
.../schemaRegion/SchemaRegionTemplateTest.java | 15 +-
.../schemaRegion/SchemaRegionTestUtil.java | 52 +-
.../schemaRegion/SchemaStatisticsTest.java | 146 +++---
.../extractor/SchemaRegionListeningQueueTest.java | 15 +-
.../db/service/DataNodeServerCommandLineTest.java | 218 ++++++++
.../compaction/AbstractCompactionTest.java | 4 +-
.../compaction/CompactionTaskManagerTest.java | 5 +-
.../FastCrossCompactionPerformerTest.java | 4 +-
...eCompactionWithFastPerformerValidationTest.java | 2 +-
...actionWithReadPointPerformerValidationTest.java | 2 +-
.../settle/SettleCompactionRecoverTest.java | 4 +-
.../dataregion/flush/CompressionRatioTest.java | 36 ++
.../wal/compression/WALCompressionTest.java | 45 +-
.../dataregion/wal/io/WALFileTest.java | 17 +
.../wal/recover/WALRepairWriterTest.java | 7 +-
.../wal/utils/WALInsertNodeCacheTest.java | 4 +-
.../org/apache/iotdb/db/tools/MLogParserTest.java | 2 +-
.../iotdb/db/tools/TsFileValidationScanTest.java | 203 ++++++++
.../apache/iotdb/db/utils/DateTimeUtilsTest.java | 63 ++-
.../db/utils/SchemaRegionSnapshotParserTest.java | 10 +-
.../reporter/iotdb/IoTDBSessionReporter.java | 2 +-
.../conf/iotdb-system.properties.template | 2 +-
.../apache/iotdb/commons/conf/CommonConfig.java | 36 +-
.../iotdb/commons/conf/CommonDescriptor.java | 17 +-
.../commons/consensus/index/ProgressIndex.java | 5 +
.../PipeRuntimeConnectorCriticalException.java | 5 +
.../pipe/PipeRuntimeNonCriticalException.java | 5 +
.../PipeRuntimeOutOfMemoryCriticalException.java | 5 +
.../iotdb/commons/pipe/config/PipeConfig.java | 17 +-
.../pipe/connector/limiter/GlobalRateLimiter.java | 2 +-
.../connector/limiter/PipeEndPointRateLimiter.java | 3 +-
.../connector/protocol/IoTDBAirGapConnector.java | 1 +
.../pipe/plugin/meta/PipePluginMetaKeeper.java | 2 +-
.../service/PipePluginExecutableManager.java | 18 +-
.../commons/pipe/progress/PipeEventCommitter.java | 1 +
.../task/subtask/PipeAbstractConnectorSubtask.java | 1 +
.../pipe/task/subtask/PipeReportableSubtask.java | 2 +-
.../schema/node/role/IMeasurementMNode.java | 2 +-
.../commons/schema/node/utils/IMNodeFactory.java | 1 +
.../iotdb/commons/service/ThriftService.java | 15 +-
.../commons/service/metric/GcTimeAlerter.java | 2 +
.../iotdb/commons/service/metric/enums/Metric.java | 2 +
.../subscription/config/SubscriptionConfig.java | 5 +
.../commons/subscription/meta/topic/TopicMeta.java | 4 +-
.../apache/iotdb/commons/udf/builtin/UDTFAbs.java | 1 +
.../org/apache/iotdb/commons/utils/IOUtils.java | 32 ++
.../apache/iotdb/commons/utils/JVMCommonUtils.java | 30 +-
.../src/main/thrift/confignode.thrift | 23 +
.../src/main/thrift/datanode.thrift | 1 +
.../org/apache/iotdb/library/dlearn/UDTFAR.java | 1 +
.../apache/iotdb/library/dmatch/UDTFPtnSym.java | 1 +
.../apache/iotdb/library/dprofile/UDTFSample.java | 1 +
pom.xml | 24 +-
420 files changed, 8618 insertions(+), 3940 deletions(-)
delete mode 100644 docker/src/main/Dockerfile-0.10.1
delete mode 100644 docker/src/main/Dockerfile-0.11.4
delete mode 100644 docker/src/main/Dockerfile-0.12.6-cluster
delete mode 100644 docker/src/main/Dockerfile-0.12.6-node
delete mode 100644 docker/src/main/Dockerfile-0.13.0-cluster
delete mode 100644 docker/src/main/Dockerfile-0.13.3-node
delete mode 100644 docker/src/main/Dockerfile-0.13.3-node-multi-platform
delete mode 100644 docker/src/main/Dockerfile-0.8.1
delete mode 100644 docker/src/main/Dockerfile-0.9.1
delete mode 100644 docker/src/main/Dockerfile-0.9.1-jre8
create mode 100644
integration-test/src/test/java/org/apache/iotdb/db/it/fill/IoTDBFillWithNewDataTypeIT.java
create mode 100644
integration-test/src/test/java/org/apache/iotdb/pipe/it/autocreate/IoTDBPipeConditionalOperationsIT.java
create mode 100644
integration-test/src/test/java/org/apache/iotdb/subscription/it/triple/AbstractSubscriptionTripleIT.java
create mode 100644
integration-test/src/test/java/org/apache/iotdb/subscription/it/triple/IoTDBSubscriptionSharingIT.java
delete mode 100644 iotdb-client/client-py/iotdb/template/InternalNode.py
rename iotdb-client/client-py/{ => resources}/pyproject.toml (51%)
delete mode 100644 iotdb-client/client-py/resources/setup.py
copy iotdb-client/client-py/{iotdb => tests/integration}/__init__.py (100%)
rename iotdb-client/client-py/tests/{ =>
integration}/tablet_performance_comparison.py (100%)
rename iotdb-client/client-py/tests/{ =>
integration}/test_aligned_timeseries.py (100%)
rename iotdb-client/client-py/tests/{ => integration}/test_dataframe.py (100%)
rename iotdb-client/client-py/tests/{ => integration}/test_delete_data.py
(100%)
rename iotdb-client/client-py/tests/{ => integration}/test_one_device.py (100%)
rename iotdb-client/client-py/tests/{ => integration}/test_session.py (100%)
rename iotdb-client/client-py/tests/{ => integration}/test_session_pool.py
(100%)
rename iotdb-client/client-py/tests/{ => integration}/test_tablet.py (100%)
rename iotdb-client/client-py/tests/{ => integration}/test_todf.py (100%)
delete mode 100644 iotdb-client/client-py/tests/test_template.py
copy iotdb-client/client-py/{iotdb => tests/unit}/__init__.py (100%)
rename iotdb-client/client-py/tests/{ => unit}/test_numpy_tablet.py (100%)
create mode 100644
iotdb-core/confignode/src/main/java/org/apache/iotdb/confignode/manager/load/balancer/region/PartiteGraphReplicationRegionGroupAllocator.java
create mode 100644
iotdb-core/confignode/src/main/java/org/apache/iotdb/confignode/manager/pipe/metric/PipeConfigNodeReceiverMetrics.java
copy
iotdb-core/confignode/src/test/java/org/apache/iotdb/confignode/procedure/impl/pipe/task/{AlterPipeProcedureV2Test.java
=> AlterPipeProcedureV3Test.java} (78%)
create mode 100644
iotdb-core/confignode/src/test/java/org/apache/iotdb/confignode/service/thrift/ConfigNodeRPCServiceProcessorTest.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/metric/PipeDataNodeReceiverMetrics.java
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/{storageengine/dataregion/wal/utils/listener/WALFlushListener.java
=>
queryengine/plan/planner/plan/node/write/ContinuousSameSearchIndexSeparatorNode.java}
(55%)
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/wal/exception/{WALRecoverException.java
=> BrokenWALFileException.java} (78%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/tools/utils/TsFileSequenceScan.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/tools/utils/TsFileValidationScan.java
create mode 100644
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/service/DataNodeServerCommandLineTest.java
create mode 100644
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/tools/TsFileValidationScanTest.java
