This is an automated email from the ASF dual-hosted git repository.
jackietien pushed a change to branch ty/TableModelGrammar
in repository https://gitbox.apache.org/repos/asf/iotdb.git
from 647b7f7c30a Support not/not null/in/like filter for id/attribute in
schema region & Multi filters for the same id & Null value attributes &
PartitionKeys for determined id query
new 27ef7a8613b Fix concurrency problem in Memory Control (#12984)
new 5f8197e04aa Ensure the flush total points statistic function works
correctly when enable_auto_create_schema is false #12990
new 3fd0c4c50fe Pipe/PipeConsensus: Fix invalid retry count in report &
enhance log in pipe consensus (#12989)
new 03daa3472f2 Replaces the internal maven property with an officially
supported alternative (#12982)
new 4fc150bfacf Reduce error log when compaction interrupted (#12985)
new cc6da4e086f Remove UNKNOWN from WALFileVersion which fails recovering
V1 WAL file (#12986)
new 84d986de5cd Pipe: Fix empty tablets generated by pattern parsing on
sender side may cause NPE on receiver side (#12994)
new 424c37f3e6a Allows for unit tests to be run in parallel (#12980)
new fd1528baa1f Pipe: add ops/latency metrics for different types of pipe
operations on receivers (DN / CN) (#12927)
new cd8fde7d56d Change some error log to warn during region migration
(#12992)
new 9f857685bdc Rectify the input parameters of LeaderBalancer (#12998)
new db22199acdb PipePlugin: Enhance PipePlugin Jar Directory Compatibility
and Update Format During CN and DN Upgrade (#13006)
new cac4ce0d79c Subscription: fix concurrent issues related to seal batch
& use hybrid realtime extractor for tsfile format topic (#12996)
new 02f7b8b6e7d Deprecate template APIs in Python SDK (#13009)
new b5120f05201 Fix sonar bugs (#13003)
new d78326a01c5 Pipe: Reported the progress of the non-forwarding events
(#13008)
new 431450066e8 Update c++ client dependency on windows (#13021)
new 5a889412bdf [Region Migration] Add retry when the read region does not
exist (#13001)
new 6d009c29705 Optimize WALNode.PlanNodeIterator.hasNext (#13013)
new 950272b67ca [IoTConsensus] Support hot reload of IoTConsensusConfig
(#13025)
new f82a43a6f96 [PY-CLIENT] Use pyproject.toml to configurate python
client project (#13027)
new c064a8308ac Subscription: display the now value as the actual
timestamp when showing the topic (#13020)
new 8ce699db53a [region migration] Fix exception capture processing logic
#13034
new 6e709c0048f avoid start pipe by PipeConsensus (#13031)
new 3af5813e649 fix md5 suffix too many (#13033)
new 891c09c992e PipeConsensus: Fix replication block when leader restart.
(#13028)
new c80450c9e20 Fix python sdk release shell issue (#13038)
new 80f93ce4f04 fix compile error
The 28 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/multi-language-client.yml | 11 +-
distribution/src/assembly/all.xml | 26 +-
distribution/src/assembly/cli.xml | 6 +-
distribution/src/assembly/client-cpp.xml | 2 +-
distribution/src/assembly/common-files.xml | 12 +-
distribution/src/assembly/confignode.xml | 10 +-
distribution/src/assembly/datanode.xml | 16 +-
distribution/src/assembly/library-udf.xml | 8 +-
.../apache/iotdb/opcua/ClientExampleRunner.java | 2 +
.../apache/iotdb/SubscriptionSessionExample.java | 52 +-
integration-test/src/assembly/mpp-test.xml | 20 +-
.../it/triple/AbstractSubscriptionTripleIT.java | 78 +++
.../it/triple/IoTDBSubscriptionSharingIT.java | 561 +++++++++++++++++++++
.../java/org/apache/iotdb/tool/ExportData.java | 134 ++---
.../java/org/apache/iotdb/tool/ImportData.java | 2 +-
.../java/org/apache/iotdb/tool/ImportTsFile.java | 5 +
.../org/apache/iotdb/tool/IoTDBDataBackTool.java | 19 +-
.../org/apache/iotdb/tool/WriteDataFileTest.java | 14 +
iotdb-client/client-py/.gitignore | 5 +-
iotdb-client/client-py/README.md | 97 +---
iotdb-client/client-py/SessionExample.py | 28 -
iotdb-client/client-py/iotdb/Session.py | 67 +++
.../client-py/iotdb/template/InternalNode.py | 41 --
.../client-py/iotdb/template/MeasurementNode.py | 10 +
iotdb-client/client-py/iotdb/template/Template.py | 8 +
.../client-py/iotdb/template/TemplateNode.py | 8 +
iotdb-client/client-py/pom.xml | 8 +-
iotdb-client/client-py/release.sh | 3 +-
iotdb-client/client-py/requirements_dev.txt | 5 +-
.../client-py/{ => resources}/pyproject.toml | 43 +-
iotdb-client/client-py/resources/setup.py | 64 ---
.../{iotdb => tests/integration}/__init__.py | 0
.../tablet_performance_comparison.py | 0
.../{ => integration}/test_aligned_timeseries.py | 0
.../tests/{ => integration}/test_dataframe.py | 0
.../tests/{ => integration}/test_delete_data.py | 0
.../tests/{ => integration}/test_one_device.py | 0
.../tests/{ => integration}/test_session.py | 0
.../tests/{ => integration}/test_session_pool.py | 0
.../tests/{ => integration}/test_tablet.py | 0
.../client-py/tests/{ => integration}/test_todf.py | 0
iotdb-client/client-py/tests/test_template.py | 110 ----
.../client-py/{iotdb => tests/unit}/__init__.py | 0
.../tests/{ => unit}/test_numpy_tablet.py | 0
.../java/org/apache/iotdb/rpc/TSStatusCode.java | 4 +-
.../iotdb/rpc/subscription/config/TopicConfig.java | 47 +-
.../exception/SubscriptionConnectionException.java | 5 +
.../SubscriptionIdentifierSemanticException.java | 5 +
.../SubscriptionIncompatibleHandlerException.java | 5 +
.../SubscriptionParameterNotValidException.java | 5 +
.../SubscriptionRuntimeCriticalException.java | 5 +
.../exception/SubscriptionRuntimeException.java | 5 +
.../SubscriptionRuntimeNonCriticalException.java | 5 +
.../org/apache/iotdb/session/NodesSupplier.java | 1 +
.../java/org/apache/iotdb/session/Session.java | 1 +
.../apache/iotdb/session/SessionConnection.java | 1 +
.../org/apache/iotdb/session/pool/SessionPool.java | 1 +
.../consumer/SubscriptionConsumer.java | 43 +-
iotdb-core/confignode/src/assembly/confignode.xml | 4 +-
.../confignode/conf/ConfigNodeDescriptor.java | 2 +
.../manager/load/balancer/RouteBalancer.java | 3 +-
.../router/leader/AbstractLeaderBalancer.java | 20 +
.../confignode/manager/load/cache/LoadCache.java | 18 +
.../manager/load/service/EventService.java | 1 +
.../ConsensusGroupStatisticsChangeEvent.java | 8 +
.../manager/pipe/metric/PipeConfigNodeMetrics.java | 2 +
.../pipe/metric/PipeConfigNodeReceiverMetrics.java | 169 +++++++
.../metric/PipeConfigNodeRemainingTimeMetrics.java | 1 +
.../metric/PipeConfigRegionConnectorMetrics.java | 1 +
.../pipe/metric/PipeTemporaryMetaMetrics.java | 1 +
.../receiver/protocol/IoTDBConfigNodeReceiver.java | 44 +-
.../iotdb/confignode/persistence/ClusterInfo.java | 6 +
.../confignode/persistence/ProcedureInfo.java | 5 +
.../iotdb/confignode/persistence/TTLInfo.java | 6 +
.../confignode/persistence/pipe/PipeInfo.java | 1 +
.../persistence/pipe/PipePluginInfo.java | 27 +-
.../schema/CNPhysicalPlanGenerator.java | 1 +
.../impl/region/AddRegionPeerProcedure.java | 6 +
.../impl/region/RemoveRegionPeerProcedure.java | 6 +
.../subscription/topic/CreateTopicProcedure.java | 6 +-
.../impl/testonly/NeverFinishProcedure.java | 6 +
.../org/apache/iotdb/consensus/IStateMachine.java | 3 +-
.../org/apache/iotdb/consensus/common/Utils.java | 6 +-
.../exception/ConsensusGroupNotExistException.java | 5 +
.../apache/iotdb/consensus/iot/IoTConsensus.java | 6 +-
.../consensus/iot/IoTConsensusServerImpl.java | 37 +-
.../consensus/iot/logdispatcher/LogDispatcher.java | 2 +-
.../apache/iotdb/consensus/pipe/PipeConsensus.java | 5 +-
.../consensus/pipe/PipeConsensusServerImpl.java | 5 +
.../iotdb/consensus/ratis/SnapshotStorage.java | 10 +-
.../apache/iotdb/consensus/ratis/utils/Utils.java | 2 +-
.../apache/iotdb/consensus/ratis/SnapshotTest.java | 5 +-
.../apache/iotdb/consensus/ratis/TestUtils.java | 11 +-
iotdb-core/datanode/pom.xml | 2 +-
iotdb-core/datanode/src/assembly/server.xml | 10 +-
.../java/org/apache/iotdb/db/conf/IoTDBConfig.java | 14 +-
.../org/apache/iotdb/db/conf/IoTDBDescriptor.java | 8 +
.../dataregion/DataRegionStateMachine.java | 6 +-
.../batch/PipeTabletEventTsFileBatch.java | 7 +-
.../request/PipeTransferTabletRawReq.java | 5 +
.../protocol/opcua/OpcUaServerBuilder.java | 139 ++---
.../pipeconsensus/PipeConsensusAsyncConnector.java | 3 +-
.../async/IoTDBDataRegionAsyncConnector.java | 1 +
.../event/realtime/PipeRealtimeEventFactory.java | 15 +-
.../realtime/PipeRealtimeDataRegionExtractor.java | 16 +-
.../PipeRealtimeDataRegionHeartbeatExtractor.java | 3 +
.../PipeRealtimeDataRegionHybridExtractor.java | 8 +-
.../PipeRealtimeDataRegionLogExtractor.java | 8 +-
.../PipeRealtimeDataRegionTsFileExtractor.java | 8 +-
.../realtime/assigner/PipeDataRegionAssigner.java | 24 +-
.../iotdb/db/pipe/metric/PipeDataNodeMetrics.java | 2 +
.../pipe/metric/PipeDataNodeReceiverMetrics.java | 333 ++++++++++++
.../PipeDataNodeRemainingEventAndTimeMetrics.java | 1 +
.../metric/PipeDataRegionConnectorMetrics.java | 1 +
.../metric/PipeDataRegionExtractorMetrics.java | 1 +
.../iotdb/db/pipe/metric/PipeProcessorMetrics.java | 1 +
.../metric/PipeSchemaRegionConnectorMetrics.java | 1 +
.../metric/PipeSchemaRegionExtractorMetrics.java | 1 +
.../metric/PipeSchemaRegionListenerMetrics.java | 1 +
.../pipe/metric/PipeWALInsertNodeCacheMetrics.java | 1 +
.../pipeconsensus/PipeConsensusReceiver.java | 143 +++---
.../protocol/thrift/IoTDBDataNodeReceiver.java | 112 ++--
.../pipe/resource/wal/PipeWALResourceManager.java | 1 +
.../pipe/task/connection/PipeEventCollector.java | 26 +-
.../protocol/thrift/impl/ClientRPCServiceImpl.java | 1 +
.../impl/DataNodeInternalRPCServiceImpl.java | 1 +
.../execution/executor/RegionReadExecutor.java | 10 +
.../operator/process/FilterAndProjectOperator.java | 3 +-
.../analyze/cache/schema/SchemaCacheEntry.java | 1 +
.../cache/schema/TimeSeriesSchemaCache.java | 3 +-
.../dualkeycache/impl/FIFOCacheEntryManager.java | 2 +
.../dualkeycache/impl/LRUCacheEntryManager.java | 2 +
.../config/executor/ClusterConfigTaskExecutor.java | 30 +-
.../scheduler/FragmentInstanceDispatcherImpl.java | 65 ++-
.../schemaregion/impl/SchemaRegionMemoryImpl.java | 3 +-
.../schemaregion/impl/SchemaRegionPBTreeImpl.java | 3 +-
.../impl/pbtree/mnode/info/CacheMNodeInfo.java | 1 +
.../mtree/impl/pbtree/schemafile/SchemaFile.java | 2 +
.../mtree/loader/MNodeFactoryLoader.java | 4 +
.../metrics/IoTDBInternalLocalReporter.java | 100 +++-
.../iotdb/db/storageengine/StorageEngine.java | 5 +
.../db/storageengine/dataregion/DataRegion.java | 1 +
.../CompactionFileCountExceededException.java | 1 +
.../CompactionLastTimeCheckFailedException.java | 1 +
.../CompactionMemoryNotEnoughException.java | 1 +
.../CompactionValidationFailedException.java | 1 +
.../performer/impl/FastCompactionPerformer.java | 9 +-
.../execute/task/SettleCompactionTask.java | 10 +-
.../execute/utils/MultiTsFileDeviceIterator.java | 3 +-
.../compaction/io/CompactionTsFileReader.java | 1 +
.../repair/RepairTimePartitionScanTask.java | 1 +
.../repair/UnsortedFileRepairTaskScheduler.java | 1 +
.../schedule/CompactionScheduleTaskManager.java | 28 +-
.../schedule/CompactionScheduleTaskWorker.java | 1 +
.../compaction/schedule/CompactionTaskManager.java | 2 +
.../compaction/schedule/TTLScheduleTask.java | 1 +
.../dataregion/compaction/tool/PrintUtil.java | 2 +-
.../compaction/tool/TimePartitionProcessTask.java | 8 +-
.../tool/TimePartitionProcessWorker.java | 9 +-
.../dataregion/modification/ModificationFile.java | 2 +-
.../io/LocalTextModificationAccessor.java | 1 +
.../dataregion/snapshot/SnapshotTaker.java | 5 +-
.../storageengine/dataregion/wal/WALManager.java | 2 +-
.../dataregion/wal/buffer/WALBuffer.java | 5 +-
.../storageengine/dataregion/wal/io/LogWriter.java | 9 +-
.../dataregion/wal/io/WALFileVersion.java | 54 +-
.../dataregion/wal/io/WALInputStream.java | 24 +-
.../dataregion/wal/io/WALMetaData.java | 18 +-
.../storageengine/dataregion/wal/io/WALWriter.java | 14 +-
.../storageengine/dataregion/wal/node/WALNode.java | 180 +++----
.../dataregion/wal/recover/WALRepairWriter.java | 30 +-
.../file/UnsealedTsFileRecoverPerformer.java | 4 +-
.../rescon/disk/DirectoryChecker.java | 2 +
.../db/storageengine/rescon/memory/SystemInfo.java | 20 +-
.../broker/SubscriptionPrefetchingQueue.java | 56 +-
.../broker/SubscriptionPrefetchingTabletQueue.java | 58 +--
.../broker/SubscriptionPrefetchingTsFileQueue.java | 68 +--
.../batch/SubscriptionPipeTabletEventBatch.java | 87 +++-
.../batch/SubscriptionPipeTsFileEventBatch.java | 68 ++-
.../db/tools/schema/PBTreeFileSketchTool.java | 11 +-
.../iotdb/db/utils/constant/TestConstant.java | 8 +-
.../schemaRegion/SchemaRegionAliasAndTagTest.java | 1 +
.../settle/SettleCompactionRecoverTest.java | 4 +-
.../wal/compression/WALCompressionTest.java | 13 +-
.../dataregion/wal/io/WALFileTest.java | 12 +-
.../wal/recover/WALRepairWriterTest.java | 7 +-
.../wal/utils/WALInsertNodeCacheTest.java | 4 +-
.../apache/iotdb/commons/conf/CommonConfig.java | 2 +-
.../commons/consensus/index/ProgressIndex.java | 5 +
.../PipeRuntimeConnectorCriticalException.java | 5 +
.../pipe/PipeRuntimeNonCriticalException.java | 5 +
.../PipeRuntimeOutOfMemoryCriticalException.java | 5 +
.../connector/limiter/PipeEndPointRateLimiter.java | 1 +
.../connector/protocol/IoTDBAirGapConnector.java | 1 +
.../pipe/plugin/meta/PipePluginMetaKeeper.java | 2 +-
.../service/PipePluginExecutableManager.java | 18 +-
.../commons/pipe/progress/PipeEventCommitter.java | 1 +
.../task/subtask/PipeAbstractConnectorSubtask.java | 1 +
.../pipe/task/subtask/PipeReportableSubtask.java | 2 +-
.../commons/service/metric/GcTimeAlerter.java | 2 +
.../iotdb/commons/service/metric/enums/Metric.java | 2 +
.../commons/subscription/meta/topic/TopicMeta.java | 4 +-
.../apache/iotdb/commons/udf/builtin/UDTFAbs.java | 1 +
.../src/main/thrift/datanode.thrift | 1 +
.../org/apache/iotdb/library/dlearn/UDTFAR.java | 1 +
.../apache/iotdb/library/dmatch/UDTFPtnSym.java | 1 +
.../apache/iotdb/library/dprofile/UDTFSample.java | 1 +
pom.xml | 2 +
208 files changed, 2792 insertions(+), 1306 deletions(-)
create mode 100644
integration-test/src/test/java/org/apache/iotdb/subscription/it/triple/AbstractSubscriptionTripleIT.java
create mode 100644
integration-test/src/test/java/org/apache/iotdb/subscription/it/triple/IoTDBSubscriptionSharingIT.java
delete mode 100644 iotdb-client/client-py/iotdb/template/InternalNode.py
rename iotdb-client/client-py/{ => resources}/pyproject.toml (51%)
delete mode 100644 iotdb-client/client-py/resources/setup.py
copy iotdb-client/client-py/{iotdb => tests/integration}/__init__.py (100%)
rename iotdb-client/client-py/tests/{ =>
integration}/tablet_performance_comparison.py (100%)
rename iotdb-client/client-py/tests/{ =>
integration}/test_aligned_timeseries.py (100%)
rename iotdb-client/client-py/tests/{ => integration}/test_dataframe.py (100%)
rename iotdb-client/client-py/tests/{ => integration}/test_delete_data.py
(100%)
rename iotdb-client/client-py/tests/{ => integration}/test_one_device.py (100%)
rename iotdb-client/client-py/tests/{ => integration}/test_session.py (100%)
rename iotdb-client/client-py/tests/{ => integration}/test_session_pool.py
(100%)
rename iotdb-client/client-py/tests/{ => integration}/test_tablet.py (100%)
rename iotdb-client/client-py/tests/{ => integration}/test_todf.py (100%)
delete mode 100644 iotdb-client/client-py/tests/test_template.py
copy iotdb-client/client-py/{iotdb => tests/unit}/__init__.py (100%)
rename iotdb-client/client-py/tests/{ => unit}/test_numpy_tablet.py (100%)
create mode 100644
iotdb-core/confignode/src/main/java/org/apache/iotdb/confignode/manager/pipe/metric/PipeConfigNodeReceiverMetrics.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/metric/PipeDataNodeReceiverMetrics.java
