This is an automated email from the ASF dual-hosted git repository.
chenyz pushed a change to branch udtf
in repository https://gitbox.apache.org/repos/asf/iotdb.git
from cb66448a1d7 add leaf operator
new 87fc892d072 save
add 5df241c044b [Table Model] Refactor UDSF/UDAF interface
add 2669666d8d2 [To dev/1.3]Script adaptation to 4 new data types (#14296)
(#14630)
add f007e8a1407 Pipe: Support 'format'='tsfile' option in table model data
sync sinks (#14108)
add 6db2249c12c Load: Add LoadWithConvertOnTypeMismatchIT on Table Model
&& Fix TSInsertTabletReq.writeToTable is not set True in
PipeTransferTabletRawReq.toTPipeTransferRawReq when inserting a Tablet in Table
mode (#14611)
add 487a1a21ce9 Pipe & Load: Retry when a file operation causes an
exception (#14598)
add d60c3d52cb1 IoTConsensusV2: Fix adoption for table-deletion #14648
add 90cdf3bbc94 Load: Consider isGeneratedByPipe mark when executing
tsfile-tablet conversion in analysis stage (#14651)
add d9214b31465 Rename interface name of FunctionArguments
add 5f8b2c08ac9 Adjust default wal_throttle_threshold_in_byte and
region_migration_speed_limit_bytes_per_second (#14653)
add 815ea9c069d Implemented query for schema related Information schema &
Fixed the bug that the deviceEntry get from cache does not reserve query memory
add a4ce6f7a1a3 Modify the condition to skip compaction schdule after
insertion compaction task selection (#14644)
add c087c24435c Do not add the source path of a view when it can be
matched by the deletion pattern (#14658)
add c60520aa41a Load: Add a new configuration to enable datatype
conversion when converting into Tablets & Refactor LoadAnalyzeException.
(#14646)
add debdb527d89 Reduced the error log cause by batch activate template
when some devices are already activated (#14661)
add a54573a2afb Use DataNodeDevicePathCache when insert by sql (#14662)
add 5f01571f4c6 Pipe: tree/table model isolation for
alter/start/stop/drop/show pipe operations & support 'mode.double-living'
(#14386)
add 7dc5179def6 Fix select-distinct when there is no order by and select
items contains expression
add e92be6f63ab [Table Model Subquery] Support uncorrelated in predicate
(#14438)
add bdf36ba1907 Skip subquery plan if subqueryExpression List is empty for
efficiency (#14647)
add 2d436e06897 Optimized the error code when table/column does not exist
& Reduced some unnecessary IT execution
add 04df3e279a2 Fixed the un-renamed ID/MEASUREMENTs in some non-runnable
ITs
add 7f0ac972839 Pipe: Fix NPE caused by forced type conversion and fix
IoTDBipipeTypeConversionISessionIT of table model (#14667)
add a5e13fd8f7c Pipe: report linked tsfile size & Subscription: decrease
reference count for other enriched events & add logging to observe possible
stuck situations (#14668)
add c8d7c5dfea9 Reopen drop column and delete devices operations
add 937df761e31 IoTConsensusV2: Fix some consensus group missing due to
recover failed or blocked (#14613)
add 5bd1eadea76 Clone value array when insert non aligned tablet with null
(#14643)
add 01e1420fa0a Add Memory Threshold (#14597)
add 82c85483572 Ignore the DatabaseNotExistsException for
adjustMaxRegionGroupNum when querying dataRegionGroupCount (#14669)
add 3ac203c4e79 Pipe: Implement table meta/deletion sync & Fix the bug
that delete data node's pattern is not parsed / pipe transferred delete data
node's progressIndex is not binded & Implement mem control for schema snapshot
parser (#14156)
add b349f565fd8 Pipe/Load: Fixed the issue where the TsFileResource of the
PlainDeviceTimeIndex type was judged to be empty (triggered by IoTDB loading
TsFiles in v3 and the tsfiles are syned to other clusters) (#14692)
add 97a08c880fb Pipe Plugin: intro class annotations for pipe plugin &
tree/table model isolation for drop/show pipe plugin operations (#14673)
add f2e20b54408 Modify tablet usage & don't cache TableSchema map in
default option (#14635)
add f129298c9e8 Pipe: Fix PipeWriteBackSink using toTPipeTransferReq
causing NPE due to uninitialized buffer (#14672)
add 701a5156e7a Pipe: Optimize the process of collecting
PipeTsFileInsertionEvent in table model to reduce unnecessary parsing of TsFile
(#14697)
add b4edfdf1d68 IoTConsensusV2: Accelerate follower receive dispatch based
on TPCx case
add b51f63c7108 Load: Fix some issues during loading tsfiles of old
version (#14649)
add c65b8264156 Pipe: strict check for synonym pipe parameters to avoid
ambiguity (#14694)
add a11401b494f add repair data for table model (#14705)
add c3df7d54496 Renaming the IT Methods of the IoTDB REST Service Table
Model (#14681)
add 6cb9035e8f6 Pipe IT: Added flush for assertData on targetEnv to avoid
IoTV2 batch mode lead to large delay (#14707)
add 42878058f11 [RATISCONSENSUS] Bump ratis version to 3.1.3 (#14709)
add 6961e2154a7 Fix NPE when concurrent closing mod files (#14708)
add 8e7605f80db Fix cache invalidation failure and add retry (#14711)
add 44e9725ab94 Publish build scans to develocity.apache.org (#14701)
add c3baa7a4978 Subscription: fix `hasNext` method in tablet batch to
support multiple tsfile in the same batch (#14715)
add 6036cabd9dc Implemented access control for
pipe/subscription/table/database
add 3072ae5a668 Implement distinct in AggFunction
add 49a91ecd348 Add more user-friendly error messages for the last and
last_by aggregation
add 141b7ba634c Extend Region & Remove Region & Reconstruct Region (#14634)
add a011b013711 Pipe: Unify the case of the table model database and fix
the incorrect use of iterators in Load File (#14718)
add 75dda9fb0ef Pipe: Fixed the bug that alter logical view statement uses
a common partial path (#14713)
add bc5fdaed8d9 Bump TsFile version to 2.0.0-250118-SNAPSHOT
add 670b45675e0 Pipe: Fix HA issues caused by exceptions not handled in
handshake (IoTDBAirGapConnector / IoTDBSyncClientManager) (#14706)
add fea2e4fc8f8 [Table Model Subquery] Support uncorrelated quantified
comparison
add 263443dcb53 The time range of device in resource file may be larger
than the actual time range in tsfile (#14712)
new 602a726ae63 merge master
new b9dd299cbe8 adjust api
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/cluster-it-1c1d.yml | 2 +-
.github/workflows/cluster-it-1c1d1a.yml | 2 +-
.github/workflows/cluster-it-1c3d.yml | 2 +-
.github/workflows/compile-check.yml | 2 +-
.github/workflows/daily-it.yml | 2 +-
.github/workflows/daily-ut.yml | 2 +-
.github/workflows/dependency-check.yml | 2 +-
.github/workflows/pipe-it-2cluster.yml | 2 +-
.github/workflows/sonar-codecov.yml | 2 +-
.github/workflows/table-cluster-it-1c1d.yml | 2 +-
.github/workflows/table-cluster-it-1c3d.yml | 2 +-
.github/workflows/unit-test.yml | 2 +-
.github/workflows/vulnerability-check.yml | 2 +-
.mvn/develocity.xml | 3 +-
.mvn/extensions.xml | 4 +-
.../java/org/apache/iotdb/CountPointProcessor.java | 2 +
.../iotdb/AlignedTimeseriesSessionExample.java | 67 +-
.../org/apache/iotdb/SessionConcurrentExample.java | 23 -
.../main/java/org/apache/iotdb/SessionExample.java | 107 +---
.../org/apache/iotdb/trigger/LoggerTrigger.java | 12 +-
.../iotdb/trigger/StatisticsUpdaterTrigger.java | 4 +-
.../apache/iotdb/udf/AggregateFunctionExample.java | 35 +-
.../apache/iotdb/udf/ScalarFunctionExample.java | 57 +-
.../db/query/udf/example/relational/AllSum.java | 49 +-
.../query/udf/example/relational/ContainNull.java | 21 +-
.../relational/{DatePlusOne.java => DatePlus.java} | 31 +-
.../query/udf/example/relational/FirstTwoSum.java | 33 +-
.../db/query/udf/example/relational/MyAvg.java | 32 +-
.../db/query/udf/example/relational/MyCount.java | 19 +-
.../java/org/apache/iotdb/it/env/EnvFactory.java | 5 +
.../iotdb/it/env/cluster/env/AbstractEnv.java | 49 ++
.../iotdb/it/env/cluster/node/AINodeWrapper.java | 5 +
.../it/env/cluster/node/AbstractNodeWrapper.java | 12 +
.../it/env/cluster/node/ConfigNodeWrapper.java | 5 +
.../iotdb/it/env/cluster/node/DataNodeWrapper.java | 5 +
.../iotdb/it/env/remote/env/RemoteServerEnv.java | 12 +
.../org/apache/iotdb/it/utils/TsFileGenerator.java | 87 +--
.../iotdb/it/utils/TsFileTableGenerator.java | 186 ++++++
.../java/org/apache/iotdb/itbase/env/BaseEnv.java | 7 +
...ionMigrateDataNodeCrashITFrameworkForIoTV1.java | 2 +-
...ionMigrateDataNodeCrashITFrameworkForIoTV2.java | 2 +-
...oTDBRegionOperationReliabilityITFramework.java} | 174 ++++--
.../IoTDBRegionGroupExpandAndShrinkForIoTV1IT.java | 157 +++++
.../commit/IoTDBRegionReconstructForIoTV1IT.java | 141 +++++
.../IoTDBRegionMigrateNormalITForIoTV2Batch.java | 4 +-
.../IoTDBRegionMigrateOtherITForIoTV2Batch.java | 4 +-
.../IoTDBRegionMigrateNormalITForIoTV2Stream.java | 4 +-
.../IoTDBRegionMigrateOtherITForIoTV2Stream.java | 4 +-
.../IoTDBRegionMigrateDataNodeCrashForIoTV1IT.java | 4 +-
...TDBRegionMigrateDataNodeCrashForIoTV2Batch.java | 4 +-
...DBRegionMigrateDataNodeCrashForIoTV2Stream.java | 4 +-
.../IoTDBRegionMigrateClusterCrashIoTV1IT.java | 4 +-
.../IoTDBRegionMigrateConfigNodeCrashIoTV1IT.java | 4 +-
.../IoTDBRegionMigrateClusterCrashIoTV2Batch.java | 4 +-
...oTDBRegionMigrateConfigNodeCrashIoTV2Batch.java | 4 +-
.../IoTDBRegionMigrateClusterCrashIoTV2Stream.java | 4 +-
...TDBRegionMigrateConfigNodeCrashIoTV2Stream.java | 4 +-
.../IoTDBRemoveDataNodeITFramework.java | 22 +-
.../org/apache/iotdb/db/it/IoTDBLoadTsFileIT.java | 80 ++-
.../iotdb/db/it/IoTDBLoadTsFileWithModIT.java | 172 ++++++
.../it/IoTDBSyntaxConventionStringLiteralIT.java | 3 +-
.../org/apache/iotdb/db/it/utils/TestUtils.java | 279 ++++++++-
.../iotdb/pipe/it/autocreate/IoTDBPipeAlterIT.java | 3 +-
.../pipe/it/autocreate/IoTDBPipeAutoDropIT.java | 10 +-
.../pipe/it/autocreate/IoTDBPipeClusterIT.java | 98 +++
.../pipe/it/autocreate/IoTDBPipeIdempotentIT.java | 93 +++
.../pipe/it/autocreate/IoTDBPipeNullValueIT.java | 28 +-
.../it/autocreate/IoTDBPipeSwitchStatusIT.java | 23 +-
.../pipe/it/manual/IoTDBPipeReqAutoSliceIT.java | 8 +-
...nclusionIT.java => IoTDBPipeTableManualIT.java} | 222 ++++---
.../manual/IoTDBPipeTypeConversionISessionIT.java | 10 +-
.../iotdb/pipe/it/tablemodel/IoTDBPipeAlterIT.java | 30 +-
.../pipe/it/tablemodel/IoTDBPipeAutoDropIT.java | 10 +-
.../pipe/it/tablemodel/IoTDBPipeClusterIT.java | 65 ++
.../pipe/it/tablemodel/IoTDBPipeDataSinkIT.java | 450 +++++++++++++-
.../it/tablemodel/IoTDBPipeDoubleLivingIT.java | 329 ++++++++++
.../pipe/it/tablemodel/IoTDBPipeIsolationIT.java | 377 ++++++++++++
.../pipe/it/tablemodel/IoTDBPipeNullValueIT.java | 4 +-
.../pipe/it/tablemodel/IoTDBPipeProtocolIT.java | 1 -
.../it/tablemodel/IoTDBPipeSwitchStatusIT.java | 23 +-
.../pipe/it/tablemodel/IoTDBPipeSyntaxIT.java | 35 +-
.../IoTDBPipeTypeConversionISessionIT.java | 97 ++-
.../pipe/it/tablemodel/IoTDBPipeWithLoadIT.java | 8 +-
.../iotdb/pipe/it/tablemodel/TableModelUtils.java | 614 +++++++++++++++----
.../relational/it/db/it/IoTDBDeletionTableIT.java | 2 +-
.../relational/it/db/it/IoTDBInsertTableIT.java | 2 +-
.../it/db/it/udf/IoTDBSQLFunctionManagementIT.java | 19 +-
.../udf/IoTDBUserDefinedAggregateFunctionIT.java | 53 +-
.../it/udf/IoTDBUserDefinedScalarFunctionIT.java | 2 +-
.../IoTDBAlignByDeviceWithTemplateTableIT.java | 4 +-
.../scalar/IoTDBFormatFunctionTableIT.java | 21 +-
...oTDBNoSelectExpressionAfterAnalyzedTableIT.java | 6 +-
.../query/old/query/IoTDBNullValueFillTableIT.java | 327 ----------
.../it/query/old/query/IoTDBResultSetTableIT.java | 2 +-
.../query/old/query/IoTDBSelectSchemaTableIT.java | 22 +-
.../it/query/recent/IoTDBTableAggregationIT.java | 656 +++++++++++++++++++-
.../recent/subquery/SubqueryDataSetUtils.java | 12 +-
.../IoTDBUncorrelatedInPredicateSubqueryIT.java | 319 ++++++++++
.../IoTDBUncorrelatedQuantifiedComparisonIT.java | 674 +++++++++++++++++++++
.../IoTDBUncorrelatedScalarSubqueryIT.java | 4 +-
.../it/rest/it/IoTDBRestServiceCaseWhenThenIT.java | 7 +-
.../it/rest/it/IoTDBRestServiceFlushQueryIT.java | 6 +-
.../relational/it/rest/it/IoTDBRestServiceIT.java | 3 +-
...IT.java => IoTDBRestServiceInsertValuesIT.java} | 20 +-
.../relational/it/schema/IoTDBDatabaseIT.java | 41 ++
.../iotdb/relational/it/schema/IoTDBDeviceIT.java | 47 +-
.../iotdb/relational/it/schema/IoTDBTableIT.java | 122 ++--
.../it/session/IoTDBSessionRelationalIT.java | 29 +-
.../iotdb/session/it/IoTDBSessionComplexIT.java | 8 +-
.../iotdb/session/it/IoTDBSessionSimpleIT.java | 78 ++-
.../session/it/IoTDBSessionSyntaxConventionIT.java | 18 +-
.../org/apache/iotdb/session/it/SessionIT.java | 32 +-
.../it/local/IoTDBSubscriptionDataTypeIT.java | 8 +-
.../java/org/apache/iotdb/util/MagicUtils.java | 61 ++
.../java/org/apache/iotdb/pipe/api/PipePlugin.java | 37 ++
.../iotdb/pipe/api/annotation/TableModel.java | 18 +-
.../iotdb/pipe/api/annotation/TreeModel.java | 18 +-
.../parameter/PipeParameterValidator.java | 42 +-
.../analysis/AggregateFunctionAnalysis.java | 62 ++
.../api/customizer/analysis/FunctionAnalysis.java | 8 +-
.../ScalarFunctionAnalysis.java} | 26 +-
...ctionParameters.java => FunctionArguments.java} | 28 +-
...tion.java => UDFArgumentNotValidException.java} | 4 +-
.../udf/api/relational/AggregateFunction.java | 48 +-
.../iotdb/udf/api/relational/ScalarFunction.java | 47 +-
.../iotdb/udf/api/relational/TableFunction.java | 51 +-
.../relational/table/TableFunctionAnalysis.java | 4 +-
.../relational/table/argument/DescribedSchema.java | 20 +-
.../api/relational/table/argument/Descriptor.java | 95 ---
.../table/argument/DescriptorArgument.java | 57 --
.../relational/table/argument/TableArgument.java | 11 -
.../processor/TableFunctionDataProcessor.java | 32 +-
.../processor/TableFunctionLeafProcessor.java | 5 +
.../DescriptorParameterSpecification.java | 57 --
.../specification/ReturnTypeSpecification.java | 68 ---
.../specification/TableParameterSpecification.java | 9 -
.../apache/iotdb/udf/api/type/ColumnCategory.java | 37 --
.../apache/iotdb/tool/data/AbstractDataTool.java | 32 +-
.../org/apache/iotdb/tool/data/ExportData.java | 43 +-
.../org/apache/iotdb/tool/data/ImportData.java | 24 +-
.../org/apache/iotdb/tool/tsfile/ExportTsFile.java | 3 -
.../java/org/apache/iotdb/rpc/TSStatusCode.java | 3 +
.../java/org/apache/iotdb/session/Session.java | 27 +-
.../payload/SubscriptionFileHandler.java | 24 +-
.../payload/SubscriptionSessionDataSet.java | 12 +-
.../org/apache/iotdb/session/util/RetryUtils.java | 37 +-
.../apache/iotdb/session/util/SessionUtils.java | 25 +-
.../java/org/apache/iotdb/session/TabletTest.java | 28 +-
.../iotdb/session/util/SessionUtilsTest.java | 16 +-
.../org/apache/iotdb/db/qp/sql/IoTDBSqlParser.g4 | 16 +-
.../antlr4/org/apache/iotdb/db/qp/sql/SqlLexer.g4 | 8 +
.../consensus/request/ConfigPhysicalPlan.java | 10 +-
.../consensus/request/ConfigPhysicalPlanType.java | 4 +
.../request/ConfigPhysicalPlanVisitor.java | 110 +++-
.../DescTable4InformationSchemaPlan.java} | 9 +-
.../ShowTable4InformationSchemaPlan.java} | 8 +-
.../payload/PipeCreateTablePlan.java} | 15 +-
.../pipe/payload/PipeDeactivateTemplatePlan.java | 22 +-
.../write/pipe/payload/PipeDeleteDevicesPlan.java | 130 ++++
.../request/write/table/PreCreateTablePlan.java | 11 +-
.../response/pipe/plugin/PipePluginTableResp.java | 27 +-
.../response/pipe/task/PipeTableResp.java | 7 +
...p.java => DescTable4InformationSchemaResp.java} | 20 +-
...p.java => ShowTable4InformationSchemaResp.java} | 17 +-
.../iotdb/confignode/manager/ClusterManager.java | 2 +-
.../iotdb/confignode/manager/ConfigManager.java | 66 +-
.../apache/iotdb/confignode/manager/IManager.java | 42 +-
.../iotdb/confignode/manager/ProcedureManager.java | 491 +++++++++++----
.../payload/PipeTransferConfigSnapshotSealReq.java | 32 +-
.../protocol/IoTDBConfigRegionAirGapConnector.java | 8 +
.../protocol/IoTDBConfigRegionConnector.java | 10 +-
.../coordinator/plugin/PipePluginCoordinator.java | 29 +
.../pipe/coordinator/task/PipeTaskCoordinator.java | 74 ++-
.../pipe/event/PipeConfigRegionSnapshotEvent.java | 3 +-
.../extractor/ConfigRegionListeningFilter.java | 89 +--
.../pipe/extractor/ConfigRegionListeningQueue.java | 23 +-
.../pipe/extractor/IoTDBConfigRegionExtractor.java | 74 ++-
...ConfigPhysicalPlanTablePatternParseVisitor.java | 135 +++++
...ConfigPhysicalPlanTreePatternParseVisitor.java} | 6 +-
.../receiver/protocol/IoTDBConfigNodeReceiver.java | 170 +++++-
.../PipeConfigPhysicalPlanTSStatusVisitor.java | 76 +++
.../manager/schema/ClusterSchemaManager.java | 158 +++--
.../persistence/executor/ConfigPlanExecutor.java | 5 +
.../confignode/persistence/pipe/PipeInfo.java | 2 +-
.../persistence/pipe/PipePluginInfo.java | 39 +-
.../confignode/persistence/pipe/PipeTaskInfo.java | 28 +-
.../schema/CNPhysicalPlanGenerator.java | 77 ++-
.../persistence/schema/ClusterSchemaInfo.java | 77 +++
.../confignode/persistence/schema/ConfigMTree.java | 29 +-
.../schema/ConfignodeSnapshotParser.java | 30 +-
.../procedure/env/ConfigNodeProcedureEnv.java | 23 +-
.../procedure/env/RegionMaintainHandler.java | 13 +-
.../impl/pipe/task/AlterPipeProcedureV2.java | 4 +-
.../impl/pipe/task/CreatePipeProcedureV2.java | 42 +-
.../impl/region/AddRegionPeerProcedure.java | 84 +--
.../impl/region/ReconstructRegionProcedure.java | 209 +++++++
.../impl/region/RegionMigrateProcedure.java | 39 +-
.../impl/region/RegionOperationProcedure.java} | 33 +-
.../impl/region/RemoveRegionPeerProcedure.java | 52 +-
.../impl/schema/DeleteDatabaseProcedure.java | 2 +-
.../impl/schema/DeleteTimeSeriesProcedure.java | 4 +-
.../table/AbstractAlterOrDropTableProcedure.java | 10 +-
.../impl/schema/table/AddTableColumnProcedure.java | 16 +-
.../impl/schema/table/CreateTableProcedure.java | 20 +-
.../impl/schema/table/DeleteDevicesProcedure.java | 43 +-
.../schema/table/DropTableColumnProcedure.java | 21 +-
.../impl/schema/table/DropTableProcedure.java | 24 +-
.../schema/table/RenameTableColumnProcedure.java | 18 +-
.../schema/table/SetTablePropertiesProcedure.java | 18 +-
.../impl/trigger/CreateTriggerProcedure.java | 9 +-
...itionState.java => ReconstructRegionState.java} | 8 +-
.../procedure/store/ProcedureFactory.java | 40 +-
.../confignode/procedure/store/ProcedureType.java | 12 +-
.../thrift/ConfigNodeRPCServiceProcessor.java | 52 +-
.../request/ConfigPhysicalPlanSerDeTest.java | 40 ++
.../response/pipe/PipePluginTableRespTest.java | 4 +-
.../connector/PipeConfigNodeThriftRequestTest.java | 12 +-
...igPhysicalPlanTablePatternParseVisitorTest.java | 133 ++++
...igPhysicalPlanTreePatternParseVisitorTest.java} | 44 +-
.../pipe/annotation/PipePluginAnnotationTest.java} | 20 +-
.../pipe/receiver/PipeEnrichedProcedureTest.java | 194 ++++++
.../schema/table/AddTableColumnProcedureTest.java | 8 +-
.../schema/table/CreateTableProcedureTest.java | 5 +-
.../schema/table/DeleteDevicesProcedureTest.java | 5 +-
.../schema/table/DropTableColumnProcedureTest.java | 4 +-
.../impl/schema/table/DropTableProcedureTest.java | 4 +-
.../table/RenameTableColumnProcedureTest.java | 4 +-
.../table/SetTablePropertiesProcedureTest.java | 6 +-
.../apache/iotdb/consensus/pipe/PipeConsensus.java | 57 +-
.../apache/iotdb/consensus/ratis/RatisClient.java | 6 +-
.../apache/iotdb/consensus/ratis/utils/Utils.java | 1 +
.../java/org/apache/iotdb/db/conf/IoTDBConfig.java | 4 +-
.../dataregion/DataExecutionVisitor.java | 9 +-
.../schemaregion/SchemaExecutionVisitor.java | 15 +-
.../schemaregion/SchemaRegionStateMachine.java | 15 +-
.../LoadAnalyzeException.java} | 8 +-
.../LoadAnalyzeTableColumnDisorderException.java} | 7 +-
.../LoadAnalyzeTypeMismatchException.java} | 6 +-
.../pipe/agent/plugin/PipeDataNodePluginAgent.java | 20 +-
.../dataregion/PipeDataRegionPluginAgent.java | 49 ++
.../agent/task/connection/PipeEventCollector.java | 35 +-
.../client/IoTDBDataNodeAsyncClientManager.java | 2 +-
.../batch/PipeTabletEventTsFileBatch.java | 384 +++---------
.../PipeTransferDataNodeHandshakeV1Req.java | 10 +-
.../evolvable/request/PipeTransferPlanNodeReq.java | 18 +-
.../request/PipeTransferSchemaSnapshotSealReq.java | 94 ++-
.../request/PipeTransferTabletInsertNodeReqV2.java | 2 +-
.../request/PipeTransferTabletRawReq.java | 4 +-
.../request/PipeTransferTabletRawReqV2.java | 9 +-
.../request/PipeTransferTsFileSealWithModReq.java | 42 +-
.../airgap/IoTDBDataNodeAirGapConnector.java | 4 +
.../airgap/IoTDBDataRegionAirGapConnector.java | 4 +
.../airgap/IoTDBSchemaRegionAirGapConnector.java | 19 +-
.../protocol/legacy/IoTDBLegacyPipeConnector.java | 2 +
.../connector/protocol/opcua/OpcUaConnector.java | 4 +
.../connector/protocol/opcua/OpcUaNameSpace.java | 44 +-
.../pipeconsensus/PipeConsensusAsyncConnector.java | 4 +
.../pipeconsensus/PipeConsensusSyncConnector.java | 4 +
.../request/PipeConsensusDeleteNodeReq.java | 4 +-
.../async/IoTDBDataRegionAsyncConnector.java | 14 +-
.../PipeTransferTabletBatchEventHandler.java | 2 +-
.../async/handler/PipeTransferTsFileHandler.java | 13 +-
.../thrift/sync/IoTDBDataNodeSyncConnector.java | 4 +
.../thrift/sync/IoTDBDataRegionSyncConnector.java | 23 +-
.../thrift/sync/IoTDBSchemaRegionConnector.java | 26 +-
.../protocol/websocket/WebSocketConnector.java | 2 +
.../protocol/writeback/WriteBackConnector.java | 199 +++++-
.../util/builder/PipeTableModeTsFileBuilder.java | 273 +++++++++
.../util/builder/PipeTreeModelTsFileBuilder.java | 268 ++++++++
.../connector/util/builder/PipeTsFileBuilder.java | 162 +++++
.../util/{ => cacher}/LeaderCacheUtils.java | 2 +-
.../sorter/PipeTableModelTabletEventSorter.java | 273 +++++++++
.../util/{ => sorter}/PipeTabletEventSorter.java | 107 +---
.../sorter/PipeTreeModelTabletEventSorter.java | 123 ++++
.../pipe/consensus/deletion/DeletionResource.java | 2 +-
.../db/pipe/event/common/PipeInsertionEvent.java | 15 +-
.../common/deletion/PipeDeleteDataNodeEvent.java | 24 +-
.../db/pipe/event/common/row/PipeRowCollector.java | 2 +-
.../schema/PipeSchemaRegionSnapshotEvent.java | 75 ++-
.../schema/PipeSchemaSerializableEventType.java | 15 +-
.../common/tablet/PipeRawTabletInsertionEvent.java | 2 +-
.../tablet/parser/TabletInsertionEventParser.java | 8 +-
.../TabletInsertionEventTreePatternParser.java | 13 +-
.../scan/TsFileInsertionEventScanParser.java | 42 +-
.../table/TsFileInsertionEventTableParser.java | 2 +-
.../dataregion/DataRegionListeningFilter.java | 21 +-
.../dataregion/IoTDBDataRegionExtractor.java | 156 ++---
...oricalDataRegionTsFileAndDeletionExtractor.java | 4 +-
.../listener/PipeInsertionDataNodeListener.java | 27 +-
.../schemaregion/IoTDBSchemaRegionExtractor.java | 28 +-
.../PipePlanTablePatternParseVisitor.java | 77 +++
...r.java => PipePlanTreePatternParseVisitor.java} | 7 +-
.../schemaregion/SchemaRegionListeningFilter.java | 87 +--
.../schemaregion/SchemaRegionListeningQueue.java | 8 +-
.../pipe/metric/PipeDataNodeReceiverMetrics.java | 48 +-
.../processor/aggregate/AggregateProcessor.java | 2 +
.../StandardStatisticsOperatorProcessor.java | 2 +
.../processor/TumblingWindowingProcessor.java | 2 +
.../changing/ChangingValueSamplingProcessor.java | 2 +
.../sdt/SwingingDoorTrendingSamplingProcessor.java | 2 +
.../tumbling/TumblingTimeSamplingProcessor.java | 2 +
.../pipeconsensus/PipeConsensusProcessor.java | 4 +
.../schemachange/RenameDatabaseProcessor.java | 11 +-
.../twostage/plugin/TwoStageCountProcessor.java | 37 +-
.../pipeconsensus/PipeConsensusReceiver.java | 33 +-
.../protocol/thrift/IoTDBDataNodeReceiver.java | 223 +++++--
.../statement/PipeConvertedInsertRowStatement.java | 30 +-
.../PipeConvertedInsertTabletStatement.java | 52 +-
.../visitor/PipePlanToStatementVisitor.java | 54 +-
.../PipeStatementTablePatternParseVisitor.java} | 24 +-
...a => PipeStatementTreePatternParseVisitor.java} | 6 +-
...leStatementDataTypeConvertExecutionVisitor.java | 10 +-
...eeStatementDataTypeConvertExecutionVisitor.java | 3 +-
...r.java => PipeTreeStatementToBatchVisitor.java} | 2 +-
.../pipe/resource/memory/PipeMemoryWeightUtil.java | 21 +-
.../resource/tsfile/PipeTsFileResourceManager.java | 5 +-
.../iotdb/db/protocol/client/ConfigNodeClient.java | 57 ++
.../impl/DataNodeInternalRPCServiceImpl.java | 2 +
.../execution/executor/RegionWriteExecutor.java | 55 +-
.../function/table/ExcludeColumnFunction.java | 32 +-
.../execution/function/table/HOPTableFunction.java | 55 +-
.../execution/function/table/SplitFunction.java | 6 +-
.../operator/process/EnforceSingleRowOperator.java | 5 +-
.../process/function/PartitionRecognizer.java | 124 +---
.../process/function/TableFunctionOperator.java | 73 ++-
.../process/function/partition/PartitionState.java | 22 +-
.../operator/process/function/partition/Slice.java | 153 +++++
.../partition/SliceCache.java} | 41 +-
.../merge/comparator/JoinKeyComparatorFactory.java | 2 +-
.../schema/source/TableDeviceQuerySource.java | 4 +-
.../InformationSchemaContentSupplierFactory.java | 372 ++++++++++--
...perator.java => MergeSortSemiJoinOperator.java} | 130 ++--
.../relational/aggregation/AccumulatorFactory.java | 265 +++++++-
.../relational/aggregation/AggregationMask.java | 204 +++++++
.../relational/aggregation/AvgAccumulator.java | 122 +++-
.../relational/aggregation/CountAccumulator.java | 22 +-
...IfAccumulator.java => CountAllAccumulator.java} | 48 +-
.../relational/aggregation/CountIfAccumulator.java | 22 +-
.../relational/aggregation/ExtremeAccumulator.java | 94 ++-
.../relational/aggregation/FirstAccumulator.java | 158 +++--
.../relational/aggregation/FirstByAccumulator.java | 164 +++--
.../aggregation/FirstByDescAccumulator.java | 132 +++-
.../aggregation/FirstDescAccumulator.java | 126 +++-
.../relational/aggregation/LastAccumulator.java | 140 ++++-
.../relational/aggregation/LastByAccumulator.java | 146 ++++-
.../aggregation/LastByDescAccumulator.java | 150 ++++-
.../aggregation/LastDescAccumulator.java | 144 ++++-
.../MaskedRecordIterator.java} | 24 +-
.../relational/aggregation/MaxAccumulator.java | 140 ++++-
.../relational/aggregation/MinAccumulator.java | 140 ++++-
.../relational/aggregation/SumAccumulator.java | 114 +++-
.../relational/aggregation/TableAccumulator.java | 2 +-
.../relational/aggregation/TableAggregator.java | 5 +-
.../aggregation/TableMaxMinByBaseAccumulator.java | 146 +++--
.../aggregation/TableModeAccumulator.java | 192 ++++--
.../aggregation/TableVarianceAccumulator.java | 158 +++--
.../UserDefinedAggregateFunctionAccumulator.java | 35 +-
.../aggregation/grouped/GroupedAccumulator.java | 4 +-
.../aggregation/grouped/GroupedAggregator.java | 6 +-
.../aggregation/grouped/GroupedAvgAccumulator.java | 119 +++-
.../grouped/GroupedCountAccumulator.java | 22 +-
.../grouped/GroupedCountIfAccumulator.java | 22 +-
.../grouped/GroupedExtremeAccumulator.java | 95 ++-
.../grouped/GroupedFirstAccumulator.java | 153 ++++-
.../grouped/GroupedFirstByAccumulator.java | 191 ++++--
.../grouped/GroupedLastAccumulator.java | 133 +++-
.../grouped/GroupedLastByAccumulator.java | 150 ++++-
.../aggregation/grouped/GroupedMaxAccumulator.java | 141 ++++-
.../grouped/GroupedMaxMinByBaseAccumulator.java | 147 ++++-
.../aggregation/grouped/GroupedMinAccumulator.java | 141 ++++-
.../grouped/GroupedModeAccumulator.java | 246 ++++++--
.../aggregation/grouped/GroupedSumAccumulator.java | 111 +++-
.../GroupedUserDefinedAggregateAccumulator.java | 14 +-
.../grouped/GroupedVarianceAccumulator.java | 159 +++--
.../aggregation/grouped/hash/GroupByHash.java | 2 +
.../aggregation/grouped/hash/MarkDistinctHash.java | 90 +++
.../relational/ColumnTransformerBuilder.java | 20 +-
.../iotdb/db/queryengine/plan/Coordinator.java | 8 +
.../db/queryengine/plan/analyze/AnalyzeUtils.java | 9 +-
.../queryengine/plan/analyze/AnalyzeVisitor.java | 30 +-
.../plan/analyze/load/LoadTsFileAnalyzer.java | 27 +-
.../analyze/load/LoadTsFileTableSchemaCache.java | 17 +-
.../load/LoadTsFileToTableModelAnalyzer.java | 34 +-
.../load/LoadTsFileToTreeModelAnalyzer.java | 22 +-
.../analyze/load/LoadTsFileTreeSchemaCache.java | 3 +-
.../load/TreeSchemaAutoCreatorAndVerifier.java | 30 +-
.../execution/config/TableConfigTaskVisitor.java | 141 ++++-
.../execution/config/TreeConfigTaskVisitor.java | 30 +-
.../config/executor/ClusterConfigTaskExecutor.java | 159 ++++-
.../config/executor/IConfigTaskExecutor.java | 19 +-
.../config/metadata/DropPipePluginTask.java | 13 +-
.../config/metadata/ShowPipePluginsTask.java | 20 +-
.../ExtendRegionTask.java} | 14 +-
.../metadata/{ => region}/MigrateRegionTask.java | 4 +-
.../ReconstructRegionTask.java} | 16 +-
.../RemoveRegionTask.java} | 14 +-
.../config/metadata/relational/ShowDBTask.java | 4 +-
.../metadata/relational/ShowTablesDetailsTask.java | 47 +-
.../config/metadata/relational/ShowTablesTask.java | 35 +-
.../execution/config/sys/pipe/AlterPipeTask.java | 8 +-
.../execution/config/sys/pipe/DropPipeTask.java | 7 +-
.../execution/config/sys/pipe/ShowPipeTask.java | 3 +-
.../execution/config/sys/pipe/StartPipeTask.java | 7 +-
.../execution/config/sys/pipe/StopPipeTask.java | 7 +-
.../db/queryengine/plan/parser/ASTVisitor.java | 38 +-
.../plan/planner/LogicalPlanVisitor.java | 7 +-
.../plan/planner/TableOperatorGenerator.java | 102 +++-
.../plan/planner/plan/node/PlanGraphPrinter.java | 34 +-
.../plan/planner/plan/node/PlanNodeType.java | 4 +
.../plan/planner/plan/node/PlanVisitor.java | 5 +
.../node/metadata/write/AlterTimeSeriesNode.java | 9 +-
.../node/metadata/write/CreateTimeSeriesNode.java | 74 +--
.../plan/node/pipe/PipeEnrichedDeleteDataNode.java | 66 +-
.../plan/node/pipe/PipeEnrichedInsertNode.java | 44 +-
.../plan/node/pipe/PipeEnrichedWritePlanNode.java | 23 +-
.../relational/analyzer/ExpressionAnalyzer.java | 37 +-
.../plan/relational/analyzer/Scope.java | 8 +-
.../relational/analyzer/StatementAnalyzer.java | 235 +++----
.../plan/relational/metadata/Metadata.java | 4 +-
.../relational/metadata/TableMetadataImpl.java | 53 +-
.../metadata/fetcher/TableDeviceSchemaFetcher.java | 14 +-
.../fetcher/TableHeaderSchemaValidator.java | 11 +-
.../plan/relational/planner/IrTypeAnalyzer.java | 15 +-
.../plan/relational/planner/QueryPlanner.java | 2 +-
.../plan/relational/planner/RelationPlanner.java | 91 +--
.../relational/planner/SimplePlanRewriter.java | 88 +++
.../plan/relational/planner/SubqueryPlanner.java | 84 ++-
.../relational/planner/TableLogicalPlanner.java | 15 +-
.../distribute/TableDistributedPlanGenerator.java | 24 +
.../planner/iterative/rule/PruneApplyColumns.java | 138 +++++
...Correlation.java => PruneApplyCorrelation.java} | 38 +-
.../iterative/rule/PruneApplySourceColumns.java | 95 +++
...ava => RemoveUnreferencedScalarApplyNodes.java} | 19 +-
.../rule/RemoveUnreferencedScalarSubqueries.java | 70 +++
.../TransformFilteringSemiJoinToInnerJoin.java | 150 +++++
...mUncorrelatedInPredicateSubqueryToSemiJoin.java | 95 +++
.../plan/relational/planner/node/Patterns.java | 7 +-
.../plan/relational/planner/node/SemiJoinNode.java | 181 ++++++
.../node/schema/CreateOrUpdateTableDeviceNode.java | 14 +-
.../DataNodeLocationSupplierFactory.java | 11 +
.../optimizations/LogicalOptimizeFactory.java | 26 +-
.../PushAggregationIntoTableScan.java | 4 +
.../optimizations/PushPredicateIntoTableScan.java | 235 ++++++-
...mQuantifiedComparisonApplyToCorrelatedJoin.java | 341 +++++++++++
.../optimizations/UnaliasSymbolReferences.java | 29 +
.../sql/ast/AbstractQueryDeviceWithCache.java | 2 +
.../relational/sql/ast/AbstractTraverseDevice.java | 8 +
.../plan/relational/sql/ast/AlterPipe.java | 22 +-
.../plan/relational/sql/ast/AstVisitor.java | 27 +-
.../{DescribeTable.java => CountStatement.java} | 40 +-
.../relational/sql/ast/CreateOrUpdateDevice.java | 31 +-
.../plan/relational/sql/ast/CreatePipe.java | 4 +-
.../plan/relational/sql/ast/Delete.java | 28 +-
.../plan/relational/sql/ast/DeleteDevice.java | 2 -
.../plan/relational/sql/ast/Descriptor.java | 80 ---
.../plan/relational/sql/ast/DescriptorField.java | 88 ---
.../plan/relational/sql/ast/DropColumn.java | 3 -
.../plan/relational/sql/ast/DropPipe.java | 10 +-
.../plan/relational/sql/ast/LoadTsFile.java | 9 +
.../plan/relational/sql/ast/PipeEnriched.java | 8 +-
.../plan/relational/sql/ast/ShowPipes.java | 10 +-
.../plan/relational/sql/ast/StartPipe.java | 8 +-
...{SetConfiguration.java => StartRepairData.java} | 6 +-
.../plan/relational/sql/ast/Statement.java | 4 +-
.../plan/relational/sql/ast/StopPipe.java | 8 +-
.../{SetConfiguration.java => StopRepairData.java} | 6 +-
.../relational/sql/ast/TableFunctionArgument.java | 5 +-
.../sql/ast/TableFunctionDescriptorArgument.java | 87 ---
.../plan/relational/sql/ast/Update.java | 6 +-
.../plan/relational/sql/parser/AstBuilder.java | 104 ++--
.../plan/relational/sql/rewrite/ShowRewrite.java | 51 +-
.../plan/relational/sql/util/SqlFormatter.java | 23 -
.../plan/scheduler/load/LoadTsFileScheduler.java | 9 +-
.../plan/statement/StatementVisitor.java | 18 +-
.../plan/statement/crud/InsertBaseStatement.java | 22 +-
.../plan/statement/crud/InsertRowStatement.java | 18 +-
.../plan/statement/crud/LoadTsFileStatement.java | 9 +
.../metadata/AlterTimeSeriesStatement.java | 7 +-
.../metadata/pipe/AlterPipeStatement.java | 31 +-
.../metadata/pipe/DropPipePluginStatement.java | 17 +-
.../statement/metadata/pipe/DropPipeStatement.java | 23 +-
.../metadata/pipe/ShowPipePluginsStatement.java | 23 +-
.../metadata/pipe/ShowPipesStatement.java | 18 +-
.../metadata/pipe/StartPipeStatement.java | 17 +-
.../statement/metadata/pipe/StopPipeStatement.java | 17 +-
.../region/ExtendRegionStatement.java} | 32 +-
.../{ => region}/MigrateRegionStatement.java | 2 +-
.../region/ReconstructRegionStatement.java} | 31 +-
.../region/RemoveRegionStatement.java} | 32 +-
.../plan/statement/pipe/PipeEnrichedStatement.java | 10 +-
.../attribute/DeviceAttributeStore.java | 10 +-
.../attribute/IDeviceAttributeStore.java | 6 +-
.../schemaregion/impl/SchemaRegionMemoryImpl.java | 136 +++--
.../schemaregion/mtree/impl/mem/MemMTreeStore.java | 52 +-
.../mtree/impl/mem/mnode/basic/BasicMNode.java | 22 +-
.../impl/mem/snapshot/MemMTreeSnapshotUtil.java | 37 +-
.../schemaengine/table/InformationSchemaUtils.java | 11 +-
.../db/service/metrics/DataNodeMetricsHelper.java | 4 +-
.../metrics/memory/ConsensusMemoryMetrics.java} | 64 +-
.../metrics/memory/GlobalMemoryMetrics.java | 132 ++++
.../metrics/memory/QueryEngineMemoryMetrics.java | 179 ++++++
.../metrics/memory/SchemaEngineMemoryMetrics.java | 124 ++++
.../metrics/memory/StorageEngineMemoryMetrics.java | 210 +++++++
.../metrics/memory/StreamEngineMemoryMetrics.java} | 64 +-
.../metrics/memory/ThresholdMemoryMetrics.java | 194 ------
.../db/storageengine/dataregion/DataRegion.java | 123 ++--
.../task/InsertionCrossSpaceCompactionTask.java | 3 -
.../utils/CompactionTableSchemaCollector.java | 2 +-
.../compaction/repair/RepairDataFileScanUtil.java | 30 +-
.../schedule/CompactionScheduleContext.java | 21 +
.../compaction/schedule/CompactionScheduler.java | 60 +-
.../impl/RewriteCrossSpaceCompactionSelector.java | 11 +-
.../dataregion/memtable/TsFileProcessor.java | 4 +
.../dataregion/modification/ModificationFile.java | 89 ++-
.../modification/PartitionLevelModFileManager.java | 5 +-
.../dataregion/snapshot/SnapshotTaker.java | 2 +-
.../dataregion/tsfile/TsFileResource.java | 10 +-
.../tsfile/timeindex/ArrayDeviceTimeIndex.java | 20 +
.../db/storageengine/load/LoadTsFileManager.java | 23 +-
.../load/active/ActiveLoadDirScanner.java | 27 +-
.../load/active/ActiveLoadTsFileLoader.java | 13 +-
.../LoadConvertedInsertTabletStatement.java | 18 +-
...leStatementDataTypeConvertExecutionVisitor.java | 10 +-
...eeStatementDataTypeConvertExecutionVisitor.java | 3 +-
.../converter/LoadTsFileDataTypeConverter.java | 78 ++-
.../load/splitter/TsFileSplitter.java | 5 +-
.../broker/SubscriptionPrefetchingQueue.java | 2 +
.../db/subscription/event/SubscriptionEvent.java | 21 +-
.../batch/SubscriptionPipeTabletEventBatch.java | 24 +-
.../batch/SubscriptionPipeTsFileEventBatch.java | 11 +-
.../db/tools/schema/SRStatementGenerator.java | 208 +++++--
.../tools/schema/SchemaRegionSnapshotParser.java | 32 +-
.../iotdb/db/utils/constant/SqlConstant.java | 1 +
.../iotdb/db/utils/datastructure/BinaryTVList.java | 6 +-
.../db/utils/datastructure/BooleanTVList.java | 6 +-
.../iotdb/db/utils/datastructure/DoubleTVList.java | 6 +-
.../iotdb/db/utils/datastructure/FloatTVList.java | 6 +-
.../iotdb/db/utils/datastructure/IntTVList.java | 6 +-
.../iotdb/db/utils/datastructure/LongTVList.java | 6 +-
.../connector/PipeDataNodeThriftRequestTest.java | 22 +-
.../PipeStatementTablePatternParseVisitorTest.java | 57 ++
... PipeStatementTreePatternParseVisitorTest.java} | 20 +-
.../pipe/connector/PipeTabletEventSorterTest.java | 263 ++++++--
.../db/pipe/consensus/DeletionResourceTest.java | 39 +-
.../pipe/event/PipeTabletInsertionEventTest.java | 25 +-
.../PipePlanTablePatternParseVisitorTest.java | 129 ++++
...va => PipePlanTreePatternParseVisitorTest.java} | 32 +-
.../annotation/PipePluginAnnotationTest.java} | 20 +-
.../resource/PipeTsFileResourceManagerTest.java | 2 +-
.../process/tvf/TableFunctionOperatorTest.java | 57 +-
.../metadata/write/MetadataWriteNodeSerDeTest.java | 2 +-
.../plan/relational/analyzer/DistinctTest.java | 91 +++
.../plan/relational/planner/SubqueryTest.java | 310 ++++++++++
.../planner/assertions/PlanMatchPattern.java | 17 +
.../{SortMatcher.java => SemiJoinMatcher.java} | 41 +-
.../storageengine/buffer/BloomFilterCacheTest.java | 5 +-
.../storageengine/dataregion/DataRegionTest.java | 24 +-
.../compaction/CompactionValidationTest.java | 7 +-
.../InsertionCrossSpaceCompactionSelectorTest.java | 4 +-
.../cross/InsertionCrossSpaceCompactionTest.java | 64 +-
.../inner/InnerSpaceCompactionSelectorTest.java | 2 +-
.../repair/RepairDataFileScanUtilTest.java | 2 +-
.../TableModelFastCompactionPerformerTest.java | 13 +-
...TableModelReadChunkCompactionPerformerTest.java | 13 +-
...TableModelReadPointCompactionPerformerTest.java | 13 +-
.../modification/ModificationFileTest.java | 90 ++-
.../reader/chunk/MemAlignedChunkLoaderTest.java | 2 +-
.../read/reader/chunk/MemChunkLoaderTest.java | 12 +-
.../file/UnsealedTsFileRecoverPerformerTest.java | 4 +-
.../iotdb/db/tools/TsFileAndModSettleToolTest.java | 2 +-
.../iotdb/db/tools/TsFileSelfCheckToolTest.java | 6 +-
.../iotdb/db/tools/TsFileSketchToolTest.java | 10 +-
.../db/utils/SchemaRegionSnapshotParserTest.java | 200 ++++--
iotdb-core/node-commons/pom.xml | 4 +
.../conf/iotdb-system.properties.template | 6 +-
.../exception}/table/ColumnNotExistsException.java | 2 +-
.../table/TableAlreadyExistsException.java | 2 +-
.../exception}/table/TableNotExistsException.java | 2 +-
.../commons/pipe/agent/plugin/PipePluginAgent.java | 17 +-
.../builtin/connector/PlaceholderConnector.java | 4 +
.../connector/donothing/DoNothingConnector.java | 4 +
.../iotdb/airgap/IoTDBAirGapConnector.java | 4 +
.../consensus/PipeConsensusAsyncConnector.java | 4 +
.../iotdb/thrift/IoTDBLegacyPipeConnector.java | 2 +
.../iotdb/thrift/IoTDBThriftAsyncConnector.java | 5 +
.../iotdb/thrift/IoTDBThriftConnector.java | 4 +
.../iotdb/thrift/IoTDBThriftSslConnector.java | 4 +
.../iotdb/thrift/IoTDBThriftSyncConnector.java | 5 +
.../builtin/connector/opcua/OpcUaConnector.java | 4 +
.../connector/websocket/WebSocketConnector.java | 2 +
.../connector/writeback/WriteBackConnector.java | 4 +
.../extractor/donothing/DoNothingExtractor.java | 4 +
.../builtin/extractor/iotdb/IoTDBExtractor.java | 4 +
.../processor/donothing/DoNothingProcessor.java | 4 +
.../throwing/ThrowingExceptionProcessor.java | 4 +
.../agent/plugin/meta/PipePluginMetaKeeper.java | 47 +-
.../commons/pipe/agent/task/PipeTaskAgent.java | 2 +-
.../pipe/agent/task/meta/PipeMetaKeeper.java | 12 +-
.../pipe/agent/task/meta/PipeStaticMeta.java | 14 +
.../config/constant/PipeExtractorConstant.java | 3 +
.../connector/client/IoTDBSyncClientManager.java | 28 +-
.../payload/thrift/request/PipeRequestType.java | 4 +-
.../thrift/request/PipeTransferFileSealReqV2.java | 13 +-
.../connector/protocol/IoTDBAirGapConnector.java | 14 +-
.../pipe/connector/protocol/IoTDBConnector.java | 10 +
.../connector/protocol/IoTDBSslSyncConnector.java | 4 +
.../options/PipeInclusionOptions.java | 43 +-
.../pipe/datastructure/pattern/TablePattern.java | 29 +-
.../pipe/datastructure/pattern/TreePattern.java | 25 +-
.../{ => resource}/PersistentResource.java | 2 +-
.../commons/pipe/datastructure/result/Result.java | 39 +-
.../pipe/datastructure/visibility/Visibility.java | 9 +-
.../visibility/VisibilityTestUtils.java | 135 +++++
.../datastructure/visibility/VisibilityUtils.java | 133 ++++
.../commons/pipe/extractor/IoTDBExtractor.java | 103 +++-
.../extractor/IoTDBNonDataRegionExtractor.java | 24 +-
.../commons/pipe/receiver/IoTDBFileReceiver.java | 49 +-
.../commons/pipe/receiver/IoTDBReceiverAgent.java | 7 +-
.../snapshot/PipeSnapshotResourceManager.java | 9 +-
.../apache/iotdb/commons/schema/node/IMNode.java | 18 +-
.../commons/schema/node/visitor/MNodeVisitor.java | 6 +-
.../schema/table/column/TsTableColumnCategory.java | 31 -
.../iotdb/commons/service/metric/enums/Metric.java | 3 +-
.../iotdb/commons/udf/access/RecordIterator.java | 9 +-
.../TableBuiltinAggregationFunction.java | 2 +
.../org/apache/iotdb/commons/utils/FileUtils.java | 49 ++
.../org/apache/iotdb/commons/utils/PathUtils.java | 2 +-
.../utils/{TestOnly.java => RetryUtils.java} | 38 +-
.../annotation/PipePluginAnnotationTest.java} | 20 +-
.../db/relational/grammar/sql/RelationalSql.g4 | 23 +-
.../thrift-commons/src/main/thrift/common.thrift | 3 +-
.../src/main/thrift/confignode.thrift | 69 +++
pom.xml | 6 +-
633 files changed, 23038 insertions(+), 7325 deletions(-)
rename
integration-test/src/main/java/org/apache/iotdb/db/query/udf/example/relational/{DatePlusOne.java
=> DatePlus.java} (55%)
create mode 100644
integration-test/src/main/java/org/apache/iotdb/it/utils/TsFileTableGenerator.java
rename
integration-test/src/test/java/org/apache/iotdb/confignode/it/regionmigration/{IoTDBRegionMigrateReliabilityITFramework.java
=> IoTDBRegionOperationReliabilityITFramework.java} (82%)
create mode 100644
integration-test/src/test/java/org/apache/iotdb/confignode/it/regionmigration/pass/commit/IoTDBRegionGroupExpandAndShrinkForIoTV1IT.java
create mode 100644
integration-test/src/test/java/org/apache/iotdb/confignode/it/regionmigration/pass/commit/IoTDBRegionReconstructForIoTV1IT.java
create mode 100644
integration-test/src/test/java/org/apache/iotdb/db/it/IoTDBLoadTsFileWithModIT.java
copy
integration-test/src/test/java/org/apache/iotdb/pipe/it/manual/{IoTDBPipeInclusionIT.java
=> IoTDBPipeTableManualIT.java} (54%)
create mode 100644
integration-test/src/test/java/org/apache/iotdb/pipe/it/tablemodel/IoTDBPipeDoubleLivingIT.java
create mode 100644
integration-test/src/test/java/org/apache/iotdb/pipe/it/tablemodel/IoTDBPipeIsolationIT.java
delete mode 100644
integration-test/src/test/java/org/apache/iotdb/relational/it/query/old/query/IoTDBNullValueFillTableIT.java
create mode 100644
integration-test/src/test/java/org/apache/iotdb/relational/it/query/recent/subquery/uncorrelated/IoTDBUncorrelatedInPredicateSubqueryIT.java
create mode 100644
integration-test/src/test/java/org/apache/iotdb/relational/it/query/recent/subquery/uncorrelated/IoTDBUncorrelatedQuantifiedComparisonIT.java
rename
integration-test/src/test/java/org/apache/iotdb/relational/it/query/recent/subquery/{
=> uncorrelated}/IoTDBUncorrelatedScalarSubqueryIT.java (99%)
rename
integration-test/src/test/java/org/apache/iotdb/relational/it/rest/it/{IoTDBRestServiceInsertAlignedValuesIT.java
=> IoTDBRestServiceInsertValuesIT.java} (96%)
create mode 100644
integration-test/src/test/java/org/apache/iotdb/util/MagicUtils.java
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/schemaengine/schemaregion/SchemaRegion.java
=>
iotdb-api/pipe-api/src/main/java/org/apache/iotdb/pipe/api/annotation/TableModel.java
(66%)
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/schemaengine/schemaregion/SchemaRegion.java
=>
iotdb-api/pipe-api/src/main/java/org/apache/iotdb/pipe/api/annotation/TreeModel.java
(66%)
create mode 100644
iotdb-api/udf-api/src/main/java/org/apache/iotdb/udf/api/customizer/analysis/AggregateFunctionAnalysis.java
copy
integration-test/src/main/java/org/apache/iotdb/itbase/category/AIClusterIT.java
=>
iotdb-api/udf-api/src/main/java/org/apache/iotdb/udf/api/customizer/analysis/FunctionAnalysis.java
(79%)
copy
iotdb-api/udf-api/src/main/java/org/apache/iotdb/udf/api/customizer/{config/UDFConfigurations.java
=> analysis/ScalarFunctionAnalysis.java} (56%)
rename
iotdb-api/udf-api/src/main/java/org/apache/iotdb/udf/api/customizer/parameter/{FunctionParameters.java
=> FunctionArguments.java} (73%)
copy
iotdb-api/udf-api/src/main/java/org/apache/iotdb/udf/api/exception/{UDFParameterNotValidException.java
=> UDFArgumentNotValidException.java} (87%)
delete mode 100644
iotdb-api/udf-api/src/main/java/org/apache/iotdb/udf/api/relational/table/argument/Descriptor.java
delete mode 100644
iotdb-api/udf-api/src/main/java/org/apache/iotdb/udf/api/relational/table/argument/DescriptorArgument.java
delete mode 100644
iotdb-api/udf-api/src/main/java/org/apache/iotdb/udf/api/relational/table/specification/DescriptorParameterSpecification.java
delete mode 100644
iotdb-api/udf-api/src/main/java/org/apache/iotdb/udf/api/relational/table/specification/ReturnTypeSpecification.java
delete mode 100644
iotdb-api/udf-api/src/main/java/org/apache/iotdb/udf/api/type/ColumnCategory.java
copy
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/cluster/NodeType.java
=>
iotdb-client/session/src/main/java/org/apache/iotdb/session/util/RetryUtils.java
(59%)
copy
iotdb-core/confignode/src/main/java/org/apache/iotdb/confignode/consensus/request/read/{subscription/ShowSubscriptionPlan.java
=> table/DescTable4InformationSchemaPlan.java} (79%)
copy
iotdb-core/confignode/src/main/java/org/apache/iotdb/confignode/consensus/request/read/{subscription/ShowSubscriptionPlan.java
=> table/ShowTable4InformationSchemaPlan.java} (79%)
copy
iotdb-core/confignode/src/main/java/org/apache/iotdb/confignode/consensus/request/write/{table/CommitCreateTablePlan.java
=> pipe/payload/PipeCreateTablePlan.java} (64%)
create mode 100644
iotdb-core/confignode/src/main/java/org/apache/iotdb/confignode/consensus/request/write/pipe/payload/PipeDeleteDevicesPlan.java
copy
iotdb-core/confignode/src/main/java/org/apache/iotdb/confignode/consensus/response/table/{FetchTableResp.java
=> DescTable4InformationSchemaResp.java} (61%)
copy
iotdb-core/confignode/src/main/java/org/apache/iotdb/confignode/consensus/response/table/{ShowTableResp.java
=> ShowTable4InformationSchemaResp.java} (64%)
create mode 100644
iotdb-core/confignode/src/main/java/org/apache/iotdb/confignode/manager/pipe/extractor/PipeConfigPhysicalPlanTablePatternParseVisitor.java
rename
iotdb-core/confignode/src/main/java/org/apache/iotdb/confignode/manager/pipe/extractor/{PipeConfigPhysicalPlanPatternParseVisitor.java
=> PipeConfigPhysicalPlanTreePatternParseVisitor.java} (98%)
create mode 100644
iotdb-core/confignode/src/main/java/org/apache/iotdb/confignode/procedure/impl/region/ReconstructRegionProcedure.java
copy
iotdb-core/{node-commons/src/main/java/org/apache/iotdb/commons/pipe/config/plugin/env/PipeTaskRuntimeEnvironment.java
=>
confignode/src/main/java/org/apache/iotdb/confignode/procedure/impl/region/RegionOperationProcedure.java}
(54%)
copy
iotdb-core/confignode/src/main/java/org/apache/iotdb/confignode/procedure/state/{RegionTransitionState.java
=> ReconstructRegionState.java} (93%)
create mode 100644
iotdb-core/confignode/src/test/java/org/apache/iotdb/confignode/manager/pipe/extractor/PipeConfigPhysicalPlanTablePatternParseVisitorTest.java
rename
iotdb-core/confignode/src/test/java/org/apache/iotdb/confignode/manager/pipe/extractor/{PipeConfigPhysicalPlanPatternParseVisitorTest.java
=> PipeConfigPhysicalPlanTreePatternParseVisitorTest.java} (91%)
copy
iotdb-core/{datanode/src/test/java/org/apache/iotdb/db/queryengine/exception/CpuNotEnoughExceptionTest.java
=>
confignode/src/test/java/org/apache/iotdb/confignode/pipe/annotation/PipePluginAnnotationTest.java}
(62%)
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/exception/{VerifyMetadataException.java
=> load/LoadAnalyzeException.java} (82%)
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/exception/{VerifyMetadataTypeMismatchException.java
=> load/LoadAnalyzeTableColumnDisorderException.java} (78%)
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/exception/{VerifyMetadataTypeMismatchException.java
=> load/LoadAnalyzeTypeMismatchException.java} (83%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/connector/util/builder/PipeTableModeTsFileBuilder.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/connector/util/builder/PipeTreeModelTsFileBuilder.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/connector/util/builder/PipeTsFileBuilder.java
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/connector/util/{ =>
cacher}/LeaderCacheUtils.java (97%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/connector/util/sorter/PipeTableModelTabletEventSorter.java
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/connector/util/{ =>
sorter}/PipeTabletEventSorter.java (56%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/connector/util/sorter/PipeTreeModelTabletEventSorter.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/extractor/schemaregion/PipePlanTablePatternParseVisitor.java
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/extractor/schemaregion/{PipePlanPatternParseVisitor.java
=> PipePlanTreePatternParseVisitor.java} (98%)
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/{queryengine/plan/relational/utils/NodeUtils.java
=> pipe/receiver/visitor/PipeStatementTablePatternParseVisitor.java} (55%)
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/receiver/visitor/{PipeStatementPatternParseVisitor.java
=> PipeStatementTreePatternParseVisitor.java} (97%)
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/receiver/visitor/{PipeStatementToBatchVisitor.java
=> PipeTreeStatementToBatchVisitor.java} (98%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/process/function/partition/Slice.java
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/process/{fill/identity/IdentityLinearFill.java
=> function/partition/SliceCache.java} (53%)
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/source/relational/{MergeSortInnerJoinOperator.java
=> MergeSortSemiJoinOperator.java} (58%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/source/relational/aggregation/AggregationMask.java
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/source/relational/aggregation/{CountIfAccumulator.java
=> CountAllAccumulator.java} (74%)
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/source/relational/{TableScanOperator.java
=> aggregation/MaskedRecordIterator.java} (58%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/source/relational/aggregation/grouped/hash/MarkDistinctHash.java
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/execution/config/metadata/{SetTTLTask.java
=> region/ExtendRegionTask.java} (79%)
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/execution/config/metadata/{
=> region}/MigrateRegionTask.java (94%)
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/execution/config/metadata/{DropFunctionTask.java
=> region/ReconstructRegionTask.java} (73%)
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/execution/config/metadata/{SetTTLTask.java
=> region/RemoveRegionTask.java} (79%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/SimplePlanRewriter.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/iterative/rule/PruneApplyColumns.java
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/iterative/rule/{PruneCorrelatedJoinCorrelation.java
=> PruneApplyCorrelation.java} (62%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/iterative/rule/PruneApplySourceColumns.java
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/iterative/rule/{RemoveRedundantEnforceSingleRowNode.java
=> RemoveUnreferencedScalarApplyNodes.java} (67%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/iterative/rule/RemoveUnreferencedScalarSubqueries.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/iterative/rule/TransformFilteringSemiJoinToInnerJoin.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/iterative/rule/TransformUncorrelatedInPredicateSubqueryToSemiJoin.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/node/SemiJoinNode.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/optimizations/TransformQuantifiedComparisonApplyToCorrelatedJoin.java
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/sql/ast/{DescribeTable.java
=> CountStatement.java} (65%)
delete mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/sql/ast/Descriptor.java
delete mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/sql/ast/DescriptorField.java
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/sql/ast/{SetConfiguration.java
=> StartRepairData.java} (85%)
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/sql/ast/{SetConfiguration.java
=> StopRepairData.java} (85%)
delete mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/sql/ast/TableFunctionDescriptorArgument.java
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/statement/{sys/MergeStatement.java
=> metadata/region/ExtendRegionStatement.java} (74%)
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/statement/metadata/{
=> region}/MigrateRegionStatement.java (96%)
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/statement/{sys/LoadConfigurationStatement.java
=> metadata/region/ReconstructRegionStatement.java} (73%)
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/statement/{sys/MergeStatement.java
=> metadata/region/RemoveRegionStatement.java} (74%)
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/{protocol/thrift/handler/InternalServiceThriftHandlerMetrics.java
=> service/metrics/memory/ConsensusMemoryMetrics.java} (52%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/service/metrics/memory/GlobalMemoryMetrics.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/service/metrics/memory/QueryEngineMemoryMetrics.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/service/metrics/memory/SchemaEngineMemoryMetrics.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/service/metrics/memory/StorageEngineMemoryMetrics.java
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/{protocol/thrift/handler/InternalServiceThriftHandlerMetrics.java
=> service/metrics/memory/StreamEngineMemoryMetrics.java} (51%)
delete mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/service/metrics/memory/ThresholdMemoryMetrics.java
create mode 100644
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/pipe/connector/PipeStatementTablePatternParseVisitorTest.java
rename
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/pipe/connector/{PipeStatementPatternParseVisitorTest.java
=> PipeStatementTreePatternParseVisitorTest.java} (94%)
create mode 100644
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/pipe/extractor/PipePlanTablePatternParseVisitorTest.java
rename
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/pipe/extractor/{PipePlanPatternParseVisitorTest.java
=> PipePlanTreePatternParseVisitorTest.java} (94%)
copy
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/{queryengine/exception/CpuNotEnoughExceptionTest.java
=> pipe/plugin/annotation/PipePluginAnnotationTest.java} (62%)
copy
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/queryengine/plan/relational/planner/assertions/{SortMatcher.java
=> SemiJoinMatcher.java} (59%)
rename
iotdb-core/{datanode/src/main/java/org/apache/iotdb/db/exception/metadata =>
node-commons/src/main/java/org/apache/iotdb/commons/exception}/table/ColumnNotExistsException.java
(95%)
rename
iotdb-core/{datanode/src/main/java/org/apache/iotdb/db/exception/metadata =>
node-commons/src/main/java/org/apache/iotdb/commons/exception}/table/TableAlreadyExistsException.java
(95%)
rename
iotdb-core/{datanode/src/main/java/org/apache/iotdb/db/exception/metadata =>
node-commons/src/main/java/org/apache/iotdb/commons/exception}/table/TableNotExistsException.java
(95%)
rename
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/pipe/datastructure/{
=> resource}/PersistentResource.java (95%)
copy
library-udf/src/main/java/org/apache/iotdb/library/match/model/SectionNext.java
=>
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/pipe/datastructure/result/Result.java
(59%)
copy
integration-test/src/main/java/org/apache/iotdb/itbase/category/MultiClusterIT2SubscriptionRegressionConsumer.java
=>
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/pipe/datastructure/visibility/Visibility.java
(86%)
create mode 100644
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/pipe/datastructure/visibility/VisibilityTestUtils.java
create mode 100644
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/pipe/datastructure/visibility/VisibilityUtils.java
copy
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/utils/{TestOnly.java
=> RetryUtils.java} (60%)
copy
iotdb-core/{datanode/src/test/java/org/apache/iotdb/db/queryengine/exception/CpuNotEnoughExceptionTest.java
=>
node-commons/src/test/java/org/apache/iotdb/commons/pipe/plugin/annotation/PipePluginAnnotationTest.java}
(62%)
