This is an automated email from the ASF dual-hosted git repository.
yongzao pushed a change to branch model-management
in repository https://gitbox.apache.org/repos/asf/iotdb.git
discard 09f98a0308c update torch version, should less than 2.8.0
omit 78922c93c70 Update builtin model path in CI envs
omit 5ad17938767 If model is already ACTIVATING or ACTIVE, skip duplicate
update and download (#16868)
omit 2b3157731f9 Fix call inference cannot specify outputLength
omit e38e67fa7e1 Add dependencies of python packages for arima,
gaussian_hmm and efficient model loading (#16865)
omit b4c10a7d969 spotless ainode codes
omit 53faad8a85f support various pipeline Interfaces and support arima with
sktime package (#16861)
omit d7d42e08997 Update model_loader.py
omit 426a406701c support pipeline for sktime models
omit 4fc01bda26f unify some parameter names
omit 3c9b8e68897 Main process manages models, child process loads models.
(#16850)
omit f5a382f9732 Support loading inference pipelines for user-defined
models (#16835)
omit 3008d202f13 Remove useless codes in CN
omit 76459b92917 call inference bug fix
omit 3335e1ebde1 refactor AIN CI tests
omit 687ae9f15ca Fix IoTDBDatabaseIT
omit 8ca64570faf Support loading inference pipelines for user-defined models
omit ca94b0694bd remove useless codes in IoTDB
omit 3448a168596 Reconstruct model management and model loading
omit 5215e53dd38 delete old code in model folder
omit 22a8589d7e5 refactor_built_in_models
add d99740860c2 may_fix (#16814)
add fb899b16b92 Fixed the bug of 305 error for unsupported table
opreations #16816
add ddcc64618b5 Fixed the calculation error of schema cache (#16818)
add 03bc2606563 Fixed the concurrency issue of region migrate and load
(#16796)
add c4114b95d94 Add 'show available urls' (#16441)
add 19ebbe31099 Added back the mis-deleted "()" in last cache (#16824)
add 8cb4b9be877 Reduced the auth log when paths are too many (#16825)
add 23be2201b81 perf: Optimize wide table write performance (#16699)
add 85b8f168d7c implement the except (distinct | all ) for table model
#16760
add 85281fac2a3 added support for Blob objects to the length() function,
allowing for calculating the byte size for Blob inputs (#16170)
add bd813d85c62 implement intersect and except related optimization rule:
MergeIntersect, MergeExcept, RemoveEmptyIntersectBranchs,
EvaluateEmptyIntersect, PruneIntersectSourceColumns, PruneExceptSourceColmns
(#16761)
add e3127e6c61f Pipe: Ignore logging when `returnSelf` is called in the
event of an exception in `AsyncClient`. (#16827)
add cb705d17b85 Fixed the schema cache calculation 2 & The potential NPE
caused by concurrent invalidate and update (#16834)
add 8cd141895b1 Fix the check of grant option for tree model (#16845)
add f68c49e349b Pipe: Fixed the bug that lower version tablet may cause
NPE when sent to 2.x version & The temporary exception may be wrongly reported
(#16843)
add 263c23bd69d Pipe: Fixed the auth implementation error (#16847)
add 45f26455926 Fix NPE during compaction when disk space is insufficient
(#16841)
add db02437f00f PipePlugin: Optimized the errorCode && Fixed the
case-sensitive semantic (#16851)
add 5bc4779ba43 Pipe: Fix NullPointerException in concurrent event access
(#16849)
add 5cf60df3afc may-c (#16859)
add f4d628a7d6a Pipe: Prevent duplicate error logging in
AsyncPipeDataTransferServiceClient (#16856)
add 1b2b32bc2a3 perf: Optimize TypeManager allocation in StatementAnalyzer
(#16870)
add 1746cdb28b1 Add system table named connections to resolve the idle
session can be found (#16846)
add c5e8663f30d refactor_built_in_models
add e07dad84280 delete old code in model folder
add f6ef03f2079 Reconstruct model management and model loading
add 20923d55c75 remove useless codes in IoTDB
add d6514890951 Support loading inference pipelines for user-defined models
add 59fe301c763 Fix IoTDBDatabaseIT
add e4412899444 refactor AIN CI tests
add 64364173236 call inference bug fix
add c247bd14c92 Remove useless codes in CN
add 1b7fb275965 Support loading inference pipelines for user-defined
models (#16835)
add 98ab1b3d8db Main process manages models, child process loads models.
(#16850)
add c066df48a6d unify some parameter names
add 0e584b73590 support pipeline for sktime models
add a8bfbcc6133 Update model_loader.py
add 8d00ce77ea6 support various pipeline Interfaces and support arima with
sktime package (#16861)
add 8e8cd6f28b7 spotless ainode codes
add 71daf3b45f8 Add dependencies of python packages for arima,
gaussian_hmm and efficient model loading (#16865)
add 381aea89a70 Fix call inference cannot specify outputLength
add f8751af22ed If model is already ACTIVATING or ACTIVE, skip duplicate
update and download (#16868)
add 89db67f32df Update builtin model path in CI envs
add ef84301661d update torch version, should less than 2.8.0
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (09f98a0308c)
\
N -- N -- N refs/heads/model-management (ef84301661d)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../org/apache/iotdb/db/it/auth/IoTDBAuthIT.java | 16 +
.../iotdb/db/it/selectinto/IoTDBSelectIntoIT.java | 13 +-
.../iotdb/db/it/utils/TSDataTypeTestUtils.java | 113 ++++
.../pipe/it/dual/tablemodel/TableModelUtils.java | 8 +-
.../treemodel/auto/basic/IoTDBPipeSyntaxIT.java | 61 ++
.../pipe/it/single/IoTDBPipePermissionIT.java | 27 +-
.../relational/it/db/it/IoTDBInsertTableIT.java | 6 +
.../scalar/IoTDBScalarFunctionTableIT.java | 23 +-
...tersectTableIT.java => IoTDBExceptTableIT.java} | 72 +-
...nFunctionIT.java => IoTDBLengthFunctionIT.java} | 80 +--
.../relational/it/schema/IoTDBDatabaseIT.java | 10 +-
.../iotdb/relational/it/schema/IoTDBTableIT.java | 16 +
.../it/session/IoTDBSessionRelationalIT.java | 14 +-
.../iotdb/session/it/IoTDBConnectionsIT.java | 364 ++++++++++
.../org/apache/iotdb/session/NodesSupplier.java | 49 +-
.../org/apache/iotdb/db/qp/sql/IdentifierParser.g4 | 2 +
.../org/apache/iotdb/db/qp/sql/IoTDBSqlParser.g4 | 7 +-
.../antlr4/org/apache/iotdb/db/qp/sql/SqlLexer.g4 | 8 +
.../org/apache/iotdb/db/auth/AuthorityChecker.java | 9 +-
.../java/org/apache/iotdb/db/conf/IoTDBConfig.java | 10 -
.../org/apache/iotdb/db/conf/IoTDBDescriptor.java | 6 -
.../db/consensus/DataRegionConsensusImpl.java | 6 +
.../dataregion/DataRegionStateMachine.java | 15 +-
.../subtask/processor/PipeProcessorSubtask.java | 7 +
.../agent/task/subtask/sink/PipeSinkSubtask.java | 7 +-
.../tablet/PipeInsertNodeTabletInsertionEvent.java | 73 +-
.../common/tsfile/PipeTsFileInsertionEvent.java | 2 +-
.../protocol/thrift/IoTDBDataNodeReceiver.java | 4 +-
.../visitor/PipeStatementExceptionVisitor.java | 8 +-
.../visitor/PipeStatementTSStatusVisitor.java | 3 +-
.../handler/PipeTransferTrackableHandler.java | 16 +-
.../async/handler/PipeTransferTsFileHandler.java | 15 +-
.../sink/util/sorter/PipeTabletEventSorter.java | 5 +
.../iotdb/db/protocol/session/IClientSession.java | 16 +
.../iotdb/db/protocol/session/SessionManager.java | 14 +
.../db/queryengine/common/ConnectionInfo.java | 64 ++
.../common/header/DatasetHeaderFactory.java | 4 +
.../fragment/FragmentInstanceContext.java | 2 +-
.../fragment/FragmentInstanceExecution.java | 2 +-
.../InformationSchemaContentSupplierFactory.java | 47 +-
.../relational/ColumnTransformerBuilder.java | 8 +-
.../iotdb/db/queryengine/plan/Coordinator.java | 19 +-
.../dualkeycache/impl/CacheEntryGroupImpl.java | 6 +
.../schema/dualkeycache/impl/DualKeyCacheImpl.java | 61 +-
.../schema/dualkeycache/impl/ICacheEntryGroup.java | 3 +
.../execution/config/TableConfigTaskVisitor.java | 23 +-
.../execution/config/TreeConfigTaskVisitor.java | 8 +
.../config/executor/ClusterConfigTaskExecutor.java | 33 +-
.../config/executor/IConfigTaskExecutor.java | 2 +
...AINodesTask.java => ShowAvailableUrlsTask.java} | 30 +-
.../db/queryengine/plan/parser/ASTVisitor.java | 6 +
.../plan/planner/plan/node/PlanGraphPrinter.java | 10 +
.../plan/planner/plan/node/PlanNodeType.java | 4 +
.../plan/planner/plan/node/PlanVisitor.java | 5 +
.../relational/analyzer/StatementAnalyzer.java | 13 +-
.../analyzer/StatementAnalyzerFactory.java | 17 +-
.../plan/relational/metadata/Metadata.java | 10 +
.../relational/metadata/TableMetadataImpl.java | 24 +-
.../fetcher/TableHeaderSchemaValidator.java | 378 +++++++++++
.../fetcher/cache/TableDeviceCacheEntry.java | 8 +-
.../fetcher/cache/TableDeviceLastCache.java | 56 +-
.../fetcher/cache/TableDeviceSchemaCache.java | 6 +-
.../fetcher/cache/TreeDeviceNormalSchema.java | 6 +-
.../plan/relational/planner/RelationPlanner.java | 24 +-
.../plan/relational/planner/TableModelPlanner.java | 9 +-
...rceColumns.java => EvaluateEmptyIntersect.java} | 41 +-
...ntIntersectAll.java => ImplementExceptAll.java} | 47 +-
...on.java => ImplementExceptDistinctAsUnion.java} | 52 +-
.../rule/ImplementIntersectDistinctAsUnion.java | 3 +-
.../rule/{MergeUnion.java => MergeExcept.java} | 15 +-
.../rule/{MergeUnion.java => MergeIntersect.java} | 13 +-
.../iterative/rule/PruneDistinctAggregation.java | 27 +-
...eColumns.java => PruneExceptSourceColumns.java} | 14 +-
...lumns.java => PruneIntersectSourceColumns.java} | 15 +-
.../iterative/rule/RemoveEmptyExceptBranches.java | 139 ++++
.../planner/iterative/rule/SetOperationMerge.java | 123 +++-
.../iterative/rule/SetOperationNodeTranslator.java | 4 +-
.../node/{IntersectNode.java => ExceptNode.java} | 21 +-
.../plan/relational/planner/node/Patterns.java | 20 +-
.../DataNodeLocationSupplierFactory.java | 1 +
.../optimizations/LogicalOptimizeFactory.java | 43 +-
.../optimizations/UnaliasSymbolReferences.java | 31 +
.../security/TreeAccessCheckVisitor.java | 24 +-
.../plan/relational/sql/ast/AstVisitor.java | 4 +
.../plan/relational/sql/ast/InsertRows.java | 19 +-
.../plan/relational/sql/ast/RenameColumn.java | 5 +-
.../plan/relational/sql/ast/RenameTable.java | 5 +-
.../{ShowAIDevices.java => ShowAvailableUrls.java} | 12 +-
.../relational/sql/ast/WrappedInsertStatement.java | 414 +++++++++---
.../plan/relational/sql/parser/AstBuilder.java | 7 +
.../plan/scheduler/load/LoadTsFileScheduler.java | 6 +-
.../plan/statement/StatementVisitor.java | 6 +
.../plan/statement/crud/InsertBaseStatement.java | 145 +++-
.../crud/InsertMultiTabletsStatement.java | 6 +
.../plan/statement/crud/InsertRowStatement.java | 42 ++
.../crud/InsertRowsOfOneDeviceStatement.java | 6 +
.../plan/statement/crud/InsertRowsStatement.java | 6 +
.../plan/statement/crud/InsertTabletStatement.java | 66 +-
...tement.java => ShowAvailableUrlsStatement.java} | 18 +-
...ormer.java => BlobLengthColumnTransformer.java} | 23 +-
.../db/schemaengine/table/DataNodeTableCache.java | 17 +-
.../iotdb/db/service/metrics/WritingMetrics.java | 3 +-
.../iotdb/db/storageengine/StorageEngine.java | 13 +-
.../db/storageengine/dataregion/DataRegion.java | 287 +++++---
.../dataregion/IDataRegionForQuery.java | 2 +-
.../dataregion/VirtualDataRegion.java | 2 +-
.../execute/task/InnerSpaceCompactionTask.java | 3 +
.../compaction/repair/RepairTimePartition.java | 2 +-
.../dataregion/memtable/TsFileProcessor.java | 18 +-
.../dataregion/snapshot/SnapshotTaker.java | 14 +-
.../file/UnsealedTsFileRecoverPerformer.java | 6 +-
.../db/storageengine/load/LoadTsFileManager.java | 26 +-
.../db/utils/datastructure/AlignedTVList.java | 4 +-
.../apache/iotdb/db/auth/AuthorityCheckerTest.java | 51 ++
.../org/apache/iotdb/db/auth/TreeAccessTest.java | 84 +++
.../cache/TreeDeviceSchemaCacheManagerTest.java | 14 +
.../plan/relational/analyzer/AnalyzerTest.java | 40 +-
.../plan/relational/analyzer/AuthTest.java | 10 +-
...zeTest.java => EvaluateEmptyIntersectTest.java} | 27 +-
.../{IntersectTest.java => ExceptTest.java} | 51 +-
.../plan/relational/analyzer/MergeExceptTest.java | 147 ++++
...{IntersectTest.java => MergeIntersectTest.java} | 99 ++-
...est.java => RemoveEmptyExceptBranchesTest.java} | 106 ++-
.../analyzer/RowPatternRecognitionTest.java | 4 +-
.../plan/relational/analyzer/TSBSMetadata.java | 13 +
.../plan/relational/analyzer/TestMetadata.java | 25 +-
.../fetcher/cache/TableDeviceSchemaCacheTest.java | 9 +
.../plan/relational/planner/PlanTester.java | 4 +-
.../plan/statement/InsertStatementTest.java | 97 ++-
.../scalar/BlobLengthColumnTransformerTest.java | 148 ++++
.../DataNodeInternalRPCServiceImplTest.java | 59 +-
.../iotdb/db/storageengine/StorageEngineTest.java | 4 +-
.../storageengine/dataregion/DataRegionTest.java | 4 +-
.../inner/InnerSpaceCompactionExceptionTest.java | 16 +
.../repair/RepairUnsortedFileSchedulerTest.java | 10 +-
.../dataregion/snapshot/IoTDBSnapshotTest.java | 2 +-
.../apache/iotdb/db/utils/TSDataTypeTestUtils.java | 111 +++
.../datastructure/PrimitiveArrayManagerTest.java | 12 +-
.../apache/iotdb/commons/client/ClientManager.java | 25 +
.../async/AsyncPipeDataTransferServiceClient.java | 21 +-
.../apache/iotdb/commons/conf/CommonConfig.java | 9 +
.../iotdb/commons/conf/CommonDescriptor.java | 5 +
.../pipe/receiver/PipeReceiverStatusHandler.java | 6 +-
.../schema/column/ColumnHeaderConstant.java | 13 +
.../commons/schema/table/InformationSchema.java | 17 +
.../schema/table/InsertNodeMeasurementInfo.java | 244 +++++++
.../schema/table/TsFileTableSchemaUtil.java | 207 ++++++
.../apache/iotdb/commons/schema/table/TsTable.java | 226 ++++---
.../schema/table/column/FieldColumnSchema.java | 1 +
.../schema/table/column/TsTableColumnSchema.java | 6 +
.../apache/iotdb/commons/utils/JVMCommonUtils.java | 7 +-
.../schema/table/TsFileTableSchemaUtilTest.java | 748 +++++++++++++++++++++
.../db/relational/grammar/sql/RelationalSql.g4 | 11 +-
153 files changed, 5419 insertions(+), 1087 deletions(-)
create mode 100644
integration-test/src/test/java/org/apache/iotdb/db/it/utils/TSDataTypeTestUtils.java
copy
integration-test/src/test/java/org/apache/iotdb/relational/it/query/recent/{IoTDBIntersectTableIT.java
=> IoTDBExceptTableIT.java} (63%)
copy
integration-test/src/test/java/org/apache/iotdb/relational/it/query/recent/{scalar/IoTDBFromBase64ColumnFunctionIT.java
=> IoTDBLengthFunctionIT.java} (51%)
create mode 100644
integration-test/src/test/java/org/apache/iotdb/session/it/IoTDBConnectionsIT.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/common/ConnectionInfo.java
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/execution/config/metadata/{relational/ShowAINodesTask.java
=> ShowAvailableUrlsTask.java} (75%)
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/iterative/rule/{PruneUnionSourceColumns.java
=> EvaluateEmptyIntersect.java} (53%)
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/iterative/rule/{ImplementIntersectAll.java
=> ImplementExceptAll.java} (65%)
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/iterative/rule/{ImplementIntersectDistinctAsUnion.java
=> ImplementExceptDistinctAsUnion.java} (69%)
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/iterative/rule/{MergeUnion.java
=> MergeExcept.java} (78%)
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/iterative/rule/{MergeUnion.java
=> MergeIntersect.java} (81%)
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/iterative/rule/{PruneUnionSourceColumns.java
=> PruneExceptSourceColumns.java} (85%)
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/iterative/rule/{PruneUnionSourceColumns.java
=> PruneIntersectSourceColumns.java} (84%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/iterative/rule/RemoveEmptyExceptBranches.java
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/node/{IntersectNode.java
=> ExceptNode.java} (82%)
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/sql/ast/{ShowAIDevices.java
=> ShowAvailableUrls.java} (84%)
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/statement/metadata/{ShowClusterStatement.java
=> ShowAvailableUrlsStatement.java} (79%)
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/transformation/dag/column/unary/scalar/{CRC32Transformer.java
=> BlobLengthColumnTransformer.java} (72%)
create mode 100644
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/auth/AuthorityCheckerTest.java
create mode 100644
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/auth/TreeAccessTest.java
copy
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/queryengine/plan/relational/analyzer/{ExplainAnalyzeTest.java
=> EvaluateEmptyIntersectTest.java} (64%)
copy
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/queryengine/plan/relational/analyzer/{IntersectTest.java
=> ExceptTest.java} (70%)
create mode 100644
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/queryengine/plan/relational/analyzer/MergeExceptTest.java
copy
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/queryengine/plan/relational/analyzer/{IntersectTest.java
=> MergeIntersectTest.java} (58%)
copy
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/queryengine/plan/relational/analyzer/{IntersectTest.java
=> RemoveEmptyExceptBranchesTest.java} (50%)
create mode 100644
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/queryengine/transformation/dag/column/unary/scalar/BlobLengthColumnTransformerTest.java
create mode 100644
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/utils/TSDataTypeTestUtils.java
create mode 100644
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/schema/table/InsertNodeMeasurementInfo.java
create mode 100644
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/schema/table/TsFileTableSchemaUtil.java
create mode 100644
iotdb-core/node-commons/src/test/java/org/apache/iotdb/commons/schema/table/TsFileTableSchemaUtilTest.java
