This is an automated email from the ASF dual-hosted git repository.
asf-gitbox-commits pushed a change to branch PHOENIX-7998
in repository https://gitbox.apache.org/repos/asf/phoenix.git
omit 5a7da6ba69 PHOENIX-7998 Declare distance operator token types as lexer
fragments
omit 8fb136eeea PHOENIX-7998 Phase 16: Index Verification, Introspection,
Consistency Checking, and Repair Tools
omit a6b14b3254 PHOENIX-7998 Phase 15: Little-endian vector encoding and
direct SIMD loads
omit 1cec6d6288 PHOENIX-7998 Phase 14: Compatibility and version validation
omit 42752010e1 PHOENIX-7998 Phase 13: Drift scorecarding and background
rebuild
omit 9d9e6b82ec PHOENIX-7998 Phase 12: Adaptive probing and hybrid filtering
omit e68ff07b6e PHOENIX-7998 Phase 11: Single-cell storage support for
vector indexes
omit 68b53f93ec PHOENIX-7998 Phase 10: BSON vector extraction
omit bc20fc3bf6 PHOENIX-7998 Phase 9: Cost-based plan selection and explain
enhancements
omit 59906cca01 PHOENIX-7998 Phase 8: IVF query execution path
omit 8d7d8bcfe6 PHOENIX-7998 Phase 7: Server-side write pipeline for IVF
omit efa4b15b56 PHOENIX-7998 Phase 6: IVF centroid management
omit b327c2b26d PHOENIX-7998 Phase 5: Vector index DDL and grammar
omit 8537d7d8ce PHOENIX-7998 Phase 4: System catalog and metadata extensions
omit d36fecf423 PHOENIX-7998 Phase 3: Exact nearest-neighbor search path
omit 84962101dd PHOENIX-7998 Phase 2: Distance functions and operators
omit 4c65327b08 PHOENIX-7998 Phase 1: Vector data type and SQL grammar
new 6a97771b4e PHOENIX-7998 Phase 1: Vector data type and SQL grammar
new 09692b34af PHOENIX-7998 Phase 2: Distance functions and operators
new a6cb4e45aa PHOENIX-7998 Phase 3: Exact nearest-neighbor search path
new aef42f6d45 PHOENIX-7998 Phase 4: System catalog and metadata extensions
new ed20fc9417 PHOENIX-7998 Phase 5: Vector index DDL and grammar
new 9f99371a58 PHOENIX-7998 Phase 6: IVF centroid management
new 87970aa53e PHOENIX-7998 Phase 7: Server-side write pipeline for IVF
new abc3d75668 PHOENIX-7998 Phase 8: IVF query execution path
new 373e6eaac8 PHOENIX-7998 Phase 9: Cost-based plan selection and explain
enhancements
new 509ce3014e PHOENIX-7998 Phase 10: BSON vector extraction
new 25acdb5ad8 PHOENIX-7998 Phase 11: Single-cell storage support for
vector indexes
new da30e97fff PHOENIX-7998 Phase 12: Adaptive probing and hybrid filtering
new 3ffef7e2e0 PHOENIX-7998 Phase 13: Drift scorecarding and background
rebuild
new deae510427 PHOENIX-7998 Phase 14: Compatibility and version validation
new be5a8678e7 PHOENIX-7998 Phase 15: Little-endian vector encoding and
direct SIMD loads
new 840c115d94 PHOENIX-7998 Phase 16: Index Verification, Introspection,
Consistency Checking, and Repair Tools
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (5a7da6ba69)
\
N -- N -- N refs/heads/PHOENIX-7998 (840c115d94)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 16 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
phoenix-client-parent/pom.xml | 38 ++
phoenix-core-client/pom.xml | 2 +-
phoenix-core-client/src/main/antlr3/PhoenixSQL.g | 15 +-
.../hbase/ipc/controller/ServerRpcController.java | 4 +
.../apache/phoenix/cache/VectorCentroidCache.java | 139 +++++--
.../phoenix/compile/CreateIndexCompiler.java | 26 +-
.../org/apache/phoenix/compile/DeleteCompiler.java | 5 +-
.../phoenix/compile/ExplainPlanAttributes.java | 2 +-
.../apache/phoenix/compile/OrderByCompiler.java | 7 +-
.../apache/phoenix/compile/ProjectionCompiler.java | 35 +-
.../org/apache/phoenix/compile/QueryCompiler.java | 2 +-
.../apache/phoenix/compile/SubselectRewriter.java | 10 +-
.../coprocessorclient/MetaDataProtocol.java | 2 -
.../apache/phoenix/exception/SQLExceptionCode.java | 4 +-
.../java/org/apache/phoenix/execute/ScanPlan.java | 5 +-
.../phoenix/execute/VectorIndexScanPlan.java | 221 ++++++++---
.../function/BsonVectorValueFunction.java | 36 +-
.../function/CosineDistanceFunction.java | 4 +-
.../expression/function/DistanceFunction.java | 33 +-
.../expression/function/DistanceKernel.java | 8 +-
.../function/InnerProductDistanceFunction.java | 5 +-
.../expression/function/L2DistanceFunction.java | 2 +-
.../function/L2DistanceSquaredFunction.java | 5 +-
.../expression/function/ScalarDistanceKernel.java | 15 +-
.../expression/function/VectorDistanceUtil.java | 14 +-
.../index/metrics/MetricsVectorIndexSource.java | 18 +-
.../org/apache/phoenix/index/IndexMaintainer.java | 176 ++++++---
.../phoenix/index/vector/CentroidManager.java | 239 ++++++++++--
.../phoenix/index/vector/ClusterSkewMetrics.java | 24 +-
.../phoenix/index/vector/GenerationSummary.java | 27 +-
.../apache/phoenix/index/vector/KMeansResult.java | 16 +-
.../apache/phoenix/index/vector/KMeansTrainer.java | 56 ++-
.../apache/phoenix/index/vector/ScorecardRow.java | 2 +-
.../phoenix/index/vector/VectorIndexRebuilder.java | 329 ++++++++--------
.../phoenix/index/vector/VectorIndexScorecard.java | 70 ++--
.../phoenix/index/vector/VectorIndexTrainer.java | 93 ++++-
.../iterate/MergeSortTopNResultIterator.java | 5 +-
.../phoenix/iterate/OrderedResultIterator.java | 4 +-
.../apache/phoenix/optimize/DistanceMetric.java | 12 +-
.../apache/phoenix/optimize/OptimizerReasons.java | 2 +
.../apache/phoenix/optimize/QueryOptimizer.java | 84 +++--
.../phoenix/optimize/VectorSearchDescriptor.java | 9 +-
.../apache/phoenix/optimize/VectorSearchUtil.java | 165 ++++++--
.../java/org/apache/phoenix/parse/ColumnDef.java | 4 +-
.../apache/phoenix/parse/CreateIndexStatement.java | 5 +-
.../phoenix/parse/DistanceFunctionParseNode.java | 8 +-
.../java/org/apache/phoenix/parse/HintNode.java | 4 +-
.../parse/IndexExpressionParseNodeRewriter.java | 16 +-
.../java/org/apache/phoenix/parse/OrderByNode.java | 62 ++-
.../org/apache/phoenix/parse/ParseNodeFactory.java | 16 +
.../apache/phoenix/parse/ParseNodeRewriter.java | 21 +-
.../phoenix/query/ConnectionQueryServices.java | 3 +-
.../phoenix/query/ConnectionQueryServicesImpl.java | 10 -
.../org/apache/phoenix/query/QueryServices.java | 23 +-
.../apache/phoenix/query/QueryServicesOptions.java | 2 +-
.../java/org/apache/phoenix/schema/ColumnRef.java | 4 +-
.../org/apache/phoenix/schema/MetaDataClient.java | 85 +++--
.../java/org/apache/phoenix/schema/PTable.java | 22 +-
.../org/apache/phoenix/schema/ValueSchema.java | 4 +-
.../phoenix/schema/types/PArrayDataType.java | 2 +-
.../phoenix/schema/types/PDataTypeFactory.java | 4 +-
.../phoenix/schema/types/PVectorDataType.java | 86 +++--
.../apache/phoenix/schema/types/PVectorDouble.java | 30 +-
.../apache/phoenix/schema/types/PVectorFloat.java | 35 +-
.../java/org/apache/phoenix/util/CostUtil.java | 10 +-
.../java/org/apache/phoenix/util/IndexUtil.java | 2 +-
.../java/org/apache/phoenix/util/MetaDataUtil.java | 7 +-
.../java/org/apache/phoenix/util/SchemaUtil.java | 2 +-
.../expression/function/DistanceKernel.java | 7 +-
.../expression/function/PanamaDistanceKernel.java | 19 +-
.../coprocessor/GlobalIndexRegionScanner.java | 5 +-
.../coprocessor/IndexRepairRegionScanner.java | 75 +++-
.../phoenix/coprocessor/MetaDataEndpointImpl.java | 14 +-
.../coprocessor/tasks/VectorIndexRebuildTask.java | 44 ++-
.../coprocessor/tasks/VectorIndexTaskRunner.java | 118 ++++++
.../tasks/VectorScorecardReconcileTask.java | 93 ++++-
.../phoenix/hbase/index/IndexRegionObserver.java | 116 +++---
.../phoenix/index/vector/ScorecardAccumulator.java | 123 +++---
.../phoenix/iterate/RegionScannerFactory.java | 26 +-
.../VectorDistanceOrderedResultIterator.java | 19 +-
.../phoenix/mapreduce/index/IndexScrutinyTool.java | 12 +-
.../apache/phoenix/mapreduce/index/IndexTool.java | 33 +-
.../phoenix/mapreduce/index/fsck/Finding.java | 5 +-
.../index/fsck/GlobalIndexFsckProvider.java | 186 ++++++---
.../mapreduce/index/fsck/IndexFsckContext.java | 13 +-
.../mapreduce/index/fsck/IndexFsckProvider.java | 43 +--
.../mapreduce/index/fsck/IndexFsckProviders.java | 14 +-
.../mapreduce/index/fsck/IndexFsckTool.java | 47 ++-
.../phoenix/mapreduce/index/fsck/OutputFormat.java | 2 +-
.../phoenix/mapreduce/index/fsck/RepairAction.java | 16 +-
.../phoenix/mapreduce/index/fsck/RepairPlan.java | 4 +-
.../phoenix/mapreduce/index/fsck/Report.java | 12 +-
.../apache/phoenix/mapreduce/index/fsck/Retry.java | 42 ++-
.../mapreduce/index/fsck/RowKeyFormatter.java | 16 +-
.../phoenix/mapreduce/index/fsck/Severity.java | 3 +-
.../mapreduce/index/fsck/VerifyFindings.java | 114 ++++--
.../mapreduce/index/fsck/ivf/IvfIndexContext.java | 47 ++-
.../index/fsck/ivf/IvfIndexFsckProvider.java | 164 +++++---
.../index/fsck/ivf/IvfIndexInspector.java | 72 ++--
.../mapreduce/index/fsck/ivf/IvfIndexReader.java | 62 +--
phoenix-core/pom.xml | 4 +-
.../phoenix/end2end/BsonServerProjectionIT.java | 57 ++-
.../apache/phoenix/end2end/IndexFsckToolIT.java | 161 +++++++-
.../end2end/IndexRepairRegionScannerIT.java | 10 +-
.../org/apache/phoenix/end2end/IvfIndexFsckIT.java | 130 ++++++-
.../apache/phoenix/end2end/IvfIndexInspectIT.java | 7 +-
.../apache/phoenix/end2end/VectorBsonIndexIT.java | 74 ++--
.../phoenix/end2end/VectorCentroidTableIT.java | 82 +++-
.../org/apache/phoenix/end2end/VectorColumnIT.java | 12 +-
.../phoenix/end2end/VectorCrossRegionMergeIT.java | 14 +-
.../phoenix/end2end/VectorDistancePushDownIT.java | 44 +--
.../phoenix/end2end/VectorExactSearchIT.java | 78 ++--
.../phoenix/end2end/VectorHybridFilterIT.java | 87 ++++-
.../org/apache/phoenix/end2end/VectorIndexIT.java | 87 +++--
.../phoenix/end2end/VectorIndexRebuildIT.java | 418 +++++++++++++++++----
.../phoenix/end2end/VectorIndexTestUtil.java | 37 +-
.../apache/phoenix/end2end/VectorIndexWriteIT.java | 110 +++---
.../apache/phoenix/end2end/VectorIvfCostIT.java | 16 +-
.../phoenix/end2end/VectorIvfLifecycleIT.java | 203 +++++++++-
.../apache/phoenix/end2end/VectorIvfQueryIT.java | 375 ++++++++++++++----
.../phoenix/end2end/VectorSystemCatalogIT.java | 23 +-
.../vector/VectorIndexRebuilderTestHooks.java | 9 +-
.../controller/ServerRpcControllerFactoryTest.java | 58 +++
.../phoenix/cache/VectorCentroidCacheTest.java | 162 +++++---
.../phoenix/compile/VectorColumnCompileTest.java | 128 +++++++
.../phoenix/compile/VectorIndexCompilerTest.java | 113 ++++--
.../tasks/VectorIndexRebuildTaskTest.java | 78 ++++
.../tasks/VectorIndexTaskRunnerTest.java | 148 ++++++++
.../phoenix/execute/VectorIndexScanPlanTest.java | 228 ++++++++++-
.../expression/ExpressionTypeOrdinalTest.java | 15 +-
.../function/BsonVectorValueFunctionTest.java | 38 +-
.../DistanceFunctionSerializationTest.java | 6 +-
.../expression/function/DistanceFunctionsTest.java | 51 +--
.../function/VectorDistanceUtilTest.java | 141 ++++++-
.../apache/phoenix/index/IndexMaintainerTest.java | 313 ++++++++++++++-
.../phoenix/index/vector/CentroidManagerTest.java | 20 +-
.../phoenix/index/vector/KMeansTrainerTest.java | 63 ++--
.../index/vector/ScorecardAccumulatorTest.java | 146 +++++++
.../index/vector/VectorIndexRebuilderTest.java | 73 +++-
.../index/vector/VectorIndexScorecardTest.java | 21 +-
.../index/vector/VectorIndexTrainerTest.java | 109 ++++++
.../phoenix/iterate/RegionScannerFactoryTest.java | 80 ++++
.../VectorDistanceOrderedResultIteratorTest.java | 124 +++++-
.../mapreduce/index/fsck/IndexFsckToolTest.java | 328 +++++++++++++++-
.../phoenix/optimize/VectorPlanOrderingTest.java | 94 ++++-
.../phoenix/optimize/VectorSearchUtilTest.java | 117 +++++-
.../phoenix/parse/DistanceOperatorParseTest.java | 55 +--
.../phoenix/parse/VectorColumnParseTest.java | 14 +-
.../apache/phoenix/schema/VectorIndexTypeTest.java | 4 +-
.../types/PArrayDataTypeVectorCoercionTest.java | 4 +-
.../apache/phoenix/schema/types/PDataTypeTest.java | 5 +-
.../phoenix/schema/types/VectorDataTypeTest.java | 118 +++---
.../org/apache/phoenix/util/MetaDataUtilTest.java | 22 --
.../function/PanamaDistanceKernelParityTest.java | 13 +-
phoenix-mapreduce-byo-shaded-hbase/pom.xml | 38 ++
phoenix-server/pom.xml | 38 ++
pom.xml | 2 +-
157 files changed, 6942 insertions(+), 2066 deletions(-)
create mode 100644
phoenix-core-server/src/main/java/org/apache/phoenix/coprocessor/tasks/VectorIndexTaskRunner.java
create mode 100644
phoenix-core/src/test/java/org/apache/hadoop/hbase/ipc/controller/ServerRpcControllerFactoryTest.java
create mode 100644
phoenix-core/src/test/java/org/apache/phoenix/compile/VectorColumnCompileTest.java
create mode 100644
phoenix-core/src/test/java/org/apache/phoenix/coprocessor/tasks/VectorIndexRebuildTaskTest.java
create mode 100644
phoenix-core/src/test/java/org/apache/phoenix/coprocessor/tasks/VectorIndexTaskRunnerTest.java
create mode 100644
phoenix-core/src/test/java/org/apache/phoenix/index/vector/ScorecardAccumulatorTest.java
create mode 100644
phoenix-core/src/test/java/org/apache/phoenix/index/vector/VectorIndexTrainerTest.java
create mode 100644
phoenix-core/src/test/java/org/apache/phoenix/iterate/RegionScannerFactoryTest.java
