This is an automated email from the ASF dual-hosted git repository.
xiangfu pushed a change to branch
dependabot/maven/org.apache.maven.plugins-maven-enforcer-plugin-3.2.1
in repository https://gitbox.apache.org/repos/asf/pinot.git
discard 3cf2bb72e2 Bump maven-enforcer-plugin from 3.0.0-M3 to 3.2.1
add 7b75ed2cb7 [multistage] [refactor] Introduce cancel on error API for
MultiStageOperator and sending and receiving mailbox (#10170)
add 76a38bffab unignore tests that pass now (#10215)
add 5f3e1dcc47 [multistage] add singleton instance stage (#10211)
add af12ebff76 Fix bug is realtime segment completion for MV Raw columns
(#10186)
add 5a89491ce1 [spark-connector] Memory optimization for GRPC data fetcher
(#10209)
add 41d51ed9dc refine a few logs for easier debugging (#10223)
add c70fcaa282 [multistage][bugfix] fix non-grouping agg result issue
(#10222)
add 4c05a37800 Change scripts to use npm ci and webpack --mode development
by default (#10221)
add ce0af82131 [hotfix] fix limit pushdown only keep 10_000 rows (#10224)
add 0bbb9455eb [multistage] [bugfix] Fix sending mailbox object leak
(#10190)
add b50c58fcd3 Fix the issue with ADLSGen2 connector when writing a file
(#10230)
add d3e0177c6f Fix Schema name in Quickstart (#10233)
add 61c9b8e580 Allow segment upload to deepstore only when server segment
store uri is configured (#10216)
add 3d6163a73b Bump h3 from 4.0.0 to 4.1.1 (#10232)
add c6b7a9ef95 [multistage] consolidate operator stats (#10235)
add b1bf79ca9a Add more information when JSON flattening throws exception
(#10236)
add 5e0a8dbaa1 adding inequality join condition test and specifically
exclude unsupported syntax (#10242)
add f17c820041 Fix race condition on removing aged deleted segments
(#10238)
add 3c9a9df8ca [multistage] Initial (phase 1) query planner support for
window functions (#10228)
add b284154731 StarTree non deterministic ordering of functions in
startree_index_map and other places (#10231)
add 0f2d51cc80 [multistage] optimize limit without order by (#10241)
add 10747952f9 [feature] Pass null literal as a boolean null value thrift
(#10217)
add a22fb68ffc update an existing API to reset only segments with Error EV
(#10239)
add 63c1e504db Add SASL_SSL support for stream github event command
(#10253)
add 2c57462d3b Avoid upsert metadata access after the manager is closed
(#10251)
add d3c3e3fead [minion] check segment existency and fail early (#10261)
add dfc84a529b Refactor logging for query killing (#10262)
add 53e80fc696 Add PinotHelixResourceManager to MetadataEventNotifier
(#10247)
add 28c439ca98 close startree index properly (#10265)
add 874d9d688b [bugfix]when the query record is 0 will infinite loop
(#10246)
add cb952899d4 use normal baseballStats (#10245)
add f5579184af improve startReplaceSegments error message (#10264)
add 6caa8a038f upload large s3 object by multi parts to overcome the 5GB
limit of PutObject (#10263)
add 41be2e4395 Make PlanMaker plugable (#10269)
add a9c827379f Fix bug with time based segment relocator where it would
try to move real-time segments that are still in the consuming state (#10272)
add c0f61354aa fix the default value for enabling multipart upload (#10275)
add f4fad9b1cd Refactor padding check to support old segments with
"\\u0000" padding (#10276)
add addb6525fc move segment existency check to preprocess (#10279)
add e3a70d1186 Update Apache Datasketches to 3.3.0 (#10281)
add a151bba929 Resolve NullPointerException when COUNT() is passed literal
and nullHandling is enabled (#10268)
add b19f3fecbb [multistage] Store Sender/Receiver Stage Id in MailboxId
(#10285)
add b66f49a63e Improve the error message for segment replacement (#10284)
add 0cb28aaba2 limit helix task result info message size to 1000 (#10287)
add 02cb2d4970 Add name case for ScalarFunction annotation for casewhen
(#10266)
add 2f345da102 Handle big In clause (#10254)
add a0162559b5 Add broker metrics to V2 Engine (#10214)
add 2187a9b63a [hotfix] use legacy case-when format before releasing 0.13
(#10291)
add cd6f9f1b3b [multistage] Allow multiStageReplicaGroup Routing for
Upsert Tables (#10258)
add 4e0afef532 Handle interrupted exception during Kinesis rate limit
(#10297)
add ba8aad7ad0 Add a tight retry block to update ZK during
endSegmentReplacement (#10290)
add 78308da90d update task metadata API response when no metadata (#10301)
add 1033a11b6d [multistage] support SEMI/ANTI join (#10294)
add 0927e94b0d [Quickstart] Improve error message and remove
NullPointerException when connection fails (#10293)
add 442bfc3af1 Minor: Limit MultiStageQuickstart output (#10307)
add 83cdb398e9 Add the support of Apache Pulsar for
StreamGithubEventsCommand (#10308)
add 1aa1368a1a [multistage] Fix Issues with OpChainScheduler (#10289)
add 0722bf077b Remove duplicate stats aggregator from V2 query metrics
(#10299)
add 6aea80c2d5 [multistage] Fix return type for AVG aggregate function
(#10314)
add fc40a97c24 [multistage] properly support query hint (#10315)
add c72873a27b Update grpc to 1.53.0 and protobuf to 3.22.0 (#10312)
add a591090f2c [multistage] Close OpChains That Return Error-Block (#10319)
add 1fce07ed77 Allow file overwrite during the segment refresh (#10317)
add 9b49bca554 Add Theta Sketch Aggregation for StarTree Indexes (#10288)
add 02baaac1c1 Relax the constraint to have a dictionary and inverted
index when forward index is disabled (#10260)
add 69420af19e fix table sorting while filter is applied (#10226)
add a1d1c3b1af Always pass the QueryContext into TransformFunctionFactory
(#10320)
add d3c8ec2ec0 Upgrade github action versions (#10327)
add 1f726dc6fc Fix the missing BIG_DECIMAL handling in range predicate
evaluator (#10330)
add e8c04d454c [multistage] Fix Flaky OpChainSchedulerServiceTest (#10340)
add 3772b55dc4 enable readonly access to controller UI for users without
table restrictions (#10329)
add 69d2fae2b4 [multi-stage] Support query plan for GROUP BY only in the
intermediary stage (#10248)
add f39550ce0a [multistage] Increase Max Inbound Message Size to 64MB for
QueryServer (#10333)
add 9d5ba2c3d3 [quickstart] configure the controller to schedule tasks
when minion is enabled (#10338)
add 3ba6d2682b Add HLL and Theta Aggregation support in Merge/Rollup
(#10328)
add 0e88f54510 Add Sketch Creation Scalar Functions for HLL/Theta (#10347)
add 555a4a35ef append UUID to segment name in SegmentGenerationAndPushTask
(#10335)
add c5dd6df27c Add Statistics grouped at Stage ID level in the V2 Engine
Response (#10337)
add 2dc26ac402 [feature] Add jdbc multistage support (#10292)
add 733cc2fa6b [multistage] Use Different Thread Pool for QueryServer
(#10349)
add ea6424e3d8 Show correct query plan when prefetch is enabled (#10172)
add 8ddbd25e18 Add min / max values to virtual column metadata to allow
segment pruning (#10343)
add 135f13e323 Adding setter to DeletedSegmentsRetentionPeriod in
TableConfigBuilder (#10351)
add 246f4db352 [multistage] Dispatch Query Plan in Parallel (#10332)
add c58a01d9f0 [bugfix] Fix bug with PERCENT_SEGMENTS_AVAILABLE metric
rounding down (#10348)
add 42588248d4 Clean up the Block APIs (#10358)
add 697adabc2e Adding table name to create unique Kafka consumer client
Ids (#10324)
add 9c7e771d22 Fixes filter aggs perf (#10356)
add 398b7f5350 Fix for flaky test
ForwardIndexHandlerTest.testEnableDictionaryForMultipleColumns (#10366)
add b6bcb05522 10249: Separating the v1beta1 & v1 ingress. (#10367)
add 2e03df02f6 Add support for multiple comparison columns (i.e. one
comparison column per producer sinking to a table) (#10234)
add 6322d39e18 [Query Killing] Enhance gc related logic, refactor logging
(#10373)
add f28525b417 Add operator level stats to response when tracing is
enabled (#10364)
add e342b1fb5d [multistage] Initial (phase 1) Query runtime for window
functions - empty OVER() and OVER(PARTITION BY) (#10286)
add 3a307f1521 Fix AVG Return Type (#10368)
add 0c0c6550d8 Add controller API to fetch all segment zookeeper metadata
for a table (#10363)
add 5d0089a6fa Fix WindowFunctionPlans test causing failures due to
changes in test columns (#10385)
add 2e55d1c04d Patch: Allow fetching comparable index from
ComparisonColumns (#10388)
add 4eeaf82185 Refactor Spark Connector into two modules for reusability
(#10321)
add 585f62a3c7 allow to customize the httpclient used to create S3PinotFS
(#10369)
add 52761fcb72 add the ability to change FilterOperatorUtils code (#10371)
add 3ffa7c3e32 [multistage][stats] clean up some stats (#10390)
add 7c3c8e8705 Allow replica group assignment support in tier configs
(#10255)
add 221db828cf Let applyAnd to be applied using different window sizes
(#10372)
add 6bd8a7dcff Enhance select order-by combine to use merge sort (#10357)
add dadd42cff1 Make startReplaceSegments() more robust for NPE (#10403)
add 5b289fc2e5 [multistage] Fix Leaks in Mailbox (#10322)
add b62033a3ac [multistage] early terminate from query dispatcher (#10401)
add 2607c0ee11 refine method name withForwardIndexDisabled (#10407)
add be9dfe74ad Bug Fix: Handle empty validDocId snapshots during reloading
(#10406)
add 6db6d65adf 10271: Updated the code for handling the slash at the end
for controller.data.dir (#10395)
add d475712d31 allow segment details page to load without indices (#10398)
add 41db04509f [Multistage] Allow queries on multiple tables of same
tenant to be executed from controller UI (#10336)
add 15cb707e86 Encapsulate changes in IndexLoadingConfig and
SegmentGeneratorConfig (#10352)
add f8d432013b [Index SPI] IndexType (#10191)
add 81ceb1d020 Simplify filtered aggregate transform operator creation
(#10410)
add 3bad67db07 feat(ui): persist oidc auth details (#10389)
add 01ff18d47c Introduce BaseProjectOperator and ValueBlock (#10405)
add 0b111a1310 Fix JDBC driver check for username (#10416)
add 856503256f Adding Pinot base build/runtime image support for Amazon
Corretto and MS OpenJDK (#10422)
add 11b6bcda50 [multistage] Fix Predicate Pushdown by Using Rule
Collection (#10409)
add 0362ad369d Support multi-arch pinot docker image (#10429)
add ce299cfe80 try fixing mailbox cancel race condition (#10432)
add 16cae5d9c6 Add support to create realtime segment in local (#10433)
add 00284b9ede Refactor: Pass context instead on individual arguments to
operator (#10413)
add 40a7911c53 Revert "Add support to create realtime segment in local
(#10433)" (#10436)
add dcddb9ff02 Adding dropResults parameter in queryOptions to drop the
resultTable field from the response (#10419)
add e01df9a7ce Data visitors (#10361)
add bc9aa75585 Add a PinotLogicalSortExchange to replace usage of
LogicalSortExchange as it will be sender and receiver aware (#10408)
add dca5d383c6 [Clean up] Remove getColumnName() from AggregationFunction
interface (#10431)
add bf5528c885 [multistage] Catch Throwable To Propagate Proper Error
Message (#10438)
add 5673c8e6b2 RebalanceStatus API changes to track the status/progress of
rebalance. (#10359)
add d9c4315ca1 Pinot Spark Connector for Spark3 (#10394)
add cb1069f11d Add Spark connector option for passing Pinot query options
(#10443)
add 3687b2b732 Trying to setup node before Compatibility Regression
Testing (#10440)
add 854e7f8d2a Index spi: index service (#10192)
add ee71befad3 Add "processAll" mode for MergeRollupTask (#10387)
add c0d0896ac3 Upgrade h2 version from 1.x to 2.x (#10456)
add 8452c25a09 fix jersey TerminalWriterInterceptor MessageBodyWriter not
found issue (#10462)
add f3c2c0d98f Bug fix: Start counting operator execution time from first
NoOp block (#10450)
add abe47bfc52 Fix unavailable instances issues for StrictReplicaGroup
(#10466)
add 11272b7eb2 Added optional force param to the table configs update API
(#10441)
add 316842f66e [multistage] Support array type for select query (#10434)
add 8bfb0b82b2 Change shell to bash (#10469)
add 40b2a65287 Fix the double destroy of segment data manager during
server shutdown (#10475)
add 3f36623c56 Enhance broker reduce to handle different column names from
server response (#10454)
add d1227e466d [feature] [null support # 1] selection only literal in
broker null support (#10376)
add ea0ecde9ed Remove "isSorted()" precondition check in the
ForwardIndexHandler (#10476)
add a6332de929 Bump maven-enforcer-plugin from 3.0.0-M3 to 3.2.1
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (3cf2bb72e2)
\
N -- N -- N
refs/heads/dependabot/maven/org.apache.maven.plugins-maven-enforcer-plugin-3.2.1
(a6332de929)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../build-multi-arch-pinot-docker-image.yml | 108 +
.../workflows/build-pinot-base-docker-image.yml | 50 +
.github/workflows/build-pinot-docker-image.yml | 6 +-
.github/workflows/build-presto-docker-image.yml | 6 +-
.github/workflows/build-superset-docker-image.yml | 6 +-
.github/workflows/pinot_compatibility_tests.yml | 20 +-
.github/workflows/pinot_tests.yml | 70 +-
.github/workflows/pinot_vuln_check.yml | 4 +-
.../scripts/.pinot_compatibility_verifier.sh | 15 +
.../.pinot_base_docker_image_build_and_push.sh | 37 +
.../scripts/docker/.pinot_build_info_gen.sh | 41 +
...not_multi_arch_docker_image_manifest_package.sh | 62 +
.../.pinot_single_platform_docker_image_build.sh | 66 +
compatibility-verifier/checkoutAndBuild.sh | 31 +-
docker/images/pinot-base/README.md | 37 +-
.../{Dockerfile => amazoncorretto.dockerfile} | 32 +-
.../{Dockerfile => ms-openjdk.dockerfile} | 4 +-
.../{Dockerfile => openjdk.dockerfile} | 4 +-
.../{Dockerfile => amazoncorretto.dockerfile} | 11 +-
.../{Dockerfile => ms-openjdk.dockerfile} | 7 +-
.../{Dockerfile => openjdk.dockerfile} | 4 +-
docker/images/pinot/Dockerfile | 4 +-
.../broker/{ingress.yaml => ingress-v1.yaml} | 31 -
.../pinot/templates/broker/ingress-v1beta1.yaml | 30 +
.../controller/{ingress.yaml => ingress-v1.yaml} | 31 -
.../templates/controller/ingress-v1beta1.yaml | 30 +
pinot-broker/pom.xml | 5 +
.../requesthandler/BaseBrokerRequestHandler.java | 17 +-
.../BrokerRequestHandlerDelegate.java | 18 +-
.../MultiStageBrokerRequestHandler.java | 43 +-
.../instanceselector/BalancedInstanceSelector.java | 60 +-
.../instanceselector/BaseInstanceSelector.java | 461 ++--
.../routing/instanceselector/InstanceSelector.java | 6 +
.../instanceselector/InstanceSelectorFactory.java | 33 +-
.../MultiStageReplicaGroupSelector.java | 42 +-
.../routing/instanceselector/NewSegmentState.java | 51 +
.../ReplicaGroupInstanceSelector.java | 126 +-
.../instanceselector/SegmentInstanceCandidate.java | 28 +-
.../routing/instanceselector/SegmentStates.java | 57 +
.../StrictReplicaGroupInstanceSelector.java | 165 +-
.../instanceselector/InstanceSelectorTest.java | 803 ++++++-
.../org/apache/pinot/client/PinotConnection.java | 49 +-
.../pinot/client/PinotPreparedStatement.java | 6 +-
.../org/apache/pinot/client/PinotStatement.java | 2 +-
.../org/apache/pinot/client/utils/DriverUtils.java | 40 +-
.../pinot/client/PinotPreparedStatementTest.java | 31 +
.../apache/pinot/client/PinotStatementTest.java | 114 +-
.../assignment/InstanceAssignmentConfigUtils.java | 20 +-
.../common/assignment/InstancePartitionsUtils.java | 16 +
.../pinot/common/datablock/DataBlockUtils.java | 5 +
.../pinot/common/datablock/MetadataBlock.java | 28 +-
.../apache/pinot/common/datatable/DataTable.java | 17 +-
.../pinot/common/exception/QueryException.java | 2 +
.../pinot/common/function/FunctionInvoker.java | 12 +
.../common/function/scalar/ObjectFunctions.java | 32 +-
.../pinot/common/lineage/SegmentLineage.java | 4 +
.../pinot/common/lineage/SegmentLineageUtils.java | 8 +-
.../pinot/common/metadata/ZKMetadataProvider.java | 31 +-
.../metadata/controllerjob/ControllerJobType.java | 4 +-
.../apache/pinot/common/request/DataSource.java | 76 +-
.../apache/pinot/common/request/Expression.java | 145 +-
.../pinot/common/request/ExpressionType.java | 7 +-
.../org/apache/pinot/common/request/Function.java | 75 +-
.../apache/pinot/common/request/Identifier.java | 47 +-
.../org/apache/pinot/common/request/Literal.java | 84 +-
.../apache/pinot/common/request/PinotQuery.java | 399 ++--
.../request/context/RequestContextUtils.java | 4 +
.../response/broker/BrokerResponseNative.java | 5 +-
.../response/broker/BrokerResponseNativeV2.java | 94 +
.../response/broker/BrokerResponseStats.java | 134 ++
.../common/tier/TimeBasedTierSegmentSelector.java | 6 +
.../org/apache/pinot/common/utils/DataSchema.java | 69 +-
.../common/utils/FileUploadDownloadClient.java | 57 +-
.../org/apache/pinot/common/utils/URIUtils.java | 13 +
.../common/utils/config/QueryOptionsUtils.java | 5 +
.../common/utils/config/TableConfigUtils.java | 13 +-
.../pinot/common/utils/request/RequestUtils.java | 20 +-
.../apache/pinot/sql/parsers/CalciteSqlParser.java | 149 +-
.../rewriter/CompileTimeFunctionsInvoker.java | 8 +-
pinot-common/src/main/proto/worker.proto | 10 +
.../pinot/common/datablock/MetadataBlockTest.java | 3 +-
.../function/FunctionDefinitionRegistryTest.java | 4 +-
.../pinot/common/lineage/SegmentLineageTest.java | 24 +
.../pinot/common/tier/TierSegmentSelectorTest.java | 24 +
.../apache/pinot/common/utils/DataSchemaTest.java | 31 +-
.../FileUploadDownloadClientWithoutServerTest.java | 29 +-
.../apache/pinot/common/utils/URIUtilsTest.java | 13 +
.../common/utils/config/TableConfigSerDeTest.java | 10 +-
.../pinot/sql/parsers/CalciteSqlCompilerTest.java | 64 +-
.../multipleIndexPartialUpsertConfig.json | 41 +
...artialUpsertMultipleComparisonColumnConfig.json | 41 +
pinot-common/src/thrift/query.thrift | 2 +
.../README.md | 22 +-
...pinot-connector-executor-server-interaction.jpg | Bin
.../documentation/read_model.md | 9 +-
.../pom.xml | 139 +-
...org.apache.spark.sql.sources.DataSourceRegister | 0
.../spark/datasource/PinotDataSourceReader.scala | 19 +-
.../spark/datasource/PinotDataSourceV2.scala | 0
.../spark/datasource/PinotInputPartition.scala | 16 +-
.../spark/datasource/TypeConverter.scala} | 20 +-
.../spark/datasource/query}/FilterPushDown.scala | 2 +-
.../src/test/resources/schema/pinot-schema.json | 0
.../src/test/resources/schema/spark-schema.json | 0
.../connector/spark/datasource}/BaseTest.scala | 2 +-
.../ExampleSparkPinotConnectorTest.scala | 4 +-
.../spark/datasource/TypeConverterTest.scala} | 45 +-
.../datasource/query}/FilterPushDownTest.scala | 4 +-
.../README.md | 23 +-
...pinot-connector-executor-server-interaction.jpg | Bin
.../documentation/read_model.md | 9 +-
.../pom.xml | 114 +-
...org.apache.spark.sql.sources.DataSourceRegister | 1 +
.../spark/v3/datasource/PinotDataSource.scala | 55 +
.../spark/v3}/datasource/PinotInputPartition.scala | 28 +-
.../connector/spark/v3/datasource/PinotScan.scala | 97 +
.../spark/v3/datasource/PinotScanBuilder.scala | 82 +
.../connector/spark/v3/datasource/PinotTable.scala | 50 +
.../spark/v3/datasource/TypeConverter.scala} | 29 +-
.../v3/datasource/query}/FilterPushDown.scala | 2 +-
.../src/test/resources/log4j2.xml | 0
.../src/test/resources/schema/pinot-schema.json | 2 +-
.../src/test/resources/schema/spark-schema.json | 2 +-
.../connector/spark/v3/datasource}/BaseTest.scala | 2 +-
.../ExampleSparkPinotConnectorTest.scala | 4 +-
.../spark/v3/datasource/PinotScanBuilderTest.scala | 51 +
.../spark/v3/datasource/PinotScanTest.scala | 51 +
.../spark/v3/datasource/TypeConverterTest.scala} | 47 +-
.../v3/datasource/query}/FilterPushDownTest.scala | 4 +-
.../pom.xml | 43 +-
.../spark/common/CaseInsensitiveStringMap.java | 201 ++
.../pinot/connector/spark/common}/HttpUtils.scala | 3 +-
.../pinot/connector/spark/common}/Logging.scala | 2 +-
.../spark/common}/PinotClusterClient.scala | 26 +-
.../spark/common}/PinotDataSourceReadOptions.scala | 66 +-
.../pinot/connector/spark/common}/exceptions.scala | 2 +-
.../pinot/connector/spark/common}/package.scala | 4 +-
.../spark/common/partition}/PinotSplitter.scala | 20 +-
.../connector/spark/common/query/ScanQuery.scala} | 4 +-
.../spark/common/query/ScanQueryGenerator.scala} | 29 +-
.../reader/PinotAbstractPartitionReader.scala | 72 +
.../reader}/PinotGrpcServerDataFetcher.scala | 30 +-
.../common/reader}/PinotServerDataFetcher.scala | 17 +-
.../src/test/resources/log4j2.xml | 0
.../pinot/connector/spark/common}/BaseTest.scala | 2 +-
.../common}/PinotDataSourceReadOptionsTest.scala | 23 +-
.../spark/common}/PinotSplitterTest.scala | 30 +-
.../common/query/ScanQueryGeneratorTest.scala} | 35 +-
.../datasource/PinotInputPartitionReader.scala | 65 -
pinot-connectors/pom.xml | 4 +-
.../presto-pinot-driver/pom.xml | 2 +-
pinot-controller/pom.xml | 6 +-
.../pinot/controller/BaseControllerStarter.java | 5 +-
.../apache/pinot/controller/ControllerConf.java | 3 +-
.../DefaultMetadataEventNotifierFactory.java | 9 +-
.../api/events/MetadataEventNotifierFactory.java | 9 +-
.../api/resources/PinotControllerAuthResource.java | 3 +-
.../PinotInstanceAssignmentRestletResource.java | 188 +-
.../api/resources/PinotQueryResource.java | 66 +-
.../api/resources/PinotRealtimeTableResource.java | 8 +-
.../api/resources/PinotSchemaRestletResource.java | 2 +-
.../api/resources/PinotSegmentRestletResource.java | 47 +-
.../api/resources/PinotTableRestletResource.java | 111 +-
.../api/resources/PinotTaskRestletResource.java | 38 +-
.../api/resources/PinotUpsertRestletResource.java | 27 +-
.../ServerRebalanceJobStatusResponse.java | 36 +-
.../api/resources/TableConfigsRestletResource.java | 11 +-
.../controller/helix/SegmentStatusChecker.java | 2 +-
.../helix/core/PinotHelixResourceManager.java | 476 +++--
.../helix/core/SegmentDeletionManager.java | 40 +-
.../instance/InstanceAssignmentDriver.java | 32 +-
.../core/minion/PinotHelixTaskResourceManager.java | 4 +-
.../realtime/PinotLLCRealtimeSegmentManager.java | 3 +-
.../core/rebalance/NoOpTableRebalanceObserver.java | 23 +-
.../helix/core/rebalance/RebalanceResult.java | 18 +-
.../core/rebalance/TableRebalanceObserver.java | 46 +
.../rebalance/TableRebalanceProgressStats.java | 130 ++
.../helix/core/rebalance/TableRebalancer.java | 225 +-
.../rebalance/ZkBasedTableRebalanceObserver.java | 197 ++
.../helix/core/relocation/SegmentRelocator.java | 5 +-
.../helix/core/retention/RetentionManager.java | 24 +-
pinot-controller/src/main/resources/app/App.tsx | 240 +--
.../app/components/AppLoadingIndicator.tsx | 32 +-
.../src/main/resources/app/components/Table.tsx | 6 +-
.../resources/app/components/auth/AuthProvider.tsx | 269 +++
pinot-controller/src/main/resources/app/index.tsx | 42 +
.../src/main/resources/app/interfaces/types.d.ts | 5 +
.../main/resources/app/utils/PinotMethodUtils.ts | 14 +-
.../src/main/resources/app/utils/Utils.tsx | 8 +-
.../src/main/resources/app/utils/axios-config.ts | 89 +-
pinot-controller/src/main/resources/package.json | 1 +
.../src/main/resources/webpack.config.js | 2 +-
.../pinot/controller/ControllerConfTest.java | 23 +
...anceAssignmentRestletResourceStatelessTest.java | 141 +-
.../api/TableConfigsRestletResourceTest.java | 40 +
.../controller/helix/SegmentStatusCheckerTest.java | 63 +
.../pinot/controller/helix/TableCacheTest.java | 2 +-
.../PinotHelixResourceManagerStatelessTest.java | 6 +-
.../instance/InstanceAssignmentTest.java | 64 +-
.../TableRebalancerClusterStatelessTest.java | 127 +-
.../TestZkBasedTableRebalanceObserver.java | 106 +
.../helix/core/retention/RetentionManagerTest.java | 20 +-
.../core/util/SegmentDeletionManagerTest.java | 11 +-
.../CPUMemThreadLevelAccountingObjects.java | 9 +
.../PerQueryCPUMemAccountantFactory.java | 70 +-
.../java/org/apache/pinot/core/common/Block.java | 37 +-
.../apache/pinot/core/common/BlockDocIdSet.java | 53 +-
.../apache/pinot/core/common/MinionConstants.java | 3 +
.../org/apache/pinot/core/common/Operator.java | 20 +
.../core/common/datablock/DataBlockBuilder.java | 8 +
.../core/data/manager/BaseTableDataManager.java | 8 +
.../core/data/manager/InstanceDataManager.java | 1 +
.../offline/ImmutableSegmentDataManager.java | 2 +-
.../manager/offline/TableDataManagerProvider.java | 13 +
.../realtime/HLRealtimeSegmentDataManager.java | 6 +-
.../realtime/LLRealtimeSegmentDataManager.java | 66 +-
.../manager/realtime/PinotFSSegmentUploader.java | 2 +
.../manager/realtime/RealtimeTableDataManager.java | 12 +-
.../manager/realtime/SegmentCommitterFactory.java | 6 +-
.../core/function/scalar/SketchFunctions.java | 134 ++
.../function/BaseBinaryGeoTransformFunction.java | 54 +-
.../function/ConstructFromTextFunction.java | 12 +-
.../function/ConstructFromWKBFunction.java | 12 +-
.../transform/function/GeoToH3Function.java | 22 +-
.../transform/function/StAreaFunction.java | 12 +-
.../transform/function/StAsBinaryFunction.java | 12 +-
.../transform/function/StAsTextFunction.java | 12 +-
.../transform/function/StContainsFunction.java | 6 +-
.../transform/function/StDistanceFunction.java | 6 +-
.../transform/function/StEqualsFunction.java | 6 +-
.../transform/function/StGeometryTypeFunction.java | 12 +-
.../transform/function/StPointFunction.java | 14 +-
.../transform/function/StPolygonFunction.java | 8 +-
.../transform/function/StWithinFunction.java | 6 +-
.../AcquireReleaseColumnsSegmentOperator.java | 13 +-
...ransformBlock.java => BaseProjectOperator.java} | 31 +-
.../core/operator/BitmapDocIdSetOperator.java | 5 +
.../apache/pinot/core/operator/ColumnContext.java | 73 +
.../pinot/core/operator/DocIdSetOperator.java | 15 +-
.../core/operator/InstanceResponseOperator.java | 4 +-
.../pinot/core/operator/ProjectionOperator.java | 31 +-
.../pinot/core/operator/blocks/DocIdSetBlock.java | 49 +-
.../pinot/core/operator/blocks/FilterBlock.java | 41 +-
.../operator/blocks/InstanceResponseBlock.java | 24 -
.../core/operator/blocks/ProjectionBlock.java | 32 +-
.../pinot/core/operator/blocks/TransformBlock.java | 52 +-
...sThroughTransformBlock.java => ValueBlock.java} | 35 +-
.../blocks/results/AggregationResultsBlock.java | 20 +-
.../operator/blocks/results/BaseResultsBlock.java | 27 +-
.../blocks/results/DistinctResultsBlock.java | 2 +-
.../blocks/results/ExceptionResultsBlock.java | 4 +-
.../blocks/results/ExplainResultsBlock.java | 3 +-
.../blocks/results/GroupByResultsBlock.java | 2 +-
.../blocks/results/MetadataResultsBlock.java | 4 +-
.../operator/blocks/results/ResultsBlockUtils.java | 14 +-
.../blocks/results/SelectionResultsBlock.java | 47 +-
.../combine/BaseSingleBlockCombineOperator.java | 2 +-
...xValueBasedSelectionOrderByCombineOperator.java | 21 +-
.../combine/merger/ResultsBlockMerger.java | 10 -
.../merger/SelectionOnlyResultsBlockMerger.java | 7 +-
.../merger/SelectionOrderByResultsBlockMerger.java | 15 +-
.../dociditerators/ArrayBasedDocIdIterator.java | 53 -
.../ExpressionScanDocIdIterator.java | 16 +
.../dociditerators/MVScanDocIdIterator.java | 20 +-
.../dociditerators/SVScanDocIdIterator.java | 50 +-
.../dociditerators/ScanBasedDocIdIterator.java | 11 +-
.../pinot/core/operator/docidsets/AndDocIdSet.java | 15 +-
.../core/operator/docidsets/BitmapDocIdSet.java | 3 +-
.../core/operator/docidsets/EmptyDocIdSet.java | 5 +-
...FilterDocIdSet.java => ExpressionDocIdSet.java} | 5 +-
.../operator/docidsets/FilterBlockDocIdSet.java | 75 -
.../core/operator/docidsets/MVScanDocIdSet.java | 3 +-
.../core/operator/docidsets/MatchAllDocIdSet.java | 3 +-
.../pinot/core/operator/docidsets/NotDocIdSet.java | 7 +-
.../pinot/core/operator/docidsets/OrDocIdSet.java | 13 +-
.../docidsets/RangelessBitmapDocIdSet.java | 3 +-
.../core/operator/docidsets/SVScanDocIdSet.java | 7 +-
.../core/operator/docidsets/SortedDocIdSet.java | 3 +-
.../operator/docvalsets/TransformBlockValSet.java | 44 +-
.../core/operator/filter/AndFilterOperator.java | 9 +-
.../operator/filter/CombinedFilterOperator.java | 11 +-
.../core/operator/filter/EmptyFilterOperator.java | 6 +
.../operator/filter/ExpressionFilterOperator.java | 32 +-
.../core/operator/filter/FilterOperatorUtils.java | 369 ++--
.../filter/H3InclusionIndexFilterOperator.java | 10 +-
.../operator/filter/H3IndexFilterOperator.java | 12 +-
.../operator/filter/MatchAllFilterOperator.java | 6 +
.../core/operator/filter/OrFilterOperator.java | 11 +-
.../filter/RangeIndexBasedFilterOperator.java | 4 +-
.../operator/filter/ScanBasedFilterOperator.java | 13 +-
.../predicate/EqualsPredicateEvaluatorFactory.java | 78 +-
.../predicate/InPredicateEvaluatorFactory.java | 85 +-
.../NotEqualsPredicateEvaluatorFactory.java | 72 +-
.../predicate/NotInPredicateEvaluatorFactory.java | 84 +-
.../predicate/PredicateEvaluatorProvider.java | 21 +-
.../operator/filter/predicate/PredicateUtils.java | 28 +-
.../predicate/RangePredicateEvaluatorFactory.java | 4 +-
.../core/operator/query/AggregationOperator.java | 27 +-
.../core/operator/query/DistinctOperator.java | 29 +-
.../operator/query/EmptySelectionOperator.java | 21 +-
.../query/FilteredAggregationOperator.java | 32 +-
.../operator/query/FilteredGroupByOperator.java | 70 +-
.../pinot/core/operator/query/GroupByOperator.java | 30 +-
.../query/LinearSelectionOrderByOperator.java | 148 +-
.../core/operator/query/SelectionOnlyOperator.java | 34 +-
.../operator/query/SelectionOrderByOperator.java | 91 +-
.../SelectionPartiallyOrderedByAscOperator.java | 16 +-
.../SelectionPartiallyOrderedByDescOperation.java | 16 +-
.../streaming/StreamingSelectionOnlyOperator.java | 33 +-
.../transform/PassThroughTransformOperator.java | 61 -
.../core/operator/transform/TransformOperator.java | 113 +-
.../function/AdditionTransformFunction.java | 22 +-
.../function/ArrayAverageTransformFunction.java | 18 +-
.../function/ArrayLengthTransformFunction.java | 20 +-
.../function/ArrayMaxTransformFunction.java | 46 +-
.../function/ArrayMinTransformFunction.java | 46 +-
.../function/ArraySumTransformFunction.java | 12 +-
.../transform/function/BaseTransformFunction.java | 196 +-
.../function/BinaryOperatorTransformFunction.java | 240 +--
.../transform/function/CaseTransformFunction.java | 84 +-
.../transform/function/CastTransformFunction.java | 170 +-
.../function/CoalesceTransformFunction.java | 101 +-
.../DateTimeConversionTransformFunction.java | 26 +-
.../function/DateTimeTransformFunction.java | 12 +-
.../function/DateTruncTransformFunction.java | 12 +-
.../function/DistinctFromTransformFunction.java | 20 +-
.../function/DivisionTransformFunction.java | 26 +-
.../function/ExtractTransformFunction.java | 12 +-
.../function/GreatestTransformFunction.java | 50 +-
.../function/GroovyTransformFunction.java | 76 +-
.../function/IdentifierTransformFunction.java | 78 +-
.../function/InIdSetTransformFunction.java | 22 +-
.../transform/function/InTransformFunction.java | 56 +-
.../function/IsNotNullTransformFunction.java | 27 +-
.../function/IsNullTransformFunction.java | 26 +-
.../function/JsonExtractKeyTransformFunction.java | 12 +-
.../JsonExtractScalarTransformFunction.java | 183 +-
.../transform/function/LeastTransformFunction.java | 50 +-
.../function/LiteralTransformFunction.java | 52 +-
.../function/LogicalOperatorTransformFunction.java | 14 +-
.../function/LookupTransformFunction.java | 119 +-
.../function/MapValueTransformFunction.java | 14 +-
.../function/ModuloTransformFunction.java | 14 +-
.../function/MultiplicationTransformFunction.java | 22 +-
.../transform/function/NotInTransformFunction.java | 12 +-
.../function/NotOperatorTransformFunction.java | 12 +-
.../transform/function/PowerTransformFunction.java | 14 +-
.../function/RegexpExtractTransformFunction.java | 12 +-
.../function/RoundDecimalTransformFunction.java | 14 +-
.../function/ScalarTransformFunctionWrapper.java | 132 +-
.../SelectTupleElementTransformFunction.java | 4 +-
.../function/SingleParamMathTransformFunction.java | 26 +-
.../function/SubtractionTransformFunction.java | 26 +-
.../function/TimeConversionTransformFunction.java | 12 +-
.../transform/function/TransformFunction.java | 119 +-
.../function/TransformFunctionFactory.java | 104 +-
.../function/TrigonometricTransformFunctions.java | 14 +-
.../function/TruncateDecimalTransformFunction.java | 14 +-
.../function/ValueInTransformFunction.java | 54 +-
.../pinot/core/plan/AggregationPlanNode.java | 32 +-
.../apache/pinot/core/plan/DistinctPlanNode.java | 8 +-
.../org/apache/pinot/core/plan/FilterPlanNode.java | 11 +-
.../apache/pinot/core/plan/GroupByPlanNode.java | 50 +-
...TransformPlanNode.java => ProjectPlanNode.java} | 41 +-
.../apache/pinot/core/plan/ProjectionPlanNode.java | 65 -
.../apache/pinot/core/plan/SelectionPlanNode.java | 37 +-
.../core/plan/StreamingSelectionPlanNode.java | 6 +-
.../core/plan/maker/InstancePlanMakerImplV2.java | 60 +-
.../apache/pinot/core/plan/maker/PlanMaker.java | 6 +
.../query/aggregation/AggregationExecutor.java | 8 +-
.../aggregation/DefaultAggregationExecutor.java | 9 +-
.../aggregation/function/AggregationFunction.java | 5 -
.../function/AggregationFunctionUtils.java | 173 +-
.../BaseSingleInputAggregationFunction.java | 5 -
.../function/CountAggregationFunction.java | 12 +-
.../function/CountMVAggregationFunction.java | 5 -
.../function/CovarianceAggregationFunction.java | 5 -
.../function/DistinctAggregationFunction.java | 14 +-
...istinctCountThetaSketchAggregationFunction.java | 26 +-
...irstDoubleValueWithTimeAggregationFunction.java | 24 +-
...FirstFloatValueWithTimeAggregationFunction.java | 25 +-
.../FirstIntValueWithTimeAggregationFunction.java | 28 +-
.../FirstLongValueWithTimeAggregationFunction.java | 24 +-
...irstStringValueWithTimeAggregationFunction.java | 20 +-
...LastDoubleValueWithTimeAggregationFunction.java | 24 +-
.../LastFloatValueWithTimeAggregationFunction.java | 25 +-
.../LastIntValueWithTimeAggregationFunction.java | 28 +-
.../LastLongValueWithTimeAggregationFunction.java | 24 +-
...LastStringValueWithTimeAggregationFunction.java | 20 +-
.../function/PercentileAggregationFunction.java | 6 -
.../function/PercentileEstAggregationFunction.java | 6 -
.../PercentileEstMVAggregationFunction.java | 6 -
.../function/PercentileMVAggregationFunction.java | 6 -
.../PercentileRawEstAggregationFunction.java | 9 -
.../PercentileRawTDigestAggregationFunction.java | 9 -
.../PercentileSmartTDigestAggregationFunction.java | 5 -
.../PercentileTDigestAggregationFunction.java | 6 -
.../PercentileTDigestMVAggregationFunction.java | 6 -
.../groupby/DefaultGroupByExecutor.java | 50 +-
.../groupby/DictionaryBasedGroupKeyGenerator.java | 29 +-
.../query/aggregation/groupby/GroupByExecutor.java | 8 +-
.../aggregation/groupby/GroupKeyGenerator.java | 18 +-
.../NoDictionaryMultiColumnGroupKeyGenerator.java | 31 +-
.../NoDictionarySingleColumnGroupKeyGenerator.java | 38 +-
.../core/query/config/QueryExecutorConfig.java | 41 +-
.../core/query/config/QueryPlannerConfig.java | 53 -
.../core/query/config/SegmentPrunerConfig.java | 5 +-
.../core/query/distinct/DistinctExecutor.java | 6 +-
.../query/distinct/DistinctExecutorFactory.java | 79 +-
...ionaryBasedMultiColumnDistinctOnlyExecutor.java | 10 +-
...aryBasedMultiColumnDistinctOrderByExecutor.java | 10 +-
...onaryBasedSingleColumnDistinctOnlyExecutor.java | 8 +-
...ryBasedSingleColumnDistinctOrderByExecutor.java | 8 +-
...BigDecimalSingleColumnDistinctOnlyExecutor.java | 8 +-
...DecimalSingleColumnDistinctOrderByExecutor.java | 8 +-
.../RawBytesSingleColumnDistinctOnlyExecutor.java | 8 +-
...awBytesSingleColumnDistinctOrderByExecutor.java | 8 +-
.../RawDoubleSingleColumnDistinctOnlyExecutor.java | 8 +-
...wDoubleSingleColumnDistinctOrderByExecutor.java | 8 +-
.../RawFloatSingleColumnDistinctOnlyExecutor.java | 8 +-
...awFloatSingleColumnDistinctOrderByExecutor.java | 8 +-
.../RawIntSingleColumnDistinctOnlyExecutor.java | 8 +-
.../RawIntSingleColumnDistinctOrderByExecutor.java | 8 +-
.../RawLongSingleColumnDistinctOnlyExecutor.java | 8 +-
...RawLongSingleColumnDistinctOrderByExecutor.java | 8 +-
.../raw/RawMultiColumnDistinctExecutor.java | 10 +-
.../RawStringSingleColumnDistinctOnlyExecutor.java | 8 +-
...wStringSingleColumnDistinctOrderByExecutor.java | 8 +-
.../pinot/core/query/executor/QueryExecutor.java | 1 +
.../query/executor/ServerQueryExecutorV1Impl.java | 128 +-
.../query/reduce/AggregationDataTableReducer.java | 38 +-
.../pinot/core/query/reduce/BaseReduceService.java | 249 ---
.../query/reduce/ExecutionStatsAggregator.java | 387 ++++
.../query/reduce/SelectionDataTableReducer.java | 92 +-
.../query/selection/SelectionOperatorService.java | 10 +-
.../query/selection/SelectionOperatorUtils.java | 207 +-
.../core/query/utils/OrderByComparatorFactory.java | 118 +-
...actory.java => DistinctCountHLLAggregator.java} | 35 +-
...ava => DistinctCountThetaSketchAggregator.java} | 38 +-
.../aggregator/ValueAggregatorFactory.java | 6 +
.../executor/StarTreeAggregationExecutor.java | 8 +-
.../startree/executor/StarTreeGroupByExecutor.java | 12 +-
...mPlanNode.java => StarTreeProjectPlanNode.java} | 58 +-
.../startree/plan/StarTreeProjectionPlanNode.java | 52 -
.../apache/pinot/core/util/trace/TraceContext.java | 7 +-
.../core/common/datablock/DataBlockTestUtils.java | 5 +
.../core/common/datatable/DataTableSerDeTest.java | 7 +
.../realtime/LLRealtimeSegmentDataManagerTest.java | 1 +
.../realtime/SegmentCommitterFactoryTest.java | 6 +-
.../core/function/scalar/SketchFunctionsTest.java | 62 +
.../blocks/results/ResultsBlockUtilsTest.java | 19 +-
.../combine/SelectionCombineOperatorTest.java | 30 +-
.../core/operator/filter/TestFilterOperator.java | 4 +-
.../predicate/InPredicateEvaluatorFactoryTest.java | 115 +
.../query/LinearSelectionOrderByOperatorTest.java | 26 +-
.../apache/pinot/core/plan/FilterPlanNodeTest.java | 4 +-
...adataAndDictionaryAggregationPlanMakerTest.java | 3 +-
.../function/AggregationFunctionFactoryTest.java | 88 +-
.../DictionaryBasedGroupKeyGeneratorTest.java | 54 +-
.../groupby/NoDictionaryGroupKeyGeneratorTest.java | 24 +-
.../query/reduce/StreamingReduceServiceTest.java | 4 +-
.../selection/SelectionOperatorServiceTest.java | 121 +-
.../pinot/core/startree/v2/BaseStarTreeV2Test.java | 9 +-
.../DistinctCountThetaSketchStarTreeV2Test.java} | 36 +-
.../pinot/queries/ExplainPlanQueriesTest.java | 1637 +++++++-------
.../pinot/queries/FilteredAggregationsTest.java | 27 +
...dexDisabledMultiValueQueriesWithReloadTest.java | 4 +-
...ForwardIndexDisabledSingleValueQueriesTest.java | 4 +-
.../ForwardIndexHandlerReloadQueriesTest.java | 14 +-
...erSegmentAggregationSingleValueQueriesTest.java | 12 +-
...InnerSegmentSelectionMultiValueQueriesTest.java | 11 +-
...erSegmentSelectionMultiValueRawQueriesTest.java | 9 +-
...nnerSegmentSelectionSingleValueQueriesTest.java | 63 +-
...terSegmentAggregationMultiValueQueriesTest.java | 2 +-
...SegmentAggregationMultiValueRawQueriesTest.java | 2 +-
.../NoDictionaryCompressionQueriesTest.java | 2 +-
.../org/apache/pinot/queries/RangeQueriesTest.java | 8 +-
.../tests/ClusterIntegrationTestUtils.java | 31 +-
.../pinot/integration/tests/ClusterTest.java | 25 +-
.../pinot/integration/tests/QueryGenerator.java | 46 +-
.../tests/BaseClusterIntegrationTestSet.java | 7 +-
.../tests/BaseRealtimeClusterIntegrationTest.java | 10 +-
.../tests/HybridClusterIntegrationTest.java | 15 +
.../MergeRollupMinionClusterIntegrationTest.java | 157 +-
...ltiStageEngineCustomTenantIntegrationTest.java} | 34 +-
.../tests/MultiStageEngineIntegrationTest.java | 15 +-
.../tests/NullHandlingIntegrationTest.java | 116 +
.../tests/OfflineClusterIntegrationTest.java | 9 +-
.../tests/ThetaSketchIntegrationTest.java | 14 +
...rformance_2014_100k_subset.test_queries_10K.sql | 1972 ++++++++---------
...rformance_2014_100k_subset.test_queries_200.sql | 70 +-
.../minion/taskfactory/TaskFactoryRegistry.java | 26 +-
.../pinot/plugin/filesystem/ADLSGen2PinotFS.java | 20 +-
pinot-plugins/pinot-file-system/pinot-s3/pom.xml | 5 +
.../apache/pinot/plugin/filesystem/S3Config.java | 99 +-
.../apache/pinot/plugin/filesystem/S3PinotFS.java | 93 +-
.../pinot/plugin/filesystem/S3ConfigTest.java | 47 +
.../pinot/plugin/filesystem/S3PinotFSTest.java | 27 +
.../pinot-input-format/pinot-protobuf/pom.xml | 2 +-
.../BaseMultipleSegmentsConversionExecutor.java | 29 +-
.../minion/tasks/SegmentConversionUtils.java | 31 +
.../mergerollup/MergeRollupTaskGenerator.java | 258 ++-
.../SegmentGenerationAndPushTaskExecutor.java | 2 +
.../SegmentGenerationAndPushTaskGenerator.java | 8 +
.../minion/tasks/SegmentConversionUtilsTest.java | 80 +
.../plugin/stream/kinesis/KinesisConsumer.java | 7 +-
.../pulsar/PulsarStreamMetadataProvider.java | 22 +-
.../stream/pulsar/server/PulsarDataProducer.java | 97 +
.../calcite/rel/hint/PinotHintStrategyTable.java | 53 +
.../rel/logical/PinotLogicalSortExchange.java | 112 +
.../PinotAggregateExchangeNodeInsertRule.java | 168 +-
.../rel/rules/PinotJoinExchangeNodeInsertRule.java | 2 +-
.../calcite/rel/rules/PinotQueryRuleSets.java | 61 +-
.../rel/rules/PinotSortExchangeCopyRule.java | 14 +-
.../rel/rules/PinotSortExchangeNodeInsertRule.java | 11 +-
.../rules/PinotWindowExchangeNodeInsertRule.java | 174 ++
.../org/apache/pinot/query/QueryEnvironment.java | 32 +-
.../query/planner/ExplainPlanStageVisitor.java | 6 +
.../apache/pinot/query/planner/PlannerUtils.java | 4 +
.../apache/pinot/query/planner/StageMetadata.java | 28 +
.../query/planner/logical/RelToStageConverter.java | 37 +-
.../pinot/query/planner/logical/RexExpression.java | 9 +
.../query/planner/logical/RexExpressionUtils.java | 2 +-
.../planner/logical/ShuffleRewriteVisitor.java | 6 +
.../planner/logical/StageMetadataVisitor.java | 8 +
.../pinot/query/planner/logical/StagePlanner.java | 34 +-
.../colocated/GreedyShuffleRewriteVisitor.java | 6 +
.../pinot/query/planner/stage/AggregateNode.java | 34 +-
.../stage/DefaultPostOrderTraversalVisitor.java | 6 +
.../apache/pinot/query/planner/stage/JoinNode.java | 19 +-
.../query/planner/stage/MailboxReceiveNode.java | 46 +
.../pinot/query/planner/stage/MailboxSendNode.java | 42 +-
.../query/planner/stage/StageNodeSerDeUtils.java | 2 +
.../query/planner/stage/StageNodeVisitor.java | 2 +
.../pinot/query/planner/stage/WindowNode.java | 152 ++
.../apache/pinot/query/routing/WorkerManager.java | 36 +-
.../org/apache/pinot/query/type/TypeFactory.java | 27 +-
.../org/apache/pinot/query/type/TypeSystem.java | 20 +
.../rel/rules/PinotSortExchangeCopyRuleTest.java | 89 +-
.../apache/pinot/query/QueryCompilationTest.java | 33 +-
.../pinot/query/QueryEnvironmentTestBase.java | 68 +
.../planner/logical/RelToStageConverterTest.java | 133 ++
.../query/queries/ResourceBasedQueryPlansTest.java | 177 ++
.../apache/pinot/query/type/TypeFactoryTest.java | 105 +
.../src/test/resources/queries/AggregatePlans.json | 124 ++
.../test/resources/queries/BasicQueryPlans.json | 63 +
.../src/test/resources/queries/GroupByPlans.json | 242 +++
.../src/test/resources/queries/JoinPlans.json | 325 +++
.../src/test/resources/queries/OrderByPlans.json | 116 +
.../resources/queries/WindowFunctionPlans.json | 2224 ++++++++++++++++++++
.../pinot/query/mailbox/GrpcMailboxService.java | 109 +-
.../pinot/query/mailbox/GrpcReceivingMailbox.java | 52 +-
.../pinot/query/mailbox/GrpcSendingMailbox.java | 90 +-
.../query/mailbox/InMemoryMailboxService.java | 87 +-
.../query/mailbox/InMemoryReceivingMailbox.java | 36 +-
.../query/mailbox/InMemorySendingMailbox.java | 50 +-
.../pinot/query/mailbox/JsonMailboxIdentifier.java | 30 +-
.../pinot/query/mailbox/MailboxIdentifier.java | 10 +
.../apache/pinot/query/mailbox/MailboxService.java | 44 +-
.../query/mailbox/MultiplexingMailboxService.java | 15 +-
.../pinot/query/mailbox/ReceivingMailbox.java | 50 +-
.../apache/pinot/query/mailbox/SendingMailbox.java | 43 +-
.../mailbox/channel/InMemoryTransferStream.java | 101 +
.../channel/MailboxContentStreamObserver.java | 163 +-
.../channel/MailboxStatusStreamObserver.java | 39 +-
.../apache/pinot/query/runtime/QueryRunner.java | 111 +-
.../query/runtime/blocks/TransferableBlock.java | 37 +-
.../runtime/blocks/TransferableBlockUtils.java | 4 +
.../query/runtime/executor/OpChainScheduler.java | 61 +-
.../runtime/executor/OpChainSchedulerService.java | 197 +-
.../runtime/executor/RoundRobinScheduler.java | 315 +--
.../query/runtime/operator/AggregateOperator.java | 186 +-
.../query/runtime/operator/FilterOperator.java | 19 +-
.../query/runtime/operator/HashJoinOperator.java | 106 +-
.../LeafStageTransferableBlockOperator.java | 55 +-
.../runtime/operator/LiteralValueOperator.java | 27 +-
.../runtime/operator/MailboxReceiveOperator.java | 210 +-
.../runtime/operator/MailboxSendOperator.java | 83 +-
.../query/runtime/operator/MultiStageOperator.java | 78 +-
.../pinot/query/runtime/operator/OpChain.java | 30 +-
.../pinot/query/runtime/operator/OpChainId.java | 46 +-
.../pinot/query/runtime/operator/OpChainStats.java | 15 +-
.../query/runtime/operator/OperatorStats.java | 86 +-
.../pinot/query/runtime/operator/SortOperator.java | 142 +-
.../query/runtime/operator/TransformOperator.java | 23 +-
.../runtime/operator/WindowAggregateOperator.java | 332 +++
.../runtime/operator/exchange/BlockExchange.java | 66 +-
.../operator/exchange/BroadcastExchange.java | 16 +-
.../runtime/operator/exchange/HashExchange.java | 35 +-
.../runtime/operator/exchange/RandomExchange.java | 21 +-
.../operator/exchange/SingletonExchange.java | 18 +-
.../runtime/operator/utils/AggregationUtils.java | 182 ++
.../runtime/operator/utils/OperatorUtils.java | 81 +-
.../query/runtime/operator/utils/SortUtils.java | 73 +
...stContext.java => OpChainExecutionContext.java} | 61 +-
.../query/runtime/plan/PhysicalPlanVisitor.java | 55 +-
.../query/runtime/plan/PlanRequestContext.java | 16 +-
.../runtime/plan/ServerRequestPlanVisitor.java | 16 +-
.../plan/server/ServerPlanRequestContext.java | 6 +-
.../apache/pinot/query/service/QueryConfig.java | 2 +-
.../apache/pinot/query/service/QueryServer.java | 32 +-
.../dispatch/AsyncQueryDispatchResponse.java | 62 +
.../query/service/dispatch/CancelObserver.java | 25 +-
.../query/service/dispatch/DispatchClient.java | 73 +
.../query/service/dispatch/DispatchObserver.java | 58 +
.../service/{ => dispatch}/QueryDispatcher.java | 185 +-
.../apache/pinot/query/QueryServerEnclosure.java | 3 +-
.../java/org/apache/pinot/query/QueryTestSet.java | 2 +-
.../query/mailbox/GrpcMailboxServiceTest.java | 223 +-
.../query/mailbox/InMemoryMailboxServiceTest.java | 125 +-
.../query/mailbox/JsonMailboxIdentifierTest.java | 51 +
.../mailbox/MultiplexingMailboxServiceTest.java | 17 +-
.../pinot/query/runtime/QueryRunnerTest.java | 30 +-
.../pinot/query/runtime/QueryRunnerTestBase.java | 182 +-
.../executor/OpChainSchedulerServiceTest.java | 259 +--
.../runtime/executor/RoundRobinSchedulerTest.java | 212 +-
.../runtime/operator/AggregateOperatorTest.java | 42 +-
.../query/runtime/operator/FilterOperatorTest.java | 40 +-
.../runtime/operator/HashJoinOperatorTest.java | 162 +-
.../LeafStageTransferableBlockOperatorTest.java | 48 +-
.../runtime/operator/LiteralValueOperatorTest.java | 19 +-
.../operator/MailboxReceiveOperatorTest.java | 589 +++++-
.../runtime/operator/MailboxSendOperatorTest.java | 109 +-
.../query/runtime/operator/OperatorTestUtil.java | 28 +-
.../query/runtime/operator/SortOperatorTest.java | 208 +-
.../runtime/operator/TransformOperatorTest.java | 27 +-
.../operator/WindowAggregateOperatorTest.java | 586 ++++++
.../operator/exchange/BlockExchangeTest.java | 81 +-
.../operator/exchange/BroadcastExchangeTest.java | 39 +-
.../operator/exchange/HashExchangeTest.java | 47 +-
.../operator/exchange/RandomExchangeTest.java | 33 +-
.../operator/exchange/SingletonExchangeTest.java | 30 +-
.../runtime/queries/ResourceBasedQueriesTest.java | 159 +-
.../pinot/query/service/QueryDispatcherTest.java | 93 -
.../pinot/query/service/QueryServerTest.java | 10 +-
.../service/dispatch/QueryDispatcherTest.java | 244 +++
.../src/test/resources/queries/Aggregates.json | 109 +-
.../src/test/resources/queries/BooleanLogic.json | 16 +-
.../src/test/resources/queries/Case.json | 6 +-
.../test/resources/queries/FromExpressions.json | 30 +-
.../src/test/resources/queries/MathFuncs.json | 16 -
.../src/test/resources/queries/OrderBy.json | 237 ++-
.../src/test/resources/queries/Parallelism.json | 16 +-
.../test/resources/queries/SelectExpressions.json | 18 +-
.../src/test/resources/queries/SpecialSyntax.json | 4 +-
.../test/resources/queries/TableExpressions.json | 16 +
.../src/test/resources/queries/TypeCasting.json | 8 +-
.../test/resources/queries/WindowFunctions.json | 1369 ++++++++++++
.../src/test/resources/queries/WithStatements.json | 65 +-
pinot-segment-local/pom.xml | 4 +
.../DistinctCountThetaSketchValueAggregator.java | 178 ++
.../local/aggregator/ValueAggregatorFactory.java | 6 +
.../local/data/manager/SegmentDataManager.java | 14 +-
.../local/data/manager/TableDataManager.java | 2 +
.../immutable/ImmutableSegmentImpl.java | 16 +-
.../indexsegment/mutable/MutableSegmentImpl.java | 131 +-
.../RealtimeIndexOffHeapMemoryManager.java | 37 +-
.../converter/stats/MutableColumnStatistics.java | 5 +
.../stats/MutableNoDictionaryColStatistics.java | 2 +-
.../local/realtime/impl/RealtimeSegmentConfig.java | 18 +-
.../creator/impl/DefaultIndexCreatorProvider.java | 26 +-
.../creator/impl/SegmentColumnarIndexCreator.java | 244 +--
.../impl/bloom/OnHeapGuavaBloomFilterCreator.java | 16 +
.../creator/impl/fwd/NoOpForwardIndexCreator.java | 64 -
.../segment/index/bloom/BloomIndexPlugin.java | 16 +-
.../local/segment/index/bloom/BloomIndexType.java | 113 +
.../DefaultNullValueVirtualColumnProvider.java | 31 +
.../index/datasource/MutableDataSource.java | 17 +-
.../segment/index/loader/BaseIndexHandler.java | 13 +-
.../segment/index/loader/ForwardIndexHandler.java | 223 +-
.../segment/index/loader/IndexHandlerFactory.java | 1 +
.../segment/index/loader/IndexLoadingConfig.java | 155 +-
...IndexAndDictionaryBasedForwardIndexCreator.java | 5 +-
.../segment/index/loader/SegmentPreProcessor.java | 1 +
.../defaultcolumn/BaseDefaultColumnHandler.java | 41 +-
.../index/readers/BaseImmutableDictionary.java | 35 +
.../startree/v2/builder/MultipleTreesBuilder.java | 4 +-
.../startree/v2/builder/StarTreeIndexCombiner.java | 15 +-
.../v2/builder/StarTreeV2BuilderConfig.java | 8 +-
.../startree/v2/store/StarTreeIndexContainer.java | 6 +-
.../startree/v2/store/StarTreeIndexMapUtils.java | 19 +-
.../upsert/BasePartitionUpsertMetadataManager.java | 176 +-
.../upsert/BaseTableUpsertMetadataManager.java | 11 +-
.../segment/local/upsert/ComparisonColumns.java | 71 +
...oncurrentMapPartitionUpsertMetadataManager.java | 14 +-
.../ConcurrentMapTableUpsertMetadataManager.java | 14 +-
.../segment/local/upsert/PartialUpsertHandler.java | 8 +-
.../upsert/PartitionUpsertMetadataManager.java | 5 +
.../local/upsert/TableUpsertMetadataManager.java | 5 +
.../pinot/segment/local/upsert/UpsertUtils.java | 79 +-
.../segment/local/utils/TableConfigUtils.java | 59 +-
...istinctCountThetaSketchValueAggregatorTest.java | 165 ++
.../ImmutableSegmentImplUpsertSnapshotTest.java | 2 +-
.../mutable/MutableSegmentImplTestUtils.java | 4 +-
.../mutable/MutableSegmentImplUpsertTest.java | 64 +-
.../DefaultNullValueVirtualColumnProviderTest.java | 41 +-
.../index/loader/ForwardIndexHandlerTest.java | 818 +++++--
.../local/segment/index/loader/LoaderTest.java | 11 +-
.../index/loader/SegmentPreProcessorTest.java | 496 ++---
.../segment/store/StarTreeIndexReaderTest.java | 41 +-
...rrentMapPartitionUpsertMetadataManagerTest.java | 58 +-
.../local/upsert/PartialUpsertHandlerTest.java | 7 +-
.../segment/local/utils/TableConfigUtilsTest.java | 66 +-
.../src/test/resources/data/test_upsert_data.json | 21 +-
.../test/resources/data/test_upsert_schema.json | 4 +
.../segment/spi/creator/IndexCreationContext.java | 2 +-
.../spi/creator/SegmentGeneratorConfig.java | 61 +-
.../segment/spi/datasource/DataSourceMetadata.java | 8 +
.../pinot/segment/spi/index/FieldIndexConfigs.java | 119 ++
.../pinot/segment/spi/index/IndexCreator.java | 65 +
.../pinot/segment/spi/index}/IndexHandler.java | 2 +-
.../pinot/segment/spi/index/IndexPlugin.java | 35 +
.../pinot/segment/spi/index/IndexReader.java | 8 +-
.../spi/index/IndexReaderConstraintException.java | 26 +-
.../segment/spi/index/IndexReaderFactory.java | 55 +
.../pinot/segment/spi/index/IndexService.java | 148 ++
.../apache/pinot/segment/spi/index/IndexType.java | 121 ++
.../pinot/segment/spi/index/StandardIndexes.java | 59 +
.../spi/index/creator/BloomFilterCreator.java | 31 +-
.../spi/index/metadata/ColumnMetadataImpl.java | 5 +-
.../spi/index/reader/BloomFilterReader.java | 4 +-
.../pinot/segment/spi/index/reader/Dictionary.java | 15 +
.../startree/AggregationFunctionColumnPair.java | 10 +-
.../segment/spi/index/FieldIndexConfigsTest.java | 33 +-
.../org/apache/pinot/server/conf/ServerConf.java | 39 +-
.../pinot/server/starter/ServerInstance.java | 5 +-
.../server/starter/helix/BaseServerStarter.java | 2 +-
.../helix/DefaultHelixStarterServerConfig.java | 73 -
.../helix/HelixInstanceDataManagerConfig.java | 15 +-
.../pinot/server/worker/WorkerQueryServer.java | 2 +-
.../apache/pinot/server/api/AccessControlTest.java | 3 +-
.../apache/pinot/server/api/BaseResourceTest.java | 3 +-
.../server/api/PinotServerAppConfigsTest.java | 3 +-
.../pinot/spi/config/table/BloomFilterConfig.java | 34 +-
.../apache/pinot/spi/config/table/IndexConfig.java | 30 +-
.../apache/pinot/spi/config/table/TableConfig.java | 12 +-
.../pinot/spi/config/table/UpsertConfig.java | 56 +-
.../java/org/apache/pinot/spi/data/FieldSpec.java | 3 +-
.../apache/pinot/spi/data/MultiValueVisitor.java | 99 +
.../apache/pinot/spi/data/SingleValueVisitor.java | 27 +-
.../spi/stream/PartitionGroupMetadataFetcher.java | 3 +-
.../apache/pinot/spi/utils/CommonConstants.java | 56 +-
.../java/org/apache/pinot/spi/utils/JsonUtils.java | 11 +-
.../pinot/spi/utils/RebalanceConfigConstants.java | 6 +
.../utils/builder/ControllerRequestURLBuilder.java | 2 +-
.../spi/utils/builder/TableConfigBuilder.java | 9 +-
.../pinot/spi/config/table/UpsertConfigTest.java | 3 +-
pinot-tools/pom.xml | 60 +
.../org/apache/pinot/tools/AuthQuickstart.java | 16 +-
.../pinot/tools/MultistageEngineQuickStart.java | 17 +-
.../apache/pinot/tools/PinotTableRebalancer.java | 2 +
.../admin/command/AbstractBaseAdminCommand.java | 5 +-
.../tools/admin/command/QuickstartRunner.java | 8 +-
.../admin/command/StartControllerCommand.java | 1 +
.../tools/admin/command/StartServerCommand.java | 1 +
.../admin/command/StartServiceManagerCommand.java | 8 +-
.../admin/command/StreamGitHubEventsCommand.java | 48 +-
.../PullRequestMergedEventsStream.java | 36 +-
.../apache/pinot/tools/utils/StreamSourceType.java | 3 +-
.../src/main/resources/appAssemblerScriptTemplate | 2 +-
.../upsertMeetupRsvp_realtime_table_config.json | 2 +-
...ertPartialMeetupRsvp_realtime_table_config.json | 2 +-
pom.xml | 23 +-
763 files changed, 30716 insertions(+), 12875 deletions(-)
create mode 100644 .github/workflows/build-multi-arch-pinot-docker-image.yml
create mode 100644 .github/workflows/build-pinot-base-docker-image.yml
create mode 100755
.github/workflows/scripts/docker/.pinot_base_docker_image_build_and_push.sh
create mode 100755 .github/workflows/scripts/docker/.pinot_build_info_gen.sh
create mode 100755
.github/workflows/scripts/docker/.pinot_multi_arch_docker_image_manifest_package.sh
create mode 100755
.github/workflows/scripts/docker/.pinot_single_platform_docker_image_build.sh
copy docker/images/pinot-base/pinot-base-build/{Dockerfile =>
amazoncorretto.dockerfile} (60%)
copy docker/images/pinot-base/pinot-base-build/{Dockerfile =>
ms-openjdk.dockerfile} (94%)
rename docker/images/pinot-base/pinot-base-build/{Dockerfile =>
openjdk.dockerfile} (95%)
copy docker/images/pinot-base/pinot-base-runtime/{Dockerfile =>
amazoncorretto.dockerfile} (84%)
copy docker/images/pinot-base/pinot-base-runtime/{Dockerfile =>
ms-openjdk.dockerfile} (91%)
rename docker/images/pinot-base/pinot-base-runtime/{Dockerfile =>
openjdk.dockerfile} (95%)
rename kubernetes/helm/pinot/templates/broker/{ingress.yaml =>
ingress-v1.yaml} (54%)
create mode 100644 kubernetes/helm/pinot/templates/broker/ingress-v1beta1.yaml
rename kubernetes/helm/pinot/templates/controller/{ingress.yaml =>
ingress-v1.yaml} (54%)
create mode 100644
kubernetes/helm/pinot/templates/controller/ingress-v1beta1.yaml
create mode 100644
pinot-broker/src/main/java/org/apache/pinot/broker/routing/instanceselector/NewSegmentState.java
copy
pinot-core/src/main/java/org/apache/pinot/core/operator/docidsets/MatchAllDocIdSet.java
=>
pinot-broker/src/main/java/org/apache/pinot/broker/routing/instanceselector/SegmentInstanceCandidate.java
(62%)
create mode 100644
pinot-broker/src/main/java/org/apache/pinot/broker/routing/instanceselector/SegmentStates.java
create mode 100644
pinot-common/src/main/java/org/apache/pinot/common/response/broker/BrokerResponseNativeV2.java
create mode 100644
pinot-common/src/main/java/org/apache/pinot/common/response/broker/BrokerResponseStats.java
rename
pinot-core/src/main/java/org/apache/pinot/core/operator/docidsets/ArrayBasedDocIdSet.java
=>
pinot-common/src/test/java/org/apache/pinot/common/utils/FileUploadDownloadClientWithoutServerTest.java
(56%)
create mode 100644
pinot-common/src/test/resources/testConfigs/multipleIndexPartialUpsertConfig.json
create mode 100644
pinot-common/src/test/resources/testConfigs/multipleIndexPartialUpsertMultipleComparisonColumnConfig.json
copy pinot-connectors/{pinot-spark-connector =>
pinot-spark-2-connector}/README.md (68%)
copy pinot-connectors/{pinot-spark-connector =>
pinot-spark-2-connector}/documentation/images/spark-pinot-connector-executor-server-interaction.jpg
(100%)
copy pinot-connectors/{pinot-spark-connector =>
pinot-spark-2-connector}/documentation/read_model.md (94%)
copy pinot-connectors/{pinot-spark-connector =>
pinot-spark-2-connector}/pom.xml (63%)
rename pinot-connectors/{pinot-spark-connector =>
pinot-spark-2-connector}/src/main/resources/META-INF/services/org.apache.spark.sql.sources.DataSourceRegister
(100%)
rename pinot-connectors/{pinot-spark-connector =>
pinot-spark-2-connector}/src/main/scala/org/apache/pinot/connector/spark/datasource/PinotDataSourceReader.scala
(85%)
rename pinot-connectors/{pinot-spark-connector =>
pinot-spark-2-connector}/src/main/scala/org/apache/pinot/connector/spark/datasource/PinotDataSourceV2.scala
(100%)
copy pinot-connectors/{pinot-spark-connector =>
pinot-spark-2-connector}/src/main/scala/org/apache/pinot/connector/spark/datasource/PinotInputPartition.scala
(63%)
copy
pinot-connectors/{pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/connector/PinotUtils.scala
=>
pinot-spark-2-connector/src/main/scala/org/apache/pinot/connector/spark/datasource/TypeConverter.scala}
(88%)
copy
pinot-connectors/{pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/connector
=>
pinot-spark-2-connector/src/main/scala/org/apache/pinot/connector/spark/datasource/query}/FilterPushDown.scala
(98%)
copy pinot-connectors/{pinot-spark-connector =>
pinot-spark-2-connector}/src/test/resources/schema/pinot-schema.json (100%)
copy pinot-connectors/{pinot-spark-connector =>
pinot-spark-2-connector}/src/test/resources/schema/spark-schema.json (100%)
copy
pinot-connectors/{pinot-spark-connector/src/test/scala/org/apache/pinot/connector/spark
=>
pinot-spark-2-connector/src/test/scala/org/apache/pinot/connector/spark/datasource}/BaseTest.scala
(94%)
copy
pinot-connectors/{pinot-spark-connector/src/test/scala/org/apache/pinot/connector/spark
=>
pinot-spark-2-connector/src/test/scala/org/apache/pinot/connector/spark/datasource}/ExampleSparkPinotConnectorTest.scala
(98%)
copy
pinot-connectors/{pinot-spark-connector/src/test/scala/org/apache/pinot/connector/spark/connector/PinotUtilsTest.scala
=>
pinot-spark-2-connector/src/test/scala/org/apache/pinot/connector/spark/datasource/TypeConverterTest.scala}
(80%)
copy
pinot-connectors/{pinot-spark-connector/src/test/scala/org/apache/pinot/connector/spark/connector
=>
pinot-spark-2-connector/src/test/scala/org/apache/pinot/connector/spark/datasource/query}/FilterPushDownTest.scala
(96%)
rename pinot-connectors/{pinot-spark-connector =>
pinot-spark-3-connector}/README.md (68%)
rename pinot-connectors/{pinot-spark-connector =>
pinot-spark-3-connector}/documentation/images/spark-pinot-connector-executor-server-interaction.jpg
(100%)
rename pinot-connectors/{pinot-spark-connector =>
pinot-spark-3-connector}/documentation/read_model.md (94%)
copy pinot-connectors/{pinot-spark-connector =>
pinot-spark-3-connector}/pom.xml (77%)
create mode 100644
pinot-connectors/pinot-spark-3-connector/src/main/resources/META-INF/services/org.apache.spark.sql.sources.DataSourceRegister
create mode 100644
pinot-connectors/pinot-spark-3-connector/src/main/scala/org/apache/pinot/connector/spark/v3/datasource/PinotDataSource.scala
rename
pinot-connectors/{pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark
=>
pinot-spark-3-connector/src/main/scala/org/apache/pinot/connector/spark/v3}/datasource/PinotInputPartition.scala
(50%)
create mode 100644
pinot-connectors/pinot-spark-3-connector/src/main/scala/org/apache/pinot/connector/spark/v3/datasource/PinotScan.scala
create mode 100644
pinot-connectors/pinot-spark-3-connector/src/main/scala/org/apache/pinot/connector/spark/v3/datasource/PinotScanBuilder.scala
create mode 100644
pinot-connectors/pinot-spark-3-connector/src/main/scala/org/apache/pinot/connector/spark/v3/datasource/PinotTable.scala
rename
pinot-connectors/{pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/connector/PinotUtils.scala
=>
pinot-spark-3-connector/src/main/scala/org/apache/pinot/connector/spark/v3/datasource/TypeConverter.scala}
(86%)
rename
pinot-connectors/{pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/connector
=>
pinot-spark-3-connector/src/main/scala/org/apache/pinot/connector/spark/v3/datasource/query}/FilterPushDown.scala
(98%)
rename pinot-connectors/{pinot-spark-connector =>
pinot-spark-3-connector}/src/test/resources/log4j2.xml (100%)
rename pinot-connectors/{pinot-spark-connector =>
pinot-spark-3-connector}/src/test/resources/schema/pinot-schema.json (99%)
rename pinot-connectors/{pinot-spark-connector =>
pinot-spark-3-connector}/src/test/resources/schema/spark-schema.json (99%)
copy
pinot-connectors/{pinot-spark-connector/src/test/scala/org/apache/pinot/connector/spark
=>
pinot-spark-3-connector/src/test/scala/org/apache/pinot/connector/spark/v3/datasource}/BaseTest.scala
(94%)
rename
pinot-connectors/{pinot-spark-connector/src/test/scala/org/apache/pinot/connector/spark
=>
pinot-spark-3-connector/src/test/scala/org/apache/pinot/connector/spark/v3/datasource}/ExampleSparkPinotConnectorTest.scala
(98%)
create mode 100644
pinot-connectors/pinot-spark-3-connector/src/test/scala/org/apache/pinot/connector/spark/v3/datasource/PinotScanBuilderTest.scala
create mode 100644
pinot-connectors/pinot-spark-3-connector/src/test/scala/org/apache/pinot/connector/spark/v3/datasource/PinotScanTest.scala
rename
pinot-connectors/{pinot-spark-connector/src/test/scala/org/apache/pinot/connector/spark/connector/PinotUtilsTest.scala
=>
pinot-spark-3-connector/src/test/scala/org/apache/pinot/connector/spark/v3/datasource/TypeConverterTest.scala}
(80%)
rename
pinot-connectors/{pinot-spark-connector/src/test/scala/org/apache/pinot/connector/spark/connector
=>
pinot-spark-3-connector/src/test/scala/org/apache/pinot/connector/spark/v3/datasource/query}/FilterPushDownTest.scala
(95%)
rename pinot-connectors/{pinot-spark-connector => pinot-spark-common}/pom.xml
(91%)
create mode 100644
pinot-connectors/pinot-spark-common/src/main/java/org/apache/pinot/connector/spark/common/CaseInsensitiveStringMap.java
rename
pinot-connectors/{pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/utils
=>
pinot-spark-common/src/main/scala/org/apache/pinot/connector/spark/common}/HttpUtils.scala
(95%)
rename
pinot-connectors/{pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/utils
=>
pinot-spark-common/src/main/scala/org/apache/pinot/connector/spark/common}/Logging.scala
(97%)
rename
pinot-connectors/{pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/connector
=>
pinot-spark-common/src/main/scala/org/apache/pinot/connector/spark/common}/PinotClusterClient.scala
(89%)
rename
pinot-connectors/{pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/datasource
=>
pinot-spark-common/src/main/scala/org/apache/pinot/connector/spark/common}/PinotDataSourceReadOptions.scala
(64%)
rename
pinot-connectors/{pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/exceptions
=>
pinot-spark-common/src/main/scala/org/apache/pinot/connector/spark/common}/exceptions.scala
(95%)
rename
pinot-connectors/{pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark
=>
pinot-spark-common/src/main/scala/org/apache/pinot/connector/spark/common}/package.scala
(95%)
rename
pinot-connectors/{pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/connector
=>
pinot-spark-common/src/main/scala/org/apache/pinot/connector/spark/common/partition}/PinotSplitter.scala
(86%)
copy
pinot-connectors/{pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/connector/query/GeneratedSQLs.scala
=>
pinot-spark-common/src/main/scala/org/apache/pinot/connector/spark/common/query/ScanQuery.scala}
(92%)
rename
pinot-connectors/{pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/connector/query/SQLSelectionQueryGenerator.scala
=>
pinot-spark-common/src/main/scala/org/apache/pinot/connector/spark/common/query/ScanQueryGenerator.scala}
(78%)
create mode 100644
pinot-connectors/pinot-spark-common/src/main/scala/org/apache/pinot/connector/spark/common/reader/PinotAbstractPartitionReader.scala
rename
pinot-connectors/{pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/connector
=>
pinot-spark-common/src/main/scala/org/apache/pinot/connector/spark/common/reader}/PinotGrpcServerDataFetcher.scala
(80%)
rename
pinot-connectors/{pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/connector
=>
pinot-spark-common/src/main/scala/org/apache/pinot/connector/spark/common/reader}/PinotServerDataFetcher.scala
(89%)
copy {pinot-broker =>
pinot-connectors/pinot-spark-common}/src/test/resources/log4j2.xml (100%)
rename
pinot-connectors/{pinot-spark-connector/src/test/scala/org/apache/pinot/connector/spark
=>
pinot-spark-common/src/test/scala/org/apache/pinot/connector/spark/common}/BaseTest.scala
(95%)
rename
pinot-connectors/{pinot-spark-connector/src/test/scala/org/apache/pinot/connector/spark/datasource
=>
pinot-spark-common/src/test/scala/org/apache/pinot/connector/spark/common}/PinotDataSourceReadOptionsTest.scala
(80%)
rename
pinot-connectors/{pinot-spark-connector/src/test/scala/org/apache/pinot/connector/spark/connector
=>
pinot-spark-common/src/test/scala/org/apache/pinot/connector/spark/common}/PinotSplitterTest.scala
(80%)
rename
pinot-connectors/{pinot-spark-connector/src/test/scala/org/apache/pinot/connector/spark/connector/query/SQLSelectionQueryGeneratorTest.scala
=>
pinot-spark-common/src/test/scala/org/apache/pinot/connector/spark/common/query/ScanQueryGeneratorTest.scala}
(74%)
delete mode 100644
pinot-connectors/pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/datasource/PinotInputPartitionReader.scala
copy
pinot-core/src/main/java/org/apache/pinot/core/operator/blocks/results/MetadataResultsBlock.java
=>
pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/ServerRebalanceJobStatusResponse.java
(50%)
copy
pinot-core/src/main/java/org/apache/pinot/core/operator/docidsets/MatchAllDocIdSet.java
=>
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/NoOpTableRebalanceObserver.java
(64%)
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/TableRebalanceObserver.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/TableRebalanceProgressStats.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/ZkBasedTableRebalanceObserver.java
rename
pinot-connectors/pinot-spark-connector/src/main/scala/org/apache/pinot/connector/spark/connector/query/GeneratedSQLs.scala
=> pinot-controller/src/main/resources/app/components/AppLoadingIndicator.tsx
(59%)
create mode 100644
pinot-controller/src/main/resources/app/components/auth/AuthProvider.tsx
create mode 100644 pinot-controller/src/main/resources/app/index.tsx
create mode 100644
pinot-controller/src/test/java/org/apache/pinot/controller/helix/core/rebalance/TestZkBasedTableRebalanceObserver.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/function/scalar/SketchFunctions.java
copy
pinot-core/src/main/java/org/apache/pinot/core/operator/{blocks/PassThroughTransformBlock.java
=> BaseProjectOperator.java} (57%)
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/operator/ColumnContext.java
rename
pinot-core/src/main/java/org/apache/pinot/core/operator/blocks/{PassThroughTransformBlock.java
=> ValueBlock.java} (59%)
delete mode 100644
pinot-core/src/main/java/org/apache/pinot/core/operator/dociditerators/ArrayBasedDocIdIterator.java
rename
pinot-core/src/main/java/org/apache/pinot/core/operator/docidsets/{ExpressionFilterDocIdSet.java
=> ExpressionDocIdSet.java} (88%)
delete mode 100644
pinot-core/src/main/java/org/apache/pinot/core/operator/docidsets/FilterBlockDocIdSet.java
delete mode 100644
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/PassThroughTransformOperator.java
rename
pinot-core/src/main/java/org/apache/pinot/core/plan/{TransformPlanNode.java =>
ProjectPlanNode.java} (63%)
delete mode 100644
pinot-core/src/main/java/org/apache/pinot/core/plan/ProjectionPlanNode.java
delete mode 100644
pinot-core/src/main/java/org/apache/pinot/core/query/config/QueryPlannerConfig.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/query/reduce/ExecutionStatsAggregator.java
copy
pinot-core/src/main/java/org/apache/pinot/core/segment/processing/aggregator/{ValueAggregatorFactory.java
=> DistinctCountHLLAggregator.java} (52%)
copy
pinot-core/src/main/java/org/apache/pinot/core/segment/processing/aggregator/{ValueAggregatorFactory.java
=> DistinctCountThetaSketchAggregator.java} (50%)
rename
pinot-core/src/main/java/org/apache/pinot/core/startree/plan/{StarTreeTransformPlanNode.java
=> StarTreeProjectPlanNode.java} (51%)
delete mode 100644
pinot-core/src/main/java/org/apache/pinot/core/startree/plan/StarTreeProjectionPlanNode.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/function/scalar/SketchFunctionsTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/operator/filter/predicate/InPredicateEvaluatorFactoryTest.java
copy
pinot-core/src/{main/java/org/apache/pinot/core/data/manager/offline/ImmutableSegmentDataManager.java
=>
test/java/org/apache/pinot/core/startree/v2/DistinctCountThetaSketchStarTreeV2Test.java}
(50%)
copy
pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/{MultiStageEngineIntegrationTest.java
=> MultiStageEngineCustomTenantIntegrationTest.java} (79%)
create mode 100644
pinot-plugins/pinot-file-system/pinot-s3/src/test/java/org/apache/pinot/plugin/filesystem/S3ConfigTest.java
create mode 100644
pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/test/java/org/apache/pinot/plugin/minion/tasks/SegmentConversionUtilsTest.java
create mode 100644
pinot-plugins/pinot-stream-ingestion/pinot-pulsar/src/main/java/org/apache/pinot/plugin/stream/pulsar/server/PulsarDataProducer.java
create mode 100644
pinot-query-planner/src/main/java/org/apache/calcite/rel/hint/PinotHintStrategyTable.java
create mode 100644
pinot-query-planner/src/main/java/org/apache/calcite/rel/logical/PinotLogicalSortExchange.java
create mode 100644
pinot-query-planner/src/main/java/org/apache/calcite/rel/rules/PinotWindowExchangeNodeInsertRule.java
create mode 100644
pinot-query-planner/src/main/java/org/apache/pinot/query/planner/stage/WindowNode.java
create mode 100644
pinot-query-planner/src/test/java/org/apache/pinot/query/planner/logical/RelToStageConverterTest.java
create mode 100644
pinot-query-planner/src/test/java/org/apache/pinot/query/queries/ResourceBasedQueryPlansTest.java
create mode 100644
pinot-query-planner/src/test/java/org/apache/pinot/query/type/TypeFactoryTest.java
create mode 100644
pinot-query-planner/src/test/resources/queries/AggregatePlans.json
create mode 100644
pinot-query-planner/src/test/resources/queries/BasicQueryPlans.json
create mode 100644
pinot-query-planner/src/test/resources/queries/GroupByPlans.json
create mode 100644
pinot-query-planner/src/test/resources/queries/JoinPlans.json
create mode 100644
pinot-query-planner/src/test/resources/queries/OrderByPlans.json
create mode 100644
pinot-query-planner/src/test/resources/queries/WindowFunctionPlans.json
create mode 100644
pinot-query-runtime/src/main/java/org/apache/pinot/query/mailbox/channel/InMemoryTransferStream.java
copy
pinot-core/src/main/java/org/apache/pinot/core/data/manager/offline/ImmutableSegmentDataManager.java
=>
pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/OpChainId.java
(50%)
create mode 100644
pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/WindowAggregateOperator.java
create mode 100644
pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/utils/AggregationUtils.java
create mode 100644
pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/utils/SortUtils.java
copy
pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/plan/{PlanRequestContext.java
=> OpChainExecutionContext.java} (58%)
create mode 100644
pinot-query-runtime/src/main/java/org/apache/pinot/query/service/dispatch/AsyncQueryDispatchResponse.java
rename
pinot-core/src/main/java/org/apache/pinot/core/common/BlockDocIdValueIterator.java
=>
pinot-query-runtime/src/main/java/org/apache/pinot/query/service/dispatch/CancelObserver.java
(64%)
create mode 100644
pinot-query-runtime/src/main/java/org/apache/pinot/query/service/dispatch/DispatchClient.java
create mode 100644
pinot-query-runtime/src/main/java/org/apache/pinot/query/service/dispatch/DispatchObserver.java
rename pinot-query-runtime/src/main/java/org/apache/pinot/query/service/{ =>
dispatch}/QueryDispatcher.java (50%)
create mode 100644
pinot-query-runtime/src/test/java/org/apache/pinot/query/mailbox/JsonMailboxIdentifierTest.java
create mode 100644
pinot-query-runtime/src/test/java/org/apache/pinot/query/runtime/operator/WindowAggregateOperatorTest.java
delete mode 100644
pinot-query-runtime/src/test/java/org/apache/pinot/query/service/QueryDispatcherTest.java
create mode 100644
pinot-query-runtime/src/test/java/org/apache/pinot/query/service/dispatch/QueryDispatcherTest.java
create mode 100644
pinot-query-runtime/src/test/resources/queries/WindowFunctions.json
create mode 100644
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/aggregator/DistinctCountThetaSketchValueAggregator.java
delete mode 100644
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/fwd/NoOpForwardIndexCreator.java
copy
pinot-controller/src/main/java/org/apache/pinot/controller/api/exception/NoTaskMetadataException.java
=>
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/bloom/BloomIndexPlugin.java
(69%)
create mode 100644
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/bloom/BloomIndexType.java
create mode 100644
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/ComparisonColumns.java
create mode 100644
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/aggregator/DistinctCountThetaSketchValueAggregatorTest.java
create mode 100644
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/FieldIndexConfigs.java
create mode 100644
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/IndexCreator.java
rename
{pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader
=>
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index}/IndexHandler.java
(96%)
create mode 100644
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/IndexPlugin.java
rename
pinot-core/src/main/java/org/apache/pinot/core/common/BlockDocIdValueSet.java
=>
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/IndexReader.java
(87%)
rename
pinot-controller/src/main/java/org/apache/pinot/controller/api/exception/NoTaskMetadataException.java
=>
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/IndexReaderConstraintException.java
(52%)
create mode 100644
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/IndexReaderFactory.java
create mode 100644
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/IndexService.java
create mode 100644
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/IndexType.java
create mode 100644
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/StandardIndexes.java
copy
pinot-clients/pinot-jdbc-client/src/test/java/org/apache/pinot/client/PinotStatementTest.java
=>
pinot-segment-spi/src/test/java/org/apache/pinot/segment/spi/index/FieldIndexConfigsTest.java
(53%)
delete mode 100644
pinot-server/src/main/java/org/apache/pinot/server/starter/helix/DefaultHelixStarterServerConfig.java
rename
pinot-query-planner/src/main/java/org/apache/pinot/query/planner/hints/PinotRelationalHints.java
=> pinot-spi/src/main/java/org/apache/pinot/spi/config/table/IndexConfig.java
(54%)
create mode 100644
pinot-spi/src/main/java/org/apache/pinot/spi/data/MultiValueVisitor.java
rename
pinot-core/src/main/java/org/apache/pinot/core/common/BlockMetadata.java =>
pinot-spi/src/main/java/org/apache/pinot/spi/data/SingleValueVisitor.java (66%)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
