This is an automated email from the ASF dual-hosted git repository.
jlli pushed a change to branch helix-1.0
in repository https://gitbox.apache.org/repos/asf/pinot.git.
discard bf3f61e Upgrade Helix to 1.0.2
add 9ed08dc Update license & notice for release 0.10.0 (#8330)
add 262dc50 Add `less` command to Pinot Docker image. (#8336)
add 91c2ebb [issue-8297] upgrade netty (#8346)
add 6fafa4c Canonicalize the function name to be lower case without
underscore when parsing the expression (#8341)
add e6a15a2 Add more aggregations that can be solved with dictionary
(#8351)
add cc33e2c upgrade jetty due to security issue (#8348)
add aa44189 Refactor streaming transformation code so it can be reused in
other places (#8335)
add c9c185a Validate schema name before permissions to return proper
message (#8357)
add 6ceecac identify running tasks based on task final states (#8354)
add 98d7810 [Clean up] Clean up some routing related classes (#8356)
add 435ff0d Change controller metrics to exclude the replaced segments
which are specified in the segment lineage entries and cannot be queried from
the table. (#8345)
add 2e8a519 Fix setting of metrics compression type in
RealtimeSegmentConverter (#8350)
add c533c6c Add .mvn/ to .gitignore to allow jvm.config file (#8359)
add 7c54ca6 adding jacoco inclusive rule (#8361)
add f60bfc8 Add support for Kinesis datasource in Github Events recipe
(#8333)
add 360a205 Add prefixesToRename config for renaming fields upon
ingestion (#8273)
add 25cd6e3 Fix the flakiness of MultiNodesOfflineClusterIntegrationTest
(#8367)
add cd311bc Use netty-bom to keep the netty-related libraries sync (#8365)
add 114e4c5 add UT for disjoint check on lineage segmentFroms (#8242)
add b05a541 Adding schema validation for recommendation engine (#8370)
add 681da61 Refresh ZK metadata when dimension table is updated (#8133)
add 24d4fd2 fix checkstyle (#8377)
add 764a330 Refactor Pinot http utils into HttpClient (#8329)
add 34002c7 Refactor Segment creation to use TransformPipeline too (#8364)
add 078c711 Add Time-Series Gapfilling functionality. (#8029)
add 4ff59aa Upgrade Helix to 1.0.2
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (bf3f61e)
\
N -- N -- N refs/heads/helix-1.0 (4ff59aa)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.gitignore | 1 +
LICENSE-binary | 135 +-
NOTICE-binary | 76 +-
config/checkstyle.xml | 1 +
docker/images/pinot/Dockerfile | 2 +-
licenses-binary/LICENSE-argparse4j.txt | 23 +
...{LICENSE-stax2.txt => LICENSE-bsd-2-clause.txt} | 7 +-
...{LICENSE-stax2.txt => LICENSE-bsd-3-clause.txt} | 7 +-
.../{LICENSE-bouncycastle.txt => LICENSE-mit.txt} | 4 +-
.../{LICENSE-azure.txt => LICENSE-pyrolite.txt} | 6 +-
licenses-binary/LICENSE-slf4j.txt | 2 +-
.../requesthandler/BaseBrokerRequestHandler.java | 108 +-
.../requesthandler/GrpcBrokerRequestHandler.java | 42 +-
.../SingleConnectionBrokerRequestHandler.java | 10 +-
.../requesthandler/DistinctCountRewriteTest.java | 4 +-
.../SelectStarWithOtherColsRewriteTest.java | 17 +-
.../common/function/TransformFunctionType.java | 8 +-
.../pinot/common/metrics/ControllerGauge.java | 5 +
.../apache/pinot/common/request/DataSource.java | 128 +-
.../common/utils/FileUploadDownloadClient.java | 413 +--
.../org/apache/pinot/common/utils/TlsUtils.java | 46 +-
.../apache/pinot/common/utils/http/HttpClient.java | 455 +++
.../pinot/common/utils/request/RequestUtils.java | 47 +-
.../pql/parsers/pql2/ast/OutputColumnAstNode.java | 2 +-
.../apache/pinot/sql/parsers/CalciteSqlParser.java | 167 +-
.../pinot/sql/parsers/rewriter/AliasApplier.java | 3 +-
...nAggregationGroupByToDistinctQueryRewriter.java | 2 +-
.../sql/parsers/rewriter/OrdinalsUpdater.java | 3 +-
.../rewriter/PredicateComparisonRewriter.java | 8 +-
.../sql/parsers/rewriter/SelectionsRewriter.java | 56 +-
.../common/utils/FileUploadDownloadClientTest.java | 3 +-
.../common/utils/config/TableConfigSerDeTest.java | 4 +-
.../apache/pinot/pql/parsers/Pql2CompilerTest.java | 17 +-
.../request/BrokerRequestSerializationTest.java | 7 +-
.../pinot/sql/parsers/CalciteSqlCompilerTest.java | 475 +--
...regationGroupByToDistinctQueryRewriterTest.java | 30 +-
pinot-common/src/thrift/query.thrift | 1 +
.../pinot-core-jdk8/pom.xml | 6 +-
.../presto/PinotScatterGatherQueryClient.java | 2 +-
.../resources/PinotInstanceRestletResource.java | 18 +-
.../api/resources/PinotQueryResource.java | 4 +-
.../api/resources/PinotSchemaRestletResource.java | 42 +-
.../controller/helix/ControllerRequestClient.java | 123 +
.../controller/helix/SegmentStatusChecker.java | 16 +-
.../core/minion/generator/TaskGeneratorUtils.java | 38 +-
.../controller/recommender/io/InputManager.java | 2 +
.../pinot/controller/ControllerTestUtils.java | 181 +-
.../pinot/controller/api/AccessControlTest.java | 4 +-
...PinotInstanceAssignmentRestletResourceTest.java | 26 +-
.../api/PinotSchemaRestletResourceTest.java | 93 +-
.../api/PinotTableRestletResourceTest.java | 44 +-
.../api/resources/ZookeeperResourceTest.java | 5 +-
.../controller/helix/SegmentStatusCheckerTest.java | 53 +-
.../helix/core/PinotHelixResourceManagerTest.java | 173 +
.../minion/generator/TaskGeneratorUtilsTest.java | 101 +
.../AggregateMetricsRuleInput.json | 2 +-
.../recommenderInput/BloomFilterInput.json | 2 +-
.../recommenderInput/EmptyQueriesInput.json | 2 +-
.../resources/recommenderInput/FlagQueryInput.json | 2 +-
.../resources/recommenderInput/InvalidInput1.json | 2 +-
.../resources/recommenderInput/InvalidInput2.json | 2 +-
.../recommenderInput/KafkaPartitionRuleInput.json | 2 +-
.../recommenderInput/KafkaPartitionRuleInput2.json | 2 +-
...NoDictionaryOnHeapDictionaryJointRuleInput.json | 2 +-
.../PinotTablePartitionRuleInput.json | 2 +-
.../recommenderInput/RangeIndexInput.json | 2 +-
.../RealtimeProvisioningInput_dateTimeColumn.json | 2 +-
.../RealtimeProvisioningInput_timeColumn.json | 2 +-
.../recommenderInput/SegmentSizeRuleInput.json | 2 +-
...gmentSizeRuleInput_noNeedToGenerateSegment.json | 2 +-
.../SegmentSizeRuleInput_realtimeOnlyTable.json | 2 +-
...put_ruleIsDisableButItNeedsToBeSilentlyRun.json | 2 +-
.../recommenderInput/SortedInvertedIndexInput.json | 2 +-
.../VariedLengthDictionaryInput.json | 2 +-
pinot-core/pom.xml | 18 +-
.../core/data/manager/offline/DimensionTable.java | 57 +
.../manager/offline/DimensionTableDataManager.java | 42 +-
.../realtime/LLRealtimeSegmentDataManager.java | 53 +-
.../realtime/Server2ControllerSegmentUploader.java | 3 +-
.../query/DictionaryBasedAggregationOperator.java | 85 +-
.../pinot/core/plan/AggregationPlanNode.java | 10 +-
.../org/apache/pinot/core/plan/FilterPlanNode.java | 10 +-
.../DistinctCountHLLAggregationFunction.java | 4 +
.../DistinctCountRawHLLAggregationFunction.java | 8 +-
...istinctCountThetaSketchAggregationFunction.java | 27 +-
.../statement/JsonStatementOptimizer.java | 12 +-
.../statement/StringPredicateFilterOptimizer.java | 44 +-
.../core/query/reduce/BrokerReduceService.java | 25 +-
.../core/query/reduce/GapfillFilterHandler.java | 80 +
.../pinot/core/query/reduce/GapfillProcessor.java | 477 +++
.../core/query/reduce/HavingFilterHandler.java | 129 +-
.../core/query/reduce/PostAggregationHandler.java | 90 +-
.../core/query/reduce/RowBasedBlockValSet.java | 192 +
.../core/query/reduce/filter/AndRowMatcher.java | 32 +-
.../query/reduce/filter/ColumnValueExtractor.java | 36 +-
.../reduce/filter/LiteralValueExtractor.java} | 28 +-
.../core/query/reduce/filter/OrRowMatcher.java | 34 +-
.../query/reduce/filter/PredicateRowMatcher.java | 66 +
.../pinot/core/query/reduce/filter/RowMatcher.java | 11 +-
.../reduce/filter/RowMatcherFactory.java} | 32 +-
.../core/query/reduce/filter/ValueExtractor.java | 23 +-
.../reduce/filter/ValueExtractorFactory.java} | 17 +-
.../core/query/request/context/QueryContext.java | 16 +-
.../BrokerRequestToQueryContextConverter.java | 17 +-
.../apache/pinot/core/transport/QueryRouter.java | 38 +-
.../pinot/core/transport/ServerChannels.java | 20 +-
.../pinot/core/transport/ServerInstance.java | 74 +-
.../org/apache/pinot/core/util/GapfillUtils.java | 296 +-
.../ServerSegmentCompletionProtocolHandler.java | 5 +-
.../offline/DimensionTableDataManagerTest.java | 56 +-
.../realtime/LLRealtimeSegmentDataManagerTest.java | 90 +-
.../query/reduce/GapfillFilterHandlerTest.java | 44 +
.../pinot/core/transport/QueryRoutingTest.java | 9 +-
.../org/apache/pinot/queries/BaseQueriesTest.java | 29 +-
.../pinot/queries/DistinctCountQueriesTest.java | 89 +
.../apache/pinot/queries/DistinctQueriesTest.java | 4 +-
.../apache/pinot/queries/GapfillQueriesTest.java | 3699 ++++++++++++++++++++
...terSegmentAggregationMultiValueQueriesTest.java | 226 +-
...erSegmentAggregationSingleValueQueriesTest.java | 89 +-
...terSegmentResultTableMultiValueQueriesTest.java | 352 +-
...erSegmentResultTableSingleValueQueriesTest.java | 397 +--
.../apache/pinot/queries/StUnionQueriesTest.java | 16 +-
.../pinot/queries/SumPrecisionQueriesTest.java | 23 +-
.../pinot/integration/tests/ClusterTest.java | 3 +-
.../MultiNodesOfflineClusterIntegrationTest.java | 49 +-
.../tests/OfflineClusterIntegrationTest.java | 7 +-
.../integration/tests/TlsIntegrationTest.java | 4 +-
.../ingestion/common/DefaultControllerRestApi.java | 15 +-
pinot-plugins/pinot-file-system/pinot-s3/pom.xml | 2 +-
.../BaseMultipleSegmentsConversionExecutor.java | 3 +-
.../tasks/BaseSingleSegmentConversionExecutor.java | 3 +-
.../minion/tasks/SegmentConversionUtils.java | 3 +-
.../SegmentGenerationAndPushTaskGenerator.java | 36 +-
.../pinot-stream-ingestion/pinot-kinesis/pom.xml | 13 +
.../stream/kinesis/server/KinesisDataProducer.java | 104 +
.../kinesis/server/KinesisDataServerStartable.java | 145 +
.../pinot-stream-ingestion/pinot-pulsar/pom.xml | 29 +-
.../converter/RealtimeSegmentConverter.java | 14 -
.../recordtransformer/ComplexTypeTransformer.java | 46 +-
.../segment/local/segment/creator/Fixtures.java | 153 +
.../RecordReaderSegmentCreationDataSource.java | 31 +-
.../local/segment/creator/TransformPipeline.java | 126 +
.../impl/SegmentIndexCreationDriverImpl.java | 46 +-
.../pinot/segment/local/utils/IngestionUtils.java | 8 +-
.../segment/local/utils/SegmentPushUtils.java | 11 +-
.../segment/local/utils/TableConfigUtils.java | 17 +
.../ComplexTypeTransformerTest.java | 97 +-
.../segment/creator/TransformPipelineTest.java | 132 +
.../segment/local/utils/IngestionUtilsTest.java | 25 +
.../segment/local/utils/TableConfigUtilsTest.java | 17 +
.../server/starter/helix/BaseServerStarter.java | 55 +-
.../config/table/ingestion/ComplexTypeConfig.java | 13 +-
.../apache/pinot/spi/utils/CommonConstants.java | 2 +-
.../apache/pinot/tools/GitHubEventsQuickstart.java | 86 +-
.../admin/command/AbstractBaseAdminCommand.java | 6 +-
.../command/GitHubEventsQuickStartCommand.java | 12 +-
.../admin/command/StreamGitHubEventsCommand.java | 41 +-
.../tools/admin/command/UploadSegmentCommand.java | 3 +-
.../PullRequestMergedEventsStream.java | 37 +
.../pinot/tools/utils/KinesisStarterUtils.java | 66 +
.../apache/pinot/tools/utils/StreamSourceType.java | 6 +-
...ergedEvents_kinesis_realtime_table_config.json} | 19 +-
pom.xml | 6 +-
163 files changed, 9321 insertions(+), 2698 deletions(-)
create mode 100644 licenses-binary/LICENSE-argparse4j.txt
copy licenses-binary/{LICENSE-stax2.txt => LICENSE-bsd-2-clause.txt} (91%)
copy licenses-binary/{LICENSE-stax2.txt => LICENSE-bsd-3-clause.txt} (79%)
copy licenses-binary/{LICENSE-bouncycastle.txt => LICENSE-mit.txt} (90%)
copy licenses-binary/{LICENSE-azure.txt => LICENSE-pyrolite.txt} (94%)
create mode 100644
pinot-common/src/main/java/org/apache/pinot/common/utils/http/HttpClient.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/helix/ControllerRequestClient.java
create mode 100644
pinot-controller/src/test/java/org/apache/pinot/controller/helix/core/minion/generator/TaskGeneratorUtilsTest.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/data/manager/offline/DimensionTable.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/query/reduce/GapfillFilterHandler.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/query/reduce/GapfillProcessor.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/query/reduce/RowBasedBlockValSet.java
copy
pinot-tools/src/main/java/org/apache/pinot/tools/scan/query/NotPredicateFilter.java
=>
pinot-core/src/main/java/org/apache/pinot/core/query/reduce/filter/AndRowMatcher.java
(55%)
copy
pinot-common/src/main/java/org/apache/pinot/common/response/broker/ResultTable.java
=>
pinot-core/src/main/java/org/apache/pinot/core/query/reduce/filter/ColumnValueExtractor.java
(57%)
copy
pinot-core/src/{test/java/org/apache/pinot/core/realtime/impl/fakestream/FakeStreamLevelConsumer.java
=>
main/java/org/apache/pinot/core/query/reduce/filter/LiteralValueExtractor.java}
(62%)
copy
pinot-tools/src/main/java/org/apache/pinot/tools/scan/query/InPredicateFilter.java
=>
pinot-core/src/main/java/org/apache/pinot/core/query/reduce/filter/OrRowMatcher.java
(55%)
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/query/reduce/filter/PredicateRowMatcher.java
copy pinot-common/src/main/java/org/apache/pinot/common/tier/TierStorage.java
=>
pinot-core/src/main/java/org/apache/pinot/core/query/reduce/filter/RowMatcher.java
(78%)
copy
pinot-core/src/main/java/org/apache/pinot/core/{segment/processing/aggregator/ValueAggregatorFactory.java
=> query/reduce/filter/RowMatcherFactory.java} (51%)
copy
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/creator/TextIndexCreator.java
=>
pinot-core/src/main/java/org/apache/pinot/core/query/reduce/filter/ValueExtractor.java
(64%)
copy
pinot-core/src/main/java/org/apache/pinot/core/{segment/processing/transformer/RecordTransformer.java
=> query/reduce/filter/ValueExtractorFactory.java} (67%)
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/query/reduce/GapfillFilterHandlerTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/queries/GapfillQueriesTest.java
create mode 100644
pinot-plugins/pinot-stream-ingestion/pinot-kinesis/src/main/java/org/apache/pinot/plugin/stream/kinesis/server/KinesisDataProducer.java
create mode 100644
pinot-plugins/pinot-stream-ingestion/pinot-kinesis/src/main/java/org/apache/pinot/plugin/stream/kinesis/server/KinesisDataServerStartable.java
create mode 100644
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/Fixtures.java
create mode 100644
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/TransformPipeline.java
create mode 100644
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/creator/TransformPipelineTest.java
create mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/utils/KinesisStarterUtils.java
copy
pinot-clients/pinot-jdbc-client/src/main/java/org/apache/pinot/client/controller/request/TableRequest.java
=>
pinot-tools/src/main/java/org/apache/pinot/tools/utils/StreamSourceType.java
(90%)
copy
pinot-tools/src/main/resources/examples/stream/githubEvents/{pullRequestMergedEvents_realtime_table_config.json
=> pullRequestMergedEvents_kinesis_realtime_table_config.json} (51%)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
