This is an automated email from the ASF dual-hosted git repository.
jackie pushed a change to branch rebalance_llc
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
discard 82cbaca Take OFFLINE segment into account for real-time rebalancer
add 9c49b40 Only set Helix cluster properties when creating the cluster
(#4339)
add 43c70c9 Remove jboss and chibi repository from pom file (#4340)
add e604eaf [TE] frontend - harleyjj/preview - remove day from graph
x-axis and set default time range to 48 hours before (#4328)
add 2377037 [TE] frontend - harleyjj/alert-overview - show bounds on
Alert Overview (#4314)
add 84b8143 Add support for an aggregation function returning serialized
hyperloglogs. (#4293)
add 5fc684d [TE] frontend - harleyjj/timeseries-chart - style legend for
current, predicted, and baseline (#4342)
add 0c339d9 [TE] detection health status (#4322)
add 5b34223 [TE] Entity Yaml (Composite Alert) Translator (#4333)
add 6005bd2 [TE] Reset null password to empty (#4343)
add 27c2736 [TE] cleaned up error messages when running in demo mode
(#4344)
add cd823b7 Refresh schema when reloading segments (#4335)
add d64aa98 [TE] Authorize service account to prevent config modification
(#4341)
add 8ed5200 [TE] Resolve runtime jackson conflicts - Upgrade and pin
jackson version (#4347)
add 4d3d02a Within a partition, only allow querying the first CONSUMING
segment for the real-time table routing (#4338)
add 17847db Refresh schema when reloading all segments (#4349)
add 3eef6ea Update quick_start.rst (#4319)
add 9497a75 [TE] Fix doc - add import sql url, disable pinot build (#4357)
add 81f433f Raise the threshold of using MMAP for inverted index creation
from 100M to 2G (#4358)
add 859c68c [TE] Tuning of Composite alert - Store metric refs in
component spec (#4359)
add b75a4c0 Adding support for udfs in Selection (#4346)
add ab6dd9f [TE] Trigger expression based grouping of anomalies - AND, OR
and combinations (#4354)
add 4ce688e [TE] update detection health (#4364)
add 866e29a Adding util methods for controller that will be used in
offline push (#4360)
add 39811df [TE] frontend - harleyjj/alert-overview - dont use anomalies
from bounds endpoint in Alert Overview (#4366)
add 49902ae [TE] set detection window based on detection granularity
(#4352)
add 583548c [TE] Bug Fix - Inject the correct metric ref in component
spec (#4369)
add db7cd63 [TE] Update name for entity anomalies (#4370)
add 89a3449 Add segment pre-processing Hadoop job (#4253)
add cc83174 Disable Hadoop Pre-process (#4375)
add 3be84f2 Include timestamp in logs (#4379)
add 5e2a810 upgrade fastutil to 8.2.3 (#4325)
add 33756ef [TE] Support Anomaly Hierarchy for Composite alerts (#4371)
add 29bad86 [TE] Inject entity name into grouper to enrich the grouped
anomalies (#4378)
add dc63643 Adding support for udf/expression in filter predicates (#4365)
add e28eeb3 [TE] Log the registry components (#4384)
add 33f583b Offheap String Dictionary Benchmark (#4381)
add 732a7b9 Add more key/value pairs into LOG2M_TO_SIZE_IN_BYTES in
HllSizeUtils (#4398)
add 49d8fa7 #4317 Feature/variable length bytes offline dictionary for
indexing bytes and string dicts. (#4321)
add 02f6181 CompletionConfig for realtime tables (#4367)
add ae1ecef [TE] add a thread pool to run preview tasks (#4405)
add de62d8e [TE] Make email template and subject pluggable under email
alert scheme (#4409)
add 7df1a43 [TE] Add Merger after metric grouper; other minor clean up
(#4410)
add e71ac1b [TE] Do not authorize when auth is disabled (#4414)
add 0b8af63 Fix codecov link from old url to new (#4412)
add 07e2c72 Move tests for API resources to a different package (#4415)
add 52f69d1 [TE] Added support for Vertica as a data source (#4404)
add a7c419a Adding Calcite SQL Parser and make another entry point to
query Pinot (#4387)
add e041ec5 [TE] frontend - harleyjj/detection-health - UI for model
performance (#4413)
add 6bec451 [TE] Inject the port from config into Commons mail SSL Smtp
port (#4420)
add dac9ae2 prompt on fail schema (#4395)
add b6fe4e6 [TE] detection health ui adjustments (#4421)
add bca6756 make kafka version number controlled by config number (#4396)
add 0993974 [TE] Fix exception handling - Propagate and display the error
message/exception on frontind (#4419)
add aadcd36 In ClusterChangeMediator, stop enqueue/process changes if
already stopped (#4422)
add 52f9b08 Add pinot community inviter (#4424)
add 2f691fc [TE] detection health - coverage fix (#4428)
add 079b86d Config for overriding initial rows threshold in segment size
auto tuning (#4376)
add 50aaaf9 [TE] added ThirdEye configuration documents (#4429)
add f671072 Add URIUtils class to handle URI/URL encoding/decoding (#4426)
add 626c43a Misc fix for controller tests (#4431)
add 12e77bd #4401 Reuse a ThreadLocal byte[] when reading String elements
from the variable length value reader. (#4432)
add e720448 Remove the massive console print and log from
CalciteSqlCompilerTest (#4434)
add 67afc71 Cleanup un-necessary setups in HelixSetupUtils (#4430)
add b1d28fd Fix potential resource leak in the way we close a collection
of closeables (#4418)
add d2b2981 add download excel (#4403)
add 7b1715c 3891: Check for validity of segment start/end time during
segment generation (#4368)
add 625482a [TE] Renamed some classes to making it accurate and
consistent (#4439)
add 31d916f [TE] Adding Entity GroupKey Anomalies Email Reporter (#4433)
add db5923c Clean up the ServerInstance class and remove the un-necessary
ip address stored (#4438)
add 5492217 Fix the potential resource leak from NettyServer (#4440)
add e897271 [TE] add ml auto config into YAML template (#4423)
add 16f028e Fix ServerInstanceTest (#4442)
add 2e07b0b [TE] Improvements to the Entity GroupKey Template (#4444)
add cd579b1 Add comments and fix typo for the SegmentCompletionTest.
(#4447)
add 85b07f3 [TE] frontend - harleyjj/preview - provide custom time range
before previewing (#4437)
add 365ee85 [TE] Detection and alert config formatters (#4443)
add 9232bb5 Moving handling of NULL values from RecordReaders to
NullValueTransfo… (#4399)
add eaecd64 Refactor pinot-connectors to break the dependencies from
kafka 0.9 (#4445)
add 024da51 Enable SegmentCompleteionIntegrationTest (#4451)
add 25e7be1 [TE] Bug Fix - Parsing of multiple alert schemes (#4456)
add d18fab9 [TE] anomaly filters potential inconsistency issue fix (#4448)
add 2d0b247 [TE] Update Swagger Resource Package (#4458)
add 6dc820a Issue #4450: Support parsing of `Order By` clause in PQL
compiler. (#4453)
add 0fded63 [TE] Inject Sub Entity Name into the anomaly properties
(#4452)
add 523f697 Add resource config for lead controller resource (#4449)
add a1fff55 Ensuring partition config is valid during Hadoop preprocess
(#4463)
add 14ef543 Remove the usage of guava deprecated methods (#4457)
add 1642f00 [TE] logs and precondition checks for pipeline re-tuning
(#4465)
add eca1d2e [TE] frontend - set analysis range using
alertDetailsDefaultWindowSize from endpoint (#4460)
add 1d45f87 Enable periodic rebalance as a temporary work-around for the
Helix issue (#4459)
add cac4f24 Allow customized metrics prefix in pinot
controller/broker/server (#4392)
add c01b852 Add table name parameter for segment push (#4454)
add c81adda [TE] Include alert description in entity email alert (#4471)
add 29bc069 [TE] Include admin recipients in every email alert (cc
address) (#4470)
add 9d64d8a Refactor HelixBrokerStarterTest (#4473)
add 8717e35 Adding Support for Kafka 2.0 Consumer (#4397)
add 551b5c3 Time-aware resizing (#4462)
add 9316610 Clean up test utils, disconnect fake instances (#4476)
add c768df8 When initializing mutable dictionary, preserve 10% buffer for
cardinality to reduce the chance of re-sizing the dictionary (#4363)
add ad6ff6b Delete extra refreshed segments after segment push (#4464)
add 08be56e minor fix double close (#4478)
add 004e4cf Take OFFLINE segment into account for real-time rebalancer
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (82cbaca)
\
N -- N -- N refs/heads/rebalance_llc (004e4cf)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.travis.yml | 6 +
.travis_install.sh | 7 +-
.travis_test.sh | 15 +-
README.md | 4 +-
docs/pluggable_streams.rst | 48 +
docs/pql_examples.rst | 2 +
.../java/org/apache/pinot/client/Connection.java | 44 +-
.../client/JsonAsyncHttpPinotClientTransport.java | 30 +-
.../apache/pinot/client/PinotClientTransport.java | 6 +
.../{PinotClientTransport.java => Request.java} | 34 +-
.../apache/pinot/client/PreparedStatementTest.java | 40 +-
.../apache/pinot/client/ResultSetGroupTest.java | 38 +-
pinot-api/src/test/resources/log4j2.xml | 2 +-
.../pinot/broker/broker/BrokerServerBuilder.java | 3 +-
.../broker/broker/helix/ClusterChangeMediator.java | 12 +-
.../broker/helix/LiveInstanceChangeHandler.java | 6 +-
.../requesthandler/BaseBrokerRequestHandler.java | 28 +-
...enNestedPredicatesFilterQueryTreeOptimizer.java | 3 +-
.../requesthandler/PinotQueryParserFactory.java | 49 +
.../broker/requesthandler/PinotQueryRequest.java | 33 +-
.../pinot/broker/routing/CfgBasedRouting.java | 64 -
.../LowLevelConsumerRoutingTableBuilder.java | 6 +-
.../builder/LowLevelRoutingTableBuilderUtil.java | 83 +-
.../EmptyBrokerOnlineOfflineStateModelFactory.java | 70 -
...EmptySegmentOnlineOfflineStateModelFactory.java | 71 -
.../broker/broker/HelixBrokerStarterTest.java | 357 ++---
.../request/PqlAndCalciteSqlCompatibilityTest.java | 123 ++
.../LowLevelConsumerRoutingTableBuilderTest.java | 74 +-
pinot-broker/src/test/resources/log4j2.xml | 2 +-
pinot-common/pom.xml | 8 +-
.../pinot/common/config/CompletionConfig.java | 63 +
.../apache/pinot/common/config/IndexingConfig.java | 20 +-
.../SegmentsValidationAndRetentionConfig.java | 30 +-
.../apache/pinot/common/config/TableConfig.java | 8 +-
.../common}/function/AggregationFunctionType.java | 7 +-
.../function/FunctionDefinitionRegistry.java} | 22 +-
.../apache/pinot/common/metrics/BrokerMetrics.java | 12 +-
.../pinot/common/metrics/ControllerMetrics.java | 9 +-
.../apache/pinot/common/metrics/ServerMetrics.java | 24 +-
.../pinot/common/request/AggregationInfo.java | 384 +++--
.../apache/pinot/common/request/BrokerRequest.java | 1533 ++++++++++----------
.../apache/pinot/common/request/DataSource.java | 216 ++-
.../apache/pinot/common/request/Expression.java | 421 +++---
.../pinot/common/request/ExpressionType.java | 11 +-
.../pinot/common/request/FilterOperator.java | 18 +-
.../apache/pinot/common/request/FilterQuery.java | 561 ++++---
.../pinot/common/request/FilterQueryMap.java | 260 ++--
.../org/apache/pinot/common/request/Function.java | 339 ++---
.../org/apache/pinot/common/request/GroupBy.java | 420 +++---
.../pinot/common/request/HavingFilterQuery.java | 562 ++++---
.../pinot/common/request/HavingFilterQueryMap.java | 259 ++--
.../apache/pinot/common/request/Identifier.java | 220 ++-
.../pinot/common/request/InstanceRequest.java | 542 ++++---
.../org/apache/pinot/common/request/Literal.java | 364 ++---
.../apache/pinot/common/request/PinotQuery.java | 1048 ++++++-------
.../apache/pinot/common/request/QuerySource.java | 218 ++-
.../org/apache/pinot/common/request/QueryType.java | 485 +++----
.../org/apache/pinot/common/request/Selection.java | 486 +++----
.../apache/pinot/common/request/SelectionSort.java | 285 ++--
.../pinot/common/response/ServerInstance.java | 180 +--
.../apache/pinot/common/utils/CommonConstants.java | 27 +-
.../common/utils/FileUploadDownloadClient.java | 84 +-
.../org/apache/pinot/common/utils/FileUtils.java | 59 +-
.../org/apache/pinot/common/utils/JsonUtils.java | 6 +-
.../org/apache/pinot/common/utils/StringUtil.java | 4 +-
.../org/apache/pinot/common/utils/URIUtils.java | 88 ++
.../common/utils/request/FilterQueryTree.java | 11 +
.../pinot/common/utils/request/RequestUtils.java | 62 +-
.../apache/pinot/common/utils/time/TimeUtils.java | 15 +
.../org/apache/pinot/filesystem/LocalPinotFS.java | 91 +-
.../pinot/{pql => }/parsers/AbstractCompiler.java | 7 +-
.../utils/BrokerRequestComparisonUtils.java | 235 +++
.../parsers/PinotQuery2BrokerRequestConverter.java | 22 +-
.../org/apache/pinot/pql/parsers/Pql2Compiler.java | 63 +-
.../parsers/pql2/ast/BetweenPredicateAstNode.java | 2 +-
.../pql2/ast/ComparisonPredicateAstNode.java | 22 +-
.../pql/parsers/pql2/ast/InPredicateAstNode.java | 2 +-
.../pinot/pql/parsers/pql2/ast/OrderByAstNode.java | 16 +-
.../pql/parsers/pql2/ast/OutputColumnAstNode.java | 22 +-
.../pql/parsers/pql2/ast/PredicateListAstNode.java | 2 +-
.../pql2/ast/RegexpLikePredicateAstNode.java | 2 +-
.../parsers/CalciteSqlCompiler.java} | 18 +-
.../apache/pinot/sql/parsers/CalciteSqlParser.java | 273 ++++
.../parsers/SqlCompilationException.java} | 19 +-
.../apache/pinot/startree/hll/HllSizeUtils.java | 14 +-
.../pinot/common/config/IndexingConfigTest.java | 3 +
.../pinot/common/config/TableConfigTest.java | 28 +
.../apache/pinot/common/utils/URIUtilsTest.java | 68 +
.../apache/pinot/pql/parsers/Pql2CompilerTest.java | 56 +-
.../request/BrokerRequestSerializationTest.java | 10 +-
.../pinot/sql/parsers/CalciteSqlCompilerTest.java | 425 ++++++
pinot-common/src/test/resources/log4j2.xml | 2 +-
pinot-common/src/test/resources/pql_queries.list | 5 +
.../{pql_queries.list => sql_queries.list} | 247 ++--
pinot-common/src/thrift/request.thrift | 1 +
pinot-connectors/pinot-connector-kafka-0.9/pom.xml | 16 +-
.../realtime/impl/kafka/KafkaStarterUtils.java | 171 ---
.../impl/kafka/server/KafkaDataProducer.java | 57 +
.../kafka/server/KafkaDataServerStartable.java | 109 ++
...inot.core.realtime.stream.StreamConsumerFactory | 19 +
.../kafka/KafkaPartitionLevelConsumerTest.java | 2 +-
.../pinot-connector-kafka-2.0/README.md | 57 +
.../pom.xml | 50 +-
.../realtime/impl/kafka2/KafkaConsumerFactory.java | 51 +
.../realtime/impl/kafka2/KafkaMessageBatch.java | 63 +
.../KafkaPartitionLevelConnectionHandler.java | 73 +
.../impl/kafka2/KafkaPartitionLevelConsumer.java | 66 +
.../kafka2/KafkaPartitionLevelStreamConfig.java | 147 ++
.../impl/kafka2/KafkaStreamLevelConsumer.java | 179 +++
.../kafka2/KafkaStreamLevelConsumerManager.java | 194 +++
.../impl/kafka2/KafkaStreamLevelStreamConfig.java | 136 ++
.../impl/kafka2/KafkaStreamMetadataProvider.java | 67 +
.../impl/kafka2/server/KafkaDataProducer.java | 77 +
.../kafka2/server/KafkaDataServerStartable.java | 98 ++
...inot.core.realtime.stream.StreamConsumerFactory | 19 +
.../kafka2/KafkaPartitionLevelConsumerTest.java | 293 ++++
.../KafkaPartitionLevelStreamConfigTest.java | 162 +++
.../impl/kafka2/utils/EmbeddedZooKeeper.java | 60 +
.../impl/kafka2/utils/MiniKafkaCluster.java | 175 +++
.../pinot-connector-kafka-base/README.md | 18 +-
.../{ => pinot-connector-kafka-base}/pom.xml | 43 +-
.../impl/kafka/KafkaAvroMessageDecoder.java | 0
.../impl/kafka/KafkaJSONMessageDecoder.java | 0
.../realtime/impl/kafka/KafkaStarterUtils.java | 118 ++
.../impl/kafka/KafkaStreamConfigProperties.java | 19 +-
.../core/realtime/impl/kafka/MessageAndOffset.java | 39 +-
pinot-connectors/pom.xml | 15 +
.../apache/pinot/controller/ControllerConf.java | 53 +-
.../apache/pinot/controller/ControllerStarter.java | 22 +-
.../api/resources/FileUploadPathProvider.java | 5 +-
.../resources/LLCSegmentCompletionHandlers.java | 25 +-
.../api/resources/PinotControllerHealthCheck.java | 4 +-
.../api/resources/PinotSegmentRestletResource.java | 20 +-
.../PinotSegmentUploadRestletResource.java | 89 +-
.../resources/PinotTableConfigRestletResource.java | 2 +
.../api/resources/PinotTableRestletResource.java | 10 +-
.../pinot/controller/api/resources/TableViews.java | 4 +-
.../controller/api/upload/SegmentValidator.java | 2 +-
.../helix/ControllerRequestBuilderUtil.java | 152 --
.../helix/ControllerRequestURLBuilder.java | 21 +-
.../EmptyBrokerOnlineOfflineStateModelFactory.java | 70 -
...EmptySegmentOnlineOfflineStateModelFactory.java | 81 --
.../helix/core/PinotHelixResourceManager.java | 34 +-
.../helix/core/PinotTableIdealStateBuilder.java | 24 -
.../helix/core/SegmentDeletionManager.java | 34 +-
...ealtimeBalanceNumSegmentAssignmentStrategy.java | 3 +-
...ltimeReplicaGroupSegmentAssignmentStrategy.java | 3 +-
.../realtime/PinotLLCRealtimeSegmentManager.java | 34 +-
.../segment/FlushThresholdUpdateManager.java | 11 +-
.../SegmentSizeBasedFlushThresholdUpdater.java | 14 +-
.../helix/core/util/HelixSetupUtils.java | 293 ++--
.../controller/validation/StorageQuotaChecker.java | 32 +-
.../src/main/resources/static/css/pinot.css | 16 +-
.../src/main/resources/static/js/init.js | 12 +
.../src/main/resources/static/query/index.html | 8 +-
.../api/{resources => }/AccessControlTest.java | 10 +-
.../FileUploadPathProviderTest.java | 3 +-
.../api/{resources => }/PinotFileUploadTest.java | 41 +-
.../PinotInstanceRestletResourceTest.java | 6 +-
.../PinotSchemaRestletResourceTest.java | 2 +-
.../PinotSegmentRestletResourceTest.java | 31 +-
.../PinotTableRestletResourceTest.java | 13 +-
.../PinotTenantRestletResourceTest.java | 14 +-
.../SegmentCompletionProtocolDeserTest.java | 2 +-
.../SegmentCompletionUtilsTest.java | 2 +-
.../{resources => }/ServerTableSizeReaderTest.java | 3 +-
.../api/{resources => }/TableSizeReaderTest.java | 2 +-
.../api/{resources => }/TableViewsTest.java | 15 +-
.../helix/ControllerInstanceToggleTest.java | 33 +-
.../helix/ControllerPeriodicTaskStarterTest.java | 38 +-
.../controller/helix/ControllerSentinelTestV2.java | 45 +-
.../controller/helix/ControllerTenantTest.java | 111 +-
.../pinot/controller/helix/ControllerTest.java | 336 ++++-
.../pinot/controller/helix/HelixHelperTest.java | 6 +-
.../controller/helix/PinotControllerModeTest.java | 275 ++--
.../controller/helix/PinotResourceManagerTest.java | 43 +-
.../helix/core/PinotHelixResourceManagerTest.java | 139 +-
.../helix/core/realtime/SegmentCompletionTest.java | 41 +-
.../segment/FlushThresholdUpdaterTest.java | 28 +-
.../ReplicaGroupRebalanceStrategyTest.java | 33 +-
.../sharding/SegmentAssignmentStrategyTest.java | 133 +-
.../validation/ValidationManagerTest.java | 49 +-
pinot-controller/src/test/resources/log4j2.xml | 2 +-
pinot-core/pom.xml | 4 -
.../core/data/aggregator/AvgValueAggregator.java | 2 +-
.../core/data/aggregator/CountValueAggregator.java | 2 +-
.../DistinctCountHLLValueAggregator.java | 2 +-
.../core/data/aggregator/MaxValueAggregator.java | 2 +-
.../aggregator/MinMaxRangeValueAggregator.java | 2 +-
.../core/data/aggregator/MinValueAggregator.java | 2 +-
.../aggregator/PercentileEstValueAggregator.java | 2 +-
.../PercentileTDigestValueAggregator.java | 2 +-
.../core/data/aggregator/SumValueAggregator.java | 2 +-
.../core/data/aggregator/ValueAggregator.java | 2 +-
.../data/aggregator/ValueAggregatorFactory.java | 4 +-
.../realtime/HLRealtimeSegmentDataManager.java | 9 +-
.../realtime/LLRealtimeSegmentDataManager.java | 95 +-
.../data/partition/MurmurPartitionFunction.java | 2 +-
.../pinot/core/data/readers/RecordReaderUtils.java | 14 +-
.../core/data/readers/sort/PinotSegmentSorter.java | 75 +-
...dTransformer.java => CompositeTransformer.java} | 20 +-
.../recordtransformer/NullValueTransformer.java | 53 +
...ansformer.java => SanitizationTransformer.java} | 6 +-
.../generator/SegmentGeneratorConfig.java | 23 +
.../indexsegment/mutable/MutableSegmentImpl.java | 18 +-
.../impl/FixedByteSingleValueMultiColReader.java | 16 +-
.../core/io/util/FixedBitIntReaderWriter.java | 16 +-
.../core/io/util/FixedByteValueReaderWriter.java | 16 +-
.../org/apache/pinot/core/io/util/ValueReader.java | 3 +-
.../io/util/VarLengthBytesValueReaderWriter.java | 278 ++++
.../impl/FixedByteSingleValueMultiColWriter.java | 23 +-
.../pinot/core/minion/BackfillDateTimeColumn.java | 4 +-
.../apache/pinot/core/minion/SegmentConverter.java | 18 +-
.../pinot/core/operator/blocks/TransformBlock.java | 4 +
.../RangelessBitmapDocIdIterator.java | 2 +-
.../dociditerators/SVScanDocIdIterator.java | 8 +-
.../operator/filter/ExpressionFilterOperator.java | 381 +++++
.../core/operator/filter/FilterOperatorUtils.java | 3 +
.../predicate/PredicateEvaluatorProvider.java | 7 +-
.../query/DictionaryBasedAggregationOperator.java | 2 +-
.../query/MetadataBasedAggregationOperator.java | 2 +-
.../core/operator/query/SelectionOnlyOperator.java | 109 --
.../core/operator/query/SelectionOperator.java | 282 ++++
.../operator/query/SelectionOrderByOperator.java | 116 --
.../transform/TransformBlockDataFetcher.java | 340 +++++
.../core/operator/transform/TransformOperator.java | 13 +-
.../org/apache/pinot/core/plan/FilterPlanNode.java | 24 +-
.../plan/MetadataBasedAggregationPlanNode.java | 2 +-
.../apache/pinot/core/plan/SelectionPlanNode.java | 38 +-
.../apache/pinot/core/plan/TransformPlanNode.java | 50 +-
.../core/plan/maker/InstancePlanMakerImplV2.java | 2 +-
.../aggregation/DefaultAggregationExecutor.java | 2 +-
.../aggregation/function/AggregationFunction.java | 1 +
.../function/AggregationFunctionFactory.java | 5 +
.../function/AggregationFunctionUtils.java | 1 +
.../function/AvgAggregationFunction.java | 1 +
.../function/AvgMVAggregationFunction.java | 1 +
.../function/CountAggregationFunction.java | 1 +
.../function/CountMVAggregationFunction.java | 1 +
.../function/DistinctCountAggregationFunction.java | 1 +
.../DistinctCountHLLAggregationFunction.java | 1 +
.../DistinctCountHLLMVAggregationFunction.java | 1 +
.../DistinctCountMVAggregationFunction.java | 1 +
...=> DistinctCountRawHLLAggregationFunction.java} | 63 +-
... DistinctCountRawHLLMVAggregationFunction.java} | 30 +-
.../function/FastHLLAggregationFunction.java | 1 +
.../function/MaxAggregationFunction.java | 1 +
.../function/MaxMVAggregationFunction.java | 1 +
.../function/MinAggregationFunction.java | 1 +
.../function/MinMVAggregationFunction.java | 1 +
.../function/MinMaxRangeAggregationFunction.java | 1 +
.../function/MinMaxRangeMVAggregationFunction.java | 1 +
.../function/PercentileAggregationFunction.java | 1 +
.../function/PercentileEstAggregationFunction.java | 1 +
.../PercentileEstMVAggregationFunction.java | 1 +
.../function/PercentileMVAggregationFunction.java | 1 +
.../PercentileTDigestAggregationFunction.java | 1 +
.../PercentileTDigestMVAggregationFunction.java | 1 +
.../function/SumAggregationFunction.java | 1 +
.../function/SumMVAggregationFunction.java | 1 +
.../function/customobject/SerializedHLL.java | 37 +-
.../groupby/DefaultGroupByExecutor.java | 2 +-
.../aggregation/groupby/utils/DoubleToIdMap.java | 2 +-
.../aggregation/groupby/utils/FloatToIdMap.java | 2 +-
.../aggregation/groupby/utils/IntToIdMap.java | 2 +-
.../aggregation/groupby/utils/LongToIdMap.java | 2 +-
.../query/pruner/ColumnValueSegmentPruner.java | 5 +
.../core/query/pruner/PartitionSegmentPruner.java | 5 +
.../core/query/request/ServerQueryRequest.java | 18 +-
.../query/selection/SelectionOperatorUtils.java | 4 +-
.../converter/RealtimeSegmentConverter.java | 21 +-
.../core/realtime/impl/RealtimeSegmentConfig.java | 20 +-
.../pinot/core/realtime/stream/StreamConfig.java | 55 +-
.../realtime/stream/StreamConfigProperties.java | 8 +-
.../core/realtime/stream/StreamDataProducer.java | 18 +-
.../core/realtime/stream/StreamDataProvider.java | 46 +
.../stream/StreamDataServerStartable.java} | 52 +-
.../segment/creator/ColumnIndexCreationInfo.java | 10 +-
.../RecordReaderSegmentCreationDataSource.java | 4 +-
.../creator/impl/SegmentColumnarIndexCreator.java | 147 +-
.../creator/impl/SegmentDictionaryCreator.java | 72 +-
.../impl/SegmentIndexCreationDriverImpl.java | 10 +-
.../fwd/SingleValueSortedForwardIndexCreator.java | 10 +-
.../inv/OffHeapBitmapInvertedIndexCreator.java | 30 +-
.../segment/index/loader/IndexLoadingConfig.java | 11 +
.../defaultcolumn/BaseDefaultColumnHandler.java | 8 +-
.../index/readers/ImmutableDictionaryReader.java | 18 +-
.../name/NormalizedDateSegmentNameGenerator.java | 18 +-
.../segment/store/SegmentLocalFSDirectory.java | 2 +-
.../pinot/core/startree/StarTreeDataTable.java | 35 +-
.../executor/StarTreeAggregationExecutor.java | 2 +-
.../startree/executor/StarTreeGroupByExecutor.java | 2 +-
.../startree/operator/StarTreeFilterOperator.java | 5 +-
.../startree/plan/StarTreeTransformPlanNode.java | 3 +-
.../startree/v2/AggregationFunctionColumnPair.java | 2 +-
.../startree/v2/builder/BaseSingleTreeBuilder.java | 2 +-
.../v2/builder/OffHeapSingleTreeBuilder.java | 53 +-
.../startree/v2/store/StarTreeLoaderUtils.java | 2 +-
.../function/FunctionExpressionEvaluatorTest.java | 1 -
.../pinot/core/data/readers/PinotSegmentUtil.java | 74 +-
.../data/readers/RecordReaderSampleDataTest.java | 8 +-
.../pinot/core/data/readers/RecordReaderTest.java | 8 +-
.../recordtransformer/RecordTransformerTest.java | 6 +-
.../MutableSegmentImplAggregateMetricsTest.java | 1 +
.../mutable/MutableSegmentImplTest.java | 3 +-
.../mutable/MutableSegmentImplTestUtils.java | 4 +-
.../util/VarLengthBytesValueReaderWriterTest.java | 170 +++
.../pinot/core/minion/SegmentConverterTest.java | 16 +-
...adataAndDictionaryAggregationPlanMakerTest.java | 12 +
.../function/AggregationFunctionFactoryTest.java | 11 +
.../function/AggregationFunctionTypeTest.java | 5 +
.../core/segment/index/ColumnMetadataTest.java | 6 +
.../segment/index/SegmentMetadataImplTest.java | 6 +
.../SegmentV1V2ToV3FormatConverterTest.java | 6 +
.../SegmentGenerationWithTimeColumnTest.java | 36 +-
.../index/loader/SegmentPreProcessorTest.java | 6 +
.../startree/StarTreeIndexTestSegmentHelper.java | 6 +
.../pinot/core/startree/TestStarTreeMetadata.java | 6 +
.../pinot/core/startree/hll/HllFieldSizeTest.java | 2 +-
.../hll/SegmentWithHllIndexCreateHelper.java | 6 +
.../v2/AggregationFunctionColumnPairTest.java | 11 +-
.../pinot/core/startree/v2/BaseStarTreeV2Test.java | 2 +-
.../org/apache/pinot/core/util/CrcUtilsTest.java | 6 +
.../pinot/queries/BaseMultiValueQueriesTest.java | 6 +
.../pinot/queries/BaseSingleValueQueriesTest.java | 6 +
.../apache/pinot/queries/FastHllQueriesTest.java | 10 +-
...InnerSegmentSelectionMultiValueQueriesTest.java | 123 +-
...nnerSegmentSelectionSingleValueQueriesTest.java | 146 +-
...terSegmentAggregationMultiValueQueriesTest.java | 29 +-
...erSegmentAggregationSingleValueQueriesTest.java | 33 +-
.../org/apache/pinot/queries/QueriesTestUtils.java | 18 +-
.../DefaultAggregationExecutorTest.java | 2 +-
.../segments/v1/creator/DictionariesTest.java | 6 +
.../pinot/segments/v1/creator/IntArraysTest.java | 6 +
.../util/IntDoubleIndexedPriorityQueueTest.java | 2 +-
.../test/java/org/apache/pinot/util/TestUtils.java | 96 +-
pinot-core/src/test/resources/log4j2.xml | 2 +-
pinot-distribution/pinot-assembly.xml | 5 +
pinot-hadoop/pom.xml | 5 +
.../pinot/hadoop/PinotHadoopJobLauncher.java | 7 +-
.../pinot/hadoop/io/CombineAvroKeyInputFormat.java | 57 +
.../apache/pinot/hadoop/job/ControllerRestApi.java | 4 +
.../pinot/hadoop/job/DefaultControllerRestApi.java | 54 +-
.../pinot/hadoop/job/InternalConfigConstants.java | 39 +
.../pinot/hadoop/job/JobConfigConstants.java | 9 +
.../pinot/hadoop/job/SegmentCreationJob.java | 33 +-
.../pinot/hadoop/job/SegmentPreprocessingJob.java | 524 +++++++
.../apache/pinot/hadoop/job/SegmentTarPushJob.java | 61 +-
.../apache/pinot/hadoop/job/SegmentUriPushJob.java | 4 +-
.../{mapper => mappers}/SegmentCreationMapper.java | 4 +-
.../job/mappers/SegmentPreprocessingMapper.java | 124 ++
.../job/partitioners/GenericPartitioner.java | 65 +
.../job/partitioners/PartitionFunctionFactory.java | 98 ++
.../job/reducers/SegmentPreprocessingReducer.java | 84 ++
.../pinot/hadoop/utils/JobPreparationHelper.java | 58 +
.../hadoop/io/DeleteExtraPushedSegmentsTest.java | 102 ++
pinot-integration-tests/pom.xml | 4 +-
.../tests/BaseClusterIntegrationTest.java | 81 +-
.../tests/BaseClusterIntegrationTestSet.java | 111 +-
.../tests/ClusterIntegrationTestUtils.java | 84 +-
.../pinot/integration/tests/ClusterTest.java | 63 +-
.../ControllerPeriodicTasksIntegrationTests.java | 89 +-
.../DeleteAPIHybridClusterIntegrationTest.java | 49 +-
...lakyConsumerRealtimeClusterIntegrationTest.java | 14 +-
...mentBuildPushOfflineClusterIntegrationTest.java | 80 +-
.../tests/HybridClusterIntegrationTest.java | 11 +-
...ridClusterIntegrationTestCommandLineRunner.java | 18 +-
...onaryAggregationPlanClusterIntegrationTest.java | 2 +-
.../MultiNodesOfflineClusterIntegrationTest.java | 7 +
.../tests/OfflineClusterIntegrationTest.java | 172 ++-
.../tests/PinotURIUploadIntegrationTest.java | 15 +-
.../tests/RealtimeClusterIntegrationTest.java | 33 +-
....java => SegmentCompletionIntegrationTest.java} | 70 +-
.../tests/SimpleMinionClusterIntegrationTest.java | 6 +-
.../tests/StarTreeClusterIntegrationTest.java | 2 +-
.../tests/StarTreeV2ClusterIntegrationTest.java | 4 +-
.../tests/UploadRefreshDeleteIntegrationTest.java | 2 +-
.../src/test/resources/log4j2.xml | 2 +-
.../org/apache/pinot/minion/MinionStarter.java | 4 +-
.../BaseMultipleSegmentsConversionExecutor.java | 17 +-
.../BaseSingleSegmentConversionExecutor.java | 13 +-
.../apache/pinot/minion/metrics/MinionMetrics.java | 6 +-
pinot-minion/src/test/resources/log4j2.xml | 2 +-
pinot-perf/pom.xml | 8 +-
.../pinot/perf/BenchmarkDictionaryCreation.java | 12 +
.../apache/pinot/perf/BenchmarkQueryEngine.java | 1 -
.../perf/BenchmarkRealtimeConsumptionSpeed.java | 8 +-
.../pinot/perf/BenchmarkStringDictionary.java | 118 +-
.../org/apache/pinot/perf/RealtimeStressTest.java | 8 +-
.../pinot/perf/StringDictionaryPerfTest.java | 86 +-
.../org/apache/pinot/server/conf/ServerConf.java | 7 +
.../server/request/ScheduledRequestHandler.java | 18 +-
.../apache/pinot/server/starter/ServerBuilder.java | 4 +-
.../starter/helix/HelixInstanceDataManager.java | 12 +-
.../server/starter/helix/HelixServerStarter.java | 6 +-
.../api/{resources => }/BaseResourceTest.java | 2 +-
.../api/{resources => }/TableSizeResourceTest.java | 2 +-
.../api/{resources => }/TablesResourceTest.java | 2 +-
.../request/ScheduledRequestHandlerTest.java | 42 +-
pinot-tools/pom.xml | 34 +-
.../org/apache/pinot/tools/HybridQuickstart.java | 39 +-
.../org/apache/pinot/tools/RealtimeQuickStart.java | 29 +-
.../command/BackfillDateTimeColumnCommand.java | 2 +-
.../tools/admin/command/StartKafkaCommand.java | 11 +-
.../admin/command/StreamAvroIntoKafkaCommand.java | 20 +-
.../tools/admin/command/UploadSegmentCommand.java | 6 +-
.../pinot/tools/backfill/BackfillSegmentUtils.java | 7 +-
.../pinot/tools/perf/PerfBenchmarkDriver.java | 19 -
.../pinot/tools/perf/PerfBenchmarkDriverConf.java | 63 -
.../tools/query/comparison/ClusterStarter.java | 4 +-
.../realtime/provisioning/MemoryEstimator.java | 18 +-
.../pinot/tools/streams/AirlineDataStream.java | 17 +-
.../pinot/tools/streams/MeetupRsvpStream.java | 28 +-
.../airlineStats_realtime_table_config.json | 0
.../meetupRsvp_realtime_table_config.json | 2 +-
.../airlineStats_realtime_table_config.json | 38 +
.../meetupRsvp_realtime_table_config.json | 5 +-
.../meetupRsvp_realtime_table_config.json | 2 +-
.../apache/pinot/transport/netty/NettyServer.java | 59 +-
.../pinot/transport/common/ServerInstanceTest.java | 62 +-
.../pinot/transport/netty/NettyTestUtils.java | 8 +-
.../transport/perf/ScatterGatherPerfClient.java | 5 +-
.../transport/perf/ScatterGatherPerfServer.java | 8 +-
.../pool/AsyncPoolResourceManagerAdapterTest.java | 14 +-
.../pinot/transport/pool/KeyedPoolImplTest.java | 25 +-
.../transport/scattergather/ScatterGatherTest.java | 24 +-
pom.xml | 138 +-
thirdeye/README.md | 3 +
thirdeye/docs/configuration.rst | 210 +++
thirdeye/docs/import_sql_metric.rst | 7 +-
thirdeye/docs/index.rst | 2 +-
thirdeye/docs/production.rst | 27 -
thirdeye/docs/quick_start.rst | 18 +-
thirdeye/install.sh | 10 +-
thirdeye/pom.xml | 19 +-
.../app/pods/components/alert-details/component.js | 97 +-
.../app/pods/components/alert-details/template.hbs | 105 +-
.../pods/components/anomaly-summary/component.js | 2 +-
.../pods/components/detection-health/component.js | 197 +++
.../pods/components/detection-health/template.hbs | 84 ++
.../pods/components/rootcause-chart/component.js | 2 +-
.../app/pods/components/stats-cards/component.js | 42 +-
.../app/pods/components/stats-cards/template.hbs | 61 +-
.../pods/components/timeseries-chart/component.js | 12 +-
.../app/pods/components/yaml-editor/template.hbs | 2 +-
.../app/pods/manage/explore/route.js | 52 +-
.../app/pods/manage/explore/template.hbs | 3 +
.../app/pods/manage/yaml/route.js | 50 +-
.../app/pods/manage/yaml/template.hbs | 4 +-
.../app/pods/screenshot/controller.js | 2 +-
.../app/pods/self-serve/create-alert/template.hbs | 2 +-
.../app/styles/components/timeseries-chart.scss | 16 +-
.../app/styles/shared/_styles.scss | 118 ++
thirdeye/thirdeye-frontend/app/utils/anomaly.js | 22 +-
.../thirdeye-frontend/app/utils/api/anomaly.js | 18 +-
thirdeye/thirdeye-frontend/app/utils/constants.js | 1 +
.../app/utils/manage-alert-utils.js | 4 +-
thirdeye/thirdeye-frontend/app/utils/utils.js | 14 +-
.../pods/components/yaml-editor/component-test.js | 2 +-
thirdeye/thirdeye-pinot/config/dashboard.yml | 3 +-
.../config/data-sources/data-sources-config.yml | 2 +-
thirdeye/thirdeye-pinot/config/h2db.mv.db | Bin 0 -> 4947968 bytes
thirdeye/thirdeye-pinot/pom.xml | 15 +
.../alert/content/BaseEmailContentFormatter.java | 95 +-
.../content/EntityGroupKeyContentFormatter.java | 151 ++
...HierarchicalAnomaliesEmailContentFormatter.java | 5 +-
...a => MetricAnomaliesEmailContentFormatter.java} | 17 +-
...nboardingNotificationEmailContentFormatter.java | 1 +
.../anomaly/alert/util/AnomalyReportGenerator.java | 2 +-
.../anomaly/alert/v2/AlertTaskRunnerV2.java | 2 +-
.../anomaly/merge/AnomalyTimeBasedSummarizer.java | 2 +
.../anomaly/utils/ThirdeyeMetricsUtil.java | 3 +
.../dashboard/DetectionPreviewConfiguration.java | 50 +
.../dashboard/ThirdEyeDashboardApplication.java | 2 +-
.../dashboard/ThirdEyeDashboardConfiguration.java | 9 +
.../dashboard/resources/v2/AnomaliesResource.java | 24 +-
.../dashboard/resources/v2/pojo/SearchFilters.java | 2 -
.../bao/jdbc/MergedAnomalyResultManagerImpl.java | 104 +-
.../datalayer/pojo/DetectionAlertConfigBean.java | 9 +
.../datalayer/pojo/DetectionConfigBean.java | 9 +
.../datasource/mock/AutoOnboardMockDataSource.java | 24 +-
.../datasource/sql/SqlResponseCacheLoader.java | 49 +-
.../pinot/thirdeye/datasource/sql/SqlUtils.java | 3 +
.../thirdeye/detection/DefaultDataProvider.java | 2 +-
.../thirdeye/detection/DetectionPipeline.java | 37 +-
.../detection/DetectionPipelineLoader.java | 10 +-
.../detection/DetectionPipelineTaskRunner.java | 11 +-
.../thirdeye/detection/DetectionResource.java | 52 +-
.../pinot/thirdeye/detection/DetectionUtils.java | 48 +
.../pinot/thirdeye/detection/ModelRetuneFlow.java | 13 +-
.../detection/alert/DetectionAlertTaskFactory.java | 13 +-
.../alert/scheme/DetectionEmailAlerter.java | 89 +-
.../annotation/registry/DetectionRegistry.java | 5 +
.../AbsoluteChangeRuleAnomalyFilter.java | 3 +-
.../thirdeye/detection/components/MockGrouper.java | 0
.../PercentageChangeRuleAnomalyFilter.java | 10 +-
.../SitewideImpactRuleAnomalyFilter.java | 5 +-
.../components/ThresholdRuleAnomalyFilter.java | 14 +-
.../components/TriggerConditionGrouper.java | 203 +++
.../detection/health/AnomalyCoverageStatus.java | 73 +
.../thirdeye/detection/health/DetectionHealth.java | 263 ++++
.../detection/health/DetectionTaskStatus.java | 127 ++
.../thirdeye/detection/health/HealthStatus.java | 28 +
.../detection/health/RegressionStatus.java | 101 ++
.../thirdeye/detection/spec/MockGrouperSpec.java | 0
.../spec/TriggerConditionGrouperSpec.java} | 40 +-
.../validators/DetectionConfigValidator.java | 23 +-
.../detection/wrapper/AnomalyDetectorWrapper.java | 6 +
.../wrapper/ChildKeepingMergeWrapper.java | 16 +-
.../thirdeye/detection/wrapper/GrouperWrapper.java | 39 +-
.../detection/yaml/DetectionConfigTuner.java | 41 +-
.../thirdeye/detection/yaml/YamlResource.java | 154 +-
.../yaml/translator/ConfigTranslator.java | 13 +
.../yaml/translator/DetectionConfigTranslator.java | 232 ++-
.../translator/DetectionMetricAttributeHolder.java | 97 ++
.../yaml/translator/DetectionMetricProperties.java | 62 +
.../translator/SubscriptionConfigTranslator.java | 14 +-
.../pinot/thirdeye/formatter/DTOFormatter.java | 38 +
.../formatter/DetectionAlertConfigFormatter.java | 46 +
.../formatter/DetectionConfigFormatter.java | 189 +++
.../apache/pinot/thirdeye/util/ThirdEyeUtils.java | 5 +-
.../detector/entity-groupkey-anomaly-report.ftl | 99 ++
...ly-report.ftl => metric-anomalies-template.ftl} | 0
.../commons/TestEmailContentFormatterFactory.java | 6 +-
.../alert/content/ContentFormatterUtils.java | 48 +
...> TestEntityGroupKeyEmailContentFormatter.java} | 138 +-
...HierarchicalAnomaliesEmailContentFormatter.java | 22 +-
... TestMetricAnomaliesEmailContentFormatter.java} | 26 +-
...TestOnboardingNotificationContentFormatter.java | 20 +-
.../pinot/thirdeye/datalayer/DaoTestUtils.java | 13 +
.../bao/TestGroupedAnomalyResultsManager.java | 6 +-
.../bao/TestMergedAnomalyResultManager.java | 46 +-
.../pinot/thirdeye/detection/DataProviderTest.java | 2 +
.../thirdeye/detection/DetectionTestUtils.java | 9 +
.../thirdeye/detection/alert/SendAlertTest.java | 2 +-
.../AbsoluteChangeRuleAnomalyFilterTest.java | 25 +-
.../PercentageChangeRuleAnomalyFilterTest.java | 60 +-
.../SitewideImpactRuleAnomalyFilterTest.java | 20 +-
.../components/ThresholdRuleAnomalyFilterTest.java | 56 +-
.../components/TriggerConditionGrouperTest.java | 231 +++
.../detection/health/DetectionHealthTest.java | 130 ++
.../wrapper/AnomalyDetectorWrapperTest.java | 4 +-
.../thirdeye/detection/yaml/YamlResourceTest.java | 27 +-
.../translator/DetectionConfigTranslatorTest.java | 18 +
.../formatter/DetectionConfigFormatterTest.java | 82 ++
.../tools/RunAdhocDatabaseQueriesTool.java | 31 +-
.../validators/entity-pipeline-config-2.yaml | 2 +-
.../detection/yaml/alertconfig/alert-config-5.yaml | 3 +
.../compositePipelineTranslatorTestResult-1.json | 93 +-
.../compositePipelineTranslatorTestResult-2.json | 1 +
.../compositePipelineTranslatorTestResult-4.json | 32 +
.../compositePipelineTranslatorTestResult-5.json | 87 ++
.../yaml/translator/pipeline-config-4.yaml | 26 +
.../translator/pipeline-config-5.yaml} | 17 +-
...st-entity-groupby-email-content-formatter.html} | 84 +-
...st-hierarchical-metric-anomalies-template.html} | 0
...er.html => test-metric-anomalies-template.html} | 0
557 files changed, 20012 insertions(+), 9976 deletions(-)
copy
pinot-api/src/main/java/org/apache/pinot/client/{PinotClientTransport.java =>
Request.java} (61%)
create mode 100644
pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/PinotQueryParserFactory.java
copy
pinot-hadoop/src/main/java/org/apache/pinot/hadoop/job/ControllerRestApi.java
=>
pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/PinotQueryRequest.java
(60%)
delete mode 100644
pinot-broker/src/main/java/org/apache/pinot/broker/routing/CfgBasedRouting.java
delete mode 100644
pinot-broker/src/test/java/org/apache/pinot/broker/broker/EmptyBrokerOnlineOfflineStateModelFactory.java
delete mode 100644
pinot-broker/src/test/java/org/apache/pinot/broker/broker/EmptySegmentOnlineOfflineStateModelFactory.java
create mode 100644
pinot-broker/src/test/java/org/apache/pinot/broker/request/PqlAndCalciteSqlCompatibilityTest.java
create mode 100644
pinot-common/src/main/java/org/apache/pinot/common/config/CompletionConfig.java
rename {pinot-core/src/main/java/org/apache/pinot/core/query/aggregation =>
pinot-common/src/main/java/org/apache/pinot/common}/function/AggregationFunctionType.java
(93%)
copy
pinot-common/src/main/java/org/apache/pinot/{pql/parsers/AbstractCompiler.java
=> common/function/FunctionDefinitionRegistry.java} (69%)
create mode 100644
pinot-common/src/main/java/org/apache/pinot/common/utils/URIUtils.java
copy pinot-common/src/main/java/org/apache/pinot/{pql =>
}/parsers/AbstractCompiler.java (81%)
create mode 100644
pinot-common/src/main/java/org/apache/pinot/parsers/utils/BrokerRequestComparisonUtils.java
copy
pinot-common/src/main/java/org/apache/pinot/{pql/parsers/AbstractCompiler.java
=> sql/parsers/CalciteSqlCompiler.java} (61%)
create mode 100644
pinot-common/src/main/java/org/apache/pinot/sql/parsers/CalciteSqlParser.java
rename
pinot-common/src/main/java/org/apache/pinot/{pql/parsers/AbstractCompiler.java
=> sql/parsers/SqlCompilationException.java} (69%)
create mode 100644
pinot-common/src/test/java/org/apache/pinot/common/utils/URIUtilsTest.java
create mode 100644
pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
copy pinot-common/src/test/resources/{pql_queries.list => sql_queries.list}
(87%)
delete mode 100644
pinot-connectors/pinot-connector-kafka-0.9/src/main/java/org/apache/pinot/core/realtime/impl/kafka/KafkaStarterUtils.java
create mode 100644
pinot-connectors/pinot-connector-kafka-0.9/src/main/java/org/apache/pinot/core/realtime/impl/kafka/server/KafkaDataProducer.java
create mode 100644
pinot-connectors/pinot-connector-kafka-0.9/src/main/java/org/apache/pinot/core/realtime/impl/kafka/server/KafkaDataServerStartable.java
create mode 100644
pinot-connectors/pinot-connector-kafka-0.9/src/main/resources/META-INF/services/org.apache.pinot.core.realtime.stream.StreamConsumerFactory
create mode 100644 pinot-connectors/pinot-connector-kafka-2.0/README.md
copy pinot-connectors/{pinot-connector-kafka-0.9 =>
pinot-connector-kafka-2.0}/pom.xml (59%)
create mode 100644
pinot-connectors/pinot-connector-kafka-2.0/src/main/java/org/apache/pinot/core/realtime/impl/kafka2/KafkaConsumerFactory.java
create mode 100644
pinot-connectors/pinot-connector-kafka-2.0/src/main/java/org/apache/pinot/core/realtime/impl/kafka2/KafkaMessageBatch.java
create mode 100644
pinot-connectors/pinot-connector-kafka-2.0/src/main/java/org/apache/pinot/core/realtime/impl/kafka2/KafkaPartitionLevelConnectionHandler.java
create mode 100644
pinot-connectors/pinot-connector-kafka-2.0/src/main/java/org/apache/pinot/core/realtime/impl/kafka2/KafkaPartitionLevelConsumer.java
create mode 100644
pinot-connectors/pinot-connector-kafka-2.0/src/main/java/org/apache/pinot/core/realtime/impl/kafka2/KafkaPartitionLevelStreamConfig.java
create mode 100644
pinot-connectors/pinot-connector-kafka-2.0/src/main/java/org/apache/pinot/core/realtime/impl/kafka2/KafkaStreamLevelConsumer.java
create mode 100644
pinot-connectors/pinot-connector-kafka-2.0/src/main/java/org/apache/pinot/core/realtime/impl/kafka2/KafkaStreamLevelConsumerManager.java
create mode 100644
pinot-connectors/pinot-connector-kafka-2.0/src/main/java/org/apache/pinot/core/realtime/impl/kafka2/KafkaStreamLevelStreamConfig.java
create mode 100644
pinot-connectors/pinot-connector-kafka-2.0/src/main/java/org/apache/pinot/core/realtime/impl/kafka2/KafkaStreamMetadataProvider.java
create mode 100644
pinot-connectors/pinot-connector-kafka-2.0/src/main/java/org/apache/pinot/core/realtime/impl/kafka2/server/KafkaDataProducer.java
create mode 100644
pinot-connectors/pinot-connector-kafka-2.0/src/main/java/org/apache/pinot/core/realtime/impl/kafka2/server/KafkaDataServerStartable.java
create mode 100644
pinot-connectors/pinot-connector-kafka-2.0/src/main/resources/META-INF/services/org.apache.pinot.core.realtime.stream.StreamConsumerFactory
create mode 100644
pinot-connectors/pinot-connector-kafka-2.0/src/test/java/org/apache/pinot/core/realtime/impl/kafka2/KafkaPartitionLevelConsumerTest.java
create mode 100644
pinot-connectors/pinot-connector-kafka-2.0/src/test/java/org/apache/pinot/core/realtime/impl/kafka2/KafkaPartitionLevelStreamConfigTest.java
create mode 100644
pinot-connectors/pinot-connector-kafka-2.0/src/test/java/org/apache/pinot/core/realtime/impl/kafka2/utils/EmbeddedZooKeeper.java
create mode 100644
pinot-connectors/pinot-connector-kafka-2.0/src/test/java/org/apache/pinot/core/realtime/impl/kafka2/utils/MiniKafkaCluster.java
copy pinot-common/src/test/resources/log4j2.xml =>
pinot-connectors/pinot-connector-kafka-base/README.md (68%)
copy pinot-connectors/{ => pinot-connector-kafka-base}/pom.xml (60%)
rename pinot-connectors/{pinot-connector-kafka-0.9 =>
pinot-connector-kafka-base}/src/main/java/org/apache/pinot/core/realtime/impl/kafka/KafkaAvroMessageDecoder.java
(100%)
rename pinot-connectors/{pinot-connector-kafka-0.9 =>
pinot-connector-kafka-base}/src/main/java/org/apache/pinot/core/realtime/impl/kafka/KafkaJSONMessageDecoder.java
(100%)
create mode 100644
pinot-connectors/pinot-connector-kafka-base/src/main/java/org/apache/pinot/core/realtime/impl/kafka/KafkaStarterUtils.java
rename pinot-connectors/{pinot-connector-kafka-0.9 =>
pinot-connector-kafka-base}/src/main/java/org/apache/pinot/core/realtime/impl/kafka/KafkaStreamConfigProperties.java
(96%)
copy pinot-core/src/main/java/org/apache/pinot/core/io/util/ValueReader.java
=>
pinot-connectors/pinot-connector-kafka-base/src/main/java/org/apache/pinot/core/realtime/impl/kafka/MessageAndOffset.java
(56%)
delete mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/helix/ControllerRequestBuilderUtil.java
delete mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/helix/EmptyBrokerOnlineOfflineStateModelFactory.java
delete mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/helix/EmptySegmentOnlineOfflineStateModelFactory.java
rename
pinot-controller/src/test/java/org/apache/pinot/controller/api/{resources =>
}/AccessControlTest.java (87%)
rename
pinot-controller/src/test/java/org/apache/pinot/controller/api/{resources =>
}/FileUploadPathProviderTest.java (95%)
rename
pinot-controller/src/test/java/org/apache/pinot/controller/api/{resources =>
}/PinotFileUploadTest.java (79%)
rename
pinot-controller/src/test/java/org/apache/pinot/controller/api/{resources =>
}/PinotInstanceRestletResourceTest.java (96%)
rename
pinot-controller/src/test/java/org/apache/pinot/controller/api/{resources =>
}/PinotSchemaRestletResourceTest.java (98%)
rename
pinot-controller/src/test/java/org/apache/pinot/controller/api/{resources =>
}/PinotSegmentRestletResourceTest.java (85%)
rename
pinot-controller/src/test/java/org/apache/pinot/controller/api/{resources =>
}/PinotTableRestletResourceTest.java (96%)
rename
pinot-controller/src/test/java/org/apache/pinot/controller/api/{resources =>
}/PinotTenantRestletResourceTest.java (89%)
rename
pinot-controller/src/test/java/org/apache/pinot/controller/api/{resources =>
}/SegmentCompletionProtocolDeserTest.java (99%)
rename
pinot-controller/src/test/java/org/apache/pinot/controller/api/{resources =>
}/SegmentCompletionUtilsTest.java (96%)
rename
pinot-controller/src/test/java/org/apache/pinot/controller/api/{resources =>
}/ServerTableSizeReaderTest.java (98%)
rename
pinot-controller/src/test/java/org/apache/pinot/controller/api/{resources =>
}/TableSizeReaderTest.java (99%)
rename
pinot-controller/src/test/java/org/apache/pinot/controller/api/{resources =>
}/TableViewsTest.java (93%)
rename
pinot-core/src/main/java/org/apache/pinot/core/data/recordtransformer/{CompoundTransformer.java
=> CompositeTransformer.java} (70%)
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/data/recordtransformer/NullValueTransformer.java
rename
pinot-core/src/main/java/org/apache/pinot/core/data/recordtransformer/{SanitationTransformer.java
=> SanitizationTransformer.java} (92%)
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/io/util/VarLengthBytesValueReaderWriter.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/operator/filter/ExpressionFilterOperator.java
delete mode 100644
pinot-core/src/main/java/org/apache/pinot/core/operator/query/SelectionOnlyOperator.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/operator/query/SelectionOperator.java
delete mode 100644
pinot-core/src/main/java/org/apache/pinot/core/operator/query/SelectionOrderByOperator.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/TransformBlockDataFetcher.java
copy
pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/{SumAggregationFunction.java
=> DistinctCountRawHLLAggregationFunction.java} (53%)
copy
pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/{CountMVAggregationFunction.java
=> DistinctCountRawHLLMVAggregationFunction.java} (64%)
copy
pinot-common/src/main/java/org/apache/pinot/common/metrics/ServerMetrics.java
=>
pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/customobject/SerializedHLL.java
(51%)
copy pinot-api/src/main/java/org/apache/pinot/client/PinotClientTransport.java
=>
pinot-core/src/main/java/org/apache/pinot/core/realtime/stream/StreamDataProducer.java
(67%)
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/realtime/stream/StreamDataProvider.java
copy pinot-core/src/main/java/org/apache/pinot/core/{io/util/ValueReader.java
=> realtime/stream/StreamDataServerStartable.java} (53%)
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/io/util/VarLengthBytesValueReaderWriterTest.java
create mode 100644
pinot-hadoop/src/main/java/org/apache/pinot/hadoop/io/CombineAvroKeyInputFormat.java
create mode 100644
pinot-hadoop/src/main/java/org/apache/pinot/hadoop/job/InternalConfigConstants.java
create mode 100644
pinot-hadoop/src/main/java/org/apache/pinot/hadoop/job/SegmentPreprocessingJob.java
rename pinot-hadoop/src/main/java/org/apache/pinot/hadoop/job/{mapper =>
mappers}/SegmentCreationMapper.java (99%)
create mode 100644
pinot-hadoop/src/main/java/org/apache/pinot/hadoop/job/mappers/SegmentPreprocessingMapper.java
create mode 100644
pinot-hadoop/src/main/java/org/apache/pinot/hadoop/job/partitioners/GenericPartitioner.java
create mode 100644
pinot-hadoop/src/main/java/org/apache/pinot/hadoop/job/partitioners/PartitionFunctionFactory.java
create mode 100644
pinot-hadoop/src/main/java/org/apache/pinot/hadoop/job/reducers/SegmentPreprocessingReducer.java
create mode 100644
pinot-hadoop/src/main/java/org/apache/pinot/hadoop/utils/JobPreparationHelper.java
create mode 100644
pinot-hadoop/src/test/java/org/apache/pinot/hadoop/io/DeleteExtraPushedSegmentsTest.java
rename
pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/{SegmentCompletionIntegrationTests.java
=> SegmentCompletionIntegrationTest.java} (88%)
rename pinot-server/src/test/java/org/apache/pinot/server/api/{resources =>
}/BaseResourceTest.java (99%)
rename pinot-server/src/test/java/org/apache/pinot/server/api/{resources =>
}/TableSizeResourceTest.java (98%)
rename pinot-server/src/test/java/org/apache/pinot/server/api/{resources =>
}/TablesResourceTest.java (99%)
copy pinot-tools/src/main/resources/sample_data/{ =>
kafka_0.9}/airlineStats_realtime_table_config.json (100%)
copy pinot-tools/src/main/resources/sample_data/{ =>
kafka_0.9}/meetupRsvp_realtime_table_config.json (94%)
create mode 100644
pinot-tools/src/main/resources/sample_data/kafka_2.0/airlineStats_realtime_table_config.json
copy pinot-tools/src/main/resources/sample_data/{ =>
kafka_2.0}/meetupRsvp_realtime_table_config.json (82%)
create mode 100644 thirdeye/docs/configuration.rst
delete mode 100644 thirdeye/docs/production.rst
create mode 100644
thirdeye/thirdeye-frontend/app/pods/components/detection-health/component.js
create mode 100644
thirdeye/thirdeye-frontend/app/pods/components/detection-health/template.hbs
create mode 100644 thirdeye/thirdeye-pinot/config/h2db.mv.db
create mode 100644
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/alert/content/EntityGroupKeyContentFormatter.java
rename
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/alert/content/{MultipleAnomaliesEmailContentFormatter.java
=> MetricAnomaliesEmailContentFormatter.java} (92%)
create mode 100644
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/DetectionPreviewConfiguration.java
rename thirdeye/thirdeye-pinot/src/{test =>
main}/java/org/apache/pinot/thirdeye/detection/components/MockGrouper.java
(100%)
create mode 100644
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/components/TriggerConditionGrouper.java
create mode 100644
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/health/AnomalyCoverageStatus.java
create mode 100644
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/health/DetectionHealth.java
create mode 100644
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/health/DetectionTaskStatus.java
create mode 100644
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/health/HealthStatus.java
create mode 100644
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/health/RegressionStatus.java
copy thirdeye/thirdeye-pinot/src/{test =>
main}/java/org/apache/pinot/thirdeye/detection/spec/MockGrouperSpec.java (100%)
rename
thirdeye/thirdeye-pinot/src/{test/java/org/apache/pinot/thirdeye/detection/spec/MockGrouperSpec.java
=>
main/java/org/apache/pinot/thirdeye/detection/spec/TriggerConditionGrouperSpec.java}
(50%)
create mode 100644
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/yaml/translator/DetectionMetricAttributeHolder.java
create mode 100644
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/yaml/translator/DetectionMetricProperties.java
create mode 100644
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/formatter/DTOFormatter.java
create mode 100644
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/formatter/DetectionAlertConfigFormatter.java
create mode 100644
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/formatter/DetectionConfigFormatter.java
create mode 100644
thirdeye/thirdeye-pinot/src/main/resources/org/apache/pinot/thirdeye/detector/entity-groupkey-anomaly-report.ftl
rename
thirdeye/thirdeye-pinot/src/main/resources/org/apache/pinot/thirdeye/detector/{holiday-anomaly-report.ftl
=> metric-anomalies-template.ftl} (100%)
create mode 100644
thirdeye/thirdeye-pinot/src/test/java/org/apache/pinot/thirdeye/alert/content/ContentFormatterUtils.java
copy
thirdeye/thirdeye-pinot/src/test/java/org/apache/pinot/thirdeye/alert/content/{TestMultipleAnomaliesEmailContentFormatter.java
=> TestEntityGroupKeyEmailContentFormatter.java} (58%)
rename
thirdeye/thirdeye-pinot/src/test/java/org/apache/pinot/thirdeye/alert/content/{TestMultipleAnomaliesEmailContentFormatter.java
=> TestMetricAnomaliesEmailContentFormatter.java} (86%)
create mode 100644
thirdeye/thirdeye-pinot/src/test/java/org/apache/pinot/thirdeye/detection/components/TriggerConditionGrouperTest.java
create mode 100644
thirdeye/thirdeye-pinot/src/test/java/org/apache/pinot/thirdeye/detection/health/DetectionHealthTest.java
create mode 100644
thirdeye/thirdeye-pinot/src/test/java/org/apache/pinot/thirdeye/formatter/DetectionConfigFormatterTest.java
create mode 100644
thirdeye/thirdeye-pinot/src/test/resources/org/apache/pinot/thirdeye/detection/yaml/translator/compositePipelineTranslatorTestResult-4.json
create mode 100644
thirdeye/thirdeye-pinot/src/test/resources/org/apache/pinot/thirdeye/detection/yaml/translator/compositePipelineTranslatorTestResult-5.json
create mode 100644
thirdeye/thirdeye-pinot/src/test/resources/org/apache/pinot/thirdeye/detection/yaml/translator/pipeline-config-4.yaml
copy
thirdeye/thirdeye-pinot/src/test/resources/org/apache/pinot/thirdeye/detection/{validators/entity-pipeline-config-2.yaml
=> yaml/translator/pipeline-config-5.yaml} (80%)
copy
thirdeye/thirdeye-pinot/src/test/resources/{test-multiple-anomalies-email-content-formatter.html
=> test-entity-groupby-email-content-formatter.html} (58%)
rename
thirdeye/thirdeye-pinot/src/test/resources/{test-hierarchical-anomalies-email-content-formatter.html
=> test-hierarchical-metric-anomalies-template.html} (100%)
rename
thirdeye/thirdeye-pinot/src/test/resources/{test-multiple-anomalies-email-content-formatter.html
=> test-metric-anomalies-template.html} (100%)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
