This is an automated email from the ASF dual-hosted git repository.
nehapawar pushed a change to branch sharded_consumer_type_support_with_kinesis
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
discard f9f2419 Avoid writing 'stream' and also 'stream.kinesis.topic.name'
discard 5ff70d2 Add unit tests in Kinesis consumer (#6410)
omit 467965b Add tests for end-of-life cases
omit f00c8dc End-of-shard as end criteria AND consume shards in order
omit efbeea2 LIcense headers
omit bf81aac Cleanup, javadocs, comments
omit f39dbb7 Remove new partition groups creation in commit
omit 666c220 Dont create new CONSUMING segment if shard has reached end of
life
omit 9c31f41 Use shardId's last digits as partitionGroupId
omit 05e126b Remove unused classes and changes
omit c35d4a6 Consumer tweaks to get it working
omit f006615 Fix offsets in StreamMetadataProvider impl
omit 781e167 Add support for stream partition offsets (#6402)
omit a341b28 Implementation fixes
omit 6240808 fixing compilation
omit eac6c90 Return message batch instead of list in the fetch result
omit abe6647 Add isEndOfPartition check in checkpoints
omit bd7f673 Handle timeout exception in consumer and make shard iterator
type configurable
omit 0af84b1 Add test code for kinesis
omit 97d3ffc Change shard metadata logic
omit 6fcdf3e Refactor: get shard iterator methods
omit 173b3c4 Handle closed connections
omit bdbab1e Refactor code
omit 2c1bb76 Handle exceptions
omit b0f82c4 Add license header
omit 4eb2aa6 fetch records with timeout
omit 2399cbb Add Kinesis config wrapper
omit f3db54c Add license headers
omit 9bdc27f Reformat code
omit d8b18cd Move shardId out of checkpoint to partition group metadata
omit f9c317d Fix consumer code
omit 86fd014 Refactor kinesis shard metadata interface and add shardId to
the metadata
omit 855e39d Refactor PartitionGroupMetadataMap interface
omit e7a5314 Add kinesis code to handle offsets
omit 5b44917 Add PartitionGroupMetdataMap interface
omit e094e49 Add initial implementation of Kinesis consumer
omit 87d0c26 Add interfaces for V2 consumers
omit 6f4336e default methods to avoid interface changes
omit 9aa7e58 Server side changes and some fixes
omit 0fa424e An attempt at server-side changes
omit eb0e4c7 Checnges in test to make it complie
omit 1cb09cf More controller side changes
omit 1f81c1e Separate PartitionGroupInfo and PartitionGroupMetadata
omit b139460 Controller side code
omit 3ef1870 Rename partitionId to partitionGroupId
omit 652a3e6 StreamPartitionOffset to implement Checkpoint
omit 54ce986 Controller side changes pseudo code
add 1bb6c14 Add FST index which works on top of REGEXP_LIKE operator.
(#6120)
add fed6750 Adding a version option to pinot admin to show all the
component versions (#6380)
add e268a79 Adding S3PinotFS as one of the default PinotFS for Quickstart
(#6379)
add ea0bfa0 Add json index support (#6216)
add 4ea345d Adding more rolling file log4j2 configs for docker images
(#6390)
add b936f90 Update pinot realtime to match helm chart. (#6392)
add 6b43aef cleanup tar.gz segment files on job exit (#6385)
add 13bd443 Fix remaining links to outdated helm chart repos. (#6394)
add 93a4515 Adding Pinot minion segment generation and push task. (#6340)
add 19101e9 Make required interfaces or classes serializable for spark
(#6384)
add 37f2e28 Segment reset API (#6336)
add e87e141 [TE] limit reflection scanning range (#6405)
add 02ab193 remove accidental junit imports (#6404)
add 8fcb17d Compatibility test for segment operations upload and delete
(#6382)
add 8def748 Make minion tasks pluggable via reflection (#6395)
add f09de82 Fix the raw single column distinct executors (#6411)
add 7e0398b data gen - use unique output file names in csv mode (#6407)
add e11bcce Real-time json index (#6408)
add d04785c Introduce 'LOOKUP' Transform Function (#6383)
add 3d4c98d [TE]frontend - Add support for Group Constituents and Entity
Metric components (#6421)
add e018695 Fix escape character in transform function literals (#6416)
add a5c4ed2 Update groovy (#6425)
add 3e4c325 [INFRA] Cancel the previous builds (#6429)
add 3d24302 refactor AnomaliesResponses to avoid duplicate code (#6406)
add 19e3d51 [TE] endpoint - harleyjj/rca - add forecast, upper, lower,
and yoXy to aggregate/chunk enpoint (#6432)
add 33830cb [TE] suppress the anomaly if current value is NaN (#6428)
add 68fbb9c [TE] frontend - harleyjj/rca - display metric funnel in
metrics table when forecast selected (#6333)
add 99fe289 [TE] fix dimensional summary in emails (#6443)
add 72a7849 Support confluent schema registry (#6434)
add 832ece7 Bug Fix column metadata, read from the correct property
config for hasFSTIndex (#6441)
add 8d3d4d4 Merge H3-index branch to master (#6409)
add e8c4636 Fixing the issue that raw table name extracts from segment
metadata might contain _OFFLINE suffix (#6445)
add 950295a Fix gcs listFiles (#6426)
add d83e371 Adding ImportData sub command in pinot admin (#6396)
add 28882ba Adding pinot minion component into helm (#6430)
add 8085fb7 setting default Data Type while setting type in Add Schema
dialog (#6452)
add ec29b82 additional scalar functions for array type (#6446)
add 5de1d4b Fixing pinot helm 0.2.3 package (#6459)
add b592c8c Adding CRON scheduler for Pinot tasks (#6451)
add 60c802c Fixing pinot hadoop fs isDirectory method (#6461)
add 174a77b [TE]frontend - Add support for feedback propagation for
Entity Monitoring (#6447)
add 079f27a Cleanup dictionary and forward index loading in
SegmentPreProcessor (#6455)
add a36d1c0 [TE] Feedback propagation (#6463)
add a56f330 Broker time segment pruner(#6189): (#6462)
add 0417e20 additional string scalar functions (#6458)
add 79c9460 Dimension table storage quota config and validation (#6465)
add 2e7cdcd Always store raw table name in segment metadata (#6457)
add 4a89534 Replace BrokerRequestOptimizer with QueryOptimizer to also
optimize the PinotQuery (#6423)
add 0e8c0e2 [TE] clean up template as to not repeat code and reduce
conditionals (#6448)
add f17be35 Adding cluster config to config number of concurrent tasks
per instance for minion task: SegmentGenerationAndPushTaskGenerator (#6468)
add 8c8897e Simplify SegmentGenerationAndPushTask handling getting schema
and table config (#6469)
add f7af798 Handle scheduler calls with proper response when it's
disabled. (#6474)
add 7649f9f Remove the usage of deprecated range delimiter (#6475)
add 04e2bcc Update superset docker image repo (#6477)
add dde3c18 TLS-support for client-pinot and pinot-internode connections
(#6418)
add 830eb94 [TE]frontend - Activate the composite anomalies route (#6472)
add ec15f41 Bump up quartz version to 2.3.2 to prevent XXE (#6484)
add 63d4266 Create CONTRIBUTING.md (#6481)
add e209230 Create CODE_OF_CONDUCT.md (#6482)
add e5bf05b Fix the overflow issue when loading the large dictionary into
the buffer (#6476)
add f292730 Fix missing copyrights (#6489)
add 98803dd Adding pinot minion metrics to jmx prometheus reporter (#6488)
add bacaed2 Guard against multiple consuming segments for same partition
(#6483)
add fde307b Adding scalar function JsonPathArray to extract arrays from
json (#6490)
add d7f4fec Support chained transform functions (#6495)
add 28bfad9 Fixing -segmentNameGeneratorType help message (#6498)
add cf35e6e Fixing pinot controller metrics prefix (#6499)
add 56df592 Fixing groovy parser with back slash (#6501)
add b8f70e7 Support generating derived column during segment load (#6494)
add 6c30c76 [TE] migrate PQL queries to standard SQL (#6486)
add 1ae53fe Adding cron scheduler metrics reporting (#6502)
add 0f398a7 Fixing quickstart launcher from IDE (#6508)
add 209f57c [TE]frontend - Add filtering support for Entity Monitoring
tables (#6514)
new 6502a7d Controller side changes pseudo code
new 7504c31 StreamPartitionOffset to implement Checkpoint
new 3892fc4 Rename partitionId to partitionGroupId
new 96621f4 Controller side code
new a7fba5a Separate PartitionGroupInfo and PartitionGroupMetadata
new 75547ec More controller side changes
new 8afc48f Checnges in test to make it complie
new 31c64a0 An attempt at server-side changes
new ae863a1 Server side changes and some fixes
new 396dae0 default methods to avoid interface changes
new 3f14cf0 Add interfaces for V2 consumers
new 72a77c6 Add initial implementation of Kinesis consumer
new 2546098 Add PartitionGroupMetdataMap interface
new 0491903 Add kinesis code to handle offsets
new 5ed893e Refactor PartitionGroupMetadataMap interface
new 92ddaab Refactor kinesis shard metadata interface and add shardId to
the metadata
new b0d8c1b Fix consumer code
new 7a4fccc Move shardId out of checkpoint to partition group metadata
new 6c8af2b Reformat code
new b05ad08 Add license headers
new 96c32c4 Add Kinesis config wrapper
new 262d326 fetch records with timeout
new 29068ca Add license header
new 760ba06 Handle exceptions
new 71ac64d Refactor code
new 0b8bb67 Handle closed connections
new be19cf6 Refactor: get shard iterator methods
new 26085a8 Change shard metadata logic
new 1288012 Add test code for kinesis
new cf23ee3 Handle timeout exception in consumer and make shard iterator
type configurable
new eb428cd Add isEndOfPartition check in checkpoints
new c5c42d4 Return message batch instead of list in the fetch result
new 6cb0ebb fixing compilation
new d079c81 Implementation fixes
new a3deab3 Add support for stream partition offsets (#6402)
new 9c33895 Fix offsets in StreamMetadataProvider impl
new 2116206 Consumer tweaks to get it working
new 570a95a Remove unused classes and changes
new d7d0408 Use shardId's last digits as partitionGroupId
new 214c007 Dont create new CONSUMING segment if shard has reached end of
life
new 40b6dbe Remove new partition groups creation in commit
new ce1a646 Cleanup, javadocs, comments
new 7d1f7a1 LIcense headers
new 751e212 End-of-shard as end criteria AND consume shards in order
new dcb2ee1 Add tests for end-of-life cases
new ab9655b Add unit tests in Kinesis consumer (#6410)
new db2136a Avoid writing 'stream' and also 'stream.kinesis.topic.name'
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (f9f2419)
\
N -- N -- N
refs/heads/sharded_consumer_type_support_with_kinesis (db2136a)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 47 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../workflows/pinot_tests-workflow-run.yml | 31 +-
.github/workflows/scripts/.pinot_quickstart.sh | 27 +
.gitignore | 1 -
CODE_OF_CONDUCT.md | 96 +
CONTRIBUTING.md | 234 +
docker/images/pinot-superset/Dockerfile | 2 +-
.../images/pinot/etc/conf/pinot-broker-log4j2.xml | 62 +
.../pinot/etc/conf/pinot-controller-log4j2.xml | 61 +
.../images/pinot/etc/conf/pinot-server-log4j2.xml | 61 +
.../etc/jmx_prometheus_javaagent/configs/pinot.yml | 52 +-
kubernetes/helm/index.yaml | 50 +-
kubernetes/helm/pinot-0.2.3.tgz | Bin 0 -> 24664 bytes
kubernetes/helm/pinot/Chart.yaml | 4 +-
kubernetes/helm/pinot/README.md | 25 +-
.../helm/pinot/pinot-realtime-quickstart.yml | 684 +-
kubernetes/helm/pinot/requirements.lock | 2 +-
kubernetes/helm/pinot/requirements.yaml | 2 +-
kubernetes/helm/pinot/templates/_helpers.tpl | 23 +
.../minion/configmap.yaml} | 16 +-
.../minion/service-headless.yaml} | 33 +-
.../{Chart.yaml => templates/minion/service.yaml} | 33 +-
.../helm/pinot/templates/minion/statefulset.yml | 108 +
kubernetes/helm/pinot/values.yaml | 50 +
kubernetes/helm/thirdeye/Chart.lock | 6 +-
kubernetes/helm/thirdeye/Chart.yaml | 3 +-
.../broker/broker/BrokerAdminApiApplication.java | 23 +-
.../broker/broker/helix/HelixBrokerStarter.java | 36 +-
.../requesthandler/BaseBrokerRequestHandler.java | 122 +-
.../SingleConnectionBrokerRequestHandler.java | 5 +-
.../routing/segmentpruner/TimeSegmentPruner.java | 52 +-
.../broker/pruner/SegmentZKMetadataPrunerTest.java | 2 +-
.../request/PqlAndCalciteSqlCompatibilityTest.java | 161 +-
.../requesthandler/BrokerRequestOptionsTest.java | 44 +-
.../LiteralOnlyBrokerRequestTest.java | 4 +-
.../requesthandler/RangeMergeOptimizerTest.java | 20 +-
.../routing/segmentpruner/SegmentPrunerTest.java | 68 +-
pinot-common/pom.xml | 4 +
.../antlr4/org/apache/pinot/pql/parsers/PQL2.g4 | 4 +
.../common/function/TransformFunctionType.java | 1 +
.../common/function/scalar/ArrayFunctions.java | 59 +
.../common/function/scalar/JsonFunctions.java | 55 +
.../common/function/scalar/StringFunctions.java | 127 +-
.../common/metadata/segment/SegmentZKMetadata.java | 21 +-
.../pinot/common/metrics/ControllerGauge.java | 5 +-
.../pinot/common/metrics/ControllerMeter.java | 3 +-
.../pinot/common/metrics/ControllerTimer.java | 2 +-
.../apache/pinot/common/minion/MinionClient.java | 2 +-
.../pinot/common/request/AggregationInfo.java | 52 +-
.../apache/pinot/common/request/BrokerRequest.java | 353 +-
.../apache/pinot/common/request/DataSource.java | 11 +-
.../apache/pinot/common/request/Expression.java | 35 +-
.../pinot/common/request/ExpressionType.java | 4 +-
.../pinot/common/request/FilterOperator.java | 11 +-
.../apache/pinot/common/request/FilterQuery.java | 46 +-
.../pinot/common/request/FilterQueryMap.java | 11 +-
.../org/apache/pinot/common/request/Function.java | 21 +-
.../org/apache/pinot/common/request/GroupBy.java | 85 +-
.../pinot/common/request/HavingFilterQuery.java | 46 +-
.../pinot/common/request/HavingFilterQueryMap.java | 11 +-
.../apache/pinot/common/request/Identifier.java | 16 +-
.../pinot/common/request/InstanceRequest.java | 76 +-
.../org/apache/pinot/common/request/Literal.java | 4 +-
.../apache/pinot/common/request/PinotQuery.java | 61 +-
.../apache/pinot/common/request/QuerySource.java | 11 +-
.../org/apache/pinot/common/request/QueryType.java | 31 +-
.../org/apache/pinot/common/request/Selection.java | 94 +-
.../apache/pinot/common/request/SelectionSort.java | 16 +-
.../request/transform/TransformExpressionTree.java | 3 +-
.../pinot/common/response/ProcessingException.java | 283 +-
.../apache/pinot/common/utils/CommonConstants.java | 27 +-
.../common/utils/FileUploadDownloadClient.java | 40 +-
.../apache/pinot/parsers/utils/ParserUtils.java | 37 +-
.../parsers/PinotQuery2BrokerRequestConverter.java | 3 +-
.../org/apache/pinot/pql/parsers/Pql2Compiler.java | 27 +-
.../parsers/pql2/ast/BetweenPredicateAstNode.java | 7 +-
.../pql2/ast/ComparisonPredicateAstNode.java | 17 +-
.../pinot/pql/parsers/pql2/ast/FilterKind.java | 3 +-
.../apache/pinot/sql/parsers/CalciteSqlParser.java | 21 +-
.../pinot/common/function/JsonFunctionsTest.java | 86 +
.../transform/TransformExpressionTreeTest.java | 11 +-
.../apache/pinot/pql/parsers/Pql2CompilerTest.java | 10 +-
pinot-common/src/thrift/request.thrift | 3 +-
.../spark/connector/PinotServerDataFetcher.scala | 2 +
pinot-controller/pom.xml | 4 +
.../apache/pinot/controller/ControllerConf.java | 47 +-
.../apache/pinot/controller/ControllerStarter.java | 29 +-
.../api/ControllerAdminApiApplication.java | 65 +-
.../api/resources/PinotQueryResource.java | 39 +-
.../api/resources/PinotSegmentRestletResource.java | 65 +-
.../api/resources/PinotTableRestletResource.java | 57 +
.../api/resources/PinotTaskRestletResource.java | 120 +
.../api/resources/ServerTableSizeReader.java | 2 +-
.../helix/ControllerRequestURLBuilder.java | 20 +-
.../helix/core/PinotHelixResourceManager.java | 178 +-
.../helix/core/minion/ClusterInfoAccessor.java | 16 +
.../helix/core/minion/CronJobScheduleJob.java | 63 +
.../helix/core/minion/PinotTaskManager.java | 268 +-
.../core/minion/TableTaskSchedulerUpdater.java | 57 +
.../generator/ConvertToRawIndexTaskGenerator.java | 7 +-
.../core/minion/generator/PinotTaskGenerator.java | 14 +-
.../RealtimeToOfflineSegmentsTaskGenerator.java | 10 +-
.../SegmentGenerationAndPushTaskGenerator.java | 371 +
.../minion/generator/TaskGeneratorRegistry.java | 58 +-
.../core/periodictask/ControllerPeriodicTask.java | 4 +
.../realtime/PinotLLCRealtimeSegmentManager.java | 19 +
.../core/realtime/PinotRealtimeSegmentManager.java | 10 +-
.../controller/util/CompletionServiceHelper.java | 7 +-
.../util/ConsumingSegmentInfoReader.java | 2 +-
.../pinot/controller/util/FileIngestionUtils.java | 2 +-
.../pinot/controller/util/ListenerConfigUtil.java | 93 -
.../util/ServerSegmentMetadataReader.java | 2 +-
.../Homepage/Operations/SchemaComponent.tsx | 42 +-
.../pinot/controller/ControllerTestUtils.java | 15 +-
.../api/ConsumingSegmentInfoReaderTest.java | 2 +-
.../controller/api/PinotSegmentsMetadataTest.java | 2 +-
.../api/PinotTableRestletResourceTest.java | 93 +
.../controller/api/SegmentCompletionUtilsTest.java | 2 +-
.../controller/api/ServerTableSizeReaderTest.java | 2 +-
.../pinot/controller/api/TableSizeReaderTest.java | 2 +-
.../helix/core/PinotHelixResourceManagerTest.java | 4 +-
.../helix/core/minion/PinotTaskManagerTest.java | 145 +
...RealtimeToOfflineSegmentsTaskGeneratorTest.java | 43 +-
.../SegmentGenerationAndPushTaskGeneratorTest.java | 80 +
.../core/util/SegmentDeletionManagerTest.java | 3 +-
.../controller/util/ListenerConfigUtilTest.java | 44 +-
pinot-core/pom.xml | 17 +-
.../org/apache/pinot/core/common/DataSource.java | 21 +
.../apache/pinot/core/common/MinionConstants.java | 8 +
.../core/data/function/FunctionEvaluator.java | 5 +
.../data/function/GroovyFunctionEvaluator.java | 5 +-
.../data/function/InbuiltFunctionEvaluator.java | 33 +-
.../data/function/TimeSpecFunctionEvaluator.java | 13 +-
.../manager/offline/DimensionTableDataManager.java | 11 +
.../realtime/HLRealtimeSegmentDataManager.java | 4 +-
.../realtime/LLRealtimeSegmentDataManager.java | 16 +-
.../core/data/partition/PartitionFunction.java | 5 +-
.../data/readers/PinotSegmentColumnReader.java | 13 +
.../recordtransformer/ExpressionTransformer.java | 40 +-
.../data/recordtransformer/RecordTransformer.java | 3 +-
.../transform/function/ScalarFunctions.java | 27 +
.../transform/function/StDistanceFunction.java | 40 +-
.../generator/SegmentGeneratorConfig.java | 91 +-
.../core/indexsegment/mutable/MutableSegment.java | 6 +-
.../indexsegment/mutable/MutableSegmentImpl.java | 75 +-
.../io/util/VarLengthBytesValueReaderWriter.java | 241 -
.../pinot/core/io/util/VarLengthValueReader.java | 121 +
.../pinot/core/io/util/VarLengthValueWriter.java | 133 +
.../operator/filter/BitmapBasedFilterOperator.java | 6 +-
.../core/operator/filter/FilterOperatorUtils.java | 18 +-
.../operator/filter/H3IndexFilterOperator.java | 246 +
.../operator/filter/JsonMatchFilterOperator.java | 59 +
.../FSTBasedRegexpPredicateEvaluatorFactory.java | 156 +
.../function/LookupTransformFunction.java | 332 +
.../function/TransformFunctionFactory.java | 1 +
.../org/apache/pinot/core/plan/FilterPlanNode.java | 78 +-
.../BaseRawFloatSingleColumnDistinctExecutor.java | 2 +-
.../BaseRawIntSingleColumnDistinctExecutor.java | 2 +-
.../BaseRawLongSingleColumnDistinctExecutor.java | 2 +-
.../BaseRawStringSingleColumnDistinctExecutor.java | 2 +-
.../optimizer/filter/MergeEqInFilterOptimizer.java | 16 +-
.../filter/MergeRangeFilterOptimizer.java | 5 +-
.../{Predicate.java => JsonMatchPredicate.java} | 57 +-
.../query/request/context/predicate/Predicate.java | 2 +-
.../request/context/predicate/RangePredicate.java | 18 +-
.../BrokerRequestToQueryContextConverter.java | 268 +-
.../context/utils/QueryContextConverterUtils.java | 4 +
.../converter/RealtimeSegmentConverter.java | 7 +-
.../RealtimeSegmentSegmentCreationDataSource.java | 3 +-
.../core/realtime/impl/RealtimeSegmentConfig.java | 46 +-
.../realtime/impl/geospatial/MutableH3Index.java | 78 +
.../RealtimeLuceneTextIndexReader.java | 6 +
.../core/realtime/impl/json/MutableJsonIndex.java | 269 +
.../core/requesthandler/RangeMergeOptimizer.java | 3 +-
.../segment/creator/ColumnIndexCreationInfo.java | 3 +-
.../core/segment/creator/ColumnStatistics.java | 3 +-
...DataSource.java => GeoSpatialIndexCreator.java} | 21 +-
...eationDataSource.java => JsonIndexCreator.java} | 21 +-
.../segment/creator/SegmentCreationDataSource.java | 3 +-
.../pinot/core/segment/creator/SegmentCreator.java | 6 +-
.../creator/SegmentIndexCreationDriver.java | 3 +-
.../segment/creator/SegmentIndexCreationInfo.java | 5 +-
.../creator/SegmentPreIndexStatsContainer.java | 5 +-
.../pinot/core/segment/creator/TextIndexType.java | 3 +-
.../creator/impl/SegmentColumnarIndexCreator.java | 145 +-
.../creator/impl/SegmentDictionaryCreator.java | 41 +-
.../core/segment/creator/impl/V1Constants.java | 5 +
.../impl/inv/BitmapInvertedIndexWriter.java | 90 +
.../inv/OffHeapBitmapInvertedIndexCreator.java | 43 +-
.../impl/inv/OnHeapBitmapInvertedIndexCreator.java | 45 +-
.../impl/inv/geospatial/BaseH3IndexCreator.java | 168 +
.../creator/impl/inv/geospatial/H3IndexConfig.java | 50 +
.../impl/inv/geospatial/H3IndexResolution.java | 71 +
.../impl/inv/geospatial/OffHeapH3IndexCreator.java | 218 +
.../impl/inv/geospatial/OnHeapH3IndexCreator.java | 49 +
.../impl/inv/json/BaseJsonIndexCreator.java | 173 +
.../impl/inv/json/OffHeapJsonIndexCreator.java | 282 +
.../impl/inv/json/OnHeapJsonIndexCreator.java | 60 +
.../impl/inv/text/LuceneFSTIndexCreator.java | 103 +
.../segment/index/column/ColumnIndexContainer.java | 23 +-
.../index/column/PhysicalColumnIndexContainer.java | 77 +-
.../converter/SegmentV1V2ToV3FormatConverter.java | 53 +-
.../segment/index/datasource/BaseDataSource.java | 27 +
.../index/datasource/ImmutableDataSource.java | 5 +-
.../index/datasource/MutableDataSource.java | 15 +-
.../segment/index/loader/IndexLoadingConfig.java | 87 +-
.../core/segment/index/loader/LoaderUtils.java | 47 +
.../segment/index/loader/SegmentPreProcessor.java | 31 +-
.../defaultcolumn/BaseDefaultColumnHandler.java | 303 +-
.../loader/defaultcolumn/DefaultColumnHandler.java | 5 +-
.../defaultcolumn/DefaultColumnHandlerFactory.java | 9 +-
.../defaultcolumn/V1DefaultColumnHandler.java | 16 +-
.../defaultcolumn/V3DefaultColumnHandler.java | 34 +-
.../index/loader/invertedindex/H3IndexHandler.java | 161 +
.../loader/invertedindex/InvertedIndexHandler.java | 19 +-
.../loader/invertedindex/JsonIndexHandler.java | 158 +
.../invertedindex/LuceneFSTIndexHandler.java | 146 +
.../loader/invertedindex/RangeIndexHandler.java | 30 +-
.../loader/invertedindex/TextIndexHandler.java | 47 +-
.../segment/index/metadata/ColumnMetadata.java | 20 +-
.../segment/index/metadata/SegmentMetadata.java | 4 +
.../index/metadata/SegmentMetadataImpl.java | 49 +-
.../index/readers/BaseImmutableDictionary.java | 15 +-
.../index/readers/BitmapInvertedIndexReader.java | 87 +-
.../{TextIndexReader.java => H3IndexReader.java} | 19 +-
.../{TextIndexReader.java => JsonIndexReader.java} | 10 +-
.../index/readers/LuceneFSTIndexReader.java | 82 +
.../segment/index/readers/TextIndexReader.java | 8 +
.../readers/geospatial/ImmutableH3IndexReader.java | 84 +
.../readers/json/ImmutableJsonIndexReader.java | 243 +
.../index/readers/text/LuceneTextIndexReader.java | 6 +
.../pinot/core/segment/memory/PinotByteBuffer.java | 15 +-
.../core/segment/name/SegmentNameGenerator.java | 3 +-
.../pinot/core/segment/store/ColumnIndexType.java | 5 +-
.../core/segment/store/FilePerIndexDirectory.java | 13 +-
.../pinot/core/segment/store/SegmentDirectory.java | 25 +-
.../core/segment/store/SegmentDirectoryPaths.java | 9 +
.../segment/store/SegmentLocalFSDirectory.java | 10 -
.../segment/store/SingleFileIndexDirectory.java | 9 +-
.../virtualcolumn/VirtualColumnIndexContainer.java | 17 +
.../core/startree/v2/store/StarTreeDataSource.java | 6 +-
.../pinot/core/transport}/ListenerConfig.java | 36 +-
.../apache/pinot/core/transport/QueryRouter.java | 34 +-
.../apache/pinot/core/transport/QueryServer.java | 51 +
.../pinot/core/transport/ServerChannels.java | 44 +
.../pinot/core/transport/ServerInstance.java | 21 +
.../core/transport/ServerRoutingInstance.java | 30 +-
.../org/apache/pinot/core/transport/TlsConfig.java | 77 +
.../H3Utils.java} | 21 +-
.../apache/pinot/core/util/ListenerConfigUtil.java | 207 +
.../apache/pinot/core/util/TableConfigUtils.java | 24 +-
.../java/org/apache/pinot/core/util/TlsUtils.java | 269 +
.../org/apache/pinot/core/util/fst/FSTBuilder.java | 65 +
.../pinot/core/util/fst/PinotBufferIndexInput.java | 89 +
.../apache/pinot/core/util/fst/RegexpMatcher.java | 164 +
.../core/data/function/InbuiltFunctionsTest.java | 71 +
.../offline/DimensionTableDataManagerTest.java | 5 +
.../ExpressionTransformerTest.java | 29 +
.../transform/StDistanceFunctionTest.java | 8 +-
.../MutableSegmentImplAggregateMetricsTest.java | 16 +-
.../util/VarLengthBytesValueReaderWriterTest.java | 172 -
.../io/util/VarLengthValueReaderWriterTest.java | 117 +
.../NoDictionaryRangePredicateEvaluatorTest.java | 104 +-
.../function/BaseTransformFunctionTest.java | 21 +
.../function/LookupTransformFunctionTest.java | 386 +
.../ScalarTransformFunctionWrapperTest.java | 297 +-
.../pinot/core/plan/CombinePlanNodeTest.java | 2 +-
.../core/query/optimizer/QueryOptimizerTest.java | 32 +-
.../query/pruner/ColumnValueSegmentPrunerTest.java | 4 +-
.../inv/geospatial/H3IndexResolutionTest.java} | 20 +-
.../core/segment/index/ColumnMetadataTest.java | 1 +
.../pinot/core/segment/index/H3IndexTest.java | 110 +
.../pinot/core/segment/index/JsonIndexTest.java | 216 +
.../index/creator/LuceneFSTIndexCreatorTest.java | 74 +
.../core/segment/index/loader/LoaderTest.java | 139 +
.../index/loader/SegmentPreProcessorTest.java | 115 +-
.../BaseDefaultColumnHandlerTest.java | 26 +-
.../segment/store/SegmentLocalFSDirectoryTest.java | 2 +-
.../pinot/core/util/TableConfigUtilsTest.java | 57 +-
.../org/apache/pinot/queries/BaseQueriesTest.java | 2 +-
.../apache/pinot/queries/DistinctQueriesTest.java | 44 +-
.../queries/FSTBasedRegexpLikeQueriesTest.java | 477 +
.../apache/pinot/queries/H3IndexQueriesTest.java | 233 +
.../pinot/queries/TextSearchQueriesTest.java | 3 +-
.../java/org/apache/pinot/util/FSTBuilderTest.java | 107 +
.../src/test/resources/data/newColumnsSchema4.json | 84 +
.../pinot/compat/tests/ClusterDescriptor.java | 10 +-
.../org/apache/pinot/compat/tests/SegmentOp.java | 256 +-
.../org/apache/pinot/compat/tests/TableOp.java | 26 +-
.../tests/AdminConsoleIntegrationTest.java | 13 +
.../tests/BaseClusterIntegrationTestSet.java | 5 +
.../tests/ClusterIntegrationTestUtils.java | 55 +-
.../pinot/integration/tests/ClusterTest.java | 66 +-
.../tests/JsonPathClusterIntegrationTest.java | 80 +-
.../tests/OfflineClusterIntegrationTest.java | 55 +-
.../tests/SimpleMinionClusterIntegrationTest.java | 71 +-
...onulls_default_column_test_extra_columns.schema | 8 +
.../compat-tests/configs/feature-test-1.json | 28 +-
.../src/test/resources/compat-tests/sample.yaml | 16 +-
pinot-minion/pom.xml | 5 +
.../org/apache/pinot/minion/MinionStarter.java | 35 +-
.../DefaultMinionEventObserver.java | 2 +-
.../DefaultMinionEventObserverFactory.java | 18 +-
.../minion/event/EventObserverFactoryRegistry.java | 85 +
.../{events => event}/MinionEventObserver.java | 2 +-
.../MinionEventObserverFactory.java | 19 +-
.../events/EventObserverFactoryRegistry.java | 50 -
.../ConvertToRawIndexTaskExecutorFactory.java | 14 +
.../executor/MergeRollupTaskExecutorFactory.java | 15 +
.../minion/executor/PinotTaskExecutorFactory.java | 14 +-
.../minion/executor/PurgeTaskExecutorFactory.java | 14 +
...altimeToOfflineSegmentsTaskExecutorFactory.java | 18 +-
.../executor/SegmentGenerationAndPushResult.java | 91 +
.../SegmentGenerationAndPushTaskExecutor.java | 314 +
...gmentGenerationAndPushTaskExecutorFactory.java} | 18 +-
.../executor/TaskExecutorFactoryRegistry.java | 61 +-
.../minion/taskfactory/TaskFactoryRegistry.java | 22 +-
.../batch/common/SegmentGenerationTaskRunner.java | 14 +-
.../ingestion/batch/common/SegmentPushUtils.java | 44 +-
.../batch/hadoop/HadoopSegmentCreationMapper.java | 3 +-
.../spark/SparkSegmentGenerationJobRunner.java | 3 +-
.../standalone/SegmentGenerationJobRunner.java | 2 +-
.../apache/pinot/hadoop/io/PinotOutputFormat.java | 3 +-
.../v0_deprecated/pinot-spark/pom.xml | 282 +-
.../src/test/java/SegmentCreationSparkTest.java | 141 +
.../src/test/resources}/test_sample_data.csv | 0
.../apache/pinot/plugin/filesystem/GcsPinotFS.java | 23 +-
.../pinot/plugin/filesystem/HadoopPinotFS.java | 13 +-
...aConfluentSchemaRegistryAvroMessageDecoder.java | 2 +-
pinot-server/pom.xml | 4 -
.../org/apache/pinot/server/conf/ServerConf.java | 20 +-
.../pinot/server/starter/ServerInstance.java | 48 +-
.../server/starter/helix/AdminApiApplication.java | 25 +-
.../server/starter/helix/HelixServerStarter.java | 40 +-
.../apache/pinot/server/api/AccessControlTest.java | 11 +-
.../apache/pinot/server/api/BaseResourceTest.java | 12 +-
.../pinot/server/api/TablesResourceTest.java | 14 +-
.../annotations/minion/EventObserverFactory.java | 20 +-
.../annotations/minion/TaskExecutorFactory.java | 20 +-
.../spi/annotations/minion/TaskGenerator.java | 20 +-
.../apache/pinot/spi/config/BaseJsonConfig.java | 3 +-
.../apache/pinot/spi/config/table/FieldConfig.java | 2 +-
.../pinot/spi/config/table/IndexingConfig.java | 10 +
.../java/org/apache/pinot/spi/data/FieldSpec.java | 3 +-
.../pinot/spi/data/IngestionSchemaValidator.java | 4 +-
.../java/org/apache/pinot/spi/data/Schema.java | 5 +-
.../apache/pinot/spi/data/TimeGranularitySpec.java | 5 +-
.../apache/pinot/spi/data/readers/GenericRow.java | 3 +-
.../pinot/spi/data/readers/RecordExtractor.java | 3 +-
.../pinot/spi/data/readers/RecordReader.java | 3 +-
.../spi/ingestion/batch/BatchConfigProperties.java | 5 +-
.../batch/spec/SegmentGenerationJobSpec.java | 26 +
.../batch/spec/SegmentGenerationTaskSpec.java | 7 +-
.../pinot/spi/ingestion/batch/spec/TlsSpec.java | 45 +-
.../org/apache/pinot/spi/plugin/PluginManager.java | 44 +
.../pinot/spi/utils/GroovyTemplateUtils.java | 4 +-
.../pinot/spi/utils/IngestionConfigUtils.java | 8 +-
.../java/org/apache/pinot/spi/utils/JsonUtils.java | 191 +-
.../org/apache/pinot/spi/utils/JsonUtilsTest.java | 145 +
.../resources/ingestion_job_spec_template.yaml | 7 +-
pinot-tools/pom.xml | 49 +
.../pinot/tools/BatchQuickstartWithMinion.java | 18 +-
.../org/apache/pinot/tools/BootstrapTableTool.java | 168 +-
.../org/apache/pinot/tools/GenericQuickstart.java | 149 +
.../org/apache/pinot/tools/JoinQuickStart.java | 7 +
.../apache/pinot/tools/JsonIndexQuickStart.java | 89 +
.../java/org/apache/pinot/tools/Quickstart.java | 17 +-
.../pinot/tools/RealtimeJsonIndexQuickStart.java | 111 +
.../pinot/tools/admin/PinotAdministrator.java | 20 +-
.../tools/admin/command/AddSchemaCommand.java | 18 +-
.../pinot/tools/admin/command/AddTableCommand.java | 17 +-
.../tools/admin/command/AddTenantCommand.java | 13 +-
.../tools/admin/command/BootstrapTableCommand.java | 6 +-
.../tools/admin/command/ChangeTableState.java | 18 +-
.../tools/admin/command/ImportDataCommand.java | 390 +
.../command/LaunchDataIngestionJobCommand.java | 9 +
.../admin/command/OperateClusterConfigCommand.java | 15 +-
.../tools/admin/command/PostQueryCommand.java | 15 +-
.../tools/admin/command/QuickStartCommand.java | 7 +
.../tools/admin/command/QuickstartRunner.java | 48 +-
.../tools/admin/command/StartMinionCommand.java | 28 +-
.../tools/admin/command/UploadSegmentCommand.java | 15 +-
.../pinot/tools/data/generator/DataGenerator.java | 2 +-
.../tools/scan/query/RangePredicateFilter.java | 36 +-
.../converter/DictionaryToRawIndexConverter.java | 4 +-
.../pinot/tools/streams/MeetupRsvpJsonStream.java | 63 +
.../pinot/tools/streams/MeetupRsvpStream.java | 149 +-
.../main/resources/conf/pinot-minion-log4j2.xml | 48 +
.../main/resources/conf/pinot-service-log4j2.xml | 8 +
.../githubEvents_offline_table_config.json | 27 +
.../batch/githubEvents/githubEvents_schema.json | 40 +
.../batch/githubEvents/ingestionJobSpec.yaml | 139 +
.../githubEvents/rawdata/githubEvents_data.json | 10000 +++++++++++++++++++
.../batch/githubEvents/sparkIngestionJobSpec.yaml | 147 +
.../batch/starbucksStores/ingestionJobSpec.yaml | 139 +
.../batch/starbucksStores/rawdata/data.csv | 6444 ++++++++++++
.../starbucksStores_offline_table_config.json | 34 +
.../starbucksStores/starbucksStores_schema.json | 29 +
.../airlineStats_offline_table_config.json | 44 +
.../batch}/airlineStats/airlineStats_schema.json | 0
.../baseballStats_offline_table_config.json | 43 +
.../batch/baseballStats/baseballStats_schema.json | 0
.../baseballStats/rawdata/baseballStats_data.csv | 0
... => json_meetupRsvp_realtime_table_config.json} | 34 +-
.../stream/meetupRsvp/json_meetupRsvp_schema.json | 51 +
.../stream/meetupRsvp/meetupRsvp_schema.json | 2 +-
.../upsert_meetupRsvp_realtime_table_config.json | 10 +
.../meetupRsvp/upsert_meetupRsvp_schema.json | 2 +-
pom.xml | 16 +-
thirdeye/pom.xml | 2 +-
.../dashboard/ThirdEyeDashboardApplication.java | 8 +-
.../dashboard/resources/AnomalyResource.java | 44 +-
.../resources/v2/RootCauseMetricResource.java | 61 +-
.../dashboard/resources/v2/RootCauseResource.java | 11 +-
.../bao/TestMergedAnomalyResultManager.java | 19 +
.../thirdeye/datasource/pinot/PqlUtilsTest.java | 26 +-
.../app/mocks/compositeAnomalies.js | 188 +-
.../app/pods/components/alert-details/template.hbs | 387 +-
.../pods/components/anomaly-summary/component.js | 14 +-
.../components/composite-anomalies/component.js | 58 +-
.../composite-anomalies/data-table/template.hbs | 24 +
.../entity-metrics-anomalies/component.js | 56 +
.../entity-metrics-anomalies/template.hbs | 1 +
.../group-constituents-anomalies/component.js | 56 +
.../group-constituents-anomalies/template.hbs | 1 +
.../parent-anomalies/component.js | 96 +-
.../parent-anomalies/template.hbs | 21 +-
.../components/composite-anomalies/template.hbs | 11 +-
.../pods/components/rootcause-anomaly/component.js | 207 +-
.../pods/components/rootcause-metrics/component.js | 220 +-
.../anomalies-list/template.hbs | 6 +-
.../criticality/template.hbs | 3 +
.../current-predicted/template.hbs | 6 +
.../dimensions/template.hbs | 3 +
.../group-name/component.js | 10 +
.../group-name/template.hbs | 3 +
.../composite-anomalies-table/metric/template.hbs | 3 +
.../resolution/component.js | 60 +-
.../resolution/template.hbs | 65 +-
.../start-duration/component.js | 10 +
.../start-duration/template.hbs | 10 +-
.../explore/composite-anomalies/controller.js | 10 +
.../explore/composite-anomalies/template.hbs | 9 +-
.../app/pods/manage/explore/route.js | 56 +-
.../pods/partials/rootcause/metrics/template.hbs | 1 +
.../app/pods/rootcause/controller.js | 687 +-
.../app/pods/services/api/anomalies/service.js | 76 +-
.../services/rootcause-aggregates-cache/service.js | 74 +-
.../app/shared/metricFunnelColumns.js | 56 +
thirdeye/thirdeye-frontend/app/styles/app.scss | 2 +-
.../app/styles/components/breadcrumb-list.scss | 2 +
.../app/styles/components/te-anomaly-table.scss | 1 +
.../pods/custom/composite-anomalies-table.scss | 53 +
.../styles/pods/custom/parent-anomalies-table.scss | 21 -
.../app/styles/shared/_styles.scss | 6 +-
.../app/utils/anomalies-tree-parser.js | 104 +-
thirdeye/thirdeye-frontend/app/utils/anomaly.js | 131 +-
.../app/utils/composite-anomalies.js | 141 +
thirdeye/thirdeye-frontend/app/utils/constants.js | 4 +-
thirdeye/thirdeye-frontend/app/utils/utils.js | 122 +-
.../composite-anomalies/component-test.js | 2 +-
.../entity-metrics-anomalies/component-test.js | 74 +
.../group-constituents-anomalies/component-test.js | 83 +
.../parent-anomalies/component-test.js | 16 +-
.../tests/unit/utils/anomalies-tree-parser-test.js | 251 +-
.../tests/unit/utils/utils-test.js | 44 +
.../anomaly/ThirdEyeAnomalyApplication.java | 7 +
.../thirdeye/common/ThirdEyeConfiguration.java | 13 +
.../resources/v2/BaselineParsingUtils.java | 15 +-
.../bao/jdbc/MergedAnomalyResultManagerImpl.java | 25 +-
.../datasource/pinot/PinotDataSourceTimeQuery.java | 12 +-
.../datasource/pinot/PinotThirdEyeDataSource.java | 12 +-
.../pinot/{PqlUtils.java => SqlUtils.java} | 63 +-
.../pinot/resources/PinotDataSourceResource.java | 2 +-
.../detection/DetectionPipelineTaskRunner.java | 4 +-
.../registry/DetectionAlertRegistry.java | 28 +-
.../annotation/registry/DetectionRegistry.java | 24 +-
.../rootcause/timeseries/BaselineAggregate.java | 21 +
.../datalayer/bao/MergedAnomalyResultManager.java | 2 +
.../installation/cloud/on-premise.md | 4 +-
479 files changed, 36865 insertions(+), 5131 deletions(-)
copy docker/images/pinot-superset/Dockerfile =>
.github/workflows/pinot_tests-workflow-run.yml (60%)
create mode 100644 CODE_OF_CONDUCT.md
create mode 100644 CONTRIBUTING.md
create mode 100644 docker/images/pinot/etc/conf/pinot-broker-log4j2.xml
create mode 100644 docker/images/pinot/etc/conf/pinot-controller-log4j2.xml
create mode 100644 docker/images/pinot/etc/conf/pinot-server-log4j2.xml
create mode 100644 kubernetes/helm/pinot-0.2.3.tgz
copy kubernetes/helm/pinot/{requirements.yaml =>
templates/minion/configmap.yaml} (72%)
copy kubernetes/helm/pinot/{Chart.yaml =>
templates/minion/service-headless.yaml} (59%)
copy kubernetes/helm/pinot/{Chart.yaml => templates/minion/service.yaml} (59%)
create mode 100644 kubernetes/helm/pinot/templates/minion/statefulset.yml
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/CronJobScheduleJob.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/TableTaskSchedulerUpdater.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/generator/SegmentGenerationAndPushTaskGenerator.java
delete mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/util/ListenerConfigUtil.java
create mode 100644
pinot-controller/src/test/java/org/apache/pinot/controller/helix/core/minion/PinotTaskManagerTest.java
create mode 100644
pinot-controller/src/test/java/org/apache/pinot/controller/helix/core/minion/generator/SegmentGenerationAndPushTaskGeneratorTest.java
delete mode 100644
pinot-core/src/main/java/org/apache/pinot/core/io/util/VarLengthBytesValueReaderWriter.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/io/util/VarLengthValueReader.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/io/util/VarLengthValueWriter.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/operator/filter/H3IndexFilterOperator.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/operator/filter/JsonMatchFilterOperator.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/operator/filter/predicate/FSTBasedRegexpPredicateEvaluatorFactory.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/LookupTransformFunction.java
copy
pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/{Predicate.java
=> JsonMatchPredicate.java} (50%)
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/realtime/impl/geospatial/MutableH3Index.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/realtime/impl/json/MutableJsonIndex.java
copy
pinot-core/src/main/java/org/apache/pinot/core/segment/creator/{SegmentCreationDataSource.java
=> GeoSpatialIndexCreator.java} (68%)
copy
pinot-core/src/main/java/org/apache/pinot/core/segment/creator/{SegmentCreationDataSource.java
=> JsonIndexCreator.java} (69%)
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/BitmapInvertedIndexWriter.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/geospatial/BaseH3IndexCreator.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/geospatial/H3IndexConfig.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/geospatial/H3IndexResolution.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/geospatial/OffHeapH3IndexCreator.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/geospatial/OnHeapH3IndexCreator.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/json/BaseJsonIndexCreator.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/json/OffHeapJsonIndexCreator.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/json/OnHeapJsonIndexCreator.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/text/LuceneFSTIndexCreator.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/segment/index/loader/invertedindex/H3IndexHandler.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/segment/index/loader/invertedindex/JsonIndexHandler.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/segment/index/loader/invertedindex/LuceneFSTIndexHandler.java
copy
pinot-core/src/main/java/org/apache/pinot/core/segment/index/readers/{TextIndexReader.java
=> H3IndexReader.java} (63%)
copy
pinot-core/src/main/java/org/apache/pinot/core/segment/index/readers/{TextIndexReader.java
=> JsonIndexReader.java} (77%)
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/segment/index/readers/LuceneFSTIndexReader.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/segment/index/readers/geospatial/ImmutableH3IndexReader.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/segment/index/readers/json/ImmutableJsonIndexReader.java
rename
{pinot-controller/src/main/java/org/apache/pinot/controller/api/listeners =>
pinot-core/src/main/java/org/apache/pinot/core/transport}/ListenerConfig.java
(67%)
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/transport/TlsConfig.java
copy
pinot-core/src/main/java/org/apache/pinot/core/{segment/creator/SegmentIndexCreationInfo.java
=> util/H3Utils.java} (70%)
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/util/ListenerConfigUtil.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/util/TlsUtils.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/util/fst/FSTBuilder.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/util/fst/PinotBufferIndexInput.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/util/fst/RegexpMatcher.java
delete mode 100644
pinot-core/src/test/java/org/apache/pinot/core/io/util/VarLengthBytesValueReaderWriterTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/io/util/VarLengthValueReaderWriterTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/operator/transform/function/LookupTransformFunctionTest.java
copy
pinot-core/src/{main/java/org/apache/pinot/core/segment/creator/SegmentIndexCreationInfo.java
=>
test/java/org/apache/pinot/core/segment/creator/impl/inv/geospatial/H3IndexResolutionTest.java}
(60%)
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/segment/index/H3IndexTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/segment/index/JsonIndexTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/segment/index/creator/LuceneFSTIndexCreatorTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/queries/FSTBasedRegexpLikeQueriesTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/queries/H3IndexQueriesTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/util/FSTBuilderTest.java
create mode 100644 pinot-core/src/test/resources/data/newColumnsSchema4.json
rename pinot-minion/src/main/java/org/apache/pinot/minion/{events =>
event}/DefaultMinionEventObserver.java (97%)
rename pinot-minion/src/main/java/org/apache/pinot/minion/{events =>
event}/DefaultMinionEventObserverFactory.java (75%)
create mode 100644
pinot-minion/src/main/java/org/apache/pinot/minion/event/EventObserverFactoryRegistry.java
rename pinot-minion/src/main/java/org/apache/pinot/minion/{events =>
event}/MinionEventObserver.java (97%)
rename pinot-minion/src/main/java/org/apache/pinot/minion/{events =>
event}/MinionEventObserverFactory.java (72%)
delete mode 100644
pinot-minion/src/main/java/org/apache/pinot/minion/events/EventObserverFactoryRegistry.java
create mode 100644
pinot-minion/src/main/java/org/apache/pinot/minion/executor/SegmentGenerationAndPushResult.java
create mode 100644
pinot-minion/src/main/java/org/apache/pinot/minion/executor/SegmentGenerationAndPushTaskExecutor.java
copy
pinot-minion/src/main/java/org/apache/pinot/minion/executor/{ConvertToRawIndexTaskExecutorFactory.java
=> SegmentGenerationAndPushTaskExecutorFactory.java} (64%)
create mode 100644
pinot-plugins/pinot-batch-ingestion/v0_deprecated/pinot-spark/src/test/java/SegmentCreationSparkTest.java
copy {pinot-core/src/test/resources/data =>
pinot-plugins/pinot-batch-ingestion/v0_deprecated/pinot-spark/src/test/resources}/test_sample_data.csv
(100%)
copy
pinot-core/src/main/java/org/apache/pinot/core/segment/creator/SegmentCreationDataSource.java
=>
pinot-spi/src/main/java/org/apache/pinot/spi/annotations/minion/EventObserverFactory.java
(57%)
copy
pinot-core/src/main/java/org/apache/pinot/core/segment/creator/SegmentCreationDataSource.java
=>
pinot-spi/src/main/java/org/apache/pinot/spi/annotations/minion/TaskExecutorFactory.java
(57%)
copy
pinot-core/src/main/java/org/apache/pinot/core/segment/creator/SegmentCreationDataSource.java
=>
pinot-spi/src/main/java/org/apache/pinot/spi/annotations/minion/TaskGenerator.java
(58%)
rename
pinot-controller/src/main/java/org/apache/pinot/controller/api/listeners/TlsConfiguration.java
=>
pinot-spi/src/main/java/org/apache/pinot/spi/ingestion/batch/spec/TlsSpec.java
(64%)
create mode 100644
pinot-spi/src/test/java/org/apache/pinot/spi/utils/JsonUtilsTest.java
copy
pinot-core/src/main/java/org/apache/pinot/core/segment/creator/SegmentIndexCreationInfo.java
=>
pinot-tools/src/main/java/org/apache/pinot/tools/BatchQuickstartWithMinion.java
(68%)
create mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/GenericQuickstart.java
create mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/JsonIndexQuickStart.java
create mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/RealtimeJsonIndexQuickStart.java
create mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/admin/command/ImportDataCommand.java
create mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/streams/MeetupRsvpJsonStream.java
create mode 100644 pinot-tools/src/main/resources/conf/pinot-minion-log4j2.xml
create mode 100644
pinot-tools/src/main/resources/examples/batch/githubEvents/githubEvents_offline_table_config.json
create mode 100644
pinot-tools/src/main/resources/examples/batch/githubEvents/githubEvents_schema.json
create mode 100644
pinot-tools/src/main/resources/examples/batch/githubEvents/ingestionJobSpec.yaml
create mode 100644
pinot-tools/src/main/resources/examples/batch/githubEvents/rawdata/githubEvents_data.json
create mode 100644
pinot-tools/src/main/resources/examples/batch/githubEvents/sparkIngestionJobSpec.yaml
create mode 100644
pinot-tools/src/main/resources/examples/batch/starbucksStores/ingestionJobSpec.yaml
create mode 100644
pinot-tools/src/main/resources/examples/batch/starbucksStores/rawdata/data.csv
create mode 100644
pinot-tools/src/main/resources/examples/batch/starbucksStores/starbucksStores_offline_table_config.json
create mode 100644
pinot-tools/src/main/resources/examples/batch/starbucksStores/starbucksStores_schema.json
create mode 100644
pinot-tools/src/main/resources/examples/minions/batch/airlineStats/airlineStats_offline_table_config.json
copy pinot-tools/src/main/resources/examples/{stream =>
minions/batch}/airlineStats/airlineStats_schema.json (100%)
create mode 100644
pinot-tools/src/main/resources/examples/minions/batch/baseballStats/baseballStats_offline_table_config.json
copy pinot-tools/src/main/resources/examples/{ =>
minions}/batch/baseballStats/baseballStats_schema.json (100%)
copy pinot-tools/src/main/resources/examples/{ =>
minions}/batch/baseballStats/rawdata/baseballStats_data.csv (100%)
copy
pinot-tools/src/main/resources/examples/stream/meetupRsvp/{upsert_meetupRsvp_realtime_table_config.json
=> json_meetupRsvp_realtime_table_config.json} (65%)
create mode 100644
pinot-tools/src/main/resources/examples/stream/meetupRsvp/json_meetupRsvp_schema.json
create mode 100644
thirdeye/thirdeye-frontend/app/pods/components/composite-anomalies/data-table/template.hbs
create mode 100644
thirdeye/thirdeye-frontend/app/pods/components/composite-anomalies/entity-metrics-anomalies/component.js
create mode 100644
thirdeye/thirdeye-frontend/app/pods/components/composite-anomalies/entity-metrics-anomalies/template.hbs
create mode 100644
thirdeye/thirdeye-frontend/app/pods/components/composite-anomalies/group-constituents-anomalies/component.js
create mode 100644
thirdeye/thirdeye-frontend/app/pods/components/composite-anomalies/group-constituents-anomalies/template.hbs
create mode 100644
thirdeye/thirdeye-frontend/app/pods/custom/composite-anomalies-table/criticality/template.hbs
create mode 100644
thirdeye/thirdeye-frontend/app/pods/custom/composite-anomalies-table/current-predicted/template.hbs
create mode 100644
thirdeye/thirdeye-frontend/app/pods/custom/composite-anomalies-table/dimensions/template.hbs
create mode 100644
thirdeye/thirdeye-frontend/app/pods/custom/composite-anomalies-table/group-name/component.js
create mode 100644
thirdeye/thirdeye-frontend/app/pods/custom/composite-anomalies-table/group-name/template.hbs
create mode 100644
thirdeye/thirdeye-frontend/app/pods/custom/composite-anomalies-table/metric/template.hbs
create mode 100644
thirdeye/thirdeye-frontend/app/pods/custom/composite-anomalies-table/start-duration/component.js
create mode 100644 thirdeye/thirdeye-frontend/app/shared/metricFunnelColumns.js
create mode 100644
thirdeye/thirdeye-frontend/app/styles/pods/custom/composite-anomalies-table.scss
delete mode 100644
thirdeye/thirdeye-frontend/app/styles/pods/custom/parent-anomalies-table.scss
create mode 100644 thirdeye/thirdeye-frontend/app/utils/composite-anomalies.js
create mode 100644
thirdeye/thirdeye-frontend/tests/integration/pods/components/composite-anomalies/entity-metrics-anomalies/component-test.js
create mode 100644
thirdeye/thirdeye-frontend/tests/integration/pods/components/composite-anomalies/group-constituents-anomalies/component-test.js
create mode 100644 thirdeye/thirdeye-frontend/tests/unit/utils/utils-test.js
rename
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/datasource/pinot/{PqlUtils.java
=> SqlUtils.java} (92%)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
