This is an automated email from the ASF dual-hosted git repository.
apucher pushed a change to branch generator-sh-fix-20210222
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
from 04e2bcc Update superset docker image repo (#6477)
add dde3c18 TLS-support for client-pinot and pinot-internode connections
(#6418)
add 830eb94 [TE]frontend - Activate the composite anomalies route (#6472)
add ec15f41 Bump up quartz version to 2.3.2 to prevent XXE (#6484)
add 63d4266 Create CONTRIBUTING.md (#6481)
add e209230 Create CODE_OF_CONDUCT.md (#6482)
add e5bf05b Fix the overflow issue when loading the large dictionary into
the buffer (#6476)
add f292730 Fix missing copyrights (#6489)
add 98803dd Adding pinot minion metrics to jmx prometheus reporter (#6488)
add bacaed2 Guard against multiple consuming segments for same partition
(#6483)
add fde307b Adding scalar function JsonPathArray to extract arrays from
json (#6490)
add d7f4fec Support chained transform functions (#6495)
add 28bfad9 Fixing -segmentNameGeneratorType help message (#6498)
add cf35e6e Fixing pinot controller metrics prefix (#6499)
add 56df592 Fixing groovy parser with back slash (#6501)
add b8f70e7 Support generating derived column during segment load (#6494)
add 6c30c76 [TE] migrate PQL queries to standard SQL (#6486)
add 1ae53fe Adding cron scheduler metrics reporting (#6502)
add 0f398a7 Fixing quickstart launcher from IDE (#6508)
add 209f57c [TE]frontend - Add filtering support for Entity Monitoring
tables (#6514)
add 12ee45c use default broker and controller ports if not defined (#6515)
add 84ae4f1 [TE] SQL migration - escape the keywords (#6528)
add d1a0688 [THIRDEYE-3799](fix): the alert item style fix (#6504)
add a2f0dbd update superset k8s example to latest image (#6534)
add b51518f Fix hadoop batch ingest (#6506)
add 91d1679 Default to enable controller task scheduler for minion tasks
(#6536)
add 2be1520 Update DimTableSegmentAssignment to include both OFFLINE and
REALTIME servers (#6530)
add 421828d Allow tar gz with > 8gb size (#6533)
add 04265e0 JSON_MATCH predicate bug fix and test cases. (#6535)
add 20ff276 Add date_trunc to scalar functions to support date_trunc
during ingestion (#6538)
add 21b83d9 [TE] Update the formatter to handle composite alert to
extract metric names (#6544)
add 9708292 Move SegmentGenerationUtils from module
pinot-ingestion-common to pinot-common (#6537)
add c736fbe Changed the compat script to run single yaml file (#6550)
add 43a3701 [TE]frontend - Change the formatting of metric display in
exploration page (#6545)
add 27e34d7 Add Access Control for REST endpoints of Controller -
Declarative Approach (#6507)
add e62addb [TE]: fix eslint reported issue under the authenticators,
helpers, and mirage folder (#6551)
add 162d0e6 [TE]frontend - Upgrade from ember v2.18 to v3.0.0 (#6553)
add d03629e add optional http basic auth to pinot broker (#6552)
add 2ee498c [TE] escape Pinot timespect SQL query (#6566)
add 8013a2d Use minion data directory as tmp directory for
SegmentGenerationAndPushTask (#6560)
add a6196ba Adding metrics for minion tasks status (#6549)
add 779422f [TE] update date format when send to Jira API (#6563)
add e4503a3 Support S3 with server side encryption mode aws:kms (#6565)
add 503d0fe Update ADLSGen2PinotFS auth; Introduce unit tests (#6531)
add 4effcf9 Upgrade zookeeper version to 3.5.8 (#6558)
add 7ac8650 Increase code coverage for pinot-java-client (#6414)
add 0cff39d Fix the raw bytes column in real-time segment (#6574)
add 1e62f70 Implement execute method of prepared statement class in JDBC
(#6570)
add 29cf33b Enhance StarTreeIndexViewer to support multiple trees (#6569)
add 19e96b5 Optimize group-key generator (#6559)
add 0c4fb58 Add the isolation level config to kafka ingestion to support
Kafka transactions (#6580)
add 50ac78c URL encoded generated segment tar name (#6571)
add a733317 [TE]: Fix all eslint warning/error under utils and shared
folders (#6542)
add d0177f9 [TE]: Fix all eslint reported errors and warnings under
app/pods/components (#6568)
add abf21ed Compatibility test: streamOp (#6419)
add b97cce5 [TE]: fix eslint reported issue under the serializers, the
models, and the adapters folders (#6547)
add a95c9b2 [TE] Enable autotuning API (#6584)
add 7a94181 Fix QueryRunner for multiple runs (#6582)
add 1a5a3e0 [TE] Enable breakdown endpoint to handle forecast offset,
repair metric funnel async, and log stacktrace for more meaningful error logs
(#6578)
add e517dda Fixing schema and table config for githubEvents demo (#6585)
add 60b0c5f Handle creation of segments with 0 rows (#6466)
add 600b91b Update the default kafka consumer factory class name to use
Kafka 2.0 (#6587)
add 53788b3 Remove the legacy group key delimiter (#6589)
add 5225ce8 Adding validation for jsonExtractKey and jsonExtractScalar
functions (#6246)
add 4c3af59 Improve Real Time Provisioning Helper tool (#6546)
add d6fd42d Adding native parquet record reader support (#6525)
add d74224a Support validating jsonExtractKey and jsonExtractScalar with
underscore (#6594)
add b06a949 Add support for getObject for datatypes supported by Pinot
(#6581)
add 916494a Fix the SIGSEGV for large index (#6577)
add 307d918 Directly store values in GroupKey and avoid ser/de (#6593)
No new revisions were added by this update.
Summary of changes:
CODE_OF_CONDUCT.md | 96 +
CONTRIBUTING.md | 234 +
LICENSE-binary | 5 +-
compatibility-verifier/compCheck.sh | 38 +-
.../etc/jmx_prometheus_javaagent/configs/pinot.yml | 57 +-
kubernetes/helm/pinot/values.yaml | 1 +
kubernetes/helm/superset.yaml | 4 +-
.../pinot/broker/api/HttpRequesterIdentity.java | 44 +-
.../broker/api/resources/PinotClientRequest.java | 33 +-
.../broker/BasicAuthAccessControlFactory.java | 169 +
.../broker/broker/BrokerAdminApiApplication.java | 23 +-
.../broker/broker/helix/HelixBrokerStarter.java | 29 +-
.../SingleConnectionBrokerRequestHandler.java | 5 +-
.../routing/segmentpruner/EmptySegmentPruner.java | 130 +
.../segmentpruner/SegmentPrunerFactory.java | 16 +-
.../routing/timeboundary/TimeBoundaryManager.java | 19 +-
.../broker/broker/BasicAuthAccessControlTest.java | 152 +
.../LiteralOnlyBrokerRequestTest.java | 4 +-
.../routing/segmentpruner/SegmentPrunerTest.java | 195 +-
.../timeboundary/TimeBoundaryManagerTest.java | 19 +
pinot-clients/pinot-java-client/pom.xml | 5 +
.../apache/pinot/client/DynamicBrokerSelector.java | 12 +-
.../apache/pinot/client/ExternalViewReader.java | 8 +-
.../pinot/client/AggregationResultSetTest.java | 119 +
.../pinot/client/DynamicBrokerSelectorTest.java | 109 +
.../apache/pinot/client/ExecutionStatsTest.java | 182 +
.../pinot/client/ExternalViewReaderTest.java | 116 +
.../apache/pinot/client/GroupByResultSetTest.java | 122 +
.../pinot/client/ResultTableResultSetTest.java | 142 +
.../pinot/client/SelectionResultSetTest.java | 110 +
.../org/apache/pinot/client/TextTableTest.java | 74 +
.../pinot/client/PinotPreparedStatement.java | 35 +-
.../org/apache/pinot/client/PinotResultSet.java | 50 +
.../pinot/client/base/AbstractBaseResultSet.java | 20 +-
.../pinot/common/function/DateTimeUtils.java | 152 +
.../apache/pinot/common}/function/TimeZoneKey.java | 4 +-
.../common/function/scalar/DateTimeFunctions.java | 52 +
.../common/function/scalar/JsonFunctions.java | 55 +
.../pinot/common/metrics/ControllerGauge.java | 8 +-
.../pinot/common/metrics/ControllerMeter.java | 3 +-
.../pinot/common/metrics/ControllerTimer.java | 2 +-
.../generation}/SegmentGenerationUtils.java | 39 +-
.../apache/pinot/common/utils/CommonConstants.java | 16 +-
.../common/utils/FileUploadDownloadClient.java | 40 +-
.../pinot/common/utils/TarGzCompressionUtils.java | 1 +
.../org/apache/pinot/common/utils/ZkStarter.java | 35 +-
.../pinot/common/utils/helix/HelixHelper.java | 7 +
.../apache/pinot/sql/parsers/CalciteSqlParser.java | 39 +
.../src/main/resources}/zone-index.properties | 0
.../pinot/common/function/JsonFunctionsTest.java | 86 +
.../generation/SegmentGenerationUtilsTest.java | 126 +
pinot-connectors/pinot-spark-connector/pom.xml | 2 +-
.../spark/connector/PinotServerDataFetcher.scala | 2 +
.../apache/pinot/controller/ControllerConf.java | 32 +-
.../apache/pinot/controller/ControllerStarter.java | 32 +-
.../api/ControllerAdminApiApplication.java | 67 +-
.../pinot/controller/api/access/AccessControl.java | 31 +-
.../controller/api/access/AccessControlUtils.java | 98 +
.../pinot/controller/api/access/AccessType.java | 20 +-
.../{AccessControl.java => Authenticate.java} | 29 +-
.../api/access/AuthenticationFilter.java | 100 +
.../resources/LLCSegmentCompletionHandlers.java | 5 +
.../api/resources/PinotBrokerRestletResource.java | 3 +
.../api/resources/PinotClusterConfigs.java | 4 +
.../resources/PinotIngestionRestletResource.java | 4 +
.../PinotInstanceAssignmentRestletResource.java | 6 +
.../resources/PinotInstanceRestletResource.java | 7 +
.../api/resources/PinotQueryResource.java | 36 +-
.../api/resources/PinotSchemaRestletResource.java | 27 +-
.../api/resources/PinotSegmentRestletResource.java | 13 +
.../PinotSegmentUploadDownloadRestletResource.java | 8 +
.../api/resources/PinotTableIndexingConfigs.java | 3 +
.../api/resources/PinotTableMetadataConfigs.java | 3 +
.../api/resources/PinotTableRestletResource.java | 26 +-
.../api/resources/PinotTableSegmentConfigs.java | 3 +
.../api/resources/PinotTableTenantConfigs.java | 3 +
.../api/resources/PinotTaskRestletResource.java | 11 +
.../api/resources/PinotTenantRestletResource.java | 6 +
.../api/resources/ServerTableSizeReader.java | 2 +-
.../api/resources/ZookeeperResource.java | 4 +
.../helix/core/PinotHelixResourceManager.java | 22 +-
.../segment/OfflineDimTableSegmentAssignment.java | 18 +-
.../helix/core/minion/CronJobScheduleJob.java | 9 +
.../helix/core/minion/PinotTaskManager.java | 58 +-
.../helix/core/minion/TaskTypeMetricsUpdater.java | 31 +-
.../SegmentGenerationAndPushTaskGenerator.java | 41 +-
.../core/periodictask/ControllerPeriodicTask.java | 4 +
.../realtime/PinotLLCRealtimeSegmentManager.java | 34 +-
.../controller/recommender/io/InputManager.java | 5 +-
.../io/metadata/DateTimeFieldSpecMetadata.java | 42 +-
.../recommender/io/metadata/FieldMetadata.java | 6 +-
.../io/metadata/SchemaWithMetaData.java | 8 +-
.../io/metadata/TimeFieldSpecMetadata.java | 27 +-
.../io/metadata/TimeGranularitySpecMetadata.java | 41 +-
.../rules/io/params/RecommenderConstants.java | 2 +-
.../controller/util/CompletionServiceHelper.java | 7 +-
.../util/ConsumingSegmentInfoReader.java | 2 +-
.../pinot/controller/util/ListenerConfigUtil.java | 93 -
.../util/ServerSegmentMetadataReader.java | 2 +-
.../api/ConsumingSegmentInfoReaderTest.java | 2 +-
.../controller/api/PinotSegmentsMetadataTest.java | 2 +-
.../controller/api/ServerTableSizeReaderTest.java | 2 +-
.../pinot/controller/api/TableSizeReaderTest.java | 2 +-
.../api/access/AuthenticationFilterTest.java | 109 +
.../helix/core/PinotHelixResourceManagerTest.java | 4 +-
.../OfflineDimTableSegmentAssignmentTest.java | 19 +
.../controller/util/ListenerConfigUtilTest.java | 44 +-
.../controller/utils/SegmentMetadataMockUtils.java | 6 +-
pinot-core/pom.xml | 17 +-
.../core/data/function/FunctionEvaluator.java | 5 +
.../data/function/GroovyFunctionEvaluator.java | 5 +-
.../data/function/InbuiltFunctionEvaluator.java | 33 +-
.../data/function/TimeSpecFunctionEvaluator.java | 13 +-
.../data/readers/PinotSegmentColumnReader.java | 13 +
.../data/readers/PinotSegmentRecordReader.java | 54 +-
.../recordtransformer/ExpressionTransformer.java | 40 +-
.../indexsegment/immutable/EmptyIndexSegment.java | 120 +
.../immutable/ImmutableSegmentLoader.java | 3 +
.../indexsegment/mutable/MutableSegmentImpl.java | 2 +
.../operator/combine/GroupByCombineOperator.java | 5 +-
.../combine/GroupByOrderByCombineOperator.java | 83 +-
.../function/DateTruncTransformFunction.java | 124 +-
.../groupby/AggregationGroupByResult.java | 21 +-
.../groupby/DefaultGroupByExecutor.java | 6 +-
.../groupby/DictionaryBasedGroupKeyGenerator.java | 539 +-
.../aggregation/groupby/GroupKeyGenerator.java | 24 +-
.../NoDictionaryMultiColumnGroupKeyGenerator.java | 57 +-
.../NoDictionarySingleColumnGroupKeyGenerator.java | 169 +-
.../aggregation/groupby/utils/BytesToIdMap.java | 5 +
.../aggregation/groupby/utils/DoubleToIdMap.java | 5 +
.../aggregation/groupby/utils/FloatToIdMap.java | 5 +
.../aggregation/groupby/utils/IntToIdMap.java | 5 +
.../aggregation/groupby/utils/LongToIdMap.java | 5 +
.../aggregation/groupby/utils/StringToIdMap.java | 5 +
.../aggregation/groupby/utils/ValueToIdMap.java | 14 +
.../core/query/pruner/SegmentPrunerService.java | 17 +-
.../core/query/reduce/GroupByDataTableReducer.java | 52 +-
.../context/utils/QueryContextConverterUtils.java | 3 +
.../converter/RealtimeSegmentRecordReader.java | 3 +-
.../dictionary/BytesOffHeapMutableDictionary.java | 5 +
.../dictionary/BytesOnHeapMutableDictionary.java | 14 +-
.../creator/impl/SegmentColumnarIndexCreator.java | 53 +-
.../impl/SegmentIndexCreationDriverImpl.java | 15 +-
.../segment/index/datasource/EmptyDataSource.java | 94 +
.../segment/index/loader/IndexLoadingConfig.java | 17 +
.../segment/index/loader/SegmentPreProcessor.java | 6 +-
.../defaultcolumn/BaseDefaultColumnHandler.java | 300 +-
.../loader/defaultcolumn/DefaultColumnHandler.java | 5 +-
.../defaultcolumn/DefaultColumnHandlerFactory.java | 9 +-
.../defaultcolumn/V1DefaultColumnHandler.java | 16 +-
.../defaultcolumn/V3DefaultColumnHandler.java | 34 +-
.../segment/index/readers/BytesDictionary.java | 5 +
.../readers/ConstantValueBytesDictionary.java | 5 +
.../core/segment/index/readers/Dictionary.java | 21 +
.../core/segment/memory/BasePinotLBuffer.java | 9 +-
.../segment/store/SingleFileIndexDirectory.java | 18 +-
.../pinot/core/startree/OffHeapStarTreeNode.java | 4 +-
.../pinot/core/transport}/ListenerConfig.java | 36 +-
.../apache/pinot/core/transport/QueryRouter.java | 34 +-
.../apache/pinot/core/transport/QueryServer.java | 51 +
.../pinot/core/transport/ServerChannels.java | 44 +
.../pinot/core/transport/ServerInstance.java | 21 +
.../core/transport/ServerRoutingInstance.java | 30 +-
.../org/apache/pinot/core/transport/TlsConfig.java | 77 +
.../apache/pinot/core/util/ListenerConfigUtil.java | 219 +
.../apache/pinot/core/util/TableConfigUtils.java | 8 -
.../java/org/apache/pinot/core/util/TlsUtils.java | 269 +
.../data/function/ArithmeticFunctionsTest.java | 74 +
.../core/data/function/ArrayFunctionsTest.java | 143 +
...nctionsTest.java => DateTimeFunctionsTest.java} | 264 +-
.../core/data/function/JsonFunctionsTest.java | 128 +
.../data/readers/RecordReaderSampleDataTest.java | 12 +
.../ExpressionTransformerTest.java | 29 +
.../function/DateTruncTransformFunctionTest.java | 7 +-
.../JsonExtractScalarTransformFunctionTest.java | 57 +-
.../SegmentGenerationWithNoRecordsTest.java | 117 +
.../index/loader/SegmentPreProcessorTest.java | 104 +-
.../BaseDefaultColumnHandlerTest.java | 26 +-
.../pinot/core/util/TableConfigUtilsTest.java | 13 +-
.../queries/DistinctCountBitmapQueriesTest.java | 5 +-
.../pinot/queries/DistinctCountQueriesTest.java | 5 +-
.../DistinctCountThetaSketchQueriesTest.java | 2 +-
.../queries/FSTBasedRegexpLikeQueriesTest.java | 21 +-
.../apache/pinot/queries/FastHllQueriesTest.java | 8 +-
.../org/apache/pinot/queries/IdSetQueriesTest.java | 14 +-
.../pinot/queries/JsonMatchPredicateTest.java | 291 +
.../queries/PercentileTDigestQueriesTest.java | 16 +-
.../org/apache/pinot/queries/QueriesTestUtils.java | 4 +-
...SegmentPartitionedDistinctCountQueriesTest.java | 5 +-
.../pinot/queries/SerializedBytesQueriesTest.java | 8 +-
.../apache/pinot/queries/StUnionQueriesTest.java | 3 +-
.../apache/pinot/queries/TransformQueriesTest.java | 3 +-
.../DictionaryBasedGroupKeyGeneratorTest.java | 52 +-
.../groupby/NoDictionaryGroupKeyGeneratorTest.java | 9 +-
.../{GroupKeyTest.java => StringGroupKeyTest.java} | 4 +-
.../pinot/query/executor/QueryExecutorTest.java | 27 +-
.../converter/RealtimeSegmentConverterTest.java | 103 +-
.../segments/v1/creator/SegmentTestUtils.java | 10 +
.../src/test/resources/data/test_empty_data.json | 0
.../pinot/compat/tests/ClusterDescriptor.java | 17 +-
.../org/apache/pinot/compat/tests/StreamOp.java | 231 +-
.../tests/BaseClusterIntegrationTest.java | 11 +-
.../tests/ClusterIntegrationTestUtils.java | 60 +
...tlyOnceKafkaRealtimeClusterIntegrationTest.java | 65 +
.../tests/JsonPathClusterIntegrationTest.java | 240 +-
.../tests/OfflineClusterIntegrationTest.java | 55 +-
.../tests/SimpleMinionClusterIntegrationTest.java | 49 +-
.../tests/StarTreeClusterIntegrationTest.java | 2 +-
...onulls_default_column_test_extra_columns.schema | 8 +
.../compat-tests/configs/FeatureTest1-schema.json | 7 +
.../feature-test-1-realtime-stream-config.json | 8 +
.../configs/feature-test-1-realtime.json | 59 +
.../test/resources/compat-tests/kafka/topic1.cfg | 5 +
.../resources/compat-tests/stream-op-publish.yaml | 31 +
.../resources/compat-tests/table-segment-op.yaml | 40 +
.../org/apache/pinot/minion/MinionStarter.java | 6 +
.../SegmentGenerationAndPushTaskExecutor.java | 6 +-
.../apache/pinot/perf/BenchmarkIntOpenHashMap.java | 146 +
.../batch/common/SegmentGenerationTaskRunner.java | 4 +-
.../batch/common/SegmentGenerationUtilsTest.java | 56 -
.../pinot-batch-ingestion-hadoop/pom.xml | 16 +
.../batch/hadoop/HadoopSegmentCreationMapper.java | 36 +-
.../hadoop/HadoopSegmentGenerationJobRunner.java | 172 +-
.../hadoop/HadoopSegmentMetadataPushJobRunner.java | 1 +
.../hadoop/HadoopSegmentTarPushJobRunner.java | 3 +-
.../HadoopSegmentGenerationJobRunnerTest.java | 175 +
.../src/test/resources/log4j2.xml | 0
.../spark/SparkSegmentGenerationJobRunner.java | 15 +-
.../pinot-batch-ingestion-standalone/pom.xml | 9 +
.../standalone/SegmentGenerationJobRunner.java | 34 +-
.../standalone/SegmentGenerationJobRunnerTest.java | 136 +-
.../v0_deprecated/pinot-spark/pom.xml | 2 +-
pinot-plugins/pinot-file-system/pinot-adls/pom.xml | 38 +-
.../pinot/plugin/filesystem/ADLSGen2PinotFS.java | 202 +-
.../datalake/store/MockADLFileInputStream.java | 94 -
.../filesystem/test/ADLSGen2PinotFSTest.java | 395 +
.../plugin/filesystem/test/AzurePinotFSTest.java | 8 +-
.../org.mockito.plugins.MockMaker | 4 +
pinot-plugins/pinot-file-system/pinot-s3/pom.xml | 2 +-
.../apache/pinot/plugin/filesystem/S3PinotFS.java | 132 +-
...ordReader.java => ParquetAvroRecordReader.java} | 13 +-
.../parquet/ParquetNativeRecordExtractor.java | 263 +
.../parquet/ParquetNativeRecordReader.java | 129 +
.../inputformat/parquet/ParquetRecordReader.java | 41 +-
.../parquet/ParquetRecordReaderConfig.java | 52 +
.../plugin/inputformat/parquet/ParquetUtils.java | 24 +-
.../parquet/ParquetRecordReaderTest.java | 77 +-
.../src/test/resources/airlineStats.snappy.parquet | Bin 0 -> 1095802 bytes
.../test/resources/baseballStats.snappy.parquet | Bin 0 -> 1993064 bytes
.../src/test/resources/githubActivities.gz.parquet | Bin 0 -> 1610474 bytes
.../src/test/resources/githubEvents.snappy.parquet | Bin 0 -> 4537684 bytes
.../test/resources/starbucksStores.snappy.parquet | Bin 0 -> 451742 bytes
.../src/test/resources/test-comparison.gz.parquet | Bin 0 -> 10617970 bytes
.../test/resources/test-comparison.snappy.parquet | Bin 0 -> 18350 bytes
...test-file-with-int96-and-decimal.snappy.parquet | Bin 0 -> 19659 bytes
.../pinot-parquet/src/test/resources/users.parquet | Bin 0 -> 4065 bytes
.../KafkaPartitionLevelConnectionHandler.java | 3 +
.../kafka20/KafkaPartitionLevelStreamConfig.java | 17 +
.../kafka20/KafkaStreamLevelConsumerManager.java | 1 +
.../KafkaPartitionLevelStreamConfigTest.java | 34 +-
.../stream/kafka/KafkaStreamConfigProperties.java | 3 +
.../org/apache/pinot/server/conf/ServerConf.java | 20 +-
.../pinot/server/starter/ServerInstance.java | 48 +-
.../server/starter/helix/AdminApiApplication.java | 25 +-
.../server/starter/helix/HelixServerStarter.java | 40 +-
.../apache/pinot/server/api/AccessControlTest.java | 11 +-
.../apache/pinot/server/api/BaseResourceTest.java | 12 +-
.../apache/pinot/spi/data/readers/GenericRow.java | 57 +-
.../batch/spec/SegmentGenerationJobSpec.java | 13 +
.../pinot/spi/ingestion/batch/spec/TlsSpec.java | 45 +-
.../org/apache/pinot/spi/stream/StreamConfig.java | 2 +-
.../pinot/spi/utils/GroovyTemplateUtils.java | 4 +-
.../spi/data/readers/AbstractRecordReaderTest.java | 5 +-
.../resources/ingestion_job_spec_template.yaml | 7 +-
pinot-tools/pom.xml | 11 +
.../org/apache/pinot/tools/BootstrapTableTool.java | 32 +-
.../apache/pinot/tools/JsonIndexQuickStart.java | 2 +-
.../apache/pinot/tools/StarTreeIndexViewer.java | 217 +-
.../tools/admin/command/AddSchemaCommand.java | 18 +-
.../pinot/tools/admin/command/AddTableCommand.java | 17 +-
.../tools/admin/command/AddTenantCommand.java | 13 +-
.../tools/admin/command/BootstrapTableCommand.java | 6 +-
.../tools/admin/command/ChangeTableState.java | 18 +-
.../tools/admin/command/GenerateDataCommand.java | 10 +-
.../tools/admin/command/ImportDataCommand.java | 2 +-
.../command/LaunchDataIngestionJobCommand.java | 9 +
.../admin/command/OperateClusterConfigCommand.java | 15 +-
.../tools/admin/command/PostQueryCommand.java | 15 +-
.../command/RealtimeProvisioningHelperCommand.java | 44 +-
.../tools/admin/command/UploadSegmentCommand.java | 15 +-
.../pinot/tools/data/generator/BytesGenerator.java | 34 +-
.../pinot/tools/data/generator/DataGenerator.java | 104 +-
.../tools/data/generator/DataGeneratorSpec.java | 16 +-
.../tools/data/generator/GeneratorFactory.java | 15 +-
.../data/generator/MultiValueGeneratorHelper.java | 54 +
.../tools/data/generator/NumberGenerator.java | 34 +-
.../tools/data/generator/StringGenerator.java | 26 +-
.../pinot/tools/data/generator/TimeGenerator.java | 70 +
.../pinot/tools/perf/PerfBenchmarkRunner.java | 12 +-
.../org/apache/pinot/tools/perf/QueryRunner.java | 157 +-
.../realtime/provisioning/MemoryEstimator.java | 200 +
.../pinot/tools/utils/KafkaStarterUtils.java | 8 +
...lRequestMergedEvents_realtime_table_config.json | 1 -
.../pullRequestMergedEvents_schema.json | 12 +-
pinot-tools/src/main/resources/star-tree.html | 2 +-
.../generator/MultiValueGeneratorHelperTest.java | 49 +
.../tools/data/generator/TimeGeneratorTest.java | 52 +
.../realtime/provisioning/MemoryEstimatorTest.java | 129 +
.../memory_estimation/schema-with-metadata.json | 93 +
.../schema-with-metadata__dateTimeFieldSpec.json | 54 +
.../resources/memory_estimation/table-config.json | 49 +
pom.xml | 35 +-
.../resources/v2/RootCauseMetricResource.java | 26 +-
.../thirdeye/detection/yaml/YamlResource.java | 35 +
.../thirdeye/datasource/pinot/PqlUtilsTest.java | 26 +-
.../formatter/DetectionConfigFormatterTest.java | 20 +
.../sample-detection-composite-config.yml | 74 +
thirdeye/thirdeye-frontend/.eslintrc.js | 14 +-
thirdeye/thirdeye-frontend/README.md | 5 +
thirdeye/thirdeye-frontend/app/adapters/base.js | 21 +-
.../thirdeye-frontend/app/adapters/performance.js | 2 +-
.../thirdeye-frontend/app/adapters/share-config.js | 2 +-
thirdeye/thirdeye-frontend/app/adapters/share.js | 2 +-
.../app/authenticators/custom-ldap.js | 21 +-
.../thirdeye-frontend/app/helpers/color-delta.js | 4 +-
.../thirdeye-frontend/app/helpers/compute-color.js | 6 +-
.../app/helpers/compute-text-color.js | 8 +-
.../thirdeye-frontend/app/helpers/format-number.js | 4 +-
thirdeye/thirdeye-frontend/app/helpers/get-safe.js | 5 +-
thirdeye/thirdeye-frontend/app/helpers/set-has.js | 4 +-
thirdeye/thirdeye-frontend/app/mirage/config.js | 25 +-
.../thirdeye-frontend/app/mirage/endpoints/auth.js | 7 +-
.../app/mirage/endpoints/entity-mapping.js | 21 +-
.../app/mirage/endpoints/rootcause.js | 198 +-
.../app/mirage/endpoints/selfserve.js | 23 +-
.../app/mirage/factories/alert.js | 93 +-
.../app/mirage/factories/anomaly.js | 8 +-
.../app/mirage/fixtures/anomalyPerformance.js | 20 +-
.../app/mirage/fixtures/applicationAnomalies.js | 30 +-
.../thirdeye-frontend/app/mirage/fixtures/jobs.js | 2 +-
.../thirdeye-frontend/app/mirage/models/alert.js | 3 +-
.../thirdeye-frontend/app/mirage/models/anomaly.js | 3 +-
.../app/mirage/scenarios/default.js | 3 +-
.../thirdeye-frontend/app/models/performance.js | 2 +-
.../thirdeye-frontend/app/models/share-config.js | 2 +-
thirdeye/thirdeye-frontend/app/models/share.js | 2 +-
.../components/alert-report-modal/component.js | 19 +-
.../pods/components/anomaly-summary/component.js | 46 +-
.../entity-metrics-anomalies/component.js | 57 +-
.../group-constituents-anomalies/component.js | 52 +-
.../parent-anomalies/component.js | 53 +-
.../app/pods/components/cube-tooltip/component.js | 4 +-
.../pods/components/detection-health/component.js | 232 +-
.../pods/components/detection-yaml/component.js | 179 +-
.../app/pods/components/entity-filter/component.js | 32 +-
.../pods/components/filter-bar-input/component.js | 27 +-
.../app/pods/components/filter-bar/component.js | 160 +-
.../app/pods/components/filter-select/component.js | 43 +-
.../app/pods/components/heatmap-chart/component.js | 140 +-
.../app/pods/components/login-form/component.js | 9 +-
.../app/pods/components/login-form/template.hbs | 2 +-
.../modals/create-event-modal/component.js | 23 +-
.../modals/entity-mapping-modal/component.js | 186 +-
.../components/modals/error-modal/component.js | 1 -
.../components/range-pill-selectors/component.js | 72 +-
.../rootcause-callgraph-table/component.js | 50 +-
.../rootcause-chart-toolbar/component.js | 36 +-
.../pods/components/rootcause-chart/component.js | 433 +-
.../rootcause-custom-baseline/component.js | 68 +-
.../rootcause-data-indicator/component.js | 22 +-
.../rootcause-dimensions-algorithm/component.js | 267 +-
.../rootcause-dimensions-settings/component.js | 51 +-
.../rootcause-dimensions-table/component.js | 142 +-
.../pods/components/rootcause-header/component.js | 11 +-
.../pods/components/rootcause-heatmap/component.js | 411 +-
.../pods/components/rootcause-legend/component.js | 135 +-
.../pods/components/rootcause-legend/template.hbs | 8 +-
.../rootcause-select-comparison-range/component.js | 41 +-
.../rootcause-select-metric-dimension/component.js | 55 +-
.../rootcause-select-metric/component.js | 81 +-
.../pods/components/rootcause-table/component.js | 63 +-
.../pods/components/rootcause-trend/component.js | 231 +-
.../self-serve-alert-details/component.js | 233 +-
.../self-serve-alert-details/template.hbs | 6 +-
.../self-serve-alert-yaml-details/component.js | 10 +
.../self-serve-alert-yaml-details/template.hbs | 155 +-
.../self-serve-config-group-table/component.js | 4 +-
.../components/share-custom-template/component.js | 64 +-
.../pods/components/subscription-yaml/component.js | 29 +-
.../app/pods/components/te-modal/component.js | 46 +-
.../app/pods/components/te-modal/template.hbs | 6 +-
.../app/pods/components/te-navbar/component.js | 1 -
.../pods/components/timeseries-chart/component.js | 73 +-
.../app/pods/components/update-table/component.js | 1 -
.../anomalies-list/template.hbs | 6 +-
.../start-duration/template.hbs | 6 +-
.../app/pods/manage/explore/route.js | 57 +-
thirdeye/thirdeye-frontend/app/serializers/base.js | 6 +-
.../app/serializers/dimensions.js | 16 +-
.../app/serializers/share-config.js | 23 +-
.../thirdeye-frontend/app/serializers/share.js | 2 +-
.../app/shared/callgraphTableColumns.js | 12 +-
.../app/shared/dimensionAnalysisTableConfig.js | 8 +-
.../app/shared/dimensionsTableColumns.js | 21 +-
.../app/shared/eventTableColumns.js | 12 +-
.../app/shared/filterBarConfig.js | 190 +-
.../app/shared/metricsTableColumns.js | 24 +-
.../app/styles/shared/_styles.scss | 44 +-
.../app/utils/anomalies-tree-parser.js | 3 +
.../thirdeye-frontend/app/utils/api/self-serve.js | 43 +-
.../thirdeye-frontend/app/utils/build-tooltip.js | 45 +-
.../app/utils/composite-anomalies.js | 97 +-
thirdeye/thirdeye-frontend/app/utils/constants.js | 4 +-
.../app/utils/date-picker-utils.js | 31 +-
.../app/utils/manage-alert-utils.js | 98 +-
thirdeye/thirdeye-frontend/app/utils/rca-utils.js | 164 +-
thirdeye/thirdeye-frontend/app/utils/translate.js | 2 +-
thirdeye/thirdeye-frontend/app/utils/utils.js | 122 +-
thirdeye/thirdeye-frontend/app/utils/yaml-tools.js | 71 +-
.../thirdeye-frontend/config/ember-cli-update.json | 21 +
thirdeye/thirdeye-frontend/jsconfig.json | 20 +-
thirdeye/thirdeye-frontend/package.json | 12 +-
thirdeye/thirdeye-frontend/testem.js | 13 +-
.../tests/unit/utils/utils-test.js | 44 +
thirdeye/thirdeye-frontend/yarn.lock | 9049 ++++++++++----------
.../datasource/pinot/PinotDataSourceTimeQuery.java | 13 +-
.../datasource/pinot/PinotThirdEyeDataSource.java | 12 +-
.../pinot/{PqlUtils.java => SqlUtils.java} | 77 +-
.../pinot/resources/PinotDataSourceResource.java | 2 +-
.../validators/DetectionConfigValidator.java | 4 +-
.../formatter/DetectionConfigFormatter.java | 20 +-
.../notification/commons/ThirdEyeJiraClient.java | 9 +-
432 files changed, 19132 insertions(+), 9944 deletions(-)
create mode 100644 CODE_OF_CONDUCT.md
create mode 100644 CONTRIBUTING.md
copy
pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/groupby/utils/ValueToIdMap.java
=>
pinot-broker/src/main/java/org/apache/pinot/broker/api/HttpRequesterIdentity.java
(55%)
create mode 100644
pinot-broker/src/main/java/org/apache/pinot/broker/broker/BasicAuthAccessControlFactory.java
create mode 100644
pinot-broker/src/main/java/org/apache/pinot/broker/routing/segmentpruner/EmptySegmentPruner.java
create mode 100644
pinot-broker/src/test/java/org/apache/pinot/broker/broker/BasicAuthAccessControlTest.java
create mode 100644
pinot-clients/pinot-java-client/src/test/java/org/apache/pinot/client/AggregationResultSetTest.java
create mode 100644
pinot-clients/pinot-java-client/src/test/java/org/apache/pinot/client/DynamicBrokerSelectorTest.java
create mode 100644
pinot-clients/pinot-java-client/src/test/java/org/apache/pinot/client/ExecutionStatsTest.java
create mode 100644
pinot-clients/pinot-java-client/src/test/java/org/apache/pinot/client/ExternalViewReaderTest.java
create mode 100644
pinot-clients/pinot-java-client/src/test/java/org/apache/pinot/client/GroupByResultSetTest.java
create mode 100644
pinot-clients/pinot-java-client/src/test/java/org/apache/pinot/client/ResultTableResultSetTest.java
create mode 100644
pinot-clients/pinot-java-client/src/test/java/org/apache/pinot/client/SelectionResultSetTest.java
create mode 100644
pinot-clients/pinot-java-client/src/test/java/org/apache/pinot/client/TextTableTest.java
create mode 100644
pinot-common/src/main/java/org/apache/pinot/common/function/DateTimeUtils.java
rename {pinot-core/src/main/java/org/apache/pinot/core/operator/transform =>
pinot-common/src/main/java/org/apache/pinot/common}/function/TimeZoneKey.java
(98%)
rename
{pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-common/src/main/java/org/apache/pinot/plugin/ingestion/batch/common
=>
pinot-common/src/main/java/org/apache/pinot/common/segment/generation}/SegmentGenerationUtils.java
(83%)
rename
{pinot-core/src/main/resources/org/apache/pinot/core/operator/transform/function
=> pinot-common/src/main/resources}/zone-index.properties (100%)
create mode 100644
pinot-common/src/test/java/org/apache/pinot/common/segment/generation/SegmentGenerationUtilsTest.java
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/api/access/AccessControlUtils.java
copy
pinot-core/src/main/java/org/apache/pinot/core/data/function/FunctionEvaluator.java
=>
pinot-controller/src/main/java/org/apache/pinot/controller/api/access/AccessType.java
(64%)
copy
pinot-controller/src/main/java/org/apache/pinot/controller/api/access/{AccessControl.java
=> Authenticate.java} (59%)
create mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/api/access/AuthenticationFilter.java
copy
pinot-integration-tests/src/test/java/org/apache/pinot/compat/tests/ClusterDescriptor.java
=>
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/TaskTypeMetricsUpdater.java
(50%)
copy
pinot-core/src/main/java/org/apache/pinot/core/segment/index/loader/defaultcolumn/DefaultColumnHandlerFactory.java
=>
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/io/metadata/DateTimeFieldSpecMetadata.java
(50%)
copy
pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/groupby/utils/ValueToIdMap.java
=>
pinot-controller/src/main/java/org/apache/pinot/controller/recommender/io/metadata/TimeGranularitySpecMetadata.java
(56%)
delete mode 100644
pinot-controller/src/main/java/org/apache/pinot/controller/util/ListenerConfigUtil.java
create mode 100644
pinot-controller/src/test/java/org/apache/pinot/controller/api/access/AuthenticationFilterTest.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/indexsegment/immutable/EmptyIndexSegment.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/segment/index/datasource/EmptyDataSource.java
rename
{pinot-controller/src/main/java/org/apache/pinot/controller/api/listeners =>
pinot-core/src/main/java/org/apache/pinot/core/transport}/ListenerConfig.java
(67%)
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/transport/TlsConfig.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/util/ListenerConfigUtil.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/util/TlsUtils.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/data/function/ArithmeticFunctionsTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/data/function/ArrayFunctionsTest.java
rename
pinot-core/src/test/java/org/apache/pinot/core/data/function/{InbuiltFunctionsTest.java
=> DateTimeFunctionsTest.java} (58%)
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/data/function/JsonFunctionsTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/segment/index/creator/SegmentGenerationWithNoRecordsTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/queries/JsonMatchPredicateTest.java
rename
pinot-core/src/test/java/org/apache/pinot/query/aggregation/groupby/{GroupKeyTest.java
=> StringGroupKeyTest.java} (94%)
copy website/src/components/Step/styles.css =>
pinot-core/src/test/resources/data/test_empty_data.json (100%)
create mode 100644
pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/ExactlyOnceKafkaRealtimeClusterIntegrationTest.java
create mode 100644
pinot-integration-tests/src/test/resources/compat-tests/configs/feature-test-1-realtime-stream-config.json
create mode 100644
pinot-integration-tests/src/test/resources/compat-tests/configs/feature-test-1-realtime.json
create mode 100644
pinot-integration-tests/src/test/resources/compat-tests/kafka/topic1.cfg
create mode 100644
pinot-integration-tests/src/test/resources/compat-tests/stream-op-publish.yaml
create mode 100644
pinot-integration-tests/src/test/resources/compat-tests/table-segment-op.yaml
create mode 100644
pinot-perf/src/main/java/org/apache/pinot/perf/BenchmarkIntOpenHashMap.java
delete mode 100644
pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-common/src/test/java/org/apache/pinot/plugin/ingestion/batch/common/SegmentGenerationUtilsTest.java
create mode 100644
pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-hadoop/src/test/java/org/apache/pinot/plugin/ingestion/batch/hadoop/HadoopSegmentGenerationJobRunnerTest.java
copy {pinot-tools =>
pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-hadoop}/src/test/resources/log4j2.xml
(100%)
delete mode 100644
pinot-plugins/pinot-file-system/pinot-adls/src/test/java/com/microsoft/azure/datalake/store/MockADLFileInputStream.java
create mode 100644
pinot-plugins/pinot-file-system/pinot-adls/src/test/java/org/apache/pinot/plugin/filesystem/test/ADLSGen2PinotFSTest.java
create mode 100644
pinot-plugins/pinot-file-system/pinot-adls/src/test/resources/mockito-extensions/org.mockito.plugins.MockMaker
copy
pinot-plugins/pinot-input-format/pinot-parquet/src/main/java/org/apache/pinot/plugin/inputformat/parquet/{ParquetRecordReader.java
=> ParquetAvroRecordReader.java} (78%)
create mode 100644
pinot-plugins/pinot-input-format/pinot-parquet/src/main/java/org/apache/pinot/plugin/inputformat/parquet/ParquetNativeRecordExtractor.java
create mode 100644
pinot-plugins/pinot-input-format/pinot-parquet/src/main/java/org/apache/pinot/plugin/inputformat/parquet/ParquetNativeRecordReader.java
create mode 100644
pinot-plugins/pinot-input-format/pinot-parquet/src/main/java/org/apache/pinot/plugin/inputformat/parquet/ParquetRecordReaderConfig.java
create mode 100644
pinot-plugins/pinot-input-format/pinot-parquet/src/test/resources/airlineStats.snappy.parquet
create mode 100644
pinot-plugins/pinot-input-format/pinot-parquet/src/test/resources/baseballStats.snappy.parquet
create mode 100644
pinot-plugins/pinot-input-format/pinot-parquet/src/test/resources/githubActivities.gz.parquet
create mode 100644
pinot-plugins/pinot-input-format/pinot-parquet/src/test/resources/githubEvents.snappy.parquet
create mode 100644
pinot-plugins/pinot-input-format/pinot-parquet/src/test/resources/starbucksStores.snappy.parquet
create mode 100644
pinot-plugins/pinot-input-format/pinot-parquet/src/test/resources/test-comparison.gz.parquet
create mode 100644
pinot-plugins/pinot-input-format/pinot-parquet/src/test/resources/test-comparison.snappy.parquet
create mode 100644
pinot-plugins/pinot-input-format/pinot-parquet/src/test/resources/test-file-with-int96-and-decimal.snappy.parquet
create mode 100644
pinot-plugins/pinot-input-format/pinot-parquet/src/test/resources/users.parquet
rename
pinot-controller/src/main/java/org/apache/pinot/controller/api/listeners/TlsConfiguration.java
=>
pinot-spi/src/main/java/org/apache/pinot/spi/ingestion/batch/spec/TlsSpec.java
(64%)
copy
pinot-common/src/main/java/org/apache/pinot/common/metrics/ControllerTimer.java
=>
pinot-tools/src/main/java/org/apache/pinot/tools/data/generator/BytesGenerator.java
(56%)
create mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/data/generator/MultiValueGeneratorHelper.java
create mode 100644
pinot-tools/src/main/java/org/apache/pinot/tools/data/generator/TimeGenerator.java
create mode 100644
pinot-tools/src/test/java/org/apache/pinot/tools/data/generator/MultiValueGeneratorHelperTest.java
create mode 100644
pinot-tools/src/test/java/org/apache/pinot/tools/data/generator/TimeGeneratorTest.java
create mode 100644
pinot-tools/src/test/java/org/apache/pinot/tools/realtime/provisioning/MemoryEstimatorTest.java
create mode 100644
pinot-tools/src/test/resources/memory_estimation/schema-with-metadata.json
create mode 100644
pinot-tools/src/test/resources/memory_estimation/schema-with-metadata__dateTimeFieldSpec.json
create mode 100644
pinot-tools/src/test/resources/memory_estimation/table-config.json
create mode 100644
thirdeye/thirdeye-dashboard/src/test/resources/sample-detection-composite-config.yml
create mode 100644 thirdeye/thirdeye-frontend/config/ember-cli-update.json
create mode 100644 thirdeye/thirdeye-frontend/tests/unit/utils/utils-test.js
rename
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/datasource/pinot/{PqlUtils.java
=> SqlUtils.java} (89%)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
