This is an automated email from the ASF dual-hosted git repository.
apucher pushed a change to branch tls-integration-test
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
discard 3037fbf patch BasicAuthRealtimeIntegrationTest with client transport
changes
discard ec47454 TLS & basic auth cross-integration test
discard 61f58f5 add protocol override config to
ServerSegmentCompletionProtocolHandler
discard 03b1348 plumbing prep for TLS-enabled tests
discard 702f325 support TLS in pinot-transport
add 08ed07c Simplify the schema check in SegmentConverter (#6931)
add 9fb6d4d Convert collections not unnested to json (#6918)
add 98891a3 Fix rules to include DateTime column (#6937) (#6938)
add 9becc57 Implement getRecord for immutableSegmentImpl (#6924)
add 1b0bb72 Update Quickstart to take tmp dir as a parameter (#5445)
add 57c1603 Fix CAST transform function for chained transforms (#6941)
add fe322f5 Adding a new Controller API to retrieve ingestion status for
realtime… (#6890)
add 6f4790f add command to infer pinot schema from json data (#6930)
add 7b44624 Fix the exception thrown in the case that a specified table
name does not exist (#6328) (#6765)
add e400566 Add the complex-type support to decoder/reader (#6945)
add f4ae3e0 Add collectionToJsonMode to schema inference (#6946)
add d63d6c5 Fix NPE issue when consumed kafka message is null or the
record value is null. (#6950)
add 16f94a1 Rename some complex type configs (#6953)
add 5188d4e Add NETTY_CONNECTION_SEND_REQUEST_LATENCY to broker timer
metrics (#6936)
add c5bca8a fixed Pinot ui can't show all the pinot servers (#6956)
add 08b909c Move TableDataManager to segment-local to unblock partial
upsert feature (#6947)
add 4d834ea Update JSON datatype test cases + minor JSON index bug fix.
(#6951)
add 0d6ca90 fixed UI crashing when server or brokers are offline (#6935)
add f15225f fixed showing all pinot servers in tenant details (#6934)
add 71e156c Make Pinot Broker/Server/Minion can start by just passing a
config file (#5446)
add 1209bcc Use stored types instead of derived types in Forward Indices.
(#6960)
add 958b4d4 Enabling compatibility tests in the script (#6959)
add 9adbe55 Include complex-type-handling quickstarts in the assembler
(#6961)
add 8878df5 Update LICENSE & NOTICE (#6944)
add 2b55004 SegmentDirectoryLoader interface and local impl (#6949)
add 4675b4a Prefetch call to fetch buffers of columns seen in the query
(#6967)
add 1efde00 Fixing comparison and hashcode for Schema, FieldSpec and
ColumnMetadata (#6965)
add 3c7dfcd Optimize TIME_CONVERT/DATE_TIME_CONVERT predicates (#6957)
add 0185482 Implement PassThroughTransformOperator to optimize select
queries(#6972) (#6973)
add c76ce2a Support null value fields in generic row ser/de (#6968)
add b8a92c4 Fix deep store directory structure (#6976)
add 6388c77 Followup to override operator name (#6972) (#6986)
add ab5ace8 feature/#6766 JSON and Startree index information in API
(#6873)
add 0b5dcb7 Unify the minion plug-in package regex path (#6980)
add 856fe09 Add annotation for deprecated fields segmentPushType and
segmentPushFrequency in TableConfigBuilder (#6993)
add d615418 Add Kinesis Stream Ingestion Plugin (#6661)
add e0b2be8 Normalize LHS and RHS numerical types for >, >=, <, and <=
operators. (#6927)
add 6fcccfc Add genericRow file reader and writer (#6997)
add 4fb234a Add constants and metadata properties for MergeRollupTask:
(#6932)
add 211cf8a Fix the race condition in realtime text index refresh thread
(#6858) (#6990)
add 57d1a2a Remove constructor from base class ColumnIndexDirectory
(#6996)
add 79507da Enhance debug endpoint for table debugging. (#6977)
add 9040571 Fix partial comparison for realtime table and allow keep
cluster on failure (#6969)
add 02d54b9 Fix broker log: keep all the pairs together, and the query at
the end (#7003)
add dbae826 Fix NPE in NumericalFilterOptimizer due to IS NULL and IS NOT
NULL operator. (#7001)
add 637cc34 Move file name constants to V1Constants (#7006)
add 8509f42 support TLS in pinot-transport
add 060edc8 plumbing prep for TLS-enabled tests
add 3cf1177 add protocol override config to
ServerSegmentCompletionProtocolHandler
add 1bd02c4 TLS & basic auth cross-integration test
add 9db052f patch BasicAuthRealtimeIntegrationTest with client transport
changes
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (3037fbf)
\
N -- N -- N refs/heads/tls-integration-test (9db052f)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/workflows/scripts/.pinot_quickstart.sh | 100 ++++-
LICENSE | 73 ----
LICENSE-binary | 109 ++----
NOTICE-binary | 115 +++---
compatibility-verifier/README.md | 12 +-
compatibility-verifier/checkoutAndBuild.sh | 4 +-
compatibility-verifier/compCheck.sh | 325 ++++++++++++----
.../queries/feature-test-2-sql-realtime.queries | 4 +
.../feature-test-2-sql-realtime.results | 3 +-
licenses-binary/LICENSE-angular.txt | 21 -
licenses-binary/LICENSE-aopalliance.txt | 2 -
.../{LICENSE-handlebars.txt => LICENSE-azure.txt} | 12 +-
.../LICENSE-codehaus-animal-sniffer.txt | 4 +-
licenses-binary/LICENSE-codemirror.txt | 21 -
licenses-binary/LICENSE-cpl-1.0.txt | 88 -----
licenses-binary/LICENSE-edl-1.0.txt | 13 +
licenses-binary/LICENSE-esotericsoftware.txt | 11 -
licenses-binary/LICENSE-foundation.txt | 22 --
...ENSE-fusesource.txt => LICENSE-google-auth.txt} | 5 +-
...SE-protobuf.txt => LICENSE-google-protobuf.txt} | 0
licenses-binary/LICENSE-jcabi.txt | 2 +-
licenses-binary/LICENSE-jcraft.txt | 32 --
licenses-binary/LICENSE-jline.txt | 32 --
.../LICENSE-jquery-requestAnimationFrame.txt | 22 --
licenses-binary/LICENSE-jquery-sizzle.txt | 36 --
licenses-binary/LICENSE-jquery.txt | 20 -
licenses-binary/LICENSE-js-beautify.txt | 9 -
licenses-binary/LICENSE-mimepull.txt | 28 --
licenses-binary/LICENSE-normalize.txt | 21 -
licenses-binary/LICENSE-paranamer.txt | 28 --
licenses-binary/LICENSE-py4j.txt | 26 --
.../LICENSE-reactive-streams.txt | 8 +-
licenses-binary/LICENSE-reflections.txt | 13 +
.../{LICENSE-scala.txt => LICENSE-scala-lang.txt} | 0
...NSE-jquery-datatables.txt => LICENSE-stax2.txt} | 4 +-
licenses-binary/LICENSE-underscore.txt | 23 --
licenses-binary/LICENSE-xmlenc.txt | 27 --
licenses-binary/LICENSE-zstd-jni.txt | 26 ++
licenses/LICENSE-angular.txt | 21 -
licenses/LICENSE-codemirror.txt | 21 -
licenses/LICENSE-foundation.txt | 22 --
licenses/LICENSE-jquery-datatables.txt | 12 -
licenses/LICENSE-jquery-requestAnimationFrame.txt | 22 --
licenses/LICENSE-jquery-sizzle.txt | 36 --
licenses/LICENSE-jquery.txt | 20 -
licenses/LICENSE-normalize.txt | 21 -
licenses/LICENSE-underscore.txt | 23 --
.../broker/broker/BrokerAdminApiApplication.java | 6 +-
.../broker/broker/helix/HelixBrokerStarter.java | 34 +-
.../broker/pruner/PartitionZKMetadataPruner.java | 4 +-
.../requesthandler/BaseBrokerRequestHandler.java | 23 +-
.../segmentpruner/PartitionSegmentPruner.java | 4 +-
.../broker/broker/HelixBrokerStarterTest.java | 7 +-
.../broker/pruner/SegmentZKMetadataPrunerTest.java | 2 +-
.../routing/segmentpruner/SegmentPrunerTest.java | 2 +-
.../pinot/common/exception/QueryException.java | 13 +-
.../metadata/segment/SegmentPartitionMetadata.java | 1 +
.../apache/pinot/common/metrics/BrokerTimer.java | 4 +
.../apache/pinot/common/metrics/ServerTimer.java | 4 +-
.../apache/pinot/common/minion/Granularity.java | 12 +-
.../common/minion/MergeRollupTaskMetadata.java | 87 +++++
.../common/minion/MinionTaskMetadataUtils.java | 28 ++
.../response/broker/BrokerResponseNative.java | 2 +
.../common/restlet/resources/SegmentErrorInfo.java | 100 +++++
.../restlet/resources/SegmentServerDebugInfo.java | 68 ++++
.../apache/pinot/common/utils/SegmentUtils.java | 2 +-
.../common/metadata/SegmentZKMetadataTest.java | 2 +-
.../segment/ColumnPartitionMetadataTest.java | 1 +
.../common/utils/config/TableConfigSerDeTest.java | 8 +-
.../apache/pinot/controller/ControllerConf.java | 24 +-
.../api/ControllerAdminApiApplication.java | 6 +-
.../pinot/controller/api/debug/TableDebugInfo.java | 119 ++++--
.../resources/LLCSegmentCompletionHandlers.java | 4 +-
.../PinotSegmentUploadDownloadRestletResource.java | 8 +-
.../api/resources/PinotTableRestletResource.java | 38 ++
.../api/resources/TableDebugResource.java | 220 +++++++++--
.../helix/ControllerRequestURLBuilder.java | 4 +-
.../helix/core/PinotHelixResourceManager.java | 11 +
.../segment/OfflineSegmentAssignment.java | 2 +-
.../helix/core/minion/ClusterInfoAccessor.java | 19 +
.../minion/generator/TaskGeneratorRegistry.java | 2 +-
.../realtime/PinotLLCRealtimeSegmentManager.java | 6 +-
.../segment/CommittingSegmentDescriptor.java | 2 +-
.../helix/core/util/ZKMetadataUtils.java | 6 +-
.../controller/recommender/io/InputManager.java | 22 +-
.../realtime/provisioning/MemoryEstimator.java | 2 +-
.../recommender/rules/impl/BloomFilterRule.java | 12 +-
.../recommender/rules/impl/FlagQueryRule.java | 13 +-
.../NoDictionaryOnHeapDictionaryJointRule.java | 2 +-
.../rules/impl/PinotTablePartitionRule.java | 2 +-
.../rules/io/params/RecommenderConstants.java | 8 +-
.../utils/QueryInvertedSortedIndexRecommender.java | 6 +-
.../util/ConsumingSegmentInfoReader.java | 48 +++
.../app/components/Homepage/ClusterConfig.tsx | 1 +
.../app/components/Homepage/InstanceTable.tsx | 1 +
.../src/main/resources/app/pages/HomePage.tsx | 10 +-
.../src/main/resources/app/pages/Tenants.tsx | 5 +-
.../api/ConsumingSegmentInfoReaderTest.java | 76 +++-
.../OfflineReplicaGroupSegmentAssignmentTest.java | 2 +-
.../PinotLLCRealtimeSegmentManagerTest.java | 2 +-
.../controller/recommender/TestConfigEngine.java | 23 +-
.../controller/utils/SegmentMetadataMockUtils.java | 6 +-
.../BloomFilterInputWithDateTimeColumn.json | 64 +++
.../resources/recommenderInput/FlagQueryInput.json | 15 +-
.../apache/pinot/core/common/MinionConstants.java | 19 +-
.../core/data/manager/BaseTableDataManager.java | 34 +-
.../core/data/manager/InstanceDataManager.java | 1 +
.../manager/offline/DimensionTableDataManager.java | 9 +-
.../offline/ImmutableSegmentDataManager.java | 2 +-
.../manager/offline/TableDataManagerProvider.java | 19 +-
.../realtime/HLRealtimeSegmentDataManager.java | 1 +
.../realtime/LLRealtimeSegmentDataManager.java | 35 +-
.../realtime/RealtimeSegmentDataManager.java | 11 +-
.../manager/realtime/RealtimeTableDataManager.java | 4 +-
.../core/metadata/DefaultMetadataExtractor.java | 2 +-
.../pinot/core/minion/RawIndexConverter.java | 6 +-
.../apache/pinot/core/minion/SegmentConverter.java | 18 +-
.../apache/pinot/core/minion/SegmentPurger.java | 2 +-
.../minion/rollup/MergeRollupSegmentConverter.java | 6 +-
.../operator/blocks/PassThroughTransformBlock.java | 32 +-
.../pinot/core/operator/blocks/TransformBlock.java | 4 +-
.../transform/PassThroughTransformOperator.java | 57 +++
.../core/operator/transform/TransformOperator.java | 6 +-
.../transform/function/BaseTransformFunction.java | 22 +-
.../transform/function/CastTransformFunction.java | 191 ++++++++-
.../apache/pinot/core/plan/TransformPlanNode.java | 9 +-
.../core/plan/maker/InstancePlanMakerImplV2.java | 6 +
.../query/executor/ServerQueryExecutorV1Impl.java | 4 +-
.../pinot/core/query/optimizer/QueryOptimizer.java | 9 +-
.../filter/MergeRangeFilterOptimizer.java | 101 +----
.../optimizer/filter/NumericalFilterOptimizer.java | 321 +++++++++++----
.../pinot/core/query/optimizer/filter/Range.java | 119 ++++++
.../filter/TimePredicateFilterOptimizer.java | 429 +++++++++++++++++++++
.../processing/collector/ConcatCollector.java | 143 +++----
.../processing/collector/SortOrderComparator.java | 72 ++++
.../genericrow/GenericRowDeserializer.java | 223 +++++++++++
.../genericrow/GenericRowFileReader.java | 81 ++++
.../genericrow/GenericRowFileWriter.java | 72 ++++
.../genericrow/GenericRowSerializer.java | 237 ++++++++++++
.../partitioner/TableConfigPartitioner.java | 2 +-
.../pinot/core/transport/AsyncQueryResponse.java | 6 +-
.../core/transport/InstanceRequestHandler.java | 27 +-
.../apache/pinot/core/transport/QueryRouter.java | 2 +-
.../pinot/core/transport/ServerChannels.java | 18 +-
.../pinot/core/transport/ServerResponse.java | 17 +-
.../pinot/core/util/GenericRowSerDeUtils.java | 316 ---------------
.../data/manager/BaseTableDataManagerTest.java | 6 +-
.../offline/DimensionTableDataManagerTest.java | 6 +-
.../realtime/LLRealtimeSegmentDataManagerTest.java | 6 +-
.../pinot/core/minion/SegmentPurgerTest.java | 17 +-
.../function/BaseTransformFunctionTest.java | 30 ++
.../function/CastTransformFunctionTest.java | 30 +-
.../core/query/executor/QueryExecutorTest.java | 6 +-
.../filter/NumericalFilterOptimizerTest.java | 395 +++++++++----------
.../filter/TimePredicateFilterOptimizerTest.java | 200 ++++++++++
.../query/pruner/ColumnValueSegmentPrunerTest.java | 2 +-
.../framework/SegmentProcessingFrameworkTest.java | 2 +-
.../processing/genericrow/GenericRowSerDeTest.java | 137 +++++++
.../pinot/queries/JsonDataTypeQueriesTest.java | 77 +++-
.../queries/SegmentWithNullValueVectorTest.java | 6 +-
.../converter/RealtimeSegmentConverterTest.java | 2 +-
.../data/test_null_value_vector_data.json | 12 -
.../data/test_null_value_vector_pinot_schema.json | 26 --
.../test/resources/data/test_upsert_schema.json | 21 -
pinot-distribution/pinot-assembly.xml | 4 +
pinot-distribution/pom.xml | 8 +
.../org/apache/pinot/compat/tests/SegmentOp.java | 4 +-
.../pinot/compat/tests/SqlResultComparator.java | 263 +++++++++++--
.../tests/BaseClusterIntegrationTestSet.java | 13 +-
.../tests/BasicAuthBatchIntegrationTest.java | 28 +-
.../tests/BasicAuthRealtimeIntegrationTest.java | 8 +-
.../pinot/integration/tests/ClusterTest.java | 40 +-
...vertToRawIndexMinionClusterIntegrationTest.java | 4 +-
.../tests/OfflineClusterIntegrationTest.java | 27 +-
...fflineSegmentsMinionClusterIntegrationTest.java | 2 +-
...PartitionLLCRealtimeClusterIntegrationTest.java | 2 +-
.../tests/ServerStarterIntegrationTest.java | 5 +-
.../tests/SimpleMinionClusterIntegrationTest.java | 161 +-------
.../minion/tasks/TestEventObserverFactory.java | 73 ++++
.../minion/tasks/TestTaskExecutorFactory.java | 81 ++++
.../plugin/minion/tasks/TestTaskGenerator.java | 72 ++++
.../pinot/minion/MinionAdminApiApplication.java | 6 +-
.../org/apache/pinot/minion/MinionStarter.java | 19 +-
.../minion/event/EventObserverFactoryRegistry.java | 9 +-
.../executor/TaskExecutorFactoryRegistry.java | 6 +-
.../pinot/perf/BenchmarkFixedBitIntReader.java | 2 +-
.../BenchmarkFixedBitSVForwardIndexReader.java | 2 +-
.../pinot/perf/BenchmarkOfflineIndexReader.java | 18 +-
.../pinot/perf/BenchmarkPinotDataBitSet.java | 2 +-
.../pinot/perf/BenchmarkPinotDataBuffer.java | 2 +-
.../apache/pinot/perf/BenchmarkQueryEngine.java | 2 +-
.../org/apache/pinot/perf/RawIndexBenchmark.java | 2 +-
.../hadoop/job/HadoopSegmentPreprocessingJob.java | 2 +-
.../job/partitioners/PartitionFunctionFactory.java | 8 +-
pinot-plugins/pinot-file-system/pinot-s3/pom.xml | 55 ++-
.../pinot/plugin/inputformat/avro/AvroUtils.java | 61 ++-
.../plugin/inputformat/avro/AvroUtilsTest.java | 20 +-
.../minion/tasks/merge_rollup/MergeProperties.java | 49 ++-
.../tasks/merge_rollup/MergeRollupTaskUtils.java | 117 ++++++
.../merge_rollup/MergeRollupTaskUtilsTest.java | 83 ++++
.../RealtimeToOfflineSegmentsTaskExecutorTest.java | 4 +-
.../filebased/FileBasedSegmentWriterTest.java | 2 +-
.../kafka20/KafkaPartitionLevelConsumer.java | 7 +-
.../pinot-stream-ingestion/pinot-kinesis/pom.xml | 218 +++++++++++
.../pinot/plugin/stream/kinesis/KinesisConfig.java | 84 ++++
.../stream/kinesis/KinesisConnectionHandler.java | 96 +++++
.../plugin/stream/kinesis/KinesisConsumer.java | 202 ++++++++++
.../stream/kinesis/KinesisConsumerFactory.java | 67 ++++
.../stream/kinesis/KinesisMsgOffsetFactory.java | 51 +++
.../kinesis/KinesisPartitionGroupOffset.java | 92 +++++
.../plugin/stream/kinesis/KinesisRecordsBatch.java | 80 ++++
.../kinesis/KinesisStreamMetadataProvider.java | 191 +++++++++
.../plugin/stream/kinesis/KinesisConsumerTest.java | 202 ++++++++++
.../kinesis/KinesisStreamMetadataProviderTest.java | 172 +++++++++
pinot-plugins/pinot-stream-ingestion/pom.xml | 1 +
pinot-plugins/pom.xml | 12 +
.../local}/data/manager/SegmentDataManager.java | 2 +-
.../local}/data/manager/TableDataManager.java | 26 +-
.../data/manager}/TableDataManagerConfig.java | 9 +-
.../indexsegment/immutable/EmptyIndexSegment.java | 4 +-
.../immutable/ImmutableSegmentImpl.java | 32 +-
.../immutable/ImmutableSegmentLoader.java | 53 ++-
.../indexsegment/mutable/IntermediateSegment.java | 4 +-
.../indexsegment/mutable/MutableSegmentImpl.java | 14 +-
.../local/io/reader/impl/FixedBitIntReader.java | 2 +-
.../impl/FixedByteSingleValueMultiColReader.java | 2 +-
.../readerwriter/PinotDataBufferMemoryManager.java | 2 +-
.../RealtimeIndexOffHeapMemoryManager.java | 2 +-
...ixedByteSingleValueMultiColumnReaderWriter.java | 2 +-
.../local/io/util/FixedBitIntReaderWriter.java | 2 +-
.../local/io/util/FixedBitIntReaderWriterV2.java | 4 +-
.../local/io/util/FixedByteValueReaderWriter.java | 2 +-
.../segment/local/io/util/PinotDataBitSet.java | 2 +-
.../segment/local/io/util/PinotDataBitSetV2.java | 4 +-
.../local/io/util/VarLengthValueReader.java | 2 +-
.../local/io/util/VarLengthValueWriter.java | 2 +-
.../local/io/writer/impl/DirectMemoryManager.java | 2 +-
.../writer/impl/FixedBitMVForwardIndexWriter.java | 2 +-
.../writer/impl/FixedBitSVForwardIndexWriter.java | 2 +-
.../impl/FixedByteSingleValueMultiColWriter.java | 2 +-
.../local/io/writer/impl/MmapMemoryManager.java | 2 +-
.../writer/impl/MutableOffHeapByteArrayStore.java | 2 +-
.../local/loader/LocalSegmentDirectoryLoader.java | 50 +++
.../dictionary/BaseOffHeapMutableDictionary.java | 2 +-
.../impl/dictionary/OffHeapMutableBytesStore.java | 2 +-
.../forward/FixedByteMVMutableForwardIndex.java | 2 +-
.../forward/FixedByteSVMutableForwardIndex.java | 2 +-
.../RealtimeLuceneIndexReaderRefreshThread.java | 52 +--
.../recordtransformer/ComplexTypeTransformer.java | 176 +++++++--
.../creator/impl/SegmentColumnarIndexCreator.java | 7 +-
.../creator/impl/SegmentDictionaryCreator.java | 3 +-
.../impl/SegmentIndexCreationDriverImpl.java | 3 +-
.../impl/bloom/OnHeapGuavaBloomFilterCreator.java | 2 +-
.../fwd/MultiValueUnsortedForwardIndexCreator.java | 2 +-
.../fwd/SingleValueFixedByteRawIndexCreator.java | 2 +-
.../fwd/SingleValueSortedForwardIndexCreator.java | 2 +-
.../SingleValueUnsortedForwardIndexCreator.java | 2 +-
.../fwd/SingleValueVarByteRawIndexCreator.java | 2 +-
.../impl/inv/BitmapInvertedIndexWriter.java | 2 +-
.../inv/OffHeapBitmapInvertedIndexCreator.java | 4 +-
.../impl/inv/OnHeapBitmapInvertedIndexCreator.java | 2 +-
.../creator/impl/inv/RangeIndexCreator.java | 4 +-
.../impl/inv/geospatial/BaseH3IndexCreator.java | 2 +-
.../impl/inv/geospatial/OffHeapH3IndexCreator.java | 2 +-
.../impl/inv/json/BaseJsonIndexCreator.java | 4 +-
.../impl/inv/json/OffHeapJsonIndexCreator.java | 2 +-
.../impl/inv/text/LuceneFSTIndexCreator.java | 2 +-
.../impl/nullvalue/NullValueVectorCreator.java | 2 +-
.../stats/AbstractColumnStatisticsCollector.java | 2 +-
.../creator/impl/text/LuceneTextIndexCreator.java | 4 +-
.../index/column/BaseVirtualColumnProvider.java | 2 +-
.../DefaultNullValueVirtualColumnProvider.java | 2 +-
.../index/column/PhysicalColumnIndexContainer.java | 15 +-
.../converter/SegmentV1V2ToV3FormatConverter.java | 35 +-
.../segment/index/datasource/EmptyDataSource.java | 2 +-
.../index/datasource/ImmutableDataSource.java | 2 +-
.../segment/index/loader/IndexLoadingConfig.java | 27 +-
.../local/segment/index/loader/LoaderUtils.java | 8 +-
.../segment/index/loader/SegmentPreProcessor.java | 18 +-
.../loader/bloomfilter/BloomFilterHandler.java | 12 +-
.../ColumnMinMaxValueGenerator.java | 10 +-
.../defaultcolumn/BaseDefaultColumnHandler.java | 10 +-
.../defaultcolumn/DefaultColumnHandlerFactory.java | 4 +-
.../defaultcolumn/V1DefaultColumnHandler.java | 4 +-
.../defaultcolumn/V3DefaultColumnHandler.java | 8 +-
.../index/loader/invertedindex/H3IndexHandler.java | 10 +-
.../loader/invertedindex/InvertedIndexHandler.java | 10 +-
.../loader/invertedindex/JsonIndexHandler.java | 10 +-
.../invertedindex/LuceneFSTIndexHandler.java | 10 +-
.../loader/invertedindex/RangeIndexHandler.java | 10 +-
.../loader/invertedindex/TextIndexHandler.java | 16 +-
.../index/readers/BaseImmutableDictionary.java | 2 +-
.../index/readers/BitmapInvertedIndexReader.java | 2 +-
.../segment/index/readers/BytesDictionary.java | 2 +-
.../segment/index/readers/DoubleDictionary.java | 2 +-
.../segment/index/readers/FloatDictionary.java | 2 +-
.../local/segment/index/readers/IntDictionary.java | 2 +-
.../segment/index/readers/LongDictionary.java | 2 +-
.../index/readers/LuceneFSTIndexReader.java | 2 +-
.../index/readers/NullValueVectorReaderImpl.java | 2 +-
.../segment/index/readers/OnHeapDictionary.java | 2 +-
.../index/readers/OnHeapDoubleDictionary.java | 2 +-
.../index/readers/OnHeapFloatDictionary.java | 2 +-
.../segment/index/readers/OnHeapIntDictionary.java | 2 +-
.../index/readers/OnHeapLongDictionary.java | 2 +-
.../index/readers/OnHeapStringDictionary.java | 2 +-
.../segment/index/readers/RangeIndexReader.java | 2 +-
.../segment/index/readers/StringDictionary.java | 2 +-
.../readers/bloom/BaseGuavaBloomFilterReader.java | 2 +-
.../readers/bloom/BloomFilterReaderFactory.java | 2 +-
.../bloom/OffHeapGuavaBloomFilterReader.java | 2 +-
.../bloom/OnHeapGuavaBloomFilterReader.java | 2 +-
.../forward/BaseChunkSVForwardIndexReader.java | 4 +-
.../forward/FixedBitMVForwardIndexReader.java | 2 +-
.../forward/FixedBitSVForwardIndexReader.java | 2 +-
.../forward/FixedBitSVForwardIndexReaderV2.java | 2 +-
.../FixedByteChunkSVForwardIndexReader.java | 2 +-
.../forward/VarByteChunkSVForwardIndexReader.java | 2 +-
.../readers/geospatial/ImmutableH3IndexReader.java | 2 +-
.../readers/json/ImmutableJsonIndexReader.java | 2 +-
.../readers/sorted/SortedIndexReaderImpl.java | 2 +-
.../index/readers/text/LuceneTextIndexReader.java | 9 +-
.../segment/readers/PinotSegmentRecordReader.java | 12 +-
.../local/segment/store/FilePerIndexDirectory.java | 57 ++-
.../segment/local/segment/store/IndexEntry.java | 2 +-
.../segment/local/segment/store/IndexKey.java | 1 +
.../segment/store/SegmentLocalFSDirectory.java | 142 +++----
.../segment/store/SingleFileIndexDirectory.java | 126 +++---
.../virtualcolumn/DocIdVirtualColumnProvider.java | 2 +-
.../virtualcolumn/VirtualColumnProvider.java | 2 +-
.../segment/local/startree/OffHeapStarTree.java | 2 +-
.../local/startree/OffHeapStarTreeNode.java | 2 +-
.../local/startree/StarTreeBuilderUtils.java | 8 +-
.../startree/v2/builder/MultipleTreesBuilder.java | 6 +-
.../v2/builder/OffHeapSingleTreeBuilder.java | 2 +-
.../startree/v2/builder/StarTreeIndexCombiner.java | 2 +-
.../v2/builder/StarTreeV2BuilderConfig.java | 4 +-
.../startree/v2/store/StarTreeIndexContainer.java | 4 +-
.../startree/v2/store/StarTreeLoaderUtils.java | 6 +-
.../local}/upsert/TableUpsertMetadataManager.java | 7 +-
.../apache/pinot/segment/local/utils/CrcUtils.java | 2 +-
.../pinot/segment/local/utils/IngestionUtils.java | 22 ++
.../segment/local/utils/SegmentPushUtils.java | 2 +-
.../local/utils/fst/PinotBufferIndexInput.java | 2 +-
.../mutable}/IntermediateSegmentTest.java | 3 +-
.../MutableSegmentImplAggregateMetricsTest.java | 3 +-
.../MutableSegmentImplNullValueVectorTest.java | 3 +-
.../mutable/MutableSegmentImplTest.java | 3 +-
.../mutable/MutableSegmentImplTestUtils.java | 3 +-
.../mutable/MutableSegmentImplUpsertTest.java | 5 +-
.../io/reader/impl/FixedBitIntReaderTest.java | 2 +-
.../segment/local/io/util/PinotDataBitSetTest.java | 2 +-
.../local/io/util/PinotDataBitSetV2Test.java | 2 +-
.../io/util/VarLengthValueReaderWriterTest.java | 2 +-
.../io/writer/impl/MmapMemoryManagerTest.java | 2 +-
.../ComplexTypeTransformerTest.java | 108 ++++++
.../creator/BitmapInvertedIndexCreatorTest.java | 4 +-
.../local/segment/creator/IntArraysTest.java | 4 +-
.../impl/SegmentColumnarIndexCreatorTest.java | 1 +
.../local/segment/index/ColumnMetadataTest.java | 4 +-
.../segment/local/segment/index/H3IndexTest.java | 4 +-
.../segment/local/segment/index/JsonIndexTest.java | 4 +-
.../segment/index/SegmentMetadataImplTest.java | 4 +-
.../DefaultNullValueVirtualColumnProviderTest.java | 2 +-
.../SegmentV1V2ToV3FormatConverterTest.java | 6 +-
.../index/creator/BloomFilterCreatorTest.java | 4 +-
.../index/creator/LuceneFSTIndexCreatorTest.java | 4 +-
.../index/creator/RangeIndexCreatorTest.java | 4 +-
.../segment/index/creator/RawIndexCreatorTest.java | 15 +-
.../SegmentGenerationWithBytesTypeTest.java | 10 +-
.../SegmentGenerationWithFilterRecordsTest.java | 5 +-
.../SegmentGenerationWithMultipleRecordsTest.java | 5 +-
.../SegmentGenerationWithNoRecordsTest.java | 5 +-
.../SegmentGenerationWithTimeColumnTest.java | 11 +-
.../index/creator/SegmentPartitionTest.java | 6 +-
.../index/forward/FixedBitMVForwardIndexTest.java | 4 +-
.../forward/FixedByteChunkSVForwardIndexTest.java | 2 +-
.../forward/SortedForwardIndexReaderTest.java | 2 +-
.../forward/VarByteChunkSVForwardIndexTest.java | 6 +-
.../local/segment/index/loader/LoaderTest.java | 67 ++--
.../index/loader/SegmentPreProcessorTest.java | 100 +++--
.../BaseDefaultColumnHandlerTest.java | 7 +-
.../FixedByteWidthRowColDataFileReaderTest.java | 2 +-
.../index/readers/ImmutableDictionaryTest.java | 4 +-
.../ImmutableDictionaryTypeConversionTest.java | 4 +-
.../readers/NullValueVectorReaderImplTest.java | 2 +-
.../FixedBitSVForwardIndexReaderV2Test.java | 2 +-
...FixedByteWidthRowColForwardIndexWriterTest.java | 4 +-
.../store/ColumnIndexDirectoryTestHelper.java | 6 +-
.../segment/store/FilePerIndexDirectoryTest.java | 6 +-
.../segment/store/SegmentDirectoryPathsTest.java | 3 +-
.../segment/store/SegmentLocalFSDirectoryTest.java | 35 +-
.../store/SingleFileIndexDirectoryTest.java | 6 +-
.../v2/builder/StarTreeV2BuilderConfigTest.java | 4 +-
.../segment/local/utils/fst/FSTBuilderTest.java | 2 +-
.../src/test/resources/data/test_upsert_data.json | 0
pinot-segment-spi/pom.xml | 4 +
.../org/apache/pinot/segment/spi/IndexSegment.java | 7 +
.../org/apache/pinot/segment/spi}/V1Constants.java | 6 +-
.../segment/spi/index}/creator/TextIndexType.java | 2 +-
.../spi}/index/metadata/ColumnMetadata.java | 39 +-
.../spi}/index/metadata/SegmentMetadataImpl.java | 56 ++-
.../segment/spi/loader/SegmentDirectoryLoader.java | 52 +--
.../spi/loader/SegmentDirectoryLoaderRegistry.java | 89 +++++
.../pinot/segment/spi/loader/SegmentLoader.java | 14 +-
.../segment/spi}/memory/BasePinotLBuffer.java | 4 +-
.../pinot/segment/spi/memory}/CleanerUtil.java | 4 +-
.../pinot/segment/spi}/memory/PinotByteBuffer.java | 5 +-
.../pinot/segment/spi}/memory/PinotDataBuffer.java | 4 +-
.../spi}/memory/PinotNativeOrderLBuffer.java | 2 +-
.../spi}/memory/PinotNonNativeOrderLBuffer.java | 2 +-
.../spi}/partition/ByteArrayPartitionFunction.java | 3 +-
.../spi}/partition/HashCodePartitionFunction.java | 3 +-
.../spi}/partition/ModuloPartitionFunction.java | 3 +-
.../spi}/partition/MurmurPartitionFunction.java | 48 ++-
.../segment/spi/partition/PartitionFunction.java | 1 -
.../spi}/partition/PartitionFunctionFactory.java | 3 +-
.../metadata}/ColumnPartitionMetadata.java | 2 +-
.../segment/spi}/store/ColumnIndexDirectory.java | 45 +--
.../pinot/segment/spi}/store/ColumnIndexType.java | 2 +-
.../pinot/segment/spi}/store/SegmentDirectory.java | 42 +-
.../segment/spi}/store/SegmentDirectoryPaths.java | 10 +-
.../segment/spi}/memory/PinotDataBufferTest.java | 2 +-
.../spi}/partition/PartitionFunctionTest.java | 7 +-
.../pinot/server/api/resources/DebugResource.java | 135 +++++++
.../server/api/resources/MmapDebugResource.java | 2 +-
.../api/resources/SegmentMetadataFetcher.java | 83 +++-
.../server/api/resources/ServerResourceUtils.java | 58 +++
.../server/api/resources/TableSizeResource.java | 4 +-
.../pinot/server/api/resources/TablesResource.java | 44 +--
.../server/starter/helix/AdminApiApplication.java | 5 +-
.../starter/helix/HelixInstanceDataManager.java | 30 +-
.../helix/HelixInstanceDataManagerConfig.java | 35 +-
.../server/starter/helix/HelixServerStarter.java | 23 +-
.../starter/helix/SegmentFetcherAndLoader.java | 2 +-
.../SegmentOnlineOfflineStateModelFactory.java | 21 +-
.../apache/pinot/server/api/AccessControlTest.java | 2 +-
.../apache/pinot/server/api/BaseResourceTest.java | 6 +-
.../pinot/server/api/TablesResourceTest.java | 5 +-
.../annotations/minion/EventObserverFactory.java | 2 +-
.../annotations/minion/TaskExecutorFactory.java | 2 +-
.../spi/annotations/minion/TaskGenerator.java | 2 +-
.../config/instance/InstanceDataManagerConfig.java | 6 +
.../SegmentsValidationAndRetentionConfig.java | 4 +-
.../apache/pinot/spi/config/table/TableStatus.java | 70 ++++
.../config/table/ingestion/ComplexTypeConfig.java | 26 +-
.../pinot/spi/data/DateTimeGranularitySpec.java | 2 +-
.../java/org/apache/pinot/spi/data/FieldSpec.java | 4 +-
.../java/org/apache/pinot/spi/data/Schema.java | 4 +
.../apache/pinot/spi/data/readers/GenericRow.java | 7 +
.../pinot/spi/env/CommonsConfigurationUtils.java | 20 +-
.../apache/pinot/spi/env/PinotConfiguration.java | 14 +-
.../apache/pinot/spi/utils/CommonConstants.java | 12 +-
.../org/apache/pinot/spi/utils/EqualityUtils.java | 27 ++
.../java/org/apache/pinot/spi/utils/JsonUtils.java | 159 ++++++++
.../main/java/org/apache/pinot/spi/utils/Pair.java | 18 +
.../pinot/spi/utils/PinotReflectionUtils.java | 20 +-
.../spi/utils/builder/TableConfigBuilder.java | 8 +
.../org/apache/pinot/spi/utils/JsonUtilsTest.java | 72 ++++
pinot-spi/src/test/resources/json_util_test.json | 50 +++
pinot-tools/pom.xml | 97 +++++
.../pinot/tools/BatchQuickstartWithMinion.java | 11 +-
.../org/apache/pinot/tools/HybridQuickstart.java | 17 +-
.../org/apache/pinot/tools/JoinQuickStart.java | 17 +-
.../apache/pinot/tools/JsonIndexQuickStart.java | 15 +-
.../OfflineComplexTypeHandlingQuickStart.java | 15 +-
.../org/apache/pinot/tools/QuickStartBase.java | 20 +-
.../java/org/apache/pinot/tools/Quickstart.java | 16 +-
.../RealtimeComplexTypeHandlingQuickStart.java | 15 +-
.../pinot/tools/RealtimeJsonIndexQuickStart.java | 15 +-
.../org/apache/pinot/tools/RealtimeQuickStart.java | 16 +-
.../org/apache/pinot/tools/SegmentDumpTool.java | 2 +-
.../org/apache/pinot/tools/UpsertQuickStart.java | 15 +-
.../pinot/tools/admin/PinotAdministrator.java | 2 +
.../admin/command/AbstractBaseAdminCommand.java | 3 -
.../pinot/tools/admin/command/AddTableCommand.java | 2 +-
.../admin/command/AvroSchemaToPinotSchema.java | 33 +-
...maToPinotSchema.java => JsonToPinotSchema.java} | 76 ++--
.../command/LaunchDataIngestionJobCommand.java | 2 +-
.../tools/admin/command/QuickStartCommand.java | 62 ++-
.../tools/admin/command/StartBrokerCommand.java | 14 +-
.../admin/command/StartControllerCommand.java | 15 +-
.../tools/admin/command/StartMinionCommand.java | 2 +-
.../tools/admin/command/StartServerCommand.java | 19 +-
.../admin/command/StartServiceManagerCommand.java | 16 +-
.../tools/admin/command/StartZookeeperCommand.java | 2 +
.../anonymizer/ArrayBasedGlobalDictionaries.java | 2 +-
.../pinot/tools/anonymizer/GlobalDictionaries.java | 2 +-
.../anonymizer/MapBasedGlobalDictionaries.java | 2 +-
.../anonymizer/PinotDataAndQueryAnonymizer.java | 4 +-
.../pinot/tools/perf/PerfBenchmarkDriver.java | 9 +-
.../pinot/tools/perf/PerfBenchmarkRunner.java | 2 +-
.../apache/pinot/tools/scan/query/Aggregation.java | 2 +-
.../apache/pinot/tools/scan/query/Projection.java | 4 +-
.../tools/scan/query/SegmentQueryProcessor.java | 4 +-
.../apache/pinot/tools/scan/query/Selection.java | 2 +-
.../converter/DictionaryToRawIndexConverter.java | 2 +-
.../converter/PinotSegmentToAvroConverter.java | 2 +-
.../segment/converter/SegmentMergeCommand.java | 2 +-
.../pinot/tools/service/PinotServiceManager.java | 28 +-
.../PinotServiceManagerAdminApiApplication.java | 5 +-
.../apache/pinot/tools/utils/PinotConfigUtils.java | 19 +-
.../src/main/resources/conf/pinot-broker.conf | 36 ++
.../src/main/resources/conf/pinot-controller.conf | 42 ++
.../src/main/resources/conf/pinot-minion.conf | 39 ++
.../src/main/resources/conf/pinot-server.conf | 42 ++
...bEvents_offline_complexTypeHandling_schema.json | 50 ++-
...s_offline_complexTypeHandling_table_config.json | 6 +-
...eHandling_meetupRsvp_realtime_table_config.json | 14 +-
.../complexTypeHandling_meetupRsvp_schema.json | 44 ++-
pom.xml | 14 +
511 files changed, 9633 insertions(+), 3576 deletions(-)
mode change 100644 => 100755 compatibility-verifier/checkoutAndBuild.sh
delete mode 100644 licenses-binary/LICENSE-angular.txt
delete mode 100644 licenses-binary/LICENSE-aopalliance.txt
rename licenses-binary/{LICENSE-handlebars.txt => LICENSE-azure.txt} (87%)
rename licenses/LICENSE-handlebars.txt =>
licenses-binary/LICENSE-codehaus-animal-sniffer.txt (95%)
delete mode 100644 licenses-binary/LICENSE-codemirror.txt
delete mode 100644 licenses-binary/LICENSE-cpl-1.0.txt
create mode 100644 licenses-binary/LICENSE-edl-1.0.txt
delete mode 100644 licenses-binary/LICENSE-esotericsoftware.txt
delete mode 100644 licenses-binary/LICENSE-foundation.txt
rename licenses-binary/{LICENSE-fusesource.txt => LICENSE-google-auth.txt}
(91%)
rename licenses-binary/{LICENSE-protobuf.txt => LICENSE-google-protobuf.txt}
(100%)
delete mode 100644 licenses-binary/LICENSE-jcraft.txt
delete mode 100644 licenses-binary/LICENSE-jline.txt
delete mode 100644 licenses-binary/LICENSE-jquery-requestAnimationFrame.txt
delete mode 100644 licenses-binary/LICENSE-jquery-sizzle.txt
delete mode 100644 licenses-binary/LICENSE-jquery.txt
delete mode 100644 licenses-binary/LICENSE-js-beautify.txt
delete mode 100644 licenses-binary/LICENSE-mimepull.txt
delete mode 100644 licenses-binary/LICENSE-normalize.txt
delete mode 100644 licenses-binary/LICENSE-paranamer.txt
delete mode 100644 licenses-binary/LICENSE-py4j.txt
rename licenses/LICENSE-js-beautify.txt =>
licenses-binary/LICENSE-reactive-streams.txt (75%)
create mode 100644 licenses-binary/LICENSE-reflections.txt
rename licenses-binary/{LICENSE-scala.txt => LICENSE-scala-lang.txt} (100%)
rename licenses-binary/{LICENSE-jquery-datatables.txt => LICENSE-stax2.txt}
(83%)
delete mode 100644 licenses-binary/LICENSE-underscore.txt
delete mode 100644 licenses-binary/LICENSE-xmlenc.txt
create mode 100644 licenses-binary/LICENSE-zstd-jni.txt
delete mode 100644 licenses/LICENSE-angular.txt
delete mode 100644 licenses/LICENSE-codemirror.txt
delete mode 100644 licenses/LICENSE-foundation.txt
delete mode 100644 licenses/LICENSE-jquery-datatables.txt
delete mode 100644 licenses/LICENSE-jquery-requestAnimationFrame.txt
delete mode 100644 licenses/LICENSE-jquery-sizzle.txt
delete mode 100644 licenses/LICENSE-jquery.txt
delete mode 100644 licenses/LICENSE-normalize.txt
delete mode 100644 licenses/LICENSE-underscore.txt
copy
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/TextIndexType.java
=> pinot-common/src/main/java/org/apache/pinot/common/minion/Granularity.java
(87%)
create mode 100644
pinot-common/src/main/java/org/apache/pinot/common/minion/MergeRollupTaskMetadata.java
create mode 100644
pinot-common/src/main/java/org/apache/pinot/common/restlet/resources/SegmentErrorInfo.java
create mode 100644
pinot-common/src/main/java/org/apache/pinot/common/restlet/resources/SegmentServerDebugInfo.java
create mode 100644
pinot-controller/src/test/resources/recommenderInput/BloomFilterInputWithDateTimeColumn.json
copy
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/store/IndexEntry.java
=>
pinot-core/src/main/java/org/apache/pinot/core/operator/blocks/PassThroughTransformBlock.java
(53%)
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/PassThroughTransformOperator.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/query/optimizer/filter/Range.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/query/optimizer/filter/TimePredicateFilterOptimizer.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/segment/processing/collector/SortOrderComparator.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/segment/processing/genericrow/GenericRowDeserializer.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/segment/processing/genericrow/GenericRowFileReader.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/segment/processing/genericrow/GenericRowFileWriter.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/segment/processing/genericrow/GenericRowSerializer.java
delete mode 100644
pinot-core/src/main/java/org/apache/pinot/core/util/GenericRowSerDeUtils.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/query/optimizer/filter/TimePredicateFilterOptimizerTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/segment/processing/genericrow/GenericRowSerDeTest.java
delete mode 100644
pinot-core/src/test/resources/data/test_null_value_vector_data.json
delete mode 100644
pinot-core/src/test/resources/data/test_null_value_vector_pinot_schema.json
delete mode 100644 pinot-core/src/test/resources/data/test_upsert_schema.json
create mode 100644
pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/plugin/minion/tasks/TestEventObserverFactory.java
create mode 100644
pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/plugin/minion/tasks/TestTaskExecutorFactory.java
create mode 100644
pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/plugin/minion/tasks/TestTaskGenerator.java
copy pinot-spi/src/main/java/org/apache/pinot/spi/utils/Pair.java =>
pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/merge_rollup/MergeProperties.java
(50%)
create mode 100644
pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/merge_rollup/MergeRollupTaskUtils.java
create mode 100644
pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/test/java/org/apache/pinot/plugin/minion/tasks/merge_rollup/MergeRollupTaskUtilsTest.java
create mode 100644 pinot-plugins/pinot-stream-ingestion/pinot-kinesis/pom.xml
create mode 100644
pinot-plugins/pinot-stream-ingestion/pinot-kinesis/src/main/java/org/apache/pinot/plugin/stream/kinesis/KinesisConfig.java
create mode 100644
pinot-plugins/pinot-stream-ingestion/pinot-kinesis/src/main/java/org/apache/pinot/plugin/stream/kinesis/KinesisConnectionHandler.java
create mode 100644
pinot-plugins/pinot-stream-ingestion/pinot-kinesis/src/main/java/org/apache/pinot/plugin/stream/kinesis/KinesisConsumer.java
create mode 100644
pinot-plugins/pinot-stream-ingestion/pinot-kinesis/src/main/java/org/apache/pinot/plugin/stream/kinesis/KinesisConsumerFactory.java
create mode 100644
pinot-plugins/pinot-stream-ingestion/pinot-kinesis/src/main/java/org/apache/pinot/plugin/stream/kinesis/KinesisMsgOffsetFactory.java
create mode 100644
pinot-plugins/pinot-stream-ingestion/pinot-kinesis/src/main/java/org/apache/pinot/plugin/stream/kinesis/KinesisPartitionGroupOffset.java
create mode 100644
pinot-plugins/pinot-stream-ingestion/pinot-kinesis/src/main/java/org/apache/pinot/plugin/stream/kinesis/KinesisRecordsBatch.java
create mode 100644
pinot-plugins/pinot-stream-ingestion/pinot-kinesis/src/main/java/org/apache/pinot/plugin/stream/kinesis/KinesisStreamMetadataProvider.java
create mode 100644
pinot-plugins/pinot-stream-ingestion/pinot-kinesis/src/test/java/org/apache/pinot/plugin/stream/kinesis/KinesisConsumerTest.java
create mode 100644
pinot-plugins/pinot-stream-ingestion/pinot-kinesis/src/test/java/org/apache/pinot/plugin/stream/kinesis/KinesisStreamMetadataProviderTest.java
rename {pinot-core/src/main/java/org/apache/pinot/core =>
pinot-segment-local/src/main/java/org/apache/pinot/segment/local}/data/manager/SegmentDataManager.java
(97%)
rename {pinot-core/src/main/java/org/apache/pinot/core =>
pinot-segment-local/src/main/java/org/apache/pinot/segment/local}/data/manager/TableDataManager.java
(84%)
rename {pinot-core/src/main/java/org/apache/pinot/core/data/manager/config =>
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/data/manager}/TableDataManagerConfig.java
(91%)
create mode 100644
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/loader/LocalSegmentDirectoryLoader.java
rename {pinot-core/src/main/java/org/apache/pinot/core =>
pinot-segment-local/src/main/java/org/apache/pinot/segment/local}/upsert/TableUpsertMetadataManager.java
(85%)
rename {pinot-core/src/test/java/org/apache/pinot/core/indexsegment =>
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/indexsegment/mutable}/IntermediateSegmentTest.java
(98%)
rename {pinot-core/src/test/java/org/apache/pinot/core =>
pinot-segment-local/src/test/java/org/apache/pinot/segment/local}/indexsegment/mutable/MutableSegmentImplAggregateMetricsTest.java
(98%)
rename {pinot-core/src/test/java/org/apache/pinot/core =>
pinot-segment-local/src/test/java/org/apache/pinot/segment/local}/indexsegment/mutable/MutableSegmentImplNullValueVectorTest.java
(97%)
rename {pinot-core/src/test/java/org/apache/pinot/core =>
pinot-segment-local/src/test/java/org/apache/pinot/segment/local}/indexsegment/mutable/MutableSegmentImplTest.java
(98%)
rename {pinot-core/src/test/java/org/apache/pinot/core =>
pinot-segment-local/src/test/java/org/apache/pinot/segment/local}/indexsegment/mutable/MutableSegmentImplTestUtils.java
(96%)
rename {pinot-core/src/test/java/org/apache/pinot/core =>
pinot-segment-local/src/test/java/org/apache/pinot/segment/local}/indexsegment/mutable/MutableSegmentImplUpsertTest.java
(95%)
rename {pinot-core =>
pinot-segment-local}/src/test/resources/data/test_upsert_data.json (100%)
rename
{pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl
=>
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi}/V1Constants.java
(94%)
copy {pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment
=>
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index}/creator/TextIndexType.java
(93%)
rename
{pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment =>
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi}/index/metadata/ColumnMetadata.java
(92%)
rename
{pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment =>
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi}/index/metadata/SegmentMetadataImpl.java
(92%)
copy
pinot-spi/src/main/java/org/apache/pinot/spi/config/instance/InstanceDataManagerConfig.java
=>
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/loader/SegmentDirectoryLoader.java
(51%)
create mode 100644
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/loader/SegmentDirectoryLoaderRegistry.java
copy
pinot-spi/src/main/java/org/apache/pinot/spi/annotations/minion/TaskExecutorFactory.java
=>
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/loader/SegmentLoader.java
(74%)
rename
{pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment =>
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi}/memory/BasePinotLBuffer.java
(98%)
rename {pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils
=>
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/memory}/CleanerUtil.java
(99%)
rename
{pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment =>
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi}/memory/PinotByteBuffer.java
(98%)
rename
{pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment =>
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi}/memory/PinotDataBuffer.java
(99%)
rename
{pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment =>
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi}/memory/PinotNativeOrderLBuffer.java
(98%)
rename
{pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment =>
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi}/memory/PinotNonNativeOrderLBuffer.java
(98%)
rename {pinot-segment-local/src/main/java/org/apache/pinot/segment/local =>
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi}/partition/ByteArrayPartitionFunction.java
(93%)
rename {pinot-segment-local/src/main/java/org/apache/pinot/segment/local =>
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi}/partition/HashCodePartitionFunction.java
(93%)
rename {pinot-segment-local/src/main/java/org/apache/pinot/segment/local =>
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi}/partition/ModuloPartitionFunction.java
(95%)
rename {pinot-segment-local/src/main/java/org/apache/pinot/segment/local =>
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi}/partition/MurmurPartitionFunction.java
(63%)
rename {pinot-segment-local/src/main/java/org/apache/pinot/segment/local =>
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi}/partition/PartitionFunctionFactory.java
(96%)
rename {pinot-common/src/main/java/org/apache/pinot/common/metadata/segment =>
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/partition/metadata}/ColumnPartitionMetadata.java
(99%)
rename
{pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment =>
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi}/store/ColumnIndexDirectory.java
(63%)
rename
{pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment =>
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi}/store/ColumnIndexType.java
(96%)
rename
{pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment =>
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi}/store/SegmentDirectory.java
(83%)
rename
{pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment =>
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi}/store/SegmentDirectoryPaths.java
(88%)
rename
{pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment =>
pinot-segment-spi/src/test/java/org/apache/pinot/segment/spi}/memory/PinotDataBufferTest.java
(99%)
rename {pinot-segment-local/src/test/java/org/apache/pinot/segment/local =>
pinot-segment-spi/src/test/java/org/apache/pinot/segment/spi}/partition/PartitionFunctionTest.java
(97%)
create mode 100644
pinot-server/src/main/java/org/apache/pinot/server/api/resources/DebugResource.java
create mode 100644
pinot-server/src/main/java/org/apache/pinot/server/api/resources/ServerResourceUtils.java
create mode 100644
pinot-spi/src/main/java/org/apache/pinot/spi/config/table/TableStatus.java
create mode 100644 pinot-spi/src/test/resources/json_util_test.json
rename
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/TextIndexType.java
=> pinot-tools/src/main/java/org/apache/pinot/tools/QuickStartBase.java (69%)
copy
pinot-tools/src/main/java/org/apache/pinot/tools/admin/command/{AvroSchemaToPinotSchema.java
=> JsonToPinotSchema.java} (63%)
create mode 100644 pinot-tools/src/main/resources/conf/pinot-broker.conf
create mode 100644 pinot-tools/src/main/resources/conf/pinot-controller.conf
create mode 100644 pinot-tools/src/main/resources/conf/pinot-minion.conf
create mode 100644 pinot-tools/src/main/resources/conf/pinot-server.conf
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
