This is an automated email from the ASF dual-hosted git repository.
xiangfu pushed a change to branch move_sample_data_to_examples
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
discard 29f025a Make travis takes kafka-2.0 into default build
discard f10757b Move sample_data to examples
add f15a7b0 Add error for empty instance list for tenant API (#4969)
add b5d1f42 Remove Kafka and Stream dependency on avro (#4968)
add 97a3c21 Refactor pinot-batch-ingestion job specs to pinot-spi (#4967)
add 21e9a2f Move sample_data to examples
add 8c7ac99 Make travis takes kafka-2.0 into default build
add b13c3ce Make offline quickstart to use launchDataIngestionJob instead
of build segment then push
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (29f025a)
\
N -- N -- N refs/heads/move_sample_data_to_examples (b13c3ce)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
docs/batch_data_ingestion.rst | 15 ++++
.../api/resources/PinotTenantRestletResource.java | 38 +++++++---
.../controller/helix/ControllerTenantTest.java | 10 +++
pinot-distribution/pinot-assembly.xml | 1 -
pinot-integration-tests/pom.xml | 5 ++
.../pinot/integration/tests/ClusterTest.java | 27 ++++---
.../pom.xml | 17 ++---
.../standalone/SegmentGenerationJobRunner.java | 87 ++++++++++++----------
.../standalone}/SegmentGenerationTaskRunner.java | 17 +++--
.../batch}/standalone/SegmentTarPushJobRunner.java | 26 +++++--
.../batch}/standalone/SegmentUriPushJobRunner.java | 26 +++++--
.../segmentCreationAndTarPushJobSpec.yaml | 5 ++
.../segmentCreationAndUriPushJobSpec.yaml | 5 ++
.../src/main/resources/segmentCreationJobSpec.yaml | 5 ++
.../src/main/resources/segmentTarPushJobSpec.yaml | 5 ++
.../src/main/resources/segmentUriPushJobSpec.yaml | 5 ++
pinot-plugins/pinot-batch-ingestion/pom.xml | 2 +-
.../inputformat/avro/AvroRecordExtractor.java | 31 +++-----
.../inputformat/avro}/KafkaAvroMessageDecoder.java | 74 ++++++++++--------
.../avro}/SimpleAvroMessageDecoder.java | 17 ++++-
.../avro}/AvroRecordToPinotRowGeneratorTest.java | 7 +-
.../kafka09/KafkaLowLevelStreamConfigTest.java | 8 +-
.../KafkaPartitionLevelStreamConfigTest.java | 5 +-
.../pinot-kafka-base/pom.xml | 4 -
.../stream/AvroRecordToPinotRowGenerator.java | 51 -------------
pinot-spi/pom.xml | 4 +
.../spi/batch/ingestion/IngestionJobLauncher.java | 35 ++++++---
.../batch/ingestion/runner/IngestionJobRunner.java | 31 ++++----
.../pinot/spi/batch/ingestion/spec}/Constants.java | 2 +-
.../ingestion/spec/ExecutionFrameworkSpec.java | 76 +++++++++++++++++++
.../batch/ingestion/spec}/PinotClusterSpec.java | 2 +-
.../spi/batch/ingestion/spec}/PinotFSSpec.java | 2 +-
.../spi/batch/ingestion/spec}/PushJobSpec.java | 2 +-
.../batch/ingestion/spec}/RecordReaderSpec.java | 2 +-
.../ingestion/spec}/SegmentGenerationJobSpec.java | 22 +++++-
.../ingestion/spec}/SegmentGenerationTaskSpec.java | 10 +--
.../ingestion/spec}/SegmentNameGeneratorSpec.java | 2 +-
.../pinot/spi/batch/ingestion/spec}/TableSpec.java | 2 +-
.../data/readers/AbstractBaseRecordExtractor.java | 30 ++++----
...ecordReaderConfig.java => RecordExtractor.java} | 12 ++-
.../pinot/spi/stream/StreamMessageDecoder.java | 2 +
pinot-tools/pom.xml | 7 +-
.../org/apache/pinot/tools/HybridQuickstart.java | 20 +++--
.../java/org/apache/pinot/tools/Quickstart.java | 12 +--
.../apache/pinot/tools/QuickstartTableRequest.java | 12 ++-
.../command/LaunchDataIngestionJobCommand.java | 5 +-
.../tools/admin/command/QuickstartRunner.java | 19 +----
.../batch/airlineStats/ingestionJobSpec.yaml | 15 ++++
.../ingestionJobSpec.yaml | 41 ++++++++--
49 files changed, 537 insertions(+), 323 deletions(-)
rename pinot-plugins/pinot-batch-ingestion/{pinot-batch-ingestion-base =>
pinot-batch-ingestion-standalone}/pom.xml (91%)
rename
pinot-plugins/pinot-batch-ingestion/{pinot-batch-ingestion-base/src/main/java/org/apache/pinot/ingestion
=>
pinot-batch-ingestion-standalone/src/main/java/org/apache/pinot/plugin/ingestion/batch}/standalone/SegmentGenerationJobRunner.java
(93%)
rename
pinot-plugins/pinot-batch-ingestion/{pinot-batch-ingestion-base/src/main/java/org/apache/pinot/ingestion/common
=>
pinot-batch-ingestion-standalone/src/main/java/org/apache/pinot/plugin/ingestion/batch/standalone}/SegmentGenerationTaskRunner.java
(91%)
rename
pinot-plugins/pinot-batch-ingestion/{pinot-batch-ingestion-base/src/main/java/org/apache/pinot/ingestion
=>
pinot-batch-ingestion-standalone/src/main/java/org/apache/pinot/plugin/ingestion/batch}/standalone/SegmentTarPushJobRunner.java
(91%)
rename
pinot-plugins/pinot-batch-ingestion/{pinot-batch-ingestion-base/src/main/java/org/apache/pinot/ingestion
=>
pinot-batch-ingestion-standalone/src/main/java/org/apache/pinot/plugin/ingestion/batch}/standalone/SegmentUriPushJobRunner.java
(91%)
rename pinot-plugins/pinot-batch-ingestion/{pinot-batch-ingestion-base =>
pinot-batch-ingestion-standalone}/src/main/resources/segmentCreationAndTarPushJobSpec.yaml
(77%)
rename pinot-plugins/pinot-batch-ingestion/{pinot-batch-ingestion-base =>
pinot-batch-ingestion-standalone}/src/main/resources/segmentCreationAndUriPushJobSpec.yaml
(78%)
rename pinot-plugins/pinot-batch-ingestion/{pinot-batch-ingestion-base =>
pinot-batch-ingestion-standalone}/src/main/resources/segmentCreationJobSpec.yaml
(78%)
rename pinot-plugins/pinot-batch-ingestion/{pinot-batch-ingestion-base =>
pinot-batch-ingestion-standalone}/src/main/resources/segmentTarPushJobSpec.yaml
(77%)
rename pinot-plugins/pinot-batch-ingestion/{pinot-batch-ingestion-base =>
pinot-batch-ingestion-standalone}/src/main/resources/segmentUriPushJobSpec.yaml
(77%)
copy
pinot-core/src/test/java/org/apache/pinot/core/realtime/impl/fakestream/FakeStreamMessageDecoder.java
=>
pinot-plugins/pinot-input-format/pinot-avro/src/main/java/org/apache/pinot/plugin/inputformat/avro/AvroRecordExtractor.java
(59%)
rename
pinot-plugins/{pinot-stream-ingestion/pinot-kafka-base/src/main/java/org/apache/pinot/plugin/stream/kafka
=>
pinot-input-format/pinot-avro/src/main/java/org/apache/pinot/plugin/inputformat/avro}/KafkaAvroMessageDecoder.java
(79%)
rename
pinot-plugins/{pinot-stream-ingestion/pinot-kafka-base/src/main/java/org/apache/pinot/plugin/stream
=>
pinot-input-format/pinot-avro/src/main/java/org/apache/pinot/plugin/inputformat/avro}/SimpleAvroMessageDecoder.java
(81%)
rename
pinot-plugins/{pinot-stream-ingestion/pinot-kafka-base/src/test/java/org/apache/pinot/plugin/stream
=>
pinot-input-format/pinot-avro/src/test/java/org/apache/pinot/plugin/inputformat/avro}/AvroRecordToPinotRowGeneratorTest.java
(87%)
delete mode 100644
pinot-plugins/pinot-stream-ingestion/pinot-kafka-base/src/main/java/org/apache/pinot/plugin/stream/AvroRecordToPinotRowGenerator.java
rename
pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-base/src/main/java/org/apache/pinot/ingestion/standalone/StandaloneIngestionJobLauncher.java
=>
pinot-spi/src/main/java/org/apache/pinot/spi/batch/ingestion/IngestionJobLauncher.java
(61%)
copy
pinot-minion/src/main/java/org/apache/pinot/minion/executor/PinotTaskExecutor.java
=>
pinot-spi/src/main/java/org/apache/pinot/spi/batch/ingestion/runner/IngestionJobRunner.java
(61%)
rename
{pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-base/src/main/java/org/apache/pinot/ingestion/common
=>
pinot-spi/src/main/java/org/apache/pinot/spi/batch/ingestion/spec}/Constants.java
(95%)
create mode 100644
pinot-spi/src/main/java/org/apache/pinot/spi/batch/ingestion/spec/ExecutionFrameworkSpec.java
rename
{pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-base/src/main/java/org/apache/pinot/ingestion/common
=>
pinot-spi/src/main/java/org/apache/pinot/spi/batch/ingestion/spec}/PinotClusterSpec.java
(96%)
rename
{pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-base/src/main/java/org/apache/pinot/ingestion/common
=>
pinot-spi/src/main/java/org/apache/pinot/spi/batch/ingestion/spec}/PinotFSSpec.java
(97%)
rename
{pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-base/src/main/java/org/apache/pinot/ingestion/common
=>
pinot-spi/src/main/java/org/apache/pinot/spi/batch/ingestion/spec}/PushJobSpec.java
(98%)
rename
{pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-base/src/main/java/org/apache/pinot/ingestion/common
=>
pinot-spi/src/main/java/org/apache/pinot/spi/batch/ingestion/spec}/RecordReaderSpec.java
(98%)
rename
{pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-base/src/main/java/org/apache/pinot/ingestion/common
=>
pinot-spi/src/main/java/org/apache/pinot/spi/batch/ingestion/spec}/SegmentGenerationJobSpec.java
(91%)
rename
{pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-base/src/main/java/org/apache/pinot/ingestion/common
=>
pinot-spi/src/main/java/org/apache/pinot/spi/batch/ingestion/spec}/SegmentGenerationTaskSpec.java
(92%)
rename
{pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-base/src/main/java/org/apache/pinot/ingestion/common
=>
pinot-spi/src/main/java/org/apache/pinot/spi/batch/ingestion/spec}/SegmentNameGeneratorSpec.java
(96%)
rename
{pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-base/src/main/java/org/apache/pinot/ingestion/common
=>
pinot-spi/src/main/java/org/apache/pinot/spi/batch/ingestion/spec}/TableSpec.java
(97%)
copy
pinot-core/src/main/java/org/apache/pinot/core/segment/virtualcolumn/VirtualColumnContext.java
=>
pinot-spi/src/main/java/org/apache/pinot/spi/data/readers/AbstractBaseRecordExtractor.java
(54%)
copy
pinot-spi/src/main/java/org/apache/pinot/spi/data/readers/{RecordReaderConfig.java
=> RecordExtractor.java} (77%)
copy pinot-tools/src/main/resources/examples/batch/{airlineStats =>
baseballStats}/ingestionJobSpec.yaml (70%)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
