> On Oct. 9, 2016, 11:52 a.m., Mike Percy wrote:
> > Looking pretty good.
> >
> > First of all, I had to make some changes to compile this. See below.
> >
> > Also, please fix the checkstyle errors.
> >
> > You can run it with the following command: $ mvn clean install -DskipTests
> >
> > This is the patch I had to apply on top of yours to get this to compile:
> >
> > diff --git a/flume-shared/flume-shared-kafka-test/pom.xml
> > b/flume-shared/flume-shared-kafka-test/pom.xml
> > index adb54fa..0f1725d 100644
> > --- a/flume-shared/flume-shared-kafka-test/pom.xml
> > +++ b/flume-shared/flume-shared-kafka-test/pom.xml
> > @@ -66,18 +66,19 @@
> > <dependency>
> > <groupId>junit</groupId>
> > <artifactId>junit</artifactId>
> > - <scope>test</scope>
> > </dependency>
> >
> > <dependency>
> > <groupId>org.apache.kafka</groupId>
> > <artifactId>kafka_2.10</artifactId>
> > - <scope>test</scope>
> > + <scope>provided</scope>
> > </dependency>
> > +
> > <dependency>
> > <groupId>org.apache.kafka</groupId>
> > <artifactId>kafka-clients</artifactId>
> > <version>${kafka.version}</version>
> > + <scope>provided</scope>
> > </dependency>
> >
> > </dependencies>
> > diff --git a/pom.xml b/pom.xml
> > index f66c1b3..ded7b7d 100644
> > --- a/pom.xml
> > +++ b/pom.xml
> > @@ -826,7 +826,6 @@ limitations under the License.
> > <groupId>junit</groupId>
> > <artifactId>junit</artifactId>
> > <version>4.10</version>
> > - <scope>test</scope>
> > </dependency>
> >
> > <dependency>
The patch got garbled. Here it is again:
diff --git a/flume-shared/flume-shared-kafka-test/pom.xml
b/flume-shared/flume-shared-kafka-test/pom.xml
index adb54fa..0f1725d 100644
--- a/flume-shared/flume-shared-kafka-test/pom.xml
+++ b/flume-shared/flume-shared-kafka-test/pom.xml
@@ -66,18 +66,19 @@
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
- <scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.10</artifactId>
- <scope>test</scope>
+ <scope>provided</scope>
</dependency>
+
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>${kafka.version}</version>
+ <scope>provided</scope>
</dependency>
</dependencies>
diff --git a/pom.xml b/pom.xml
index f66c1b3..ded7b7d 100644
--- a/pom.xml
+++ b/pom.xml
@@ -826,7 +826,6 @@ limitations under the License.
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.10</version>
- <scope>test</scope>
</dependency>
<dependency>
- Mike
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52598/#review151939
-----------------------------------------------------------
On Oct. 8, 2016, 2:41 p.m., Tristan Stevens wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52598/
> -----------------------------------------------------------
>
> (Updated Oct. 8, 2016, 2:41 p.m.)
>
>
> Review request for Flume and Grant Henke.
>
>
> Repository: flume-git
>
>
> Description
> -------
>
> This feature is useful for anyone who needs greater control of which
> partitions are being written to - normally in a situation where multiple
> Flume agents are being deployed in order to horizontally scale, or
> alternatively if there is a scenario where there is a skew in data that might
> lead to one or more partitions hotspotting.
> We also have the ability to specify custom partitions on to the Kafka
> Producer itself using the kafka.* configuration properties.
>
> The Kafka Producer provides the ability to set the partition ID using the
> following constructor
> (https://kafka.apache.org/090/javadoc/org/apache/kafka/clients/producer/ProducerRecord.html#ProducerRecord%28java.lang.String,%20java.lang.Integer,%20K,%20V%29
> ), this is just a matter of providing the option to use this constructor.
>
> This is specified in one of two ways: either via the staticPartition
> configuration property, which means that every message goes to the specified
> partition, or via the partitionHeader configuration property, which directs
> the implementation to retrieve the partitionId from one of the event headers.
>
>
> Diffs
> -----
>
> flume-ng-channels/flume-kafka-channel/pom.xml c1cc844
>
> flume-ng-channels/flume-kafka-channel/src/main/java/org/apache/flume/channel/kafka/KafkaChannel.java
> 66b553a
>
> flume-ng-channels/flume-kafka-channel/src/main/java/org/apache/flume/channel/kafka/KafkaChannelConfiguration.java
> 3ab807b
>
> flume-ng-channels/flume-kafka-channel/src/test/java/org/apache/flume/channel/kafka/TestKafkaChannel.java
> 57c0b28
> flume-ng-doc/sphinx/FlumeUserGuide.rst ab71d38
> flume-ng-sinks/flume-ng-kafka-sink/pom.xml 195c921
>
> flume-ng-sinks/flume-ng-kafka-sink/src/main/java/org/apache/flume/sink/kafka/KafkaSink.java
> 89bdd84
>
> flume-ng-sinks/flume-ng-kafka-sink/src/main/java/org/apache/flume/sink/kafka/KafkaSinkConstants.java
> 1bf380c
>
> flume-ng-sinks/flume-ng-kafka-sink/src/test/java/org/apache/flume/sink/kafka/TestKafkaSink.java
> 76eca37
> flume-ng-sources/flume-kafka-source/pom.xml c89ea1a
> flume-shared/flume-shared-kafka-test/pom.xml PRE-CREATION
>
> flume-shared/flume-shared-kafka-test/src/main/java/org/apache/flume/shared/kafka/test/KafkaPartitionTestUtil.java
> PRE-CREATION
>
> flume-shared/flume-shared-kafka-test/src/main/java/org/apache/flume/shared/kafka/test/PartitionOption.java
> PRE-CREATION
>
> flume-shared/flume-shared-kafka-test/src/main/java/org/apache/flume/shared/kafka/test/PartitionTestScenario.java
> PRE-CREATION
> flume-shared/pom.xml PRE-CREATION
> pom.xml 2332a29
>
> Diff: https://reviews.apache.org/r/52598/diff/
>
>
> Testing
> -------
>
> Unit testing done for both Kafka Channel and Kafka Sink.
>
>
> Thanks,
>
> Tristan Stevens
>
>
