> On Oct. 7, 2016, 3:38 p.m., Attila Simon wrote: > > Patch applies clearly on trunk currently (probably will conflicts after > > FLUME-2971). Code compiles with "mvn clean install -DskipTests", site > > builds without error message on the generated html. When running tests I > > got errors in the "mvn test -pl flume-ng-sinks/flume-ng-kafka-sink -fae" > > results: > > > > Failed tests: > > - testDefaultTopic(org.apache.flume.sink.kafka.TestKafkaSink): > > expected:<[default-topic-test]> but was:<[9]> > > - testEmptyChannel(org.apache.flume.sink.kafka.TestKafkaSink) > > "mvn test -pl flume-ng-sinks/flume-ng-kafka-sink -fae" passes on trunk, so > > this patch broke the tests. > > > > "mvn test -pl flume-ng-channels/flume-kafka-channel -fae" passed.
Okay, found the cause of this one - now fixed. - Tristan ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52598/#review151794 ----------------------------------------------------------- On Oct. 10, 2016, 8:49 a.m., Tristan Stevens wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52598/ > ----------------------------------------------------------- > > (Updated Oct. 10, 2016, 8:49 a.m.) > > > Review request for Flume and Grant Henke. > > > Repository: flume-git > > > Description > ------- > > This feature is useful for anyone who needs greater control of which > partitions are being written to - normally in a situation where multiple > Flume agents are being deployed in order to horizontally scale, or > alternatively if there is a scenario where there is a skew in data that might > lead to one or more partitions hotspotting. > We also have the ability to specify custom partitions on to the Kafka > Producer itself using the kafka.* configuration properties. > > The Kafka Producer provides the ability to set the partition ID using the > following constructor > (https://kafka.apache.org/090/javadoc/org/apache/kafka/clients/producer/ProducerRecord.html#ProducerRecord%28java.lang.String,%20java.lang.Integer,%20K,%20V%29 > ), this is just a matter of providing the option to use this constructor. > > This is specified in one of two ways: either via the staticPartition > configuration property, which means that every message goes to the specified > partition, or via the partitionHeader configuration property, which directs > the implementation to retrieve the partitionId from one of the event headers. > > > Diffs > ----- > > flume-ng-channels/flume-kafka-channel/pom.xml c1cc844 > > flume-ng-channels/flume-kafka-channel/src/main/java/org/apache/flume/channel/kafka/KafkaChannel.java > 66b553a > > flume-ng-channels/flume-kafka-channel/src/main/java/org/apache/flume/channel/kafka/KafkaChannelConfiguration.java > 3ab807b > > flume-ng-channels/flume-kafka-channel/src/test/java/org/apache/flume/channel/kafka/TestKafkaChannel.java > 57c0b28 > flume-ng-doc/sphinx/FlumeUserGuide.rst ab71d38 > flume-ng-sinks/flume-ng-kafka-sink/pom.xml 195c921 > > flume-ng-sinks/flume-ng-kafka-sink/src/main/java/org/apache/flume/sink/kafka/KafkaSink.java > 89bdd84 > > flume-ng-sinks/flume-ng-kafka-sink/src/main/java/org/apache/flume/sink/kafka/KafkaSinkConstants.java > 1bf380c > > flume-ng-sinks/flume-ng-kafka-sink/src/test/java/org/apache/flume/sink/kafka/TestKafkaSink.java > 76eca37 > flume-ng-sources/flume-kafka-source/pom.xml c89ea1a > flume-shared/flume-shared-kafka-test/pom.xml PRE-CREATION > > flume-shared/flume-shared-kafka-test/src/main/java/org/apache/flume/shared/kafka/test/KafkaPartitionTestUtil.java > PRE-CREATION > > flume-shared/flume-shared-kafka-test/src/main/java/org/apache/flume/shared/kafka/test/PartitionOption.java > PRE-CREATION > > flume-shared/flume-shared-kafka-test/src/main/java/org/apache/flume/shared/kafka/test/PartitionTestScenario.java > PRE-CREATION > flume-shared/pom.xml PRE-CREATION > pom.xml 2332a29 > > Diff: https://reviews.apache.org/r/52598/diff/ > > > Testing > ------- > > Unit testing done for both Kafka Channel and Kafka Sink. > > > Thanks, > > Tristan Stevens > >
