spark git commit: [SPARK-26121][STRUCTURED STREAMING] Allow users to define prefix of Kafka's consumer group (group.id)

Closes #23103 from zouzias/SPARK-26121. Authored-by: Anastasios Zouzias Signed-off-by: cody koeninger Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/2512a1d4 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/2512a1d4 Di

spark git commit: [SPARK-25233][STREAMING] Give the user the option of specifying a minimum message per partition per batch when using kafka direct API with backpressure

asf/spark/tree/135ff16a Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/135ff16a Branch: refs/heads/master Commit: 135ff16a3510a4dfb3470904004dae9848005019 Parents: 9e0f959 Author: Reza Safi Authored: Thu Aug 30 13:26:03 2018 -0500 Committer: cody koeninger Committed: Thu Aug 30 13:26:03 2

spark git commit: [SPARK-24987][SS] - Fix Kafka consumer leak when no new offsets for TopicPartition

picked from commit b7fdf8eb2011ae76f0161caa9da91e29f52f05e4) Signed-off-by: cody koeninger Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/14b50d7f Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/14b50d7f Diff: http://git-wip-us.apache.o

spark git commit: [SPARK-24987][SS] - Fix Kafka consumer leak when no new offsets for TopicPartition

29f52f05e4 Parents: 55e3ae6 Author: Yuval Itzchakov Authored: Sat Aug 4 14:44:10 2018 -0500 Committer: cody koeninger Committed: Sat Aug 4 14:44:10 2018 -0500 -- .../main/scala/org/apache/spark/sql/kafka010/KafkaSourceRDD.scala | 3 +--

spark git commit: [SPARK-24713] AppMatser of spark streaming kafka OOM if there are hund…

ents: 0ce11d0 Author: Yuanbo Liu Authored: Fri Jul 13 07:37:24 2018 -0600 Committer: cody koeninger Committed: Fri Jul 13 07:37:24 2018 -0600 -- .../spark/streaming/kafka010/DirectKafkaInputDStream.scala| 7 ++- 1 fi

spark git commit: [SPARK-24743][EXAMPLES] Update the JavaDirectKafkaWordCount example to support the new API of kafka

5 09:06:25 2018 -0500 Committer: cody koeninger Committed: Thu Jul 5 09:06:25 2018 -0500 -- .../streaming/JavaDirectKafkaWordCount.java | 24 +--- 1 file changed, 16 insertions(

spark git commit: [SPARK-24067][BACKPORT-2.3][STREAMING][KAFKA] Allow non-consecutive offsets

ion spark.streaming.kafka.allowNonConsecutiveOffsets to allow streaming jobs to proceed on compacted topics (or other situations involving gaps between offsets in the log). ## How was this patch tested? Added new unit test justinrmiller has been testing this branch in production for a few weeks Author: cody koenin

spark git commit: [SPARK-21168] KafkaRDD should always set kafka clientId.

red: Mon Apr 23 13:56:11 2018 -0500 Committer: cody koeninger Committed: Mon Apr 23 13:56:11 2018 -0500 -- .../src/main/scala/org/apache/spark/streaming/kafka/KafkaRDD.scala | 1 + 1 file changed, 1 insert

spark git commit: [SPARK-22968][DSTREAM] Throw an exception on partition revoking issue

org/repos/asf/spark/tree/5fccdae1 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/5fccdae1 Branch: refs/heads/master Commit: 5fccdae18911793967b315c02c058eb737e46174 Parents: 1ca3c50 Author: jerryshao Authored: Tue Apr 17 21:08:42 2018 -0500 Committer: cody koeninger Committed: Tue Apr 17

spark git commit: [SPARK-18580][DSTREAM][KAFKA] Add spark.streaming.backpressure.initialRate to direct Kafka streams

ads/master Commit: 2b89e4aa2e8bd8b88f6e5eb60d95c1a58e5c4ace Parents: 918c7e9 Author: akonopko Authored: Wed Mar 21 14:40:21 2018 -0500 Committer: cody koeninger Committed: Wed Mar 21 14:40:21 2018 -0500 -- .../kafka

spark git commit: [SPARK-18371][STREAMING] Spark Streaming backpressure generates batch with large number of records

mit: dffeac3691daa620446ae949c5b147518d128e08 Parents: 5414abc Author: Sebastian Arzt Authored: Fri Mar 16 12:25:58 2018 -0500 Committer: cody koeninger Committed: Fri Mar 16 12:25:58 2018 -0500 -- .../kafka010/DirectKafkaInputDStream.scala | 6 +-- .../kafka

spark-website git commit: Add Cody Koeninger to committers

Repository: spark-website Updated Branches: refs/heads/asf-site 2c2f85561 -> 32ff6fa97 Add Cody Koeninger to committers Project: http://git-wip-us.apache.org/repos/asf/spark-website/repo Commit: http://git-wip-us.apache.org/repos/asf/spark-website/commit/32ff6fa9 Tree: http://git-