Re: Insanely long recovery time with Kafka 0.11.0.2

> > The change you mention is not part of 0.11.0.2. > > Ismael > > On Sat, Jan 6, 2018 at 3:31 PM, Ted Yu wrote: > > > bq. WARN Found a corrupted index file due to requirement failed: Corrupt > > index found, index file > > (/data/kafka/data-processed-15/00

Re: Broker won't exit...

I think that is the default signal. >From the script: SIGNAL=${SIGNAL:-TERM} FYI On Wed, Jan 10, 2018 at 2:35 AM, Sam Pegler wrote: > Have you tried a normal kill (sigterm) against the java process? > > __ > > Sam Pegler > > PRODUCTION ENGINEER > > T. +44(0) 07 562 867 486 > >

Re: Broker won't exit...

Skip:Can you pastebin the stack trace of the stuck broker ? Thanks Original message From: Skip Montanaro Date: 1/10/18 3:52 AM (GMT-08:00) To: users@kafka.apache.org Subject: Re: Broker won't exit... Did you stop the broker before stoping zookeeper? Yes. My stop script exec

Re: [ANNOUNCE] New committer: Matthias J. Sax

Congratulations, Matthias. On Fri, Jan 12, 2018 at 2:59 PM, Guozhang Wang wrote: > Hello everyone, > > The PMC of Apache Kafka is pleased to announce Matthias J. Sax as our > newest Kafka committer. > > Matthias has made tremendous contributions to Kafka Streams API since early > 2016. His footp

Re: [VOTE] KIP-247: Add public test utils for Kafka Streams

+1 Original message From: Guozhang Wang Date: 1/17/18 10:38 PM (GMT-08:00) To: users@kafka.apache.org Subject: Re: [VOTE] KIP-247: Add public test utils for Kafka Streams +1 (binding). On Wed, Jan 17, 2018 at 6:09 PM, Matthias J. Sax wrote: > Hi, > > I would like to start

Re: Memory Leak in Kafka

Did you attach two .png files ? Please use third party site since the attachment didn't come thru. On Tue, Jan 23, 2018 at 5:20 PM, Avinash Herle wrote: > > Hi, > > I'm using Kafka as a messaging system in my data pipeline. I've a couple > of producer processes in my pipeline and Spark Streamin

Re: Recommended max number of topics (and data separation)

After brief search, I found KAFKA-6469 FYI Original message From: Andrey Falko Date: 2/1/18 5:28 PM (GMT-08:00) To: users@kafka.apache.org Subject: Re: Recommended max number of topics (and data separation) Indeed David, I confirmed that I can't push my clusters to more than

Re: Strange Topic ...

Which Kafka version are you using ? Older versions of kafka (0.10 and prior) had some bugs in the log-cleaner thread that might sometimes cause it to crash. Please check the log-cleaner.log file to see if there was some clue. Cheers On Sun, Feb 4, 2018 at 11:14 AM, adrien ruffie wrote: > Hello

Re: log.retention.bytes not working as expected

bq. but is aborted. See the following in LogManager#asyncDelete(): //We need to wait until there is no more cleaning task on the log to be deleted before actually deleting it. if (cleaner != null && !isFuture) { cleaner.abortCleaning(topicPartition) FYI On Tue, Feb 6, 2018

Re: log.retention.bytes not working as expected

The log cleaner abortion in the log file preceded log deletion. On Tue, Feb 6, 2018 at 1:36 PM, Raghav wrote: > Ted > > Sorry, I did not understand your point here. > > On Tue, Feb 6, 2018 at 1:09 PM, Ted Yu wrote: > > > bq. but is aborted. > > > > See the

Re: [VOTE] 1.0.1 RC0

+1 Checked signature Ran test suite where there was one flaky test (KAFKA-5889): kafka.metrics.MetricsTest > testMetricsLeak FAILED java.lang.AssertionError: expected:<1365> but was:<1368> at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:8

Re: Cancel partition reassignment?

Have you seen this thread ? http://search-hadoop.com/m/Kafka/uyzND1pHiNuYt8hc1?subj=Re+Question+Kafka+Reassign+partitions+tool On Thu, Feb 8, 2018 at 4:12 PM, Dylan Martin wrote: > Hi all. > > > I'm trying to cancel a failed partition reassignment. I've heard that > this can be done by deletin

Re: mistake in Kafka the definitive guide

I think Paolo's comment is correct. BTW the paragraph above the figure says Kafka currently lacks transactions The above seems to have been written some time ago. Cheers On Thu, Feb 8, 2018 at 2:10 PM, adrien ruffie wrote: > I take a look the errata page, and I saw that Paolo had already ent

Re: Question Related to Zookeeper and kafka java error message.

Are you using zookeeper 3.4.11 ? Please take a look at ZOOKEEPER-2976 Can you pastebin more log before the NoRouteToHostException ? Cheers On Thu, Feb 8, 2018 at 11:39 AM, Raymond, Shawn P CTR USARMY NETCOM (US) < shawn.p.raymond@mail.mil> wrote: > Afternoon all, > > I was wondering if any

Re: Information regarding Kafka 2.10-0.10.1.1 version

For #1, there is record-size-avg metric Not sure about #2 On Thu, Feb 8, 2018 at 10:28 AM, Pawan K wrote: > Hi, > I am currently trying to research answers for the following questions. Can > you please let me know where/how could i find these in the configuration. > > 1) Average record size (KB

Re: question on serialization ..

If I read the code correctly, the operation on this line prepares the input for the (stringSerde, stringSerde) specified on line 142: .leftJoin(userRegionsTable, (clicks: Long, region: String) => (if (region == null) "UNKNOWN" else region, clicks)) FYI On Sat, Feb 10, 2018 at 11:00 AM, Deb

Re: question on serialization ..

his will serialize/de-serialize with the default serializers which are > both stringSerde for keys and values. > > or am I missing something ? > > regards. > > On Sun, Feb 11, 2018 at 2:30 AM, Ted Yu wrote: > > > If I read the code correctly, the operation on this

Re: DumpLogSegment

For --index-sanity-check, according to dumpIndex(): if (indexSanityOnly) { index.sanityCheck println(s"$file passed sanity check.") Do you see the print above ? Which release of Kafka are you using ? Cheers On Sat, Feb 10, 2018 at 1:54 PM, adrien ruffie wrote: > Hi all,

Re: DumpLogSegment

my release 2.11-1.0.0 > > > thank for you response Ted ! > > > De : Ted Yu > Envoyé : samedi 10 février 2018 23:00:02 > À : users@kafka.apache.org > Objet : Re: DumpLogSegment > > For --index-sanity-check, according to dumpIndex(): > > if

Re: [VOTE] 1.0.1 RC1

+1 Ran test suite which passed. BTW it seems the staging repo hasn't been updated yet: https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka-clients/ On Mon, Feb 12, 2018 at 10:16 AM, Ewen Cheslack-Postava wrote: > And of course I'm +1 since I've already done normal rele

Re: org.apache.kafka.clients.consumer.OffsetOutOfRangeException

Have you looked at SPARK-19888 ? Please give the full stack trace of the exception you saw. Cheers On Mon, Feb 12, 2018 at 12:38 PM, Mina Aslani wrote: > Hi Matthias, > Are you referring to https://issues.apache.org/jira/browse/SPARK-19976? > Doesn't look like that the jira was not fixed. (e.g

Typo in "Processing with local state"

Hi, In "Kafka the definitive guide", page 257: to calculate the minimum and average price ... It seems average should be maximum. Cheers

Re: error when attempting a unit test of spring kafka producer

LoginType was in 0.10.x release. This seems to indicate Kafka version mismatch. Can you check the dependencies of your test ? Thanks On Tue, Feb 13, 2018 at 8:03 PM, Ian Ewing wrote: > I have been trying to figure out how to unit test a kafka producer. Should > take in a simple integer and pe

Re: Kafka cluster instablility

For #2 and #3, you would get better stability if zookeeper and Kafka get dedicated machines. Have you profiled the performance of the nodes where multiple processes ran (zookeeper / Kafka / Druid) ? How was disk and network IO like ? Cheers On Wed, Feb 14, 2018 at 9:38 AM, Avinash Herle wrote:

Re: Kafka connect mysql

Have you tried adding -g/--globoff ? What is the OS you use ? Cheers On Sat, Feb 17, 2018 at 11:04 AM, adrien ruffie wrote: > Hello all, > > > In Kafka the definitive guide, on page 146 I found the following command: > > > curl http://localhost:8083/connector-plugins [{"class":"org.apache.kafk

Re: Kafka connect mysql

ng complaint ? Cheers On Sat, Feb 17, 2018 at 1:59 PM, adrien ruffie wrote: > yes like suggested :-) but nothing, > > Debian 9 for the OS > > > thx Ted > > > De : Ted Yu > Envoyé : samedi 17 février 2018 22:10:19 > À : users@kafka.a

Re: Kafka connect mysql

Feb 17, 2018 at 2:51 PM, adrien ruffie wrote: > yes the fact that my jdbcSourceConnector wasn't installd 😊 > > > By checking with the REST api connect list and status, no connector was > deployed > > ________ > De : Ted Yu > Envoyé : samed

Re: Zookeeper Error

What are the entries in /etc/hosts w.r.t. localhost ? I wonder if the exception had something to do with ipv6. On Sat, Feb 17, 2018 at 5:49 PM, Maria Pilar wrote: > When i try to create a topic in that multicluster, > > kafka-topics.bat --create --topic my-kafka-topic --zookeeper locahost:2181

Re: KafkaUtils.createStream(..) is removed for API

createStream() is still in external/kafka-0-8/src/main/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala But it is not in external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/KafkaUtils.scala FYI On Sun, Feb 18, 2018 at 5:17 PM, naresh Goud wrote: > Hello Team, > > I s

Re: Testing with MockConsumer

For #2, I think the assumption is that the records are processed by the loop: https://github.com/apache/kafka/blob/73be1e1168f91ee2a9d68e1d1c75c14018cf7d3a/clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java#L164 On Mon, Feb 19, 2018 at 4:39 AM, Gabriel Giussi wrote: > H

Re: Testing with MockConsumer

n: retry >- Others: close consumer > > Thanks for your help to solve #2. I'm wondering about 1# and 3# yet. > > 2018-02-19 11:46 GMT-03:00 Ted Yu : > > > For #2, I think the assumption is that the records are processed by the > > loop: > > > > ht

Re: Kafka Zookeeper Connection

Have you seen this thread (GC might be the cause for session time out) ? http://search-hadoop.com/m/Kafka/uyzND15d4h21yZnIZ?subj=Re+zookeeper+session+time+out Please check zookeeper server logs when you have a chance. Please share the versions of Kafka and zookeeper. On Sat, Feb 24, 2018 at 12:

Re: Kafka Zookeeper Connection

Please take a look at maxSessionTimeout under: http://zookeeper.apache.org/doc/r3.3.3/zookeeperAdmin.html#sc_advancedConfiguration On Sat, Feb 24, 2018 at 9:46 AM, Soheil Pourbafrani wrote: > Thanks, Manna, Can you say which property (in Kafka or Zookeeper) should I > increase? > > On Sat, Feb 2

Re: 答复: which Kafka StateStore could I use ?

Jie: Which DB are you using ? 600 records/second is very low rate. Probably your DB needs some tuning. Cheers On Fri, Mar 2, 2018 at 9:32 AM, Guozhang Wang wrote: > Hello Jie, > > By default Kafka Streams uses caching on top of its internal state stores > to de-dup output streams to the final

Re: [VOTE] 1.1.0 RC1

+1 Checked signature Ran test suite - apart from flaky testMetricsLeak, other tests passed. On Tue, Mar 6, 2018 at 2:45 AM, Damian Guy wrote: > Hello Kafka users, developers and client-developers, > > This is the second candidate for release of Apache Kafka 1.1.0. > > This is minor version rele

Re: [DISCUSS] KIP-267: Add Processor Unit Test Support to Kafka Streams Test Utils

Looks good. See if you can add punctuator into the sample code. On Wed, Mar 7, 2018 at 7:10 PM, John Roesler wrote: > Dear Kafka community, > > I am proposing KIP-267 to augment the public Streams test utils API. > The goal is to simplify testing of Kafka Streams applications. > > Please find d

Re: Kafka 0.10.2.2 release

Only found the following from brief search: http://search-hadoop.com/m/Kafka/uyzND1kSw4Xm3wyj1?subj=0+10+2+2+bug+fix+release FYI On Thu, Mar 8, 2018 at 10:13 AM, Devendar Rao wrote: > Hi, > > We're hitting an issue with log cleaner and I see this is fixed in 0.10.2.2 > release as per https://i

Re: Kafka Over TLS Error - Failed to send SSL Close message - Broken Pipe

Looking at KAFKA-3702, it is still Open. FYI On Thu, Mar 15, 2018 at 5:51 PM, Raghav wrote: > I am hitting this issue possible in 10.2.1. Can someone please confirm if > this issue was fixed in 10.2.1 or not ? > > R > > On Wed, Jun 7, 2017 at 11:50 AM, IT Consultant <0binarybudd...@gmail.com> >

Re: Two instances of Kafka consumer reading same partition

Can you attach broker and consumer logs to the JIRA ? Thanks On Mon, Mar 19, 2018 at 6:15 AM, Narayan Periwal wrote: > Hi, > > We are facing an issue with the Kafka consumer, the new library that got > introduced in 0.9 > > We are using Kafka broker 0.10.2.1 and consumer client version is also

Re: information about kafka

Can you clarify the question ? Are you looking for different systems for stream processing ? What's your use case ? Thanks On Mon, Mar 19, 2018 at 10:24 AM, abdelhadi FEKHAR wrote: > Hi, > >in science for collecting data in real time, does it exist only stream > processing and batch proce

Re: Running kafka in containers

Hi,Can you give a little more detail on the type and size of the containers ? Thanks Original message From: Thomas Crayford Date: 3/22/18 11:19 AM (GMT-08:00) To: Users Subject: Re: Running kafka in containers We (heroku) have run databases in containers since 2012, and kafk

Re: Apache Kafka / Spark Integration - Exception - The server disconnected before a response was received.

Can you post the stack trace for NetworkException (pastebin) ? Please also check the broker logs to see if there was some clue around the time this happened. Thanks On Mon, Mar 26, 2018 at 9:30 AM, M Singh wrote: > Hi: > I am working with spark 2.2.1 and spark kafka 0.10 client integration wit

Re: suggestion for 1.1.0 release notes

Nice reminder. zookeeper community seems to be close to releasing 3.4.12 : http://search-hadoop.com/m/ZooKeeper/SvKKH1jwLHJq2tr22?subj=Re+VOTE+Apache+ZooKeeper+release+3+4+12+candidate+1 FYI On Fri, Mar 30, 2018 at 3:38 PM, Gibson, Bill (ArcSight) < bill.gib...@microfocus.com> wrote: > The rel

Re: SSL does not work

Have you seen this thread ? http://search-hadoop.com/m/Kafka/uyzND1aC37obDa1W1?subj=Re+advertised+listeners On Tue, Apr 3, 2018 at 12:28 AM, Eduard wrote: > Hey! > > I've configured SSL for client connection and it does not work for me: > > My configuration: > > listeners=PLAINTEXT://0.0.0.0:70

Re: kafka-streams TopologyTestDriver problem with EXACTLY_ONCE

You saw the error because TopologyTestDriverTest doesn't explicitly call MockProducer#initTransactions(). To do that, TopologyTestDriver needs to expose such method to the TopologyTestDriverTest. You can log a JIRA if you think adding such capability is needed. Cheers On Wed, Apr 4, 2018 at 3:0

Re: kafka-streams Invalid transition attempted from state READY to state ABORTING_TRANSACTION

Looking at isTransitionValid(): case ABORTING_TRANSACTION: return source == IN_TRANSACTION || source == ABORTABLE_ERROR; The source state is not supposed to be READY. I don't see READY in the log you posted. Please consider logging a JIRA where you can atta

Re: TimeoutException not being triggered, stuck on OUT_OF_ORDER_SEQUENCE_NUMBER

Which Kafka release are you using ? I was looking at some potentially related JIRA(s), such as KAFKA-6015. FYI On Wed, Apr 4, 2018 at 3:05 AM, Saheb Motiani wrote: > Hi all, > > We have been seeing this issue intermittently, and hence it's difficult to > give a step by step instructions to rep

Re: kafka-streams Invalid transition attempted from state READY to state ABORTING_TRANSACTION

case, we never initialize TX > and thus aborting the TX fails. > > Please open a JIRA for the issue. > > -Matthias > > On 4/4/18 9:32 AM, Ted Yu wrote: > > Looking at isTransitionValid(): > > > > case ABORTING_TRANSACTION: > > >

Re: kafka-streams Invalid transition attempted from state READY to state ABORTING_TRANSACTION

t; -Matthias > > On 4/4/18 2:57 PM, Ted Yu wrote: > > How about the following change ? > > > > diff --git > > a/streams/src/main/java/org/apache/kafka/streams/processor/internals/ > StreamTask.java > > b/streams/src/main/java/org/apache/kafka/streams/proce

Re: kafka-streams Invalid transition attempted from state READY to state ABORTING_TRANSACTION

I created KAFKA-6747. Frederic's Id was on KAFKA-6323. Can some committer change the reporter of KAFKA-6747 to Frederic ? Thanks On Wed, Apr 4, 2018 at 4:35 PM, Ted Yu wrote: > https://github.com/apache/kafka/pull/4826 > > I will fill in JIRA Id once Frederic creates the JI

Re: join 2 topic streams --> to another topic

The blog is an interesting reading material. There was a minor typo: bq. an event arrives for either the left of right input stream 'of' above should be 'or'. Cheers On Sun, Apr 8, 2018 at 2:04 PM, Matthias J. Sax wrote: > Check out this blog post that explain how the different joins work: >

Re: subscribe kafka user group

See https://kafka.apache.org/contact Original message From: "vinly.zhao" Date: 4/9/18 5:21 AM (GMT-08:00) To: users@kafka.apache.org Subject: subscribe kafka user group Thanks, *Vincent Zhao*

Re: FetcherLagMetrics is not displaying in JConsole

Can you take a look at KAFKA-6156 and see if your cluster had the same issue ? Thanks On Thu, Apr 12, 2018 at 7:50 AM, ishwar panjari wrote: > HI, > > After configuration the kafka and zookeeper. it is not displaying the > kafka.consumer metrics. > > > I am not able to see the following metrics

Re: log retention bytes and log segment bytes

For log.retention.bytes : A size-based retention policy for logs. Segments are pruned from the log unless the remaining segments drop below log.retention.bytes This config is per partition. For log.segment.bytes : The maximum size of a log segment file. When this size is reached a new log segment

Re: Re: kafka streams with TimeWindows ,incorrect result

Noticed a typo in streams in subject. Corrected it in this reply. Original message From: 杰 杨 Date: 4/27/18 1:28 AM (GMT-08:00) To: 杰 杨 , users Subject: Re: Re: kafka steams with TimeWindows ,incorrect result and I checked windowStore interface found it has put method not get

Re: source code location for KSQL

https://github.com/confluentinc/ksql FYI On Fri, Apr 27, 2018 at 4:23 PM, Henry Cai wrote: > I think KSQL is also open sourced, where is the source code location for > KSQL in github? > > Thanks. >

Re: Kafka-Connect : Invalid value java.sql.SQLException: No suitable driver found for jdbc:mysql://127.0.0.1:3306/connect_test

Have you seen this thread ? https://stackoverflow.com/questions/49433199/kafka-connect-jdbc-source-connector-not-working-for-microsoft-sql-server On Tue, May 1, 2018 at 5:52 AM, Jagannath Bilgi wrote: > Hi Team, > Getting subject error message while trying to create Kafka-JDBC > connector. Abl

Re: Kafka offset problem when using Spark Streaming...

Can you give us more information: release of Spark and Kafka you're using anything interesting from broker / client logs - feel free to use pastebin to pass snippet if needed Thanks On Wed, May 9, 2018 at 3:42 AM, Pena Quijada Alexander < a.penaquij...@reply.it> wrote: > Hi all, > > We're faci

Re: Consulting ReadOnlyKeyValueStore from Processor can lead to deadlock

bq. the docs and CachingKVS behavior could improve I would agree. Pointing out the usage of ReadWriteLock and mentioning the withCachingDisabled() method in doc would help other developers. On Thu, May 10, 2018 at 11:21 AM, Steven Schlansker < sschlans...@opentable.com> wrote: > > > On May 10,

Re: Kafka consumer slowly consume the data

bq. load on the node is increasing tremendously Can you do some profiling to see where the bottleneck was ? You can pastebin some stack traces. Which Kafka release do you use ? Thanks On Fri, May 11, 2018 at 6:41 AM, Karthick Kumar wrote: > Hi, > > I'm using tomcat nodes for Kafka producer an

Re: Help Needed: Leadership Issue upon Kafka Upgrade (ZooKeeper 3.4.9)

Was there any previous connectivity issue to 1.1.1.143:3888 before the upgrade ? I assume you have verified that connectivity between broker and 1.1.1.143 is Okay. Which zookeeper release are you running ? Cheers On Fri, May 11, 2018 at 3:16 PM, Raghav wrote: > Hi > >

Re: Kafka 2.11-1.1.0 crashes brokers and brings cluster down on Windows

There are two outstanding issues: KAFKA-6059 and KAFKA-6200 which bear some resemblance. Can you try to find how the other process uses the file being deleted ? https://superuser.com/questions/117902/find-out-which-process-is-locking-a-file-or-folder-in-windows https://www.computerhope.com/issues

Re: Kafka 2.11-1.1.0 crashes brokers and brings cluster down on Windows

nd out which thread is >> locking the file. Apologise but it's a huge application so it might get me >> some time to get around it :) >> >> Please do update if you find something new. >> >> Regards, >> >> On 12 May 2018 at 22:15, Ted Yu wrote

Re: Kafka 2.11-1.1.0 crashes brokers and brings cluster down on Windows

serving logs in dir C:\kafka1 > (kafka.log.LogManager) > [2018-05-14 16:35:29,517] INFO [ExpirationReaper-1-topic]: Starting > (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) > [2018-05-14 16:35:29,517] ERROR Shutdown broker because all log dirs in > C:\kafka1 have faile

Re: Kafka Scala debugging and diagnostics

Have you set the following two environment variables: export KAFKA_DEBUG=y; export DEBUG_SUSPEND_FLAG=y; The above would freeze the process until debugger is attached. Cheers On Wed, May 16, 2018 at 2:46 PM, M. Manna wrote: > Hello, > > Is there anyone who can provide a few steps for setting

Re: Kafka Scala debugging and diagnostics

anna wrote: > > > Must have missed that in Kafka-run-class.bat which sets suspend=n by > > default. Thanks for pointing that out, giving it a try. > > > > > > > > And didn > > > > On Wed, 16 May 2018 at 22:57, Ted Yu wrote: > > > >

Re: INVALID_FETCH_SESSION_EPOCH after upgrade to 1.1.0

Before Errors.INVALID_FETCH_SESSION_EPOCH is returned, FetchSession.scala would log the reason for the response. There are 3 cases, 2 with info log and 1 with debug log. Here is one code snippet: if (session.epoch != reqMetadata.epoch()) { debug(s"Created a new error Fet

Re: Details of segment deletion

Minor clarification (since new segment appeared twice) : bq. before a new one is deleted. The 'new one' (in the last sentence) would become old when another segment is created. Cheers On Tue, Jun 12, 2018 at 6:42 PM, Gwen Shapira wrote: > See below: > > On Mon, Jun 11, 2018 at 3:36 AM, Simon

Re: INVALID_FETCH_SESSION_EPOCH after upgrade to 1.1.0

13, 2018 at 9:08 AM, Mark Anderson wrote: > Ted > > I don't see any other INFO log messages so I assume that means it is the > DEBUG case I'm seeing? > > I don't have DEBUG enabled at the moment. > > Thanks > > On Wed, 13

Re: NPE in low level Kafka topology

Can you show the related code from OneToManyGroupedProcessor ? Thanks On Mon, Jun 18, 2018 at 4:29 AM, Frank Lyaruu wrote: > Hi, I've upgraded our 0.11 based stream application to the trunk version, > and I get an intermittent NPE. It's is quite a big topology, and I haven't > succeeded in repr

Re: [VOTE] 1.1.1 RC0

+1 Ran unit test suite which passed. Checked signatures. On Tue, Jun 19, 2018 at 4:47 PM, Dong Lin wrote: > Re-send to kafka-clie...@googlegroups.com > > On Tue, Jun 19, 2018 at 4:29 PM, Dong Lin wrote: > > > Hello Kafka users, developers and client-developers, > > > > This is the first candi

Re: [VOTE] 1.1.1 RC1

+1 Ran test suite. Checked signatures. On Fri, Jun 22, 2018 at 12:14 PM, Vahid S Hashemian < vahidhashem...@us.ibm.com> wrote: > +1 (non-binding) > > Built from source and ran quickstart successfully on Ubuntu (with Java 8). > > Thanks Dong! > --Vahid > > > > From: Dong Lin > To: d...@kaf

Re: [VOTE] 0.11.0.3 RC0

+1 Checked signatures. Ran unit test suite. On Fri, Jun 22, 2018 at 4:56 PM, Vahid S Hashemian < vahidhashem...@us.ibm.com> wrote: > +1 (non-binding) > > Built from source and ran quickstart successfully on Ubuntu (with Java 8). > > Thanks Matthias! > --Vahid > > > > > From: "Matthias J. Sax"

Re: [DISCUSS] KIP-328: Ability to suppress updates for KTables

of the methods I listed. > > For example: > > windowCounts > .suppress( > Suppression > .suppressLateEvents(Duration.ofMinutes(10)) > .suppressIntermediateEvents( > IntermediateSuppression.emitAfter(Duration.ofMinutes(10)) >

Re: [VOTE] 0.10.2.2 RC1

+1 Ran test suite. Checked signatures. On Fri, Jun 29, 2018 at 10:21 AM, Jason Gustafson wrote: > +1 (binding). I checked release notes, documentation, and went through the > quickstart. > > Thanks Matthias! > > On Fri, Jun 22, 2018 at 6:43 PM, Matthias J. Sax > wrote: > > > Hello Kafka users

Re: Subscribe to mailing list

Please see https://kafka.apache.org/contact On Fri, Jun 29, 2018 at 10:54 AM, chinchu chinchu wrote: > >

Re: Partitions reassignment is failing in Kafka 1.1.0

Seems to be related to KIP-113. server.properties didn't go thru. Do you mind pastebin'ing its content ? If you can pastebin logs from broker, that should help. Thanks On Fri, Jun 29, 2018 at 10:37 AM, Debraj Manna wrote: > Hi > > I altered a topic like below in kafka 1.1.0 > > /home/ubuntu/d

Re: [VOTE] 1.0.2 RC1

+1 Ran test suite. On Fri, Jun 29, 2018 at 10:02 PM, Matthias J. Sax wrote: > Hello Kafka users, developers and client-developers, > > This is the second candidate for release of Apache Kafka 1.0.2. > > This is a bug fix release addressing 27 tickets: > https://cwiki.apache.org/confluence/displ

Re: Partitions reassignment is failing in Kafka 1.1.0

> > >> socket.request.max.bytes=104857600 > > >> log.dirs=/var/lib/kafka/kafka-logs > > >> num.recovery.threads.per.data.dir=1 > > >> log.retention.hours=36 > > >> log.retention.bytes=1073741824 > > >> log.segment.

Re: Partitions reassignment is failing in Kafka 1.1.0

See KAFKA-7124 Cheers On Sun, Jul 1, 2018 at 3:26 AM, Debraj Manna wrote: > Is there any bug filed for this ? > > On Sun, Jul 1, 2018 at 6:37 AM, Ted Yu wrote: > > > Code snippet from ReassignPartitionsCommand.scala : > > > >

Re: Cannot run the inbuild zookeeper server.

'Address already in use' indicates that there was some (zookeeper) server running on the same machine. Can you double check ? Thanks On Sun, Jul 1, 2018 at 4:45 AM, Vineet Herenj wrote: > Hello, > > I am not able to run the inbuild zookeeper server. > Following error shows: > > *java.net.BindE

Re: [VOTE] 1.1.1 RC3

+1 Ran test suite. Checked signatures. On Sun, Jul 8, 2018 at 3:36 PM Dong Lin wrote: > Hello Kafka users, developers and client-developers, > > This is the fourth candidate for release of Apache Kafka 1.1.1. > > Apache Kafka 1.1.1 is a bug-fix release for the 1.1 branch that was first > relea

Re: [VOTE] 2.0.0 RC3

+1 Checked signatures Ran test suite which passed. On Tue, Jul 24, 2018 at 8:28 AM Rajini Sivaram wrote: > Hello Kafka users, developers and client-developers, > > > This is the fourth candidate for release of Apache Kafka 2.0.0. > > > This is a major version release of Apache Kafka. It includ

Re: Next Release

Have you seen this thread ? http://search-hadoop.com/m/Kafka/uyzND1050Xu1lWkZV?subj=+VOTE+2+0+0+RC3 On Wed, Jul 25, 2018 at 7:04 PM xiejing wrote: > Hi, > > > > Do you have a schedule for the next release, say 1.2. or 2.0? > > > -- > > 谢静.Kane > > 大数据平台部 > [image: ci

Re: I can't use the command to close my kafka

Can you provide a bit more information ? Release of Kafka Have you checked broker logs after running kafka-server-stop.sh ? Cheers On Fri, Jul 27, 2018 at 3:53 PM ? ?? wrote: > Dear apache staff, > I'm sorry to bother you during your busy schedule, but I had a problem > with kafka! When I sta

Re: Apache Kafka Process showing high CPU (100 to 200+) usage in Linux when Idle

bq. only one specific node is showing this issue Is controller running on this node ? updating the metrics is expensive. Cheers On Sat, Aug 4, 2018 at 3:00 PM Abhijith Sreenivasan < abhijithonl...@gmail.com> wrote: > Hello > > We are seeing high CPU usage for the Kafka process. I am using 0.11

Re: Kafka flow

Have you looked at http://kafka.apache.org/documentation/ ? Chapters 4 and 5. FYI On Wed, Aug 8, 2018 at 1:46 AM darekAsz wrote: > Hi, > I need informations about workflow in kafka, I want to know what is going > on when producer send message, as much details as possible about > requests-respo

Re: Error reading field 'version' when execute kafka-consumer-groups.sh

For the last error, please see KAFKA-6868 On Wed, Aug 8, 2018 at 8:18 AM 刘鹏 wrote: > hi, > I get an error when I run kafka-consumer-groups.sh. > > > firstly,I run: > ./kafka-consumer-groups.sh --command-config consumer.properties > --bootstrap-server ip:port --list > > the result

Re: Java API to read metrics via JMX

Boris: BrokerWithJMX is referenced but I didn't find the class source after a brief search. FYI On Wed, Aug 8, 2018 at 7:10 PM Boris Lublinsky < boris.lublin...@lightbend.com> wrote: > Its actually quite simple, unfortunately you have to read, and then write > to TSDB. > Enclosed is an example d

Re: Issue in Kafka 2.0.0 ?

I think what happened in your use case was that the following implicit from ImplicitConversions.scala kept wrapping the resultant KTable from filter(): implicit def wrapKTable[K, V](inner: KTableJ[K, V]): KTable[K, V] = leading to stack overflow. Cheers On Mon, Aug 20, 2018 at 12:50 PM Druhin

Re: Issue in Kafka 2.0.0 ?

t; > > On August 20, 2018 at 1:30:42 PM, Ted Yu (yuzhih...@gmail.com yuzhih...@gmail.com>) wrote: > > I think what happened in your use case was that the following implicit > from ImplicitConversions.scala kept wrapping the resultant KTable from > filter(): > > implicit d

Re: Issue in Kafka 2.0.0 ?

hanks for reporting and for creating the ticket! > > > > -Matthias > > > > On 8/20/18 5:17 PM, Ted Yu wrote: > > > I was able to reproduce what you saw with modification > > > to StreamToTableJoinScalaIntegrationTestImplicitSerdes.scala > > > I have

Re: Issue in Kafka 2.0.0 ?

tthias J. Sax (matth...@confluent.io > <mailto:matth...@confluent.io>) wrote: > > Thanks for reporting and for creating the ticket! > > -Matthias > > On 8/20/18 5:17 PM, Ted Yu wrote: > > I was able to reproduce what you saw with modification > > to StreamToTableJoinS

Re: Issue in Kafka 2.0.0 ?

Sent out a PR #5543 which fixes the reported bug, with StreamToTableJoinScalaIntegrationTestImplicitSerdes.testShouldCountClicksPerRegion modified adding the filter methods. FYI On Mon, Aug 20, 2018 at 5:26 PM Ted Yu wrote: > Thanks for pointing me to that PR. > > I applied the PR lo

<    1   2