Re: Kafka InvalidStateStoreException

bq. at com.mytest.csd.kafka.KafkaDeduplicator.getQuotaStore(KafkaDe duplicator.java:147) Can you show us relevant code snippet for the above method ? On Fri, Sep 15, 2017 at 2:20 AM, Jari Väimölä wrote: > Hello all, > I have an apache kafka stream application running

Re: Kafka MirrorMaker - target or source datacenter deployment

Wouldn't KAFKA-5494 make remote produce more reliable? Original message From: Todd Palino Date: 9/14/17 6:53 PM (GMT-08:00) To: users@kafka.apache.org Subject: Re: Kafka MirrorMaker - target or source datacenter deployment Always in the target datacenter.

Re: Kafka Streams application Unable to Horizontally scale and the application on other instances refusing to start.

ere getting closed and started which might >> have >> > resulted in the "CommitFailedException due to non avialability of >> > processing processors. >> > >> > After some time the issue got propagated to other servers, I have >> attached &

Re: Kafka Streams application Unable to Horizontally scale and the application on other instances refusing to start.

Hi, Were you using 0.11.0.0 ? I ask this because some related fixes, KAFKA-5167 and KAFKA-5152, are only in 0.11.0.1 Mind trying out 0.11.0.1 release and see whether the problem persists ? On Fri, Sep 15, 2017 at 12:52 PM, Ted Yu <yuzhih...@gmail.com> wrote: > bq. 1)

Re: Kafka Streams application Unable to Horizontally scale and the application on other instances refusing to start.

ere any upgrade which needs to be > done on kafka broker side as well ? > > On Sat, Sep 16, 2017 at 5:18 AM, Ted Yu <yuzhih...@gmail.com> wrote: > > > Hi, > > Were you using 0.11.0.0 ? > > > > I ask this because some related fixes, KAFKA-5167 and KAFKA-5152, ar

Re: Improving Kafka State Store performance

Looking at the calculation of totalTimeTakenToStoreRecords, it covers store.put() call. Can you tell us more about what the put() does ? Does it involve external key value store ? Are you using 0.11.0.0 ? Thanks On Sat, Sep 16, 2017 at 6:14 AM, dev loper wrote: > Hi Kafka

Re: Improving Kafka State Store performance

We're using rocksdb 5.3.6 It would make more sense to perform next round of experiment using rocksdb 5.7.3 which is latest. Cheers On Mon, Sep 18, 2017 at 5:00 AM, Bill Bejeck wrote: > I'm following up from your other thread as well here. Thanks for the info > above, that

Re: Kafka Connect Sink Connector for multiple JDBC sinks

Have you looked at https://github.com/confluentinc/kafka-connect-jdbc ? On Sat, Sep 16, 2017 at 1:39 PM, M. Manna wrote: > Sure. But all these are not available via Kafka open source (requires > manual coding), correct? Only Confluence seems to provide some > off-the-shelf

Re: Kafka Streams application Unable to Horizontally scale and the application on other instances refusing to start.

ORDS_CONFIG >> > =6 and the processors were getting closed and started which might >> have >> > resulted in the "CommitFailedException due to non avialability of >> > processing processors. >> > >> > After some tim

Re: Data loss while upgrading confluent 3.0.0 kafka cluster to confluent 3.2.2

Looks like the screen shots didn't come through.  Consider pasting the text. Thanks Original message From: Yogesh Sangvikar Date: 9/19/17 4:33 AM (GMT-08:00) To: users@kafka.apache.org Cc: Sumit Arora , Bharathreddy

Re: Hi

Please follow instructions on http://kafka.apache.org/contact On Thu, Sep 21, 2017 at 1:30 PM, Daniele Ascione wrote: > hi, I would like to subscribe >

Re: Kafka Internals Video/Blog

Kafka has been evolving rapidly. Is there area(s) which you're particularly interested ? You can find all the relevant KIPs under: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals Normally KIPs contain fair amount of technical details. >From each KIP, you can find

Re: Kafka Streams application Unable to Horizontally scale and the application on other instances refusing to start.

Ram, Storage : EBS > > Kafka Streams Instance : 3 Kafka Streams Application Instances (Current > CPU Usage 8%- 24%) > > Instance Type : AWS M4 Large > Machine Configuration : 2 VCPU;s, 8gb Ram, Storage : EBS (Dedicated > EBS bandwidth 450 mbps) > Thanks >

Re: Kafka MirrorMaker - exit on send failure?

>From MirrorMaker.scala : // Defaults to no data loss settings. maybeSetDefaultProperty(producerProps, ProducerConfig.RETRIES_CONFIG, Int.MaxValue.toString) maybeSetDefaultProperty(producerProps, ProducerConfig.MAX_BLOCK_MS_CONFIG, Long.MaxValue.toString) I think the settings

Re: Kafka + ACL Topic based security ,Spring Boot

Attachment didn't come thru. Have you read ? https://cwiki.apache.org/confluence/display/KAFKA/Security especially the https://cwiki.apache.org/confluence/display/KAFKA/Security#Security-ImplementingthePermissionManager section ? On Fri, Sep 22, 2017 at 2:45 AM, Pooppillikudiyil, Joby <

Re: Kafka ACL - Access Denied while publishing the message

>From the example off: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Authorization+Command+Line+Interface it seems following 'User:', the formation is te...@example.com Can you double check ? On Wed, Oct 4, 2017 at 8:55 PM, Awadhesh Gupta wrote: > Hi, > > I

Re: Kafka Streams with embedded RocksDB

I assume you have read https://github.com/facebook/rocksdb/wiki/Building-on-Windows Please also see https://github.com/facebook/rocksdb/issues/2531 BTW your question should be directed to rocksdb forum. On Fri, Oct 6, 2017 at 6:39 AM, Valentin Forst wrote: > Hi there, > >

Re: kafka broker loosing offsets?

gt; > >: > > > > > > > > > Reprocessing same events again - is fine for us (idempotent). While > > > > loosing > > > > > data is more critical. > > > > > > > > > > What are reasons of such behaviour? Consumers

Re: kafka broker loosing offsets?

tion to find out what happens. Also in general with current > semantics of offset reset policy IMO using anything but none is not really > an option unless it is ok for consumer to loose some data (latest) or > reprocess it second time (earliest). > > fre 6 okt. 2017 kl. 17:44 skrev

Re: __consumers topic is overloaded

Graph image didn't come through. Consider using third party site for hosting image. On Fri, Oct 6, 2017 at 6:46 AM, Alexander Petrovsky wrote: > Hello! > > I observe the follow strange behavior in my kafka graphs: > > > As you can see, the topic __consumer_offsets have very

Re: kafka broker loosing offsets?

tes (336 hrs)? > > > On Fri, Oct 6, 2017 at 8:58 PM, Dmitriy Vsekhvalnov < > dvsekhval...@gmail.com> > wrote: > > > Hi Ted, > > > > Broker: v0.11.0.0 > > > > Consumer: > > kafka-clients v0.11.0.0 > > auto.offset.reset = earliest >

Re: kafka broker loosing offsets?

What's the value for auto.offset.reset ? Which release are you using ? Cheers On Fri, Oct 6, 2017 at 7:52 AM, Dmitriy Vsekhvalnov wrote: > Hi all, > > we several time faced situation where consumer-group started to re-consume > old events from beginning. Here is

Re: NPE on ConsumerRecords$ConcatenatedIterable$1.makeNext() while iterating records

Was there any consumer interceptor involved ? Cheers On Sun, Oct 8, 2017 at 6:29 AM, Michael Keinan wrote: > Hi > Using Kafka 0.10.2.0 > I get a NPE while iterating the records after polling them using poll > method. > - Any idea where does it come from ? > - How can I

Re: Kafka stream api to consume json values using kafka connect json deserializer

Have you taken a look at streams/examples/src/main/java/org/apache/kafka/streams/examples/pageview/PageViewUntypedDemo.java ? final Deserializer jsonDeserializer = new JsonDeserializer(); final Serde jsonSerde = Serdes.serdeFrom(jsonSerializer, jsonDeserializer); final

Re: Store - RocksDB - aging data off

Past thread related to TTL: http://search-hadoop.com/m/Kafka/uyzND1RLg4VOJ84U?subj=Re+Streams+TTLCacheStore On Thu, Oct 5, 2017 at 9:54 AM, Ted Yu <yuzhih...@gmail.com> wrote: > Looking at > streams/src/main/java/org/apache/kafka/streams/state/internals/Rocks

Re: Store - RocksDB - aging data off

Looking at streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java : throw new UnsupportedOperationException("Change log is not supported for store " + this.name + " since it is TTL based."); // TODO: support TTL with change log?

Re: Kafka connect rebalance problem

bq. There are 7 connectors configured with each connector configured with 8 tasks (Averaging about 4 tasks per connector) Pardon. I don't quite understand the above setup. Can you describe in more detail ? Which version of connector are you using ? Cheers On Mon, Oct 16, 2017 at 11:30 PM,

Re: Kafka 0.9.0.1 Direct buffer memory OOM

Can you take a look at KAFKA-5470 ? On Tue, Oct 17, 2017 at 2:32 AM, 杨文 wrote: > Hi Kafka Users, > We are using kafka 0.9.0.1 and frequently we have seen below exception > which causes the broker > to die.We even increased the MaxDirectMemory to 1G but still see this. >

Re: when will zk path "/kafka/brokers/topics/__consumer_offsets/partitions/{number}/state" be updated

The logs didn't go through. Consider using pastebin. For steps 1 and 2, it seems you used some image which didn't go through. Please use pastebin instead. Where did you get the docker image ? Cheers On Fri, Sep 8, 2017 at 4:42 AM, Nick <394299...@qq.com> wrote: > Dears, > > May I seek your

Re: To Know about Kafka and it's usage

Parth : bq. an application which could take some real time data Can you be a bit more specific on what your goal is ? It would help narrow down the choices you have. Cheers On Thu, Sep 28, 2017 at 8:01 AM, David Garcia wrote: > If you’re on the AWS bandwagon, you can

Re: Is anyone using MX4J loader?

Looks like Ralph logged KAFKA-4946 for this already. On Fri, Sep 29, 2017 at 12:40 AM, Dong Lin wrote: > Hi Kafka users, > > I am wondering if anyone is currently using feature from MX4J loader. This > feature is currently enabled by default. But if kafka_mx4jenable is >

Re: windowed store excessive memory consumption

Have you seen this comment ? https://issues.apache.org/jira/browse/KAFKA-5122?focusedCommentId=15984467=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15984467 On Wed, Sep 27, 2017 at 12:44 PM, Stas Chizhov wrote: > Hi, > > I am running a simple

Re: kafka-console-producer.sh --security-protocol

I think in producer.properties you should use: security.protocol=SASL_PLAINTEXT FYI On Tue, Oct 3, 2017 at 7:17 AM, Pekka Sarnila wrote: > Hi, > > kafka_2.11-0.11.0.0 > > If I try to give --security-protocol xyz (xyz any value e.g. > SASL_PLAINTEXT, PLAINTEXTSASL, SASL_SSL) I

Re: Kafka Streams Avro SerDe version/id caching

I did a quick search in the code base - there doesn't seem to be caching as you described. On Tue, Oct 3, 2017 at 6:36 AM, Kristopher Kane wrote: > If using a Byte SerDe and schema registry in the consumer configs of a > Kafka streams application, does it cache the Avro

Re: Fetching topic metadata issue

Please give us more information: release of Kafka Did your consumer get any error ? Have you inspected broker log(s) ? Cheers On Sun, Sep 3, 2017 at 11:08 PM, Sagar Nadagoud < sagar.nadag...@wildjasmine.com> wrote: > Hi, > > Please help me in this issue i am unable to read the message from

Re: Kafka cluster Error

Can you check the health of zookeeper quorum ? Cheers On Tue, Oct 10, 2017 at 3:35 PM, Kannappan, Saravanan (Contractor) < saravanan_kannap...@comcast.com> wrote: > Hello, Someone can you help me kafka server not starting after rebooting , > the below is the error message > > [2017-10-10

Re: Regarding kafka-manager topics parameters

Images didn't come thru. Consider using third party website. On Tue, Oct 17, 2017 at 9:36 PM, Pavan Patani wrote: > Hello, > > Previously I was using old version of Kafka-manager and it was showing > "Producer Message/Sec and Summed Recent Offsets" parameters in

Re: Kafka FileStreamSinkConnector handling of bad messages

Considering Ewen's response, you can open a JIRA for applying the suggestion toward FileStreamSinkConnector. Cheers On Wed, Oct 18, 2017 at 10:39 AM, Marina Popova wrote: > Hi, > I wanted to give this question a second try as I feel it is very > important to

Re: Kafka problem

Can you provide more information (such as pastebin of relevant logs) ? Cheers On Wed, Nov 22, 2017 at 1:55 AM, Linux实训项目 wrote: > Hi: >Kafka node zookeeper is lost after a period of time.What causes the > node to be lost in zookeeper？ > > >environment： >

Re: Too many open files in kafka 0.9

There is KAFKA-3317 which is still open. Have you seen this ? http://search-hadoop.com/m/Kafka/uyzND1KvOlt1p5UcE?subj=Re+Brokers+is+down+by+java+io+IOException+Too+many+open+files+ On Wed, Nov 29, 2017 at 8:55 AM, REYMOND Jean-max (BPCE-IT - SYNCHRONE TECHNOLOGIES)

Re: StackOverflowError in kafka-coordinator-heartbeat-thread with Kafka 1.0

Interesting. Looks like disconnection resulted in the stack overflow. I think the following would fix the overflow: https://pastebin.com/Pm5g5V2L On Thu, Dec 14, 2017 at 7:40 AM, Jörg Heinicke wrote: > > Hi everyone, > > We recently switched to Kafka 1.0 and are facing

Re: caching permantly disabled in Kafka Streams

In StreamsConfig.java , CACHE_MAX_BYTES_BUFFERING_CONFIG is defined as: .define(CACHE_MAX_BYTES_BUFFERING_CONFIG, Type.LONG, 10 * 1024 * 1024L, I think using numeral should be accepted (as shown by the Demo.java classes). On Thu, Dec 14,

Re: Kafka isync replica

Can you look at the log from controller to see if there is some clue w.r.t. partition 82 ? Was unclean leader election enabled ? BTW which release of Kafka are you using ? Cheers On Thu, Dec 14, 2017 at 11:49 AM, Tarun Garg wrote: > I checked log.dir of the all nodes and

Re: Producer is blocking at second commitTransaction

Can you capture stack trace on the broker and pastebin it ? Broker log may also provide some clue. Thanks On Mon, Dec 18, 2017 at 4:46 AM, HKT wrote: > Hello, > > I was testing the transactional message on kafka. > but I get a problem. > the producer always blocking at

Re: Need help in storing key along with value of _schemas topic using s3 sink connector.

Have you checked s3 connector issue listing ? Cheers On Wed, Dec 13, 2017 at 2:38 AM, Pratik Shah wrote: > Hi All, > I am using kafka s3 sink connector ( version 4.0) to backup the > _schemas topic for its backup. > The issue is that the content stored

Re: Class Files\OpenLink not found

Which release of Kafka did you download ? The error seems to be (in English): unable to find or load the main class I searched Kafka source code but didn't find reference to OpenLink. Can you give the complete error ? Thanks On Wed, Dec 13, 2017 at 12:55 AM, Thomas JASSEM

Re: ConcurrentModificationException

>From the stack trace, it seems you hit KAFKA-6349 (where Damian provided a PR). FYI On Mon, Dec 11, 2017 at 11:29 PM, Mr.Wang <1282183...@qq.com> wrote: > Hi~ > We found this error when we read kafka data using storm: > java.util.ConcurrentModificationException at >

Re: Producer is blocking at second commitTransaction

ing > transaction metadata from __transaction_state-15 > (kafka.coordinator.transaction.TransactionStateManager) > [2017-12-19 08:26:08,471] INFO [Transaction State Manager 0]: Loading > transaction metadata from __transaction_state-18 > (kafka.coordinator.transaction.Transacti

Re: Time-Based Index for Consuming Messages Up to Certain Timestamp

bq. an older timestamp that allowed I guess you meant 'than allowed' Cheers On Tue, Nov 21, 2017 at 2:57 PM, Matthias J. Sax wrote: > This is possible, but I think you don't need the time-based index for it :) > > You will just buffer up all messages for a 5 minute

Re: documentation link broken

See: https://github.com/apache/kafka/blob/1.0/clients/src/main/java/org/apache/kafka/clients/admin/AdminClient.java On Thu, Nov 2, 2017 at 5:51 AM, diane wrote: > Hi > > I was trying to look at the documentation for the AdminClient API , but > the link from page >

Re: Getting rid of Attempting to send response via channel for which there is no open connection

Have you seen this thread ? http://search-hadoop.com/m/Kafka/uyzND1ppGvmNscWc?subj=Re+Data+loss+while+upgrading+confluent+3+0+0+kafka+cluster+to+confluent+3+2+2 On Thu, Nov 2, 2017 at 1:46 PM, Karim Lamouri wrote: > Hi, > > I wanted to know in which cases those warning arise

Re: GlobalKTable never finishes restoring

Alex: In the future, please use pastebin if the log is not too large. When people find this thread in mailing list archive, the attachment wouldn't be there. Thanks On Tue, Nov 7, 2017 at 8:32 AM, Matthias J. Sax wrote: > Alex, > > I am not sure, but maybe it's a bug. I

Re: Migrating from log4j 1 to log4j 2

PropertyConfigurator is used here: https://github.com/apache/kafka/blob/trunk/tools/src/main/java/org/apache/kafka/tools/VerifiableLog4jAppender.java#L233 On Thu, Nov 9, 2017 at 2:17 PM, Arunkumar wrote: > Hi All > We have a requirement to migrate log4J 1.x to

Re: Kafka JVM heap limit

Did you use G1GC ? Thanks Original message From: John Yost Date: 11/8/17 5:48 AM (GMT-08:00) To: users@kafka.apache.org Cc: ja...@scholz.cz Subject: Re: Kafka JVM heap limit In addition, in my experience, a memory heap > 8 GB leads to long GC pauses

Re: Kafka Monitoring..

Have you seen this thread ? http://search-hadoop.com/m/Kafka/uyzND1Q6wyNBj42g?subj=Kafka+Monitoring On Wed, Nov 8, 2017 at 5:10 PM, chidigam . wrote: > Hi All, > What is the simplest way of monitoring the metrics in kaka brokers? > Is there any opensource available? >

Re: Client Offset Deleted by Broker without cause

bq. attached screenshots from the log viewer The screenshots didn't go through. Consider using 3rd party site. On Wed, Nov 1, 2017 at 9:18 AM, Elmar Weber wrote: > Hello, > > I had this morning the issue that a client offset got deleted from a > broker as it seems. > >

Re: Offset commit for partitions not owned by consumer

Can you pastebin relevant logs from client and broker ? Thanks On Fri, Nov 3, 2017 at 1:37 PM, Manan G wrote: > Hello, > > I am using 0.11.0.0 version of Kakfa broker and Java client library. My > consumer code tracks offsets for each assigned partition and at some time >

Re: Offset commit for partitions not owned by consumer

tebin.com/PuQhud91 > > Consumer 2 logs: https://pastebin.com/yfJDSGPA > > > > server.log: https://pastebin.com/QKpk0zLn > > controller.log: https://pastebin.com/9T0niwEw > > state-change.log: https://pastebin.com/nrftHPC9 > > > > > > On Fri, Nov 3, 2017 at 1:5

Re: Getting install error ...

In the guide, note the following (see bin/windows): change the script extension to .bat On Mon, Dec 11, 2017 at 2:57 AM, kish babu wrote: > Hi, > I am trying to setup Kafka on my windows box which has no cygwin. > > I am following instructions from > >

Re: Servers Getting disconnected

Please take a look at KAFKA-4477 On Mon, Oct 30, 2017 at 2:06 AM, Yuanjia wrote: > Hi, > My cluster version is 0.10.0.0, 10 nodes. One server has many error > log, like that > java.io.IOException: Connection to 6 was disconnected before the response > was read >

Re: [DISCUSS] KIP-213 Support non-key joining in KTable

Have you seen the email a moment ago from Onur which uses the same KIP number ? Looks like there was race condition in modifying wiki. Please consider bumping the KIP number. Thanks On Wed, Oct 25, 2017 at 4:14 PM, Jan Filipiak wrote: > Hello Kafka-users, > > I want

Re: Consumer poll returning 0 results

Can you provide a bit more information ? Release of Kafka Java / Scala version Thanks On Wed, Oct 25, 2017 at 6:40 PM, Susheel Kumar wrote: > Hello Kafka Users, > > I am trying to run below sample code mentioned in Kafka documentation under > Automatic Offset Committing

Re: Failure to reset consumer offsets for specific topics

08978014698 offset: 254410828 > > > timestamp: 1508978014959 offset: 254410834 > > > timestamp: 1508978015677 offset: 254410854 > > > timestamp: 1508978016980 offset: 254410870 > > > timestamp: 1508978017212 offset: 254410883 > > > >

Re: Failure to reset consumer offsets for specific topics

then tried to manually reset the consumer group's offset to a few > minutes before I restarted the broker, only to discover this strange > behavior where no matter which datetime value I provided, it kept resetting > to the latest offset. > > > On 25 October 2017 at 22:48, Ted Y

Re: Failure to reset consumer offsets for specific topics

Clarification: my most recent reply was w.r.t. the strange situation Dan described, not the offset resetting. On Wed, Oct 25, 2017 at 1:24 PM, Ted Yu <yuzhih...@gmail.com> wrote: > I wonder if you have hit KAFKA-5600. > > Is it possible that you try out 0.11.0.1 ? > > Than

Re: Failure to reset consumer offsets for specific topics

Do you mind providing a bit more information ? Release of Kafka you use Any difference between data1_log and the other, normal topic ? Probably check the broker log where data1_log is hosted - see if there is some clue. Thanks On Wed, Oct 25, 2017 at 12:11 PM, Dan Markhasin

Re: Kafka source code - Build Error

This should fix what you observed: https://github.com/apache/kafka/pull/4127 On Tue, Oct 24, 2017 at 12:21 PM, Vishal Shukla wrote: > Hi All, > I took latest source code from kafka git repo and tried to setup my local > env. in Eclipse. > I am getting a compilation error in

Re: Kafka source code - Build Error

JDK 1.8.0_91 Cheers On Tue, Oct 24, 2017 at 1:23 PM, Guozhang Wang <wangg...@gmail.com> wrote: > Hmm.. which Java version were you using? > > > Guozhang > > On Tue, Oct 24, 2017 at 1:09 PM, Ted Yu <yuzhih...@gmail.com> wrote: > > > This should fix what y

Re: Kafka Streams 0.11.0.1 - Task Assignment Stickiness Behavior

Eric: I wonder if it is possible to load up 1.0.0 RC3 on test cluster and see what the new behavior is ? Thanks On Tue, Oct 24, 2017 at 5:41 PM, Eric Lalonde wrote: > > >>> Could it be, that the first KafkaStreams instance was still in status > >>> "rebalancing" when you

Re: Re: Servers Getting disconnected

R's down to itself, and all clients work well. > I notice KAFKA-5153, it maybe same to mine. But it doesn't update for > a long time. > > > From: Ted Yu > Date: 2017-10-30 17:13 > To: users > Subject: Re: Servers Getting disconnected > Please take a look at KAFKA-4477

Re: Producer is blocking at second commitTransaction

ProducerRecord<Long, String> record = new > ProducerRecord<>("test", 0, (long) 0, Long.toString(0)); > producer.send(record); > producer.commitTransaction(); > producer.beginTransaction(); > record = new ProducerRec

Re: Kafka 2.11-1.1.0 crashes brokers and brings cluster down on Windows

rough debugger to find out which thread is >> locking the file. Apologise but it's a huge application so it might get me >> some time to get around it :) >> >> Please do update if you find something new. >> >> Regards, >> >> On 12 May 2018 at 22:15, Te

Re: Kafka 2.11-1.1.0 crashes brokers and brings cluster down on Windows

5:29,439] INFO Stopping serving logs in dir C:\kafka1 > (kafka.log.LogManager) > [2018-05-14 16:35:29,517] INFO [ExpirationReaper-1-topic]: Starting > (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) > [2018-05-14 16:35:29,517] ERROR Shutdown broker because all log

Re: Kafka offset problem when using Spark Streaming...

Can you give us more information: release of Spark and Kafka you're using anything interesting from broker / client logs - feel free to use pastebin to pass snippet if needed Thanks On Wed, May 9, 2018 at 3:42 AM, Pena Quijada Alexander < a.penaquij...@reply.it> wrote: > Hi all, > > We're

Re: Re: kafka streams with TimeWindows ,incorrect result

Noticed a typo in streams in subject. Corrected it in this reply. Original message From: 杰 杨 Date: 4/27/18 1:28 AM (GMT-08:00) To: 杰 杨 , users Subject: Re: Re: kafka steams with TimeWindows ,incorrect result and I

Re: source code location for KSQL

https://github.com/confluentinc/ksql FYI On Fri, Apr 27, 2018 at 4:23 PM, Henry Cai wrote: > I think KSQL is also open sourced, where is the source code location for > KSQL in github? > > Thanks. >

Re: Kafka 2.11-1.1.0 crashes brokers and brings cluster down on Windows

There are two outstanding issues: KAFKA-6059 and KAFKA-6200 which bear some resemblance. Can you try to find how the other process uses the file being deleted ? https://superuser.com/questions/117902/find-out-which-process-is-locking-a-file-or-folder-in-windows

Re: Help Needed: Leadership Issue upon Kafka Upgrade (ZooKeeper 3.4.9)

Was there any previous connectivity issue to 1.1.1.143:3888 before the upgrade ? I assume you have verified that connectivity between broker and 1.1.1.143 is Okay. Which zookeeper release are you running ? Cheers On Fri, May 11, 2018 at 3:16 PM, Raghav

Re: Consulting ReadOnlyKeyValueStore from Processor can lead to deadlock

bq. the docs and CachingKVS behavior could improve I would agree. Pointing out the usage of ReadWriteLock and mentioning the withCachingDisabled() method in doc would help other developers. On Thu, May 10, 2018 at 11:21 AM, Steven Schlansker < sschlans...@opentable.com> wrote: > > > On May 10,

Re: Kafka consumer slowly consume the data

bq. load on the node is increasing tremendously Can you do some profiling to see where the bottleneck was ? You can pastebin some stack traces. Which Kafka release do you use ? Thanks On Fri, May 11, 2018 at 6:41 AM, Karthick Kumar wrote: > Hi, > > I'm using tomcat

Re: Kafka Scala debugging and diagnostics

16 May 2018 at 23:04, M. Manna <manme...@gmail.com> wrote: > > > Must have missed that in Kafka-run-class.bat which sets suspend=n by > > default. Thanks for pointing that out, giving it a try. > > > > > > > > And didn > > > > On Wed, 16 May 2018

Re: Kafka Scala debugging and diagnostics

Have you set the following two environment variables: export KAFKA_DEBUG=y; export DEBUG_SUSPEND_FLAG=y; The above would freeze the process until debugger is attached. Cheers On Wed, May 16, 2018 at 2:46 PM, M. Manna wrote: > Hello, > > Is there anyone who can provide a

Re: NPE in low level Kafka topology

Can you show the related code from OneToManyGroupedProcessor ? Thanks On Mon, Jun 18, 2018 at 4:29 AM, Frank Lyaruu wrote: > Hi, I've upgraded our 0.11 based stream application to the trunk version, > and I get an intermittent NPE. It's is quite a big topology, and I haven't > succeeded in

Re: Details of segment deletion

Minor clarification (since new segment appeared twice) : bq. before a new one is deleted. The 'new one' (in the last sentence) would become old when another segment is created. Cheers On Tue, Jun 12, 2018 at 6:42 PM, Gwen Shapira wrote: > See below: > > On Mon, Jun 11, 2018 at 3:36 AM,

Re: INVALID_FETCH_SESSION_EPOCH after upgrade to 1.1.0

Before Errors.INVALID_FETCH_SESSION_EPOCH is returned, FetchSession.scala would log the reason for the response. There are 3 cases, 2 with info log and 1 with debug log. Here is one code snippet: if (session.epoch != reqMetadata.epoch()) { debug(s"Created a new error

Re: INVALID_FETCH_SESSION_EPOCH after upgrade to 1.1.0

13, 2018 at 9:08 AM, Mark Anderson wrote: > Ted > > I don't see any other INFO log messages so I assume that means it is the > DEBUG case I'm seeing? > > I don't have DEBUG enabled at the moment. > > Thanks > > On Wed, 13 Jun 2018, 00:2

Re: [VOTE] 1.0.2 RC1

+1 Ran test suite. On Fri, Jun 29, 2018 at 10:02 PM, Matthias J. Sax wrote: > Hello Kafka users, developers and client-developers, > > This is the second candidate for release of Apache Kafka 1.0.2. > > This is a bug fix release addressing 27 tickets: >

Re: Partitions reassignment is failing in Kafka 1.1.0

See KAFKA-7124 Cheers On Sun, Jul 1, 2018 at 3:26 AM, Debraj Manna wrote: > Is there any bug filed for this ? > > On Sun, Jul 1, 2018 at 6:37 AM, Ted Yu wrote: > > > Code snippet from ReassignPartitionsCommand.scala : > > > >

Re: Cannot run the inbuild zookeeper server.

'Address already in use' indicates that there was some (zookeeper) server running on the same machine. Can you double check ? Thanks On Sun, Jul 1, 2018 at 4:45 AM, Vineet Herenj wrote: > Hello, > > I am not able to run the inbuild zookeeper server. > Following error shows: > >

Re: [VOTE] 0.10.2.2 RC1

+1 Ran test suite. Checked signatures. On Fri, Jun 29, 2018 at 10:21 AM, Jason Gustafson wrote: > +1 (binding). I checked release notes, documentation, and went through the > quickstart. > > Thanks Matthias! > > On Fri, Jun 22, 2018 at 6:43 PM, Matthias J. Sax > wrote: > > > Hello Kafka

Re: Partitions reassignment is failing in Kafka 1.1.0

Seems to be related to KIP-113. server.properties didn't go thru. Do you mind pastebin'ing its content ? If you can pastebin logs from broker, that should help. Thanks On Fri, Jun 29, 2018 at 10:37 AM, Debraj Manna wrote: > Hi > > I altered a topic like below in kafka 1.1.0 > >

Re: [DISCUSS] KIP-328: Ability to suppress updates for KTables

hods I listed. > > For example: > > windowCounts > .suppress( > Suppression > .suppressLateEvents(Duration.ofMinutes(10)) > .suppressIntermediateEvents( > IntermediateSuppression.emitAfter(Duration.ofMinutes(10)) > )

Re: [VOTE] 1.1.1 RC1

+1 Ran test suite. Checked signatures. On Fri, Jun 22, 2018 at 12:14 PM, Vahid S Hashemian < vahidhashem...@us.ibm.com> wrote: > +1 (non-binding) > > Built from source and ran quickstart successfully on Ubuntu (with Java 8). > > Thanks Dong! > --Vahid > > > > From: Dong Lin > To:

Re: [VOTE] 0.11.0.3 RC0

+1 Checked signatures. Ran unit test suite. On Fri, Jun 22, 2018 at 4:56 PM, Vahid S Hashemian < vahidhashem...@us.ibm.com> wrote: > +1 (non-binding) > > Built from source and ran quickstart successfully on Ubuntu (with Java 8). > > Thanks Matthias! > --Vahid > > > > > From: "Matthias J.

Re: [VOTE] 1.1.1 RC0

+1 Ran unit test suite which passed. Checked signatures. On Tue, Jun 19, 2018 at 4:47 PM, Dong Lin wrote: > Re-send to kafka-clie...@googlegroups.com > > On Tue, Jun 19, 2018 at 4:29 PM, Dong Lin wrote: > > > Hello Kafka users, developers and client-developers, > > > > This is the first

Re: Partitions reassignment is failing in Kafka 1.1.0

;> socket.request.max.bytes=104857600 > > >> log.dirs=/var/lib/kafka/kafka-logs > > >> num.recovery.threads.per.data.dir=1 > > >> log.retention.hours=36 > > >> log.retention.bytes=1073741824 > > >> log.segment.bytes=536870912

Re: Kafka-Connect : Invalid value java.sql.SQLException: No suitable driver found for jdbc:mysql://127.0.0.1:3306/connect_test

Have you seen this thread ? https://stackoverflow.com/questions/49433199/kafka-connect-jdbc-source-connector-not-working-for-microsoft-sql-server On Tue, May 1, 2018 at 5:52 AM, Jagannath Bilgi wrote: > Hi Team, > Getting subject error message while trying to create

Re: [DISCUSS] KIP-213 Support non-key joining in KTable

Please fill in thread link and JIRA number. * prefix of the output key wich is the same as serializing K Typo: wich * @param joiner Please add explanation for joiner (in both APIs) Please add javadoc for valueOtherSerde and joinValueSerde * an event in this KTabl

Re: reg Kafka Node num

Here is related code: log.debug("Using older server API v{} to send {} {} with correlation id {} to node {}", header.apiVersion(), clientRequest.apiKey(), request, clientRequest.correlationId(), nodeId); 2147483647 is the node Id. On Fri, Oct 20, 2017 at

Re: reg kafka API versions

See code in NetworkClient : int latestClientVersion = clientRequest .apiKey().latestVersion(); if (header.apiVersion() == latestClientVersion) { ... } else { log.debug("Using older server API v{} to send {} {} with correlation id {} to node

  1   2   >