Re: Number of kafka topics/partitions supported per cluster of n nodes

-- Jiefu Gong University of California, Berkeley | Class of 2017 B.A Computer Science | College of Letters and Sciences jg...@berkeley.edu elise...@berkeley.edu | (925) 400-3427

Understanding the Simple Consumer

help would definitely be appreciated! Thanks! -- Jiefu Gong University of California, Berkeley | Class of 2017 B.A Computer Science | College of Letters and Sciences jg...@berkeley.edu elise...@berkeley.edu | (925) 400-3427

Re: KafkaConfigurationError: No topics or partitions configured

From: JIEFU GONG jg...@berkeley.edu Sent: Tuesday, July 28, 2015 01:31 PM To: users@kafka.apache.org Subject: Re: KafkaConfigurationError: No topics or partitions configured Can you confirm that there are indeed messages in the topic that you published

Re: KafkaConfigurationError: No topics or partitions configured

keith.wi...@atigeo.com [atigeo]http://atigeo.com/ [twitter]https://twitter.com/atigeo [LinkedIn] https://www.linkedin.com/company/atigeo [YouTube] https://www.youtube.com/user/AtigeoXpatterns/ [blog] http://xpatterns.com/blog/ -- Jiefu Gong University of California, Berkeley | Class

Re: Log Deletion Behavior

PM, JIEFU GONG jg...@berkeley.edu wrote: Mayuresh, thanks for your comment. I won't be able to change these settings until next Monday, but just so confirm you are saying that if I restart the brokers my logs should delete themselves with respect to the newest settings, correct? ᐧ

Log Deletion Behavior

/anticipated/wanted behavior? Thanks for the help! -- Jiefu Gong University of California, Berkeley | Class of 2017 B.A Computer Science | College of Letters and Sciences jg...@berkeley.edu elise...@berkeley.edu | (925) 400-3427

Re: Log Deletion Behavior

not set other retention settings. Just use the hours settings. Let me know about this :) Thanks, Mayuresh On Fri, Jul 24, 2015 at 6:43 PM, JIEFU GONG jg...@berkeley.edu wrote: Mayuresh, thanks for your comment. I won't be able to change these settings until next Monday, but just so

Re: Log Deletion Behavior

log.retention.minutes log.retention.hours We should probably be having just the milliseconds. Thanks, Mayuresh On Fri, Jul 24, 2015 at 4:12 PM, JIEFU GONG jg...@berkeley.edu wrote: Hi all, I have a few broad questions on how log deletion works, specifically in conjunction

Re: kafka-topics.sh - Include topic deletion information?

have this info via the Kafka scripts. -Jaikiran -- Regards, Ashish -- Jiefu Gong University of California, Berkeley | Class of 2017 B.A Computer Science | College of Letters and Sciences jg...@berkeley.edu elise...@berkeley.edu | (925) 400-3427

Implementing a custom partitioner

the producer has the line: this.partitioner = new Partitioner(); which more or less ignores my input. Any ideas? Help is appreciated! -- Jiefu Gong University of California, Berkeley | Class of 2017 B.A Computer Science | College of Letters and Sciences jg...@berkeley.edu elise...@berkeley.edu

Re: Implementing a custom partitioner

/KIP-+22+-+Expose+a+Partitioner+interface+in+the+new+producer -- Harsha On July 21, 2015 at 2:27:05 PM, JIEFU GONG (jg...@berkeley.edu) wrote: Hi all, If I wanted to write my own partitioner, all I would need to do is write a class that extends Partitioner and override the partition function

Re: Delete topic using kafka-topics.sh also not working (or takes forever)

(org.apache.zookeeper.server.PrepRequestProcessor) Could someone please let me know if I am missing anything? Thank you, Siva. -- Regards, Ashish -- Jiefu Gong University of California, Berkeley | Class of 2017 B.A Computer Science | College of Letters and Sciences jg

Deleting persisted Kafka messages?

like I only configure the directory where logs are written, can anyone point me in the right direction? Much appreciated! -- Jiefu Gong University of California, Berkeley | Class of 2017 B.A Computer Science | College of Letters and Sciences jg...@berkeley.edu elise...@berkeley.edu | (925) 400

Re: Deleting persisted Kafka messages?

everything older than 60 seconds (eventually). bin/kafka-topics.sh --zookeeper zookeeper.yoursite.com --alter --topic your-topic --config retention.ms=6 Remember to set it back to something sane after cleanup. Foo On Thu, Jul 16, 2015 at 4:30 PM, JIEFU GONG jg...@berkeley.edu wrote: I'm

Re: Kafka partitioning is pretty much broken

in old consumer so I will be sure that the partition the message will be sent to is super random. But that does not seems to work either. Please people, help me. -- Stefan Miklosovic -- Jiefu Gong University of California, Berkeley | Class of 2017 B.A Computer Science | College

Re: stunning error - Request of length 1550939497 is not valid, it is larger than the maximum size of 104857600 bytes

export KAFKA_HEAP_OPTS=-Xmx256M -Xms128M in kafka-server-start.sh So.why? -- Jiefu Gong University of California, Berkeley | Class of 2017 B.A Computer Science | College of Letters and Sciences jg...@berkeley.edu elise...@berkeley.edu | (925) 400-3427

Re: How to run the three producers test

:48,056] WARN Error in I/O with producer1-link-0/ 192.168.1.2 (org.apache.kafka.common.network.Selector) java.net.ConnectException: Connection refused. What could be the cause? Thank you guys! On Tue, Jul 14, 2015 at 2:47 PM, JIEFU GONG jg

Re: How to run the three producers test

on the brokers, it seems that the zookeeper or the kafka server process is not running on this broker...Thank you guys. I will see if it happens again. On Tue, Jul 14, 2015 at 4:53 PM, JIEFU GONG jg...@berkeley.edu wrote: Hmm..yeah some error logs would be nice like Gwen pointed out. Do

Re: stunning error - Request of length 1550939497 is not valid, it is larger than the maximum size of 104857600 bytes

taking a look at this internally. Do you have any thoughts on this? -Todd On Tue, Jul 14, 2015 at 11:18 AM, JIEFU GONG jg...@berkeley.edu wrote: @Gwen I am having a very very similar issue where I am attempting to send a rather small message and it's blowing up on me (my specific error

Re: How to run the three producers test

-- Jiefu Gong University of California, Berkeley | Class of 2017 B.A Computer Science | College of Letters and Sciences jg...@berkeley.edu elise...@berkeley.edu | (925) 400-3427

Re: kafka benchmark tests

more time fiddling with tweaking producer performance may have more insight. -Ewen On Mon, Jul 13, 2015 at 10:08 AM, JIEFU GONG jg...@berkeley.edu wrote: Hi all, I was wondering if any of you guys have done benchmarks on Kafka performance before

kafka benchmark tests

tests of 3x synchronous replication and 3x async replication he changed the batch.size, as well as why he is seemingly publishing to incorrect topics: Configs: https://gist.github.com/jkreps/c7ddb4041ef62a900e6c Any help is greatly appreciated! -- Jiefu Gong University of California, Berkeley

Re: performance benchmarking of kafka

the following error: Error: Could not find or load main class org.apache.kafka.client.tools.ProducerPerformance What should I fix? Thank you! -- Jiefu Gong University of California, Berkeley | Class of 2017 B.A Computer Science | College of Letters and Sciences jg...@berkeley.edu elise

Sending a list of ProducerRecords

not have the ability to send multiple producerRecords using one send call. However, in the producer under javaapi, it's possible to send a list of keyedMessages. Can someone explain the difference or point me in the right direction? Much appreciated. -- Jiefu Gong University of California

Re: Kafka consumer does not work from eclipse when kafka running inside VM

. Kafka is running on port 6667 and zookeeper on 2181. -- Regards Shivam Tiwari -- Regards Shivam Tiwari -- Jiefu Gong University of California, Berkeley | Class of 2017 B.A Computer Science | College of Letters and Sciences jg...@berkeley.edu elise...@berkeley.edu

Kafka rebalancing and general questions

Hi all, I've very recently become interested in Kafka and have studied it a great deal but find myself confused on some of the more basic design aspects of the technology and despite considering various online resources still haven't found the answer I was looking for -- hoping this community