Hi,
I think for 2) you can use Kafka Consumer and push messages to vertex event
bus, which already have REST implementation (vertx-jersey).
I would say, Vertx cluster can be used as receive data irrespective of
topic and then publish to particular kafka topic. Then consume messages
from kafka by
Hello Apache Kafka community,
I like that kafka-clients is now separate module, and has no scala
dependency even. I'd like to propose that kafka.admin package gets
published as separate module too.
I'm writing some tests, and to be able to use kafka.admin tools/utils in
them I have to bring in
You can use kafka-console-consumer consuming the topic from the beginning
*kafka-console-consumer.sh --zookeeper localhost:2181 --topic test
--from-beginning*
On Thu, Mar 26, 2015 at 12:17 AM, Victor L vlyamt...@gmail.com wrote:
Can someone let me know how to dump contents of topics?
I have
Hi Stevo,
JFYI: we are working now on new centralized API for Admin commands.
This will include:
- Public API to perform TopicCommands (Create/Alter/Delete/List/Describe)
- Out-of-box java client for Admin API: AdminClient - will be part of
/clients
- Interactive cli for admin commands
The plan
Yeah, that would be great having a separate admin package. +1.
Thanks,
Mayuresh
On Wed, Mar 25, 2015 at 8:26 AM, Stevo Slavić ssla...@gmail.com wrote:
Hello Apache Kafka community,
I like that kafka-clients is now separate module, and has no scala
dependency even. I'd like to propose that
You can use the DumpLogSegment tool.
Thanks,
Mayuresh
On Wed, Mar 25, 2015 at 9:17 AM, Victor L vlyamt...@gmail.com wrote:
Can someone let me know how to dump contents of topics?
I have producers sending messages to 3 brokers but about half of them don't
seem to be consumed. I suppose they
Can someone let me know how to dump contents of topics?
I have producers sending messages to 3 brokers but about half of them don't
seem to be consumed. I suppose they are getting stuck in queues but how can
i figure out where?
Thks,
For 3, Confluent wrote a REST proxy that's pretty comprehensive. See the
docs: http://confluent.io/docs/current/kafka-rest/docs/intro.html and a
blog post describing it + future directions:
http://blog.confluent.io/2015/03/25/a-comprehensive-open-source-rest-proxy-for-kafka/
There are a few other
Ah, great, thanks for heads up Andrii!
On Mar 25, 2015 5:39 PM, Andrii Biletskyi andrii.bilets...@stealth.ly
wrote:
Hi Stevo,
JFYI: we are working now on new centralized API for Admin commands.
This will include:
- Public API to perform TopicCommands (Create/Alter/Delete/List/Describe)
-
DumpLogSegments will give you output something like this :
offset: 780613873770 isvalid: true payloadsize: 8055 magic: 1 compresscodec:
GZIPCompressionCodec
If this is what you want you can use the tool, to detect if the messages
are getting to your brokers.
Console-Consumer will output the
Hello Kafka Community,
What is expected behavior on Producer side when one or more Brokers’ disk
is full, but have not reached retention period for topics (by size or by
time limit).
Does producer send data to that particular brokers and/or Producer Queue
gets full and always throws Queue
If you want to do a seamless migration. I think a better way is to build a
cross datacenter Kafka cluster temporarily. So the process is:
1. Add several new Kafka brokers in your new datacenter and add them to
the old cluster.
2. Use replica assignment tool to reassign all the partitions to
Hi Team,
in my project, we have built a new datacenter for Kafka brokers and wants
to migrate from current datacenter to new one.
Switching producers and consumers wont be a problem provided New Datacenter
has all the messages of existing Datacenter.
i have only 1 topic with 2 partition that
You can use the Mirror maker to move data from one data center to other and
once all the data has been moved you can shut down the source data center
by doing a controlled shutdown.
Thanks,
Mayuresh
On Wed, Mar 25, 2015 at 2:35 PM, Jiangjie Qin j...@linkedin.com.invalid
wrote:
If you want to
Ewen Cheslack-Postava ewen@... writes:
Im also searching the shortest way to find topic partition count, so that the
initialization code in the thread pool can set up the right number of threads.
so far i found below is the shortest way.
public static void main(String[] args){
15 matches
Mail list logo
