In Kafka, we detect failures using ZK. So, if the network connectivity btw
the producer and the broker is down, but the one btw the broker and ZK is
up, we assume the broker is still alive and will continue to send the data
to it. Within the the same data center, we assume this is extremely rare.
I
Hey guys,
Sriram and I took another pass at the site and documentation:
http://kafka.apache.org
This includes the following changes:
1. New logo!
2. Get rid of blue header
3. New font (Google webfont, Source Sans Pro)
4. Update the information on the design section to cover 0.8.
5. Move to a si
Kam,
I am perfectly fine if you pick this up. After thinking about it for a
while, we are going to upgrade to Kafka 0.8.0 and also use Camus as it
more closely matches our use case, with the caveat of we do not use Avro.
With that said, I will try and work on the back-port of custom data writer
pat
We've done a bit of work over at Wikimedia to debianize Kafka and make it
behave like a regular service.
https://github.com/wikimedia/operations-debs-kafka/blob/debian/debian
Most relevant, Ken, is an init script for Kafka:
https://github.com/wikimedia/operations-debs-kafka/blob/debian/debian
I would like to do this refactoring since I did a high level consumer a while
ago.
A few weeks ago I had opened KAFKA-949 Kafka on Yarn which I was also hoping to
add to contribute.
It's almost done. KAFKA-949 is paired with BIGTOP-989 which adds kafka 0.8 to
the bigtop distribution.
KAFKA-949
You can package the lib to use another Scala version in 0.7.X
project/build.properties
Change
build.scala.versions=2.8.0
And then ./sbt package
To get the lib for your Scala version
/***
Joe Stein
Founder, Principal Consultant
Big Data Open Source S
Hi guys,
Is it possible to compile Kafka 0.7.2 for Scala 2.9? Has anyone done this?
Thanks in advance.
--
Ian Friedman
The goal is to use sync producer and find out that network is down as
soon as possible.
--
Viktor
2013/8/13 Viktor Kolodrevskiy :
> Felix,
> the thing is that I was using sync producer.
>
> --
> Viktor
>
> 2013/8/13 Felix GV :
>> Async production is meant to work this way. You have no delivery gu
Felix,
the thing is that I was using sync producer.
--
Viktor
2013/8/13 Felix GV :
> Async production is meant to work this way. You have no delivery guarantee
> nor any exception because the producer sends the message independently of
> the code that called the aync production function.
>
> It i
Async production is meant to work this way. You have no delivery guarantee
nor any exception because the producer sends the message independently of
the code that called the aync production function.
It is meant to be faster than sync production, but it is obviously intended
for non-critical messa
That's a classic: just delete your ~/.ivy2 as well and be prepared to
download the internet again :(
Erik.
Op 06-08-13 05:36, Rob Withers schreef:
I deleted all the sbt project and target stuff, for both ~/.sbt and
kafka/project/build. I had previously had php stuff in my global sbt stuf
The closed by interrupt exception is because the underlying
"leader-finder-thread" has been interrupted - this is done for e.g.,
on a rebalance. Do you see any rebalancing messages in your consumer
logs? How many topics are you consuming? Are the topics present on the
broker and is there a leader f
Very clean logs.
Just deletes and rolls messages:
[2013-08-12 16:49:15,371] INFO [Kafka Log on Broker 4], Rolling incoming-6
due to full data log (kafka.log.Log)
[2013-08-12 16:49:15,429] INFO [Kafka Log on Broker 4], Rolling log
'incoming-6' to /mnt/data/kafka/incoming-6/00434346.log
The consumer uses a sync.producer to look up topic metadata for topics
that it consumes. For some reason, it is unable to establish a
connection to the brokers. Are the brokers all up with clean logs?
On Mon, Aug 12, 2013 at 8:39 AM, Eric Sites wrote:
> I am getting some new messages in my 0.8 Ja
I am getting some new messages in my 0.8 Java producer app.
I am running 6 Kafka brokers.
The app producing these logs messages is not doing anything, i.e. Not producing
messages.
Anywhere form every 5 minutes to 10 seconds sometimes.
[2013-08-12 14:45:20,714] ERROR Producer connection to kafk
This is the Kafka C client for 0.8 we are using at Wikimedia:
https://github.com/edenhill/librdkafka
If you're using Debian/Ubuntu: you use the debian branch here to build a .deb:
https://github.com/paravoid/librdkafka/tree/debian
On Aug 12, 2013, at 12:06 AM, Jun Rao wrote:
> At Linked
16 matches
Mail list logo
