On Thu, Apr 30, 2015 at 2:15 AM, Nimi Wariboko Jr n...@channelmeter.com
wrote:
My mistake, it seems the Java drivers are a lot more advanced than the
Shopify's Kafka driver (or I am missing something) - and I haven't used
Kafka before.
With the Go driver - it seems you have to manage offsets
Related to this topic: why the choice of murmur2 over murmur3? I'm not
super-familiar with the differences between the two, but I'd assume murmur3
would be faster or have a more even distribution or something.
Evan
P.S. Also, there appear to be many murmur3 implementations for other
languages,
that purgatory has been
rewritten (again) in 0.8.3, so might it be worth trying a trunk build? Is
there an ETA for a beta release of 0.8.3?
Thanks,
Evan
On Tue, Apr 14, 2015 at 8:40 PM, Evan Huus evan.h...@shopify.com wrote:
On Tue, Apr 14, 2015 at 8:31 PM, Jiangjie Qin j...@linkedin.com.invalid
Hi Victor, two points:
- Based on the backtrace, you are using a very old version of Sarama. You
might have better luck using a more recent stable version.
- Are you setting `MaxBufferTime` in the configuration to 0 or a very small
value? If so the loop will spin on that timer. Try making this
= 1024
How's it versioned? Should i just download zip file from project github?
By default go will use whatever version you originally checked out with `go
get` or the like. You can use a properly versioned copy via
http://gopkg.in/Shopify/sarama.v1.
On Wed, Apr 15, 2015 at 12:48 PM, Evan Huus
, 2015 at 1:57 PM, Evan Huus evan.h...@shopify.com
wrote:
The field is a time.Duration (https://golang.org/pkg/time/#Duration)
and
it
is the maximum duration to buffer messages before triggering a flush to
the
broker. The default was 1 millisecond (this value does not even exist
for efficiency, whatever your use case dictates.
Evan
On Wed, Apr 15, 2015 at 1:47 PM, Victor L vlyamt...@gmail.com wrote:
I inherited this code, it just set to 1000, what would be reasonable time:
1000 * time.Millisecond?
On Wed, Apr 15, 2015 at 1:41 PM, Evan Huus evan.h...@shopify.com wrote
the
`fetch.purgatory.purge.interval.requests` and
`producer.purgatory.purge.interval.requests` configuration values on
cluster B from 1000 to 200, but it had no effect, which I find really weird.
Thanks,
Evan
Jiangjie (Becket) Qin
On 4/14/15, 6:55 AM, Evan Huus evan.h...@shopify.com wrote:
Any ideas on this? It's still occurring
Any ideas on this? It's still occurring...
Is there a separate mailing list or project for mirrormaker that I could
ask?
Thanks,
Evan
On Thu, Apr 9, 2015 at 4:36 PM, Evan Huus evan.h...@shopify.com wrote:
Hey Folks, we're running into an odd issue with mirrormaker and the fetch
request
Hey Folks, we're running into an odd issue with mirrormaker and the fetch
request purgatory on the brokers. Our setup consists of two six-node
clusters (all running 0.8.2.1 on identical hw with the same config). All
normal producing and consuming happens on cluster A. Mirrormaker has been
set up
Hey all, it seems that 0.8.2 has added a handful more errors to the
protocol which are not yet reflected on the wiki page [1]. Specifically,
[2] seems to indicate that codes 17-20 now have associated meanings.
My questions are:
- Which of these are exposed publicly? (for example, the existing
- - - - - - - - - - - - - - - - -
http://www.stealth.ly
- - - - - - - - - - - - - - - - -
On Wed, Mar 4, 2015 at 12:44 PM, Evan Huus evan.h...@shopify.com wrote:
Hey all, it seems that 0.8.2 has added a handful more errors to the
protocol which are not yet reflected on the wiki page [1]. Specifically,
[2] seems
/ KafkaProducer class. Could you file a JIRA?
Guozhang
On Sun, Mar 1, 2015 at 7:11 PM, Evan Huus evan.h...@shopify.com wrote:
Which I think is my point - based on my current understanding, there is
*no* way to find out the total number of partitions for a topic besides
hard-coding it or manually
On Sun, Mar 1, 2015 at 1:46 AM, Guozhang Wang wangg...@gmail.com wrote:
Hi Honghai,
1. If a partition has no leader (i.e. all of its replicas are down) it will
become offline, and hence the metadata response will not have this
partition's info.
If I am understanding this correctly, then
send retry successfully.
Hence, if some of your partitions becomes offline for too long some data
will be lost on the producer side (unless you set infinite retry, of
course).
Guozhang
On Sun, Mar 1, 2015 at 5:25 AM, Evan Huus evan.h...@shopify.com wrote:
On Sun, Mar 1, 2015 at 1:46 AM
use the total number of partitions for the topic, not
the number of available partitions to compute partition id.
Guozhang
On Sun, Mar 1, 2015 at 6:42 PM, Evan Huus evan.h...@shopify.com wrote:
My concern is more with the partitioner that determines the partition of
the message. IIRC
On Thu, Feb 19, 2015 at 8:43 PM, Joel Koshy jjkosh...@gmail.com wrote:
If you are using v0 of OffsetCommit/FetchRequest then you can issue
that to any broker. For version 0 you will need to issue it to the
coordinator. You can discover the coordinator by sending a
ConsumerMetadataRequest to
Hi there,
As far as I know, RequestTimedOut is returned exclusively when the
`Timeout` value provided in a Produce Request is exceeded. The message
will likely still be committed on the local broker, but it means that
some replicas may not have received it yet. I'm honestly not sure what
the use
request yourself and then you need to handle the
error code accordingly.
I agree that such documentation is kind of lacking for now, and we would
better add that in the response class Java doc.
Guozhang
On Tue, Nov 25, 2014 at 12:29 PM, Evan Huus evan.h...@shopify.com wrote:
Hi folks
Hi folks,
I was wondering in which cases the PartitionMetadata section of a
MetadataResponse [1] can contain useful information?
I had been working under the assumption that when the PartitionErrorCode
was 0 (NoError) then the rest of the data would be legitimate, and that
when the error was
20 matches
Mail list logo
