Yeah the real question is really are the products built on top of Kafka
(Kafka with a hat on). The last place I worked we ended up using Kinesis
rather than Kafka basically for the reason Niek mentions, it seemed easier
to accept the limitations and pay Amazon rather than run Kafka (small
company <
I've wondered that about Azure Event Hubs as well. They both use a
different consumer offset tracking mechanism than the one in 0.8 for their
higher level consumers.
Christian
On Thu, Nov 13, 2014 at 2:32 PM, Joseph Lawson wrote:
> Oh man they look similar. Any comments?
>
I used the 0.8.2 producer in a 0.8.1 cluster in a nonproduction
environment. No problems to report it worked great, but my testing at that
time was not particularly extensive for failure scenarios.
Christian
On Wed, Nov 12, 2014 at 10:37 PM, Shlomi Hazan wrote:
> I was asking to know if there's
I may be out of date, but I believe security measures are only in the
proposal stage. Your use case most likely involves sending data from the
internet at large to the Kafka instance. This will result in all data sent
to the Kafka instance being consumable by the internet at large. This is
unlikely
Knowing that the partitioning is consistent for a given key means that
(apart from other benefits) a given consumer only deals with a partition of
the keyspace. So if you are in a system with tens of millions of users each
consumer only has to store state on a small number of them with
inconsistent
Wouldn't this work only for producers using random partitioning?
On Tue, Oct 14, 2014 at 1:51 PM, Kyle Banker wrote:
> Consider a 12-node Kafka cluster with a 200-parition topic having a
> replication factor of 3. Let's assume, in addition, that we're running
> Kafka v0.8.2, we've disabled uncle
Albert, you certainly can use Kafka (and it will probably work quite well)
you'll just need to make sure your consumers are written to match the
available options. I think I may not have a good picture of what you need
to do. Is it that you have a stream of documents coming in and then each
documen
at 5:45 PM, Patrick Barker
wrote:
> O, I'm not trying to use it for persistence, I'm wanting to sync 3
> databases: sql, mongo, graph. I want to publish to kafka and then have it
> update the db's. I'm wanting to keep this as efficient as possible.
>
> O
I would say that it depends upon what you mean by persistence. I don't
believe Kafka is intended to be your permanent data store, but it would
work if you were basically write once with appropriate query patterns. It
would be an odd way to describe it though.
Christian
On Fri, Sep 12, 2014 at 4:0
Based on the phrasing of your first question I might recommend taking
either a closer look at how Kafka works. Kafka stores data on its broker
servers in its own fashion as that is a key part of what makes it useful.
It is not written to use another database for message storage. Consumers of
the da
I believe there are architectures for the chat system that can use Kafka in
a sensible fashion to achieve certain of the difficult aspects. However
doing partition per user would not be advisable, nor I imagine would
relying on Kafka's storage for checking for past or expired messages. (I've
done t
I believe I downloaded from trunk and compiled a jar from that. The hardest
part of that seemed to be configuring gradle to sign the jar having never
done it before.
Christian
On Fri, Aug 15, 2014 at 11:00 AM, Roger Hoover
wrote:
> Hi,
>
> I want to try out the new producer api
> (org.apache.k
Message retention in Kafka is disconnected from message consumption.
Messages are all persisted to disk and the queues do not need to fit in RAM
unlike some other systems. There are configuration values that control
maximum log size in terms of MB and the duration of retention which is
typically in
This code says to send this message infinitely as fast as the machine can
thereby consuming as much of one CPU as possible. You may want to consider
an alternate test, perhaps one that records the number of messages sent in
a given time period.
> > public static void main(String[] args) {
> >
Thanks, that's quite helpful. According to this post,
http://blog.empathybox.com/ , it looks like it will be beta then which
seems good enough. Assuming that the beta designation is correct, is that
because it won't have as many features/the same flexibility as it's
expected to by 0.9?
Christian
15 matches
Mail list logo
