Thanks, Hans for the insight. Will use compacted topic.
On Thu, Dec 15, 2016 at 3:53 PM, Hans Jespersen wrote:
> for #2 definitely use a compacted topic. Compaction will remove old
> messages and keep the last update for each key. To use this function you
> will need to publish messages as Key/V
for #2 definitely use a compacted topic. Compaction will remove old
messages and keep the last update for each key. To use this function you
will need to publish messages as Key/Value pairs. Apache Kafka 0.10.1 has
some important fixes to make compacted topics more reliable when scaling to
large nu
Thanks, Kenny for confirming. Message updates I mean to say that for same
document/message there will be updates coming in (for e.g. person details
may change). As you mentioned using the proper key should make that happen
so good on that.
On Thu, Dec 15, 2016 at 1:16 PM, Kenny Gorman wrote:
>
A couple thoughts..
- If you plan on fetching old messages in a non-contiguous manner then this may
not be the best design. For instance, “give me messages from mondays for the
last 3 quarters” is better served with a database. But if you want to say “give
me messages from the last month until
Sorry, I do not have any info on backup and recovery plan at this point of
time. Please consider both cases (no backup AND back up)
On Thu, Dec 15, 2016 at 1:06 PM, Tauzell, Dave wrote:
> What is the plan for backup and recovery of the kafka data?
>
> -Dave
>
> -Original Message-
> From
What is the plan for backup and recovery of the kafka data?
-Dave
-Original Message-
From: Susheel Kumar [mailto:susheel2...@gmail.com]
Sent: Thursday, December 15, 2016 12:00 PM
To: users@kafka.apache.org
Subject: Kafka as a database/repository question
Hello Folks,
I am going thru an
