Here was the JIRA:
https://issues.apache.org/jira/browse/KAFKA-191
-Jay
On Thu, May 10, 2012 at 3:47 PM, Jay Kreps wrote:
> Our current understanding is that there is synchronization in linux so
> the underlying fsync and write calls actually block each other so even
> if you have a separate th
Our current understanding is that there is synchronization in linux so
the underlying fsync and write calls actually block each other so even
if you have a separate thread do the flush it doesn't help. We would
be interested in finding a way to work around this.
Good blog on this:
http://antirez.c
>> couldn't this be written in a way that it doesn't block?
This can't be done since file channels are blocking in nature.
Thanks,
Neha
On Wed, May 9, 2012 at 9:35 PM, S Ahmed wrote:
> couldn't this be written in a way that it doesn't block? or limits the
> time, like it makes a copy of the re
couldn't this be written in a way that it doesn't block? or limits the
time, like it makes a copy of the reference, then replaces it when a newed
up channel (synchronized).
Oh, this isn't possible because the var is mapped to a file at the o/s
level?
On Tue, May 8, 2012 at 5:38 PM, Jay Kreps wro
filechannel.force() always fully syncs the file to disk. This is done
irrespective of message boundaries. The file is locked during this
time so other appends are blocked.
-Jay
On Tue, May 8, 2012 at 1:44 PM, S Ahmed wrote:
> Slowly trying to understand it, have to wramp up on my scala.
>
> When
Slowly trying to understand it, have to wramp up on my scala.
When the flush/sink occurrs, does it pull items of the collection 1 by 1 or
does it do this in bulk somehow while locking the collection?
On Mon, May 7, 2012 at 3:14 PM, Neha Narkhede wrote:
> Ahmed,
>
> The related code is in kafka.l
Ahmed,
The related code is in kafka.log.*. The message to file persistence is
inside FileMessageSet.scala.
Thanks,
Neha
On Mon, May 7, 2012 at 12:12 PM, S Ahmed wrote:
> I can barely read scala, but I'm curious where the applications performs
> the operation of taking the in-memory log and per
