[
https://issues.apache.org/jira/browse/KAFKA-1646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316747#comment-14316747
]
Jay Kreps commented on KAFKA-1646:
----------------------------------
Hey [~waldenchen] yeah I think what you are saying is that the offset in the
recovery point checkpoint should always be the last sync'd offset irrespective
of segment file boundaries and we should optimize recovery to just recovery
from that offset rather than always recovering the full last segment. That
would work, and actually makes more sense than the current approach, but is a
fairly involved and correctness critical change. One of the challenges is the
offset index needs to be reconstructed from that point on as well, but there is
a bit of a chicken and egg problem, because how do you search into the log at
all if the index itself is corrupt?
Another approach could be to truncate off the preallocated file extent on clean
shutdown. This is actually effectively what we do for the offset indexes
anyway. This would also avoid windows specific code since we could do this in
all cases.
> Improve consumer read performance for Windows
> ---------------------------------------------
>
> Key: KAFKA-1646
> URL: https://issues.apache.org/jira/browse/KAFKA-1646
> Project: Kafka
> Issue Type: Improvement
> Components: log
> Affects Versions: 0.8.1.1
> Environment: Windows
> Reporter: xueqiang wang
> Assignee: xueqiang wang
> Labels: newbie, patch
> Attachments: Improve consumer read performance for Windows.patch,
> KAFKA-1646-truncate-off-trailing-zeros-on-broker-restart-if-bro.patch,
> KAFKA-1646_20141216_163008.patch
>
>
> This patch is for Window platform only. In Windows platform, if there are
> more than one replicas writing to disk, the segment log files will not be
> consistent in disk and then consumer reading performance will be dropped down
> greatly. This fix allocates more disk spaces when rolling a new segment, and
> then it will improve the consumer reading performance in NTFS file system.
> This patch doesn't affect file allocation of other filesystems, for it only
> adds statements like 'if(Os.iswindow)' or adds methods used on Windows.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
