[
https://issues.apache.org/jira/browse/KAFKA-1646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310912#comment-14310912
]
Jay Kreps commented on KAFKA-1646:
----------------------------------
Hey guys if this forces full recovery the impact on startup time will be
considerable if you have a large number of partitions.
Say you have 2000 partitions per machine and a 1GB log segment file size. On
average these files will have about 500MB per partition when a restart occurs.
The result is running recovery on 2000 * 500MB = 1TB of data. This will take
about 5.5 hours at 50MB/sec.
[~qixia] not sure how the above reasoning compares to your test?
I think this would be a blocker issue, no?
> Improve consumer read performance for Windows
> ---------------------------------------------
>
> Key: KAFKA-1646
> URL: https://issues.apache.org/jira/browse/KAFKA-1646
> Project: Kafka
> Issue Type: Improvement
> Components: log
> Affects Versions: 0.8.1.1
> Environment: Windows
> Reporter: xueqiang wang
> Assignee: xueqiang wang
> Labels: newbie, patch
> Attachments: Improve consumer read performance for Windows.patch,
> KAFKA-1646-truncate-off-trailing-zeros-on-broker-restart-if-bro.patch,
> KAFKA-1646_20141216_163008.patch
>
>
> This patch is for Window platform only. In Windows platform, if there are
> more than one replicas writing to disk, the segment log files will not be
> consistent in disk and then consumer reading performance will be dropped down
> greatly. This fix allocates more disk spaces when rolling a new segment, and
> then it will improve the consumer reading performance in NTFS file system.
> This patch doesn't affect file allocation of other filesystems, for it only
> adds statements like 'if(Os.iswindow)' or adds methods used on Windows.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
