[
https://issues.apache.org/jira/browse/SOLR-8575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15144833#comment-15144833
]
Mark Miller commented on SOLR-8575:
-----------------------------------
bq. Actually, it looks like this patch fixed more than just performance
Right, it's not just a performance fix or a 'status' numbers fix. The issue was
the size hdfs was returning to us was wrong and we were going off the wrong
size info. That made it so that when we had to open a new reader, we then did
so every update. That seems to have hidden some of the issues here. There was
no way to know if there was a bug users where hitting here beyond super, super
slow replay while buffering performance though. For example, you were not
seeing inconsistency fails with that code. It was obviously a bug no matter
what flushing happened though, because we were basing our logic on file sizes
that did not relate to reality (and did not generally change at all between
calls).
> Fix HDFSLogReader replay status numbers, a performance bug where we can
> reopen FSDataInputStream much too often, and an hdfs tlog data integrity bug.
> -----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-8575
> URL: https://issues.apache.org/jira/browse/SOLR-8575
> Project: Solr
> Issue Type: Bug
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: master
>
> Attachments: SOLR-8575.patch, SOLR-8575.patch
>
>
> [~pdvo...@cloudera.com] noticed some funny transaction log replay status
> logging a while back:
> active=true starting pos=444978 current pos=2855956 current size=16262 %
> read=17562
> active=true starting pos=444978 current pos=5748869 current size=16262 %
> read=35352
> 17562% read? Current size does not change as expected in this case?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
