[
https://issues.apache.org/jira/browse/LUCENE-7113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15202684#comment-15202684
]
ASF subversion and git services commented on LUCENE-7113:
---------------------------------------------------------
Commit e458385b5e94d146853fabe5a94f2c9b0af31800 in lucene-solr's branch
refs/heads/branch_6_0 from Mike McCandless
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e458385 ]
LUCENE-7113: add best-effort checksum verification to temp files written by
OfflineSorter and BKDWriter
Conflicts:
lucene/core/src/java/org/apache/lucene/util/bkd/BKDWriter.java
lucene/core/src/java/org/apache/lucene/util/bkd/HeapPointWriter.java
lucene/core/src/java/org/apache/lucene/util/bkd/OfflinePointReader.java
lucene/core/src/java/org/apache/lucene/util/bkd/OfflinePointWriter.java
> OfflineSorter and BKD should verify checksums in their temp files
> -----------------------------------------------------------------
>
> Key: LUCENE-7113
> URL: https://issues.apache.org/jira/browse/LUCENE-7113
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Michael McCandless
> Assignee: Michael McCandless
> Fix For: master, 6.0
>
> Attachments: LUCENE-7113.patch, LUCENE-7113.patch
>
>
> I am trying to index all 3.2 B points from the latest OpenStreetMaps export.
> My SSDs were not up to this, so I added a spinning magnets disk to beast2.
> But then I was hitting scary bug-like exceptions
> ({{ArrayIndexOutOfBoundsException}}) when indexing the first 2B points, and I
> finally checked dmesg and saw that my hard drive is dying.
> I think it's important that our temp file usages also validate checksums
> (like we do for all our index files, either at reader open or at merge or
> {{CheckIndex}}), so we can hopefully easily differentiate a bit-flipping IO
> system from a possible Lucene bug, in the future.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
