[jira] [Commented] (LUCENE-5636) SegmentCommitInfo continues to list unneeded gen'd files
[ https://issues.apache.org/jira/browse/LUCENE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14004644#comment-14004644 ] ASF subversion and git services commented on LUCENE-5636: - Commit 1596570 from [~shaie] in branch 'dev/trunk' [ https://svn.apache.org/r1596570 ] LUCENE-5618, LUCENE-5636: write each DocValues update in a separate file; stop referencing old fieldInfos files SegmentCommitInfo continues to list unneeded gen'd files Key: LUCENE-5636 URL: https://issues.apache.org/jira/browse/LUCENE-5636 Project: Lucene - Core Issue Type: Bug Components: core/index Reporter: Shai Erera Assignee: Shai Erera Attachments: LUCENE-5636.patch I thought I handled it in LUCENE-5246, but turns out I didn't handle it fully. I'll upload a patch which improves the test to expose the bug. I know where it is, but I'm not sure how to fix it without breaking index back-compat. Can we do that on experimental features? The problem is that if you update different fields in different gens, the FieldInfos files of older gens remain referenced (still!!). I open a new issue since LUCENE-5246 is already resolved and released, so don't want to mess up our JIRA... The severity of the bug is that unneeded files are still referenced in the index. Everything still works correctly, it's just that .fnm files are still there. But as I wrote, I'm still not sure how to solve it without requiring apps that use dv updates to reindex. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5636) SegmentCommitInfo continues to list unneeded gen'd files
[ https://issues.apache.org/jira/browse/LUCENE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14004715#comment-14004715 ] ASF subversion and git services commented on LUCENE-5636: - Commit 1596582 from [~shaie] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1596582 ] LUCENE-5618, LUCENE-5636: write each DocValues update in a separate file; stop referencing old fieldInfos files SegmentCommitInfo continues to list unneeded gen'd files Key: LUCENE-5636 URL: https://issues.apache.org/jira/browse/LUCENE-5636 Project: Lucene - Core Issue Type: Bug Components: core/index Reporter: Shai Erera Assignee: Shai Erera Attachments: LUCENE-5636.patch I thought I handled it in LUCENE-5246, but turns out I didn't handle it fully. I'll upload a patch which improves the test to expose the bug. I know where it is, but I'm not sure how to fix it without breaking index back-compat. Can we do that on experimental features? The problem is that if you update different fields in different gens, the FieldInfos files of older gens remain referenced (still!!). I open a new issue since LUCENE-5246 is already resolved and released, so don't want to mess up our JIRA... The severity of the bug is that unneeded files are still referenced in the index. Everything still works correctly, it's just that .fnm files are still there. But as I wrote, I'm still not sure how to solve it without requiring apps that use dv updates to reindex. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5636) SegmentCommitInfo continues to list unneeded gen'd files
[ https://issues.apache.org/jira/browse/LUCENE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13988005#comment-13988005 ] Shai Erera commented on LUCENE-5636: I chatted with Robert about this. The current situation is that the old .fnm files continue to be referenced even when not needed, however when the segment is merged, they go away (as all gen'd files). Given that there's no way to solve it without breaking back-compat, unless we introduce hacks such as checking for a .fnm suffix, we discussed how to solve this going forward. By going forward I mean to not change existing segments, but if they contain future updates, write the new information in a better way. Perhaps old .fnm files will still be referenced by those segments, until they're merged away, but new segments will fix that bug. I think that this might be doable together with LUCENE-5618, by writing per-field gen'd DV file, so I'll try to solve it there and if it works I'll resolve that issue as appropriate. SegmentCommitInfo continues to list unneeded gen'd files Key: LUCENE-5636 URL: https://issues.apache.org/jira/browse/LUCENE-5636 Project: Lucene - Core Issue Type: Bug Components: core/index Reporter: Shai Erera Assignee: Shai Erera Attachments: LUCENE-5636.patch I thought I handled it in LUCENE-5246, but turns out I didn't handle it fully. I'll upload a patch which improves the test to expose the bug. I know where it is, but I'm not sure how to fix it without breaking index back-compat. Can we do that on experimental features? The problem is that if you update different fields in different gens, the FieldInfos files of older gens remain referenced (still!!). I open a new issue since LUCENE-5246 is already resolved and released, so don't want to mess up our JIRA... The severity of the bug is that unneeded files are still referenced in the index. Everything still works correctly, it's just that .fnm files are still there. But as I wrote, I'm still not sure how to solve it without requiring apps that use dv updates to reindex. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5636) SegmentCommitInfo continues to list unneeded gen'd files
[ https://issues.apache.org/jira/browse/LUCENE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986859#comment-13986859 ] Shai Erera commented on LUCENE-5636: Hmm, actually I think I can fix it w/o breaking index back-compat, by manually removing the old .fnm files in SegCommitInfo (I'll also improve how this is done in general). Assuming very little apps, if at all, write their own FieldInfosFormat, I think that's acceptable? I'll work on a patch later. SegmentCommitInfo continues to list unneeded gen'd files Key: LUCENE-5636 URL: https://issues.apache.org/jira/browse/LUCENE-5636 Project: Lucene - Core Issue Type: Bug Components: core/index Reporter: Shai Erera Assignee: Shai Erera Priority: Critical Attachments: LUCENE-5636.patch I thought I handled it in LUCENE-5246, but turns out I didn't handle it fully. I'll upload a patch which improves the test to expose the bug. I know where it is, but I'm not sure how to fix it without breaking index back-compat. Can we do that on experimental features? The problem is that if you update different fields in different gens, the FieldInfos files of older gens remain referenced (still!!). I open a new issue since LUCENE-5246 is already resolved and released, so don't want to mess up our JIRA... The severity of the bug is that unneeded files are still referenced in the index. Everything still works correctly, it's just that .fnm files are still there. But as I wrote, I'm still not sure how to solve it without requiring apps that use dv updates to reindex. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
