[jira] [Commented] (LUCENE-6902) Fail fsync immediately
[ https://issues.apache.org/jira/browse/LUCENE-6902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15020929#comment-15020929 ] ASF subversion and git services commented on LUCENE-6902: - Commit 1715618 from [~thetaphi] in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1715618 ] Merged revision(s) 1715617 from lucene/dev/trunk: LUCENE-6902: Don't retry to fsync files / directories; fail immediately > Fail fsync immediately > -- > > Key: LUCENE-6902 > URL: https://issues.apache.org/jira/browse/LUCENE-6902 > Project: Lucene - Core > Issue Type: Improvement > Components: core/store >Affects Versions: 5.3.1 >Reporter: Daniel Mitterdorfer >Assignee: Uwe Schindler >Priority: Minor > Fix For: Trunk, 5.4 > > Attachments: ioutils-fsync-fail-fast.patch, > ioutils-fsync-fail-fast.patch > > > While analysing a build issue in Elasticsearch I stumpled upon > org.apache.lucene.util.IOUtils.fsync. It has a retry loop in fsync whenever > an IOException occurs. However, there are lots of instances where a retry is > not useful, e.g. when a channel has been closed, a ClosedChannelException is > thrown and IOUtils#fsync still tries to fsync multiple times on the closed > channel. > After bringing the issue to Robert's attention, he even opted for removing > the retry logic entirely for fsyncing. > Please find attached a patch that removes the retry logic. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6902) Fail fsync immediately
[ https://issues.apache.org/jira/browse/LUCENE-6902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15020928#comment-15020928 ] ASF subversion and git services commented on LUCENE-6902: - Commit 1715617 from [~thetaphi] in branch 'dev/trunk' [ https://svn.apache.org/r1715617 ] LUCENE-6902: Don't retry to fsync files / directories; fail immediately > Fail fsync immediately > -- > > Key: LUCENE-6902 > URL: https://issues.apache.org/jira/browse/LUCENE-6902 > Project: Lucene - Core > Issue Type: Improvement > Components: core/store >Affects Versions: 5.3.1 >Reporter: Daniel Mitterdorfer >Assignee: Uwe Schindler >Priority: Minor > Fix For: 5.4 > > Attachments: ioutils-fsync-fail-fast.patch, > ioutils-fsync-fail-fast.patch > > > While analysing a build issue in Elasticsearch I stumpled upon > org.apache.lucene.util.IOUtils.fsync. It has a retry loop in fsync whenever > an IOException occurs. However, there are lots of instances where a retry is > not useful, e.g. when a channel has been closed, a ClosedChannelException is > thrown and IOUtils#fsync still tries to fsync multiple times on the closed > channel. > After bringing the issue to Robert's attention, he even opted for removing > the retry logic entirely for fsyncing. > Please find attached a patch that removes the retry logic. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6902) Fail fsync immediately
[ https://issues.apache.org/jira/browse/LUCENE-6902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15015494#comment-15015494 ] Daniel Mitterdorfer commented on LUCENE-6902: - Sorry, wasn't aware of that. Thanks for pointing it out. > Fail fsync immediately > -- > > Key: LUCENE-6902 > URL: https://issues.apache.org/jira/browse/LUCENE-6902 > Project: Lucene - Core > Issue Type: Improvement > Components: core/store >Affects Versions: 5.3.1 >Reporter: Daniel Mitterdorfer >Assignee: Uwe Schindler >Priority: Minor > Fix For: 5.4 > > Attachments: ioutils-fsync-fail-fast.patch > > > While analysing a build issue in Elasticsearch I stumpled upon > org.apache.lucene.util.IOUtils.fsync. It has a retry loop in fsync whenever > an IOException occurs. However, there are lots of instances where a retry is > not useful, e.g. when a channel has been closed, a ClosedChannelException is > thrown and IOUtils#fsync still tries to fsync multiple times on the closed > channel. > After bringing the issue to Robert's attention, he even opted for removing > the retry logic entirely for fsyncing. > Please find attached a patch that removes the retry logic. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6902) Fail fsync immediately
[ https://issues.apache.org/jira/browse/LUCENE-6902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15015486#comment-15015486 ] Uwe Schindler commented on LUCENE-6902: --- Looks fine! I will check and commit this later. You can leave the old versions of the patch online, don't delete them! If you upload a new version of the same filename, the old one is grayed out, but is still accessible for doucmentation purposes. > Fail fsync immediately > -- > > Key: LUCENE-6902 > URL: https://issues.apache.org/jira/browse/LUCENE-6902 > Project: Lucene - Core > Issue Type: Improvement > Components: core/store >Affects Versions: 5.3.1 >Reporter: Daniel Mitterdorfer >Assignee: Uwe Schindler >Priority: Minor > Fix For: 5.4 > > Attachments: ioutils-fsync-fail-fast.patch > > > While analysing a build issue in Elasticsearch I stumpled upon > org.apache.lucene.util.IOUtils.fsync. It has a retry loop in fsync whenever > an IOException occurs. However, there are lots of instances where a retry is > not useful, e.g. when a channel has been closed, a ClosedChannelException is > thrown and IOUtils#fsync still tries to fsync multiple times on the closed > channel. > After bringing the issue to Robert's attention, he even opted for removing > the retry logic entirely for fsyncing. > Please find attached a patch that removes the retry logic. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6902) Fail fsync immediately
[ https://issues.apache.org/jira/browse/LUCENE-6902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15018471#comment-15018471 ] Uwe Schindler commented on LUCENE-6902: --- Hi the patch fails on windows, because it does not swallow the IOException when fsyncing a directory. I will correct it and upload new patch. > Fail fsync immediately > -- > > Key: LUCENE-6902 > URL: https://issues.apache.org/jira/browse/LUCENE-6902 > Project: Lucene - Core > Issue Type: Improvement > Components: core/store >Affects Versions: 5.3.1 >Reporter: Daniel Mitterdorfer >Assignee: Uwe Schindler >Priority: Minor > Fix For: 5.4 > > Attachments: ioutils-fsync-fail-fast.patch > > > While analysing a build issue in Elasticsearch I stumpled upon > org.apache.lucene.util.IOUtils.fsync. It has a retry loop in fsync whenever > an IOException occurs. However, there are lots of instances where a retry is > not useful, e.g. when a channel has been closed, a ClosedChannelException is > thrown and IOUtils#fsync still tries to fsync multiple times on the closed > channel. > After bringing the issue to Robert's attention, he even opted for removing > the retry logic entirely for fsyncing. > Please find attached a patch that removes the retry logic. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6902) Fail fsync immediately
[
https://issues.apache.org/jira/browse/LUCENE-6902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014188#comment-15014188
]
Michael McCandless commented on LUCENE-6902:
+1, patch looks good, except {{testFsyncFile}} is actually just fsync'ing the
dir?
> Fail fsync immediately
> --
>
> Key: LUCENE-6902
> URL: https://issues.apache.org/jira/browse/LUCENE-6902
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Daniel Mitterdorfer
>Priority: Minor
> Attachments: ioutils-fsync-fail-fast.patch
>
>
> While analysing a build issue in Elasticsearch I stumpled upon
> org.apache.lucene.util.IOUtils.fsync. It has a retry loop in fsync whenever
> an IOException occurs. However, there are lots of instances where a retry is
> not useful, e.g. when a channel has been closed, a ClosedChannelException is
> thrown and IOUtils#fsync still tries to fsync multiple times on the closed
> channel.
> After bringing the issue to Robert's attention, he even opted for removing
> the retry logic entirely for fsyncing.
> Please find attached a patch that removes the retry logic.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6902) Fail fsync immediately
[ https://issues.apache.org/jira/browse/LUCENE-6902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014501#comment-15014501 ] Uwe Schindler commented on LUCENE-6902: --- btw. I was About to remove the loop when touching the code last time (adding directory fsync). But as this was another issue I left it alive. > Fail fsync immediately > -- > > Key: LUCENE-6902 > URL: https://issues.apache.org/jira/browse/LUCENE-6902 > Project: Lucene - Core > Issue Type: Improvement > Components: core/store >Affects Versions: 5.3.1 >Reporter: Daniel Mitterdorfer >Assignee: Uwe Schindler >Priority: Minor > Fix For: 5.4 > > Attachments: ioutils-fsync-fail-fast.patch > > > While analysing a build issue in Elasticsearch I stumpled upon > org.apache.lucene.util.IOUtils.fsync. It has a retry loop in fsync whenever > an IOException occurs. However, there are lots of instances where a retry is > not useful, e.g. when a channel has been closed, a ClosedChannelException is > thrown and IOUtils#fsync still tries to fsync multiple times on the closed > channel. > After bringing the issue to Robert's attention, he even opted for removing > the retry logic entirely for fsyncing. > Please find attached a patch that removes the retry logic. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6902) Fail fsync immediately
[ https://issues.apache.org/jira/browse/LUCENE-6902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014485#comment-15014485 ] Uwe Schindler commented on LUCENE-6902: --- Hi, patch looks good. I will take this issue and commit it after doing my usual checks (modifications on fsync are Mike is right, the testFsyncFile is wrong, it syncs the directory. The assume on not windows is not needed for the directory test, please remove it! Because fsyncing a directory may always fail on any operating system and this is correctly checked in the code (supresses exception). So just remove the assume on windows. Removing it actually checks that our assert statement in the code works correct. > Fail fsync immediately > -- > > Key: LUCENE-6902 > URL: https://issues.apache.org/jira/browse/LUCENE-6902 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Daniel Mitterdorfer >Priority: Minor > Attachments: ioutils-fsync-fail-fast.patch > > > While analysing a build issue in Elasticsearch I stumpled upon > org.apache.lucene.util.IOUtils.fsync. It has a retry loop in fsync whenever > an IOException occurs. However, there are lots of instances where a retry is > not useful, e.g. when a channel has been closed, a ClosedChannelException is > thrown and IOUtils#fsync still tries to fsync multiple times on the closed > channel. > After bringing the issue to Robert's attention, he even opted for removing > the retry logic entirely for fsyncing. > Please find attached a patch that removes the retry logic. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6902) Fail fsync immediately
[ https://issues.apache.org/jira/browse/LUCENE-6902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15015362#comment-15015362 ] Daniel Mitterdorfer commented on LUCENE-6902: - Btw, I think LUCENE-6169 could also be resolved now. The according JDK bug is fixed by now according to their issue tracker. > Fail fsync immediately > -- > > Key: LUCENE-6902 > URL: https://issues.apache.org/jira/browse/LUCENE-6902 > Project: Lucene - Core > Issue Type: Improvement > Components: core/store >Affects Versions: 5.3.1 >Reporter: Daniel Mitterdorfer >Assignee: Uwe Schindler >Priority: Minor > Fix For: 5.4 > > Attachments: ioutils-fsync-fail-fast.patch > > > While analysing a build issue in Elasticsearch I stumpled upon > org.apache.lucene.util.IOUtils.fsync. It has a retry loop in fsync whenever > an IOException occurs. However, there are lots of instances where a retry is > not useful, e.g. when a channel has been closed, a ClosedChannelException is > thrown and IOUtils#fsync still tries to fsync multiple times on the closed > channel. > After bringing the issue to Robert's attention, he even opted for removing > the retry logic entirely for fsyncing. > Please find attached a patch that removes the retry logic. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6902) Fail fsync immediately
[ https://issues.apache.org/jira/browse/LUCENE-6902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15015381#comment-15015381 ] Uwe Schindler commented on LUCENE-6902: --- Hi, can you also remove the assume on Windows from the fsync on directory test? The test must also work on Windows (it would just be a no-op). LUCENE-6169 is still open because OpenJDK team is still working on an "official" API. > Fail fsync immediately > -- > > Key: LUCENE-6902 > URL: https://issues.apache.org/jira/browse/LUCENE-6902 > Project: Lucene - Core > Issue Type: Improvement > Components: core/store >Affects Versions: 5.3.1 >Reporter: Daniel Mitterdorfer >Assignee: Uwe Schindler >Priority: Minor > Fix For: 5.4 > > Attachments: ioutils-fsync-fail-fast.patch > > > While analysing a build issue in Elasticsearch I stumpled upon > org.apache.lucene.util.IOUtils.fsync. It has a retry loop in fsync whenever > an IOException occurs. However, there are lots of instances where a retry is > not useful, e.g. when a channel has been closed, a ClosedChannelException is > thrown and IOUtils#fsync still tries to fsync multiple times on the closed > channel. > After bringing the issue to Robert's attention, he even opted for removing > the retry logic entirely for fsyncing. > Please find attached a patch that removes the retry logic. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6902) Fail fsync immediately
[ https://issues.apache.org/jira/browse/LUCENE-6902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15013833#comment-15013833 ] Robert Muir commented on LUCENE-6902: - +1 to remove this retry. Its especially bad since its just "any IOException". I think it is a relic from when this method used RandomAccessFile (and would sometimes create files). If it turns out some hack is really needed (e.g. on windows), then we should at least contain it better to a more specific case or think about how we can avoid it. > Fail fsync immediately > -- > > Key: LUCENE-6902 > URL: https://issues.apache.org/jira/browse/LUCENE-6902 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Daniel Mitterdorfer >Priority: Minor > Attachments: ioutils-fsync-fail-fast.patch > > > While analysing a build issue in Elasticsearch I stumpled upon > org.apache.lucene.util.IOUtils.fsync. It has a retry loop in fsync whenever > an IOException occurs. However, there are lots of instances where a retry is > not useful, e.g. when a channel has been closed, a ClosedChannelException is > thrown and IOUtils#fsync still tries to fsync multiple times on the closed > channel. > After bringing the issue to Robert's attention, he even opted for removing > the retry logic entirely for fsyncing. > Please find attached a patch that removes the retry logic. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
