[jira] [Commented] (LUCENE-8617) FSDirectory tries to create MMapDirectory on non default file system
[
https://issues.apache.org/jira/browse/LUCENE-8617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16727548#comment-16727548
]
Philippe Marschall commented on LUCENE-8617:
Is it ok if I just close this as chasing this would negatively affect tests?
> FSDirectory tries to create MMapDirectory on non default file system
>
>
> Key: LUCENE-8617
> URL: https://issues.apache.org/jira/browse/LUCENE-8617
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/store
>Affects Versions: 5.0
>Reporter: Philippe Marschall
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{org.apache.lucene.store.FSDirectory.open(Path, LockFactory)}} and
> {{org.apache.lucene.store.FSDirectory.open(Path)}} always assume the path is
> created on the default file system. If the path is not on the default file
> system {{MMapDirectory}} is the wrong choice as only the default file system
> can provide {{mmap()}}.
> In case of a non default file system {{SimpleFSDirectory}} is a good choice.
> See this bug for an example
> https://github.com/marschall/memoryfilesystem/issues/113
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8617) FSDirectory tries to create MMapDirectory on non default file system
[ https://issues.apache.org/jira/browse/LUCENE-8617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725986#comment-16725986 ] Philippe Marschall commented on LUCENE-8617: > We provide mmap with non-default filesystems for testing purposes and we > override the mmap() method: Yes you implement and override the the mmap() method but in the end you delegate to a `FileChannel` instance created on the default file system. All the file system you provide are actually "just" decorations around the default file system. You have to because you can not create a subclass or instance of `MappedByteBuffer` because all the constructors are package-private. The only ways to get your hands on a `MappedByteBuffer` is by either calling `FileChannel#map` or calling `ByteBuffer#allocateDirect()`. Implementing `FileChannel#map` by calling `FileChannel#map` means it will only work on the default file system as only it can create instance of `MappedByteBuffer`. Implementing `FileChannel#map` by calling `ByteBuffer#allocateDirect()` relies on implementation quirks and would be quite tricky. > That change would have a real impact on test coverage since we use > non-default filesystems to check for things like file descriptor leaks That's a valid point, I was not aware of this. My use case case is very niche, an in-memory file system for testing, and a work around exists. I don't see an easy way to detect wether a file system supports mmap() other than by calling it. > FSDirectory tries to create MMapDirectory on non default file system > > > Key: LUCENE-8617 > URL: https://issues.apache.org/jira/browse/LUCENE-8617 > Project: Lucene - Core > Issue Type: Bug > Components: core/store >Affects Versions: 5.0 >Reporter: Philippe Marschall >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > `org.apache.lucene.store.FSDirectory.open(Path, LockFactory)` and > `org.apache.lucene.store.FSDirectory.open(Path)` always assume the path is > created on the default file system. If the path is not on the default file > system `MMapDirectory` is the wrong choice as only the default file system > can provide `mmap()`. > In case of a non default file system `SimpleFSDirectory` is a good choice. > See this bug for an example > https://github.com/marschall/memoryfilesystem/issues/113 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8617) FSDirectory tries to create MMapDirectory on non default file system
[
https://issues.apache.org/jira/browse/LUCENE-8617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725940#comment-16725940
]
Robert Muir commented on LUCENE-8617:
-
I'm not sure I agree with this statement: {quote}as only the default file
system can provide `mmap()`.{quote}
We provide mmap with non-default filesystems for testing purposes and we
override the mmap() method:
https://github.com/apache/lucene-solr/blob/master/lucene/test-framework/src/java/org/apache/lucene/mockfile/FilterFileChannel.java#L119-L122
So I don't think it makes sense to disable it based on a Filesystem.getDefault
check like the PR does.
That change would have a real impact on test coverage since we use non-default
filesystems to check for things like file descriptor leaks:
[https://lucene.apache.org/core/7_6_0/test-framework/org/apache/lucene/mockfile/package-summary.html#package.description]
> FSDirectory tries to create MMapDirectory on non default file system
>
>
> Key: LUCENE-8617
> URL: https://issues.apache.org/jira/browse/LUCENE-8617
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/store
>Affects Versions: 5.0
>Reporter: Philippe Marschall
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> `org.apache.lucene.store.FSDirectory.open(Path, LockFactory)` and
> `org.apache.lucene.store.FSDirectory.open(Path)` always assume the path is
> created on the default file system. If the path is not on the default file
> system `MMapDirectory` is the wrong choice as only the default file system
> can provide `mmap()`.
> In case of a non default file system `SimpleFSDirectory` is a good choice.
> See this bug for an example
> https://github.com/marschall/memoryfilesystem/issues/113
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
