[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2021-01-16 Thread GitBox
zacharymorn commented on pull request #2052: URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-761700207 > If it returns 0, there may be more to read, just not yet available. This is important for asynchronous channels! Oh I see! I guess since it's reading into a

[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2021-01-16 Thread GitBox
zacharymorn commented on pull request #2052: URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-761623319 > The issue with seek to filesize is well-known to me, that was also my first idea I had yesterday in the evening. But I was ready to go to bed and of course I had a

[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2021-01-16 Thread GitBox
zacharymorn commented on pull request #2052: URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-761525653 > > I will try to find the issue tomorrow...! (not sure if I will have time, so if you can reproduce on Linux feel free to help out) > > Hmmm that's strange. On

[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2021-01-15 Thread GitBox
zacharymorn commented on pull request #2052: URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-761298120 > I will try to find the issue tomorrow...! (not sure if I will have time, so if you can reproduce on Linux feel free to help out) Hmmm that's strange. On my

[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2021-01-14 Thread GitBox
zacharymorn commented on pull request #2052: URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-760697048 hi @mikemccand, I just took a look, and pushed some more commits to resolve latest merge conflicts and adopt new formatting.

[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2021-01-03 Thread GitBox
zacharymorn commented on pull request #2052: URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-753800497 > Hi @dweiss, for the JDK compiler issue, I ended up submitting a bug report to open jdk (internal review id 9068426) as I'm hoping to be able to refer to this issue in

[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2021-01-02 Thread GitBox
zacharymorn commented on pull request #2052: URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-753560734 Hi @dweiss, for the JDK compiler issue, I ended up submitting a bug report to open jdk (internal review id 9068426) as I'm hoping to be able to refer to this issue in

[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2021-01-02 Thread GitBox
zacharymorn commented on pull request #2052: URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-753557228 Hi Uwe, I've gone ahead and pushed a commit for the following: 1. Add checksum computation 2. Add a new test to exercise the DirectIODirectory with actual index

[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2021-01-02 Thread GitBox
zacharymorn commented on pull request #2052: URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-753556939 Hi Uwe, I've gone ahead and pushed a commit for the following: 1. Add checksum This is an automated

[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2020-12-28 Thread GitBox
zacharymorn commented on pull request #2052: URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-751925231 Hi @uschindler, I will have some time in the next few days. Do you mind me taking a look at what's left here and potentially pushing some changes? Don't want to get in

[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2020-12-20 Thread GitBox
zacharymorn commented on pull request #2052: URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-748804203 > This may also happen on other operating systems. It depends on if the buffer is already aligned when you call alignSlice(). If it is not aligned (seems to happen 100%

[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2020-12-19 Thread GitBox
zacharymorn commented on pull request #2052: URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-748563011 Oh wow thanks for the updates there, and sorry to hear the current changes are not yet ready! Yes the original implementation hasn't been used for a while now (as

[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2020-12-19 Thread GitBox
zacharymorn commented on pull request #2052: URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-748509146 > > Hmm I don't have a windows box to test this. The blockSize is dynamically computed though. If this works on linux/darwin but fails on windows, does it signal

[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2020-12-19 Thread GitBox
zacharymorn commented on pull request #2052: URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-748506650 > Hi, I implemented the approach and also cleaned up the lookup code to not require unchecked casts. I also made the code safer for exceptions, you only catched

[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2020-12-19 Thread GitBox
zacharymorn commented on pull request #2052: URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-748443350 > You can do it statically, look at how the unmapper in Mmapdir works: > Look up the reflective in static intitalizer and save the result at very end in static final.

[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2020-12-19 Thread GitBox
zacharymorn commented on pull request #2052: URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-748441602 > I'll try to help out later today, Zach. Sounds good. Thanks Dawid! This is an automated

[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2020-12-18 Thread GitBox
zacharymorn commented on pull request #2052: URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-748427066 Wow thanks everyone for the great suggestions and ideas! Really appreciate them! Didn't quite anticipate we would go so deep into looking at java compiler when picking

[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2020-12-17 Thread GitBox
zacharymorn commented on pull request #2052: URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-747890887 > > Alas, `gradlew precommit` from the command-line on Linux box is still angry for me: > > ``` > > > Task :lucene:misc:compileJava > >

[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2020-12-14 Thread GitBox
zacharymorn commented on pull request #2052: URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-745039717 > Alas, `gradlew precommit` from the command-line on Linux box is still angry for me: > > ``` > > Task :lucene:misc:compileJava >

[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2020-12-07 Thread GitBox
zacharymorn commented on pull request #2052: URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-740306114 > Hmm, `gradlew precommit` is angry: > > ``` > > Task :lucene:misc:compileJava >

[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2020-12-02 Thread GitBox
zacharymorn commented on pull request #2052: URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-737614580 Just want to have a quick follow up on this PR. Are there any more changes expected from my end? This

[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2020-11-20 Thread GitBox
zacharymorn commented on pull request #2052: URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-731491257 > > > Second, it is extremely experimental and not clear when it provides benefits / what risks there are / etc. We need to learn much more about it, in diverse usage,

[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2020-11-20 Thread GitBox
zacharymorn commented on pull request #2052: URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-731491171 > > I am not sure if we should preserve the native PosixUtil. It's used nowhere and was only added by @mikemccand back at that time to do more testing. > > Nowadays,

[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2020-11-18 Thread GitBox
zacharymorn commented on pull request #2052: URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-729520080 > I stated a bit earlier, removing WindowsDirectory should be a separate PR. It's not the same thing (it's not about DirectIO), it works around aproblem with positional

[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2020-11-17 Thread GitBox
zacharymorn commented on pull request #2052: URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-729393685 I've been following the comment thread on `WindowsDirectory`, but not sure if there's a consensus to remove it in this PR? For the time being, I've pushed a standalone

[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2020-11-17 Thread GitBox
zacharymorn commented on pull request #2052: URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-729389857 > > I think we just need to resolve the "replace now or not" question? > > I'd like to improve the testcase and use a real index on top of a combination with

[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2020-11-17 Thread GitBox
zacharymorn commented on pull request #2052: URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-729387261 > > Second, it is extremely experimental and not clear when it provides benefits / what risks there are / etc. We need to learn much more about it, in diverse usage, to

[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory

2020-11-06 Thread GitBox
zacharymorn commented on pull request #2052: URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-723400516 > Thank you @zacharymorn! It's so nice we can move this to pure java now -- this makes it much easier and lower risk for Lucene users to test. > > I suspect for