The count for FSDirectory on OSX is around 700-800. The problem is that the filesystem timestamps on OSX has abysmal resolution (nothing finer than 1 whole second!)
Wow! Write once, debug everywhere...
Is this an OSX "feature" or is this a feature of a particular disk / filesystem / driver / whatnot that is in use in the current run?
If this is unavoidable on OSX, perhaps other platforms are also affected? Does anyone know the timestamp resolution on Wintel, Lintel, Solaris, etc?
Finally, what if we allocate a "long" in the beginning of the files that need to be touched (like the segments and other files) and use it to store the full System.currentTimeMillis timestamp? Does this method return a more granular number on OSX? This would be a big change, but it would work the same on all platforms and will not have a slowdown.
So the touch method will spin its wheels for anything up to a second before we've registered an actual 'touch'.
This isn't so bad as files aren't touch that often, but it's a lot more problematic in the case of SegmentInfos.write(), where a new file is created, then copied over the old one. The testDeleteReaderWriteConflict testcase fails due to this. The old file and the new have the same timestamp, and so the staleness check fails. How should this be fixed?
Also, won't the fix for FSDirectory mean there's a window where other threads could end up with bad info, if we're taking up to a second to touch a file?
I don't know off hand, but I believe there is file locking in place to prevent other threads from getting into this code until the update operation is completed, and it will not be completed until the touch finishes. This still sucks with a 1-second blocking...
Dmitry.
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
