Dear derby-dev and mike matrigali:
Do you have any time to write a test case of the I/O calls separate from
derby? This has the feel of a JVM bug, but maybe the interfaces are not
being used correctly. This is a relatively new feature in jvm's (and
new code in derby) so could be a bug in either area.
Sure. Here's my guesswork/bugwork to get Derby running comfortably on OSX given the latest discussion(s).
mike matrigali wrote (Thu, 02 Sep 2004 09:53:45):
Joe I don't have access to an OSX 10.3.5. This error is strange, from my
reading of the java interfaces and the code the first call is creating
the file, and the second call should just be opening the existing file -
and should not be getting a file exists error.
Yes. I read (presumably) the same docs at (FYI):
http://java.sun.com/j2se/1.4.2/docs/api/java/io/RandomAccessFile.html
To quote:
"r"
Open for reading only. Invoking any of the write methods of the resulting object will cause an IOException to be thrown.
"rw"
Open for reading and writing. If the file does not already exist then an attempt will be made to create it.
"rws"
Open for reading and writing, as with "rw", and also require that every update to the file's content or metadata be written synchronously to the underlying storage device.
"rwd"��
Open for reading and writing, as with "rw", and also require that every update to the file's content be written synchronously to the underlying storage device.
Specifically, under "rw", it says "If the file does not already exist then an attempt will be made to create it." Plus, "rws" (and "rwd") say "as with "rw"", so autocreate should apply to them also.
However, I have experimented and discovered that "rwd" works (on OSX), while "rws" does not work. The only difference appears to be that "rws" writes metadata in addition to the content. Hmm...
I don't know what the metadata should be but...
Could this "rws" metadata explain the dual writes behavior someone discovered on Windows using "rws" (i.e., one write for content file, and one for metadata)?
Could this metadata write explain the "File not found" message (i.e., perhaps the metadata file is not found) when "rws" is used to open an already existing (content) file (which has no matching metadata file)? I don' t know if the OSX JVM "File not found" behavior is correct/preferred/other (i.e., whether metadata file should be autocreated, even if the content file already existed without a metadata file), or whether it's just ambiguous (the docs at the URL do not cover this odd case), or whether my hypotheses are misguided, but it led me to the following.
It would be nice if a work around can be found to get this up and
running in the OSX environment but not have to pay the extra performance
cost across all the other JVM's.
When I replace "rws" with "rwd", the code (seems to) works on OSX. I presume it also works on:
I have tested this on a sun jdk 1.4.2 and an ibm 1.42 on windows and I believe it has also been tested against a linux jdk 1.4.2.
but a double-check would be worthwhile if the patch seems appropriate. To be consistent with this hypothesis, I (went out on a limb and) replaced all (five) LogToFile.java occurrences of "rws" with "rwd". I'm sure you'll sanity check that approach.
-=-
Also, along the way, I found what I believe is a subtle operator precedence bug in DirFile4.java which prevented "rwd" from taking effect (it became "rw" instead). So I offer a patch for DirFile4.java as well. This time I used 'diff -c1' from the derby root directory (hopefully more convenient to apply patches).
Two patches follow.
*** java/engine/org/apache/derby/impl/store/raw/log/LogToFile.java.org Thu Aug 26 12:07:32 2004
--- java/engine/org/apache/derby/impl/store/raw/log/LogToFile.java Fri Sep 3 18:14:17 2004
***************
*** 897,899 ****
theLog.close();
! theLog= privRandomAccessFile(logFile, "rws");
//postion the log at the current end postion
--- 897,899 ----
theLog.close();
! theLog= privRandomAccessFile(logFile, "rwd");
//postion the log at the current end postion
***************
*** 933,935 ****
if(isWriteSynced)
! theLog = privRandomAccessFile(logFile, "rws");
else
--- 933,935 ----
if(isWriteSynced)
! theLog = privRandomAccessFile(logFile, "rwd");
else
***************
*** 1933,1935 ****
newLog.close();
! newLog= privRandomAccessFile(newLogFile, "rws");
newLog.seek(endPosition);
--- 1933,1935 ----
newLog.close();
! newLog= privRandomAccessFile(newLogFile, "rwd");
newLog.seek(endPosition);
***************
*** 3023,3025 ****
theLog.close();
! theLog= privRandomAccessFile(logFile, "rws");
//postion the log at the current log end postion
--- 3023,3025 ----
theLog.close();
! theLog= privRandomAccessFile(logFile, "rwd");
//postion the log at the current log end postion
***************
*** 4849,4851 ****
if(isWriteSynced)
! return privRandomAccessFile(logFile, "rws");
else
--- 4849,4851 ----
if(isWriteSynced)
! return privRandomAccessFile(logFile, "rwd");
else
-=-
I believe the following code was intended to preserve the "rwd" if rwsOK==TRUE (as with the "rws" preservation). On that assumption and to enable "rwd" to take effect (it was replaced with "rw" under original code), I force intended operator precedence with an additional pair of parens.
*** java/engine/org/apache/derby/impl/io/DirFile4.java.org Tue Aug 31 22:09:51 2004
--- java/engine/org/apache/derby/impl/io/DirFile4.java Fri Sep 3 17:23:12 2004
***************
*** 241,244 ****
// Assume that modes "rws" and "rwd" are not supported.
! if(!rwsOK && "rws".equals( mode) || "rwd".equals( mode))
mode = "rw";
return new DirRandomAccessFile4( (File) this, mode);
--- 241,245 ----
// Assume that modes "rws" and "rwd" are not supported.
! if( !rwsOK && ("rws".equals( mode) || "rwd".equals( mode)))
mode = "rw";
return new DirRandomAccessFile4( (File) this, mode);
-=-
I hope that all makes sense and proves useful.
Cheers,
= Joe =
