Knut Anders Hatlen wrote:
Mike Matrigali <[EMAIL PROTECTED]> writes:
There are comments/code in this file about getting synchonization on
page 0. I went through the comments in the file and in the derby issue
and my understanding is that the problem is caused by the behavior on
windows where the use of a channel affects the current position of the
file that the channel is based off of. The case in point is one thread
using non-channel code to operate on page 0 while another thread is
using channel code.
I think this code intended to solve a different problem. Consider this
situation:
Thread 1 tries to update the borrowed space on page 0. It does the
following:
a) synchronize on the container object
b) read AllocPage.MAX_BORROWED_SPACE bytes from page 0 into a byte
array
c) update the borrowed space portion of the byte array
d) write AllocPage.MAX_BORROWED_SPACE bytes back to page 0
e) release synchronization on the container object
At the same time in Thread 2, the page cache decides to clean page 0. If
that happens after (b) and before (d), all the non-borrowed space in the
first AllocPage.MAX_BORROWED_SPACE bytes of page 0 will come from the
old version of the page, whereas the rest of the bytes come from the new
version of the page. This means that the new version of page is not
fully written to disk, although the page cache thinks the page is
"clean".
If we synchronize on the container object when cleaning the page, the
above problem goes away, since the cleaning of the page must happen
before (a) or after (e). In those cases we'll always end up with the new
version of the page on the disk. Note that the problem is only present
at page 0, since the cleaning of any other page would not touch the same
bytes as Thread 1.
What I am missing is why the problem is limited to channel code on page
0? What stops the case of non-channel code operating on page 0 while
channel code is operating on another page and affecting the non-channel
page 0 operation?
I considered doing the work to just get rid of the non-channel code, but
small platform environments including foundation 1.0, 1.1 and 1.1.2
don't support nio.
I thought all the non-channel code in RAFContainer was overloaded by
methods in RAFContainer4 after DERBY-3347. Which code did you find that
still used the old I/O API when running JDK>=1.4?
This is by all means not a complete list, at this point I am still just
doing code inspection, so it is hard to figure out what can't be called
because it is overridden. I just concentrated on the calls in the code
that used seek and then io do do writes. So far here are the ones I
have seen called, just by doing a quick limited set of tests (I also
verified that RAFContainer4 calls were being made also):
Got to FileContainer.java!getEmbryonicPage
Got to FileContainer.java!writeAtOffset
Got to RAFContainer!padFile()
Got to RAFContainer!writePage()
I have not looked at all the stack traces, but here is one for each of them:
java.lang.Throwable
at
org.apache.derby.impl.store.raw.data.FileContainer.writeAtOffset(FileContainer.java:912)
at
org.apache.derby.impl.store.raw.data.FileContainer.writeHeader(FileContainer.java:889)
at
org.apache.derby.impl.store.raw.data.RAFContainer.writeRAFHeader(RAFContainer.java:702)
at
org.apache.derby.impl.store.raw.data.RAFContainer.clean(RAFContainer.java:549)
at
org.apache.derby.impl.services.cache.ConcurrentCache.cleanAndUnkeepEntry(ConcurrentCache.java:551)
at
org.apache.derby.impl.services.cache.ConcurrentCache.cleanCache(ConcurrentCache.java:509)
at
org.apache.derby.impl.services.cache.ConcurrentCache.cleanAll(ConcurrentCache.java:460)
at
org.apache.derby.impl.store.raw.data.BaseDataFileFactory.checkpoint(BaseDataFileFactory.java:1203)
at
org.apache.derby.impl.store.raw.log.LogToFile.checkpointWithTran(LogToFile.java:1645)
at
org.apache.derby.impl.store.raw.log.LogToFile.checkpoint(LogToFile.java:1484)
at org.apache.derby.impl.store.raw.RawStore.stop(RawStore.java:363)
at
org.apache.derby.impl.services.monitor.TopService.stop(TopService.java:405)
at
org.apache.derby.impl.services.monitor.TopService.shutdown(TopService.java:349)
at
org.apache.derby.impl.services.monitor.BaseMonitor.shutdown(BaseMonitor.java:237)
at
org.apache.derby.impl.services.monitor.BaseMonitor.shutdown(BaseMonitor.java:203)
at
org.apache.derby.jdbc.InternalDriver.connect(InternalDriver.java:231)
at
org.apache.derby.jdbc.AutoloadedDriver.connect(AutoloadedDriver.java:119)
at java.sql.DriverManager.getConnection(Unknown Source)
at java.sql.DriverManager.getConnection(Unknown Source)
at org.apache.derby.impl.tools.ij.utilMain.cleanupGo(utilMain.java:416)
at org.apache.derby.impl.tools.ij.utilMain.go(utilMain.java:249)
at org.apache.derby.impl.tools.ij.Main.go(Main.java:215)
at org.apache.derby.impl.tools.ij.Main.mainCore(Main.java:181)
at org.apache.derby.impl.tools.ij.Main.main(Main.java:73)
at org.apache.derby.tools.ij.main(ij.java:59)
at
org.apache.derby.impl.store.raw.data.FileContainer.getEmbryonicPage(FileContainer.java:959)
at
org.apache.derby.impl.store.raw.data.RAFContainer.writeRAFHeader(RAFContainer.java:697)
at
org.apache.derby.impl.store.raw.data.RAFContainer.clean(RAFContainer.java:549)
at
org.apache.derby.impl.services.cache.ConcurrentCache.cleanAndUnkeepEntry(ConcurrentCache.java:551)
at
org.apache.derby.impl.services.cache.ConcurrentCache.cleanCache(ConcurrentCache.java:509)
at
org.apache.derby.impl.services.cache.ConcurrentCache.cleanAll(ConcurrentCache.java:460)
at
org.apache.derby.impl.store.raw.data.BaseDataFileFactory.checkpoint(BaseDataFileFactory.java:1203)
at
org.apache.derby.impl.store.raw.log.LogToFile.checkpointWithTran(LogToFile.java:1645)
at
org.apache.derby.impl.store.raw.log.LogToFile.checkpoint(LogToFile.java:1484)
at org.apache.derby.impl.store.raw.RawStore.stop(RawStore.java:363)
at
org.apache.derby.impl.services.monitor.TopService.stop(TopService.java:405)
at
org.apache.derby.impl.services.monitor.TopService.shutdown(TopService.java:349)
at
org.apache.derby.impl.services.monitor.BaseMonitor.shutdown(BaseMonitor.java:237)
at
org.apache.derby.impl.services.monitor.BaseMonitor.shutdown(BaseMonitor.java:203)
at
org.apache.derby.jdbc.InternalDriver.connect(InternalDriver.java:231)
at
org.apache.derby.jdbc.AutoloadedDriver.connect(AutoloadedDriver.java:119)
at java.sql.DriverManager.getConnection(Unknown Source)
at java.sql.DriverManager.getConnection(Unknown Source)
at org.apache.derby.impl.tools.ij.utilMain.cleanupGo(utilMain.java:416)
at org.apache.derby.impl.tools.ij.utilMain.go(utilMain.java:249)
at org.apache.derby.impl.tools.ij.Main.go(Main.java:215)
at org.apache.derby.impl.tools.ij.Main.mainCore(Main.java:181)
at org.apache.derby.impl.tools.ij.Main.main(Main.java:73)
at org.apache.derby.tools.ij.main(ij.java:59)
at
org.apache.derby.impl.store.raw.data.RAFContainer.padFile(RAFContainer.java:449)
at
org.apache.derby.impl.store.raw.data.RAFContainer.writeRAFHeader(RAFContainer.java:711)
at
org.apache.derby.impl.store.raw.data.RAFContainer.run(RAFContainer.java:1343)
at java.security.AccessController.doPrivileged(AccessController.java:251)
at
org.apache.derby.impl.store.raw.data.RAFContainer.createContainer(RAFContainer.java:824)
at
org.apache.derby.impl.store.raw.data.RAFContainer4.createContainer(RAFContainer4.java:159)
at
org.apache.derby.impl.store.raw.data.FileContainer.createIdent(FileContainer.java:459)
at
org.apache.derby.impl.store.raw.data.RAFContainer.createIdentity(RAFContainer.java:149)
at
org.apache.derby.impl.services.cache.ConcurrentCache.create(ConcurrentCache.java:388)
at
org.apache.derby.impl.store.raw.data.BaseDataFileFactory.addContainer(BaseDataFileFactory.java:812)
at org.apache.derby.impl.store.raw.xact.Xact.addContainer(Xact.java:1329)
at org.apache.derby.impl.store.access.heap.Heap.create(Heap.java:274)
java.lang.Throwable
at
org.apache.derby.impl.store.raw.data.RAFContainer.writePage(RAFContainer.java:301)
at
org.apache.derby.impl.store.raw.data.CachedPage.writePage(CachedPage.java:782)
at
org.apache.derby.impl.store.raw.data.CachedPage.createIdentity(CachedPage.java:303)
at
org.apache.derby.impl.services.cache.ConcurrentCache.create(ConcurrentCache.java:388)
at
org.apache.derby.impl.store.raw.data.FileContainer.initPage(FileContainer.java:2348)
at
org.apache.derby.impl.store.raw.data.FileContainer.makeAllocPage(FileContainer.java:2282)
at
org.apache.derby.impl.store.raw.data.FileContainer.findAllocPageForAdd(FileContainer.java:2061)
at
org.apache.derby.impl.store.raw.data.FileContainer.newPage(FileContainer.java:1622)
at
org.apache.derby.impl.store.raw.data.BaseContainer.addPage(BaseContainer.java:314)
at
org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(BaseContainerHandle.java:183)
at
org.apache.derby.impl.store.raw.data.BaseDataFileFactory.addContainer(BaseDataFileFactory.java:872)
at
org.apache.derby.impl.store.raw.xact.Xact.addContainer(Xact.java:1329)
at org.apache.derby.impl.store.access.heap.Heap.create(Heap.java:274)
at
org.apache.derby.impl.store.access.heap.HeapConglomerateFactory.createConglomerate(HeapConglomerateFactory.java:195)
at
org.apache.derby.impl.store.access.RAMTransaction.createConglomerate(RAMTransaction.java:826)
at
org.apache.derby.impl.sql.execute.CreateTableConstantAction.executeConstantAction(CreateTableConstantAction.java:201)
at
org.apache.derby.impl.sql.execute.MiscResultSet.open(MiscResultSet.java:64)
at
org.apache.derby.impl.sql.GenericPreparedStatement.execute(GenericPreparedStatement.java:384)
at
org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(EmbedStatement.java:1235)
at
org.apache.derby.impl.jdbc.EmbedStatement.execute(EmbedStatement.java:625)
at
org.apache.derby.impl.jdbc.EmbedStatement.execute(EmbedStatement.java:555)
at org.apache.derby.impl.tools.ij.ij.executeImmediate(ij.java:329)
at org.apache.derby.impl.tools.ij.utilMain.doCatch(utilMain.java:508)
at
org.apache.derby.impl.tools.ij.utilMain.runScriptGuts(utilMain.java:350)
at org.apache.derby.impl.tools.ij.utilMain.go(utilMain.java:248)
at org.apache.derby.impl.tools.ij.Main.go(Main.java:215)
at org.apache.derby.impl.tools.ij.Main.mainCore(Main.java:181)
at org.apache.derby.impl.tools.ij.Main.main(Main.java:73)
at org.apache.derby.tools.ij.main(ij.java:59)
