Replication fails when creating index -------------------------------------
Key: DERBY-3533 URL: https://issues.apache.org/jira/browse/DERBY-3533 Project: Derby Issue Type: Bug Components: Replication Affects Versions: 10.4.0.0 Reporter: Øystein Grøvlen I created an index while replication was running, and the slave seems to have failed at that time. Below is excerpts from derby.log on the slave. The container id referred to, matches the id of the index on the master. ------------ BEGIN SHUTDOWN ERROR STACK ------------- ---- BEGIN REPLICATION ERROR MESSAGE (3/11/08 2:32 PM) ---- ERROR XSLA7: Cannot redo operation Page Operation: Page(349,Container(0, 1041)) pageVersion 180 : Insert : Slot=177 recordId=183 in the log. at org.apache.derby.iapi.error.StandardException.newException(StandardException.java:296) at org.apache.derby.impl.store.raw.log.FileLogger.redo(FileLogger.java:1525) at org.apache.derby.impl.store.raw.log.LogToFile.recover(LogToFile.java:920) at org.apache.derby.impl.store.raw.RawStore.boot(RawStore.java:334) at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1999) at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:291) at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:553) at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:427) at org.apache.derby.impl.store.access.RAMAccessManager.boot(RAMAccessManager.java:1019) at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1999) at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:291) at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:553) at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:427) at org.apache.derby.impl.db.BasicDatabase.bootStore(BasicDatabase.java:789) at org.apache.derby.impl.db.BasicDatabase.boot(BasicDatabase.java:205) at org.apache.derby.impl.db.SlaveDatabase.bootBasicDatabase(SlaveDatabase.java:421) at org.apache.derby.impl.db.SlaveDatabase.access$000(SlaveDatabase.java:70) at org.apache.derby.impl.db.SlaveDatabase$SlaveDatabaseBootThread.run(SlaveDatabase.java:308) at java.lang.Thread.run(Thread.java:619) Caused by: ERROR XSDG0: Page Page(349,Container(0, 1041)) could not be read from disk. at org.apache.derby.iapi.error.StandardException.newException(StandardException.java:336) at org.apache.derby.impl.store.raw.data.CachedPage.readPage(CachedPage.java:688) at org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(CachedPage.java:190) at org.apache.derby.impl.services.cache.ConcurrentCache.find(ConcurrentCache.java:288) at org.apache.derby.impl.store.raw.data.FileContainer.getAnyPage(FileContainer.java:2493) at org.apache.derby.impl.store.raw.data.BaseContainer.getAnyPage(BaseContainer.java:496) at org.apache.derby.impl.store.raw.data.BaseContainerHandle.getAnyPage(BaseContainerHandle.java:611) at org.apache.derby.impl.store.raw.data.PageBasicOperation.findpage(PageBasicOperation.java:304) at org.apache.derby.impl.store.raw.data.PageBasicOperation.needsRedo(PageBasicOperation.java:160) at org.apache.derby.impl.store.raw.log.FileLogger.redo(FileLogger.java:1395) ... 17 more Caused by: java.io.EOFException: Reached end of file while attempting to read a whole page. at org.apache.derby.impl.store.raw.data.RAFContainer4.readFull(RAFContainer4.java:378) at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage(RAFContainer4.java:201) at org.apache.derby.impl.store.raw.data.CachedPage.readPage(CachedPage.java:664) ... 25 more ============= begin nested exception, level (1) =========== ERROR XSDG0: Page Page(349,Container(0, 1041)) could not be read from disk. at org.apache.derby.iapi.error.StandardException.newException(StandardException.java:336) at org.apache.derby.impl.store.raw.data.CachedPage.readPage(CachedPage.java:688) at org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(CachedPage.java:190) at org.apache.derby.impl.services.cache.ConcurrentCache.find(ConcurrentCache.java:288) at org.apache.derby.impl.store.raw.data.FileContainer.getAnyPage(FileContainer.java:2493) at org.apache.derby.impl.store.raw.data.BaseContainer.getAnyPage(BaseContainer.java:496) at org.apache.derby.impl.store.raw.data.BaseContainerHandle.getAnyPage(BaseContainerHandle.java:611) at org.apache.derby.impl.store.raw.data.PageBasicOperation.findpage(PageBasicOperation.java:304) at org.apache.derby.impl.store.raw.data.PageBasicOperation.needsRedo(PageBasicOperation.java:160) at org.apache.derby.impl.store.raw.log.FileLogger.redo(FileLogger.java:1395) at org.apache.derby.impl.store.raw.log.LogToFile.recover(LogToFile.java:920) at org.apache.derby.impl.store.raw.RawStore.boot(RawStore.java:334) at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1999) at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:291) at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:553) at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:427) at org.apache.derby.impl.store.access.RAMAccessManager.boot(RAMAccessManager.java:1019) at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1999) at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:291) at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:553) at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:427) at org.apache.derby.impl.db.BasicDatabase.bootStore(BasicDatabase.java:789) at org.apache.derby.impl.db.BasicDatabase.boot(BasicDatabase.java:205) at org.apache.derby.impl.db.SlaveDatabase.bootBasicDatabase(SlaveDatabase.java:421) at org.apache.derby.impl.db.SlaveDatabase.access$000(SlaveDatabase.java:70) at org.apache.derby.impl.db.SlaveDatabase$SlaveDatabaseBootThread.run(SlaveDatabase.java:308) at java.lang.Thread.run(Thread.java:619) Caused by: java.io.EOFException: Reached end of file while attempting to read a whole page. at org.apache.derby.impl.store.raw.data.RAFContainer4.readFull(RAFContainer4.java:378) at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage(RAFContainer4.java:201) at org.apache.derby.impl.store.raw.data.CachedPage.readPage(CachedPage.java:664) ... 25 more ============= end nested exception, level (1) =========== ============= begin nested exception, level (2) =========== java.io.EOFException: Reached end of file while attempting to read a whole page. at org.apache.derby.impl.store.raw.data.RAFContainer4.readFull(RAFContainer4.java:378) at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage(RAFContainer4.java:201) at org.apache.derby.impl.store.raw.data.CachedPage.readPage(CachedPage.java:664) at org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(CachedPage.java:190) at org.apache.derby.impl.services.cache.ConcurrentCache.find(ConcurrentCache.java:288) at org.apache.derby.impl.store.raw.data.FileContainer.getAnyPage(FileContainer.java:2493) at org.apache.derby.impl.store.raw.data.BaseContainer.getAnyPage(BaseContainer.java:496) at org.apache.derby.impl.store.raw.data.BaseContainerHandle.getAnyPage(BaseContainerHandle.java:611) at org.apache.derby.impl.store.raw.data.PageBasicOperation.findpage(PageBasicOperation.java:304) at org.apache.derby.impl.store.raw.data.PageBasicOperation.needsRedo(PageBasicOperation.java:160) at org.apache.derby.impl.store.raw.log.FileLogger.redo(FileLogger.java:1395) at org.apache.derby.impl.store.raw.log.LogToFile.recover(LogToFile.java:920) at org.apache.derby.impl.store.raw.RawStore.boot(RawStore.java:334) at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1999) at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:291) at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:553) at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:427) at org.apache.derby.impl.store.access.RAMAccessManager.boot(RAMAccessManager.java:1019) at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1999) at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:291) at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:553) at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:427) at org.apache.derby.impl.db.BasicDatabase.bootStore(BasicDatabase.java:789) at org.apache.derby.impl.db.BasicDatabase.boot(BasicDatabase.java:205) at org.apache.derby.impl.db.SlaveDatabase.bootBasicDatabase(SlaveDatabase.java:421) at org.apache.derby.impl.db.SlaveDatabase.access$000(SlaveDatabase.java:70) at org.apache.derby.impl.db.SlaveDatabase$SlaveDatabaseBootThread.run(SlaveDatabase.java:308) at java.lang.Thread.run(Thread.java:619) ============= end nested exception, level (2) =========== ------------ END SHUTDOWN ERROR STACK ------------- Replication slave role was stopped for database 'slaveDB'. Replication slave got a fatal error for database 'slaveDB'. Replication will be stopped. ERROR XSLAA: The store has been marked for shutdown by an earlier exception. at org.apache.derby.iapi.error.StandardException.newException(StandardException.java:279) at org.apache.derby.impl.store.raw.log.LogToFile.appendLogRecord(LogToFile.java:3704) at org.apache.derby.impl.store.replication.slave.SlaveController$SlaveLogReceiverThread.handleLogChunk(SlaveController.java:535) at org.apache.derby.impl.store.replication.slave.SlaveController$SlaveLogReceiverThread.run(SlaveController.java:481) Caused by: ERROR XSLA7: Cannot redo operation Page Operation: Page(349,Container(0, 1041)) pageVersion 180 : Insert : Slot=177 recordId=183 in the log. at org.apache.derby.iapi.error.StandardException.newException(StandardException.java:296) at org.apache.derby.impl.store.raw.log.FileLogger.redo(FileLogger.java:1525) at org.apache.derby.impl.store.raw.log.LogToFile.recover(LogToFile.java:920) at org.apache.derby.impl.store.raw.RawStore.boot(RawStore.java:334) at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1999) at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:291) at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:553) at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:427) at org.apache.derby.impl.store.access.RAMAccessManager.boot(RAMAccessManager.java:1019) at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1999) at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:291) at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:553) at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:427) at org.apache.derby.impl.db.BasicDatabase.bootStore(BasicDatabase.java:789) at org.apache.derby.impl.db.BasicDatabase.boot(BasicDatabase.java:205) at org.apache.derby.impl.db.SlaveDatabase.bootBasicDatabase(SlaveDatabase.java:421) at org.apache.derby.impl.db.SlaveDatabase.access$000(SlaveDatabase.java:70) at org.apache.derby.impl.db.SlaveDatabase$SlaveDatabaseBootThread.run(SlaveDatabase.java:308) at java.lang.Thread.run(Thread.java:619) Caused by: ERROR XSDG0: Page Page(349,Container(0, 1041)) could not be read from disk. at org.apache.derby.iapi.error.StandardException.newException(StandardException.java:336) at org.apache.derby.impl.store.raw.data.CachedPage.readPage(CachedPage.java:688) at org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(CachedPage.java:190) at org.apache.derby.impl.services.cache.ConcurrentCache.find(ConcurrentCache.java:288) at org.apache.derby.impl.store.raw.data.FileContainer.getAnyPage(FileContainer.java:2493) at org.apache.derby.impl.store.raw.data.BaseContainer.getAnyPage(BaseContainer.java:496) at org.apache.derby.impl.store.raw.data.BaseContainerHandle.getAnyPage(BaseContainerHandle.java:611) at org.apache.derby.impl.store.raw.data.PageBasicOperation.findpage(PageBasicOperation.java:304) at org.apache.derby.impl.store.raw.data.PageBasicOperation.needsRedo(PageBasicOperation.java:160) at org.apache.derby.impl.store.raw.log.FileLogger.redo(FileLogger.java:1395) ... 17 more Caused by: java.io.EOFException: Reached end of file while attempting to read a whole page. at org.apache.derby.impl.store.raw.data.RAFContainer4.readFull(RAFContainer4.java:378) at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage(RAFContainer4.java:201) at org.apache.derby.impl.store.raw.data.CachedPage.readPage(CachedPage.java:664) ... 25 more ============= begin nested exception, level (1) =========== ERROR XSLA7: Cannot redo operation Page Operation: Page(349,Container(0, 1041)) pageVersion 180 : Insert : Slot=177 recordId=183 in the log. at org.apache.derby.iapi.error.StandardException.newException(StandardException.java:296) at org.apache.derby.impl.store.raw.log.FileLogger.redo(FileLogger.java:1525) at org.apache.derby.impl.store.raw.log.LogToFile.recover(LogToFile.java:920) at org.apache.derby.impl.store.raw.RawStore.boot(RawStore.java:334) at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1999) at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:291) at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:553) at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:427) at org.apache.derby.impl.store.access.RAMAccessManager.boot(RAMAccessManager.java:1019) at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1999) at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:291) at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:553) at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:427) at org.apache.derby.impl.db.BasicDatabase.bootStore(BasicDatabase.java:789) at org.apache.derby.impl.db.BasicDatabase.boot(BasicDatabase.java:205) at org.apache.derby.impl.db.SlaveDatabase.bootBasicDatabase(SlaveDatabase.java:421) at org.apache.derby.impl.db.SlaveDatabase.access$000(SlaveDatabase.java:70) at org.apache.derby.impl.db.SlaveDatabase$SlaveDatabaseBootThread.run(SlaveDatabase.java:308) at java.lang.Thread.run(Thread.java:619) Caused by: ERROR XSDG0: Page Page(349,Container(0, 1041)) could not be read from disk. at org.apache.derby.iapi.error.StandardException.newException(StandardException.java:336) at org.apache.derby.impl.store.raw.data.CachedPage.readPage(CachedPage.java:688) at org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(CachedPage.java:190) at org.apache.derby.impl.services.cache.ConcurrentCache.find(ConcurrentCache.java:288) at org.apache.derby.impl.store.raw.data.FileContainer.getAnyPage(FileContainer.java:2493) at org.apache.derby.impl.store.raw.data.BaseContainer.getAnyPage(BaseContainer.java:496) at org.apache.derby.impl.store.raw.data.BaseContainerHandle.getAnyPage(BaseContainerHandle.java:611) at org.apache.derby.impl.store.raw.data.PageBasicOperation.findpage(PageBasicOperation.java:304) at org.apache.derby.impl.store.raw.data.PageBasicOperation.needsRedo(PageBasicOperation.java:160) at org.apache.derby.impl.store.raw.log.FileLogger.redo(FileLogger.java:1395) ... 17 more Caused by: java.io.EOFException: Reached end of file while attempting to read a whole page. at org.apache.derby.impl.store.raw.data.RAFContainer4.readFull(RAFContainer4.java:378) at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage(RAFContainer4.java:201) at org.apache.derby.impl.store.raw.data.CachedPage.readPage(CachedPage.java:664) ... 25 more ============= end nested exception, level (1) =========== ============= begin nested exception, level (2) =========== ERROR XSDG0: Page Page(349,Container(0, 1041)) could not be read from disk. at org.apache.derby.iapi.error.StandardException.newException(StandardException.java:336) at org.apache.derby.impl.store.raw.data.CachedPage.readPage(CachedPage.java:688) at org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(CachedPage.java:190) at org.apache.derby.impl.services.cache.ConcurrentCache.find(ConcurrentCache.java:288) at org.apache.derby.impl.store.raw.data.FileContainer.getAnyPage(FileContainer.java:2493) at org.apache.derby.impl.store.raw.data.BaseContainer.getAnyPage(BaseContainer.java:496) at org.apache.derby.impl.store.raw.data.BaseContainerHandle.getAnyPage(BaseContainerHandle.java:611) at org.apache.derby.impl.store.raw.data.PageBasicOperation.findpage(PageBasicOperation.java:304) at org.apache.derby.impl.store.raw.data.PageBasicOperation.needsRedo(PageBasicOperation.java:160) at org.apache.derby.impl.store.raw.log.FileLogger.redo(FileLogger.java:1395) at org.apache.derby.impl.store.raw.log.LogToFile.recover(LogToFile.java:920) at org.apache.derby.impl.store.raw.RawStore.boot(RawStore.java:334) at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1999) at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:291) at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:553) at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:427) at org.apache.derby.impl.store.access.RAMAccessManager.boot(RAMAccessManager.java:1019) at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1999) at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:291) at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:553) at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:427) at org.apache.derby.impl.db.BasicDatabase.bootStore(BasicDatabase.java:789) at org.apache.derby.impl.db.BasicDatabase.boot(BasicDatabase.java:205) at org.apache.derby.impl.db.SlaveDatabase.bootBasicDatabase(SlaveDatabase.java:421) at org.apache.derby.impl.db.SlaveDatabase.access$000(SlaveDatabase.java:70) at org.apache.derby.impl.db.SlaveDatabase$SlaveDatabaseBootThread.run(SlaveDatabase.java:308) at java.lang.Thread.run(Thread.java:619) Caused by: java.io.EOFException: Reached end of file while attempting to read a whole page. at org.apache.derby.impl.store.raw.data.RAFContainer4.readFull(RAFContainer4.java:378) at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage(RAFContainer4.java:201) at org.apache.derby.impl.store.raw.data.CachedPage.readPage(CachedPage.java:664) ... 25 more ============= end nested exception, level (2) =========== ============= begin nested exception, level (3) =========== java.io.EOFException: Reached end of file while attempting to read a whole page. at org.apache.derby.impl.store.raw.data.RAFContainer4.readFull(RAFContainer4.java:378) at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage(RAFContainer4.java:201) at org.apache.derby.impl.store.raw.data.CachedPage.readPage(CachedPage.java:664) at org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(CachedPage.java:190) at org.apache.derby.impl.services.cache.ConcurrentCache.find(ConcurrentCache.java:288) at org.apache.derby.impl.store.raw.data.FileContainer.getAnyPage(FileContainer.java:2493) at org.apache.derby.impl.store.raw.data.BaseContainer.getAnyPage(BaseContainer.java:496) at org.apache.derby.impl.store.raw.data.BaseContainerHandle.getAnyPage(BaseContainerHandle.java:611) at org.apache.derby.impl.store.raw.data.PageBasicOperation.findpage(PageBasicOperation.java:304) at org.apache.derby.impl.store.raw.data.PageBasicOperation.needsRedo(PageBasicOperation.java:160) at org.apache.derby.impl.store.raw.log.FileLogger.redo(FileLogger.java:1395) at org.apache.derby.impl.store.raw.log.LogToFile.recover(LogToFile.java:920) at org.apache.derby.impl.store.raw.RawStore.boot(RawStore.java:334) at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1999) at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:291) at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:553) at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:427) at org.apache.derby.impl.store.access.RAMAccessManager.boot(RAMAccessManager.java:1019) at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1999) at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:291) at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:553) at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:427) at org.apache.derby.impl.db.BasicDatabase.bootStore(BasicDatabase.java:789) at org.apache.derby.impl.db.BasicDatabase.boot(BasicDatabase.java:205) at org.apache.derby.impl.db.SlaveDatabase.bootBasicDatabase(SlaveDatabase.java:421) at org.apache.derby.impl.db.SlaveDatabase.access$000(SlaveDatabase.java:70) at org.apache.derby.impl.db.SlaveDatabase$SlaveDatabaseBootThread.run(SlaveDatabase.java:308) at java.lang.Thread.run(Thread.java:619) ============= end nested exception, level (3) =========== -------------------- END REPLICATION ERROR MESSAGE --------------------- Replication slave role was stopped for database 'slaveDB'. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.