FYI -- I disabled the WAL (add -Dstorage.useWAL=false to ORIENTDB_SETTINGS in server.[bat|sh]) and I haven't had the problem since (albeit with limited usage).
On Monday, July 7, 2014 5:33:34 PM UTC-4, Rob Gratz wrote: > > You are not alone. > > On Thursday, July 3, 2014 1:44:18 PM UTC-5, Brian O'Keefe wrote: >> >> Every day I grab the latest 1.7.5 snapshot but this still is occurring >> and it is getting frustrating. I've modified the way I am creating and >> destroying database connections to try to resolve the issue, but that did >> not stop it. I disabled the WAL this time, so I am hoping that this stops >> this from happening. The recurring theme always seems to be accessing the >> logSegments ArrayList in the WAL with index = -1. >> >> This seems way to simple, but adding the line: >> if (logSegments.size() == 0) return; >> >> Above the 2 trouble lines below resolve this? >> >> LogSegment last = logSegments.get(logSegments.size() - 1); >> [OWriteAheadLog.java:733] >> and >> LogSegment last = logSegments.get(logSegments.size() - 1); >> [OWriteAheadLog.java:764] >> >> I'm not touching the code because I don't know this well enough, but it >> seems like that is an issue waiting to happen unless you can be absolutely >> sure "logSegments" will never be empty. >> >> >> On Tuesday, July 1, 2014 1:44:42 PM UTC-4, Brian O'Keefe wrote: >>> >>> It happened again today with 1.7.5-SNAPSHOT and this time it did >>> corrupt. >>> >>> Server at time of exception: >>> -1 >>> -> java.util.ArrayList.elementData(ArrayList.java:400) >>> -> java.util.ArrayList.get(ArrayList.java:413) >>> -> >>> com.orientechnologies.orient.core.storage.impl.local.paginated.wal.OWriteAheadLog.flush(OWriteAheadLog.java:733) >>> -> >>> com.orientechnologies.orient.core.index.hashindex.local.cache.OWOWCache.flushPage(OWOWCache.java:1082) >>> -> >>> com.orientechnologies.orient.core.index.hashindex.local.cache.OWOWCache.access$1000(OWOWCache.java:67) >>> -> >>> com.orientechnologies.orient.core.index.hashindex.local.cache.OWOWCache$PeriodicFlushTask.iterateBySubRing(OWOWCache. >>> java:291) >>> -> >>> com.orientechnologies.orient.core.index.hashindex.local.cache.OWOWCache$PeriodicFlushTask.flushRing(OWOWCache.java:24 >>> 6) >>> -> >>> com.orientechnologies.orient.core.index.hashindex.local.cache.OWOWCache$PeriodicFlushTask.run(OWOWCache.java:222) >>> -> >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) >>> -> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) >>> -> >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) >>> >>> -> >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) >>> -> >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>> -> >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>> -> java.lang.Thread.run(Thread.java:744)Exception during data flush. >>> >>> Loading application after database restart: >>> com.orientechnologies.orient.core.exception.ODatabaseException: Error on >>> retrieving record #19:11575 (cluster: *redacted*) >>> at >>> com.orientechnologies.orient.core.db.raw.ODatabaseRaw.read(ODatabaseRaw.java:252) >>> at >>> com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.executeReadRecord(ODatabaseRecordAbstract.java:1017) >>> at >>> com.orientechnologies.orient.core.tx.OTransactionNoTx.loadRecord(OTransactionNoTx.java:65) >>> at >>> com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.load(ODatabaseRecordTx.java:251) >>> at >>> com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.load(ODatabaseRecordTx.java:40) >>> at >>> com.orientechnologies.orient.core.db.ODatabaseRecordWrapperAbstract.load(ODatabaseRecordWrapperAbstract.java:227) >>> at >>> com.orientechnologies.orient.core.db.ODatabaseRecordWrapperAbstract.load(ODatabaseRecordWrapperAbstract.java:52) >>> at >>> com.orientechnologies.orient.core.record.impl.ODocument.field(ODocument.java:591) >>> at >>> com.orientechnologies.orient.core.record.impl.ODocument.merge(ODocument.java:972) >>> at >>> com.orientechnologies.orient.core.record.impl.ODocument.merge(ODocument.java:918) >>> at >>> com.orientechnologies.orient.server.network.protocol.binary.OBinaryNetworkProtocolAbstract.updateRecord(OBinaryNetworkProtocolAbstract.java:307) >>> at >>> com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.updateRecord(ONetworkProtocolBinary.java:1343) >>> at >>> com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.executeRequest(ONetworkProtocolBinary.java:309) >>> at >>> com.orientechnologies.orient.server.network.protocol.binary.OBinaryNetworkProtocolAbstract.execute(OBinaryNetworkProtocolAbstract.java:169) >>> at >>> com.orientechnologies.common.thread.OSoftThread.run(OSoftThread.java:45) >>> Caused by: >>> com.orientechnologies.orient.core.exception.ODatabaseException: Error >>> during data decompression. >>> at >>> com.orientechnologies.orient.core.compression.impl.OSnappyCompression.uncompress(OSnappyCompression.java:54) >>> at >>> com.orientechnologies.orient.core.storage.impl.local.paginated.OPaginatedCluster.readRecord(OPaginatedCluster.java:631) >>> at >>> com.orientechnologies.orient.core.storage.impl.local.paginated.OLocalPaginatedStorage.readRecord(OLocalPaginatedStorage.java:1480) >>> at >>> com.orientechnologies.orient.core.storage.impl.local.paginated.OLocalPaginatedStorage.readRecord(OLocalPaginatedStorage.java:745) >>> at >>> com.orientechnologies.orient.core.db.raw.ODatabaseRaw.read(ODatabaseRaw.java:246) >>> ... 14 more >>> Caused by: java.io.IOException: FAILED_TO_UNCOMPRESS(5) >>> at org.xerial.snappy.SnappyNative.throw_error(SnappyNative.java:84) >>> at org.xerial.snappy.SnappyNative.rawUncompress(Native Method) >>> at org.xerial.snappy.Snappy.rawUncompress(Snappy.java:444) >>> at org.xerial.snappy.Snappy.uncompress(Snappy.java:480) >>> at >>> com.orientechnologies.orient.core.compression.impl.OSnappyCompression.uncompress(OSnappyCompression.java:50) >>> ... 18 more >>> 2014-07-01 13:39:07,811 [main ] ERROR >>> TamrResultCsvExportParser - Unknown Exception >>> com.orientechnologies.orient.core.exception.ODatabaseException: Error on >>> retrieving record #19:31049 (cluster: *redacted*) >>> at >>> com.orientechnologies.orient.core.db.raw.ODatabaseRaw.read(ODatabaseRaw.java:252) >>> at >>> com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.executeReadRecord(ODatabaseRecordAbstract.java:1017) >>> at >>> com.orientechnologies.orient.core.tx.OTransactionNoTx.loadRecord(OTransactionNoTx.java:65) >>> at >>> com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.load(ODatabaseRecordTx.java:251) >>> at >>> com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.load(ODatabaseRecordTx.java:40) >>> at >>> com.orientechnologies.orient.core.db.ODatabaseRecordWrapperAbstract.load(ODatabaseRecordWrapperAbstract.java:227) >>> at >>> com.orientechnologies.orient.core.db.ODatabaseRecordWrapperAbstract.load(ODatabaseRecordWrapperAbstract.java:52) >>> at >>> com.orientechnologies.orient.core.record.impl.ODocument.field(ODocument.java:591) >>> at >>> com.orientechnologies.orient.core.record.impl.ODocument.merge(ODocument.java:972) >>> at >>> com.orientechnologies.orient.core.record.impl.ODocument.merge(ODocument.java:918) >>> at >>> com.orientechnologies.orient.server.network.protocol.binary.OBinaryNetworkProtocolAbstract.updateRecord(OBinaryNetworkProtocolAbstract.java:307) >>> at >>> com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.updateRecord(ONetworkProtocolBinary.java:1343) >>> at >>> com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.executeRequest(ONetworkProtocolBinary.java:309) >>> at >>> com.orientechnologies.orient.server.network.protocol.binary.OBinaryNetworkProtocolAbstract.execute(OBinaryNetworkProtocolAbstract.java:169) >>> at >>> com.orientechnologies.common.thread.OSoftThread.run(OSoftThread.java:45) >>> Caused by: >>> com.orientechnologies.orient.core.exception.ODatabaseException: Error >>> during data decompression. >>> at >>> com.orientechnologies.orient.core.compression.impl.OSnappyCompression.uncompress(OSnappyCompression.java:54) >>> at >>> com.orientechnologies.orient.core.storage.impl.local.paginated.OPaginatedCluster.readRecord(OPaginatedCluster.java:631) >>> at >>> com.orientechnologies.orient.core.storage.impl.local.paginated.OLocalPaginatedStorage.readRecord(OLocalPaginatedStorage.java:1480) >>> at >>> com.orientechnologies.orient.core.storage.impl.local.paginated.OLocalPaginatedStorage.readRecord(OLocalPaginatedStorage.java:745) >>> at >>> com.orientechnologies.orient.core.db.raw.ODatabaseRaw.read(ODatabaseRaw.java:246) >>> ... 14 more >>> Caused by: java.io.IOException: FAILED_TO_UNCOMPRESS(5) >>> at org.xerial.snappy.SnappyNative.throw_error(SnappyNative.java:84) >>> at org.xerial.snappy.SnappyNative.rawUncompress(Native Method) >>> at org.xerial.snappy.Snappy.rawUncompress(Snappy.java:444) >>> at org.xerial.snappy.Snappy.uncompress(Snappy.java:480) >>> at >>> com.orientechnologies.orient.core.compression.impl.OSnappyCompression.uncompress(OSnappyCompression.java:50) >>> ... 18 more >>> 2014-07-01 13:39:08,584 [main ] ERROR >>> TamrResultCsvExportParser - Unknown Exception >>> com.orientechnologies.orient.core.exception.ODatabaseException: Error on >>> retrieving record #19:47480 (cluster: *redacted*) >>> at >>> com.orientechnologies.orient.core.db.raw.ODatabaseRaw.read(ODatabaseRaw.java:252) >>> at >>> com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.executeReadRecord(ODatabaseRecordAbstract.java:1017) >>> at >>> com.orientechnologies.orient.core.tx.OTransactionNoTx.loadRecord(OTransactionNoTx.java:65) >>> at >>> com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.load(ODatabaseRecordTx.java:251) >>> at >>> com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.load(ODatabaseRecordTx.java:40) >>> at >>> com.orientechnologies.orient.core.db.ODatabaseRecordWrapperAbstract.load(ODatabaseRecordWrapperAbstract.java:227) >>> at >>> com.orientechnologies.orient.core.db.ODatabaseRecordWrapperAbstract.load(ODatabaseRecordWrapperAbstract.java:52) >>> at >>> com.orientechnologies.orient.core.record.impl.ODocument.field(ODocument.java:591) >>> at >>> com.orientechnologies.orient.core.record.impl.ODocument.merge(ODocument.java:972) >>> at >>> com.orientechnologies.orient.core.record.impl.ODocument.merge(ODocument.java:918) >>> at >>> com.orientechnologies.orient.server.network.protocol.binary.OBinaryNetworkProtocolAbstract.updateRecord(OBinaryNetworkProtocolAbstract.java:307) >>> at >>> com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.updateRecord(ONetworkProtocolBinary.java:1343) >>> at >>> com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.executeRequest(ONetworkProtocolBinary.java:309) >>> at >>> com.orientechnologies.orient.server.network.protocol.binary.OBinaryNetworkProtocolAbstract.execute(OBinaryNetworkProtocolAbstract.java:169) >>> at >>> com.orientechnologies.common.thread.OSoftThread.run(OSoftThread.java:45) >>> Caused by: >>> com.orientechnologies.orient.core.exception.ODatabaseException: Error >>> during data decompression. >>> at >>> com.orientechnologies.orient.core.compression.impl.OSnappyCompression.uncompress(OSnappyCompression.java:54) >>> at >>> com.orientechnologies.orient.core.storage.impl.local.paginated.OPaginatedCluster.readRecord(OPaginatedCluster.java:631) >>> at >>> com.orientechnologies.orient.core.storage.impl.local.paginated.OLocalPaginatedStorage.readRecord(OLocalPaginatedStorage.java:1480) >>> at >>> com.orientechnologies.orient.core.storage.impl.local.paginated.OLocalPaginatedStorage.readRecord(OLocalPaginatedStorage.java:745) >>> at >>> com.orientechnologies.orient.core.db.raw.ODatabaseRaw.read(ODatabaseRaw.java:246) >>> ... 14 more >>> Caused by: java.io.IOException: FAILED_TO_UNCOMPRESS(5) >>> at org.xerial.snappy.SnappyNative.throw_error(SnappyNative.java:84) >>> at org.xerial.snappy.SnappyNative.rawUncompress(Native Method) >>> at org.xerial.snappy.Snappy.rawUncompress(Snappy.java:444) >>> at org.xerial.snappy.Snappy.uncompress(Snappy.java:480) >>> at >>> com.orientechnologies.orient.core.compression.impl.OSnappyCompression.uncompress(OSnappyCompression.java:50) >>> ... 18 more >>> 2014-07-01 13:39:19,058 [main ] ERROR >>> TamrResultCsvExportParser - Unknown Exception >>> com.orientechnologies.orient.core.exception.ODatabaseException: Error on >>> retrieving record #19:24451 (cluster: *redacted*) >>> at >>> com.orientechnologies.orient.core.db.raw.ODatabaseRaw.read(ODatabaseRaw.java:252) >>> at >>> com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.executeReadRecord(ODatabaseRecordAbstract.java:1017) >>> at >>> com.orientechnologies.orient.core.tx.OTransactionNoTx.loadRecord(OTransactionNoTx.java:65) >>> at >>> com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.load(ODatabaseRecordTx.java:251) >>> at >>> com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.load(ODatabaseRecordTx.java:40) >>> at >>> com.orientechnologies.orient.core.db.ODatabaseRecordWrapperAbstract.load(ODatabaseRecordWrapperAbstract.java:227) >>> at >>> com.orientechnologies.orient.core.db.ODatabaseRecordWrapperAbstract.load(ODatabaseRecordWrapperAbstract.java:52) >>> at >>> com.orientechnologies.orient.core.record.impl.ODocument.field(ODocument.java:591) >>> at >>> com.orientechnologies.orient.core.record.impl.ODocument.merge(ODocument.java:972) >>> at >>> com.orientechnologies.orient.core.record.impl.ODocument.merge(ODocument.java:918) >>> at >>> com.orientechnologies.orient.server.network.protocol.binary.OBinaryNetworkProtocolAbstract.updateRecord(OBinaryNetworkProtocolAbstract.java:307) >>> at >>> com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.updateRecord(ONetworkProtocolBinary.java:1343) >>> at >>> com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.executeRequest(ONetworkProtocolBinary.java:309) >>> at >>> com.orientechnologies.orient.server.network.protocol.binary.OBinaryNetworkProtocolAbstract.execute(OBinaryNetworkProtocolAbstract.java:169) >>> at >>> com.orientechnologies.common.thread.OSoftThread.run(OSoftThread.java:45) >>> Caused by: >>> com.orientechnologies.orient.core.exception.OStorageException: Content of >>> record #19:24451 was broken. >>> at >>> com.orientechnologies.orient.core.storage.impl.local.paginated.OPaginatedCluster.readFullEntry(OPaginatedCluster.java:656) >>> at >>> com.orientechnologies.orient.core.storage.impl.local.paginated.OPaginatedCluster.readRecord(OPaginatedCluster.java:607) >>> at >>> com.orientechnologies.orient.core.storage.impl.local.paginated.OLocalPaginatedStorage.readRecord(OLocalPaginatedStorage.java:1480) >>> at >>> com.orientechnologies.orient.core.storage.impl.local.paginated.OLocalPaginatedStorage.readRecord(OLocalPaginatedStorage.java:745) >>> at >>> com.orientechnologies.orient.core.db.raw.ODatabaseRaw.read(ODatabaseRaw.java:246) >>> ... 14 more >>> >>> Server after restart: >>> 2014-07-01 13:38:06:756 INFO OrientDB Server v1.7.5-SNAPSHOT is active. >>> [OServer] >>> 2014-07-01 13:38:35:261 WARN segment file 'database.ocf' was not closed >>> correctly last time [OSingleFileSegment] >>> 2014-07-01 13:38:35:330 SEVE Can not restore 0 WAL master record for >>> storage linked crc check is failed [OWriteAheadLog] >>> >>> 2014-07-01 13:38:36:675 WARN Storage linked was not closed properly. >>> Will try to restore from write ahead log. [OLocalPa >>> ginatedStorage] >>> 2014-07-01 13:38:36:675 SEVE Restore is not possible because write ahead >>> log is empty. [OLocalPaginatedStorage] >>> 2014-07-01 13:38:36:691 INFO Storage data restore was completed >>> [OLocalPaginatedStorage] >>> 2014-07-01 13:38:55:688 WARN Page in file lpaperson.pcl with index 16191 >>> was placed in wrong free list, this error will >>> be fixed automatically. [OPaginatedCluster] >>> >>> -- --- You received this message because you are subscribed to the Google Groups "OrientDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
