Hey Luca,
After we did the 'upgrade graph' and it finished without any errors or
warnings some documents have invalid relationship content. For example our
acccounts have a 'Friend' relationship, and after the upgrade 1851/254352
records don't have any friends anymore. I picked one of those broken
records for example:
When trying to resolve the friends I get an empty list:
orientdb {upgrade.broken}> select out('Friend') from #12:155580
----+-----+----
# |@RID |out
----+-----+----
0 |#-2:1|[0]
----+-----+----
1 item(s) found. Query executed in 0.098 sec(s).
Where the same query BEFORE the upgrade returned 35 results:
orientdb {before.upgrade}> select out('Friend') from #12:155580
----+-----+----
# |@RID |out
----+-----+----
0 |#-2:1|[35]
----+-----+----
1 item(s) found. Query executed in 0.098 sec(s).
So I checked to see what's wrong and did the following query:
orientdb {upgrade.broken}> select out_Friend from #12:155580
----+-----+-----------------------------------------------------------------------------------------------------------------
# |@RID |out_Friend
----+-----+-----------------------------------------------------------------------------------------------------------------
0
|#-2:1|%AQAAACMAEwAAAAAAATybABMAAAAAAAE9OAATAAAAAAABP7kAEwAAAAAAAUAcABMAAAAAAAFB6QATAAAAAAABRFwAEwAAAAAAAUTPABMAAAAAAAFJ1wATAAAAAAABSqkAEwAAAAAAAU6CABMAAAAAAAFO9gATAAAAAAABUyYAEwAAAAAAAVadABMAAAAAAAFcZgATAAAAAAABXYgAEwAAAAAAAV9bABMAAAAAAAFf2AATAAAAAAABYOgAEwAAAAAAAWnaABMAAAAAAAFulgATAAAAAAABbqcAEwAAAAAAAXFxABMAAAAAAAF1mQATAAAAAAABiIYAEwAAAAAAAYmvABMAAAAAAAGLzAATAAAAAAABlBcAEwAAAAAAAZx0ABMAAAAAAAGktQATAAAAAAABqPkAEwAAAAAAAakTABMAAAAAAAG98wATAAAAAAAB3o4AEwAAAAAAAeyEABMAAAAAAAH9WQ==;
----+-----+-----------------------------------------------------------------------------------------------------------------
1 item(s) found. Query executed in 0.012 sec(s).
This seems to be the base64 encoded RidBag content. How can we fix the
relationships BEFORE the upgrade to prevent such broken records AFTER the
upgrade?
When trying to add a new 'Friend' relation to such broken records I get the
following exception:
java.lang.IllegalStateException: Relationship content is invalid on field
out_Friend. Found:
%AQAAACMAEwAAAAAAATybABMAAAAAAAE9OAATAAAAAAABP7kAEwAAAAAAAUAcABMAAAAAAAFB6QATAAAAAAABRFwAEwAAAAAAAUTPABMAAAAAAAFJ1wATAAAAAAABSqkAEwAAAAAAAU6CABMAAAAAAAFO9gATAAAAAAABUyYAEwAAAAAAAVadABMAAAAAAAFcZgATAAAAAAABXYgAEwAAAAAAAV9bABMAAAAAAAFf2AATAAAAAAABYOgAEwAAAAAAAWnaABMAAAAAAAFulgATAAAAAAABbqcAEwAAAAAAAXFxABMAAAAAAAF1mQATAAAAAAABiIYAEwAAAAAAAYmvABMAAAAAAAGLzAATAAAAAAABlBcAEwAAAAAAAZx0ABMAAAAAAAGktQATAAAAAAABqPkAEwAAAAAAAakTABMAAAAAAAG98wATAAAAAAAB3o4AEwAAAAAAAeyEABMAAAAAAAH9WQ==;
at
com.tinkerpop.blueprints.impls.orient.OrientVertex.createLink(OrientVertex.java:603)
at
com.tinkerpop.blueprints.impls.orient.OrientVertex.addEdge(OrientVertex.java:285)
...
Any hints?
Thanks a million!
Daniel
Am Mittwoch, 23. April 2014 01:34:16 UTC+2 schrieb Lvc@:
>
> Yes, should fix the tx problem.
>
> Lvc@
>
>
> On 22 April 2014 15:19, Daniel Hanelt [STARGAST Systems GmbH] <
> [email protected] <javascript:>> wrote:
>
>> Hey Luca,
>>
>> thanks, we will try it. But it won't help with our distributed
>> transactions, right?
>>
>> Thanks!
>> Daniel
>>
>> Am 22.04.2014 um 12:14 schrieb "Luca Garulli" <[email protected]<javascript:>
>> >:
>>
>> Hi Daniel,
>> to avoid using OMVRBTreeRIDSet you should migrate the graph:
>>
>>
>> https://github.com/orientechnologies/orientdb/wiki/Upgrade#migrate-graph-to-ridbag
>>
>> Lvc@
>>
>>
>> On 21 April 2014 19:58, Daniel Hanelt [STARGAST Systems GmbH] <
>> [email protected] <javascript:>> wrote:
>>
>>> Hey Luca,
>>>
>>> thanks for your reply.
>>>
>>> -We chose writeQuorum=1 because we wanted to have some sort of
>>> asynchronous replication. We just work with node01, node02 is just needed
>>> for handling the backup
>>>
>>> - the db is plocal. Back in time it was created with local but then
>>> with EXPORT exported and imported in a newly created plocal database with
>>> 1.6 (But we were wondering too regarding the logs with OMVRBTreeRIDSet)
>>>
>>> thanks a million
>>> Daniek
>>>
>>> Am 20.04.2014 um 10:36 schrieb "Luca Garulli"
>>> <[email protected]<javascript:>
>>> >:
>>>
>>> Hi guys,
>>> As Mateusz pointed the quorum settings should be to 2 and
>>> failureAvailableNodesLessQuorum
>>> = true to maintain consistency. Then I suggest you to try 1.7-SNAPSHOT
>>> where we fixed a few issues against distributed configuration.
>>>
>>> About your database is it a plocal? The fact your logs show
>>> OMVRBTreeRIDSet
>>> usage let me think this is an old database created with older version of
>>> OrientDB, maybe with local?
>>>
>>> Lvc@
>>>
>>>
>>>
>>> On 17 April 2014 18:10, Andrey Lomakin <[email protected]<javascript:>
>>> > wrote:
>>>
>>>> Hi guys,
>>>> Could you create issue about it so we will not forget to look on it ?
>>>>
>>>>
>>>> On Thu, Apr 17, 2014 at 6:10 PM, Daniel Hanelt [STARGAST Systems GmbH]
>>>> <[email protected] <javascript:>> wrote:
>>>>
>>>>> Hey,
>>>>>
>>>>> we are using the 1.7 RC2 but we've had the same problems with 1.6
>>>>>
>>>>> You are right with the numbers of nodes, three would make more
>>>>> sense. But I doubt that this would solve the problems we've experienced,
>>>>> the exceptions started right away and there was no network issue at that
>>>>> time.
>>>>>
>>>>> Thanks
>>>>> Daniel
>>>>>
>>>>> Am 17.04.2014 um 03:34 schrieb "Mateusz Dymczyk"
>>>>> <[email protected]<javascript:>
>>>>> >:
>>>>>
>>>>> Which version are you using? I remember similar exceptions before
>>>>> switching to 1.7, seems they had some bugs.
>>>>>
>>>>> Also I don't know if Luca suggested exactly that setup but running
>>>>> such a DB with 2 nodes doesn't seem like a great idea - should you have a
>>>>> link (or well any sort of network error) between them or if one of them
>>>>> should go down for some time you will have inconsistent data (since both
>>>>> quorums are set to only 1). An odd number of servers with quorum (N/2)+1
>>>>> is
>>>>> a better idea.
>>>>>
>>>>> Mateusz
>>>>>
>>>>> On Wednesday, April 16, 2014 9:22:31 PM UTC+9, Daniel wrote:
>>>>>>
>>>>>> Because we've had daily outages of 15 minutes due to the backup of
>>>>>> the
>>>>>> DB, we follwed the advice of Luca to set up a distributed environment
>>>>>> with two nodes.
>>>>>>
>>>>>> Here is the config:
>>>>>>
>>>>>>
>>>>>> default-distributed-db-config.json
>>>>>>
>>>>>> {
>>>>>> "replication": true,
>>>>>> "autoDeploy": true,
>>>>>> "hotAlignment": true,
>>>>>> "resyncEvery": 15,
>>>>>> "clusters": {
>>>>>> "internal": {
>>>>>> "replication": false
>>>>>> },
>>>>>> "index": {
>>>>>> "replication": false
>>>>>> },
>>>>>> "ODistributedConflict": {
>>>>>> "replication": false
>>>>>> },
>>>>>> "*": {
>>>>>> "replication": true,
>>>>>> "readQuorum": 1,
>>>>>> "writeQuorum": 1,
>>>>>> "failureAvailableNodesLessQuorum": false,
>>>>>> "readYourWrites": true,
>>>>>> "partitioning": {
>>>>>> "strategy": "round-robin",
>>>>>> "default": 0,
>>>>>> "partitions": [
>>>>>> [ "<NEW_NODE>" ]
>>>>>> ]
>>>>>> }
>>>>>> }
>>>>>> }
>>>>>> }
>>>>>>
>>>>>>
>>>>>> So now here is what happended and finally led to a corrupted DB:
>>>>>>
>>>>>>
>>>>>> - Stopped the application server
>>>>>> - Setup and configured replication with the settings above
>>>>>> - Two nodes: node01 and node02
>>>>>> - node02 had no existing database, so node01 exported and zipped the
>>>>>> db
>>>>>> and sent it to node02
>>>>>> - node02 extracted the db successfully, log message: INFO [node02]
>>>>>> installed database [OHazelcastPlugin]
>>>>>> - We started the application server
>>>>>> - After some time the following exception (sometimes) appeares in the
>>>>>> orient-server.log:
>>>>>>
>>>>>> Cannot route TX operation against distributed node
>>>>>> Error on committing distributed transaction
>>>>>> -> com.orientechnologies.orient.server.distributed.
>>>>>> ODistributedStorage.commit(ODistributedStorage.java:502)
>>>>>> -> com.orientechnologies.orient.core.tx.
>>>>>> OTransactionOptimistic.commit(OTransactionOptimistic.java:109)
>>>>>> -> com.orientechnologies.orient.core.db.record.
>>>>>> ODatabaseRecordTx.commit(ODatabaseRecordTx.java:146)
>>>>>> -> com.orientechnologies.orient.core.db.document.
>>>>>> ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:440)
>>>>>> -> com.orientechnologies.orient.core.db.document.
>>>>>> ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:435)
>>>>>> -> com.orientechnologies.orient.server.network.protocol.
>>>>>> binary.ONetworkProtocolBinary.commit(ONetworkProtocolBinary.
>>>>>> java:1253)
>>>>>> -> com.orientechnologies.orient.server.network.protocol.
>>>>>> binary.ONetworkProtocolBinary.executeRequest(
>>>>>> ONetworkProtocolBinary.java:325)
>>>>>> -> com.orientechnologies.orient.server.network.protocol.binary.
>>>>>> OBinaryNetworkProtocolAbstract.execute(OBinaryNetworkProtocolAbstract
>>>>>> .java:126)
>>>>>> -> com.orientechnologies.common.thread.OSoftThread.run(
>>>>>> OSoftThread.java:45)
>>>>>>
>>>>>>
>>>>>> - After a while the following exception appeared in our application
>>>>>> with increased frequency:
>>>>>>
>>>>>> Caused by:
>>>>>> com.orientechnologies.orient.core.exception.OTransactionException:
>>>>>> Cannot insert item in mvrb-tree because the transactional item was not
>>>>>> found.
>>>>>> at com.orientechnologies.orient.core.type.tree.OMVRBTreeRID.
>>>>>> internalPut(OMVRBTreeRID.java:156)
>>>>>> at com.orientechnologies.orient.core.type.tree.OMVRBTreeRID.
>>>>>> internalPut(OMVRBTreeRID.java:57)
>>>>>> at com.orientechnologies.orient.core.type.tree.
>>>>>> OMVRBTreePersistent.put(OMVRBTreePersistent.java:468)
>>>>>> at com.orientechnologies.orient.core.type.tree.provider.
>>>>>> OMVRBTreeRIDProvider.lazyUnmarshall(OMVRBTreeRIDProvider.java:227)
>>>>>> at com.orientechnologies.orient.core.type.tree.OMVRBTreeRID.
>>>>>> getTreeSize(OMVRBTreeRID.java:332)
>>>>>> at com.orientechnologies.orient.core.type.tree.OMVRBTreeRID.
>>>>>> size(OMVRBTreeRID.java:318)
>>>>>> at com.orientechnologies.orient.core.type.tree.
>>>>>> OMVRBTreeRIDSet.size(OMVRBTreeRIDSet.java:91)
>>>>>> at com.orientechnologies.common.collection.OMultiValue.
>>>>>> getSize(OMultiValue.java:82)
>>>>>> at com.orientechnologies.orient.core.serialization.serializer.
>>>>>> record.string.ORecordSerializerSchemaAware2CSV.toString(
>>>>>> ORecordSerializerSchemaAware2CSV.java:165)
>>>>>> at com.orientechnologies.orient.core.serialization.serializer.
>>>>>> record.string.ORecordSerializerStringAbstract.toStream(
>>>>>> ORecordSerializerStringAbstract.java:92)
>>>>>> at com.orientechnologies.orient.core.serialization.serializer.
>>>>>> record.string.ORecordSerializerSchemaAware2CSV.toStream(
>>>>>> ORecordSerializerSchemaAware2CSV.java:518)
>>>>>> at com.orientechnologies.orient.core.record.
>>>>>> ORecordSchemaAwareAbstract.toStream(ORecordSchemaAwareAbstract.
>>>>>> java:127)
>>>>>> at com.orientechnologies.orient.core.record.
>>>>>> ORecordSchemaAwareAbstract.toStream(ORecordSchemaAwareAbstract.
>>>>>> java:122)
>>>>>> at com.orientechnologies.orient.core.record.impl.ODocument.
>>>>>> toStream(ODocument.java:391)
>>>>>> at com.orientechnologies.orient.client.remote.OStorageRemote.
>>>>>> commitEntry(OStorageRemote.java:1919)
>>>>>> ... 75 more
>>>>>>
>>>>>>
>>>>>> - Then we took a look at the corresponding dataset to those
>>>>>> exception via the orientdb console, i.e.:
>>>>>>
>>>>>> select from #12:155580
>>>>>>
>>>>>> Error:
>>>>>> com.orientechnologies.orient.core.exception.OTransactionException:
>>>>>> Cannot insert item in mvrb-tree because the transactional item was
>>>>>> not found.
>>>>>>
>>>>>>
>>>>>> - Simple properties could be selected without problems, i.e.:
>>>>>>
>>>>>> select email from #12:155580
>>>>>> ----+-----+---------
>>>>>> # |@RID |email
>>>>>> ----+-----+---------
>>>>>> 0 |#-2:1|<removed>
>>>>>> ----+-----+---------
>>>>>>
>>>>>>
>>>>>> - Selections of linked Edges resulted sometimes in errors, i.e.:
>>>>>>
>>>>>> select out_Friend from #12:155580
>>>>>> Error:
>>>>>> com.orientechnologies.orient.core.exception.OTransactionException:
>>>>>> Cannot insert item in mvrb-tree because the transactional item was
>>>>>> not found.
>>>>>>
>>>>>>
>>>>>> - Whereas others worked
>>>>>>
>>>>>> select in_Friend from #12:155580
>>>>>> ----+-----+---------
>>>>>> # |@RID |in_Friend
>>>>>> ----+-----+---------
>>>>>> 0 |#-2:1|[63]
>>>>>> ----+-----+---------
>>>>>> 1 item(s) found. Query executed in 0.004 sec(s).
>>>>>>
>>>>>>
>>>>>> Any idea or hint for us?
>>>>>>
>>>>>> Thanks a million
>>>>>> Daniel
>>>>>>
>>>>> --
>>>>>
>>>>> ---
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "OrientDB" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to [email protected] <javascript:>.
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>>> --
>>>>>
>>>>> ---
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "OrientDB" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to [email protected] <javascript:>.
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Andrey Lomakin.
>>>>
>>>> Orient Technologies
>>>> the Company behind OrientDB
>>>>
>>>> --
>>>>
>>>> ---
>>>> You received this message because you are subscribed to the Google
>>>> Groups "OrientDB" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to [email protected] <javascript:>.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>> --
>>>
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "OrientDB" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected] <javascript:>.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>> --
>>>
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "OrientDB" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected] <javascript:>.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "OrientDB" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <javascript:>.
>> For more options, visit https://groups.google.com/d/optout.
>>
>> --
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "OrientDB" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <javascript:>.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
---
You received this message because you are subscribed to the Google Groups
"OrientDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
