I have limited the transactions to the smallest granularity and it still failing... I even tried to remove transactions, but I don't know before hand the classes of my model and without transactions OrientDB complains that class XPTO doesn't exist prior inserting. How would you do that?
As I have an XML, each node name can be a class and each attribute can become a property of the class. In particular, there are tags that represent relations between two elements. In this case it can be turned into edges. My XML files are about 40MB, they aren't that big. I have 2GB of ram dedicated to OrientDB and more 4GB dedicated to my XML processing API. And my enviromnent is an xlarge instance of Amazon AWS. On Monday, March 30, 2015 at 12:23:17 PM UTC-3, Chris Wilper wrote: > > Just another data point, I did some testing of large transactions on the > graph API a while back and also ran into memory issues, but they went away > when I reduced transaction size, and called commit periodically. Have you > tried just committing the transaction periodically to rule out the > possibility of an OOM coming from some other source? Your point about the > wanting to do everything atomically makes sense, but there does seem to be > a practical limit w/orient as currently designed. I can't remember exactly > where the limit was in my testing, but it seemed reasonable for my > application (thousands of operations vs millions). > > On Mon, Mar 30, 2015 at 10:52 AM, Fábio Costa <[email protected] > <javascript:>> wrote: > >> I forgot to mention that the documentation regarding graph API and tuning >> is outdated on the OrientDB's website. >> >> >> On Monday, March 30, 2015 at 10:52:23 AM UTC-3, Fábio Costa wrote: >>> >>> ==> I'm a newbie in OrientDB :D >>> >>> Is there any documentation regarding transactions, performance tuning >>> and the Java API for graphs? The closest I got were some github issues, as >>> it is a kind of bug. Some suggested to limit the number of records within >>> the same commit. But it seems absurd to me, because this kind of action can >>> potentially break the ACID properties. Actually I have, lets say, 30 XML >>> files - big but not huge - that I'm interpreting to do import to OrientDB. >>> I could just convert them to JSON, but I do some parsing to improve the >>> design of the data, like creating edges. >>> I got OutOfMemory many times. First, I created an index to my natural >>> id, which improved a lot the overall performance, but now I'm facing the >>> outofmemory coming from commit operations. >>> What's the proper way to delimit transactions? Another point is that I'm >>> using the embedded mode(plocal). It's not necessary though, since I >>> installed OrientDB in the same server. Is OrientDB really more powerful >>> when running on server? >>> It would be very useful to create a more complex or real world tutorial >>> and post on the website. I could even do so if I had the knowlege. >>> Thanks in advance! >>> >> -- >> >> --- >> You received this message because you are subscribed to the Google Groups >> "OrientDB" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > > -- --- You received this message because you are subscribed to the Google Groups "OrientDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
