Hi Pawel, You are right and we know about it, personally I am against of locking strategies in database at all, but on current stage we can not avoid them. We have umbrella issue to rid off many current locks in database https://github.com/orientechnologies/orientdb/issues/1678 .
On Tue, Feb 11, 2014 at 12:11 AM, Pawel K. <[email protected]> wrote: > Hi Andrey, > I tried to imlement similar loader using .NET and binary protocol. > The algorithm is following: > > 1. Read bunch of lines related to the same DbPedia Resource > 2. Identify existing objects (simple SELECT FROM DbResource WHERE > ResourceId IN ['AWK','ResId'.....] > 3. Add lightweigt edge to existing objects > 4. Create new non existing resources and add lightweight edge > 5. Commit all changes in one optimistic commit > > One thread is able to save 8000-10000 items per second. It's quite ok. > Obviously I tried to implement it in multithread scenario. Step 1 is done > in one thread and steps 2-5 in other threads pool using work issued from > Step 1. > Unfortunately performance with two threads is really bad (~270 items per > second). Commits get *com.orientechnologies.common.concur.OTimeoutException. > Timeout on acquiring exclusive lock against resource of class: class > com.orientechnologies.orient.core.index.OIndexNotUnique with timeout=5000.* > Seems that index locking is real bottleneck. I was playing with unique, > hash, not unique but always result was similar. > > Best regards, > Pawel > > > > > > > On Monday, February 10, 2014 1:24:54 PM UTC+1, Pawel K. wrote: >> >> Thanks! I will look at it. >> Pawel >> >> On Monday, February 10, 2014 10:32:48 AM UTC+1, Andrey Lomakin wrote: >>> >>> Hi, >>> Here you are https://github.com/laa/orientdb-wikipedia-benchmark . >>> >>> >>> On Wed, Feb 5, 2014 at 11:16 PM, Andrey Lomakin <[email protected]>wrote: >>> >>>> Hi Pawel, >>>> >>>> I have merged it in develop branch couple of hours ago. >>>> >>>> About your questions. >>>> >>>> 1. I will share tomorrow. >>>> 2. sorry you should use java remote client it is new data structure, we >>>> will publish specification for it so non java users can use it too. >>>> >>>> The main parameter of plocal is amount of RAM for disk cache, so as you >>>> could see db is used 24 GB of RAM so you should set >>>> storage.diskCache.bufferSize >>>> parameter in megabytes. >>>> >>>> >>>> >>>> >>>> On Wed, Feb 5, 2014 at 10:38 PM, Pawel K. <[email protected]> wrote: >>>> >>>>> Andrey, >>>>> >>>>> Can you share benchmark sources? I am trying to repeat your scenario >>>>> and results. >>>>> >>>>> Can I achieve similar performance boost using binary protocol with >>>>> non-Java client? >>>>> >>>>> Best regards, >>>>> Pawel >>>>> >>>>> -- >>>>> >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "OrientDB" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>> >>>> >>>> >>>> >>>> -- >>>> Best regards, >>>> Andrey Lomakin. >>>> >>>> Orient Technologies >>>> the Company behind OrientDB >>>> >>>> >>> >>> >>> -- >>> Best regards, >>> Andrey Lomakin. >>> >>> Orient Technologies >>> the Company behind OrientDB >>> >>> -- > > --- > You received this message because you are subscribed to the Google Groups > "OrientDB" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > -- Best regards, Andrey Lomakin. Orient Technologies the Company behind OrientDB -- --- You received this message because you are subscribed to the Google Groups "OrientDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
