Actually I think that multicore support will give us 2-3 times speed up because I bet you work on PC with 4 cores at least )). So multi core scalability for us not less important than single thread performance.
On Tue, Feb 11, 2014 at 9:12 AM, Andrey Lomakin <[email protected]>wrote: > Hi Pawel, > You are right and we know about it, personally I am against of locking > strategies in database at all, but on current stage we can not avoid them. > We have umbrella issue to rid off many current locks in database > https://github.com/orientechnologies/orientdb/issues/1678 . > > > On Tue, Feb 11, 2014 at 12:11 AM, Pawel K. <[email protected]> wrote: > >> Hi Andrey, >> I tried to imlement similar loader using .NET and binary protocol. >> The algorithm is following: >> >> 1. Read bunch of lines related to the same DbPedia Resource >> 2. Identify existing objects (simple SELECT FROM DbResource WHERE >> ResourceId IN ['AWK','ResId'.....] >> 3. Add lightweigt edge to existing objects >> 4. Create new non existing resources and add lightweight edge >> 5. Commit all changes in one optimistic commit >> >> One thread is able to save 8000-10000 items per second. It's quite ok. >> Obviously I tried to implement it in multithread scenario. Step 1 is done >> in one thread and steps 2-5 in other threads pool using work issued from >> Step 1. >> Unfortunately performance with two threads is really bad (~270 items per >> second). Commits get *com.orientechnologies.common.concur.OTimeoutException. >> Timeout on acquiring exclusive lock against resource of class: class >> com.orientechnologies.orient.core.index.OIndexNotUnique with timeout=5000.* >> Seems that index locking is real bottleneck. I was playing with unique, >> hash, not unique but always result was similar. >> >> Best regards, >> Pawel >> >> >> >> >> >> >> On Monday, February 10, 2014 1:24:54 PM UTC+1, Pawel K. wrote: >>> >>> Thanks! I will look at it. >>> Pawel >>> >>> On Monday, February 10, 2014 10:32:48 AM UTC+1, Andrey Lomakin wrote: >>>> >>>> Hi, >>>> Here you are https://github.com/laa/orientdb-wikipedia-benchmark . >>>> >>>> >>>> On Wed, Feb 5, 2014 at 11:16 PM, Andrey Lomakin >>>> <[email protected]>wrote: >>>> >>>>> Hi Pawel, >>>>> >>>>> I have merged it in develop branch couple of hours ago. >>>>> >>>>> About your questions. >>>>> >>>>> 1. I will share tomorrow. >>>>> 2. sorry you should use java remote client it is new data structure, >>>>> we will publish specification for it so non java users can use it too. >>>>> >>>>> The main parameter of plocal is amount of RAM for disk cache, so as >>>>> you could see db is used 24 GB of RAM so you should set >>>>> storage.diskCache.bufferSize >>>>> parameter in megabytes. >>>>> >>>>> >>>>> >>>>> >>>>> On Wed, Feb 5, 2014 at 10:38 PM, Pawel K. <[email protected]> wrote: >>>>> >>>>>> Andrey, >>>>>> >>>>>> Can you share benchmark sources? I am trying to repeat your scenario >>>>>> and results. >>>>>> >>>>>> Can I achieve similar performance boost using binary protocol with >>>>>> non-Java client? >>>>>> >>>>>> Best regards, >>>>>> Pawel >>>>>> >>>>>> -- >>>>>> >>>>>> --- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "OrientDB" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Best regards, >>>>> Andrey Lomakin. >>>>> >>>>> Orient Technologies >>>>> the Company behind OrientDB >>>>> >>>>> >>>> >>>> >>>> -- >>>> Best regards, >>>> Andrey Lomakin. >>>> >>>> Orient Technologies >>>> the Company behind OrientDB >>>> >>>> -- >> >> --- >> You received this message because you are subscribed to the Google Groups >> "OrientDB" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/groups/opt_out. >> > > > > -- > Best regards, > Andrey Lomakin. > > Orient Technologies > the Company behind OrientDB > > -- Best regards, Andrey Lomakin. Orient Technologies the Company behind OrientDB -- --- You received this message because you are subscribed to the Google Groups "OrientDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
