Hi all, As far as I know, FOR UPDATE would only break SQLite, and I can live with that.
Gareth, would optimistic instead of pessimistic work for your situation? The database does less of the work and it's more scalable. I haven't tested it out, but OFBiz provides optimistic locking if you turn on enable-lock. Cheers Paul Foxworthy On 27 October 2017 at 03:49, Jacques Le Roux <[email protected]> wrote: > I agree with Scott and Taher, > > About your question Gareth, we once had a hard coded "for update" > somewhere in OFBiz code (in SequenceUtil.SequenceBank.fillBank() ). > I put it in. Then Jacopo crossed an issue in a cluster and rightly changed > it to something better > See https://issues.apache.org/jira/browse/OFBIZ-2353 for more where > Adrian gave some directions also. > > Since I did not read all in the thread and there is no patch yet I don't > know if something alike applies to EntityQuery API. Putting a for update is > certainly much simpler, so just saying for now. > > Jacques > > > > Le 26/10/2017 à 10:46, Gareth Carter a écrit : > >> HI all >> >> Thank for the response >> >> It could be considered a hint, if the underlying db does not support it >> then don't use it. Adding an attribute on the datasource could be the way >> to go but I also believe we need methods to tell the system when to use it, >> you don't need it all the time only in selective cases. >> >> Until support is added I see ofbiz as purely RDMS. I don't believe ofbiz >> should restrict itself on the assumption it might support different >> databases such as nosql. >> >> There could be an alternative to forUpdate, anyone considered >> implementing a distributed locking mechanism? The whole point for using >> forUpdate is to prevent one or more code blocks changing the same database >> records concurrently, db row locks help but are not supported by all >> databases but an external locking mechanism may provide a generic way of >> dealing with this. OFBiz already provides a semaphore on services but I >> believe you need something at entity level >> Gareth Carter >> Software Development Analyst >> Stannah Management Services Ltd >> IT Department >> Ext: >> 7036 >> DDI: >> 01264 364311 >> >> >> Please consider the environment before printing this email. >> >> -----Original Message----- >> From: Taher Alkhateeb [mailto:[email protected]] >> Sent: 26 October 2017 6:13 AM >> To: OFBIZ Development Mailing List <[email protected]> >> Subject: Re: Updates to EntityQuery >> >> Hmmm usually my first gut reaction would be "keep things consistent", >> however Scott made a good argument for supporting non-common DB features. >> So my suggestion would be to use something like the widget's >> <platform-specific ...> tag (or equivalent in code) for any features not >> supported in all DBMSs. By labeling it as such the user is aware that the >> DB selection is restricted. >> >> Now with regards to your comment about NoSQL, such databases are >> fundamentally different, and act more like a key-value based entities >> without a schema or the resulting ACID properties. It is possible to make >> it work with the entity engine, but then you'd lose the flexibility it >> gives you because you want to restrict yourself to the relational model >> that we apply in OFBiz. The idea of NoSQL is to think at a different >> conceptual level of your domain model. >> >> On Thu, Oct 26, 2017 at 4:33 AM, Scott Gray <[email protected]> >> wrote: >> >>> I think it's better to have for-update available for use for the >>> majority of use-cases than it is to exclude it altogether. I'd rather >>> lose the databases that don't support it than constrain the ones that >>> do and are used most by pretty much everyone. It would be nice to >>> hear from anyone using a database that doesn't support it since I'm >>> just guessing most of us use postgres or mysql. >>> >>> Has anyone actually looked at how NoSQL databases might work with the >>> OFBiz entity engine or are the concepts too far removed from SQL for >>> it to even be workable? >>> >>> If we did want to support for-update globally, we could always have >>> the delegator simulate it by issuing an UPDATE to lock the rows prior >>> to performing the SELECT. Although it might not work perfectly in all >>> cases, it could be a good compromise. e.g. >>> UPDATE table SET lastUpdatedTxStamp = NOW() WHERE ...conditions -- >>> lock the rows SELECT * FROM table WHERE ...conditions -- select the >>> rows >>> >>> Regards >>> Scott >>> >>> >>> >>> On 26 October 2017 at 13:45, Paul Foxworthy <[email protected]> wrote: >>> >>> Hi Gareth, >>>> >>>> FOR UPDATE is standard SQL, but even so SQLite for one doesn't support >>>> it: >>>> >>>> http://sqlite.1065341.n5.nabble.com/SELECT-FOR-UPDATE-td89630.html >>>> https://sqlite.org/lang_select.html >>>> >>>> FOR UPDATE only makes sense for relational databases. Adding >>>> assumptions it's available would make it harder to use a NoSQL database >>>> in future. >>>> >>>> In the case of SQLite, in effect it's got serialized isolation >>>> anyway, so the FOR UPDATE is not strictly necessary. >>>> >>>> I suggest OFBiz shouldn't have a method that generates syntactically >>>> incorrect SQL for any database we want to support, and we claim to >>>> support several. Where there are variations between SQL dialects, we >>>> have attributes in the data source, such as "join-style". >>>> >>>> So think our choices are: >>>> >>>> 1. Add an attribute to say "SELECT ... FOR UPDATE" is supported by a >>>> data source. OFBiz would only add the "FOR UPDATE" if it is >>>> supported. In the case of SQLite, I suspect we'd get away without it, >>>> but there may be other databases where the absence of "FOR UPDATE" >>>> might bring on unexpected behaviour. >>>> >>>> 2. Don't provide a method to expose FOR UPDATE. >>>> >>>> Could you achieve the results you want with a higher transaction >>>> isolation level like repeatable read, or with optimistic locking >>>> (enable-lock)? >>>> >>>> What do you think? >>>> >>>> Cheers >>>> >>>> Paul Foxworthy >>>> >>>> >>>> On 26 October 2017 at 07:50, Gareth Carter >>>> <[email protected]> >>>> wrote: >>>> >>>> Ok I will create a Jira with patch for review. >>>>> >>>>> >>>>> >>>>> Sent from my Samsung Galaxy smartphone. >>>>> >>>>> >>>>> >>>>> Gareth Carter >>>>> >>>>> >>>>> Software Development Analyst >>>>> >>>>> >>>>> Stannah Management Services Ltd >>>>> >>>>> >>>>> IT Department >>>>> >>>>> >>>>> Ext: >>>>> >>>>> >>>>> 7036 >>>>> >>>>> >>>>> DDI: >>>>> >>>>> >>>>> 01264 364311 >>>>> >>>>> >>>>> >>>>> >>>>> [http://logos.stannah.co.uk/stan150.jpg] >>>>> >>>>> >>>>> [http://logos.stannah.co.uk/enviro.jpg]Please consider the >>>>> environment before printing this email. >>>>> >>>>> >>>>> -------- Original message -------- >>>>> From: deepak nigam <[email protected]> >>>>> Date: 25/10/2017 14:06 (GMT+00:00) >>>>> To: [email protected] >>>>> Subject: Re: Updates to EntityQuery >>>>> >>>>> Looking good to me also. Let me know if I can be of any help. >>>>> >>>>> >>>>> Thanks & Regards >>>>> -- >>>>> Deepak Nigam >>>>> >>>>> On Wed, Oct 25, 2017 at 6:26 PM, Yash Sharma < >>>>> [email protected]> >>>>> wrote: >>>>> >>>>> I am all up for it, please let me know if I could help. >>>>>> >>>>>> Thanks & Regards, >>>>>> -- >>>>>> *Pradhan Yash Sharma* >>>>>> *HotWax Systems* | >>>>>> www.hotwaxsystems.com<http://www.hotwaxsystems.com> >>>>>> >>>>>> On Wed, Oct 25, 2017 at 6:23 PM, Yash Sharma < >>>>>> [email protected]> >>>>>> wrote: >>>>>> >>>>>> Yes, Usage of the stream will surely enhance performance to a >>>>>>> certain extent and removes ceremony from the code base. I think >>>>>>> Parallel >>>>>>> >>>>>> streams >>>>> >>>>>> will add enhancements in many folds as we are using a multicore >>>>>>> >>>>>> processor >>>>> >>>>>> (I have not tested yet), but the Functional approach seems promising. >>>>>>> >>>>>>> Thanks & Regards, >>>>>>> -- >>>>>>> *Pradhan Yash Sharma* >>>>>>> *HotWax Systems* | www.hotwaxsystems.com<http:// >>>>>>> >>>>>> www.hotwaxsystems.com> >>>> >>>>> On Wed, Oct 25, 2017 at 5:59 PM, Gareth Carter < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>> forUpdate >>>>>>>> >>>>>>>> We patched EntityFindOptions with a new field "forUpdate" with >>>>>>>> >>>>>>> shorthand >>>>> >>>>>> methods in EntityQuery to enable. We then made a change to >>>>>>>> GenericDAO.selectListIteratorByCondition to add "FOR UPDATE" >>>>>>>> on the >>>>>>>> >>>>>>> end >>>>> >>>>>> of the SQL select statement - this allows for DB row locks (we >>>>>>>> use >>>>>>>> >>>>>>> postgres >>>>>> >>>>>>> and works but have not tested other databases). I believe >>>>>>>> there may >>>>>>>> >>>>>>> have >>>>> >>>>>> been a discussion about this before >>>>>>>> >>>>>>>> >>>>>>>> forEach on EntityQuery >>>>>>>> >>>>>>>> Use Consumer in java and groovy to iterate over a query. This >>>>>>>> can >>>>>>>> >>>>>>> reduce >>>>> >>>>>> memory consumption (replacement for queryList()) and boiler >>>>>>>> plate >>>>>>>> >>>>>>> code >>>> >>>>> (eg >>>>>> >>>>>>> queryIterator(), while loop and close) >>>>>>>> >>>>>>>> Example: >>>>>>>> >>>>>>>> EntityQuery.use(delegator).from("Foobar") >>>>>>>> .forEach(item -> >>>>>>>> Debug.logInfo(item.toString(), module); ); >>>>>>>> >>>>>>>> A further update could be to provide stream capabilities >>>>>>>> >>>>>>>> Hope this helps, I can provide a patch aswell >>>>>>>> >>>>>>>> Gareth Carter >>>>>>>> Software Development Analyst >>>>>>>> Stannah Management Services Ltd IT Department >>>>>>>> Ext: >>>>>>>> 7036 >>>>>>>> DDI: >>>>>>>> 01264 364311 >>>>>>>> >>>>>>>> >>>>>>>> Please consider the environment before printing this email. >>>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Arun Patidar [mailto:[email protected]] >>>>>>>> Sent: 25 October 2017 5:49 AM >>>>>>>> To: [email protected] >>>>>>>> Cc: [email protected] >>>>>>>> Subject: Re: Updates to EntityQuery >>>>>>>> >>>>>>>> Hello Gareth, >>>>>>>> >>>>>>>> Please provide some more details or patch to understand - forUpdate >>>>>>>> >>>>>>> and >>>>> >>>>>> forEach method utility. >>>>>>>> >>>>>>>> getFieldMap method looks good to me. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Thanks & Regards >>>>>>>> --- >>>>>>>> Arun Patidar >>>>>>>> Manager, Enterprise Software Development >>>>>>>> >>>>>>>> HotWax Systems Pvt Ltd. >>>>>>>> >>>>>>>> www.hotwaxsystems.com<http://www.hotwaxsystems.com> >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter < >>>>>>>> [email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>> Hi all >>>>>>>>> >>>>>>>>> We have internally patched EntityQuery with some additional >>>>>>>>> functionality and before I create a Jira was going to see what the >>>>>>>>> >>>>>>>> community thinks. >>>>>>>> >>>>>>>>> New functionality: >>>>>>>>> >>>>>>>>> - New method getFieldMap which returns a Map object of >>>>>>>>> >>>>>>>> selected >>>>> >>>>>> fields from GenericValue objects, useful for creating cache map >>>>>>>>> objects for lookup >>>>>>>>> >>>>>>>>> - Support forUpdate >>>>>>>>> >>>>>>>>> - forEach method to accept Consumer >>>>>>>>> >>>>>>>>> We have found these useful and believe the project can benefit, >>>>>>>>> >>>>>>>> let >>>> >>>>> me >>>>> >>>>>> know what you think >>>>>>>>> >>>>>>>>> Regards >>>>>>>>> >>>>>>>>> Gareth Carter >>>>>>>>> >>>>>>>>> >>>>>>>>> Software Development Analyst >>>>>>>>> >>>>>>>>> >>>>>>>>> Stannah Management Services Ltd >>>>>>>>> >>>>>>>>> >>>>>>>>> IT Department >>>>>>>>> >>>>>>>>> >>>>>>>>> Ext: >>>>>>>>> >>>>>>>>> >>>>>>>>> 7036 >>>>>>>>> >>>>>>>>> >>>>>>>>> DDI: >>>>>>>>> >>>>>>>>> >>>>>>>>> 01264 364311 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> [http://logos.stannah.co.uk/stan150.jpg] >>>>>>>>> >>>>>>>>> >>>>>>>>> [http://logos.stannah.co.uk/enviro.jpg]Please consider the >>>>>>>>> >>>>>>>> environment >>>>>> >>>>>>> before printing this email. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> This email is intended only for the above addressee. It may >>>>>>>>> >>>>>>>> contain >>>> >>>>> privileged information. If you are not the addressee you must not >>>>>>>>> copy, distribute, disclose or use any of the information in it. If >>>>>>>>> >>>>>>>> you >>>>> >>>>>> have received it in error, please delete it and notify the sender. >>>>>>>>> >>>>>>>>> Stannah Lift Holdings Ltd registered No. 686996, Stannah >>>>>>>>> >>>>>>>> Management >>>> >>>>> Services Ltd registered No. 2483693, Stannah Lift Services Ltd >>>>>>>>> registered No. 1189799, Stannah Microlifts Ltd registered No. >>>>>>>>> >>>>>>>> 964804, >>>>> >>>>>> Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd >>>>>>>>> registered No. 1401451, Global Upholstery Solutions Ltd registered >>>>>>>>> >>>>>>>> No. >>>>> >>>>>> 02452728. >>>>>>>> >>>>>>>>> All registered offices at Watt Close, East Portway, Andover, >>>>>>>>> Hampshire, >>>>>>>>> SP10 3SD, England. >>>>>>>>> >>>>>>>>> All Registered in England and Wales. >>>>>>>>> >>>>>>>>> This message has been scanned for malware by Websense. >>>>>>>>> www.websense.com<http://www.websense.com> >>>>>>>>> >>>>>>>>> >>>>>>>> To report this email as spam, please send the original message, >>>>>>>> >>>>>>> complete >>>>> >>>>>> with headers to [email protected] >>>>>>>> >>>>>>>> >>>>>>> >>>> >>>> -- >>>> Coherent Software Australia Pty Ltd >>>> PO Box 2773 >>>> Cheltenham Vic 3192 >>>> Australia >>>> >>>> Phone: +61 3 9585 6788 >>>> Web: http://webdefence.global.blackspider.com/urlwrap/?q=AXicFYxN >>>> CsIwEEa_E02K-NPqRlDs2t5gKJGIaVIm0wYPVnDr0rt4COP28d5Di8MHWN6A >>>> -OeqZkoy08B338egEj31ccDcXDfn7tJV63q3bdCyWHV0YlErx6QcArsi0vSA >>>> Ux33xuScC3BWbNAUb5pL8l8RTwbA9wX8ANMdKWg&Z >>>> Email: [email protected] >>>> >>>> > -- Coherent Software Australia Pty Ltd PO Box 2773 Cheltenham Vic 3192 Australia Phone: +61 3 9585 6788 Web: http://www.coherentsoftware.com.au/ Email: [email protected]
![http://logos.stannah.co.uk/enviro.jpg]Please](http://logos.stannah.co.uk/enviro.jpg]Please){kind=link}