Hi Pinaki, You are right... I will change the code according to your comment. Thanks for your review and input. Will repost the patch once the regression test is done.
Teresa On Jan 17, 2008 10:07 AM, Pinaki Poddar <[EMAIL PROTECTED]> wrote: > Hi, > Having a get/setBatchLimit() in DBDictionary does not exclude > OracleDictionary or any other specific databases to set their respective > default limits. The specifc ones just call super.setBatchLimit(xyz) in > their consturctors. If they do not then whatever value is set by > DBDictionary becomes their default. That is the pattern used for any other > DBDictionary property and I am not being able to fathom why batch limit > property has to be different. > > Whichever way you may want to interpret the numeric values of the batch > limit (e.g. 0 means no batching, -1 means unlimited batch size or > whatever), put that logic in validateBatchProcess(). The point is there is > no need to hold separate flag BATCH_LIMIT_NOT_SET -- it can all be > interpreted by the numeric value of the batch limit set as default or > explictly by the user. > > > -----Original Message----- > From: Teresa Kan [mailto:[EMAIL PROTECTED] > Sent: Thursday, January 17, 2008 9:47 AM > To: [email protected] > Subject: Re: [jira] Commented: (OPENJPA-464) Performance improvement with > Statement Batching support > > Hi Pinaki, > We want to let each db dictionary to make the decision on the batch limit > because it gives more flexibility to each db. In addition, we considered > user's "out of box" experience. In stead of requiring user to set the batch > limit in the property explicitly, by default, DB2 and Oracle have set the > batch limit to 100. Unless user does not satisfy the limit, no need to add > this property in the xml file. > I don't think we can use the check ((if (getBatchLimit()<-1) return > false;) as your suggested because 0 is > -1 and it is known as disabled the > batch. > Otherwise, how can we interpret the meaning of 0? > > Thanks, > Teresa > > On Jan 16, 2008 4:48 PM, Pinaki Poddar (JIRA) <[EMAIL PROTECTED]> wrote: > > > > > [ > > https://issues.apache.org/jira/browse/OPENJPA-464?page=com.atlassian.j > > ira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=125 > > 59710#action_12559710] > > > > Pinaki Poddar commented on OPENJPA-464: > > --------------------------------------- > > > > I read your original posting again to see your usage of batch limit. > > Why not have a single batch limit bean-style property defined in > > DBDictionary? > > Also in > > DBDictionary.validateBatchProcess() { > > if (getBatchLimit()<-1) return false; // other auto-assign logic } > > > > 1. You will not require to define batch limit for individual database > > dictionaries 2. the concept of batch_limit_unset is not required. Just > > interpret any number less that -1 as an indication of switching off > > batching. Interpret -1 as maximum. > > > > > Performance improvement with Statement Batching support > > > ------------------------------------------------------- > > > > > > Key: OPENJPA-464 > > > URL: https://issues.apache.org/jira/browse/OPENJPA-464 > > > Project: OpenJPA > > > Issue Type: Bug > > > Components: kernel > > > Affects Versions: 1.1.0 > > > Reporter: Teresa Kan > > > Fix For: 1.1.0 > > > > > > Attachments: OPENJPA-464.patch, OPENJPA-464.patch2, > > > statement > > batch design_1211.doc > > > > > > > > > The current OpenJPA implementation did not provide the SQL statement > > batching. All SQL statements will be executed one statement at a time > > to the database. Consequently, the runtime performance was decreased > > due to lots of database flows. JDBC Specification provides the batch > > capability for insert, update and delete statements through the > addBatch() and executeBatch() APIs. > > We should be able to take advantage of this capability to support SQL > > statement batching in OpenJPA. > > > According to the old version of the OpenJPA manual (i.e., Kodo), > > statement batching was part of the initial functions. Conscious > > decision by BEA that this function was not contributed back to > > OpenJPA. We can still use this info as the implementation base with some > modifications. > > > I have completed the work for this statement batching support and > > > the > > patch has been tested by CTS against Derby and DB2, OPENJPA regression > > test as well as our internal FVT test bucket. The following section > > describes the design and implementation info. I also attached the > > whole design documentation and the patch in this jira. Once the design > > and implementation are accepted, then I will update the OPENJPA manual > > to include this function. Thanks, > > > Design and implementation: > > > * Configuration: > > > o Batch Limit value: > > > § 0 - Disable batch support. > > > § -1 - Unlimited number of statements for a batch. > > > § Any positive number - Maximum number of statements for a batch. > > > o By default, the batch support is based on each Dictionary to > > define the default batch limit. Currently only DB2 and Oracle > > dictionaries are set the default batch limit to 100. The default batch > > limit for rest of the dictionaries is set to zero (disabled). > > > o To enable the batch support, user can specify the following > > property in the persistence.xml file: > > > <property name="openjpa.jdbc.DBDictionary" value="BatchLimit=25"/> > > > or <property name="openjpa.jdbc.DBDictionary" > > > value="db2(batchLimit=25)"/> > > > * Basic design is to cache all the insert/update/delete statements > > during the execution of the > > PreparedStatementManagerImpl.flushInternal() > > method. There is a cache structure which uses the LinkHashMap to > > maintain the order of the SQL statements for execution: > > > o _cacheSql - a LinkHashMap to store the rows that associate with > > one PrepareStatement. Key: SQL statement string; Value: array list of > rows. > > > During the PreparedStatementManagerImpl.flush() process, it will go > > through the cache to prepare the SQL statement; add the statement to > > the batch; and execute the batch when the batch limit is reached or > > all the rows are processed for that statement. Validate the update > > count after the > > executeBatch() method. > > > * If the batch limit =0 (disabled), execute the statement as the > > normal process; no need to use the batching process. Same rule > > applies to the statement that only has one row, execute it as the normal > process. > > > * The batch process will be disabled if the primary key generation > > is used the Identity strategy. When the GeneratedType=IDENTITY, we > > need to get the ID value right away for the in-memory entity to use. > > Therefore, we can't batch this kind of statement. > > > * Batch exception process: a checkUpdateCount() is used to > validate > > the batch process after the executeBatch(). According to the javadoc, > > there are three cases to consider: > > > o Case of EXECUTE_FAILED: (-3): > > > § This is a failure case. If the action is UPDATE or there is > > FailedObject, treats it as OptimisticException. Otherwise, throws the > > SQLException. > > > § This is the same process as current implementation. > > > o Case of SUCCESS_NO_INFO: (-2): > > > § We treat this as successful case and log the info in the log. > > > o Case of 0: > > > § If there is a FailedObject or the action is INSERT, then throws > > the SQLException. Otherwise, treats it as successful case. > > > > -- > > This message is automatically generated by JIRA. > > - > > You can reply to this email to add a comment to the issue online. > > > > > > Notice: This email message, together with any attachments, may contain > information of BEA Systems, Inc., its subsidiaries and affiliated > entities, that may be confidential, proprietary, copyrighted and/or > legally privileged, and is intended solely for the use of the individual or > entity named in this message. If you are not the intended recipient, and > have received this message in error, please immediately return this by email > and then delete it. >
