GlAccountOrganization looks like it could be contentious for locks due to the postedBalance field (lots of separate long running transactions trying to update multiple rows at the same time). The service updating it probably needs refactoring so that the update is moved to a smaller transaction that executes after the main one has committed successfully.
Regards Scott On 17/05/2014, at 3:46 pm, Adrian Crum <adrian.c...@sandglass-software.com> wrote: > Deadlock issues are hard to reproduce because they are usually timing > dependent. If your issue is easily reproducible, then most likely it is > caused by an action (SECA) trying to change data that is being iterated over > by another action (the service invoking the SECA). > > Look through the stack trace and see if there is a situation where the > EntityListIterator is being used, and during iteration another service tries > to change the table being iterated over. If yes, then the EntityListIterator > needs to be converted to a java.util.List. > > Adrian Crum > Sandglass Software > www.sandglass-software.com > > On 5/7/2014 11:47 AM, S K Pradeep Kumar wrote: >> Hi All, >> >> What might be the possible reason for this issue : >> >> This issue starts only when we have the separate servers for Database and >> Application. but this issue is not coming on the single server. >> >> We have deployed the application on Amazon EC2 servers. >> >> Application is deployed on : EC2 Medium instance with 3.75 GB RAM >> MySql Database is on : EC2 Medium instance with 3.75 GB RAM >> >> 2014-05-07 19:36:54,593 (TP-Processor107) [ CallService.java:333:ERROR] >> ---- exception report >> ---------------------------------------------------------- Exception: >> java.lang.Exception Message: Error: ERROR: Could not complete the Update >> GlAccountOrganization >> [file:/applications/accounting/script/org/ofbiz/accounting/ledger/GeneralLedgerServices.xml#updateGlAccountOrganization] >> process [problem storing the lookedUpValue value: Error while updating: >> [GenericEntity:GlAccountOrganization][createdStamp,2013-08-19 >> 12:32:25.0(java.sql.Timestamp)][createdTxStamp,2013-08-19 >> 12:32:25.0(java.sql.Timestamp)][fromDate,2001-01-01 >> 00:00:00.0(java.sql.Timestamp)][glAccountId,214000(java.lang.String)][lastUpdatedStamp,2014-05-07 >> 19:36:54.579(java.sql.Timestamp)][lastUpdatedTxStamp,2014-05-07 >> 19:36:54.334(java.sql.Timestamp)][organizationPartyId,Company(java.lang.String)][postedBalance,701133.69(java.math.BigDecimal)][roleTypeId,null()][thruDate,null()] >> (SQL Exception while executing the following:UPDATE GL_ACCOUNT_ORGANIZATION >> SET ROLE_TYPE_ID=?, FROM_DATE=?, THRU_DATE=?, POSTED_BALANCE=?, >> LAST_UPDATED_STAMP=?, LAST_UPDATED_TX_STAMP=?, CREATED_STAMP=?, >> CREATED_TX_STAMP=? WHERE GL_ACCOUNT_ID=? AND ORGANIZATION_PARTY_ID=? >> (Deadlock found when trying to get lock; try restarting >> transaction))] calling service updateGlAccountOrganization in >> postAcctgTrans ---- stack trace >> --------------------------------------------------------------- >> java.lang.Exception: Error: ERROR: Could not complete the Update >> GlAccountOrganization >> [file:/applications/accounting/script/org/ofbiz/accounting/ledger/GeneralLedgerServices.xml#updateGlAccountOrganization] >> process [problem storing the lookedUpValue value: Error while updating: >> [GenericEntity:GlAccountOrganization][createdStamp,2013-08-19 >> 12:32:25.0(java.sql.Timestamp)][createdTxStamp,2013-08-19 >> 12:32:25.0(java.sql.Timestamp)][fromDate,2001-01-01 >> 00:00:00.0(java.sql.Timestamp)][glAccountId,214000(java.lang.String)][lastUpdatedStamp,2014-05-07 >> 19:36:54.579(java.sql.Timestamp)][lastUpdatedTxStamp,2014-05-07 >> 19:36:54.334(java.sql.Timestamp)][organizationPartyId,Company(java.lang.String)][postedBalance,701133.69(java.math.BigDecimal)][roleTypeId,null()][thruDate,null()] >> (SQL Exception while executing the following:UPDATE GL_ACCOUNT_ORGANIZATION >> SET ROLE_TYPE_ID=?, FROM_DATE=?, THRU_DATE=?, POSTED_BALANCE=?, >> LAST_UPDATED_STAMP=?, LAST_UPDATED_TX_STAMP=?, CREATED_STAMP=?, >> CREATED_TX_STAMP=? WHERE GL_ACCOUNT_ID=? AND ORGANIZATION_PARTY_ID=? >> (Deadlock found when trying to get lock; try restarting >> transaction))] calling service updateGlAccountOrganization in >> postAcctgTrans >> org.ofbiz.minilang.method.callops.CallService.exec(CallService.java:333) >> org.ofbiz.minilang.SimpleMethod.runSubOps(SimpleMethod.java:920) >> org.ofbiz.minilang.method.envops.Iterate.exec(Iterate.java:118) >> org.ofbiz.minilang.SimpleMethod.runSubOps(SimpleMethod.java:920) >> org.ofbiz.minilang.method.ifops.IfCompare.exec(IfCompare.java:135) >> org.ofbiz.minilang.SimpleMethod.runSubOps(SimpleMethod.java:920) >> org.ofbiz.minilang.SimpleMethod.exec(SimpleMethod.java:739) >> org.ofbiz.minilang.SimpleMethod.runSimpleMethod(SimpleMethod.java:160) >> org.ofbiz.minilang.SimpleMethod.runSimpleService(SimpleMethod.java:142) >> org.ofbiz.minilang.SimpleServiceEngine.serviceInvoker(SimpleServiceEngine.java:78) >> org.ofbiz.minilang.SimpleServiceEngine.runSync(SimpleServiceEngine.java:53) >> org.ofbiz.service.ModelServiceReader$GenericInvokerImpl.runSync(ModelServiceReader.java:761) >> _$gen.file_58$.youmart.youmart_95$v3.trunk.applications.accounting.servicedef.services_95$ledger_46$xml_35$postAcctgTrans.runSync(file:/youmart/youmart_v3/trunk/applications/accounting/servicedef/services_ledger.xml#postAcctgTrans:283) >> org.ofbiz.service.ServiceDispatcher.runSync(ServiceDispatcher.java:399) >> org.ofbiz.service.ServiceDispatcher.runSync(ServiceDispatcher.java:226) >> org.ofbiz.service.GenericDispatcher.runSync(GenericDispatcher.java:163) >> org.ofbiz.service.eca.ServiceEcaAction.runAction(ServiceEcaAction.java:138) >> org.ofbiz.service.eca.ServiceEcaRule.eval(ServiceEcaRule.java:151) >> org.ofbiz.service.eca.ServiceEcaUtil.evalRules(ServiceEcaUtil.java:162) >> org.ofbiz.service.ServiceDispatcher.runSync(ServiceDispatcher.java:500) >> org.ofbiz.service.ServiceDispatcher.runSync(ServiceDispatcher.java:226) >> org.ofbiz.service.GenericDispatcher.runSync(GenericDispatcher.java:177) >> org.ofbiz.minilang.method.callops.CallService.exec(CallService.java:247) >> org.ofbiz.minilang.SimpleMethod.runSubOps(SimpleMethod.java:920) >> org.ofbiz.minilang.SimpleMethod.exec(SimpleMethod.java:739) >> org.ofbiz.minilang.SimpleMethod.runSimpleMethod(SimpleMethod.java:160) >> org.ofbiz.minilang.SimpleMethod.runSimpleService(SimpleMethod.java:142) >> org.ofbiz.minilang.SimpleServiceEngine.serviceInvoker(SimpleServiceEngine.java:78) >> org.ofbiz.minilang.SimpleServiceEngine.runSync(SimpleServiceEngine.java:53) >> org.ofbiz.service.ModelServiceReader$GenericInvokerImpl.runSync(ModelServiceReader.java:761) >> _$gen.file_58$.youmart.youmart_95$v3.trunk.applications.accounting.servicedef.services_95$ledger_46$xml_35$createAcctgTransForShipmentReceipt.runSync(file:/youmart/youmart_v3/trunk/applications/accounting/servicedef/services_ledger.xml#createAcctgTransForShipmentReceipt:357) >> org.ofbiz.service.ServiceDispatcher.runSync(ServiceDispatcher.java:399) >> org.ofbiz.service.ServiceDispatcher.runSync(ServiceDispatcher.java:226) >> org.ofbiz.service.GenericDispatcher.runSync(GenericDispatcher.java:163) >> org.ofbiz.service.eca.ServiceEcaAction.runAction(ServiceEcaAction.java:138) >> org.ofbiz.service.eca.ServiceEcaRule.eval(ServiceEcaRule.java:151) >> org.ofbiz.service.eca.ServiceEcaUtil.evalRules(ServiceEcaUtil.java:162) >> org.ofbiz.service.ServiceDispatcher.runSync(ServiceDispatcher.java:500) >> org.ofbiz.service.ServiceDispatcher.runSync(ServiceDispatcher.java:226) >> org.ofbiz.service.GenericDispatcher.runSync(GenericDispatcher.java:177) >> org.ofbiz.minilang.method.callops.CallService.exec(CallService.java:247) >> org.ofbiz.minilang.SimpleMethod.runSubOps(SimpleMethod.java:920) >> org.ofbiz.minilang.method.envops.Loop.exec(Loop.java:82) >> org.ofbiz.minilang.SimpleMethod.runSubOps(SimpleMethod.java:920) >> org.ofbiz.minilang.SimpleMethod.exec(SimpleMethod.java:739) >> org.ofbiz.minilang.SimpleMethod.runSimpleMethod(SimpleMethod.java:160) >> org.ofbiz.minilang.SimpleMethod.runSimpleService(SimpleMethod.java:142) >> org.ofbiz.minilang.SimpleServiceEngine.serviceInvoker(SimpleServiceEngine.java:78) >> org.ofbiz.minilang.SimpleServiceEngine.runSync(SimpleServiceEngine.java:53) >> org.ofbiz.service.ModelServiceReader$GenericInvokerImpl.runSync(ModelServiceReader.java:761) >> _$gen.file_58$.youmart.youmart_95$v3.trunk.applications.product.servicedef.services_95$shipment_46$xml_35$receiveInventoryProduct.runSync(file:/youmart/youmart_v3/trunk/applications/product/servicedef/services_shipment.xml#receiveInventoryProduct:775) >> org.ofbiz.service.ServiceDispatcher.runSync(ServiceDispatcher.java:399) >> org.ofbiz.service.ServiceDispatcher.runSync(ServiceDispatcher.java:226) >> org.ofbiz.service.GenericDispatcher.runSync(GenericDispatcher.java:163) >> org.ofbiz.order.shoppinglist.ShoppingListEvents.dailyUpload(ShoppingListEvents.java:1423) >> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> java.lang.reflect.Method.invoke(Method.java:597) >> org.ofbiz.webapp.event.JavaEventHandler.invoke(JavaEventHandler.java:92) >> org.ofbiz.webapp.event.JavaEventHandler.invoke(JavaEventHandler.java:78) >> org.ofbiz.webapp.control.RequestHandler.runEvent(RequestHandler.java:642) >> org.ofbiz.webapp.control.RequestHandler.doRequest(RequestHandler.java:388) >> org.ofbiz.webapp.control.ControlServlet.doGet(ControlServlet.java:226) >> org.ofbiz.webapp.control.ControlServlet.doPost(ControlServlet.java:89) >> javax.servlet.http.HttpServlet.service(HttpServlet.java:637) >> javax.servlet.http.HttpServlet.service(HttpServlet.java:717) >> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) >> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) >> org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:339) >> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) >> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) >> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) >> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) >> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) >> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) >> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) >> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:554) >> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) >> org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190) >> org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291) >> org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:776) >> org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:705) >> org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:898) >> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690) >> java.lang.Thread.run(Thread.java:619) >> -------------------------------------------------------------------------------- >> >> >> >> With regards, >> S K Pradeep Kumar, >> 9035009495 >>
