----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74566/#review225694 -----------------------------------------------------------
security-admin/src/main/java/org/apache/ranger/service/RangerTransactionService.java Lines 115 (patched) <https://reviews.apache.org/r/74566/#comment314146> Abhay - I guess I misread your comment. Details of the failure are pinted in #117 here, as shown in the following exception logged. Are you suggesting to add more details in the log? ``` 2023-08-27 08:30:52,649 [pool-5-thread-1] ERROR [RangerTransactionService.java:117] Failed to commit TransactionService transaction javax.persistence.PersistenceException: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.7.12.v20230209-e5c4074ef3): org.eclipse.persistence.exceptions.DatabaseException Internal Exception: com.mysql.cj.jdbc.exceptions.MySQLTransactionRollbackException: Lock wait timeout exceeded; try restarting transaction Error Code: 1205 Call: UPDATE x_plugin_info SET info = ?, UPDATE_TIME = ? WHERE (id = ?) bind => [3 parameters bound] at org.eclipse.persistence.internal.jpa.EntityManagerImpl.flush(EntityManagerImpl.java:980) at sun.reflect.GeneratedMethodAccessor49.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.orm.jpa.SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(SharedEntityManagerCreator.java:315) at com.sun.proxy.$Proxy33.flush(Unknown Source) at org.apache.ranger.common.db.BaseDao.update(BaseDao.java:162) at org.apache.ranger.db.XXPluginInfoDao.update(XXPluginInfoDao.java:50) at org.apache.ranger.biz.AssetMgr.doCreateOrUpdateXXPluginInfo(AssetMgr.java:1030) at org.apache.ranger.biz.AssetMgr.access$000(AssetMgr.java:76) at org.apache.ranger.biz.AssetMgr$6.run(AssetMgr.java:829) at org.apache.ranger.service.RangerTransactionService$1$1.doInTransaction(RangerTransactionService.java:110) at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140) at org.apache.ranger.service.RangerTransactionService$1.run(RangerTransactionService.java:108) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) ``` - Madhan Neethiraj On Aug. 25, 2023, 8:27 p.m., Madhan Neethiraj wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74566/ > ----------------------------------------------------------- > > (Updated Aug. 25, 2023, 8:27 p.m.) > > > Review request for ranger, Abhay Kulkarni, Monika Kachhadiya, Pradeep > Agrawal, Ramesh Mani, Subhrat Chaudhary, and Velmurugan Periasamy. > > > Bugs: RANGER-4375 > https://issues.apache.org/jira/browse/RANGER-4375 > > > Repository: ranger > > > Description > ------- > > - when ranger.plugin.activity.log.not.modified=false, x_plugin_info record > update is skipped if no changes are downloaded to the plugin > - updated RangerTransactionService to support threadpool size via > configuration > - updated RangerTransactionService to periodically log status of number of > pending tasks > > > Diffs > ----- > > security-admin/src/main/java/org/apache/ranger/biz/AssetMgr.java 1dc3d372d > > security-admin/src/main/java/org/apache/ranger/service/RangerTransactionService.java > 49d07fcd0 > > > Diff: https://reviews.apache.org/r/74566/diff/3/ > > > Testing > ------- > > - verified that update of x_plugin_info record is skipped when no changes are > downloaded to the plugin > - verified following stats logs: > ``` > 2023-08-25 23:18:47,008 [http-nio-6080-exec-1] INFO > [RangerTransactionService.java:156] RangerTransactionService: > tasks(scheduled=1, executed=0, failed=0, pending=1) > 2023-08-25 23:19:47,009 [http-nio-6080-exec-7] INFO > [RangerTransactionService.java:156] RangerTransactionService: > tasks(scheduled=16413, executed=16411, failed=0, pending=2) > 2023-08-25 23:20:47,014 [http-nio-6080-exec-5] INFO > [RangerTransactionService.java:156] RangerTransactionService: > tasks(scheduled=31327, executed=31326, failed=0, pending=1) > 2023-08-25 23:21:47,031 [http-nio-6080-exec-10] INFO > [RangerTransactionService.java:156] RangerTransactionService: > tasks(scheduled=45031, executed=45030, failed=0, pending=1) > 2023-08-25 23:22:47,034 [http-nio-6080-exec-4] INFO > [RangerTransactionService.java:156] RangerTransactionService: > tasks(scheduled=61300, executed=49132, failed=0, pending=12168) > 2023-08-25 23:23:47,064 [http-nio-6080-exec-11] INFO > [RangerTransactionService.java:156] RangerTransactionService: > tasks(scheduled=75351, executed=49134, failed=2, pending=26217) > 2023-08-25 23:24:47,162 [http-nio-6080-exec-2] INFO > [RangerTransactionService.java:156] RangerTransactionService: > tasks(scheduled=92338, executed=49136, failed=3, pending=43202) > 2023-08-25 23:25:47,163 [pool-5-thread-1] INFO > [RangerTransactionService.java:156] RangerTransactionService: > tasks(scheduled=105358, executed=98645, failed=3, pending=6713) > 2023-08-25 23:26:47,192 [http-nio-6080-exec-6] INFO > [RangerTransactionService.java:156] RangerTransactionService: > tasks(scheduled=120623, executed=120622, failed=3, pending=1) > 2023-08-25 23:27:47,193 [http-nio-6080-exec-29] INFO > [RangerTransactionService.java:156] RangerTransactionService: > tasks(scheduled=137638, executed=137637, failed=3, pending=1) > 2023-08-25 23:28:47,194 [http-nio-6080-exec-4] INFO > [RangerTransactionService.java:156] RangerTransactionService: > tasks(scheduled=149134, executed=149133, failed=3, pending=1) > 2023-08-25 23:29:47,196 [http-nio-6080-exec-20] INFO > [RangerTransactionService.java:156] RangerTransactionService: > tasks(scheduled=166921, executed=166920, failed=3, pending=1) > 2023-08-25 23:30:47,222 [http-nio-6080-exec-10] INFO > [RangerTransactionService.java:156] RangerTransactionService: > tasks(scheduled=181683, executed=181682, failed=3, pending=1) > 2023-08-25 23:31:47,228 [http-nio-6080-exec-13] INFO > [RangerTransactionService.java:156] RangerTransactionService: > tasks(scheduled=198283, executed=198282, failed=3, pending=1) > ``` > > > Thanks, > > Madhan Neethiraj > >
