[ https://issues.apache.org/jira/browse/OOZIE-3717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17751281#comment-17751281 ]
Hadoop QA commented on OOZIE-3717: ---------------------------------- Testing JIRA OOZIE-3717 Cleaning local git workspace ---------------------------- {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:green}+1 RAW_PATCH_ANALYSIS{color} . {color:green}+1{color} the patch does not introduce any @author tags . {color:green}+1{color} the patch does not introduce any tabs . {color:green}+1{color} the patch does not introduce any trailing spaces . {color:green}+1{color} the patch does not introduce any star imports . {color:green}+1{color} the patch does not introduce any line longer than 132 . {color:green}+1{color} the patch adds/modifies 1 testcase(s) {color:green}+1 RAT{color} . {color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} . {color:green}+1{color} Javadoc generation succeeded with the patch . {color:green}+1{color} the patch does not seem to introduce new Javadoc warning(s) {color:green}+1 COMPILE{color} . {color:green}+1{color} HEAD compiles . {color:green}+1{color} patch compiles . {color:green}+1{color} the patch does not seem to introduce new javac warnings {color:red}-1{color} There are [3] new bugs found below threshold in total that must be fixed. . {color:green}+1{color} There are no new bugs found in [examples]. . {color:green}+1{color} There are no new bugs found in [fluent-job/fluent-job-api]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [sharelib/git]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:green}+1{color} There are no new bugs found in [server]. . {color:green}+1{color} There are no new bugs found in [docs]. . {color:green}+1{color} There are no new bugs found in [webapp]. . {color:red}-1{color} There are [3] new bugs found below threshold in [core] that must be fixed. . You can find the SpotBugs diff here (look for the red and orange ones): core/findbugs-new.html . The most important SpotBugs errors are: . At BulkJPAExecutor.java:[line 206]: This use of javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query; can be vulnerable to SQL/JPQL injection . At BulkJPAExecutor.java:[line 176]: At BulkJPAExecutor.java:[line 175] . At BulkJPAExecutor.java:[line 205]: At BulkJPAExecutor.java:[line 199] . Unsafe comparison of hash that are susceptible to timing attack: At BulkJPAExecutor.java:[line 206] . At ShareLibService.java:[line 689]: At ShareLibService.java:[line 695] . {color:green}+1{color} There are no new bugs found in [tools]. . {color:green}+1{color} There are no new bugs found in [client]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} . {color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations . {color:green}+1{color} the patch does not modify JPA files {color:green}+1 TESTS{color} . Tests run: 3262 . {color:orange}Tests failed at first run:{color} TestCoordActionInputCheckXCommand#testCoordActionInputCheckXCommandUniqueness . For the complete list of flaky tests, see TEST-SUMMARY-FULL files. {color:green}+1 DISTRO{color} . {color:green}+1{color} distro tarball builds with the patch {color:green}+1 MODERNIZER{color} ---------------------------- {color:red}*-1 Overall result, please check the reported -1(s)*{color} The full output of the test-patch run is available at . https://ci-hadoop.apache.org/job/PreCommit-OOZIE-Build/213/ > When fork actions parallel submit, becasue ForkedActionStartXCommand and > ActionStartXCommand has the same name, so ForkedActionStartXCommand would be > lost, and cause deadlock > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > > Key: OOZIE-3717 > URL: https://issues.apache.org/jira/browse/OOZIE-3717 > Project: Oozie > Issue Type: Bug > Components: action > Affects Versions: 5.2.1 > Reporter: chenhaodan > Assignee: chenhaodan > Priority: Major > Attachments: OOZIE-3717-001.patch, OOZIE-3717-002.patch, > OOZIE-3717-003.patch > > > when fork actions parallel submit will add ForkedActionStartXCommand and > RecoveryService will check pending action may add ActionStartXCommand, if > ForkedActionStartXCommand enqueue and there is a ActionStartXCommand(the same > action) in queue, it would be lose. The thread parallel submit actions block > at CallableQueueService.blockingWait() wait for ForkedActionStartXCommand to > finish, but ForkedActionStartXCommand had lost and cause deadlock. > {code:java} > Thread 1 Thread 2 > (ForkedActionStartXCommand) (ActionStartXCommand) > +----------------------------+ +---------+ > | removeFromUniqueCallables | | ..... | > +----------------------------+ +---------+ > | ...... | | queue | > +----------------------------+ +---------+ > | queue | enqueue successed, in uniqueCallables > +----------------------------+ > | wrapper.filterDuplicates() | > +----------------------------+ > Thread 1 and Thread 2 execute CallableWrapper's execute function order like: > 1. Thread 1 execute removeFromUniqueCallables; > 2. Thread 2 execute queue add ActionStartXCommand into queue and add to > uniqueCallables; > 3. Thread 1 execute queue add ForkedActionStartXCommand into queue, but > filterDuplicates() function found a same name XCommand in uniqueCallables, so > skip add to queue; > Becasue ForkedActionStartXCommand and ActionStartXCommand has the same name, > Thread 2 add ActionStartXCommand enqueue before Thread 1, so > ForkedActionStartXCommand would be lost(never execute), and the thread that > fork actions parallel submit block at CallableQueueService.blockingWait(). > {code} > > *CallableWrapper's code* > {code:java} > public class CallableWrapper<E> extends PriorityDelayQueue.QueueElement<E> > implements Runnable, Callable<E> { > private Instrumentation.Cron cron; > public void run() { > XCallable<?> callable = null; > try { > removeFromUniqueCallables(); > if (Services.get().getSystemMode() == SYSTEM_MODE.SAFEMODE) { > log.info("Oozie is in SAFEMODE, requeuing callable [{0}] with > [{1}]ms delay", getElement().getType(), > SAFE_MODE_DELAY); > setDelay(SAFE_MODE_DELAY, TimeUnit.MILLISECONDS); > queue(this, true); > return; > } > callable = getElement(); > if (callableBegin(callable)) { > cron.stop(); > addInQueueCron(cron); > XLog log = XLog.getLog(getClass()); > log.trace("executing callable [{0}]", callable.getName()); > try { > //FutureTask.run() will invoke cllable.call() > super.run(); > incrCounter(INSTR_EXECUTED_COUNTER, 1); > log.trace("executed callable [{0}]", callable.getName()); > } > catch (Exception ex) { > incrCounter(INSTR_FAILED_COUNTER, 1); > log.warn("exception callable [{0}], {1}", > callable.getName(), ex.getMessage(), ex); > } > } > else { > log.warn("max concurrency for callable [{0}] exceeded, > requeueing with [{1}]ms delay", callable > .getType(), CONCURRENCY_DELAY); > setDelay(CONCURRENCY_DELAY, TimeUnit.MILLISECONDS); > queue(this, true); > incrCounter(callable.getType() + "#exceeded.concurrency", 1); > } > } > catch (Throwable t) { > incrCounter(INSTR_FAILED_COUNTER, 1); > log.warn("exception callable [{0}], {1}", callable == null ? > "N/A" : callable.getName(), > t.getMessage(), t); > } > finally { > if (callable != null) { > callableEnd(callable); > } > } > } > } > {code} > > -- This message was sent by Atlassian Jira (v8.20.10#820010)