[
https://issues.apache.org/jira/browse/OOZIE-636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13194294#comment-13194294
]
[email protected] commented on OOZIE-636:
-----------------------------------------------------
bq. On 2012-01-26 23:22:06, Virag Kothari wrote:
bq. >
trunk/core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java,
line 52
bq. > <https://reviews.apache.org/r/3486/diff/3/?file=70926#file70926line52>
bq. >
bq. > I understand that this change is not related to the patch.
bq. > But changed to public as constants usually should not be kept private
bq.
bq. Alejandro Abdelnur wrote:
bq. currently they are not needed as public, nobody else uses it. as a
rule, expose as little as possible, this is not necessary to expose
bq.
bq. Virag Kothari wrote:
bq. Found an interesting article on usage of "public/private static final"
bq.
http://geekwhorled.blogspot.com/2004/07/simple-java-questions-1-private-static.html
interesting point of view. Still, if it is not need outside of the class, it
should private, the less interface you expose the better, specially if it is
not used.
- Alejandro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3486/#review4636
-----------------------------------------------------------
On 2012-01-26 22:42:27, Virag Kothari wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3486/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-01-26 22:42:27)
bq.
bq.
bq. Review request for oozie, Mohammad Islam and Angelo K. Huang.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Validate fork and join at wf submission time
bq. https://issues.apache.org/jira/browse/OOZIE-636
bq.
bq. Brief description of algo:
bq.
bq. A modified dfs algorithm is used. Two stacks, one for dfs traversal and
other for maintaining fork join status, are kept. When a fork is encountered
during traversal, it is added to the forkjoin stack and number of paths
associated with the fork is also stored. When a node’s child is seen as a
join, the join is added to the forkJoin stack and the no. of paths to it is
updated. When the number of paths for fork and join are equal, then the
fork/join pair is removed from the forkJoin stack and join is pushed to the
dfsStack.
bq.
bq. Nodes other than fork and join are only pushed to the dfs stack.
bq. If a action node is seen, only the node's "ok-to" transition is considered
bq.
bq.
bq. While(!stack.isEmpty()){
bq. Node n = DfsStack.pop()
bq. n.traversed = true;
bq. If(n.type==fork){
bq. ForkJoinStack.push(new Element(n, n.paths) );
bq. }
bq. List<Node> childs = getUnvisitedChildnodes(n)
bq. For(Node n: childs){
bq. If (n.type==join){
bq. Boolean b=isForkJoinCleared(ForkJoinStack)
bq. If(!b){
bq. Continue;
bq. }
bq. stack.push(n);
bq. n.traversed = true;
bq. }
bq. }
bq.
bq.
bq. This addresses bug OOZIE-636.
bq. https://issues.apache.org/jira/browse/OOZIE-636
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/core/src/test/resources/wf-schema-valid.xml 1236413
bq.
trunk/core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java
1236413
bq.
trunk/core/src/test/java/org/apache/oozie/service/TestLiteWorkflowAppService.java
1236413
bq.
trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
1236413
bq. trunk/core/src/main/java/org/apache/oozie/ErrorCode.java 1236413
bq.
bq. Diff: https://reviews.apache.org/r/3486/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Test case to validate fork-join added
bq.
bq.
bq. Thanks,
bq.
bq. Virag
bq.
bq.
> Check fork and join in the workflow in the submission time
> -----------------------------------------------------------
>
> Key: OOZIE-636
> URL: https://issues.apache.org/jira/browse/OOZIE-636
> Project: Oozie
> Issue Type: Bug
> Reporter: Virag Kothari
>
> Enhancement: Oozie should check that the fork node and join node are correct
> in pair when user submits the job. This should be a static check, not when
> the workflow is running.
> Current logic bug:
> A workflow with different number of forks and joins was run. The wf job
> should have been killed but it succeeded. Also, strangely, the action was
> killed.
> Following are the different types of tests run and their results with varying
> delays.
> test1: wf job SUCCEEDED, action java12 KILLED.
> delay11=11
> delay12=12
> delay121=1
> delay122=2
> delay21=1
> delay22=1
> test2: wf job SUCCEEDED, action java12 KILLED.
> delay11=1
> delay12=12
> delay121=1
> delay122=2
> delay21=1
> delay22=1
> test3: wf job SUCCEEED, all actions OK. question: why wf job always pass in
> this scenario, even when fork-join not in
> pair?
> delay11=10
> delay12=10
> delay121=15
> delay122=15
> delay21=20
> delay22=20
> workflow.xml
> ============
> <workflow-app xmlns='uri:oozie:workflow:0.1' name='fork-join-4735180-wf'>
> <start to='fork1' />
> <fork name="fork1">
> <path start="java11" />
> <path start="fork12" />
> </fork>
> <action name='java11'>
> <java>
> <job-tracker>${jobTracker}</job-tracker>
> <name-node>${nameNode}</name-node>
> <configuration>
> <property>
> <name>mapred.job.queue.name</name>
> <value>${queueName}</value>
> </property>
> </configuration>
> <main-class>qa.test.tests.testsleep</main-class>
> <arg>${delay11}</arg>
> </java>
> <ok to="java12" />
> <error to="fail" />
> </action>
> <action name='java12'>
> <java>
> <job-tracker>${jobTracker}</job-tracker>
> <name-node>${nameNode}</name-node>
> <configuration>
> <property>
> <name>mapred.job.queue.name</name>
> <value>${queueName}</value>
> </property>
> </configuration>
> <main-class>qa.test.tests.testsleep</main-class>
> <arg>${delay12}</arg>
> </java>
> <ok to="join1" />
> <error to="fail" />
> </action>
> <fork name="fork12">
> <path start="java121" />
> <path start="java122" />
> </fork>
> <action name='java121'>
> <java>
> <job-tracker>${jobTracker}</job-tracker>
> <name-node>${nameNode}</name-node>
> <configuration>
> <property>
> <name>mapred.job.queue.name</name>
> <value>${queueName}</value>
> </property>
> </configuration>
> <main-class>qa.test.tests.testsleep</main-class>
> <arg>${delay121}</arg>
> </java>
> <ok to="join12" />
> <error to="fail" />
> </action>
> <action name='java122'>
> <java>
> <job-tracker>${jobTracker}</job-tracker>
> <name-node>${nameNode}</name-node>
> <configuration>
> <property>
> <name>mapred.job.queue.name</name>
> <value>${queueName}</value>
> </property>
> </configuration>
> <main-class>qa.test.tests.testsleep</main-class>
> <arg>${delay122}</arg>
> </java>
> <ok to="join12" />
> <error to="fail" />
> </action>
> <join name="join12" to="fork2" />
> <fork name="fork2">
> <path start="java21" />
> <path start="java22" />
> </fork>
> <action name='java21'>
> <java>
> <job-tracker>${jobTracker}</job-tracker>
> <name-node>${nameNode}</name-node>
> <configuration>
> <property>
> <name>mapred.job.queue.name</name>
> <value>${queueName}</value>
> </property>
> </configuration>
> <main-class>qa.test.tests.testsleep</main-class>
> <arg>${delay21}</arg>
> </java>
> <ok to="join1" />
> <error to="fail" />
> </action>
> <action name='java22'>
> <java>
> <job-tracker>${jobTracker}</job-tracker>
> <name-node>${nameNode}</name-node>
> <configuration>
> <property>
> <name>mapred.job.queue.name</name>
> <value>${queueName}</value>
> </property>
> </configuration>
> <main-class>qa.test.tests.testsleep</main-class>
> <arg>${delay22}</arg>
> </java>
> <ok to="join1" />
> <error to="fail" />
> </action>
> <join name="join1" to="end" />
> <kill name="fail">
> <message>Streaming Map/Reduce failed, error
> message[${wf:errorMessage(wf:lastErrorNode())}]</message>
> </kill>
> <end name='end' />
> </workflow-app>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
