[jira] [Commented] (OOZIE-3139) Oozie validates workflow incorrectly
[
https://issues.apache.org/jira/browse/OOZIE-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16283434#comment-16283434
]
Peter Bacsko commented on OOZIE-3139:
-
+1
thanks for the contribution [~dionusos], committed to master!
> Oozie validates workflow incorrectly
>
>
> Key: OOZIE-3139
> URL: https://issues.apache.org/jira/browse/OOZIE-3139
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.3.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Fix For: 5.0.0b1
>
> Attachments: OOZIE-3139.001.patch, OOZIE-3139.002.patch,
> OOZIE-3139.003.patch, OOZIE-3139.004.patch, OOZIE-3139.005.patch,
> exception.log, workflow.pdf
>
>
> We found that after OOZIE-1978 the attached workflow becomes invalid. Aa we
> found out the following line brings the problem:
> https://github.com/apache/oozie/commit/8e9b9042b3270dc5ff975c44a5c977fcc41250e4#diff-a38ae78886b7c3afb711d32cfb4dbbbdR267
> . The parameter "okPath" changed from okPath to true.
> The following workflow is problematic:
> {noformat}
>
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> Shell action failed, error
> message[${wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-3139) Oozie validates workflow incorrectly
[
https://issues.apache.org/jira/browse/OOZIE-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16283418#comment-16283418
]
Peter Bacsko commented on OOZIE-3139:
-
[~dionusos] I'll check the latest patch then +1 and commit it if it looks good.
> Oozie validates workflow incorrectly
>
>
> Key: OOZIE-3139
> URL: https://issues.apache.org/jira/browse/OOZIE-3139
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.3.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Attachments: OOZIE-3139.001.patch, OOZIE-3139.002.patch,
> OOZIE-3139.003.patch, OOZIE-3139.004.patch, OOZIE-3139.005.patch,
> exception.log, workflow.pdf
>
>
> We found that after OOZIE-1978 the attached workflow becomes invalid. Aa we
> found out the following line brings the problem:
> https://github.com/apache/oozie/commit/8e9b9042b3270dc5ff975c44a5c977fcc41250e4#diff-a38ae78886b7c3afb711d32cfb4dbbbdR267
> . The parameter "okPath" changed from okPath to true.
> The following workflow is problematic:
> {noformat}
>
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> Shell action failed, error
> message[${wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-3139) Oozie validates workflow incorrectly
[
https://issues.apache.org/jira/browse/OOZIE-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16283416#comment-16283416
]
Denes Bodo commented on OOZIE-3139:
---
Does anybody have any comments? Is it okay to merge? Change is on review board,
also.
> Oozie validates workflow incorrectly
>
>
> Key: OOZIE-3139
> URL: https://issues.apache.org/jira/browse/OOZIE-3139
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.3.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Attachments: OOZIE-3139.001.patch, OOZIE-3139.002.patch,
> OOZIE-3139.003.patch, OOZIE-3139.004.patch, OOZIE-3139.005.patch,
> exception.log, workflow.pdf
>
>
> We found that after OOZIE-1978 the attached workflow becomes invalid. Aa we
> found out the following line brings the problem:
> https://github.com/apache/oozie/commit/8e9b9042b3270dc5ff975c44a5c977fcc41250e4#diff-a38ae78886b7c3afb711d32cfb4dbbbdR267
> . The parameter "okPath" changed from okPath to true.
> The following workflow is problematic:
> {noformat}
>
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> Shell action failed, error
> message[${wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-3139) Oozie validates workflow incorrectly
[
https://issues.apache.org/jira/browse/OOZIE-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281763#comment-16281763
]
Hadoop QA commented on OOZIE-3139:
--
Testing JIRA OOZIE-3139
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 3 testcase(s)
{color:red}-1 RAT{color}
.{color:red}-1{color} the patch seems to introduce 2 new RAT warning(s)
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2077
.Tests rerun: 60
.Tests failed at first run:
org.apache.oozie.action.hadoop.TestJavaActionExecutor,org.apache.oozie.service.TestCoordMaterializeTriggerService,
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/277/
> Oozie validates workflow incorrectly
>
>
> Key: OOZIE-3139
> URL: https://issues.apache.org/jira/browse/OOZIE-3139
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.3.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Attachments: OOZIE-3139.001.patch, OOZIE-3139.002.patch,
> OOZIE-3139.003.patch, OOZIE-3139.004.patch, exception.log, workflow.pdf
>
>
> We found that after OOZIE-1978 the attached workflow becomes invalid. Aa we
> found out the following line brings the problem:
> https://github.com/apache/oozie/commit/8e9b9042b3270dc5ff975c44a5c977fcc41250e4#diff-a38ae78886b7c3afb711d32cfb4dbbbdR267
> . The parameter "okPath" changed from okPath to true.
> The following workflow is problematic:
> {noformat}
>
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
[jira] [Commented] (OOZIE-3139) Oozie validates workflow incorrectly
[
https://issues.apache.org/jira/browse/OOZIE-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280108#comment-16280108
]
Peter Bacsko commented on OOZIE-3139:
-
Sorry I my suggestion above was incorrect - you're not building the WF from
objects, instead it's loaded from an XML, so it doesn't work.
However, I'd like to ask one final (minor) change in the two new tests:
{noformat}
LiteWorkflowApp app =
parser.validateAndParse(IOUtils.getResourceAsReader(
"wf-multiple-error-parent.xml", -1), new Configuration());
{noformat]
The {{app}} variable is unused, therefore it generates a warning. Just drop the
variable declaration and ignore the return value of "parser".
> Oozie validates workflow incorrectly
>
>
> Key: OOZIE-3139
> URL: https://issues.apache.org/jira/browse/OOZIE-3139
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.3.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Attachments: OOZIE-3139.001.patch, OOZIE-3139.002.patch,
> OOZIE-3139.003.patch, exception.log, workflow.pdf
>
>
> We found that after OOZIE-1978 the attached workflow becomes invalid. Aa we
> found out the following line brings the problem:
> https://github.com/apache/oozie/commit/8e9b9042b3270dc5ff975c44a5c977fcc41250e4#diff-a38ae78886b7c3afb711d32cfb4dbbbdR267
> . The parameter "okPath" changed from okPath to true.
> The following workflow is problematic:
> {noformat}
>
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> Shell action failed, error
> message[${wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-3139) Oozie validates workflow incorrectly
[
https://issues.apache.org/jira/browse/OOZIE-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279670#comment-16279670
]
Abhishek Bafna commented on OOZIE-3139:
---
[~dionusos] Looks good to me. +1.
Can we also include the below scenario. Thanks.
{code}
kill
{code}
> Oozie validates workflow incorrectly
>
>
> Key: OOZIE-3139
> URL: https://issues.apache.org/jira/browse/OOZIE-3139
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.3.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Attachments: OOZIE-3139.001.patch, OOZIE-3139.002.patch,
> exception.log, workflow.pdf
>
>
> We found that after OOZIE-1978 the attached workflow becomes invalid. Aa we
> found out the following line brings the problem:
> https://github.com/apache/oozie/commit/8e9b9042b3270dc5ff975c44a5c977fcc41250e4#diff-a38ae78886b7c3afb711d32cfb4dbbbdR267
> . The parameter "okPath" changed from okPath to true.
> The following workflow is problematic:
> {noformat}
>
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> Shell action failed, error
> message[${wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-3139) Oozie validates workflow incorrectly
[
https://issues.apache.org/jira/browse/OOZIE-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278841#comment-16278841
]
Denes Bodo commented on OOZIE-3139:
---
[~pbacsko] thanks for your quick response. I've also uploaded the patch to
https://reviews.apache.org/r/64343/
Also great thanks for [~csomaati] finding this wrong parametrization.
> Oozie validates workflow incorrectly
>
>
> Key: OOZIE-3139
> URL: https://issues.apache.org/jira/browse/OOZIE-3139
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.3.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Attachments: OOZIE-3139.001.patch, OOZIE-3139.002.patch,
> exception.log, workflow.pdf
>
>
> We found that after OOZIE-1978 the attached workflow becomes invalid. Aa we
> found out the following line brings the problem:
> https://github.com/apache/oozie/commit/8e9b9042b3270dc5ff975c44a5c977fcc41250e4#diff-a38ae78886b7c3afb711d32cfb4dbbbdR267
> . The parameter "okPath" changed from okPath to true.
> The following workflow is problematic:
> {noformat}
>
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> Shell action failed, error
> message[${wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-3139) Oozie validates workflow incorrectly
[
https://issues.apache.org/jira/browse/OOZIE-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278710#comment-16278710
]
Peter Bacsko commented on OOZIE-3139:
-
[~dionusos] could you please attach the exception w/ stack trace that you
receive for this workflow? (I assume there is one)
> Oozie validates workflow incorrectly
>
>
> Key: OOZIE-3139
> URL: https://issues.apache.org/jira/browse/OOZIE-3139
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.3.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Attachments: workflow.pdf
>
>
> We found that after OOZIE-1978 the attached workflow becomes invalid. Aa we
> found out the following line brings the problem:
> https://github.com/apache/oozie/commit/8e9b9042b3270dc5ff975c44a5c977fcc41250e4#diff-a38ae78886b7c3afb711d32cfb4dbbbdR267
> . The parameter "okPath" changed from okPath to true.
> The following workflow is problematic:
> {noformat}
>
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> Shell action failed, error
> message[${wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-3139) Oozie validates workflow incorrectly
[
https://issues.apache.org/jira/browse/OOZIE-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278707#comment-16278707
]
Peter Bacsko commented on OOZIE-3139:
-
[~dionusos] I made that change, Robert reviewed & committed it. I'll check what
happens here.
> Oozie validates workflow incorrectly
>
>
> Key: OOZIE-3139
> URL: https://issues.apache.org/jira/browse/OOZIE-3139
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.3.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Attachments: workflow.pdf
>
>
> We found that after OOZIE-1978 the attached workflow becomes invalid. Aa we
> found out the following line brings the problem:
> https://github.com/apache/oozie/commit/8e9b9042b3270dc5ff975c44a5c977fcc41250e4#diff-a38ae78886b7c3afb711d32cfb4dbbbdR267
> . The parameter "okPath" changed from okPath to true.
> The following workflow is problematic:
> {noformat}
>
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> Shell action failed, error
> message[${wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-3139) Oozie validates workflow incorrectly
[
https://issues.apache.org/jira/browse/OOZIE-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278695#comment-16278695
]
Denes Bodo commented on OOZIE-3139:
---
[~rkanter] Can you please take a look?
> Oozie validates workflow incorrectly
>
>
> Key: OOZIE-3139
> URL: https://issues.apache.org/jira/browse/OOZIE-3139
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.3.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Attachments: workflow.pdf
>
>
> We found that after OOZIE-1978 the attached workflow becomes invalid. Aa we
> found out the following line brings the problem:
> https://github.com/apache/oozie/commit/8e9b9042b3270dc5ff975c44a5c977fcc41250e4#diff-a38ae78886b7c3afb711d32cfb4dbbbdR267
> The following workflow is problematic:
> {noformat}
>
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
>
>
>
>
>
>
>
> Shell action failed, error
> message[${wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
