[jira] [Commented] (OOZIE-3474) workflow jobs stuck in prep state on single manager and 3 data node cluster.
[ https://issues.apache.org/jira/browse/OOZIE-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16818851#comment-16818851 ] Andras Salamon commented on OOZIE-3474: --- Can you please upload a simple workflow and job.properties file which can be used to reproduce the problem? Could you test it against a more recent Oozie version? > workflow jobs stuck in prep state on single manager and 3 data node cluster. > > > Key: OOZIE-3474 > URL: https://issues.apache.org/jira/browse/OOZIE-3474 > Project: Oozie > Issue Type: Bug > Components: workflow >Affects Versions: 4.2.0 >Reporter: Akash S >Priority: Major > > I have a HDP cluster with single manager node and 3 data nodes. workflow > job's status shows as running but the task status shows a prep and it is > stuck in the same state forever. > The same job works fine in an HDP cluster with 3 manager and 4 data node > setup. > > I have also verified that the jobs.properties is pointing to proper job > tracker and nameNode. > There is nothing unusual or any relevant warning seen in the oozie.log or in > oozie-error.log. > There is no resource crunch in Yarn too, as there are no other jobs using > yarn's resources. > oozie status too says 'Normal' > I have recreated the setup multiple times and the issue is consistent. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (OOZIE-3265) properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together
[ https://issues.apache.org/jira/browse/OOZIE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton reassigned OOZIE-3265: - Assignee: Julia Kinga Marton (was: TIAN XING) > properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear > together > -- > > Key: OOZIE-3265 > URL: https://issues.apache.org/jira/browse/OOZIE-3265 > Project: Oozie > Issue Type: Task >Affects Versions: 5.0.0 >Reporter: TIAN XING >Assignee: Julia Kinga Marton >Priority: Minor > Attachments: OOZIE-3265-v1.patch, OOZIE-3265-v2.patch, > OOZIE-3265-v3.patch, rerun.patch > > > Currently when you re-run a workflow with property "oozie.wf.rerun.failnodes" > set to true, > you can no longer re-run it again with "oozie.wf.rerun.skip.nodes" property > specified, even if you set "oozie.wf.rerun.failnodes" to false. > This kind of limitation is not reasonable. There is only one case where > "oozie.wf.rerun.failnodes" is true and "oozie.wf.rerun.skip.nodes" is not > null or empty, that should be disallowed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2972) Server goes inconsistent when prepare war called with secure without SSL
[
https://issues.apache.org/jira/browse/OOZIE-2972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16818823#comment-16818823
]
Andras Salamon commented on OOZIE-2972:
---
[~dionusos] [~kmarton] We still support WAR creation, but it's not really
tested since we started to use embedded Jetty. The documentation is definitely
not useful, I've already opened OOZIE-3428 about that.
> Server goes inconsistent when prepare war called with secure without SSL
>
>
> Key: OOZIE-2972
> URL: https://issues.apache.org/jira/browse/OOZIE-2972
> Project: Oozie
> Issue Type: Bug
> Components: security
>Affects Versions: 4.3.0
>Reporter: Prabhu Joseph
>Priority: Major
>
> When prepare-war with secure is called by some user by mistake on a Oozie
> Server which is not configured with SSL causes inconsistent state. Oozie
> Server runs fine but the oozie clients are failed with Authentication failure
> status 302. Checking curl verbose, Oozie Server redirects client to https
> port even though it is not listening. We need to validate the prepare-war
> command when SSL is not configured instead of going to inconsistent state.
> Repro:
> {code}
> Oozie Server without SSL
> /usr/hdp/current/oozie-server/bin/oozie-setup.sh prepare-war -secure
> Start Oozie Server
> curl -ikv -L --negotiate -u:
> http://prabhuzeppelin2.openstacklocal:11000/oozie/v1/admin/status
> * About to connect() to prabhuzeppelin2.openstacklocal port 11000 (#0)
> * Trying 172.26.93.73... connected
> * Connected to prabhuzeppelin2.openstacklocal (172.26.93.73) port 11000 (#0)
> > GET /oozie/v1/admin/status HTTP/1.1
> > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.27.1
> > zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> > Host: prabhuzeppelin2.openstacklocal:11000
> > Accept: */*
> >
> < HTTP/1.1 302 Found
> HTTP/1.1 302 Found
> < Server: Apache-Coyote/1.1
> Server: Apache-Coyote/1.1
> < Pragma: No-cache
> Pragma: No-cache
> < Cache-Control: no-cache
> Cache-Control: no-cache
> < Expires: Thu, 01 Jan 1970 00:00:00 UTC
> Expires: Thu, 01 Jan 1970 00:00:00 UTC
> < Location: https://prabhuzeppelin2.openstacklocal:11443/oozie/v1/admin/status
> Location: https://prabhuzeppelin2.openstacklocal:11443/oozie/v1/admin/status
> < Content-Length: 0
> Content-Length: 0
> < Date: Tue, 27 Jun 2017 11:05:45 GMT
> Date: Tue, 27 Jun 2017 11:05:45 GMT
> <
> * Connection #0 to host prabhuzeppelin2.openstacklocal left intact
> * Issue another request to this URL:
> 'https://prabhuzeppelin2.openstacklocal:11443/oozie/v1/admin/status'
> * About to connect() to prabhuzeppelin2.openstacklocal port 11443 (#1)
> * Trying 172.26.93.73... Connection refused
> * couldn't connect to host
> * Closing connection #1
> curl: (7) couldn't connect to host
> * Closing connection #0
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2972) Server goes inconsistent when prepare war called with secure without SSL
[
https://issues.apache.org/jira/browse/OOZIE-2972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16818782#comment-16818782
]
Julia Kinga Marton commented on OOZIE-2972:
---
[~dionusos], it needs to be checked with the actual master, because we still
support war creation.
> Server goes inconsistent when prepare war called with secure without SSL
>
>
> Key: OOZIE-2972
> URL: https://issues.apache.org/jira/browse/OOZIE-2972
> Project: Oozie
> Issue Type: Bug
> Components: security
>Affects Versions: 4.3.0
>Reporter: Prabhu Joseph
>Priority: Major
>
> When prepare-war with secure is called by some user by mistake on a Oozie
> Server which is not configured with SSL causes inconsistent state. Oozie
> Server runs fine but the oozie clients are failed with Authentication failure
> status 302. Checking curl verbose, Oozie Server redirects client to https
> port even though it is not listening. We need to validate the prepare-war
> command when SSL is not configured instead of going to inconsistent state.
> Repro:
> {code}
> Oozie Server without SSL
> /usr/hdp/current/oozie-server/bin/oozie-setup.sh prepare-war -secure
> Start Oozie Server
> curl -ikv -L --negotiate -u:
> http://prabhuzeppelin2.openstacklocal:11000/oozie/v1/admin/status
> * About to connect() to prabhuzeppelin2.openstacklocal port 11000 (#0)
> * Trying 172.26.93.73... connected
> * Connected to prabhuzeppelin2.openstacklocal (172.26.93.73) port 11000 (#0)
> > GET /oozie/v1/admin/status HTTP/1.1
> > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.27.1
> > zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> > Host: prabhuzeppelin2.openstacklocal:11000
> > Accept: */*
> >
> < HTTP/1.1 302 Found
> HTTP/1.1 302 Found
> < Server: Apache-Coyote/1.1
> Server: Apache-Coyote/1.1
> < Pragma: No-cache
> Pragma: No-cache
> < Cache-Control: no-cache
> Cache-Control: no-cache
> < Expires: Thu, 01 Jan 1970 00:00:00 UTC
> Expires: Thu, 01 Jan 1970 00:00:00 UTC
> < Location: https://prabhuzeppelin2.openstacklocal:11443/oozie/v1/admin/status
> Location: https://prabhuzeppelin2.openstacklocal:11443/oozie/v1/admin/status
> < Content-Length: 0
> Content-Length: 0
> < Date: Tue, 27 Jun 2017 11:05:45 GMT
> Date: Tue, 27 Jun 2017 11:05:45 GMT
> <
> * Connection #0 to host prabhuzeppelin2.openstacklocal left intact
> * Issue another request to this URL:
> 'https://prabhuzeppelin2.openstacklocal:11443/oozie/v1/admin/status'
> * About to connect() to prabhuzeppelin2.openstacklocal port 11443 (#1)
> * Trying 172.26.93.73... Connection refused
> * couldn't connect to host
> * Closing connection #1
> curl: (7) couldn't connect to host
> * Closing connection #0
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2972) Server goes inconsistent when prepare war called with secure without SSL
[
https://issues.apache.org/jira/browse/OOZIE-2972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16818771#comment-16818771
]
Denes Bodo commented on OOZIE-2972:
---
[~asalamon74] Am I right that with eliminating Tomcat and using Jetty instead
this situation is no more possible? Thanks
> Server goes inconsistent when prepare war called with secure without SSL
>
>
> Key: OOZIE-2972
> URL: https://issues.apache.org/jira/browse/OOZIE-2972
> Project: Oozie
> Issue Type: Bug
> Components: security
>Affects Versions: 4.3.0
>Reporter: Prabhu Joseph
>Priority: Major
>
> When prepare-war with secure is called by some user by mistake on a Oozie
> Server which is not configured with SSL causes inconsistent state. Oozie
> Server runs fine but the oozie clients are failed with Authentication failure
> status 302. Checking curl verbose, Oozie Server redirects client to https
> port even though it is not listening. We need to validate the prepare-war
> command when SSL is not configured instead of going to inconsistent state.
> Repro:
> {code}
> Oozie Server without SSL
> /usr/hdp/current/oozie-server/bin/oozie-setup.sh prepare-war -secure
> Start Oozie Server
> curl -ikv -L --negotiate -u:
> http://prabhuzeppelin2.openstacklocal:11000/oozie/v1/admin/status
> * About to connect() to prabhuzeppelin2.openstacklocal port 11000 (#0)
> * Trying 172.26.93.73... connected
> * Connected to prabhuzeppelin2.openstacklocal (172.26.93.73) port 11000 (#0)
> > GET /oozie/v1/admin/status HTTP/1.1
> > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.27.1
> > zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> > Host: prabhuzeppelin2.openstacklocal:11000
> > Accept: */*
> >
> < HTTP/1.1 302 Found
> HTTP/1.1 302 Found
> < Server: Apache-Coyote/1.1
> Server: Apache-Coyote/1.1
> < Pragma: No-cache
> Pragma: No-cache
> < Cache-Control: no-cache
> Cache-Control: no-cache
> < Expires: Thu, 01 Jan 1970 00:00:00 UTC
> Expires: Thu, 01 Jan 1970 00:00:00 UTC
> < Location: https://prabhuzeppelin2.openstacklocal:11443/oozie/v1/admin/status
> Location: https://prabhuzeppelin2.openstacklocal:11443/oozie/v1/admin/status
> < Content-Length: 0
> Content-Length: 0
> < Date: Tue, 27 Jun 2017 11:05:45 GMT
> Date: Tue, 27 Jun 2017 11:05:45 GMT
> <
> * Connection #0 to host prabhuzeppelin2.openstacklocal left intact
> * Issue another request to this URL:
> 'https://prabhuzeppelin2.openstacklocal:11443/oozie/v1/admin/status'
> * About to connect() to prabhuzeppelin2.openstacklocal port 11443 (#1)
> * Trying 172.26.93.73... Connection refused
> * couldn't connect to host
> * Closing connection #1
> curl: (7) couldn't connect to host
> * Closing connection #0
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3470) PurgeXCommand coordActionDel variable assignment should be standardized
[
https://issues.apache.org/jira/browse/OOZIE-3470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16818747#comment-16818747
]
Junfan Zhang commented on OOZIE-3470:
-
Thanks [~asalamon74] :)
> PurgeXCommand coordActionDel variable assignment should be standardized
> ---
>
> Key: OOZIE-3470
> URL: https://issues.apache.org/jira/browse/OOZIE-3470
> Project: Oozie
> Issue Type: Bug
>Reporter: Junfan Zhang
>Assignee: Junfan Zhang
>Priority: Major
> Fix For: 5.2.0
>
> Attachments: OOZIE-3470-1.patch
>
>
> PurgeXCommand coordActionDel variable assignment should be standardized, like
> {{wfDel}}, {{coordDel}} and {{bundleDel}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (OOZIE-3474) workflow jobs stuck in prep state on single manager and 3 data node cluster.
Akash S created OOZIE-3474: -- Summary: workflow jobs stuck in prep state on single manager and 3 data node cluster. Key: OOZIE-3474 URL: https://issues.apache.org/jira/browse/OOZIE-3474 Project: Oozie Issue Type: Bug Components: workflow Affects Versions: 4.2.0 Reporter: Akash S I have a HDP cluster with single manager node and 3 data nodes. workflow job's status shows as running but the task status shows a prep and it is stuck in the same state forever. The same job works fine in an HDP cluster with 3 manager and 4 data node setup. I have also verified that the jobs.properties is pointing to proper job tracker and nameNode. There is nothing unusual or any relevant warning seen in the oozie.log or in oozie-error.log. There is no resource crunch in Yarn too, as there are no other jobs using yarn's resources. oozie status too says 'Normal' I have recreated the setup multiple times and the issue is consistent. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3470) PurgeXCommand coordActionDel variable assignment should be standardized
[
https://issues.apache.org/jira/browse/OOZIE-3470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16818739#comment-16818739
]
Andras Salamon commented on OOZIE-3470:
---
Test error is unrelated.
Thanks for the contributions [~zuston], +1, committed to master.
> PurgeXCommand coordActionDel variable assignment should be standardized
> ---
>
> Key: OOZIE-3470
> URL: https://issues.apache.org/jira/browse/OOZIE-3470
> Project: Oozie
> Issue Type: Bug
>Reporter: Junfan Zhang
>Assignee: Junfan Zhang
>Priority: Major
> Attachments: OOZIE-3470-1.patch
>
>
> PurgeXCommand coordActionDel variable assignment should be standardized, like
> {{wfDel}}, {{coordDel}} and {{bundleDel}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] Subscription: Oozie Patch Available
Issue Subscription Filter: Oozie Patch Available (94 issues) Subscriber: ooziedaily Key Summary OOZIE-3470 PurgeXCommand coordActionDel variable assignment should be standardized https://issues.apache.org/jira/browse/OOZIE-3470 OOZIE-3468 Use modernizer plugin https://issues.apache.org/jira/browse/OOZIE-3468 OOZIE-3464 Use UTF8 charset instead of default one https://issues.apache.org/jira/browse/OOZIE-3464 OOZIE-3461 CoordMaterializeTriggerService code cleanup https://issues.apache.org/jira/browse/OOZIE-3461 OOZIE-3455 Inconsistent CoordMaterializeTransitionXCommand logging https://issues.apache.org/jira/browse/OOZIE-3455 OOZIE-3449 Make spark-2 as the default profile https://issues.apache.org/jira/browse/OOZIE-3449 OOZIE-3447 Run test case in local : It shows oozie-hsqldb-orm.xml exception https://issues.apache.org/jira/browse/OOZIE-3447 OOZIE-3418 Upgrade to Guava 27 https://issues.apache.org/jira/browse/OOZIE-3418 OOZIE-3404 The env variable of SPARK_HOME needs to be set when running pySpark https://issues.apache.org/jira/browse/OOZIE-3404 OOZIE-3393 Add Oozie instrumentation delayed metric in CoordMaterializeTriggerService https://issues.apache.org/jira/browse/OOZIE-3393 OOZIE-3375 Can't use empty in coordinator https://issues.apache.org/jira/browse/OOZIE-3375 OOZIE-3367 Using && in EL expressions in oozie bundle.xml files generates parse errors https://issues.apache.org/jira/browse/OOZIE-3367 OOZIE-3366 Update workflow status and subworkflow status on suspend command https://issues.apache.org/jira/browse/OOZIE-3366 OOZIE-3364 Rerunning Oozie bundle jobs starts the coordinators in indeterminate order https://issues.apache.org/jira/browse/OOZIE-3364 OOZIE-3362 When killed, SSH action should kill the spawned processes on target host https://issues.apache.org/jira/browse/OOZIE-3362 OOZIE-3335 Cleanup parseFilter methods https://issues.apache.org/jira/browse/OOZIE-3335 OOZIE-3328 Create Hive compatibility action executor to run hive actions using beeline https://issues.apache.org/jira/browse/OOZIE-3328 OOZIE-3320 Oozie ShellAction should support absolute bash file path https://issues.apache.org/jira/browse/OOZIE-3320 OOZIE-3319 Log SSH action callback error output https://issues.apache.org/jira/browse/OOZIE-3319 OOZIE-3301 Update NOTICE file https://issues.apache.org/jira/browse/OOZIE-3301 OOZIE-3274 Remove slf4j https://issues.apache.org/jira/browse/OOZIE-3274 OOZIE-3266 Coord action rerun support RERUN_SKIP_NODES option https://issues.apache.org/jira/browse/OOZIE-3266 OOZIE-3265 properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together https://issues.apache.org/jira/browse/OOZIE-3265 OOZIE-3256 refactor OozieCLI class https://issues.apache.org/jira/browse/OOZIE-3256 OOZIE-3199 Let system property restriction configurable https://issues.apache.org/jira/browse/OOZIE-3199 OOZIE-3196 Authorization: restrict world readability by user https://issues.apache.org/jira/browse/OOZIE-3196 OOZIE-3179 Adding a configurable config-default.xml location to a workflow https://issues.apache.org/jira/browse/OOZIE-3179 OOZIE-3170 Oozie Diagnostic Bundle tool fails with NPE due to missing service class https://issues.apache.org/jira/browse/OOZIE-3170 OOZIE-3137 Add support for log4j2 in HiveMain https://issues.apache.org/jira/browse/OOZIE-3137 OOZIE-3135 Configure log4j2 in SqoopMain https://issues.apache.org/jira/browse/OOZIE-3135 OOZIE-3091 Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: org/apache/avro/mapred/AvroWrapper" https://issues.apache.org/jira/browse/OOZIE-3091 OOZIE-3071 Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0 https://issues.apache.org/jira/browse/OOZIE-3071 OOZIE-3063 Sanitizing variables that are part of openjpa.ConnectionProperties https://issues.apache.org/jira/browse/OOZIE-3063 OOZIE-3062 Set HADOOP_CONF_DIR for spark action https://issues.apache.org/jira/browse/OOZIE-3062 OOZIE-2952 Fix Findbugs warnings in oozie-sharelib-oozie https://issues.apache.org/jira/browse/OOZIE-2952 OOZIE-2927 Append new line character for Hive2 query using query tag https://issues.apache.org/jira/browse/OOZIE-2927 OOZIE-2834 ParameterVerifier logging non-useful warning for workflow definition https://issues.apache.org/jira/browse/OOZIE-2834 OOZIE-2833 when using uber mode the regex pattern used in the extractHeapSizeMB method does not allow heap sizes specified in bytes. https://issues.apache.org/jira/browse/OOZIE-2833 OOZIE-2812
