[jira] [Commented] (OOZIE-3307) oozie workflow gets failed throwing error virtual memory limits

2018-08-31 Thread Vladimir Prus (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598887#comment-16598887
 ] 

Vladimir Prus commented on OOZIE-3307:
--

Although this particular error message can be solved by tweaking yarn settings, 
there is an issue with Oozie itself. It allocated 2GB Yarn container, however, 
I don't see any Xmx option above. On many systems, default Xmx is above 2GB, so 
with this default configuration, Ozzie AM can easily hit physical memory size 
limit, and be likewise killed. Oozie 4 used to pass -Xmx1024 to AM by default.

 

Is this a issue that can be discussed in this Jira, or a new one would be more 
appropriate?

> oozie workflow gets failed throwing error virtual memory limits
> ---
>
> Key: OOZIE-3307
> URL: https://issues.apache.org/jira/browse/OOZIE-3307
> Project: Oozie
>  Issue Type: Bug
>Reporter: Sabir Naikwadi
>Priority: Critical
>
> Application application_1531909575787_0039 failed 2 times due to AM Container 
> for appattempt_1531909575787_0039_02 exited with exitCode: -103
>  Failing this attempt.Diagnostics: Container 
> [pid=11516,containerID=container_1531909575787_0039_02_01] is running 
> beyond virtual memory limits. Current usage: 469.8 MB of 2 GB physical memory 
> used; 10.0 GB of 10 GB virtual memory used. Killing container.
>  Dump of the process-tree for container_1531909575787_0039_02_01 :
> | - PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) 
> SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE|
> | - 11516 11514 11516 11516 (bash) 1 3 115863552 682 /bin/bash -c 
> /usr/lib/jvm/java-openjdk/bin/java 
> -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true 
> -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01
>  -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA 
> -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev 
> org.apache.oozie.action.hadoop.LauncherAM 
> 1>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stdout
>  
> 2>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stderr|
> | - 11755 11516 11516 11516 (java) 1142 71 10658242560 119576 
> /usr/lib/jvm/java-openjdk/bin/java 
> -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true 
> -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01
>  -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA 
> -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev 
> org.apache.oozie.action.hadoop.LauncherAM
>  Container killed on request. Exit code is 143
>  Container exited with a non-zero exit code 143
>  For more detailed output, check the application tracking page: 
> [http://ip-10-20-201-36.us-gov-west-1.compute.internal:8088/cluster/app/application_1531909575787_0039]
>  Then click on links to logs of each attempt.
>  . Failing the application.|



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3298) OYA: external ID is not filled properly and failing MR job is treated as SUCCEEDED

2018-08-31 Thread Hadoop QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598686#comment-16598686
 ] 

Hadoop QA commented on OOZIE-3298:
--


Testing JIRA OOZIE-3298

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch adds/modifies 6 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} Javadoc generation succeeded with the patch
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warning(s)
.{color:orange}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:orange}0{color} There are [1] new bugs found in total that would be nice 
to have fixed.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:orange}0{color} There are [1] new bugs found in [core] that would be 
nice to have fixed.
. You can find the FindBugs diff here: core/findbugs-new.html
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in 
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2943
.{color:orange}Tests failed at first run:{color}
TestCoordActionsKillXCommand#testActionKillCommandDate
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:green}*+1 Overall result, good!, no -1s*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/802/



> OYA: external ID is not filled properly and failing MR job is treated as 
> SUCCEEDED
> --
>
> Key: OOZIE-3298
> URL: https://issues.apache.org/jira/browse/OOZIE-3298
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 5.0.0
>Reporter: Peter Bacsko
>Assignee: Andras Piros
>Priority: Blocker
> Fix For: 5.1.0
>
> Attachments: OOZIE-3298.001.patch, OOZIE-3298.002.patch, 
> OOZIE-3298.003.patch, OOZIE-3298.004.patch, OOZIE-3298.005.patch, 
> OOZIE-3298.007.patch, OOZIE-3298.008.patch, OOZIE-3298.009.patch, 
> OOZIE-3298.010.patch
>
>
> When a MapReduce action is launched from Oozie (OYA), we don't properly fill 
> the external ID field. It gets populated with the YARN id of the LauncherAM, 
> not with the id of the actual MR job. If the MR job is succesfully submitted 
> then fails, it will be treated as a successfully executed action, which is 
> very misleading and can potentially confuse Oozie users.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-2877) Oozie Git Action

2018-08-31 Thread Hadoop QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-2877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598624#comment-16598624
 ] 

Hadoop QA commented on OOZIE-2877:
--


Testing JIRA OOZIE-2877

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch adds/modifies 6 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} Javadoc generation succeeded with the patch
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warning(s)
.{color:orange}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:orange}0{color} There are [2] new bugs found in total that would be nice 
to have fixed.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in 
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:orange}0{color} There are [2] new bugs found in [sharelib/git] that 
would be nice to have fixed.
. You can find the FindBugs diff here: sharelib/git/findbugs-new.html
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2939
.{color:orange}Tests failed at first run:{color}
TestCoordActionInputCheckXCommand#testNone
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:green}*+1 Overall result, good!, no -1s*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/801/



> Oozie Git Action
> 
>
> Key: OOZIE-2877
> URL: https://issues.apache.org/jira/browse/OOZIE-2877
> Project: Oozie
>  Issue Type: Sub-task
>  Components: action
>Affects Versions: 5.0.0
>Reporter: Clay B.
>Assignee: Clay B.
>Priority: Major
>  Labels: action
> Fix For: 5.1.0
>
> Attachments: 0001-OOZIE-2877-Oozie-Git-Action.patch, 
> 0002-OOZIE-2877-Oozie-Git-Action.patch, 
> 0003-OOZIE-2877-Oozie-Git-Action.patch, 
> 0004-OOZIE-2877-Oozie-Git-Action.patch, 
> 0005-OOZIE-2877-Oozie-Git-Action.patch, 
> 0006-OOZIE-2877-Oozie-Git-Action.patch, 
> 0007-OOZIE-2877-Oozie-Git-Action.patch, 
> 0008-OOZIE-2877-Oozie-Git-Action.patch, 
> 0009-OOZIE-2877-Oozie-Git-Action.patch, OOZIE-2877.010.patch, 
> OOZIE-2877.011.patch, OOZIE-2877.012.patch, OOZIE-2877.013-1.patch, 
> OOZIE-2877.013.patch, OOZIE-2877.014-1.patch, OOZIE-2877.014-2.patch, 
> OOZIE-2877.014-3.patch, OOZIE-2877.015.patch, OOZIE-2877.016.patch, 
> OOZIE-2877.017.patch
>
>
> To aide in deploying ASCII artifacts to clusters, let's provide a tie-in for 
> a source-code management system. Git would be my preferred choice.
> Ideally, this could handle a user's key material e.g. for an ssh key to pull 
> down from a

[jira] [Commented] (OOZIE-3061) Kill only those child jobs which are not already killed

2018-08-31 Thread Andras Piros (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598607#comment-16598607
 ] 

Andras Piros commented on OOZIE-3061:
-

[~matijhs] as I see the patch 005 consists of following parts:
# refactoring {{JavaActionExecutor#kill()}} - doesn't need any new tests
# extending {{LauncherMain#killChildYarnJobs()}} - definitely needs new tests 
in {{TestLauncherMain}}

Apart from testing perspective, could you please modify the condition in 
{{LauncherMain#killChildYarnJobs():265}} so that instead of getting only the 
{{ApplicationReport}} in {{FinalApplicationStatus.UNDEFINED}}, we instead get 
the one that is not {{SUCCEED}}, {{FAILED}}, or {{KILLED}}? So that we can stay 
future proof just in case someone extends {{FinalApplicationStatus}} with a new 
field.

> Kill only those child jobs which are not already killed
> ---
>
> Key: OOZIE-3061
> URL: https://issues.apache.org/jira/browse/OOZIE-3061
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 4.3.0
>Reporter: Satish Subhashrao Saley
>Assignee: Mate Juhasz
>Priority: Trivial
>  Labels: newbie, newbiee
> Fix For: 5.1.0
>
> Attachments: OOZIE-3061-001.patch, OOZIE-3061-002.patch, 
> OOZIE-3061-003.patch, OOZIE-3061-004.patch, OOZIE-3061-005.patch
>
>
> Here we kill all child jobs. 
> https://github.com/apache/oozie/blob/master/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java#L265
> We should check before killing for already killed application.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3298) OYA: external ID is not filled properly and failing MR job is treated as SUCCEEDED

2018-08-31 Thread Hadoop QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598577#comment-16598577
 ] 

Hadoop QA commented on OOZIE-3298:
--

PreCommit-OOZIE-Build started


> OYA: external ID is not filled properly and failing MR job is treated as 
> SUCCEEDED
> --
>
> Key: OOZIE-3298
> URL: https://issues.apache.org/jira/browse/OOZIE-3298
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 5.0.0
>Reporter: Peter Bacsko
>Assignee: Andras Piros
>Priority: Blocker
> Fix For: 5.1.0
>
> Attachments: OOZIE-3298.001.patch, OOZIE-3298.002.patch, 
> OOZIE-3298.003.patch, OOZIE-3298.004.patch, OOZIE-3298.005.patch, 
> OOZIE-3298.007.patch, OOZIE-3298.008.patch, OOZIE-3298.009.patch, 
> OOZIE-3298.010.patch
>
>
> When a MapReduce action is launched from Oozie (OYA), we don't properly fill 
> the external ID field. It gets populated with the YARN id of the LauncherAM, 
> not with the id of the actual MR job. If the MR job is succesfully submitted 
> then fails, it will be treated as a successfully executed action, which is 
> very misleading and can potentially confuse Oozie users.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3298) OYA: external ID is not filled properly and failing MR job is treated as SUCCEEDED

2018-08-31 Thread Andras Piros (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598578#comment-16598578
 ] 

Andras Piros commented on OOZIE-3298:
-

Addressing review comments.

> OYA: external ID is not filled properly and failing MR job is treated as 
> SUCCEEDED
> --
>
> Key: OOZIE-3298
> URL: https://issues.apache.org/jira/browse/OOZIE-3298
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 5.0.0
>Reporter: Peter Bacsko
>Assignee: Andras Piros
>Priority: Blocker
> Fix For: 5.1.0
>
> Attachments: OOZIE-3298.001.patch, OOZIE-3298.002.patch, 
> OOZIE-3298.003.patch, OOZIE-3298.004.patch, OOZIE-3298.005.patch, 
> OOZIE-3298.007.patch, OOZIE-3298.008.patch, OOZIE-3298.009.patch, 
> OOZIE-3298.010.patch
>
>
> When a MapReduce action is launched from Oozie (OYA), we don't properly fill 
> the external ID field. It gets populated with the YARN id of the LauncherAM, 
> not with the id of the actual MR job. If the MR job is succesfully submitted 
> then fails, it will be treated as a successfully executed action, which is 
> very misleading and can potentially confuse Oozie users.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3298) OYA: external ID is not filled properly and failing MR job is treated as SUCCEEDED

2018-08-31 Thread Andras Piros (JIRA) 


 [ 
https://issues.apache.org/jira/browse/OOZIE-3298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros updated OOZIE-3298:

Attachment: OOZIE-3298.010.patch

> OYA: external ID is not filled properly and failing MR job is treated as 
> SUCCEEDED
> --
>
> Key: OOZIE-3298
> URL: https://issues.apache.org/jira/browse/OOZIE-3298
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 5.0.0
>Reporter: Peter Bacsko
>Assignee: Andras Piros
>Priority: Blocker
> Fix For: 5.1.0
>
> Attachments: OOZIE-3298.001.patch, OOZIE-3298.002.patch, 
> OOZIE-3298.003.patch, OOZIE-3298.004.patch, OOZIE-3298.005.patch, 
> OOZIE-3298.007.patch, OOZIE-3298.008.patch, OOZIE-3298.009.patch, 
> OOZIE-3298.010.patch
>
>
> When a MapReduce action is launched from Oozie (OYA), we don't properly fill 
> the external ID field. It gets populated with the YARN id of the LauncherAM, 
> not with the id of the actual MR job. If the MR job is succesfully submitted 
> then fails, it will be treated as a successfully executed action, which is 
> very misleading and can potentially confuse Oozie users.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 68237: OOZIE-3298 OYA: external ID is not filled properly and failing MR job is treated as SUCCEEDED

2018-08-31 Thread András Piros via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68237/
---

(Updated Aug. 31, 2018, 10:50 a.m.)


Review request for oozie, Andras Salamon, Attila Sasvari, Kinga Marton, and 
Peter Bacsko.


Changes
---

Addressing review comments.


Repository: oozie-git


Description
---

OOZIE-3298 OYA: external ID is not filled properly and failing MR job is 
treated as SUCCEEDED


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
8f0f244013932476d8ae454d224f235948529b34 
  
core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 
83a23f5220aa72ba15edc8b98ef80a74213fcee8 
  core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java 
f39bba2c691435354dac6da7794e5142b511d937 
  core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 
a31079a41d30677d35a253a4a69505c21aa585f6 
  
core/src/test/java/org/apache/oozie/action/hadoop/TestYarnApplicationIdComparator.java
 PRE-CREATION 
  
core/src/test/java/org/apache/oozie/action/hadoop/TestYarnApplicationIdFinder.java
 PRE-CREATION 
  
core/src/test/java/org/apache/oozie/action/oozie/TestSubWorkflowActionExecutor.java
 893405e38ad71d22460768b76ed03ac4d9a0b95d 
  
sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java
 f460b6bd11f60dfb397c6bba82be1427c2d1b570 


Diff: https://reviews.apache.org/r/68237/diff/7/

Changes: https://reviews.apache.org/r/68237/diff/6-7/


Testing
---

Tested on a real cluster, plus added test cases to 
`TestMapReduceActionExecutor` and new test classes.

Please note that I could find no proper way of actually getting MapReduce to 
start a new job while using `OozieClient#submit()` - apparently no YARN child 
application is created. Please advise what's the best way to advance, maybe 
call `JobClient#submitJob()` directly from 
`ActionExecutorTestCase#startWorkflowAndFailChildMRJob()`.


Thanks,

András Piros

Re: Review Request 68237: OOZIE-3298 OYA: external ID is not filled properly and failing MR job is treated as SUCCEEDED

2018-08-31 Thread András Piros via Review Board 


> On Aug. 30, 2018, 1:34 p.m., Peter Bacsko wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java
> > Lines 575 (patched)
> > 
> >
> > Can't we just compare the IDs as strings? What's the rationale of this 
> > class?
> > 
> > If it's not passed to other data structures or classes which use a 
> > Comparator, then I guess it doesn't have to implement it.

The rationale behind this class is to compare two YARN application ID 
`String`s, when the cluster startup time is different. Newer application IDs 
will be treated as "greater".

We cannot really use plain `String` comparison because we could face situations 
where different application IDs are read with different cluster startup times.


- András


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68237/#review208130
---


On Aug. 24, 2018, 2:18 p.m., András Piros wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68237/
> ---
> 
> (Updated Aug. 24, 2018, 2:18 p.m.)
> 
> 
> Review request for oozie, Andras Salamon, Attila Sasvari, Kinga Marton, and 
> Peter Bacsko.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-3298 OYA: external ID is not filled properly and failing MR job is 
> treated as SUCCEEDED
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> 8f0f244013932476d8ae454d224f235948529b34 
>   
> core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java
>  83a23f5220aa72ba15edc8b98ef80a74213fcee8 
>   
> core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java 
> f39bba2c691435354dac6da7794e5142b511d937 
>   
> core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 
> a31079a41d30677d35a253a4a69505c21aa585f6 
>   
> core/src/test/java/org/apache/oozie/action/hadoop/TestYarnApplicationIdComparator.java
>  PRE-CREATION 
>   
> core/src/test/java/org/apache/oozie/action/hadoop/TestYarnApplicationIdFinder.java
>  PRE-CREATION 
>   
> core/src/test/java/org/apache/oozie/action/oozie/TestSubWorkflowActionExecutor.java
>  893405e38ad71d22460768b76ed03ac4d9a0b95d 
>   
> sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java
>  f460b6bd11f60dfb397c6bba82be1427c2d1b570 
> 
> 
> Diff: https://reviews.apache.org/r/68237/diff/6/
> 
> 
> Testing
> ---
> 
> Tested on a real cluster, plus added test cases to 
> `TestMapReduceActionExecutor` and new test classes.
> 
> Please note that I could find no proper way of actually getting MapReduce to 
> start a new job while using `OozieClient#submit()` - apparently no YARN child 
> application is created. Please advise what's the best way to advance, maybe 
> call `JobClient#submitJob()` directly from 
> `ActionExecutorTestCase#startWorkflowAndFailChildMRJob()`.
> 
> 
> Thanks,
> 
> András Piros
> 
>

[jira] [Updated] (OOZIE-2734) Switch docs from twiki to markdown

2018-08-31 Thread Andras Piros (JIRA) 


 [ 
https://issues.apache.org/jira/browse/OOZIE-2734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros updated OOZIE-2734:

Fix Version/s: 5.1.0

> Switch docs from twiki to markdown
> --
>
> Key: OOZIE-2734
> URL: https://issues.apache.org/jira/browse/OOZIE-2734
> Project: Oozie
>  Issue Type: Improvement
>  Components: docs
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Andras Salamon
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-2734-1.patch, OOZIE-2734-2.patch, 
> OOZIE-2734-3.patch, OOZIE-2734-4.patch, OOZIE-2734-5.patch, 
> OOZIE-2734-6.patch, oozie_generated_html_20180731.zip, 
> oozie_generated_html_markdown_20180802.zip, twiki2md_20180730.zip
>
>
> This won't be a fun JIRA, but it would be great if we could switch to 
> something easier to work with and better supported like markdown, and get rid 
> of the super old twiki we're using for docs.
> I found two tools that I haven't tried out that claim to be able to convert 
> this for us:
> # http://pandoc.org/
> # https://github.com/jcodagnone/twiki2markdown
> The former supports a lot of different formats, and according to [this blog 
> post|http://goer.org/Journal/2013/02/converting_twiki_foswiki_to_markdown_restructuredtext.html]
>  it's better to convert from the rendered html instead of the twiki.  But who 
> knows which is better, or if the second tool is better than either.
> I think once we switch to a better format, it will be easier to keep the docs 
> in better shape, as they could really use some reorganizing.  Though we can 
> keep that for a separate task.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 68505: OOZIE-2877 Git action

2018-08-31 Thread András Piros via Review Board 


> On Aug. 31, 2018, 10:03 a.m., Andras Salamon wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/GitActionExecutor.java
> > Line 175 (original), 175 (patched)
> > 
> >
> > Shouldn't we register this error code in registerErrors method?

`ActionExecutor#registerError()` is used when a specific `Exception` subclass 
should always be treated given the same `errorType` and `errorCode`. This is 
not the case here.


> On Aug. 31, 2018, 10:03 a.m., Andras Salamon wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/GitActionExecutor.java
> > Line 183 (original), 183 (patched)
> > 
> >
> > Shouldn't we register this error code in registerErrors method?

`ActionExecutor#registerError()` is used when a specific `Exception` subclass 
should always be treated given the same `errorType` and `errorCode`. This is 
not the case here.


> On Aug. 31, 2018, 10:03 a.m., Andras Salamon wrote:
> > sharelib/git/src/test/java/org/apache/oozie/action/hadoop/TestGitMain.java
> > Line 106 (original), 107 (patched)
> > 
> >
> > If we want to check the length then assertEquals("...", 
> > credentialFileData.length(), testOutput.length()) would be better

This is how it looked like before Peter Bacsko pointed out it's not a good idea 
to check for a given length of a test output. So we're checking here only that 
the file is indeed created and non-empty.


- András


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68505/#review208163
---


On Aug. 31, 2018, 9:50 a.m., András Piros wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68505/
> ---
> 
> (Updated Aug. 31, 2018, 9:50 a.m.)
> 
> 
> Review request for oozie, Andras Salamon, Clay B., Peter Cseh, Kinga Marton, 
> and Peter Bacsko.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-2877 Git action
> 
> 
> Diffs
> -
> 
>   client/src/main/resources/git-action-1.0.xsd PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/GitActionExecutor.java 
> PRE-CREATION 
>   core/src/main/resources/oozie-default.xml 
> b69d2c9aacc1a7f5f903fb80f77bbf5482feb3b5 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java 
> 661970d57ac038d7311856a93f5221f081e9ce15 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki 
> 76cbe21ecd79eae86885a8db77831bf2c40f2d4f 
>   examples/src/main/apps/git/job.properties PRE-CREATION 
>   examples/src/main/apps/git/workflow.xml PRE-CREATION 
>   examples/src/main/java/org/apache/oozie/example/fluentjob/Git.java 
> PRE-CREATION 
>   fluent-job/fluent-job-api/pom.xml cefd43c143550a24ffba5a1d6922b8241dd5756a 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/action/GitAction.java
>  PRE-CREATION 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/action/GitActionBuilder.java
>  PRE-CREATION 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/DistcpConfigurationConverter.java
>  fffb734e7c1b920e94f76e6a2b3062dd5aa73a86 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/ExplicitNodeConverter.java
>  7bb82e5cbbffbee02785361eb2bbde2fc1c31e7a 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/GitConfigurationConverter.java
>  PRE-CREATION 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/GitPrepareConverter.java
>  PRE-CREATION 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/Hive2ConfigurationConverter.java
>  c67b5aeab41e0c2d963c0ad3cf44fd19fe7c66e3 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/HiveConfigurationConverter.java
>  5f9a2b180623b89c793e8e6bb6e03d445c0ad991 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/InlineWorkflowConfigurationConverter.java
>  b1e17c9b5b8c29a81792ebfc637a53564bdb60fe 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/ShellConfigurationConverter.java
>  ab73dfd2f24cbbfa60a37d75e5057f3ff2a1c0fa 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/SparkConfigurationConverter.java
>  8827769ea84f8f08a79e5e2d0189e2d1ded4bd9d 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/SqoopConfigurationConverter.java
>  1d4f615af583f8b941e570f3e3f239979cc8f825 
>   
>

Re: Review Request 68505: OOZIE-2877 Git action

2018-08-31 Thread Andras Salamon 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68505/#review208163
---




core/src/main/java/org/apache/oozie/action/hadoop/GitActionExecutor.java
Line 134 (original), 134 (patched)


Shouldn't we register this error code in registerErrors method?



core/src/main/java/org/apache/oozie/action/hadoop/GitActionExecutor.java
Line 175 (original), 175 (patched)


Shouldn't we register this error code in registerErrors method?



core/src/main/java/org/apache/oozie/action/hadoop/GitActionExecutor.java
Line 183 (original), 183 (patched)


Shouldn't we register this error code in registerErrors method?



sharelib/git/src/test/java/org/apache/oozie/action/hadoop/TestGitMain.java
Line 106 (original), 107 (patched)


If we want to check the length then assertEquals("...", 
credentialFileData.length(), testOutput.length()) would be better


- Andras Salamon


On Aug. 31, 2018, 9:50 a.m., András Piros wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68505/
> ---
> 
> (Updated Aug. 31, 2018, 9:50 a.m.)
> 
> 
> Review request for oozie, Andras Salamon, Clay B., Peter Cseh, Kinga Marton, 
> and Peter Bacsko.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-2877 Git action
> 
> 
> Diffs
> -
> 
>   client/src/main/resources/git-action-1.0.xsd PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/GitActionExecutor.java 
> PRE-CREATION 
>   core/src/main/resources/oozie-default.xml 
> b69d2c9aacc1a7f5f903fb80f77bbf5482feb3b5 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java 
> 661970d57ac038d7311856a93f5221f081e9ce15 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki 
> 76cbe21ecd79eae86885a8db77831bf2c40f2d4f 
>   examples/src/main/apps/git/job.properties PRE-CREATION 
>   examples/src/main/apps/git/workflow.xml PRE-CREATION 
>   examples/src/main/java/org/apache/oozie/example/fluentjob/Git.java 
> PRE-CREATION 
>   fluent-job/fluent-job-api/pom.xml cefd43c143550a24ffba5a1d6922b8241dd5756a 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/action/GitAction.java
>  PRE-CREATION 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/action/GitActionBuilder.java
>  PRE-CREATION 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/DistcpConfigurationConverter.java
>  fffb734e7c1b920e94f76e6a2b3062dd5aa73a86 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/ExplicitNodeConverter.java
>  7bb82e5cbbffbee02785361eb2bbde2fc1c31e7a 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/GitConfigurationConverter.java
>  PRE-CREATION 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/GitPrepareConverter.java
>  PRE-CREATION 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/Hive2ConfigurationConverter.java
>  c67b5aeab41e0c2d963c0ad3cf44fd19fe7c66e3 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/HiveConfigurationConverter.java
>  5f9a2b180623b89c793e8e6bb6e03d445c0ad991 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/InlineWorkflowConfigurationConverter.java
>  b1e17c9b5b8c29a81792ebfc637a53564bdb60fe 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/ShellConfigurationConverter.java
>  ab73dfd2f24cbbfa60a37d75e5057f3ff2a1c0fa 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/SparkConfigurationConverter.java
>  8827769ea84f8f08a79e5e2d0189e2d1ded4bd9d 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/SqoopConfigurationConverter.java
>  1d4f615af583f8b941e570f3e3f239979cc8f825 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/serialization/WorkflowMarshaller.java
>  ec56554ba4cd313a9d5ab8472ab3c62b6bc17380 
>   fluent-job/fluent-job-api/src/main/resources/action_mappings.xml 
> a5f890eef9ccc352a5a852781f4212d96913e6d2 
>   fluent-job/fluent-job-api/src/main/xjb/bindings.xml 
> 48f68905ac2df2818d88907b26e863326d6d2c61 
>   
> fluent-job/fluent-job-client/src/test/java/org/apache/oozie/jobs/client/minitest/TestGitAction.java
>  PRE-CREATION 
>   pom.xml e0dbe85e5238fdc11cbd44cc31f65fc768c7454d 
>   sharelib/git/pom.xml PRE-CREATION 
>   sharelib/git/src/main/java/org/apache/oozie/action/hadoop/GitMain.java

[jira] [Commented] (OOZIE-2877) Oozie Git Action

2018-08-31 Thread Hadoop QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-2877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598530#comment-16598530
 ] 

Hadoop QA commented on OOZIE-2877:
--

PreCommit-OOZIE-Build started


> Oozie Git Action
> 
>
> Key: OOZIE-2877
> URL: https://issues.apache.org/jira/browse/OOZIE-2877
> Project: Oozie
>  Issue Type: Sub-task
>  Components: action
>Affects Versions: 5.0.0
>Reporter: Clay B.
>Assignee: Clay B.
>Priority: Major
>  Labels: action
> Fix For: 5.1.0
>
> Attachments: 0001-OOZIE-2877-Oozie-Git-Action.patch, 
> 0002-OOZIE-2877-Oozie-Git-Action.patch, 
> 0003-OOZIE-2877-Oozie-Git-Action.patch, 
> 0004-OOZIE-2877-Oozie-Git-Action.patch, 
> 0005-OOZIE-2877-Oozie-Git-Action.patch, 
> 0006-OOZIE-2877-Oozie-Git-Action.patch, 
> 0007-OOZIE-2877-Oozie-Git-Action.patch, 
> 0008-OOZIE-2877-Oozie-Git-Action.patch, 
> 0009-OOZIE-2877-Oozie-Git-Action.patch, OOZIE-2877.010.patch, 
> OOZIE-2877.011.patch, OOZIE-2877.012.patch, OOZIE-2877.013-1.patch, 
> OOZIE-2877.013.patch, OOZIE-2877.014-1.patch, OOZIE-2877.014-2.patch, 
> OOZIE-2877.014-3.patch, OOZIE-2877.015.patch, OOZIE-2877.016.patch, 
> OOZIE-2877.017.patch
>
>
> To aide in deploying ASCII artifacts to clusters, let's provide a tie-in for 
> a source-code management system. Git would be my preferred choice.
> Ideally, this could handle a user's key material e.g. for an ssh key to pull 
> down from a secured repository. This would free users from handling their own 
> key staging and clean-up on YARN nodes and only require them to store the key 
> secured in HDFS.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3155) [ui] Job DAG is not refreshed when a job is finished

2018-08-31 Thread Andras Piros (JIRA) 


 [ 
https://issues.apache.org/jira/browse/OOZIE-3155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros updated OOZIE-3155:

Summary: [ui] Job DAG is not refreshed when a job is finished  (was: Job 
DAG is not refreshed when a job is finished)

> [ui] Job DAG is not refreshed when a job is finished
> 
>
> Key: OOZIE-3155
> URL: https://issues.apache.org/jira/browse/OOZIE-3155
> Project: Oozie
>  Issue Type: Bug
>  Components: client
>Affects Versions: trunk, 5.0.0b1
>Reporter: Julia Kinga Marton
>Assignee: Andras Salamon
>Priority: Critical
> Fix For: 5.1.0
>
> Attachments: OOZIE-3155-1.patch, Screen Shot 2018-01-10 at 
> 16.23.28.png
>
>
> When opening a Job DAG when the job is still in progress, the DAG is OK, but 
> when the job will finish, this diagram is not refreshed.
> Steps to reproduce: 
> - submit a workflow (e.g. the subwf from the examples) - the wf should not 
> end very quickly, to have time to catch it when it is still running
> - check the Job DAG when the job is still running
> - wait for the job to finish
> - check again the Job DAG -> is the same with the running one
> Note: if check it in a new tab, is OK, and also a browser refresh will solve 
> it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-2877) Oozie Git Action

2018-08-31 Thread Andras Piros (JIRA) 


 [ 
https://issues.apache.org/jira/browse/OOZIE-2877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros updated OOZIE-2877:

Attachment: OOZIE-2877.017.patch

> Oozie Git Action
> 
>
> Key: OOZIE-2877
> URL: https://issues.apache.org/jira/browse/OOZIE-2877
> Project: Oozie
>  Issue Type: Sub-task
>  Components: action
>Affects Versions: 5.0.0
>Reporter: Clay B.
>Assignee: Clay B.
>Priority: Major
>  Labels: action
> Fix For: 5.1.0
>
> Attachments: 0001-OOZIE-2877-Oozie-Git-Action.patch, 
> 0002-OOZIE-2877-Oozie-Git-Action.patch, 
> 0003-OOZIE-2877-Oozie-Git-Action.patch, 
> 0004-OOZIE-2877-Oozie-Git-Action.patch, 
> 0005-OOZIE-2877-Oozie-Git-Action.patch, 
> 0006-OOZIE-2877-Oozie-Git-Action.patch, 
> 0007-OOZIE-2877-Oozie-Git-Action.patch, 
> 0008-OOZIE-2877-Oozie-Git-Action.patch, 
> 0009-OOZIE-2877-Oozie-Git-Action.patch, OOZIE-2877.010.patch, 
> OOZIE-2877.011.patch, OOZIE-2877.012.patch, OOZIE-2877.013-1.patch, 
> OOZIE-2877.013.patch, OOZIE-2877.014-1.patch, OOZIE-2877.014-2.patch, 
> OOZIE-2877.014-3.patch, OOZIE-2877.015.patch, OOZIE-2877.016.patch, 
> OOZIE-2877.017.patch
>
>
> To aide in deploying ASCII artifacts to clusters, let's provide a tie-in for 
> a source-code management system. Git would be my preferred choice.
> Ideally, this could handle a user's key material e.g. for an ssh key to pull 
> down from a secured repository. This would free users from handling their own 
> key staging and clean-up on YARN nodes and only require them to store the key 
> secured in HDFS.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 68505: OOZIE-2877 Git action

2018-08-31 Thread András Piros via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68505/
---

(Updated Aug. 31, 2018, 9:50 a.m.)


Review request for oozie, Andras Salamon, Clay B., Peter Cseh, Kinga Marton, 
and Peter Bacsko.


Repository: oozie-git


Description
---

OOZIE-2877 Git action


Diffs (updated)
-

  client/src/main/resources/git-action-1.0.xsd PRE-CREATION 
  core/src/main/java/org/apache/oozie/action/hadoop/GitActionExecutor.java 
PRE-CREATION 
  core/src/main/resources/oozie-default.xml 
b69d2c9aacc1a7f5f903fb80f77bbf5482feb3b5 
  core/src/test/java/org/apache/oozie/test/XTestCase.java 
661970d57ac038d7311856a93f5221f081e9ce15 
  docs/src/site/twiki/WorkflowFunctionalSpec.twiki 
76cbe21ecd79eae86885a8db77831bf2c40f2d4f 
  examples/src/main/apps/git/job.properties PRE-CREATION 
  examples/src/main/apps/git/workflow.xml PRE-CREATION 
  examples/src/main/java/org/apache/oozie/example/fluentjob/Git.java 
PRE-CREATION 
  fluent-job/fluent-job-api/pom.xml cefd43c143550a24ffba5a1d6922b8241dd5756a 
  
fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/action/GitAction.java
 PRE-CREATION 
  
fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/action/GitActionBuilder.java
 PRE-CREATION 
  
fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/DistcpConfigurationConverter.java
 fffb734e7c1b920e94f76e6a2b3062dd5aa73a86 
  
fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/ExplicitNodeConverter.java
 7bb82e5cbbffbee02785361eb2bbde2fc1c31e7a 
  
fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/GitConfigurationConverter.java
 PRE-CREATION 
  
fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/GitPrepareConverter.java
 PRE-CREATION 
  
fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/Hive2ConfigurationConverter.java
 c67b5aeab41e0c2d963c0ad3cf44fd19fe7c66e3 
  
fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/HiveConfigurationConverter.java
 5f9a2b180623b89c793e8e6bb6e03d445c0ad991 
  
fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/InlineWorkflowConfigurationConverter.java
 b1e17c9b5b8c29a81792ebfc637a53564bdb60fe 
  
fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/ShellConfigurationConverter.java
 ab73dfd2f24cbbfa60a37d75e5057f3ff2a1c0fa 
  
fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/SparkConfigurationConverter.java
 8827769ea84f8f08a79e5e2d0189e2d1ded4bd9d 
  
fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/SqoopConfigurationConverter.java
 1d4f615af583f8b941e570f3e3f239979cc8f825 
  
fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/serialization/WorkflowMarshaller.java
 ec56554ba4cd313a9d5ab8472ab3c62b6bc17380 
  fluent-job/fluent-job-api/src/main/resources/action_mappings.xml 
a5f890eef9ccc352a5a852781f4212d96913e6d2 
  fluent-job/fluent-job-api/src/main/xjb/bindings.xml 
48f68905ac2df2818d88907b26e863326d6d2c61 
  
fluent-job/fluent-job-client/src/test/java/org/apache/oozie/jobs/client/minitest/TestGitAction.java
 PRE-CREATION 
  pom.xml e0dbe85e5238fdc11cbd44cc31f65fc768c7454d 
  sharelib/git/pom.xml PRE-CREATION 
  sharelib/git/src/main/java/org/apache/oozie/action/hadoop/GitMain.java 
PRE-CREATION 
  sharelib/git/src/main/java/org/apache/oozie/action/hadoop/GitOperations.java 
PRE-CREATION 
  sharelib/git/src/test/java/org/apache/oozie/action/hadoop/GitServer.java 
PRE-CREATION 
  
sharelib/git/src/test/java/org/apache/oozie/action/hadoop/TestGitActionExecutor.java
 PRE-CREATION 
  sharelib/git/src/test/java/org/apache/oozie/action/hadoop/TestGitMain.java 
PRE-CREATION 
  
sharelib/git/src/test/java/org/apache/oozie/action/hadoop/TestIntegrationGitActionExecutor.java
 PRE-CREATION 
  sharelib/pom.xml 39cea257e750eb8b3f3c4cb3f137093fccc03016 
  src/main/assemblies/sharelib.xml 07dc69c8276895b254e5af0cc021cce6ebad18f4 
  webapp/pom.xml fd3f89feedd76657f1431898a268298549826d6f 


Diff: https://reviews.apache.org/r/68505/diff/3/

Changes: https://reviews.apache.org/r/68505/diff/2-3/


Testing
---

Multiple unit and integration tests, as well as tests performed on a local 
Oozie server w/ pseudo-distributed Hadoop.

Based on the work Clay B. has done in previous patchsets till 
[OOZIE-2877.014-3.patch](https://issues.apache.org/jira/secure/attachment/12934336/OOZIE-2877.014-3.patch).


Thanks,

András Piros

[jira] [Commented] (OOZIE-3155) Job DAG is not refreshed when a job is finished

2018-08-31 Thread Hadoop QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598513#comment-16598513
 ] 

Hadoop QA commented on OOZIE-3155:
--


Testing JIRA OOZIE-3155

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} Javadoc generation succeeded with the patch
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warning(s)
.{color:orange}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in 
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2931
.{color:orange}Tests failed at first run:{color}
TestCoordActionsKillXCommand#testActionKillCommandDate
TestJMSJobEventListener#testConnectionDrop
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/800/



> Job DAG is not refreshed when a job is finished
> ---
>
> Key: OOZIE-3155
> URL: https://issues.apache.org/jira/browse/OOZIE-3155
> Project: Oozie
>  Issue Type: Bug
>  Components: client
>Affects Versions: trunk, 5.0.0b1
>Reporter: Julia Kinga Marton
>Assignee: Andras Salamon
>Priority: Critical
> Fix For: 5.1.0
>
> Attachments: OOZIE-3155-1.patch, Screen Shot 2018-01-10 at 
> 16.23.28.png
>
>
> When opening a Job DAG when the job is still in progress, the DAG is OK, but 
> when the job will finish, this diagram is not refreshed.
> Steps to reproduce: 
> - submit a workflow (e.g. the subwf from the examples) - the wf should not 
> end very quickly, to have time to catch it when it is still running
> - check the Job DAG when the job is still running
> - wait for the job to finish
> - check again the Job DAG -> is the same with the running one
> Note: if check it in a new tab, is OK, and also a browser refresh will solve 
> it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3334) Don't use org.apache.hadoop.hbase.security.User in HDFSCredentials

2018-08-31 Thread Hadoop QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598497#comment-16598497
 ] 

Hadoop QA commented on OOZIE-3334:
--


Testing JIRA OOZIE-3334

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} Javadoc generation succeeded with the patch
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warning(s)
.{color:orange}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:orange}0{color} There are [1] new bugs found in total that would be nice 
to have fixed.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:orange}0{color} There are [1] new bugs found in [core] that would be 
nice to have fixed.
. You can find the FindBugs diff here: core/findbugs-new.html
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in 
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2931
.{color:orange}Tests failed at first run:{color}
TestCoordActionInputCheckXCommand#testNone
TestCoordActionsKillXCommand#testActionKillCommandActionNumbers
TestCoordMaterializeTriggerService#testCoordMaterializeTriggerService3
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/799/



> Don't use org.apache.hadoop.hbase.security.User in HDFSCredentials
> --
>
> Key: OOZIE-3334
> URL: https://issues.apache.org/jira/browse/OOZIE-3334
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 5.0.0b1
>Reporter: Peter Cseh
>Assignee: Peter Cseh
>Priority: Major
> Attachments: OOZIE-3334.01.patch, OOZIE-3334.02.patch
>
>
> HDFSCredentials is using org.apache.hadoop.hbase.security.User to obtain 
> delegation tokens.
> It introduces a runtime dependency on HBase which is not required in Oozie 
> core.
> It was reported on GitHub:
> https://github.com/apache/oozie/commit/c29d9c5fce27f5f5335662ce2649387258f1c7fd#commitcomment-30224511



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (OOZIE-3136) Upgrade from Log4j 1.x to 2.x

2018-08-31 Thread Andras Piros (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598440#comment-16598440
 ] 

Andras Piros edited comment on OOZIE-3136 at 8/31/18 8:44 AM:
--

Hi [~bugg_tb],

we're targeting to resolve this issue in the next couple of weeks.

One who wants to get this resolved sooner needs to address following other 
JIRAs as well:
* OOZIE-3135
* OOZIE-3137


was (Author: andras.piros):
Hi [~bugg_tb],

we're targeting to resolve this issue in the next couple of weeks.

One who wants to get this resolved sooner needs to address following other 
JIRAs:
* OOZIE-3135
* OOZIE-3137

> Upgrade from Log4j 1.x to 2.x
> -
>
> Key: OOZIE-3136
> URL: https://issues.apache.org/jira/browse/OOZIE-3136
> Project: Oozie
>  Issue Type: Improvement
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
>
> {{5 August 2015 --The Apache Logging Services™ Project Management Committee 
> (PMC) has announced that the Log4j™ 1.x logging framework has reached its end 
> of life (EOL) and is no longer officially supported.}} 
> https://blogs.apache.org/foundation/entry/apache_logging_services_project_announces
> We should upgrade from Log4j 1.x to 2.x . Perhaps we could use slf4j .
> Related tickets: MAPREDUCE-6983, HADOOP-12956, OOZIE-3135



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (OOZIE-3136) Upgrade from Log4j 1.x to 2.x

2018-08-31 Thread Andras Piros (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598440#comment-16598440
 ] 

Andras Piros edited comment on OOZIE-3136 at 8/31/18 8:44 AM:
--

Hi [~bugg_tb],

we're targeting to resolve this issue in the next couple of weeks.

One who wants to get this resolved sooner needs to address following other 
JIRAs:
* OOZIE-3135
* OOZIE-3137


was (Author: andras.piros):
Hi [~bugg_tb],

we're targeting to resolve this issue in the next couple of weeks.

The one who wants to get this resolved sooner needs to address following other 
JIRAs:
* OOZIE-3135
* OOZIE-3137

> Upgrade from Log4j 1.x to 2.x
> -
>
> Key: OOZIE-3136
> URL: https://issues.apache.org/jira/browse/OOZIE-3136
> Project: Oozie
>  Issue Type: Improvement
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
>
> {{5 August 2015 --The Apache Logging Services™ Project Management Committee 
> (PMC) has announced that the Log4j™ 1.x logging framework has reached its end 
> of life (EOL) and is no longer officially supported.}} 
> https://blogs.apache.org/foundation/entry/apache_logging_services_project_announces
> We should upgrade from Log4j 1.x to 2.x . Perhaps we could use slf4j .
> Related tickets: MAPREDUCE-6983, HADOOP-12956, OOZIE-3135



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3136) Upgrade from Log4j 1.x to 2.x

2018-08-31 Thread Andras Piros (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598440#comment-16598440
 ] 

Andras Piros commented on OOZIE-3136:
-

Hi [~bugg_tb],

we're targeting to resolve this issue in the next couple of weeks.

The one that wants to get this resolved sooner needs to address following other 
JIRAs:
* OOZIE-3135
* OOZIE-3137

> Upgrade from Log4j 1.x to 2.x
> -
>
> Key: OOZIE-3136
> URL: https://issues.apache.org/jira/browse/OOZIE-3136
> Project: Oozie
>  Issue Type: Improvement
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
>
> {{5 August 2015 --The Apache Logging Services™ Project Management Committee 
> (PMC) has announced that the Log4j™ 1.x logging framework has reached its end 
> of life (EOL) and is no longer officially supported.}} 
> https://blogs.apache.org/foundation/entry/apache_logging_services_project_announces
> We should upgrade from Log4j 1.x to 2.x . Perhaps we could use slf4j .
> Related tickets: MAPREDUCE-6983, HADOOP-12956, OOZIE-3135



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (OOZIE-3136) Upgrade from Log4j 1.x to 2.x

2018-08-31 Thread Andras Piros (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598440#comment-16598440
 ] 

Andras Piros edited comment on OOZIE-3136 at 8/31/18 8:44 AM:
--

Hi [~bugg_tb],

we're targeting to resolve this issue in the next couple of weeks.

The one who wants to get this resolved sooner needs to address following other 
JIRAs:
* OOZIE-3135
* OOZIE-3137


was (Author: andras.piros):
Hi [~bugg_tb],

we're targeting to resolve this issue in the next couple of weeks.

The one that wants to get this resolved sooner needs to address following other 
JIRAs:
* OOZIE-3135
* OOZIE-3137

> Upgrade from Log4j 1.x to 2.x
> -
>
> Key: OOZIE-3136
> URL: https://issues.apache.org/jira/browse/OOZIE-3136
> Project: Oozie
>  Issue Type: Improvement
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
>
> {{5 August 2015 --The Apache Logging Services™ Project Management Committee 
> (PMC) has announced that the Log4j™ 1.x logging framework has reached its end 
> of life (EOL) and is no longer officially supported.}} 
> https://blogs.apache.org/foundation/entry/apache_logging_services_project_announces
> We should upgrade from Log4j 1.x to 2.x . Perhaps we could use slf4j .
> Related tickets: MAPREDUCE-6983, HADOOP-12956, OOZIE-3135



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3155) Job DAG is not refreshed when a job is finished

2018-08-31 Thread Hadoop QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598391#comment-16598391
 ] 

Hadoop QA commented on OOZIE-3155:
--

PreCommit-OOZIE-Build started


> Job DAG is not refreshed when a job is finished
> ---
>
> Key: OOZIE-3155
> URL: https://issues.apache.org/jira/browse/OOZIE-3155
> Project: Oozie
>  Issue Type: Bug
>  Components: client
>Affects Versions: trunk, 5.0.0b1
>Reporter: Julia Kinga Marton
>Assignee: Andras Salamon
>Priority: Critical
> Fix For: 5.1.0
>
> Attachments: OOZIE-3155-1.patch, Screen Shot 2018-01-10 at 
> 16.23.28.png
>
>
> When opening a Job DAG when the job is still in progress, the DAG is OK, but 
> when the job will finish, this diagram is not refreshed.
> Steps to reproduce: 
> - submit a workflow (e.g. the subwf from the examples) - the wf should not 
> end very quickly, to have time to catch it when it is still running
> - check the Job DAG when the job is still running
> - wait for the job to finish
> - check again the Job DAG -> is the same with the running one
> Note: if check it in a new tab, is OK, and also a browser refresh will solve 
> it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3334) Don't use org.apache.hadoop.hbase.security.User in HDFSCredentials

2018-08-31 Thread Hadoop QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598390#comment-16598390
 ] 

Hadoop QA commented on OOZIE-3334:
--

PreCommit-OOZIE-Build started


> Don't use org.apache.hadoop.hbase.security.User in HDFSCredentials
> --
>
> Key: OOZIE-3334
> URL: https://issues.apache.org/jira/browse/OOZIE-3334
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 5.0.0b1
>Reporter: Peter Cseh
>Assignee: Peter Cseh
>Priority: Major
> Attachments: OOZIE-3334.01.patch, OOZIE-3334.02.patch
>
>
> HDFSCredentials is using org.apache.hadoop.hbase.security.User to obtain 
> delegation tokens.
> It introduces a runtime dependency on HBase which is not required in Oozie 
> core.
> It was reported on GitHub:
> https://github.com/apache/oozie/commit/c29d9c5fce27f5f5335662ce2649387258f1c7fd#commitcomment-30224511



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3155) Job DAG is not refreshed when a job is finished

2018-08-31 Thread Andras Salamon (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-3155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598389#comment-16598389
 ] 

Andras Salamon commented on OOZIE-3155:
---

Jenkins build failed but the problem is not related to this patch: 
[https://builds.apache.org/job/PreCommit-OOZIE-Build/798/]

[~andras.piros] Can you please retrigger it?

> Job DAG is not refreshed when a job is finished
> ---
>
> Key: OOZIE-3155
> URL: https://issues.apache.org/jira/browse/OOZIE-3155
> Project: Oozie
>  Issue Type: Bug
>  Components: client
>Affects Versions: trunk, 5.0.0b1
>Reporter: Julia Kinga Marton
>Assignee: Andras Salamon
>Priority: Critical
> Fix For: 5.1.0
>
> Attachments: OOZIE-3155-1.patch, Screen Shot 2018-01-10 at 
> 16.23.28.png
>
>
> When opening a Job DAG when the job is still in progress, the DAG is OK, but 
> when the job will finish, this diagram is not refreshed.
> Steps to reproduce: 
> - submit a workflow (e.g. the subwf from the examples) - the wf should not 
> end very quickly, to have time to catch it when it is still running
> - check the Job DAG when the job is still running
> - wait for the job to finish
> - check again the Job DAG -> is the same with the running one
> Note: if check it in a new tab, is OK, and also a browser refresh will solve 
> it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (OOZIE-2458) 'oozie-setup.sh sharelib create' should ensure uploaded jars are world readable

2018-08-31 Thread Ferenc Denes (JIRA) 


 [ 
https://issues.apache.org/jira/browse/OOZIE-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ferenc Denes reassigned OOZIE-2458:
---

Assignee: (was: Ferenc Denes)

> 'oozie-setup.sh sharelib create' should ensure uploaded jars are world 
> readable
> ---
>
> Key: OOZIE-2458
> URL: https://issues.apache.org/jira/browse/OOZIE-2458
> Project: Oozie
>  Issue Type: Bug
>  Components: scripts
>Affects Versions: 4.1.0
>Reporter: Ferenc Denes
>Priority: Critical
> Fix For: trunk
>
> Attachments: OOZIE-2458-1.patch
>
>
> If the default umask of HDFS does not include world read then the Oozie 
> sharelib gets created accordingly, and then the jars can not be read at 
> runtime because Oozie workflows impersonate the submitting user.
> The 'oozie-setup.sh sharelib create' command should ensure that the installed 
> jars are world readable.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] Subscription: Oozie Patch Available

2018-08-31 Thread jira 
Issue Subscription
Filter: Oozie Patch Available (96 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-3334  Don't use org.apache.hadoop.hbase.security.User in HDFSCredentials
https://issues.apache.org/jira/browse/OOZIE-3334
OOZIE-3326  Sqoop Action should support tez delegation tokens for hive-import
https://issues.apache.org/jira/browse/OOZIE-3326
OOZIE-3320  Oozie ShellAction should support absolute bash file path
https://issues.apache.org/jira/browse/OOZIE-3320
OOZIE-3319  Log SSH action callback error output
https://issues.apache.org/jira/browse/OOZIE-3319
OOZIE-3301  Update NOTICE file
https://issues.apache.org/jira/browse/OOZIE-3301
OOZIE-3298  OYA: external ID is not filled properly and failing MR job is 
treated as SUCCEEDED
https://issues.apache.org/jira/browse/OOZIE-3298
OOZIE-3277  [build] Check for star imports in patches in pre-commit
https://issues.apache.org/jira/browse/OOZIE-3277
OOZIE-3274  Remove slf4j
https://issues.apache.org/jira/browse/OOZIE-3274
OOZIE-3266  Coord action rerun support RERUN_SKIP_NODES option
https://issues.apache.org/jira/browse/OOZIE-3266
OOZIE-3265  properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to 
appear together
https://issues.apache.org/jira/browse/OOZIE-3265
OOZIE-3256  refactor OozieCLI class
https://issues.apache.org/jira/browse/OOZIE-3256
OOZIE-3249  [tools] Instrumentation log parser
https://issues.apache.org/jira/browse/OOZIE-3249
OOZIE-3229  Improved filtering options in V2SLAServlet
https://issues.apache.org/jira/browse/OOZIE-3229
OOZIE-3218  Oozie Sqoop action with command splits the select clause into 
multiple parts due to delimiter being space
https://issues.apache.org/jira/browse/OOZIE-3218
OOZIE-3199  Let system property restriction configurable
https://issues.apache.org/jira/browse/OOZIE-3199
OOZIE-3196  Authorization: restrict world readability by user
https://issues.apache.org/jira/browse/OOZIE-3196
OOZIE-3194  Oozie should set proper permissions to sharelib after upload
https://issues.apache.org/jira/browse/OOZIE-3194
OOZIE-3186  Oozie is unable to use configuration linked using jceks://file/...
https://issues.apache.org/jira/browse/OOZIE-3186
OOZIE-3179  Adding a configurable config-default.xml location to a workflow
https://issues.apache.org/jira/browse/OOZIE-3179
OOZIE-3170  Oozie Diagnostic Bundle tool fails with NPE due to missing service 
class
https://issues.apache.org/jira/browse/OOZIE-3170
OOZIE-3160  PriorityDelayQueue put()/take() can cause significant CPU load due 
to busy waiting
https://issues.apache.org/jira/browse/OOZIE-3160
OOZIE-3155  Job DAG is not refreshed when a job is finished
https://issues.apache.org/jira/browse/OOZIE-3155
OOZIE-3135  Configure log4j2 in SqoopMain
https://issues.apache.org/jira/browse/OOZIE-3135
OOZIE-3091  Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: 
org/apache/avro/mapred/AvroWrapper"
https://issues.apache.org/jira/browse/OOZIE-3091
OOZIE-3071  Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 
than Spark 2.2.0
https://issues.apache.org/jira/browse/OOZIE-3071
OOZIE-3063  Sanitizing variables that are part of openjpa.ConnectionProperties
https://issues.apache.org/jira/browse/OOZIE-3063
OOZIE-3062  Set HADOOP_CONF_DIR for spark action
https://issues.apache.org/jira/browse/OOZIE-3062
OOZIE-3061  Kill only those child jobs which are not already killed
https://issues.apache.org/jira/browse/OOZIE-3061
OOZIE-2952  Fix Findbugs warnings in oozie-sharelib-oozie
https://issues.apache.org/jira/browse/OOZIE-2952
OOZIE-2949  Escape quotes whitespaces in Sqoop  field
https://issues.apache.org/jira/browse/OOZIE-2949
OOZIE-2927  Append new line character for Hive2 query using query tag
https://issues.apache.org/jira/browse/OOZIE-2927
OOZIE-2877  Oozie Git Action
https://issues.apache.org/jira/browse/OOZIE-2877
OOZIE-2834  ParameterVerifier logging non-useful warning for workflow definition
https://issues.apache.org/jira/browse/OOZIE-2834
OOZIE-2833  when using uber mode the regex pattern used in the 
extractHeapSizeMB method does not allow heap sizes specified in bytes.
https://issues.apache.org/jira/browse/OOZIE-2833
OOZIE-2812  SparkConfigurationService should support loading configurations 
from multiple Spark versions
https://issues.apache.org/jira/browse/OOZIE-2812
OOZIE-2795  Create lib directory or symlink for Oozie CLI during packaging
https://issues.apache.org/jira/browse/OOZIE-2795
OOZIE-2784  Include WEEK as a parameter in the Coordinator Expression Language 
Evaulator