Jenkins build is back to normal : oozie-trunk-find-patches-available #166415

2017-08-02 Thread Apache Jenkins Server
See 




Build failed in Jenkins: oozie-trunk-find-patches-available #166414

2017-08-02 Thread Apache Jenkins Server
See 


--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on H1 (Hadoop) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/oozie.git # timeout=10
Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/oozie.git
 > git --version # timeout=10
 > git fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/oozie.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 4f3afcacad072792feb02d59fe1f440f1487bf00 
(refs/remotes/origin/master)
Commit message: "OOZIE-2852 Remove simple-json dependency from oozie sharelib 
(Artem Ervits via gezapeti)"
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 4f3afcacad072792feb02d59fe1f440f1487bf00
 > git rev-list 4f3afcacad072792feb02d59fe1f440f1487bf00 # timeout=10
[oozie-trunk-find-patches-available] $ /bin/bash 
/tmp/jenkins2748576035989058940.sh
mkdir: cannot create directory 
':
 File exists
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  
0 00 00 0  0  0 --:--:--  0:00:01 --:--:-- 0  0 
00 00 0  0  0 --:--:--  0:00:02 --:--:-- 0  0   
  00 00 0  0  0 --:--:--  0:00:03 --:--:-- 0  0 
00 00 0  0  0 --:--:--  0:00:04 --:--:-- 0  0 0 
   0 00 0  0  0 --:--:--  0:00:05 --:--:-- 0  0 0   
 0 00 0  0  0 --:--:--  0:00:06 --:--:-- 0  0 0
0 00 0  0  0 --:--:--  0:00:07 --:--:-- 0  0 00 
00 0  0  0 --:--:--  0:00:08 --:--:-- 0  0 00   
  00 0  0  0 --:--:--  0:00:09 --:--:-- 0  0 00 
00 0  0  0 --:--:--  0:00:10 --:--:-- 0  0 00 0 
   0 0  0  0 --:--:--  0:00:11 --:--:-- 0  0 00 0   
 0 0  0  0 --:--:--  0:00:12 --:--:-- 0  0 00 0
0 0  0  0 --:--:--  0:00:13 --:--:-- 0  0 00 00 
0  0  0 --:--:--  0:00:14 --:--:-- 0  0 00 00   
  0  0  0 --:--:--  0:00:15 --:--:-- 0  0 00 00 
0  0  0 --:--:--  0:00:16 --:--:-- 0  0 00 00 0 
 0  0 --:--:--  0:00:17 --:--:-- 0  0 00 00 0   
   0  0 --:--:--  0:00:18 --:--:-- 0  0 00 00 0 
 0  0 --:--:--  0:00:19 --:--:-- 0  0 00 00 0  
0  0 --:--:--  0:00:20 --:--:-- 0  0 00 00 0  0 
 0 --:--:--  0:00:21 --:--:-- 0  0 00 00 0  0   
   0 --:--:--  0:00:22 --:--:-- 0  0 00 00 0  0 
 0 --:--:--  0:00:23 --:--:-- 0  0 00 00 0  0  
0 --:--:--  0:00:24 --:--:-- 0  0 00 00 0  0  0 
--:--:--  0:00:25 --:--:-- 0  0 00 00 0  0  0 
--:--:--  0:00:26 --:--:-- 0  0 00 00 0  0  0 
--:--:--  0:00:27 --:--:-- 0  0 00 00 0  0  0 
--:--:--  0:00:28 --:--:-- 0  0 00 00 0  0  0 
--:--:--  0:00:29 --:--:-- 0  0 00 00 0  0  0 
--:--:--  0:00:30 --:--:-- 0  0 00 00 0  0  0 
--:--:--  0:00:31 --:--:-- 0  0 00 00 0  0  0 
--:--:--  0:00:32 --:--:-- 0  0 00 00 0  0  0 
--:--:--  0:00:33 --:--:-- 0  0 00 00 0  0  0 
--:--:--  0:00:34 --:--:-- 0  0 00 00 0  0  0 
--:--:--  0:00:35 --:--:-- 0  0 00 00 0  0  0 
--:--:--  0:00:36 --:--:-- 0  0 00 00 0  0  0 
--:--:--  0:00:37 --:--:-- 0  0 00 00 0  0  0 
--:--:--  0:00:38 --:--:-- 0  0 00 00 0  0  0 
--:--:--  0:00:39 --:--:-- 0  0 00 00 0  0  0 
--:--:--  0:00:40 --:--:-- 0  0 00 00 0  0  0 
--:--:--  0:00:41 --:--:-- 0  0 00 00 0 

Jenkins build is back to normal : oozie-trunk-find-patches-available #166407

2017-08-02 Thread Apache Jenkins Server
See 




Build failed in Jenkins: oozie-trunk-find-patches-available #166406

2017-08-02 Thread Apache Jenkins Server
See 


--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on H1 (Hadoop) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/oozie.git # timeout=10
Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/oozie.git
 > git --version # timeout=10
 > git fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/oozie.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 4f3afcacad072792feb02d59fe1f440f1487bf00 
(refs/remotes/origin/master)
Commit message: "OOZIE-2852 Remove simple-json dependency from oozie sharelib 
(Artem Ervits via gezapeti)"
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 4f3afcacad072792feb02d59fe1f440f1487bf00
 > git rev-list 4f3afcacad072792feb02d59fe1f440f1487bf00 # timeout=10
[oozie-trunk-find-patches-available] $ /bin/bash 
/tmp/jenkins6527428919269624782.sh
mkdir: cannot create directory 
':
 File exists
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  
0 00 00 0  0  0 --:--:--  0:00:01 --:--:-- 0100 
 103k0  103k0 0  49778  0 --:--:--  0:00:02 --:--:-- 49784100  
255k0  255k0 0  80738  0 --:--:--  0:00:03 --:--:-- 80756100  
439k0  439k0 0   107k  0 --:--:--  0:00:04 --:--:--  107k100  
551k0  551k0 0   110k  0 --:--:--  0:00:04 --:--:--  110k100  
711k0  711k0 0   115k  0 --:--:--  0:00:06 --:--:--  146k100  
960k0  960k0 0   135k  0 --:--:--  0:00:07 --:--:--  173k100 
1311k0 1311k0 0   163k  0 --:--:--  0:00:08 --:--:--  220k100 
1471k0 1471k0 0   163k  0 --:--:--  0:00:09 --:--:--  210k100 
1679k0 1679k0 0   165k  0 --:--:--  0:00:10 --:--:--  219k100 
1751k0 1751k0 0   159k  0 --:--:--  0:00:10 --:--:--  215k
curl: (18) transfer closed with outstanding read data remaining
Could not retrieve available patches from JIRA
Build step 'Execute shell' marked build as failure


[jira] [Updated] (OOZIE-2897) LauncherAM should support ACLs

2017-08-02 Thread Peter Bacsko (JIRA)

 [ 
https://issues.apache.org/jira/browse/OOZIE-2897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Bacsko updated OOZIE-2897:

Description: 
In MapReduce, you can define ACL-related properties:

{noformat}
mapreduce.job.acl-view-job
mapreduce.job.acl-modify-job
{noformat}

{{acl-view-job}} defines a list of users/groups who can retrieve the job 
statistics. {{acl-modify-job}} defines a list of users/groups who can kill a 
job or adjust the priority of it.
Docs: 
https://hadoop.apache.org/docs/r1.2.1/mapred_tutorial.html#Job+Authorization

In YARN, we can provide backward compatibility for these properties. Example 
code:

{code}
Map acls = new HashMap();
acls.put(ApplicationAccessType.MODIFY_APP, "*");
acls.put(ApplicationAccessType.VIEW_APP, "*");
amContainer.setApplicationACLs(acls);
{code}

This has to be done before application submission. We have to do what 
YARNRunner.java does: 
https://github.com/apache/hadoop/blob/3721cfe1fbd98c5b6aa46aefdfcf62276c28c4a4/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java#L502-L507
 

  was:
In MapReduce, you can define ACL-related properties:

{noformat}
mapreduce.job.acl-view-job
mapreduce.job.acl-modify-job
{noformat}

{{acl-view-job}} defines a list of users/groups who can retrieve the job 
statistics. {{acl-modify-job}} defines a list of users/groups who can kill a 
job or adjust the priority of it.
Docs: 
https://hadoop.apache.org/docs/r1.2.1/mapred_tutorial.html#Job+Authorization

In YARN, we can provide backward compatibility for these properties. Example 
code:

{code}
Map acls = new HashMap();
acls.put(ApplicationAccessType.MODIFY_APP, "*");
acls.put(ApplicationAccessType.VIEW_APP, "*");
amContainer.setApplicationACLs(acls);
{code}

This has to be done before application submission. We have to open 
{{mapred-site.xml}} and check if {{mapred.acls.enabled}} is true. If so, then 
read the values of view-job/modify-job and configure the launcher context as 
described above.


> LauncherAM should support ACLs
> --
>
> Key: OOZIE-2897
> URL: https://issues.apache.org/jira/browse/OOZIE-2897
> Project: Oozie
>  Issue Type: Sub-task
>Reporter: Peter Bacsko
>
> In MapReduce, you can define ACL-related properties:
> {noformat}
> mapreduce.job.acl-view-job
> mapreduce.job.acl-modify-job
> {noformat}
> {{acl-view-job}} defines a list of users/groups who can retrieve the job 
> statistics. {{acl-modify-job}} defines a list of users/groups who can kill a 
> job or adjust the priority of it.
> Docs: 
> https://hadoop.apache.org/docs/r1.2.1/mapred_tutorial.html#Job+Authorization
> In YARN, we can provide backward compatibility for these properties. Example 
> code:
> {code}
> Map acls = new HashMap String>();
> acls.put(ApplicationAccessType.MODIFY_APP, "*");
> acls.put(ApplicationAccessType.VIEW_APP, "*");
> amContainer.setApplicationACLs(acls);
> {code}
> This has to be done before application submission. We have to do what 
> YARNRunner.java does: 
> https://github.com/apache/hadoop/blob/3721cfe1fbd98c5b6aa46aefdfcf62276c28c4a4/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java#L502-L507
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OOZIE-2983) Stream the Launcher AM Logs

2017-08-02 Thread Peter Cseh (JIRA)

[ 
https://issues.apache.org/jira/browse/OOZIE-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110794#comment-16110794
 ] 

Peter Cseh commented on OOZIE-2983:
---

This sound like a good idea.

> Stream the Launcher AM Logs
> ---
>
> Key: OOZIE-2983
> URL: https://issues.apache.org/jira/browse/OOZIE-2983
> Project: Oozie
>  Issue Type: Sub-task
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Assignee: Andras Piros
> Attachments: OOZIE-2983.001.patch
>
>
> Because OYA will get rid of the MR-based Launcher Job and have a custom AM 
> instead, the Launcher Job info and logs will no longer show up in the JHS. It 
> will show up in the ATSv2, however, the ATSv2 isn't quite ready yet and that 
> doesn't help users on older versions of Hadoop.  Launcher Jobs will stil show 
> up in the RM, but the RM doesn't remember things forever (and can be quite 
> short depending on cluster busyness and configs).
> We should have Oozie serve those logs itself (which would also be a nice 
> thing to have anyway, as the launcher logs are typically the most useful). 
> The code for reading those files is pretty easy to call. The yarn CLI, JHS, 
> and this [Aggregated Log Files 
> Exploder|https://github.com/rkanter/ALFExploder] tool I wrote all use it; the 
> latter proves that you can do it outside of Hadoop. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OOZIE-2852) Remove simple-json dependency from oozie sharelib

2017-08-02 Thread Peter Cseh (JIRA)

[ 
https://issues.apache.org/jira/browse/OOZIE-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110782#comment-16110782
 ] 

Peter Cseh commented on OOZIE-2852:
---

+1


> Remove simple-json dependency from oozie sharelib
> -
>
> Key: OOZIE-2852
> URL: https://issues.apache.org/jira/browse/OOZIE-2852
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: oya
>Reporter: Peter Cseh
>Assignee: Artem Ervits
> Fix For: oya
>
> Attachments: OOZIE-2852-2.patch
>
>
> {{MRStats}} is the only class that needs json-simple in the oozie sharelib. 
> It's only used by SqoopAcitonExecutor and MRActionExecutor. It should be 
> moved to oozie-core so the json-simple depencency can be removed from the 
> oozie sharelib.
> This is affecting only the oya branch right now.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OOZIE-2983) Stream the Launcher AM Logs

2017-08-02 Thread Andras Piros (JIRA)

[ 
https://issues.apache.org/jira/browse/OOZIE-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110783#comment-16110783
 ] 

Andras Piros commented on OOZIE-2983:
-

[~rkanter] +1 on the original approach, your patch seems pretty advanced (apart 
from TODOs). I like the idea of Oozie being able to stream the launcher AM logs 
on its own, rather than relying on a running JobHistoryServer and the client's 
capability of following redirects in the first place.

I'd be curious how Spark does that, though. [~wilfreds] can you point to the 
Spark sources where Spark does the redirect? I [might be 
missing|https://github.com/apache/spark/blob/6735433cde44b3430fb44edfff58ef8149c66c13/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala#L376-L380]
 the place.

> Stream the Launcher AM Logs
> ---
>
> Key: OOZIE-2983
> URL: https://issues.apache.org/jira/browse/OOZIE-2983
> Project: Oozie
>  Issue Type: Sub-task
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Assignee: Andras Piros
> Attachments: OOZIE-2983.001.patch
>
>
> Because OYA will get rid of the MR-based Launcher Job and have a custom AM 
> instead, the Launcher Job info and logs will no longer show up in the JHS. It 
> will show up in the ATSv2, however, the ATSv2 isn't quite ready yet and that 
> doesn't help users on older versions of Hadoop.  Launcher Jobs will stil show 
> up in the RM, but the RM doesn't remember things forever (and can be quite 
> short depending on cluster busyness and configs).
> We should have Oozie serve those logs itself (which would also be a nice 
> thing to have anyway, as the launcher logs are typically the most useful). 
> The code for reading those files is pretty easy to call. The yarn CLI, JHS, 
> and this [Aggregated Log Files 
> Exploder|https://github.com/rkanter/ALFExploder] tool I wrote all use it; the 
> latter proves that you can do it outside of Hadoop. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


Build failed in Jenkins: oozie-trunk-precommit-build #4020

2017-08-02 Thread Apache Jenkins Server
See 


Changes:

[gezapeti] OOZIE-2931 Fix warnings during build for "no @param for" (Artem 
Ervits

[asasvari] OOZIE-3021 Error on job or SLA event listening: WARN logs instead of

[asasvari] Revert "OOZIE-3021 Error on job or SLA event listening: WARN logs

[asasvari] OOZIE-3021 Error on job or SLA event listening: WARN logs instead of

--
[...truncated 3.45 MB...]
  testSLAOperations(org.apache.oozie.sla.TestSLAService)
  testBasicService(org.apache.oozie.sla.TestSLAService)
  
testCreateprotoConfWithLibPath(org.apache.oozie.service.TestLiteWorkflowAppService)
  testReadDefinition(org.apache.oozie.service.TestLiteWorkflowAppService)
  testActionNameLength(org.apache.oozie.service.TestLiteWorkflowAppService)
  testParsing(org.apache.oozie.service.TestLiteWorkflowAppService)
  testMaxWfDefinition(org.apache.oozie.service.TestLiteWorkflowAppService)
  testExtSchema(org.apache.oozie.service.TestLiteWorkflowAppService)
  
testCreateprotoConfWithMulipleLibPath(org.apache.oozie.service.TestLiteWorkflowAppService)
  
testCreateProtoConfWithSubWorkflowLib1(org.apache.oozie.service.TestLiteWorkflowAppService)
  
testCreateProtoConfWithSubWorkflowLib2(org.apache.oozie.service.TestLiteWorkflowAppService)
  
testCreateProtoConfWithSubWorkflowLib3(org.apache.oozie.service.TestLiteWorkflowAppService)
  
testCreateProtoConfWithSubWorkflowLib4(org.apache.oozie.service.TestLiteWorkflowAppService)
  
testCreateProtoConfWithSubWorkflowLib5(org.apache.oozie.service.TestLiteWorkflowAppService)
  
testCreateProtoConfWithSubWorkflowLib6(org.apache.oozie.service.TestLiteWorkflowAppService)
  testCreateprotoConf(org.apache.oozie.service.TestLiteWorkflowAppService)
  testNoAppPath(org.apache.oozie.service.TestLiteWorkflowAppService)
  testService(org.apache.oozie.service.TestLiteWorkflowAppService)
  testSchema(org.apache.oozie.service.TestLiteWorkflowAppService)
  
testCoordELActionMater(org.apache.oozie.command.coord.TestCoordELExtensions)
  testSimpleFork(org.apache.oozie.workflow.lite.TestLiteWorkflowLib)
  
testFailWithRunningNodes(org.apache.oozie.workflow.lite.TestLiteWorkflowLib)
  
testDoneWithRunningNodes(org.apache.oozie.workflow.lite.TestLiteWorkflowLib)
  testNestedFork(org.apache.oozie.workflow.lite.TestLiteWorkflowLib)
  testJobPersistance(org.apache.oozie.workflow.lite.TestLiteWorkflowLib)
  testLoopFork(org.apache.oozie.workflow.lite.TestLiteWorkflowLib)
  testEmptyWorkflow(org.apache.oozie.workflow.lite.TestLiteWorkflowLib)
  
testKillWithRunningNodes(org.apache.oozie.workflow.lite.TestLiteWorkflowLib)
  
testInvalidExecutionPath(org.apache.oozie.workflow.lite.TestLiteWorkflowLib)
  testWorkflowStates(org.apache.oozie.workflow.lite.TestLiteWorkflowLib)
  testForkedContext(org.apache.oozie.workflow.lite.TestLiteWorkflowLib)
  testSelfTransition(org.apache.oozie.workflow.lite.TestLiteWorkflowLib)
  testLoopSimple(org.apache.oozie.workflow.lite.TestLiteWorkflowLib)
  
testJobPersistanceMoreThan64K(org.apache.oozie.workflow.lite.TestLiteWorkflowLib)
  testSynchDouble(org.apache.oozie.workflow.lite.TestLiteWorkflowLib)
  
testWFKillWithRunningNodes(org.apache.oozie.workflow.lite.TestLiteWorkflowLib)
  testSynchSimple(org.apache.oozie.workflow.lite.TestLiteWorkflowLib)
  testKillWorkflow(org.apache.oozie.workflow.lite.TestLiteWorkflowLib)
  testImmediateError(org.apache.oozie.workflow.lite.TestLiteWorkflowLib)
  testActionOKError(org.apache.oozie.workflow.lite.TestLiteWorkflowLib)
  testNodeContext(org.apache.oozie.workflow.lite.TestLiteWorkflowLib)
  testDecision(org.apache.oozie.workflow.lite.TestLiteWorkflowLib)
  testAsynchSimple(org.apache.oozie.workflow.lite.TestLiteWorkflowLib)
  
testWfFailWithRunningNodes(org.apache.oozie.workflow.lite.TestLiteWorkflowLib)
  
testCoordActionGet(org.apache.oozie.executor.jpa.TestCoordActionGetForInputCheckJPAExecutor)
  testRerunFork(org.apache.oozie.command.wf.TestReRunXCommand)
  
testRerunWithExistingCoodConf(org.apache.oozie.command.wf.TestReRunXCommand)
  testRerun(org.apache.oozie.command.wf.TestReRunXCommand)
  testRedeploy(org.apache.oozie.command.wf.TestReRunXCommand)
  testRerunWithExistingConf(org.apache.oozie.command.wf.TestReRunXCommand)
  testRerunFromFailNodes(org.apache.oozie.command.wf.TestReRunXCommand)
  testRerunVariableSub(org.apache.oozie.command.wf.TestReRunXCommand)
  testRerunDisableForChild(org.apache.oozie.command.wf.TestReRunXCommand)
  testSafeMode(org.apache.oozie.servlet.TestAdminServlet)
  testPurgeServiceV2(org.apache.oozie.servlet.TestAdminServlet)
  testOsEnv(org.apache.oozie.servlet.TestAdminServlet)
  testConfiguration(org.apache.oozie.servlet.TestAdminServlet)
  

[jira] [Commented] (OOZIE-2978) Remove code that handles Pig versions before 0.8

2017-08-02 Thread Peter Cseh (JIRA)

[ 
https://issues.apache.org/jira/browse/OOZIE-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110711#comment-16110711
 ] 

Peter Cseh commented on OOZIE-2978:
---

+1

> Remove code that handles Pig versions before 0.8 
> -
>
> Key: OOZIE-2978
> URL: https://issues.apache.org/jira/browse/OOZIE-2978
> Project: Oozie
>  Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2978-01.patch, OOZIE-2978-02.patch, 
> OOZIE-2978-03.patch, OOZIE-2978-04.patch, OOZIE-2978-05.patch
>
>
> Oozie submits Pig jobs according to given Pig version. For example, Pig 
> versions before 0.8 are treated in the following way:
>  
> [Main.main(args)|https://github.com/apache/oozie/blob/678f1b2b2cc9a2456c7e3d9f693d2637f1f03257/sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java#L382]
> However, we dropped support for *Hadoop 1*, *Hadoop 0.23* support via 
> {{OOZIE-2316}}. I wonder whether submitting jobs using Pig version before 0.8 
> will work at all:
> - 
> https://pig.apache.org/releases.html#13+May%2C+2010%3A+release+0.7.0+available
>  says "This release works with Hadoop 0.20". Pig *0.10.1* is the first pig 
> release that  works with Hadoop 2.
> - Looking at [PIG-2125|https://issues.apache.org/jira/browse/PIG-2125], I 
> believe the answer is NO.  
> Therefore we can safely delete corresponding parts from the codebase. There 
> is also another class {{PigMainWithOldAPI}} that is only used by tests in 
> {{TestPigMainWithOldAPI}}. So we can remove these too.
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (OOZIE-2608) Comma in oozie.service.JPAService.jdbc.password value results in authentication error

2017-08-02 Thread Peter Cseh (JIRA)

 [ 
https://issues.apache.org/jira/browse/OOZIE-2608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Cseh updated OOZIE-2608:
--
Attachment: OOZIE-2608-002.patch

Attaching rebased patch.
Also, I've removed the "url" from the comma-separated list which will probably 
make the change introduced in OOZIE-2574 not needed any more. [~abhishekbafna], 
can you check on this issue? I don't have easy access to a replication-enabled 
MySQL.

> Comma in oozie.service.JPAService.jdbc.password value results in 
> authentication error
> -
>
> Key: OOZIE-2608
> URL: https://issues.apache.org/jira/browse/OOZIE-2608
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.2.0
> Environment: HDP 2.4 latest, Oozie 4.2.0.2.4.2.0-258
>Reporter: Nikolai Grigoriev
>Assignee: Peter Cseh
>Priority: Minor
> Attachments: OOZIE-2608-001.patch, OOZIE-2608-002.patch
>
>
> I have attempted to install Oozie with Postgres db backend via HDP 2.4. I was 
> able to test DB connection via Ambari, everything was correct. However, the 
> server failed to start with this error:
> {code}
> 2016-07-15 11:46:39,580  INFO URIHandlerService:520 - 
> SERVER[hadoop-master01.production.sociablelabs.net] Loaded default urihandler 
> org.apache.oozie.dependency.FSURIHandler
> 2016-07-15 11:46:40,323  INFO HadoopAccessorService:520 - 
> SERVER[hadoop-master01.production.sociablelabs.net] password key is 
> oozie.service.JPAService.jdbc.password
> 2016-07-15 11:46:40,331  INFO HadoopAccessorService:520 - 
> SERVER[hadoop-master01.production.sociablelabs.net] credential provider path 
> is null
> 2016-07-15 11:46:40,337  INFO HadoopAccessorService:520 - 
> SERVER[hadoop-master01.production.sociablelabs.net] password is null
> 2016-07-15 11:46:41,176 FATAL Services:514 - 
> SERVER[hadoop-master01.production.sociablelabs.net] Runtime Exception during 
> Services Load. Check your list of 'oozie.services' or 'o
> ozie.services.ext'
> 2016-07-15 11:46:41,194 FATAL Services:514 - 
> SERVER[hadoop-master01.production.sociablelabs.net] E0103: Could not load 
> service classes, Cannot create PoolableConnectionFactory (F
> ATAL: password authentication failed for user "oozie")
> org.apache.oozie.service.ServiceException: E0103: Could not load service 
> classes, Cannot create PoolableConnectionFactory (FATAL: password 
> authentication failed for user "oozie")
> at org.apache.oozie.service.Services.loadServices(Services.java:309)
> at org.apache.oozie.service.Services.init(Services.java:213)
> 
> Caused by: org.postgresql.util.PSQLException: FATAL: password authentication 
> failed for user "oozie"
> at 
> org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:291)
> at 
> org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108)
> at 
> org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)
> ...
> {code}
> Schema verification at startup worked fine and I could not find any bugs 
> related to it. So, I started getting suspicious about my password value, 
> which was like "pO,e43DpjK" (not exactly but it was a bunch of letters, 
> number and one comma). I suspected that the comma was probably interpreted as 
> a value list separator so I have changed the password removing the comma. 
> Server started flawlessly.
> I am not 100% sure if it is Ambari that should be escaping (or validating?) 
> the value or Oozie that fails to read the value as single string.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OOZIE-2608) Comma in oozie.service.JPAService.jdbc.password value results in authentication error

2017-08-02 Thread Attila Sasvari (JIRA)

[ 
https://issues.apache.org/jira/browse/OOZIE-2608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110682#comment-16110682
 ] 

Attila Sasvari commented on OOZIE-2608:
---

[~gezapeti] could you please rebase the patch? Thanks

> Comma in oozie.service.JPAService.jdbc.password value results in 
> authentication error
> -
>
> Key: OOZIE-2608
> URL: https://issues.apache.org/jira/browse/OOZIE-2608
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.2.0
> Environment: HDP 2.4 latest, Oozie 4.2.0.2.4.2.0-258
>Reporter: Nikolai Grigoriev
>Assignee: Peter Cseh
>Priority: Minor
> Attachments: OOZIE-2608-001.patch
>
>
> I have attempted to install Oozie with Postgres db backend via HDP 2.4. I was 
> able to test DB connection via Ambari, everything was correct. However, the 
> server failed to start with this error:
> {code}
> 2016-07-15 11:46:39,580  INFO URIHandlerService:520 - 
> SERVER[hadoop-master01.production.sociablelabs.net] Loaded default urihandler 
> org.apache.oozie.dependency.FSURIHandler
> 2016-07-15 11:46:40,323  INFO HadoopAccessorService:520 - 
> SERVER[hadoop-master01.production.sociablelabs.net] password key is 
> oozie.service.JPAService.jdbc.password
> 2016-07-15 11:46:40,331  INFO HadoopAccessorService:520 - 
> SERVER[hadoop-master01.production.sociablelabs.net] credential provider path 
> is null
> 2016-07-15 11:46:40,337  INFO HadoopAccessorService:520 - 
> SERVER[hadoop-master01.production.sociablelabs.net] password is null
> 2016-07-15 11:46:41,176 FATAL Services:514 - 
> SERVER[hadoop-master01.production.sociablelabs.net] Runtime Exception during 
> Services Load. Check your list of 'oozie.services' or 'o
> ozie.services.ext'
> 2016-07-15 11:46:41,194 FATAL Services:514 - 
> SERVER[hadoop-master01.production.sociablelabs.net] E0103: Could not load 
> service classes, Cannot create PoolableConnectionFactory (F
> ATAL: password authentication failed for user "oozie")
> org.apache.oozie.service.ServiceException: E0103: Could not load service 
> classes, Cannot create PoolableConnectionFactory (FATAL: password 
> authentication failed for user "oozie")
> at org.apache.oozie.service.Services.loadServices(Services.java:309)
> at org.apache.oozie.service.Services.init(Services.java:213)
> 
> Caused by: org.postgresql.util.PSQLException: FATAL: password authentication 
> failed for user "oozie"
> at 
> org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:291)
> at 
> org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108)
> at 
> org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)
> ...
> {code}
> Schema verification at startup worked fine and I could not find any bugs 
> related to it. So, I started getting suspicious about my password value, 
> which was like "pO,e43DpjK" (not exactly but it was a bunch of letters, 
> number and one comma). I suspected that the comma was probably interpreted as 
> a value list separator so I have changed the password removing the comma. 
> Server started flawlessly.
> I am not 100% sure if it is Ambari that should be escaping (or validating?) 
> the value or Oozie that fails to read the value as single string.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Comment Edited] (OOZIE-3021) Error on job or SLA event listening: WARN logs instead of DEBUG

2017-08-02 Thread Attila Sasvari (JIRA)

[ 
https://issues.apache.org/jira/browse/OOZIE-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110644#comment-16110644
 ] 

Attila Sasvari edited comment on OOZIE-3021 at 8/2/17 10:03 AM:


[~andras.piros] thanks. +1 Committed to master


was (Author: asasvari):
[~andras.piros] thanks. +1

> Error on job or SLA event listening: WARN logs instead of DEBUG
> ---
>
> Key: OOZIE-3021
> URL: https://issues.apache.org/jira/browse/OOZIE-3021
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.3.0
>Reporter: Andras Piros
>Assignee: Andras Piros
> Fix For: 5.0.0
>
> Attachments: OOZIE-3021.001.patch, OOZIE-3021.002.patch
>
>
> When handling a job or SLA event inside {{EventHandlingService}}, when a 
> {{Throwable}} happens when 
> [*{{EventWorker.run()}}*|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/service/EventHandlerService.java#L255-L265]
>  is called, only {{DEBUG}} level logs are printed.
> Since in general most users don't set Oozie log level to {{DEBUG}} while in 
> production, we should employ {{WARN}} level to get an idea when e.g. an SLA 
> event processing didn't succeed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OOZIE-3021) Error on job or SLA event listening: WARN logs instead of DEBUG

2017-08-02 Thread Attila Sasvari (JIRA)

[ 
https://issues.apache.org/jira/browse/OOZIE-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110644#comment-16110644
 ] 

Attila Sasvari commented on OOZIE-3021:
---

[~andras.piros] thanks. +1

> Error on job or SLA event listening: WARN logs instead of DEBUG
> ---
>
> Key: OOZIE-3021
> URL: https://issues.apache.org/jira/browse/OOZIE-3021
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.3.0
>Reporter: Andras Piros
>Assignee: Andras Piros
> Fix For: 5.0.0
>
> Attachments: OOZIE-3021.001.patch, OOZIE-3021.002.patch
>
>
> When handling a job or SLA event inside {{EventHandlingService}}, when a 
> {{Throwable}} happens when 
> [*{{EventWorker.run()}}*|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/service/EventHandlerService.java#L255-L265]
>  is called, only {{DEBUG}} level logs are printed.
> Since in general most users don't set Oozie log level to {{DEBUG}} while in 
> production, we should employ {{WARN}} level to get an idea when e.g. an SLA 
> event processing didn't succeed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (OOZIE-2931) Fix warnings during build for "no @param for"

2017-08-02 Thread Peter Cseh (JIRA)

[ 
https://issues.apache.org/jira/browse/OOZIE-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110512#comment-16110512
 ] 

Peter Cseh commented on OOZIE-2931:
---

+1

> Fix warnings during build for "no @param for"
> -
>
> Key: OOZIE-2931
> URL: https://issues.apache.org/jira/browse/OOZIE-2931
> Project: Oozie
>  Issue Type: Sub-task
>  Components: action, client, core
>Affects Versions: 4.3.0
>Reporter: Artem Ervits
>Assignee: Artem Ervits
>Priority: Trivial
> Fix For: trunk, 5.0.0
>
> Attachments: OOZIE-2931-0.0.patch, OOZIE-2931-0.patch, 
> OOZIE-2931-1.patch, OOZIE-2931-2.patch
>
>
> {noformat}
> ./bin/mkdistro.sh -DskipTests=true
> {noformat}
> generates many warnings for "no @param for" message. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] Subscription: Oozie Patch Available

2017-08-02 Thread jira
Issue Subscription
Filter: Oozie Patch Available (110 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-2987  Coord action missing dependencies should show URI template with 
unresolved dependencies
https://issues-test.apache.org/jira/browse/OOZIE-2987
OOZIE-2980  JAXB Proof of Concept
https://issues-test.apache.org/jira/browse/OOZIE-2980
OOZIE-2978  Remove code that handles Pig versions before 0.8 
https://issues-test.apache.org/jira/browse/OOZIE-2978
OOZIE-2977  error: self-closing element not allowed
https://issues-test.apache.org/jira/browse/OOZIE-2977
OOZIE-2975  code clean up in pig sharelib, replace Exception with more 
explicit, add try with resources, StringBuilder instead of StringBuffer
https://issues-test.apache.org/jira/browse/OOZIE-2975
OOZIE-2974  Change TestLiteWorkflowAppParser so that it works with Hadoop 3
https://issues-test.apache.org/jira/browse/OOZIE-2974
OOZIE-2969  Drop support for Java 1.7
https://issues-test.apache.org/jira/browse/OOZIE-2969
OOZIE-2964  getting at 
org.apache.maven.plugin.javadoc.AbstractJavadocMojo.executeJavadocCommandLine(AbstractJavadocMojo.java:5163)
https://issues-test.apache.org/jira/browse/OOZIE-2964
OOZIE-2963  getting error in build ArtifactNotFoundException: Could not find 
artifact org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.5-jhyde
https://issues-test.apache.org/jira/browse/OOZIE-2963
OOZIE-2962  bump maven-javadoc-plugin to 2.10.4
https://issues-test.apache.org/jira/browse/OOZIE-2962
OOZIE-2961  build contains multiple warnings for is not a parameter name
https://issues-test.apache.org/jira/browse/OOZIE-2961
OOZIE-2960  many warnings for @return tag has no arguments
https://issues-test.apache.org/jira/browse/OOZIE-2960
OOZIE-2957  Documentation states that starting a coordinator is possible
https://issues-test.apache.org/jira/browse/OOZIE-2957
OOZIE-2956  Fix Findbugs warnings related to reliance on default encoding in 
oozie-core
https://issues-test.apache.org/jira/browse/OOZIE-2956
OOZIE-2955  Fix Findbugs warnings related to reliance on default encoding in 
oozie-client
https://issues-test.apache.org/jira/browse/OOZIE-2955
OOZIE-2954  Fix Checkstyle issues in oozie-client
https://issues-test.apache.org/jira/browse/OOZIE-2954
OOZIE-2953  Fix Checkstyle issues in oozie-tools
https://issues-test.apache.org/jira/browse/OOZIE-2953
OOZIE-2952  Fix Findbugs warnings in oozie-sharelib-oozie
https://issues-test.apache.org/jira/browse/OOZIE-2952
OOZIE-2949  Escape quotes whitespaces in Sqoop  field
https://issues-test.apache.org/jira/browse/OOZIE-2949
OOZIE-2942  Fix Findbugs warnings in oozie-examples
https://issues-test.apache.org/jira/browse/OOZIE-2942
OOZIE-2937  Remove redundant groupId from the child pom's
https://issues-test.apache.org/jira/browse/OOZIE-2937
OOZIE-2934  Fix "Exceptional return value of java.io.File.mkdirs() ignored" 
Findbugs error in oozie-sharelib-spark
https://issues-test.apache.org/jira/browse/OOZIE-2934
OOZIE-2931  multiple warnings during build for "no @param for"
https://issues-test.apache.org/jira/browse/OOZIE-2931
OOZIE-2927  Append new line character for Hive2 query using query tag
https://issues-test.apache.org/jira/browse/OOZIE-2927
OOZIE-2916  Set a job name for the MR Action's child job
https://issues-test.apache.org/jira/browse/OOZIE-2916
OOZIE-2914  Consolidate Trim 
https://issues-test.apache.org/jira/browse/OOZIE-2914
OOZIE-2883  OOZIE throw the error "Missing 
[oozie.service.ProxyUserService.proxyuser.oozie.service.ProxyUserService.proxyuser.mr.groups]
 property"
https://issues-test.apache.org/jira/browse/OOZIE-2883
OOZIE-2877  Oozie Git Action
https://issues-test.apache.org/jira/browse/OOZIE-2877
OOZIE-2867  Timezone handling for Coordinators: emphasize "Continent/City" 
format
https://issues-test.apache.org/jira/browse/OOZIE-2867
OOZIE-2834  ParameterVerifier logging non-useful warning for workflow definition
https://issues-test.apache.org/jira/browse/OOZIE-2834
OOZIE-2833  when using uber mode the regex pattern used in the 
extractHeapSizeMB method does not allow heap sizes specified in bytes.
https://issues-test.apache.org/jira/browse/OOZIE-2833
OOZIE-2829  Improve sharelib upload to accept multiple source folders
https://issues-test.apache.org/jira/browse/OOZIE-2829
OOZIE-2826  Falcon feed fails to aws s3; Oozie joda time version does not meet 
required jar version 2.2 or later
https://issues-test.apache.org/jira/browse/OOZIE-2826
OOZIE-2812  SparkConfigurationService should support loading configurations 
from multiple Spark versions
https://issues-test.apache.org/jira/browse/OOZIE-2812

[jira] Subscription: Oozie Patch Available

2017-08-02 Thread jira
Issue Subscription
Filter: Oozie Patch Available (111 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-3022  fix for warning has no file and won't be listed in dependency files 
details
https://issues.apache.org/jira/browse/OOZIE-3022
OOZIE-3021  Error on job or SLA event listening: WARN logs instead of DEBUG
https://issues.apache.org/jira/browse/OOZIE-3021
OOZIE-3002  address findbugs errors in client lib
https://issues.apache.org/jira/browse/OOZIE-3002
OOZIE-3001  core library has many instances of warnings with trailing spaces 
and lines longer than 132 chars
https://issues.apache.org/jira/browse/OOZIE-3001
OOZIE-2999  minioozie can't compile
https://issues.apache.org/jira/browse/OOZIE-2999
OOZIE-2997  files contain trailing white spaces in client lib
https://issues.apache.org/jira/browse/OOZIE-2997
OOZIE-2996  add option for -UseGCOverheadLimit to maven opts as sometimes local 
testing fails
https://issues.apache.org/jira/browse/OOZIE-2996
OOZIE-2980  JAXB Proof of Concept
https://issues.apache.org/jira/browse/OOZIE-2980
OOZIE-2978  Remove code that handles Pig versions before 0.8 
https://issues.apache.org/jira/browse/OOZIE-2978
OOZIE-2975  code clean up in pig sharelib, replace Exception with more 
explicit, add try with resources, StringBuilder instead of StringBuffer
https://issues.apache.org/jira/browse/OOZIE-2975
OOZIE-2969  Drop support for Java 1.7
https://issues.apache.org/jira/browse/OOZIE-2969
OOZIE-2964  Add -Xdoclint:none to javadoc opts to avoid warnings
https://issues.apache.org/jira/browse/OOZIE-2964
OOZIE-2963  getting error in build ArtifactNotFoundException: Could not find 
artifact org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.5-jhyde
https://issues.apache.org/jira/browse/OOZIE-2963
OOZIE-2962  bump maven-javadoc-plugin to 2.10.4
https://issues.apache.org/jira/browse/OOZIE-2962
OOZIE-2957  Documentation states that starting a coordinator is possible
https://issues.apache.org/jira/browse/OOZIE-2957
OOZIE-2956  Fix Findbugs warnings related to reliance on default encoding in 
oozie-core
https://issues.apache.org/jira/browse/OOZIE-2956
OOZIE-2955  Fix Findbugs warnings related to reliance on default encoding in 
oozie-client
https://issues.apache.org/jira/browse/OOZIE-2955
OOZIE-2954  Fix Checkstyle issues in oozie-client
https://issues.apache.org/jira/browse/OOZIE-2954
OOZIE-2953  Fix Checkstyle issues in oozie-tools
https://issues.apache.org/jira/browse/OOZIE-2953
OOZIE-2952  Fix Findbugs warnings in oozie-sharelib-oozie
https://issues.apache.org/jira/browse/OOZIE-2952
OOZIE-2949  Escape quotes whitespaces in Sqoop  field
https://issues.apache.org/jira/browse/OOZIE-2949
OOZIE-2942  Fix Findbugs warnings in oozie-examples
https://issues.apache.org/jira/browse/OOZIE-2942
OOZIE-2937  Remove redundant groupId from the child pom's
https://issues.apache.org/jira/browse/OOZIE-2937
OOZIE-2934  Fix "Exceptional return value of java.io.File.mkdirs() ignored" 
Findbugs error in oozie-sharelib-spark
https://issues.apache.org/jira/browse/OOZIE-2934
OOZIE-2931  Fix warnings during build for "no @param for"
https://issues.apache.org/jira/browse/OOZIE-2931
OOZIE-2927  Append new line character for Hive2 query using query tag
https://issues.apache.org/jira/browse/OOZIE-2927
OOZIE-2916  Set a job name for the MR Action's child job
https://issues.apache.org/jira/browse/OOZIE-2916
OOZIE-2914  Consolidate Trim 
https://issues.apache.org/jira/browse/OOZIE-2914
OOZIE-2883  OOZIE throw the error "Missing 
[oozie.service.ProxyUserService.proxyuser.oozie.service.ProxyUserService.proxyuser.mr.groups]
 property"
https://issues.apache.org/jira/browse/OOZIE-2883
OOZIE-2877  Oozie Git Action
https://issues.apache.org/jira/browse/OOZIE-2877
OOZIE-2867  Timezone handling for Coordinators: emphasize "Continent/City" 
format
https://issues.apache.org/jira/browse/OOZIE-2867
OOZIE-2852  Remove simple-json dependency from oozie sharelib
https://issues.apache.org/jira/browse/OOZIE-2852
OOZIE-2834  ParameterVerifier logging non-useful warning for workflow definition
https://issues.apache.org/jira/browse/OOZIE-2834
OOZIE-2833  when using uber mode the regex pattern used in the 
extractHeapSizeMB method does not allow heap sizes specified in bytes.
https://issues.apache.org/jira/browse/OOZIE-2833
OOZIE-2829  Improve sharelib upload to accept multiple source folders
https://issues.apache.org/jira/browse/OOZIE-2829
OOZIE-2826  Falcon feed fails to aws s3; Oozie joda time version does not meet 
required jar version 2.2 or later
https://issues.apache.org/jira/browse/OOZIE-2826
OOZIE-2812