[jira] Subscription: Oozie Patch Available

[jira]

Issue Subscription
Filter: Oozie Patch Available (86 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-2619  Make  Hive action defaults to match hive defaults when running from 
command line
https://issues-test.apache.org/jira/browse/OOZIE-2619
OOZIE-2618  Include hive-common in hive-beeline as compile component for 
hive-server2 (hive2 action)
https://issues-test.apache.org/jira/browse/OOZIE-2618
OOZIE-2617  Read properties files in action configurations
https://issues-test.apache.org/jira/browse/OOZIE-2617
OOZIE-2616  Add Tez profile for Hive and Pig sharelibs
https://issues-test.apache.org/jira/browse/OOZIE-2616
OOZIE-2606  Set spark.yarn.jars to fix Spark 2.0 with Oozie
https://issues-test.apache.org/jira/browse/OOZIE-2606
OOZIE-2589  CompletedActionXCommand priority param is of no effect
https://issues-test.apache.org/jira/browse/OOZIE-2589
OOZIE-2588  Support getting credentials for same cluster hcat when credentials 
config is empty
https://issues-test.apache.org/jira/browse/OOZIE-2588
OOZIE-2585  Disable or remove test case 
TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and 
testMemoryUsageAndSpeed
https://issues-test.apache.org/jira/browse/OOZIE-2585
OOZIE-2584  Eliminate Thread.sleep() calls in TestMemoryLocks
https://issues-test.apache.org/jira/browse/OOZIE-2584
OOZIE-2583  oozie throws EL Exception when reference variable name containing 
dot
https://issues-test.apache.org/jira/browse/OOZIE-2583
OOZIE-2582  Populating external child Ids for action failures
https://issues-test.apache.org/jira/browse/OOZIE-2582
OOZIE-2573  dataset url  which contains spaces can not be handled rightly
https://issues-test.apache.org/jira/browse/OOZIE-2573
OOZIE-2572  SLA DURATION miss not shown when job is running for longer than 
expected time
https://issues-test.apache.org/jira/browse/OOZIE-2572
OOZIE-2571  Add scala.binary.version Maven property so that Scala 2.11 can be 
used
https://issues-test.apache.org/jira/browse/OOZIE-2571
OOZIE-2569  Adding yarn-site, core-site, hdfs-site and mapred-site into spark 
launcher 
https://issues-test.apache.org/jira/browse/OOZIE-2569
OOZIE-2568  SSH  action pretends to retry automaticly when it failed
https://issues-test.apache.org/jira/browse/OOZIE-2568
OOZIE-2564  Create new log4j config for unit tests so that logged messages 
contain threads
https://issues-test.apache.org/jira/browse/OOZIE-2564
OOZIE-2552  Update ActiveMQ version for security and other fixes
https://issues-test.apache.org/jira/browse/OOZIE-2552
OOZIE-2538  Update HttpClient versions to close security vulnerabilities
https://issues-test.apache.org/jira/browse/OOZIE-2538
OOZIE-2534  Versioned action libs (similar to how SharedLibs works)
https://issues-test.apache.org/jira/browse/OOZIE-2534
OOZIE-2530  Remove jline from parent and sharelib pig pom
https://issues-test.apache.org/jira/browse/OOZIE-2530
OOZIE-2528  Print out environment variables in LauncherMapper
https://issues-test.apache.org/jira/browse/OOZIE-2528
OOZIE-2521  Filter options are case sensitive for jobtye=bundle
https://issues-test.apache.org/jira/browse/OOZIE-2521
OOZIE-2518  Oozie is creating many files and directories in /tmp per day
https://issues-test.apache.org/jira/browse/OOZIE-2518
OOZIE-2517  Add support for startCreatedTime and endCreatedTime filters for 
coord and bundles
https://issues-test.apache.org/jira/browse/OOZIE-2517
OOZIE-2513  log.scan.duration should not be used for error and audit logs
https://issues-test.apache.org/jira/browse/OOZIE-2513
OOZIE-2512  ShareLibservice returns incorrect path for jar
https://issues-test.apache.org/jira/browse/OOZIE-2512
OOZIE-2508  Documentation change for Coord action rerun [OOZIE-1735]
https://issues-test.apache.org/jira/browse/OOZIE-2508
OOZIE-2501  ZK reentrant lock doesn't work for few cases
https://issues-test.apache.org/jira/browse/OOZIE-2501
OOZIE-2500  -DtestJarSimple option mentioned in minioozie doc does not work
https://issues-test.apache.org/jira/browse/OOZIE-2500
OOZIE-2499  map-reduce launcher does not need distributed files, archives 
except  jar of input/outputformat class
https://issues-test.apache.org/jira/browse/OOZIE-2499
OOZIE-2498  Oozie CallerId configuration for downstream components 
https://issues-test.apache.org/jira/browse/OOZIE-2498
OOZIE-2495  change action status from  ErrorType.NON_TRANSIENT to TRANSIENT 
when SSH action occurs AUTH_FAILED occasionally
https://issues-test.apache.org/jira/browse/OOZIE-2495
OOZIE-2491  oozie acl cannot specify group,it does`t work
https://issues-test.apache.org/jira/browse/OOZIE-2491
OOZIE

[jira] Subscription: Oozie Patch Available

[jira]

Issue Subscription
Filter: Oozie Patch Available (78 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-2694  Add logging for FsActionExecutor 
https://issues.apache.org/jira/browse/OOZIE-2694
OOZIE-2693  SimpleHCatDependencyCache.removeMissingDependency can throw NPE
https://issues.apache.org/jira/browse/OOZIE-2693
OOZIE-2692  Oozie job submit doesn't report error message to user if there is 
any issue with job conf
https://issues.apache.org/jira/browse/OOZIE-2692
OOZIE-2691  Show workflow action retry count to UI
https://issues.apache.org/jira/browse/OOZIE-2691
OOZIE-2690  OOZIE NPE while executing kill()
https://issues.apache.org/jira/browse/OOZIE-2690
OOZIE-2684  Bad database schema error for WF_ACTIONS table
https://issues.apache.org/jira/browse/OOZIE-2684
OOZIE-2674  Improve oozie commads documentation
https://issues.apache.org/jira/browse/OOZIE-2674
OOZIE-2670  Upgrade Hbase to 1.2
https://issues.apache.org/jira/browse/OOZIE-2670
OOZIE-2668  Status update and recovery problems when coord action and its 
children not in sync
https://issues.apache.org/jira/browse/OOZIE-2668
OOZIE-2666  Support embedding Jetty into Oozie
https://issues.apache.org/jira/browse/OOZIE-2666
OOZIE-2662  DB migration fails if DB is too big
https://issues.apache.org/jira/browse/OOZIE-2662
OOZIE-2658  --driver-class-path can overwrite the classpath in SparkMain
https://issues.apache.org/jira/browse/OOZIE-2658
OOZIE-2654  Zookeeper dependent services should not depend on Connectionstate 
to be valid before cleaning up
https://issues.apache.org/jira/browse/OOZIE-2654
OOZIE-2652  Skip trash while deleting hive table partition
https://issues.apache.org/jira/browse/OOZIE-2652
OOZIE-2650  Retry coord start on database exceptions
https://issues.apache.org/jira/browse/OOZIE-2650
OOZIE-2641  rerunning an oozie coordinator job stucks on waiting
https://issues.apache.org/jira/browse/OOZIE-2641
OOZIE-2630  Oozie Coordinator EL Functions to get first day of the week/month
https://issues.apache.org/jira/browse/OOZIE-2630
OOZIE-2619  Make  Hive action defaults to match hive defaults when running from 
command line
https://issues.apache.org/jira/browse/OOZIE-2619
OOZIE-2618  Include hive-common in hive-beeline as compile component for 
hive-server2 (hive2 action)
https://issues.apache.org/jira/browse/OOZIE-2618
OOZIE-2616  Add Tez profile for Hive and Pig sharelibs
https://issues.apache.org/jira/browse/OOZIE-2616
OOZIE-2613  Upgrade hive version from 0.13.1 to 1.2.0
https://issues.apache.org/jira/browse/OOZIE-2613
OOZIE-2585  Disable or remove test case 
TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and 
testMemoryUsageAndSpeed
https://issues.apache.org/jira/browse/OOZIE-2585
OOZIE-2573  dataset url  which contains spaces can not be handled rightly
https://issues.apache.org/jira/browse/OOZIE-2573
OOZIE-2568  SSH  action pretends to retry automaticly when it failed
https://issues.apache.org/jira/browse/OOZIE-2568
OOZIE-2564  Create new log4j config for unit tests so that logged messages 
contain threads
https://issues.apache.org/jira/browse/OOZIE-2564
OOZIE-2534  Versioned action libs (similar to how SharedLibs works)
https://issues.apache.org/jira/browse/OOZIE-2534
OOZIE-2528  Print out environment variables in LauncherMapper
https://issues.apache.org/jira/browse/OOZIE-2528
OOZIE-2518  Oozie is creating many files and directories in /tmp per day
https://issues.apache.org/jira/browse/OOZIE-2518
OOZIE-2513  log.scan.duration should not be used for error and audit logs
https://issues.apache.org/jira/browse/OOZIE-2513
OOZIE-2499  map-reduce launcher does not need distributed files, archives 
except  jar of input/outputformat class
https://issues.apache.org/jira/browse/OOZIE-2499
OOZIE-2495  change action status from  ErrorType.NON_TRANSIENT to TRANSIENT 
when SSH action occurs AUTH_FAILED occasionally
https://issues.apache.org/jira/browse/OOZIE-2495
OOZIE-2473  Connection pool for SMTP connection
https://issues.apache.org/jira/browse/OOZIE-2473
OOZIE-2461  Workflow, Coordinator and Bundle job querying should have last 
modified filter
https://issues.apache.org/jira/browse/OOZIE-2461
OOZIE-2458  'oozie-setup.sh sharelib create' should ensure uploaded jars are 
world readable
https://issues.apache.org/jira/browse/OOZIE-2458
OOZIE-2457  Oozie log parsing regex consume more than 90% cpu
https://issues.apache.org/jira/browse/OOZIE-2457
OOZIE-2427  With Kerberos enabled, the authentication failure for the 
javascript resources under /ext-2.2 
https://issues.apache.org/jira/browse/OOZIE-2427
O

Build failed in Jenkins: oozie-trunk-precommit-build #3355

[Apache Jenkins Server]

See 

--
[...truncated 5885 lines...]
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-tools ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-tools ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-tools ---
[INFO] Reading assembly descriptor: ../src/main/assemblies/tools.xml
[WARNING] The following patterns were never triggered in this artifact 
exclusion filter:
o  '*:*:pom:*'

[INFO] Copying files to 

[WARNING] Assembly file: 

 is not a regular file (it may be a directory). It cannot be attached to the 
project build for installation or deployment.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie MiniOozie 4.4.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-mini ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-mini 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-mini ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-mini ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-mini ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-mini ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-mini ---
[INFO] Reading assembly descriptor: src/main/assemblies/empty.xml
[INFO] 
[INFO] 
[INFO] Building Apache Oozie Distro 4.4.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-distro 
---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-distro 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-distro ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-distro ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-distro ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-distro ---
[INFO] 
[INFO] --- maven-antrun-plugin:1.6:run (default) @ oozie-distro ---
[INFO] Executing tasks

main:
  [get] Getting: 
http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.44/bin/apache-tomcat-6.0.44.tar.gz
  [get] To: 

[jira] [Commented] (OOZIE-2668) Status update and recovery problems when coord action and its children not in sync

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15537741#comment-15537741
 ] 

Hadoop QA commented on OOZIE-2668:
--

Testing JIRA OOZIE-2668

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1814
.Tests failed: 7
.Tests errors: 0

.The patch failed the following testcases:

.  
testBulkCoordSuspendResumeKillSuccess(org.apache.oozie.command.coord.TestBulkCoordXCommand)
.  
testCoordKillSuccess2(org.apache.oozie.command.coord.TestCoordKillXCommand)
.  
testCoordKillSuccess3(org.apache.oozie.command.coord.TestCoordKillXCommand)
.  
testCoordKillForBackwardSupport(org.apache.oozie.command.coord.TestCoordKillXCommand)
.  
testCoordActionInputCheckXCommandUniqueness(org.apache.oozie.command.coord.TestCoordActionInputCheckXCommandNonUTC)
.  
testNone(org.apache.oozie.command.coord.TestCoordActionInputCheckXCommand)
.  
testBundleStatusTransitServiceKilled2(org.apache.oozie.service.TestStatusTransitService)

{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/3355/

> Status update and recovery problems when coord action and its children not in 
> sync
> --
>
> Key: OOZIE-2668
> URL: https://issues.apache.org/jira/browse/OOZIE-2668
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Attachments: OOZIE-2668-1.patch, OOZIE-2668-2.patch, 
> OOZIE-2668-4.patch, OOZIE-2688-3.patch
>
>
> In cases where workflow is already in terminal status (except failed) but the 
> coord action is not yet updated and still running, following will happen if a 
> kill command is issued on the coord job: 
> Kill on Coordjob will make the kill on coordaction pending until the children 
> are also killed. However, as the wf in terminal state (except failed), the wf 
> will not be killed and preverifycondition will fail. The wf doesn't update 
> its parent and hence the coordaction kill will still be pending. Two 
> problems: Status transit service will not resolve the state of this coord job 
> as some the actions are still pending Recovery service will try to recover 
> this killed coord action and keep on reissuing the kill command.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 52227: OOZIE-2225 Add wild card filter for gathering jobs

[Sai Chirravuri]

> On Sept. 28, 2016, 11:12 a.m., Abhishek Bafna wrote:
> > client/src/main/java/org/apache/oozie/client/OozieClient.java, line 129
> > 
> >
> > Can you also test cases into the TestOozieCLI#testJobsStatus for this 
> > new filter which is being added.

TestOozieCLI#testJobsStatus uses MockDagEngineService that doesn't filter only 
on 'status'. Hence skipped this.


- Sai


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52227/#review150704
---


On Oct. 1, 2016, 12:29 a.m., Sai Chirravuri wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52227/
> ---
> 
> (Updated Oct. 1, 2016, 12:29 a.m.)
> 
> 
> Review request for oozie and Robert Kanter.
> 
> 
> Bugs: OOZIE-2225
> https://issues.apache.org/jira/browse/OOZIE-2225
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> commit 05c108f61f4c3da437d826b73251e92662a4518b
> Author: krish 
> Date:   Fri Sep 23 13:50:24 2016 -0700
> 
> OOZIE-2225 Add wild card filter for gathering jobs
> 
> :100644 100644 f1d0f2b... f10de36... M  
> client/src/main/java/org/apache/oozie/cli/OozieCLI.java
> :100644 100644 a882cab... 53a6ecf... M  
> client/src/main/java/org/apache/oozie/client/OozieClient.java
> :100644 100644 01aeddf... 7597142... M  
> core/src/main/java/org/apache/oozie/DagEngine.java
> :100644 100644 3c03267... 275aabf... M  
> core/src/main/java/org/apache/oozie/executor/jpa/WorkflowsJobGetJPAExecutor.java
> :100644 100644 fb1db29... 80f0756... M  
> core/src/main/java/org/apache/oozie/store/StoreStatusFilter.java
> :00 100644 000... dd1e943... A  
> core/src/main/java/org/apache/oozie/util/db/JPAUtils.java
> :100644 100644 0b1f644... 87fccd6... M  
> core/src/test/java/org/apache/oozie/executor/jpa/TestBundleJobInfoGetJPAExecutor.java
> :100644 100644 7d468f1... 4378286... M  
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobInfoGetJPAExecutor.java
> :100644 100644 ddb5506... 12cb16b... M  
> core/src/test/java/org/apache/oozie/executor/jpa/TestWorkflowsJobGetJPAExecutor.java
> :100644 100644 ea778bd... 2fed1f8... M  
> core/src/test/java/org/apache/oozie/test/XDataTestCase.java
> :100644 100644 567e260... 3daf38a... M  
> docs/src/site/twiki/DG_CommandLineTool.twiki
> :100644 100644 13e1691... a66e564... M  
> docs/src/site/twiki/WebServicesAPI.twiki
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 
> f1d0f2b7442e04e01bea93622cf378fc92934497 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java 
> a882cab95c4b75c3fc641fb8e5511357d482eeff 
>   core/src/main/java/org/apache/oozie/CoordinatorEngine.java 
> cec6347905233d9a5fe0eb0e2fa27336a6a967a9 
>   core/src/main/java/org/apache/oozie/DagEngine.java 
> 01aeddfc282261d9ad396565a9d3d30efec2dc59 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/WorkflowsJobGetJPAExecutor.java
>  3c032678d06641ae7bb2f2a9c0631948190ea662 
>   core/src/main/java/org/apache/oozie/store/StoreStatusFilter.java 
> fb1db29cdfe1dbe37c8e2176079f39f841232ee3 
>   core/src/main/java/org/apache/oozie/util/JobsFilterUtils.java 
> 18df836713918a2864566b56646f711dd84a5f97 
>   
> core/src/test/java/org/apache/oozie/executor/jpa/TestBundleJobInfoGetJPAExecutor.java
>  0b1f644c1ff74ce24b78eadf0ed7c0cbc87e93a3 
>   
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobInfoGetJPAExecutor.java
>  7d468f16981710d96473155df25b8de0d5ef322b 
>   
> core/src/test/java/org/apache/oozie/executor/jpa/TestWorkflowsJobGetJPAExecutor.java
>  ddb5506de90ae32d760657f8a867ab46599f1a86 
>   docs/src/site/twiki/DG_CommandLineTool.twiki 
> 567e260dd3e05c7619d957520ed49882b0cf37f2 
>   docs/src/site/twiki/WebServicesAPI.twiki 
> 13e1691c3497bed94957ef959e16e788d1196c67 
> 
> Diff: https://reviews.apache.org/r/52227/diff/
> 
> 
> Testing
> ---
> 
> CLI manual testing looks good and unit tests passed
> 
> 
> Thanks,
> 
> Sai Chirravuri
> 
>

Re: Review Request 52227: OOZIE-2225 Add wild card filter for gathering jobs

[Sai Chirravuri]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52227/
---

(Updated Oct. 1, 2016, 12:29 a.m.)


Review request for oozie and Robert Kanter.


Bugs: OOZIE-2225
https://issues.apache.org/jira/browse/OOZIE-2225


Repository: oozie-git


Description
---

commit 05c108f61f4c3da437d826b73251e92662a4518b
Author: krish 
Date:   Fri Sep 23 13:50:24 2016 -0700

OOZIE-2225 Add wild card filter for gathering jobs

:100644 100644 f1d0f2b... f10de36... M  
client/src/main/java/org/apache/oozie/cli/OozieCLI.java
:100644 100644 a882cab... 53a6ecf... M  
client/src/main/java/org/apache/oozie/client/OozieClient.java
:100644 100644 01aeddf... 7597142... M  
core/src/main/java/org/apache/oozie/DagEngine.java
:100644 100644 3c03267... 275aabf... M  
core/src/main/java/org/apache/oozie/executor/jpa/WorkflowsJobGetJPAExecutor.java
:100644 100644 fb1db29... 80f0756... M  
core/src/main/java/org/apache/oozie/store/StoreStatusFilter.java
:00 100644 000... dd1e943... A  
core/src/main/java/org/apache/oozie/util/db/JPAUtils.java
:100644 100644 0b1f644... 87fccd6... M  
core/src/test/java/org/apache/oozie/executor/jpa/TestBundleJobInfoGetJPAExecutor.java
:100644 100644 7d468f1... 4378286... M  
core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobInfoGetJPAExecutor.java
:100644 100644 ddb5506... 12cb16b... M  
core/src/test/java/org/apache/oozie/executor/jpa/TestWorkflowsJobGetJPAExecutor.java
:100644 100644 ea778bd... 2fed1f8... M  
core/src/test/java/org/apache/oozie/test/XDataTestCase.java
:100644 100644 567e260... 3daf38a... M  
docs/src/site/twiki/DG_CommandLineTool.twiki
:100644 100644 13e1691... a66e564... M  docs/src/site/twiki/WebServicesAPI.twiki


Diffs (updated)
-

  client/src/main/java/org/apache/oozie/cli/OozieCLI.java 
f1d0f2b7442e04e01bea93622cf378fc92934497 
  client/src/main/java/org/apache/oozie/client/OozieClient.java 
a882cab95c4b75c3fc641fb8e5511357d482eeff 
  core/src/main/java/org/apache/oozie/CoordinatorEngine.java 
cec6347905233d9a5fe0eb0e2fa27336a6a967a9 
  core/src/main/java/org/apache/oozie/DagEngine.java 
01aeddfc282261d9ad396565a9d3d30efec2dc59 
  
core/src/main/java/org/apache/oozie/executor/jpa/WorkflowsJobGetJPAExecutor.java
 3c032678d06641ae7bb2f2a9c0631948190ea662 
  core/src/main/java/org/apache/oozie/store/StoreStatusFilter.java 
fb1db29cdfe1dbe37c8e2176079f39f841232ee3 
  core/src/main/java/org/apache/oozie/util/JobsFilterUtils.java 
18df836713918a2864566b56646f711dd84a5f97 
  
core/src/test/java/org/apache/oozie/executor/jpa/TestBundleJobInfoGetJPAExecutor.java
 0b1f644c1ff74ce24b78eadf0ed7c0cbc87e93a3 
  
core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobInfoGetJPAExecutor.java
 7d468f16981710d96473155df25b8de0d5ef322b 
  
core/src/test/java/org/apache/oozie/executor/jpa/TestWorkflowsJobGetJPAExecutor.java
 ddb5506de90ae32d760657f8a867ab46599f1a86 
  docs/src/site/twiki/DG_CommandLineTool.twiki 
567e260dd3e05c7619d957520ed49882b0cf37f2 
  docs/src/site/twiki/WebServicesAPI.twiki 
13e1691c3497bed94957ef959e16e788d1196c67 

Diff: https://reviews.apache.org/r/52227/diff/


Testing
---

CLI manual testing looks good and unit tests passed


Thanks,

Sai Chirravuri

Re: Review Request 52373: OOZIE-2688 YAML representation of Oozie unit testcase

[Satish Saley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52373/#review151099
---




yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/GenerateBatchJsonMojo.java
 (line 19)


Can we rename it to org.apache.oozie.yamlunit? 
We are already under "oozie", so prefix "oozie" in "oozieyamlunit" not 
needed.


- Satish Saley


On Sept. 28, 2016, 5:53 p.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52373/
> ---
> 
> (Updated Sept. 28, 2016, 5:53 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2688
> https://issues.apache.org/jira/browse/OOZIE-2688
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Oozie YAML Unit test
> 
> Maven configuration
> YAML configuration
> Declaration
> Import
> setup
> Initialization
> Job Definition
> HDFS operation
> HCat operation
> Job stubbing
> Usage Guideline
> validation
> output validation
> job validation
> Usage Guideline
> TearUp
> Tag
> Examples
> Bundle Test
> Coordinator test
> Coordinator test with HCat
> Workflow Test
> Workflow Test with skip Job submission
> API
> Maven configuration
> 
> To enable oozie unit yaml, one need add oozie-yaml plugin to maven.
> 
> 
> org.apache.oozie
> oozie-yaml
> 0.1-SNAPSHOT
> 
> 
> 
> oozieyaml
> 
> 
> 
> 
> Oozie-YAML plugin to Maven will search for any oozie YAML file and if it's 
> there, it will convert it into respective JUnit test case and will execute 
> them. There will be a surefire report.
> YAML configuration
> 
> Whole YAML configuration is divided into following sections
> 
> Declaration
> Import
> Setup
> Initialization
> Job Definition
> Stubbing
> HDFS operation
> HCat operation
> Validation
> TearUp
> Tag
> Declaration
> 
> This section can be used to defined test case properties like test case name, 
> package name, etc.
> 
> name: String, required
> Name of the test cases. Java file will be created with the same name.
> package: String, Optional
> The Java package of the generated test class.
> skip_test : boolean, Optional
> A boolean that indicates whether or not to execute this test. This parameter 
> defaults to false.
> copy_dependencies_files_hdfs: List of Strings, Optional
> List of HDFS of paths where all Maven dependent jar will be copied. This is 
> useful if for pig/hive jobs.
> Examples:
> 
> package: org.apache.oozie.yamltest
> name: TestWorkflowActionFromBundle
> copy_dependencies_files_hdfs:
>- /tmp/workflow-test/lib
> Import
> 
> imports: List of Strings, Optional
> A list of Java classes to import in the generated Oozie unit Java file. * 
> Examples: *
>  import:
>   - org.apache.oozie.util.DateUtils
>   - java.util.Date
> setup
> 
> setup: List of Strings, Optional
> List of java statement that can be used for JUnit setup method.
> setup can also include HDFS and HCat operation.
> Initialization
> 
> initialization: List of Strings, Optional
> List of java statement that can be used for initialization. This statement 
> will be added first in test case.
> Examples:
> 
> initialization:
>   - "Date date = new Date()"
>   - "String currentTime = DateUtils.formatDateOozieTZ(date)"
> Job Definition
> 
> This section can be used to define job properties
> 
> job_property: String, Optional
> job_property can be a Java Properties file (.properties) or a Hadoop XML 
> Configuration file (.xml).
> properties: String, Optional
> Overridden key/value configuration use in job submission. This value 
> overrides the value of job_property.
> Examples:
> 
> job:
> properties:
> raw_logs_path: "/tmp/test"
> aggregated_logs_path: "/user/test"
> workflowPath: "/testDirPath"
> start: "2010-02-01T10:00Z"
> job:
> job_property : src/test/resources/job-1.properties
> properties:
> oozie.wf.application.path: "/tmp/workflow-test/"
> jobTracker: !func getJobTrackerUri()
> nameNode: !func getNameNodeUri()
> HDFS operation
> 
> Set of HDFS operation. Can be used to create dummy input dataset for 
> coordinator or input file for workflow. Supported operation are
> 
> mkdir: List, Optional
> The mkdir command creates the specified directory. If the directory already 
> exists, it does a no-op.
> touchz: List, Optional
> Touchz creates a zero length file in the specified path if none exists. If 
> one already exists, then touchz will perform a touch operation.
> delete: List, Optional
> List of file/dir to be deleted.
> delete: List, Optional
> Deletes the specified path.
> chmod: List, Optional
> The chmod command changes the permissi

[jira] [Commented] (OOZIE-2668) Status update and recovery problems when coord action and its children not in sync

[Satish Subhashrao Saley (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15537274#comment-15537274
 ] 

Satish Subhashrao Saley commented on OOZIE-2668:


{quote}
 We should only increment pending when a command is queued.
{quote}
- Done
- Fixing the test failure


> Status update and recovery problems when coord action and its children not in 
> sync
> --
>
> Key: OOZIE-2668
> URL: https://issues.apache.org/jira/browse/OOZIE-2668
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Attachments: OOZIE-2668-1.patch, OOZIE-2668-2.patch, 
> OOZIE-2668-4.patch, OOZIE-2688-3.patch
>
>
> In cases where workflow is already in terminal status (except failed) but the 
> coord action is not yet updated and still running, following will happen if a 
> kill command is issued on the coord job: 
> Kill on Coordjob will make the kill on coordaction pending until the children 
> are also killed. However, as the wf in terminal state (except failed), the wf 
> will not be killed and preverifycondition will fail. The wf doesn't update 
> its parent and hence the coordaction kill will still be pending. Two 
> problems: Status transit service will not resolve the state of this coord job 
> as some the actions are still pending Recovery service will try to recover 
> this killed coord action and keep on reissuing the kill command.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2668) Status update and recovery problems when coord action and its children not in sync

[Satish Subhashrao Saley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Satish Subhashrao Saley updated OOZIE-2668:
---
Attachment: OOZIE-2668-4.patch

> Status update and recovery problems when coord action and its children not in 
> sync
> --
>
> Key: OOZIE-2668
> URL: https://issues.apache.org/jira/browse/OOZIE-2668
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Attachments: OOZIE-2668-1.patch, OOZIE-2668-2.patch, 
> OOZIE-2668-4.patch, OOZIE-2688-3.patch
>
>
> In cases where workflow is already in terminal status (except failed) but the 
> coord action is not yet updated and still running, following will happen if a 
> kill command is issued on the coord job: 
> Kill on Coordjob will make the kill on coordaction pending until the children 
> are also killed. However, as the wf in terminal state (except failed), the wf 
> will not be killed and preverifycondition will fail. The wf doesn't update 
> its parent and hence the coordaction kill will still be pending. Two 
> problems: Status transit service will not resolve the state of this coord job 
> as some the actions are still pending Recovery service will try to recover 
> this killed coord action and keep on reissuing the kill command.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build is back to normal : oozie-trunk-precommit-build #3354

[Apache Jenkins Server]

See 

[jira] [Commented] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15537238#comment-15537238
 ] 

Hadoop QA commented on OOZIE-2613:
--

Testing JIRA OOZIE-2613

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch does adds/modifies 6 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1813
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:green}*+1 Overall result, good!, no -1s*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/3354/

> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch, OOZIE-2613-05.patch, 
> OOZIE-2613-06.patch, OOZIE-2613-07.patch, OOZIE-2613-08.patch, 
> OOZIE-2613-09.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Oozie going forward

[Mohammad Islam]

Very good Rohini.+1 from me as well.
 

On Friday, September 30, 2016 1:11 PM, Rohini Palaniswamy 
 wrote:
 

 Abhishek,  I did rename the 4.4.0 version to 5.0.0 now. Work on a feature 
branch will still be marked with the "oya" version. When it is merged, then the 
5.0.0 version can be added in addition to the "oya" version. This is usually 
what is done for work in feature branches.
Peter,  It will have to be 4) We create a patch file containing all changes and 
apply it.   You will have to issue a review request with all the changes in the 
oya branch for review by the wider community before committing it to master.
Regards,Rohini

On Fri, Sep 30, 2016 at 1:00 PM, Abhishek Bafna  wrote:

Hi Robert,

Can you please create a fix version 5.0.0, so that OYA related stuff can be 
tagged correctly.

Thanks,
Abhishek
> On Oct 1, 2016, at 12:13 AM, Mohammad Islam  
> wrote:
>
> Robert,Very good plan!
> How other  products (pig/hive etc) are handling the Hadoop 1 issue?If they 
> are planning in the same way, this will be easy for us.
> Regards,Mohammad
>
>
>
>
>    On Thursday, September 29, 2016 5:56 PM, Robert Kanter 
> wrote:
>
>
> Hi,
>
> Now that Oozie 4.3 is branched and will be happening soon, I imagine the
> next release we do with be Oozie 5 with Oozie On Yarn.  We can continue to
> keep OYA in a feature branch until later when it's more stable so we don't
> disrupt the master branch in the meantime.  We've already removed 0.23 and
> made Hadoop 2 the default.  Removing Hadoop 1 will simplify things, and is
> currently in the way of other tasks like upgrading HBase.  I imagine it
> will continue to do so more as time goes on.
>
> I'd like to propose the following:
>
>   1. Oozie 4.3 is the last release to support Hadoop 1.  If we want to do
>   another Oozie 4.x release later, we can branch it from branch-4.3 instead
>   of trunk.  Though I expect 4.3 will be the last minor release of Oozie 4.
>   2. Oozie 5 (the master branch; aka trunk) will be the first release to
>   support only Hadoop 2 and Hadoop 3 (these should be much more similar than
>   Hadoop 1 to 2 was; I expect things will work as-is by just changing the
>   Hadoop version so we probably won't even need profiles).  We'll have Oozie
>   On Yarn as the marquee feature.
>   3. We'll drop Hadoop 1 from the master branch.  We can reuse parts
>   of OOZIE-2316.
>   4. The master branch will become 5.0.0-SNAPSHOT instead of
>   4.4.0-SNAPSHOT.
>
> I'd like us to take care of #3 soon, because it's getting in the way.  We
> might as well also take care of #4 soon too.
>
> Thoughts?
>
>
> thanks
> - Robert
>
>





   

[jira] [Commented] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

[Rohini Palaniswamy (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536981#comment-15536981
 ] 

Rohini Palaniswamy commented on OOZIE-2613:
---

Will take a look later today. Had missed all the earlier questions mentioned 
with my name. Just found out today that gmail has been filtering most of the 
"XXX mentioned you" jira emails to the Promotions tab and I have been missing 
them for more than a year. Sigh :(. 

> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch, OOZIE-2613-05.patch, 
> OOZIE-2613-06.patch, OOZIE-2613-07.patch, OOZIE-2613-08.patch, 
> OOZIE-2613-09.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2696) Fix NPE when is provided

[Rohini Palaniswamy (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536924#comment-15536924
 ] 

Rohini Palaniswamy commented on OOZIE-2696:
---

Can you add more details to the jira?

> Fix NPE when  is provided
> --
>
> Key: OOZIE-2696
> URL: https://issues.apache.org/jira/browse/OOZIE-2696
> Project: Oozie
>  Issue Type: Sub-task
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

[Peter Cseh (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536929#comment-15536929
 ] 

Peter Cseh commented on OOZIE-2613:
---

Yaay, I'm glad it fixed the other tests as well :)

I'll reach out for some Hive experts on Monday, but they are not familiar with 
our MiniHCat stuff, so they might not be able to help much.

> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch, OOZIE-2613-05.patch, 
> OOZIE-2613-06.patch, OOZIE-2613-07.patch, OOZIE-2613-08.patch, 
> OOZIE-2613-09.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Oozie going forward

[Rohini Palaniswamy]

Abhishek,
  I did rename the 4.4.0 version to 5.0.0 now. Work on a feature branch
will still be marked with the "oya" version. When it is merged, then the
5.0.0 version can be added in addition to the "oya" version. This is
usually what is done for work in feature branches.

Peter,
  It will have to be 4) We create a patch file containing all changes and
apply it.   You will have to issue a review request with all the changes in
the oya branch for review by the wider community before committing it to
master.

Regards,
Rohini

On Fri, Sep 30, 2016 at 1:00 PM, Abhishek Bafna 
wrote:

> Hi Robert,
>
> Can you please create a fix version 5.0.0, so that OYA related stuff can
> be tagged correctly.
>
> Thanks,
> Abhishek
> > On Oct 1, 2016, at 12:13 AM, Mohammad Islam 
> wrote:
> >
> > Robert,Very good plan!
> > How other  products (pig/hive etc) are handling the Hadoop 1 issue?If
> they are planning in the same way, this will be easy for us.
> > Regards,Mohammad
> >
> >
> >
> >
> >On Thursday, September 29, 2016 5:56 PM, Robert Kanter <
> rkan...@cloudera.com> wrote:
> >
> >
> > Hi,
> >
> > Now that Oozie 4.3 is branched and will be happening soon, I imagine the
> > next release we do with be Oozie 5 with Oozie On Yarn.  We can continue
> to
> > keep OYA in a feature branch until later when it's more stable so we
> don't
> > disrupt the master branch in the meantime.  We've already removed 0.23
> and
> > made Hadoop 2 the default.  Removing Hadoop 1 will simplify things, and
> is
> > currently in the way of other tasks like upgrading HBase.  I imagine it
> > will continue to do so more as time goes on.
> >
> > I'd like to propose the following:
> >
> >   1. Oozie 4.3 is the last release to support Hadoop 1.  If we want to do
> >   another Oozie 4.x release later, we can branch it from branch-4.3
> instead
> >   of trunk.  Though I expect 4.3 will be the last minor release of Oozie
> 4.
> >   2. Oozie 5 (the master branch; aka trunk) will be the first release to
> >   support only Hadoop 2 and Hadoop 3 (these should be much more similar
> than
> >   Hadoop 1 to 2 was; I expect things will work as-is by just changing the
> >   Hadoop version so we probably won't even need profiles).  We'll have
> Oozie
> >   On Yarn as the marquee feature.
> >   3. We'll drop Hadoop 1 from the master branch.  We can reuse parts
> >   of OOZIE-2316.
> >   4. The master branch will become 5.0.0-SNAPSHOT instead of
> >   4.4.0-SNAPSHOT.
> >
> > I'd like us to take care of #3 soon, because it's getting in the way.  We
> > might as well also take care of #4 soon too.
> >
> > Thoughts?
> >
> >
> > thanks
> > - Robert
> >
> >
>
>

[jira] [Commented] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536918#comment-15536918
 ] 

Abhishek Bafna commented on OOZIE-2613:
---

This fix works for other two testcases as well. The issue was that 
table/partition get create with the current system user and our testcase was 
using the 'test' user and they were failing due to permission issue. 

I agree for the point to do more investigation and fix them properly. If the 
lasted patch {{-09}} works, then we can decide whether we want to move ahead 
with it or not. Thanks for the feedback.

> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch, OOZIE-2613-05.patch, 
> OOZIE-2613-06.patch, OOZIE-2613-07.patch, OOZIE-2613-08.patch, 
> OOZIE-2613-09.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Oozie going forward

[Rohini Palaniswamy]

+1. Pig 0.16 was the last release to support Hadoop 1.x. The next release
will not support it. Hive has already dropped it from the Hive 2.x releases.

On Fri, Sep 30, 2016 at 11:43 AM, Mohammad Islam  wrote:

> Robert,Very good plan!
> How other  products (pig/hive etc) are handling the Hadoop 1 issue?If they
> are planning in the same way, this will be easy for us.
> Regards,Mohammad
>
>
>
>
> On Thursday, September 29, 2016 5:56 PM, Robert Kanter <
> rkan...@cloudera.com> wrote:
>
>
>  Hi,
>
> Now that Oozie 4.3 is branched and will be happening soon, I imagine the
> next release we do with be Oozie 5 with Oozie On Yarn.  We can continue to
> keep OYA in a feature branch until later when it's more stable so we don't
> disrupt the master branch in the meantime.  We've already removed 0.23 and
> made Hadoop 2 the default.  Removing Hadoop 1 will simplify things, and is
> currently in the way of other tasks like upgrading HBase.  I imagine it
> will continue to do so more as time goes on.
>
> I'd like to propose the following:
>
>   1. Oozie 4.3 is the last release to support Hadoop 1.  If we want to do
>   another Oozie 4.x release later, we can branch it from branch-4.3 instead
>   of trunk.  Though I expect 4.3 will be the last minor release of Oozie 4.
>   2. Oozie 5 (the master branch; aka trunk) will be the first release to
>   support only Hadoop 2 and Hadoop 3 (these should be much more similar
> than
>   Hadoop 1 to 2 was; I expect things will work as-is by just changing the
>   Hadoop version so we probably won't even need profiles).  We'll have
> Oozie
>   On Yarn as the marquee feature.
>   3. We'll drop Hadoop 1 from the master branch.  We can reuse parts
>   of OOZIE-2316.
>   4. The master branch will become 5.0.0-SNAPSHOT instead of
>   4.4.0-SNAPSHOT.
>
> I'd like us to take care of #3 soon, because it's getting in the way.  We
> might as well also take care of #4 soon too.
>
> Thoughts?
>
>
> thanks
> - Robert
>
>
>
>

[jira] [Commented] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

[Peter Cseh (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536906#comment-15536906
 ] 

Peter Cseh commented on OOZIE-2613:
---

I'm not sure setting this is the right way to fix it. If we decide to do it 
anyway as the problem is with the test, not the actual code, please file a 
follow up jira to get to the bottom of this testing issue. There are still two 
more test cases which I haven't had the chance to look at, so this fix might 
not be sufficient.

> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch, OOZIE-2613-05.patch, 
> OOZIE-2613-06.patch, OOZIE-2613-07.patch, OOZIE-2613-08.patch, 
> OOZIE-2613-09.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2613:
--
Attachment: OOZIE-2613-09.patch

> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch, OOZIE-2613-05.patch, 
> OOZIE-2613-06.patch, OOZIE-2613-07.patch, OOZIE-2613-08.patch, 
> OOZIE-2613-09.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Oozie going forward

[Abhishek Bafna]

Hi Robert,

Can you please create a fix version 5.0.0, so that OYA related stuff can be 
tagged correctly.

Thanks,
Abhishek
> On Oct 1, 2016, at 12:13 AM, Mohammad Islam  
> wrote:
> 
> Robert,Very good plan!
> How other  products (pig/hive etc) are handling the Hadoop 1 issue?If they 
> are planning in the same way, this will be easy for us.
> Regards,Mohammad
> 
> 
> 
> 
>On Thursday, September 29, 2016 5:56 PM, Robert Kanter 
>  wrote:
> 
> 
> Hi,
> 
> Now that Oozie 4.3 is branched and will be happening soon, I imagine the
> next release we do with be Oozie 5 with Oozie On Yarn.  We can continue to
> keep OYA in a feature branch until later when it's more stable so we don't
> disrupt the master branch in the meantime.  We've already removed 0.23 and
> made Hadoop 2 the default.  Removing Hadoop 1 will simplify things, and is
> currently in the way of other tasks like upgrading HBase.  I imagine it
> will continue to do so more as time goes on.
> 
> I'd like to propose the following:
> 
>   1. Oozie 4.3 is the last release to support Hadoop 1.  If we want to do
>   another Oozie 4.x release later, we can branch it from branch-4.3 instead
>   of trunk.  Though I expect 4.3 will be the last minor release of Oozie 4.
>   2. Oozie 5 (the master branch; aka trunk) will be the first release to
>   support only Hadoop 2 and Hadoop 3 (these should be much more similar than
>   Hadoop 1 to 2 was; I expect things will work as-is by just changing the
>   Hadoop version so we probably won't even need profiles).  We'll have Oozie
>   On Yarn as the marquee feature.
>   3. We'll drop Hadoop 1 from the master branch.  We can reuse parts
>   of OOZIE-2316.
>   4. The master branch will become 5.0.0-SNAPSHOT instead of
>   4.4.0-SNAPSHOT.
> 
> I'd like us to take care of #3 soon, because it's getting in the way.  We
> might as well also take care of #4 soon too.
> 
> Thoughts?
> 
> 
> thanks
> - Robert
> 
> 

[jira] [Updated] (OOZIE-2684) Bad database schema error for WF_ACTIONS table

[Robert Kanter (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Kanter updated OOZIE-2684:
-
Fix Version/s: (was: 4.3.0)
   4.4.0

> Bad database schema error for WF_ACTIONS table
> --
>
> Key: OOZIE-2684
> URL: https://issues.apache.org/jira/browse/OOZIE-2684
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.4.0
>
> Attachments: OOZIE-2684-00.patch
>
>
> In {{SchemaCheckerService}}, Oozie compares expected and found indexed 
> columns and generates the below error message which could be confusing to the 
> users.
> {noformat}
> 2016-09-16 12:39:26,564  INFO SchemaCheckXCommand:520 - 
> SERVER[c6402.ambari.apache.org] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] 
> ACTION[-] About to check database schema
> 2016-09-16 12:39:26,703 ERROR SchemaCheckXCommand:517 - 
> SERVER[c6402.ambari.apache.org] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] 
> ACTION[-] Found [1] extra indexes for columns in table [WF_ACTIONS]: [wf_id]
> 2016-09-16 12:39:26,723 ERROR SchemaCheckXCommand:517 - 
> SERVER[c6402.ambari.apache.org] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] 
> ACTION[-] Database schema is BAD! Check previous error log messages for 
> details
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-1986) Add FindBugs report to pre-commit build

[Robert Kanter (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Kanter updated OOZIE-1986:
-
Fix Version/s: (was: 4.3.0)
   4.4.0

> Add FindBugs report to pre-commit build
> ---
>
> Key: OOZIE-1986
> URL: https://issues.apache.org/jira/browse/OOZIE-1986
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
>Reporter: Robert Kanter
>Assignee: Andras Piros
> Fix For: 4.4.0
>
> Attachments: OOZIE-1986.001.patch, OOZIE-1986.002.patch
>
>
> Now that we have OOZIE-1793, it will be really useful to have the pre-commit 
> build generate a findbugs report.  It should probably before and after the 
> patch and only complain it there are more after.  And it should link to the 
> report.  Ideally, this would do some kind of diff (like Hadoop does) so it's 
> easy to identify the new findbugs items.
> You can generate the findbugs reports by running:
> {noformat}
> mvn verify -DskipTests
> {noformat}
> This also runs checkstyle



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2634) Queue dump command message is confusing when the queue is empty

[Robert Kanter (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536859#comment-15536859
 ] 

Robert Kanter commented on OOZIE-2634:
--

+1

> Queue dump command message is confusing when the queue is empty
> ---
>
> Key: OOZIE-2634
> URL: https://issues.apache.org/jira/browse/OOZIE-2634
> Project: Oozie
>  Issue Type: Improvement
>Reporter: Robert Kanter
>Assignee: Andras Piros
>Priority: Minor
>  Labels: newbie
> Fix For: 4.4.0
>
> Attachments: OOZIE-2634.001.patch, OOZIE-2634.002.patch, 
> OOZIE-2634.003.patch, OOZIE-2634.004.patch
>
>
> If the callable queue is empty, and you run the queue dump command, the 
> message says:
> {noformat}
> # oozie admin queuedump
> [Server Queue Dump]:
> Queue dump is null!
> **
> [Server Uniqueness Map Dump]:
> Uniqueness dump is null!
> {noformat}
> This message makes it sound like a bad thing, especially the exclamation 
> mark.  We should change the message to something more neutral and helpful, 
> like "The queue is empty" or something like that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536857#comment-15536857
 ] 

Abhishek Bafna commented on OOZIE-2613:
---

[~rkanter] i.e. a great method, I guess. I will test it and then submit a 
patch, If it works. :)

> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch, OOZIE-2613-05.patch, 
> OOZIE-2613-06.patch, OOZIE-2613-07.patch, OOZIE-2613-08.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

[Robert Kanter (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536830#comment-15536830
 ] 

Robert Kanter commented on OOZIE-2613:
--

If you use {{XTestCase#setSystemProperty}} to do it, then Oozie will 
automatically take care of that :)

> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch, OOZIE-2613-05.patch, 
> OOZIE-2613-06.patch, OOZIE-2613-07.patch, OOZIE-2613-08.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536818#comment-15536818
 ] 

Abhishek Bafna commented on OOZIE-2613:
---

[~gezapeti] If we set this into the system property then it will be set for the 
rest of the testcase execution, which can cause the other testcases to fail. 
May be we can reset it to its original value at the end of the test. cc: 
[~rkanter]

> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch, OOZIE-2613-05.patch, 
> OOZIE-2613-06.patch, OOZIE-2613-07.patch, OOZIE-2613-08.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

[Peter Cseh (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536790#comment-15536790
 ] 

Peter Cseh commented on OOZIE-2613:
---

It's easier to add the line 
{code}System.setProperty(XTestCase.TEST_USER1_PROP, ); 
{code} 
to {{testDeleteHCat}} to test this. This will make the test to delete the table 
as your current user and will make the test pass.

[~rkanter], I could not change the user of {{MiniHCatServer}}, but I'm sure 
it's possible. I'll look into this more on Monday.


> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch, OOZIE-2613-05.patch, 
> OOZIE-2613-06.patch, OOZIE-2613-07.patch, OOZIE-2613-08.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

[Robert Kanter (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536777#comment-15536777
 ] 

Robert Kanter commented on OOZIE-2613:
--

I could be wrong here, but I imagine that in a production environment, the HCat 
Server would be running as the 'hive' user, and the clients would be running as 
the actual user.  So the HCat Server should allow for doing things as other 
users, but in our setup, it seems like it does things only as the HCat Server 
user, which is the OS user.  Perhaps we need to look into how HCat Client 
determines the user and use that to run the client as 'test' and then hopefully 
the HCat Server will do things on behalf of 'test'?

> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch, OOZIE-2613-05.patch, 
> OOZIE-2613-06.patch, OOZIE-2613-07.patch, OOZIE-2613-08.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

[Peter Cseh (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536726#comment-15536726
 ] 

Peter Cseh commented on OOZIE-2613:
---

I managed to fix {{TestFsActionExecutor#testDeleteHcat}} by using my OS user 
name, {{gezapeti}}, in {{FsActionExecutor#delete}} instead of the user 
{{test}}. But this "fix" would actually break things in production.

The problem is with the test:
The table is created by {{MiniHCatServer}} and it's {{gezapeti}} who creates 
the table, and user {{test}} don't have the permissions to delete it. 
Unfortunately, I could not find a way to make {{MiniHCatServer}} to use the 
user {{test}} or give permissions to drop table to that user. Anyone has any 
ideas? [~abhishekbafna]?



> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch, OOZIE-2613-05.patch, 
> OOZIE-2613-06.patch, OOZIE-2613-07.patch, OOZIE-2613-08.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2622) ExtJS 2.2 is no longer available

[Robert Kanter (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536713#comment-15536713
 ] 

Robert Kanter commented on OOZIE-2622:
--

We already got a +1 from [~jaydeepvishwakarma] so [~rohini] doesn't need to 
bother.
I'll commit this shortly.

> ExtJS 2.2 is no longer available
> 
>
> Key: OOZIE-2622
> URL: https://issues.apache.org/jira/browse/OOZIE-2622
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 4.2.0
>Reporter: Sergey Svinarchuk
>Assignee: Robert Kanter
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2622.001.patch
>
>
> ExtJS 2.2 is no longer available by link from Oozie docs - 
> http://dev.sencha.com/deploy/ext-2.2.zip 
> (http://oozie.apache.org/docs/4.2.0/DG_QuickStart.html)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Oozie going forward

[Mohammad Islam]

Robert,Very good plan!
How other  products (pig/hive etc) are handling the Hadoop 1 issue?If they are 
planning in the same way, this will be easy for us.
Regards,Mohammad


 

On Thursday, September 29, 2016 5:56 PM, Robert Kanter 
 wrote:
 

 Hi,

Now that Oozie 4.3 is branched and will be happening soon, I imagine the
next release we do with be Oozie 5 with Oozie On Yarn.  We can continue to
keep OYA in a feature branch until later when it's more stable so we don't
disrupt the master branch in the meantime.  We've already removed 0.23 and
made Hadoop 2 the default.  Removing Hadoop 1 will simplify things, and is
currently in the way of other tasks like upgrading HBase.  I imagine it
will continue to do so more as time goes on.

I'd like to propose the following:

  1. Oozie 4.3 is the last release to support Hadoop 1.  If we want to do
  another Oozie 4.x release later, we can branch it from branch-4.3 instead
  of trunk.  Though I expect 4.3 will be the last minor release of Oozie 4.
  2. Oozie 5 (the master branch; aka trunk) will be the first release to
  support only Hadoop 2 and Hadoop 3 (these should be much more similar than
  Hadoop 1 to 2 was; I expect things will work as-is by just changing the
  Hadoop version so we probably won't even need profiles).  We'll have Oozie
  On Yarn as the marquee feature.
  3. We'll drop Hadoop 1 from the master branch.  We can reuse parts
  of OOZIE-2316.
  4. The master branch will become 5.0.0-SNAPSHOT instead of
  4.4.0-SNAPSHOT.

I'd like us to take care of #3 soon, because it's getting in the way.  We
might as well also take care of #4 soon too.

Thoughts?


thanks
- Robert


   

[jira] [Commented] (OOZIE-2690) OOZIE NPE while executing kill()

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536704#comment-15536704
 ] 

Abhishek Bafna commented on OOZIE-2690:
---

[~gezapeti] I wanted to write a testcase before submitting the patch, But do 
not know how to fail the job submission (JobClient#submitJob) for one action 
and not for the other action. We found this issue, during testing due to a 
issue into other component. The underlined issue was also intermittent in 
nature. That is the reason, I provided the scenario as comment for clarity of 
the issue and fix provided for it. Is there anything you will can be done, 
please do let me know. Thanks.

> OOZIE NPE while executing kill()
> 
>
> Key: OOZIE-2690
> URL: https://issues.apache.org/jira/browse/OOZIE-2690
> Project: Oozie
>  Issue Type: Bug
>Reporter: Cheng Xu
>Assignee: Abhishek Bafna
> Attachments: OOZIE-2690-00.patch
>
>
> ION[027-160917110703953-oozie-oozi-W@java2] Exception while executing 
> kill(). Error Code [NullPointerException], Message[NullPointerException: null]
> org.apache.oozie.action.ActionExecutorException: NullPointerException: null
> at 
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
> at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.kill(JavaActionExecutor.java:1632)
> at 
> org.apache.oozie.command.wf.ActionKillXCommand.execute(ActionKillXCommand.java:146)
> at 
> org.apache.oozie.command.wf.ActionKillXCommand.execute(ActionKillXCommand.java:62)
> at org.apache.oozie.command.XCommand.call(XCommand.java:287)
> at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:331)
> at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:260)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:178)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.mapreduce.TypeConverter.toYarn(TypeConverter.java:79)
> at 
> org.apache.hadoop.mapred.ClientServiceDelegate.(ClientServiceDelegate.java:120)
> at org.apache.hadoop.mapred.ClientCache.getClient(ClientCache.java:68)
> at org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:573)
> at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:184)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:601)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:599)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
> at 
> org.apache.hadoop.mapred.JobClient.getJobUsingCluster(JobClient.java:599)
> at org.apache.hadoop.mapred.JobClient.getJobInner(JobClient.java:609)
> at org.apache.hadoop.mapred.JobClient.getJob(JobClient.java:639)
> at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.getJobClientWithRetries(JavaActionExecutor.java:1432)
> at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.getRunningJob(JavaActionExecutor.java:1420)
> at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.kill(JavaActionExecutor.java:1623)
> ... 10 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2622) ExtJS 2.2 is no longer available

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536664#comment-15536664
 ] 

Abhishek Bafna commented on OOZIE-2622:
---

[~rohini] Can you please take a look at this? Thanks.

> ExtJS 2.2 is no longer available
> 
>
> Key: OOZIE-2622
> URL: https://issues.apache.org/jira/browse/OOZIE-2622
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 4.2.0
>Reporter: Sergey Svinarchuk
>Assignee: Robert Kanter
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2622.001.patch
>
>
> ExtJS 2.2 is no longer available by link from Oozie docs - 
> http://dev.sencha.com/deploy/ext-2.2.zip 
> (http://oozie.apache.org/docs/4.2.0/DG_QuickStart.html)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-1986) Add FindBugs report to pre-commit build

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536563#comment-15536563
 ] 

Hadoop QA commented on OOZIE-1986:
--

Testing JIRA OOZIE-1986

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 7 line(s) longer than 132 
characters
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1812
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/3353/

> Add FindBugs report to pre-commit build
> ---
>
> Key: OOZIE-1986
> URL: https://issues.apache.org/jira/browse/OOZIE-1986
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
>Reporter: Robert Kanter
>Assignee: Andras Piros
> Fix For: 4.3.0
>
> Attachments: OOZIE-1986.001.patch, OOZIE-1986.002.patch
>
>
> Now that we have OOZIE-1793, it will be really useful to have the pre-commit 
> build generate a findbugs report.  It should probably before and after the 
> patch and only complain it there are more after.  And it should link to the 
> report.  Ideally, this would do some kind of diff (like Hadoop does) so it's 
> easy to identify the new findbugs items.
> You can generate the findbugs reports by running:
> {noformat}
> mvn verify -DskipTests
> {noformat}
> This also runs checkstyle



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: oozie-trunk-precommit-build #3353

[Apache Jenkins Server]

See 

--
[...truncated 5873 lines...]
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-tools ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 11 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-tools ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-tools ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-tools ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-tools ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-tools ---
[INFO] Reading assembly descriptor: ../src/main/assemblies/tools.xml
[WARNING] The following patterns were never triggered in this artifact 
exclusion filter:
o  '*:*:pom:*'

[INFO] Copying files to 

[WARNING] Assembly file: 

 is not a regular file (it may be a directory). It cannot be attached to the 
project build for installation or deployment.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie MiniOozie 4.4.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-mini ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-mini 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-mini ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-mini ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-mini ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-mini ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-mini ---
[INFO] Reading assembly descriptor: src/main/assemblies/empty.xml
[INFO] 
[INFO] 
[INFO] Building Apache Oozie Distro 4.4.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-distro 
---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-distro 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-distro ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-distro ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @

Jenkins build is back to normal : oozie-trunk-find-patches-available #134828

[Apache Jenkins Server]

See 

Build failed in Jenkins: oozie-trunk-find-patches-available #134827

[Apache Jenkins Server]

See 

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on H1 (yahoo-not-h2 Hadoop Mapreduce Hdfs Pig) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/oozie.git # timeout=10
Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/oozie.git
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/oozie.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 3eca3c2bdecb61c995c2cf2180345cbc56ffb8a5 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 3eca3c2bdecb61c995c2cf2180345cbc56ffb8a5
 > git rev-list 3eca3c2bdecb61c995c2cf2180345cbc56ffb8a5 # timeout=10
[oozie-trunk-find-patches-available] $ /bin/bash 
/tmp/hudson2584038103195789071.sh
mkdir: cannot create directory 
':
 File exists
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
100 713520 713520 0   109k  0 --:--:-- --:--:-- --:--:--  109k
100  403k0  403k0 0   235k  0 --:--:--  0:00:01 --:--:--  235k
100  693k0  693k0 0   262k  0 --:--:--  0:00:02 --:--:--  262k
100 1140k0 1140k0 0   308k  0 --:--:--  0:00:03 --:--:--  308k
100 1360k0 1360k0 0   306k  0 --:--:--  0:00:04 --:--:--  306k
curl: (18) transfer closed with outstanding read data remaining
Could not retrieve available patches from JIRA
Build step 'Execute shell' marked build as failure

[jira] [Commented] (OOZIE-1986) Add FindBugs report to pre-commit build

[Andras Piros (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536355#comment-15536355
 ] 

Andras Piros commented on OOZIE-1986:
-

Hi Robert,

thanks for your feedback!

My thoughts:
* Good idea, just did it :)
* The output will be shown in the report later on (when this new {{.sh}} file 
will also run) and there will be diffs generated like 
{{core/findbugs-new.html}} but I cannot tell how to make that filenames inside 
the reports clickable since I cannot tell the real path of the generated file 
inside JIRA. But these will also appear in the generated folder of the Jenkins 
job
* Nice catch on {{shellcheck}}, using it. The actual patchset doesn't have any 
warnings anymore with this marvellous tool

> Add FindBugs report to pre-commit build
> ---
>
> Key: OOZIE-1986
> URL: https://issues.apache.org/jira/browse/OOZIE-1986
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
>Reporter: Robert Kanter
>Assignee: Andras Piros
> Fix For: 4.3.0
>
> Attachments: OOZIE-1986.001.patch, OOZIE-1986.002.patch
>
>
> Now that we have OOZIE-1793, it will be really useful to have the pre-commit 
> build generate a findbugs report.  It should probably before and after the 
> patch and only complain it there are more after.  And it should link to the 
> report.  Ideally, this would do some kind of diff (like Hadoop does) so it's 
> easy to identify the new findbugs items.
> You can generate the findbugs reports by running:
> {noformat}
> mvn verify -DskipTests
> {noformat}
> This also runs checkstyle



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-1986) Add FindBugs report to pre-commit build

[Andras Piros (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros updated OOZIE-1986:

Attachment: OOZIE-1986.002.patch

{{shellcheck}} modifications.

> Add FindBugs report to pre-commit build
> ---
>
> Key: OOZIE-1986
> URL: https://issues.apache.org/jira/browse/OOZIE-1986
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
>Reporter: Robert Kanter
>Assignee: Andras Piros
> Fix For: 4.3.0
>
> Attachments: OOZIE-1986.001.patch, OOZIE-1986.002.patch
>
>
> Now that we have OOZIE-1793, it will be really useful to have the pre-commit 
> build generate a findbugs report.  It should probably before and after the 
> patch and only complain it there are more after.  And it should link to the 
> report.  Ideally, this would do some kind of diff (like Hadoop does) so it's 
> easy to identify the new findbugs items.
> You can generate the findbugs reports by running:
> {noformat}
> mvn verify -DskipTests
> {noformat}
> This also runs checkstyle



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 52429: OOZIE-1986 Add FindBugs report to pre-commit build

[András Piros]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52429/
---

Review request for oozie and Robert Kanter.


Repository: oozie-git


Description
---

We go for that solution:

1. create a new script as `bin/test-patch-11-findbugs-diff` so that it comes 
right after compile

2. `mvn clean verify -DskipTests` before and after applying the patch, and 
saving each and every module's `target/findbugs/findbugsXml.xml` to a secure 
folder

3. call the `findbugs-diff` app for all the before / after FindBugs xml pairs

4. analyze all the `findbugs-new.xml` entries for nodes with XPath 
`/BugCollection/BugInstance` entries. Stop on any BugInstance w/ `priority` and 
/ or `rank` lower than threshold

5. have the reports merged and be downloadable in any case in following 
scenarios:

* -1 for bugs that are too serious to apply the patch with them
* 0 for other new bugs
* +1 if there is no new bug


Diffs
-

  bin/test-patch-11-findbugs-diff PRE-CREATION 
  pom.xml 704a2ee 

Diff: https://reviews.apache.org/r/52429/diff/


Testing
---


Thanks,

András Piros

[jira] [Updated] (OOZIE-1986) Add FindBugs report to pre-commit build

[Andras Piros (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros updated OOZIE-1986:

Summary: Add FindBugs report to pre-commit build  (was: Add findbugs report 
to pre-commit build)

> Add FindBugs report to pre-commit build
> ---
>
> Key: OOZIE-1986
> URL: https://issues.apache.org/jira/browse/OOZIE-1986
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
>Reporter: Robert Kanter
>Assignee: Andras Piros
> Fix For: 4.3.0
>
> Attachments: OOZIE-1986.001.patch
>
>
> Now that we have OOZIE-1793, it will be really useful to have the pre-commit 
> build generate a findbugs report.  It should probably before and after the 
> patch and only complain it there are more after.  And it should link to the 
> report.  Ideally, this would do some kind of diff (like Hadoop does) so it's 
> easy to identify the new findbugs items.
> You can generate the findbugs reports by running:
> {noformat}
> mvn verify -DskipTests
> {noformat}
> This also runs checkstyle



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (OOZIE-2699) Refactor JavaActionExecutor and executor class hierarchy

[Peter Bacsko (JIRA)]

Peter Bacsko created OOZIE-2699:
---

 Summary: Refactor JavaActionExecutor and executor class hierarchy
 Key: OOZIE-2699
 URL: https://issues.apache.org/jira/browse/OOZIE-2699
 Project: Oozie
  Issue Type: Sub-task
Reporter: Peter Bacsko


Currently JavaActionExecutor has _way_ too much responsibility. A lot of things 
inside the class are absolutely unrelated to the Java action itself.

Not only that, the class is too big. A lot of code should be moved to somewhere 
else, eg. submitting the action to YARN should be handled separately in a 
different class.

If implemented properly, old and new classes will be more testable and code 
coverage will increase.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: oozie-trunk-precommit-build #3352

[Apache Jenkins Server]

See 

--
[...truncated 6641 lines...]
[INFO] Copying commons-beanutils-core-1.8.0.jar to 

[INFO] Copying hadoop-yarn-server-common-2.4.0.jar to 

[INFO] Copying geronimo-j2ee-management_1.1_spec-1.0.1.jar to 

[INFO] Copying datanucleus-core-3.2.10.jar to 

[INFO] Copying slf4j-log4j12-1.6.6.jar to 

[INFO] Copying aopalliance-1.0.jar to 

[INFO] Copying hadoop-mapreduce-client-core-2.4.0.jar to 

[INFO] Copying datanucleus-api-jdo-3.2.6.jar to 

[INFO] Copying jung-api-2.0.1.jar to 

[INFO] Copying jta-1.1.jar to 

[INFO] Copying jung-graph-impl-2.0.1.jar to 

[INFO] Copying hive-metastore-0.13.1.jar to 

[INFO] Copying jung-visualization-2.0.1.jar to 

[INFO] Copying hadoop-client-2.4.0.jar to 

[INFO] Copying hsqldb-1.8.0.10.jar to 

[INFO] Copying jackson-annotations-2.2.2.jar to 

[INFO] Copying openjpa-jdbc-2.4.1.jar to 

[INFO] Copying apache-jstl-9.2.19.v20160908.pom to 

[INFO] Copying jsr305-1.3.9.jar to 

[INFO] Copying avro-1.7.4.jar to 

[INFO] Copying jaxb-api-2.2.2.jar to 

[INFO] Copying jetty-webapp-9.2.19.v20160908.jar to 

[INFO] Copying jetty-security-9.2.19.v20160908.jar to 

[INFO] Copying jackson-jaxrs-1.8.3.jar to 

[INFO] Copying hive-common-0.13.1.jar to 

[INFO] Copying metrics-ganglia-3.1.2.jar to 

[INFO] Copying hadoop-auth-2.4.0.jar to 

[INFO] Copying asm-tree-5.0.1.jar to 

[INFO] Copying geronimo-jpa_2.0_spec-1.1.jar to 

[INFO] Copying commons-collections-3.2.1.jar to 

[INFO] Copying jersey-client-1.9.jar to 

[jira] [Commented] (OOZIE-2666) Support embedding Jetty into Oozie

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535906#comment-15535906
 ] 

Hadoop QA commented on OOZIE-2666:
--

Testing JIRA OOZIE-2666

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:red}-1{color} the patch contains 12 line(s) with tabs
.{color:red}-1{color} the patch contains 2 line(s) with trailing spaces
.{color:red}-1{color} the patch contains 12 line(s) longer than 132 
characters
.{color:green}+1{color} the patch does adds/modifies 2 testcase(s)
{color:red}-1 RAT{color}
.{color:red}-1{color} the patch seems to introduce 3 new RAT warning(s)
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:red}-1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:red}-1{color} the patch seems to introduce 3 new javac warning(s)
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1820
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/3352/

> Support embedding Jetty into Oozie
> --
>
> Key: OOZIE-2666
> URL: https://issues.apache.org/jira/browse/OOZIE-2666
> Project: Oozie
>  Issue Type: Sub-task
>  Components: core
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2666.PATCH-01.patch, OOZIE-2666.PATCH-02.patch, 
> OOZIE-2666.PATCH-03.patch
>
>
> The purpose of this task is to allow Oozie to run as a standalone application 
> rather than being deployed to a container inside a WAR file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (OOZIE-2698) Refactor LauncherAM to make it more testable

[Peter Bacsko (JIRA)]

Peter Bacsko created OOZIE-2698:
---

 Summary: Refactor LauncherAM to make it more testable
 Key: OOZIE-2698
 URL: https://issues.apache.org/jira/browse/OOZIE-2698
 Project: Oozie
  Issue Type: Sub-task
Reporter: Peter Bacsko
Assignee: Peter Bacsko






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (OOZIE-2697) Implement proper UGI calls in LauncherAM for secure/non-secure clusters

[Peter Bacsko (JIRA)]

Peter Bacsko created OOZIE-2697:
---

 Summary: Implement proper UGI calls in LauncherAM for 
secure/non-secure clusters
 Key: OOZIE-2697
 URL: https://issues.apache.org/jira/browse/OOZIE-2697
 Project: Oozie
  Issue Type: Sub-task
Reporter: Peter Bacsko
Assignee: Peter Bacsko






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (OOZIE-2696) Fix NPE when is provided

[Peter Bacsko (JIRA)]

Peter Bacsko created OOZIE-2696:
---

 Summary: Fix NPE when  is provided
 Key: OOZIE-2696
 URL: https://issues.apache.org/jira/browse/OOZIE-2696
 Project: Oozie
  Issue Type: Sub-task
Reporter: Peter Bacsko
Assignee: Peter Bacsko






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (OOZIE-2597) OYA: Sqoop Action

[Peter Bacsko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Bacsko resolved OOZIE-2597.
-
Resolution: Fixed

> OYA: Sqoop Action
> -
>
> Key: OOZIE-2597
> URL: https://issues.apache.org/jira/browse/OOZIE-2597
> Project: Oozie
>  Issue Type: Sub-task
>Affects Versions: oya
>Reporter: Robert Kanter
>Assignee: Peter Bacsko
> Fix For: oya
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (OOZIE-2594) OYA: MapReduce Action

[Peter Bacsko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Bacsko resolved OOZIE-2594.
-
Resolution: Fixed

> OYA: MapReduce Action
> -
>
> Key: OOZIE-2594
> URL: https://issues.apache.org/jira/browse/OOZIE-2594
> Project: Oozie
>  Issue Type: Sub-task
>Affects Versions: oya
>Reporter: Robert Kanter
>Assignee: Peter Bacsko
> Fix For: oya
>
> Attachments: OOZIE-2594.001.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (OOZIE-2595) OYA: Pig Action

[Peter Bacsko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Bacsko resolved OOZIE-2595.
-
Resolution: Fixed

> OYA: Pig Action
> ---
>
> Key: OOZIE-2595
> URL: https://issues.apache.org/jira/browse/OOZIE-2595
> Project: Oozie
>  Issue Type: Sub-task
>Affects Versions: oya
>Reporter: Robert Kanter
>Assignee: Peter Bacsko
> Fix For: oya
>
> Attachments: OOZIE-2595.01.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (OOZIE-2593) OYA: DistCp Action

[Peter Bacsko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Bacsko resolved OOZIE-2593.
-
Resolution: Fixed

> OYA: DistCp Action
> --
>
> Key: OOZIE-2593
> URL: https://issues.apache.org/jira/browse/OOZIE-2593
> Project: Oozie
>  Issue Type: Sub-task
>Affects Versions: oya
>Reporter: Robert Kanter
>Assignee: Peter Bacsko
> Fix For: oya
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (OOZIE-2591) OYA: Java Action

[Peter Bacsko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Bacsko resolved OOZIE-2591.
-
Resolution: Fixed

> OYA: Java Action
> 
>
> Key: OOZIE-2591
> URL: https://issues.apache.org/jira/browse/OOZIE-2591
> Project: Oozie
>  Issue Type: Sub-task
>Affects Versions: oya
>Reporter: Robert Kanter
>Assignee: Peter Cseh
> Fix For: oya
>
> Attachments: OOZIE-2591.01.patch, OOZIE-2591.02.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2666) Support embedding Jetty into Oozie

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535787#comment-15535787
 ] 

Hadoop QA commented on OOZIE-2666:
--

Testing JIRA OOZIE-2666

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:red}-1{color} the patch contains 12 line(s) with tabs
.{color:red}-1{color} the patch contains 2 line(s) with trailing spaces
.{color:red}-1{color} the patch contains 12 line(s) longer than 132 
characters
.{color:green}+1{color} the patch does adds/modifies 2 testcase(s)
{color:red}-1 RAT{color}
.{color:red}-1{color} the patch seems to introduce 3 new RAT warning(s)
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:red}-1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:red}-1{color} the patch seems to introduce 3 new javac warning(s)
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1820
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/3351/

> Support embedding Jetty into Oozie
> --
>
> Key: OOZIE-2666
> URL: https://issues.apache.org/jira/browse/OOZIE-2666
> Project: Oozie
>  Issue Type: Sub-task
>  Components: core
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2666.PATCH-01.patch, OOZIE-2666.PATCH-02.patch, 
> OOZIE-2666.PATCH-03.patch
>
>
> The purpose of this task is to allow Oozie to run as a standalone application 
> rather than being deployed to a container inside a WAR file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: oozie-trunk-precommit-build #3351

[Apache Jenkins Server]

See 

--
[...truncated 6667 lines...]
[INFO] Copying commons-beanutils-core-1.8.0.jar to 

[INFO] Copying hadoop-yarn-server-common-2.4.0.jar to 

[INFO] Copying geronimo-j2ee-management_1.1_spec-1.0.1.jar to 

[INFO] Copying datanucleus-core-3.2.10.jar to 

[INFO] Copying slf4j-log4j12-1.6.6.jar to 

[INFO] Copying aopalliance-1.0.jar to 

[INFO] Copying hadoop-mapreduce-client-core-2.4.0.jar to 

[INFO] Copying datanucleus-api-jdo-3.2.6.jar to 

[INFO] Copying jung-api-2.0.1.jar to 

[INFO] Copying jta-1.1.jar to 

[INFO] Copying jung-graph-impl-2.0.1.jar to 

[INFO] Copying hive-metastore-0.13.1.jar to 

[INFO] Copying jung-visualization-2.0.1.jar to 

[INFO] Copying hadoop-client-2.4.0.jar to 

[INFO] Copying hsqldb-1.8.0.10.jar to 

[INFO] Copying jackson-annotations-2.2.2.jar to 

[INFO] Copying openjpa-jdbc-2.4.1.jar to 

[INFO] Copying apache-jstl-9.2.19.v20160908.pom to 

[INFO] Copying jsr305-1.3.9.jar to 

[INFO] Copying avro-1.7.4.jar to 

[INFO] Copying jaxb-api-2.2.2.jar to 

[INFO] Copying jetty-webapp-9.2.19.v20160908.jar to 

[INFO] Copying jetty-security-9.2.19.v20160908.jar to 

[INFO] Copying jackson-jaxrs-1.8.3.jar to 

[INFO] Copying hive-common-0.13.1.jar to 

[INFO] Copying metrics-ganglia-3.1.2.jar to 

[INFO] Copying hadoop-auth-2.4.0.jar to 

[INFO] Copying asm-tree-5.0.1.jar to 

[INFO] Copying geronimo-jpa_2.0_spec-1.1.jar to 

[INFO] Copying commons-collections-3.2.1.jar to 

[INFO] Copying jersey-client-1.9.jar to 

[jira] [Commented] (OOZIE-2634) Queue dump command message is confusing when the queue is empty

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535712#comment-15535712
 ] 

Hadoop QA commented on OOZIE-2634:
--

Testing JIRA OOZIE-2634

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1812
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:green}*+1 Overall result, good!, no -1s*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/3350/

> Queue dump command message is confusing when the queue is empty
> ---
>
> Key: OOZIE-2634
> URL: https://issues.apache.org/jira/browse/OOZIE-2634
> Project: Oozie
>  Issue Type: Improvement
>Reporter: Robert Kanter
>Assignee: Andras Piros
>Priority: Minor
>  Labels: newbie
> Fix For: 4.4.0
>
> Attachments: OOZIE-2634.001.patch, OOZIE-2634.002.patch, 
> OOZIE-2634.003.patch, OOZIE-2634.004.patch
>
>
> If the callable queue is empty, and you run the queue dump command, the 
> message says:
> {noformat}
> # oozie admin queuedump
> [Server Queue Dump]:
> Queue dump is null!
> **
> [Server Uniqueness Map Dump]:
> Uniqueness dump is null!
> {noformat}
> This message makes it sound like a bad thing, especially the exclamation 
> mark.  We should change the message to something more neutral and helpful, 
> like "The queue is empty" or something like that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build is back to normal : oozie-trunk-precommit-build #3350

[Apache Jenkins Server]

See 

Build failed in Jenkins: oozie-trunk-precommit-build #3349

[Apache Jenkins Server]

See 

--
[...truncated 5899 lines...]
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-tools ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-tools ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-tools ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-tools ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-tools ---
[INFO] Reading assembly descriptor: ../src/main/assemblies/tools.xml
[WARNING] The following patterns were never triggered in this artifact 
exclusion filter:
o  '*:*:pom:*'

[INFO] Copying files to 

[WARNING] Assembly file: 

 is not a regular file (it may be a directory). It cannot be attached to the 
project build for installation or deployment.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie MiniOozie 4.4.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-mini ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-mini 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-mini ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-mini ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-mini ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-mini ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-mini ---
[INFO] Reading assembly descriptor: src/main/assemblies/empty.xml
[INFO] 
[INFO] 
[INFO] Building Apache Oozie Distro 4.4.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-distro 
---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-distro 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-distro ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-distro ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-distro ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-distro ---
[INFO] 
[INFO] --- 

[jira] [Commented] (OOZIE-1814) Oozie should mask any passwords in logs and REST interfaces

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535701#comment-15535701
 ] 

Hadoop QA commented on OOZIE-1814:
--

Testing JIRA OOZIE-1814

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 11 line(s) longer than 132 
characters
.{color:green}+1{color} the patch does adds/modifies 3 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1814
.Tests failed: 0
.Tests errors: 1

.The patch failed the following testcases:

.  

{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/3349/

> Oozie should mask any passwords in logs and REST interfaces
> ---
>
> Key: OOZIE-1814
> URL: https://issues.apache.org/jira/browse/OOZIE-1814
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: trunk
>Reporter: Bowen Zhang
>Assignee: Andras Piros
>  Labels: newbie
> Fix For: 4.3.0
>
> Attachments: OOZIE-1814.002.patch, OOZIE-1814.003.patch, 
> OOZIE-1814.004.patch, OOZIE-1814.005.patch, OOZIE-1814.006.patch, 
> OOZIE-1814.007.patch, oozie-1814.patch
>
>
> Following passwords are currently visible in the instrumentation log, REST 
> endpoints, WebUI, and CLI (WebUI and CLI simply call the REST endpoints):
> * {{javax.net.ssl.trustStorePassword}}
> * {{oozie.https.keystore.pass}}
> * {{HADOOP_CREDSTORE_PASSWORD}}
> * {{OOZIE_HTTPS_KEYSTORE_PASSWORD}}
> * {{OOZIE_HTTPS_TRUSTSTORE_PASSWORD}}
> There are a few examples that illustrate password leakage.
> {noformat}
> # grep -i pass /var/log/oozie/oozie-instrumentation.log
> OOZIE_HTTPS_TRUSTSTORE_PASSWORD = password
> javax.net.ssl.trustStorePassword = password
> oozie.https.keystore.pass = password
> HADOOP_CREDSTORE_PASSWORD = password
> OOZIE_HTTPS_KEYSTORE_PASSWORD = password
> CATALINA_OPTS = -Xms603979776 -Xmx603979776 
> -XX:+HeapDumpOnOutOfMemoryError 
> -XX:HeapDumpPath=/tmp/OOZIE-1_OOZIE-1-OOZIE_SERVER-2e75cc1293d9058eef7250a18f347c43_pid30867.hprof
>  -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh 
> -Doozie.home.dir=/usr/lib/oozie 
> -Doozie.config.dir=/var/run/cloudera-scm-agent/process/320-oozie-OOZIE_SERVER 
> -Doozie.log.dir=/var/log/oozie 
> -Doozie.log.file=oozie-cmf-OOZIE-1-OOZIE_SERVER-nightly-1.gce.cloudera.com.log.out
>  -Doozie.config.file=oozie-site.xml -Doozie.log4j.file=log4j.properties 
> -Doozie.log4j.reload=10 -Doozie.http.hostname=nightly-1.gce.cloudera.com 
> -Doozie.http.port=11000 -Djava.net.preferIPv4Stack=true 
> -Doozie.admin.port=11001 -Dderby.stream.error.file=/var/log/oozie/derby.log 
> -Doozie.instance.id=nightly-1.gce.cloudera.com 
> -Djava.library.path=/usr/lib/hadoop/lib/native -Doozie.https.port=11443 
> -Djavax.net.ssl.trustStore=/etc/cdep-ssl-conf/CA_STANDARD/truststore.jks 
> -Djavax.net.ssl.trustStorePassword=password
> {noformat}
> Oozie dumps the env vars and Java sys props to the instrumentation log on 
> startup.
> {noformat}
> # curl --negotiate -u foo:bar -k 
> https://nightly-1.gce.cloudera.com:11443/oozie/v2/admin/os-env | python -m 
> json.tool | grep -i pass
> "CATALINA_OPTS": "-Xms603979776 -Xmx603979776 -XX:+HeapDumpOnOutOfMemoryError 
> -XX:HeapDumpPath=/tmp/OOZIE-1_OOZIE-1-OOZIE_SERVER-2e75cc1293d9058eef7250a18f347c43_pid30867.hprof
>  -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh 
> -Doozie.home.dir=/usr/lib/oozie 
> -Doozie.config.dir=/var/run/cloudera-scm-agent/process/320-oozie-OOZIE_SERVER 
> -Doozie.log.dir=/var/log/oozie 
> -Doozie.log.file=oozie-cmf-OOZIE-1-OOZIE_SERVE

[jira] [Updated] (OOZIE-2666) Support embedding Jetty into Oozie

[Attila Sasvari (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Sasvari updated OOZIE-2666:
--
Attachment: OOZIE-2666.PATCH-03.patch

Refactor oozied.sh

> Support embedding Jetty into Oozie
> --
>
> Key: OOZIE-2666
> URL: https://issues.apache.org/jira/browse/OOZIE-2666
> Project: Oozie
>  Issue Type: Sub-task
>  Components: core
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2666.PATCH-01.patch, OOZIE-2666.PATCH-02.patch, 
> OOZIE-2666.PATCH-03.patch
>
>
> The purpose of this task is to allow Oozie to run as a standalone application 
> rather than being deployed to a container inside a WAR file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 52399: OOZIE-2666 Support embedding Jetty into Oozie

[Attila Sasvari]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52399/
---

(Updated Sept. 30, 2016, 10:45 a.m.)


Review request for oozie, Peter Cseh, Peter Bacsko, and Robert Kanter.


Changes
---

Refactor oozied.sh - move jetty related things to new script file


Repository: oozie-git


Description
---

Embedding jetty into Oozie so that it can run as a standalone application. The 
changes also try to address OOZIE-2317 (i.e. Tomcat 6 is EOL).

New functionality
- New module (server) is added that sets up an embedded Jetty server and start 
Oozie services. Servlet mapping is done by reading web.xml of webapp at 
runtime. JSP is handled with custom code. Server version is not revealed in 
server repsonses.
- SSL protocols and ciphers can be configured via system properties and 
environment variables. Precedence: system properties, environment variables, 
default values
   
Changes in existing code
- Excluded jetty 6 dependencies from core and updated tests accordingly  

Packaging
- oozie.sh is modified so that it starts Oozie with embedded jetty by default. 
If someone would like to use tomcat for any reason, they can set an environment 
variable (e.g. OOZIE_USE_TOMCAT=1).

TODO:
- Add more tests
- Add more documentation
- Code cleanup + refactoring in packaging and core parts
- Maven clean up
- Allow to tune more Jetty settings (for example threadpool)
- More security measures (e.g. protect against clickjacking, CSRF, etc.)
- Update Oozie Documentation


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/util/Instrumentation.java 
fa1e92a0f1fadfb66c5e66fea5f26f57579080e7 
  distro/src/main/bin/oozie-jetty-server.sh PRE-CREATION 
  distro/src/main/bin/oozied.sh a869c3da177c863a068f2af45c7bca9d5cb771ac 
  pom.xml 704a29f4e4805e3e08c2a547b2a375f6b1b2 
  server/pom.xml PRE-CREATION 
  server/src/main/assemblies/empty.xml PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java 
PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/JspHandler.java PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/SSLServerConnectorFactory.java 
PRE-CREATION 
  server/src/main/resources/checkstyle-header.txt PRE-CREATION 
  server/src/main/resources/checkstyle.xml PRE-CREATION 
  server/src/test/java/org/apache/oozie/server/TestEmbeddedOozieServer.java 
PRE-CREATION 
  
server/src/test/java/org/apache/oozie/server/TestSSLServerConnectorFactory.java 
PRE-CREATION 
  src/main/assemblies/distro.xml 1ffbfd6d2ba33b390999e9094cbb336fbce45c21 

Diff: https://reviews.apache.org/r/52399/diff/


Testing
---

- Tested basic functionality by executing a workflow that uses the sample 
JavaAction
- without SSL - on a 2.4.0 pseudo Hadoop cluster
- SSL with Kerberos is using a test CDH cluster 
- Added new unit tests that check
- If oozie.ssl.enabled is not specified, server starts without SSL settings 
- If oozie.ssl.enabled is specified, server starts with SSL settings
- SSL protocols and ciphers can be configured via system properties and 
environment variables 
- Ran subset of tests using Hadoop-2 profile
- mvn clean package assembly:single   -DjavaVersion=1.8 -DtargetVersion=1.7 
 -Dtest=TestJavaActionExecutor  -Phadoop-2 -Dhadoop.version=2.4.0


Thanks,

Attila Sasvari

Re: Review Request 52399: OOZIE-2666 Support embedding Jetty into Oozie

[Attila Sasvari]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52399/
---

(Updated Sept. 30, 2016, 9:27 a.m.)


Review request for oozie, Peter Cseh, Peter Bacsko, and Robert Kanter.


Changes
---

Change Jetty version to 9.2.19.v20160908 because Jetty 9.3 requires Java 8 (see 
http://www.eclipse.org/jetty/documentation/current/what-jetty-version.html); no 
code changes were needed


Repository: oozie-git


Description
---

Embedding jetty into Oozie so that it can run as a standalone application. The 
changes also try to address OOZIE-2317 (i.e. Tomcat 6 is EOL).

New functionality
- New module (server) is added that sets up an embedded Jetty server and start 
Oozie services. Servlet mapping is done by reading web.xml of webapp at 
runtime. JSP is handled with custom code. Server version is not revealed in 
server repsonses.
- SSL protocols and ciphers can be configured via system properties and 
environment variables. Precedence: system properties, environment variables, 
default values
   
Changes in existing code
- Excluded jetty 6 dependencies from core and updated tests accordingly  

Packaging
- oozie.sh is modified so that it starts Oozie with embedded jetty by default. 
If someone would like to use tomcat for any reason, they can set an environment 
variable (e.g. OOZIE_USE_TOMCAT=1).

TODO:
- Add more tests
- Add more documentation
- Code cleanup + refactoring in packaging and core parts
- Maven clean up
- Allow to tune more Jetty settings (for example threadpool)
- More security measures (e.g. protect against clickjacking, CSRF, etc.)
- Update Oozie Documentation


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/util/Instrumentation.java 
fa1e92a0f1fadfb66c5e66fea5f26f57579080e7 
  distro/src/main/bin/oozie-sys.sh 97d55a2b69c34ede007d4f65cdfc66f1ac2cfd13 
  distro/src/main/bin/oozied.sh a869c3da177c863a068f2af45c7bca9d5cb771ac 
  pom.xml 704a29f4e4805e3e08c2a547b2a375f6b1b2 
  server/pom.xml PRE-CREATION 
  server/src/main/assemblies/empty.xml PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java 
PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/JspHandler.java PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/SSLServerConnectorFactory.java 
PRE-CREATION 
  server/src/main/resources/checkstyle-header.txt PRE-CREATION 
  server/src/main/resources/checkstyle.xml PRE-CREATION 
  server/src/test/java/org/apache/oozie/server/TestEmbeddedOozieServer.java 
PRE-CREATION 
  
server/src/test/java/org/apache/oozie/server/TestSSLServerConnectorFactory.java 
PRE-CREATION 
  src/main/assemblies/distro.xml 1ffbfd6d2ba33b390999e9094cbb336fbce45c21 

Diff: https://reviews.apache.org/r/52399/diff/


Testing
---

- Tested basic functionality by executing a workflow that uses the sample 
JavaAction
- without SSL - on a 2.4.0 pseudo Hadoop cluster
- SSL with Kerberos is using a test CDH cluster 
- Added new unit tests that check
- If oozie.ssl.enabled is not specified, server starts without SSL settings 
- If oozie.ssl.enabled is specified, server starts with SSL settings
- SSL protocols and ciphers can be configured via system properties and 
environment variables 
- Ran subset of tests using Hadoop-2 profile
- mvn clean package assembly:single   -DjavaVersion=1.8 -DtargetVersion=1.7 
 -Dtest=TestJavaActionExecutor  -Phadoop-2 -Dhadoop.version=2.4.0


Thanks,

Attila Sasvari

[jira] [Updated] (OOZIE-2666) Support embedding Jetty into Oozie

[Attila Sasvari (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Sasvari updated OOZIE-2666:
--
Attachment: OOZIE-2666.PATCH-02.patch

- Change Jetty version to 9.2.19.v20160908 because Jetty 9.3 requires Java 8 
(see 
http://www.eclipse.org/jetty/documentation/current/what-jetty-version.html); no 
code changes were needed

> Support embedding Jetty into Oozie
> --
>
> Key: OOZIE-2666
> URL: https://issues.apache.org/jira/browse/OOZIE-2666
> Project: Oozie
>  Issue Type: Sub-task
>  Components: core
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2666.PATCH-01.patch, OOZIE-2666.PATCH-02.patch
>
>
> The purpose of this task is to allow Oozie to run as a standalone application 
> rather than being deployed to a container inside a WAR file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-1814) Oozie should mask any passwords in logs and REST interfaces

[Andras Piros (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535490#comment-15535490
 ] 

Andras Piros commented on OOZIE-1814:
-

Thanks so much!
# Done.
# These 11 lines are all in the new {{.json}} files that are also filtered out 
in the patch.

> Oozie should mask any passwords in logs and REST interfaces
> ---
>
> Key: OOZIE-1814
> URL: https://issues.apache.org/jira/browse/OOZIE-1814
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: trunk
>Reporter: Bowen Zhang
>Assignee: Andras Piros
>  Labels: newbie
> Fix For: 4.3.0
>
> Attachments: OOZIE-1814.002.patch, OOZIE-1814.003.patch, 
> OOZIE-1814.004.patch, OOZIE-1814.005.patch, OOZIE-1814.006.patch, 
> OOZIE-1814.007.patch, oozie-1814.patch
>
>
> Following passwords are currently visible in the instrumentation log, REST 
> endpoints, WebUI, and CLI (WebUI and CLI simply call the REST endpoints):
> * {{javax.net.ssl.trustStorePassword}}
> * {{oozie.https.keystore.pass}}
> * {{HADOOP_CREDSTORE_PASSWORD}}
> * {{OOZIE_HTTPS_KEYSTORE_PASSWORD}}
> * {{OOZIE_HTTPS_TRUSTSTORE_PASSWORD}}
> There are a few examples that illustrate password leakage.
> {noformat}
> # grep -i pass /var/log/oozie/oozie-instrumentation.log
> OOZIE_HTTPS_TRUSTSTORE_PASSWORD = password
> javax.net.ssl.trustStorePassword = password
> oozie.https.keystore.pass = password
> HADOOP_CREDSTORE_PASSWORD = password
> OOZIE_HTTPS_KEYSTORE_PASSWORD = password
> CATALINA_OPTS = -Xms603979776 -Xmx603979776 
> -XX:+HeapDumpOnOutOfMemoryError 
> -XX:HeapDumpPath=/tmp/OOZIE-1_OOZIE-1-OOZIE_SERVER-2e75cc1293d9058eef7250a18f347c43_pid30867.hprof
>  -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh 
> -Doozie.home.dir=/usr/lib/oozie 
> -Doozie.config.dir=/var/run/cloudera-scm-agent/process/320-oozie-OOZIE_SERVER 
> -Doozie.log.dir=/var/log/oozie 
> -Doozie.log.file=oozie-cmf-OOZIE-1-OOZIE_SERVER-nightly-1.gce.cloudera.com.log.out
>  -Doozie.config.file=oozie-site.xml -Doozie.log4j.file=log4j.properties 
> -Doozie.log4j.reload=10 -Doozie.http.hostname=nightly-1.gce.cloudera.com 
> -Doozie.http.port=11000 -Djava.net.preferIPv4Stack=true 
> -Doozie.admin.port=11001 -Dderby.stream.error.file=/var/log/oozie/derby.log 
> -Doozie.instance.id=nightly-1.gce.cloudera.com 
> -Djava.library.path=/usr/lib/hadoop/lib/native -Doozie.https.port=11443 
> -Djavax.net.ssl.trustStore=/etc/cdep-ssl-conf/CA_STANDARD/truststore.jks 
> -Djavax.net.ssl.trustStorePassword=password
> {noformat}
> Oozie dumps the env vars and Java sys props to the instrumentation log on 
> startup.
> {noformat}
> # curl --negotiate -u foo:bar -k 
> https://nightly-1.gce.cloudera.com:11443/oozie/v2/admin/os-env | python -m 
> json.tool | grep -i pass
> "CATALINA_OPTS": "-Xms603979776 -Xmx603979776 -XX:+HeapDumpOnOutOfMemoryError 
> -XX:HeapDumpPath=/tmp/OOZIE-1_OOZIE-1-OOZIE_SERVER-2e75cc1293d9058eef7250a18f347c43_pid30867.hprof
>  -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh 
> -Doozie.home.dir=/usr/lib/oozie 
> -Doozie.config.dir=/var/run/cloudera-scm-agent/process/320-oozie-OOZIE_SERVER 
> -Doozie.log.dir=/var/log/oozie 
> -Doozie.log.file=oozie-cmf-OOZIE-1-OOZIE_SERVER-nightly-1.gce.cloudera.com.log.out
>  -Doozie.config.file=oozie-site.xml -Doozie.log4j.file=log4j.properties 
> -Doozie.log4j.reload=10 -Doozie.http.hostname=nightly-1.gce.cloudera.com 
> -Doozie.http.port=11000 -Djava.net.preferIPv4Stack=true 
> -Doozie.admin.port=11001 -Dderby.stream.error.file=/var/log/oozie/derby.log 
> -Doozie.instance.id=nightly-1.gce.cloudera.com 
> -Djava.library.path=/usr/lib/hadoop/lib/native -Doozie.https.port=11443 
> -Djavax.net.ssl.trustStore=/etc/cdep-ssl-conf/CA_STANDARD/truststore.jks 
> -Djavax.net.ssl.trustStorePassword=password ",
> "HADOOP_CREDSTORE_PASSWORD": "password",
> "OOZIE_HTTPS_KEYSTORE_PASSWORD": "password",
> "OOZIE_HTTPS_TRUSTSTORE_PASSWORD": "password",
> {noformat}
> {noformat}
> # curl --negotiate -u foo:bar -k 
> https://nightly-1.gce.cloudera.com:11443/oozie/v2/admin/java-sys-properties | 
> python -m json.tool | grep -i pass
> "javax.net.ssl.trustStorePassword": "password",
> "oozie.https.keystore.pass": "password",
> {noformat}
> The REST API has two endpoints, {{admin/os-env}} and 
> {{admin/java-sys-properties}}, which are also available in the Web UI and 
> CLI.  These expose the env vars and Java sys props too.
> We should mask these like we do for the configuration endpoint.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-1814) Oozie should mask any passwords in logs and REST interfaces

[Andras Piros (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-1814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros updated OOZIE-1814:

Attachment: OOZIE-1814.007.patch

Some small refactoring.

> Oozie should mask any passwords in logs and REST interfaces
> ---
>
> Key: OOZIE-1814
> URL: https://issues.apache.org/jira/browse/OOZIE-1814
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: trunk
>Reporter: Bowen Zhang
>Assignee: Andras Piros
>  Labels: newbie
> Fix For: 4.3.0
>
> Attachments: OOZIE-1814.002.patch, OOZIE-1814.003.patch, 
> OOZIE-1814.004.patch, OOZIE-1814.005.patch, OOZIE-1814.006.patch, 
> OOZIE-1814.007.patch, oozie-1814.patch
>
>
> Following passwords are currently visible in the instrumentation log, REST 
> endpoints, WebUI, and CLI (WebUI and CLI simply call the REST endpoints):
> * {{javax.net.ssl.trustStorePassword}}
> * {{oozie.https.keystore.pass}}
> * {{HADOOP_CREDSTORE_PASSWORD}}
> * {{OOZIE_HTTPS_KEYSTORE_PASSWORD}}
> * {{OOZIE_HTTPS_TRUSTSTORE_PASSWORD}}
> There are a few examples that illustrate password leakage.
> {noformat}
> # grep -i pass /var/log/oozie/oozie-instrumentation.log
> OOZIE_HTTPS_TRUSTSTORE_PASSWORD = password
> javax.net.ssl.trustStorePassword = password
> oozie.https.keystore.pass = password
> HADOOP_CREDSTORE_PASSWORD = password
> OOZIE_HTTPS_KEYSTORE_PASSWORD = password
> CATALINA_OPTS = -Xms603979776 -Xmx603979776 
> -XX:+HeapDumpOnOutOfMemoryError 
> -XX:HeapDumpPath=/tmp/OOZIE-1_OOZIE-1-OOZIE_SERVER-2e75cc1293d9058eef7250a18f347c43_pid30867.hprof
>  -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh 
> -Doozie.home.dir=/usr/lib/oozie 
> -Doozie.config.dir=/var/run/cloudera-scm-agent/process/320-oozie-OOZIE_SERVER 
> -Doozie.log.dir=/var/log/oozie 
> -Doozie.log.file=oozie-cmf-OOZIE-1-OOZIE_SERVER-nightly-1.gce.cloudera.com.log.out
>  -Doozie.config.file=oozie-site.xml -Doozie.log4j.file=log4j.properties 
> -Doozie.log4j.reload=10 -Doozie.http.hostname=nightly-1.gce.cloudera.com 
> -Doozie.http.port=11000 -Djava.net.preferIPv4Stack=true 
> -Doozie.admin.port=11001 -Dderby.stream.error.file=/var/log/oozie/derby.log 
> -Doozie.instance.id=nightly-1.gce.cloudera.com 
> -Djava.library.path=/usr/lib/hadoop/lib/native -Doozie.https.port=11443 
> -Djavax.net.ssl.trustStore=/etc/cdep-ssl-conf/CA_STANDARD/truststore.jks 
> -Djavax.net.ssl.trustStorePassword=password
> {noformat}
> Oozie dumps the env vars and Java sys props to the instrumentation log on 
> startup.
> {noformat}
> # curl --negotiate -u foo:bar -k 
> https://nightly-1.gce.cloudera.com:11443/oozie/v2/admin/os-env | python -m 
> json.tool | grep -i pass
> "CATALINA_OPTS": "-Xms603979776 -Xmx603979776 -XX:+HeapDumpOnOutOfMemoryError 
> -XX:HeapDumpPath=/tmp/OOZIE-1_OOZIE-1-OOZIE_SERVER-2e75cc1293d9058eef7250a18f347c43_pid30867.hprof
>  -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh 
> -Doozie.home.dir=/usr/lib/oozie 
> -Doozie.config.dir=/var/run/cloudera-scm-agent/process/320-oozie-OOZIE_SERVER 
> -Doozie.log.dir=/var/log/oozie 
> -Doozie.log.file=oozie-cmf-OOZIE-1-OOZIE_SERVER-nightly-1.gce.cloudera.com.log.out
>  -Doozie.config.file=oozie-site.xml -Doozie.log4j.file=log4j.properties 
> -Doozie.log4j.reload=10 -Doozie.http.hostname=nightly-1.gce.cloudera.com 
> -Doozie.http.port=11000 -Djava.net.preferIPv4Stack=true 
> -Doozie.admin.port=11001 -Dderby.stream.error.file=/var/log/oozie/derby.log 
> -Doozie.instance.id=nightly-1.gce.cloudera.com 
> -Djava.library.path=/usr/lib/hadoop/lib/native -Doozie.https.port=11443 
> -Djavax.net.ssl.trustStore=/etc/cdep-ssl-conf/CA_STANDARD/truststore.jks 
> -Djavax.net.ssl.trustStorePassword=password ",
> "HADOOP_CREDSTORE_PASSWORD": "password",
> "OOZIE_HTTPS_KEYSTORE_PASSWORD": "password",
> "OOZIE_HTTPS_TRUSTSTORE_PASSWORD": "password",
> {noformat}
> {noformat}
> # curl --negotiate -u foo:bar -k 
> https://nightly-1.gce.cloudera.com:11443/oozie/v2/admin/java-sys-properties | 
> python -m json.tool | grep -i pass
> "javax.net.ssl.trustStorePassword": "password",
> "oozie.https.keystore.pass": "password",
> {noformat}
> The REST API has two endpoints, {{admin/os-env}} and 
> {{admin/java-sys-properties}}, which are also available in the Web UI and 
> CLI.  These expose the env vars and Java sys props too.
> We should mask these like we do for the configuration endpoint.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Oozie going forward

[Peter Bacsko]

Hi,

one question from me: how shall we integrate our OYA changes to master?

I can think of four options:
1. Simply git merge our commits from the oya branch. This will mean at
least 20-30 commits.
2. We squash our commits to have fewer and then merge the oya branch (right
now I can't say in advance how many commits we'll end up with)
3. We squash _all_ commits into one and then merge/cherry-pick it.
4. We create a patch file containing all changes and apply it

Ideas, suggestions are welcome.

Peter

On Fri, Sep 30, 2016 at 2:56 AM, Robert Kanter  wrote:

> Hi,
>
> Now that Oozie 4.3 is branched and will be happening soon, I imagine the
> next release we do with be Oozie 5 with Oozie On Yarn.  We can continue to
> keep OYA in a feature branch until later when it's more stable so we don't
> disrupt the master branch in the meantime.  We've already removed 0.23 and
> made Hadoop 2 the default.  Removing Hadoop 1 will simplify things, and is
> currently in the way of other tasks like upgrading HBase.  I imagine it
> will continue to do so more as time goes on.
>
> I'd like to propose the following:
>
>1. Oozie 4.3 is the last release to support Hadoop 1.  If we want to do
>another Oozie 4.x release later, we can branch it from branch-4.3
> instead
>of trunk.  Though I expect 4.3 will be the last minor release of Oozie
> 4.
>2. Oozie 5 (the master branch; aka trunk) will be the first release to
>support only Hadoop 2 and Hadoop 3 (these should be much more similar
> than
>Hadoop 1 to 2 was; I expect things will work as-is by just changing the
>Hadoop version so we probably won't even need profiles).  We'll have
> Oozie
>On Yarn as the marquee feature.
>3. We'll drop Hadoop 1 from the master branch.  We can reuse parts
>of OOZIE-2316.
>4. The master branch will become 5.0.0-SNAPSHOT instead of
>4.4.0-SNAPSHOT.
>
> I'd like us to take care of #3 soon, because it's getting in the way.  We
> might as well also take care of #4 soon too.
>
> Thoughts?
>
>
> thanks
> - Robert
>

[jira] [Commented] (OOZIE-2634) Queue dump command message is confusing when the queue is empty

[Andras Piros (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535429#comment-15535429
 ] 

Andras Piros commented on OOZIE-2634:
-

Done, thx.

> Queue dump command message is confusing when the queue is empty
> ---
>
> Key: OOZIE-2634
> URL: https://issues.apache.org/jira/browse/OOZIE-2634
> Project: Oozie
>  Issue Type: Improvement
>Reporter: Robert Kanter
>Assignee: Andras Piros
>Priority: Minor
>  Labels: newbie
> Fix For: 4.4.0
>
> Attachments: OOZIE-2634.001.patch, OOZIE-2634.002.patch, 
> OOZIE-2634.003.patch, OOZIE-2634.004.patch
>
>
> If the callable queue is empty, and you run the queue dump command, the 
> message says:
> {noformat}
> # oozie admin queuedump
> [Server Queue Dump]:
> Queue dump is null!
> **
> [Server Uniqueness Map Dump]:
> Uniqueness dump is null!
> {noformat}
> This message makes it sound like a bad thing, especially the exclamation 
> mark.  We should change the message to something more neutral and helpful, 
> like "The queue is empty" or something like that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2634) Queue dump command message is confusing when the queue is empty

[Andras Piros (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros updated OOZIE-2634:

Attachment: OOZIE-2634.004.patch

Some small refactoring.

> Queue dump command message is confusing when the queue is empty
> ---
>
> Key: OOZIE-2634
> URL: https://issues.apache.org/jira/browse/OOZIE-2634
> Project: Oozie
>  Issue Type: Improvement
>Reporter: Robert Kanter
>Assignee: Andras Piros
>Priority: Minor
>  Labels: newbie
> Fix For: 4.4.0
>
> Attachments: OOZIE-2634.001.patch, OOZIE-2634.002.patch, 
> OOZIE-2634.003.patch, OOZIE-2634.004.patch
>
>
> If the callable queue is empty, and you run the queue dump command, the 
> message says:
> {noformat}
> # oozie admin queuedump
> [Server Queue Dump]:
> Queue dump is null!
> **
> [Server Uniqueness Map Dump]:
> Uniqueness dump is null!
> {noformat}
> This message makes it sound like a bad thing, especially the exclamation 
> mark.  We should change the message to something more neutral and helpful, 
> like "The queue is empty" or something like that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2690) OOZIE NPE while executing kill()

[Peter Cseh (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535395#comment-15535395
 ] 

Peter Cseh commented on OOZIE-2690:
---

Could you add a testcase based upon your scenario description?

> OOZIE NPE while executing kill()
> 
>
> Key: OOZIE-2690
> URL: https://issues.apache.org/jira/browse/OOZIE-2690
> Project: Oozie
>  Issue Type: Bug
>Reporter: Cheng Xu
>Assignee: Abhishek Bafna
> Attachments: OOZIE-2690-00.patch
>
>
> ION[027-160917110703953-oozie-oozi-W@java2] Exception while executing 
> kill(). Error Code [NullPointerException], Message[NullPointerException: null]
> org.apache.oozie.action.ActionExecutorException: NullPointerException: null
> at 
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
> at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.kill(JavaActionExecutor.java:1632)
> at 
> org.apache.oozie.command.wf.ActionKillXCommand.execute(ActionKillXCommand.java:146)
> at 
> org.apache.oozie.command.wf.ActionKillXCommand.execute(ActionKillXCommand.java:62)
> at org.apache.oozie.command.XCommand.call(XCommand.java:287)
> at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:331)
> at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:260)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:178)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.mapreduce.TypeConverter.toYarn(TypeConverter.java:79)
> at 
> org.apache.hadoop.mapred.ClientServiceDelegate.(ClientServiceDelegate.java:120)
> at org.apache.hadoop.mapred.ClientCache.getClient(ClientCache.java:68)
> at org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:573)
> at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:184)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:601)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:599)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
> at 
> org.apache.hadoop.mapred.JobClient.getJobUsingCluster(JobClient.java:599)
> at org.apache.hadoop.mapred.JobClient.getJobInner(JobClient.java:609)
> at org.apache.hadoop.mapred.JobClient.getJob(JobClient.java:639)
> at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.getJobClientWithRetries(JavaActionExecutor.java:1432)
> at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.getRunningJob(JavaActionExecutor.java:1420)
> at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.kill(JavaActionExecutor.java:1623)
> ... 10 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2694) Add logging for FsActionExecutor

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535297#comment-15535297
 ] 

Abhishek Bafna commented on OOZIE-2694:
---

{code}
LOG.debug("doOperations = [{0}], nameNodePath = [{1}], path = [{2}]", "mkdir", 
nameNodePath, path);
{code}

One trivial thing, {{doOperations}} can be {{doOperation}} as we are printing 
about one operation only. For all the log statements.

{{+1 non-binding}}

> Add logging for FsActionExecutor 
> -
>
> Key: OOZIE-2694
> URL: https://issues.apache.org/jira/browse/OOZIE-2694
> Project: Oozie
>  Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Purshotam Shah
> Attachments: OOZIE-2694-V1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2693) SimpleHCatDependencyCache.removeMissingDependency can throw NPE

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535278#comment-15535278
 ] 

Abhishek Bafna commented on OOZIE-2693:
---

{{+1 non-binding}}

> SimpleHCatDependencyCache.removeMissingDependency can throw NPE
> ---
>
> Key: OOZIE-2693
> URL: https://issues.apache.org/jira/browse/OOZIE-2693
> Project: Oozie
>  Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Purshotam Shah
> Attachments: OOZIE-2693-V1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)