Build failed in Jenkins: oozie-trunk-find-patches-available #142621

2016-12-13 Thread Apache Jenkins Server 
See 

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on H1 (Hadoop) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/oozie.git # timeout=10
Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/oozie.git
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/oozie.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision fe182fd8127cb73cdc5502e1536e69724e87f69a 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f fe182fd8127cb73cdc5502e1536e69724e87f69a
 > git rev-list fe182fd8127cb73cdc5502e1536e69724e87f69a # timeout=10
[oozie-trunk-find-patches-available] $ /bin/bash 
/tmp/hudson7438413154075622044.sh
mkdir: cannot create directory 
':
 File exists
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
100 901120 901120 0   179k  0 --:--:-- --:--:-- --:--:--  178k
100  368k0  368k0 0   249k  0 --:--:--  0:00:01 --:--:--  249k
100  657k0  657k0 0   270k  0 --:--:--  0:00:02 --:--:--  269k
100  912k0  912k0 0   261k  0 --:--:--  0:00:03 --:--:--  261k
100 1264k0 1264k0 0   284k  0 --:--:--  0:00:04 --:--:--  284k
100 1608k0 1608k0 0   294k  0 --:--:--  0:00:05 --:--:--  306k
100 1904k0 1904k0 0   292k  0 --:--:--  0:00:06 --:--:--  305k
100 2071k0 2071k0 0   291k  0 --:--:--  0:00:07 --:--:--  302k
curl: (18) transfer closed with outstanding read data remaining
Could not retrieve available patches from JIRA
Build step 'Execute shell' marked build as failure

[jira] [Updated] (OOZIE-2629) Get OozieShareLibCLI to perform a final rename to destPath when creating a Sharelib

2016-12-13 Thread Yishan Yang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/OOZIE-2629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yishan Yang updated OOZIE-2629:
---
Attachment: OOZIE-2629-4.patch

> Get OozieShareLibCLI to perform a final rename to destPath when creating a 
> Sharelib
> ---
>
> Key: OOZIE-2629
> URL: https://issues.apache.org/jira/browse/OOZIE-2629
> Project: Oozie
>  Issue Type: Improvement
>Affects Versions: 4.3.0
>Reporter: Yishan Yang
>Priority: Minor
> Fix For: trunk
>
> Attachments: OOZIE-2629-1.patch, OOZIE-2629-2.patch, 
> OOZIE-2629-3.patch, OOZIE-2629-4.patch
>
>
> Change the OozieShareLibCLI to perform a final 'rename' to avoid a condition 
> where 'half' of the sharelib could exist during sharelib installation, if 
> Oozie attempts to run a workflow at the same time as the sharelib 
> installation. Oozie will only start if the sharelib exists, but this by 
> itself is not enough since the sharelib could be in an unusable state.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: oozie-trunk-precommit-build #3510

2016-12-13 Thread Apache Jenkins Server 
See 

--
[...truncated 14653 lines...]
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/client/retry/package-summary.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/client/retry/class-use/ConnectionRetriableClient.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/client/retry/package-frame.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/client/retry/package-use.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/client/CoordinatorJob.Timeunit.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/client/OozieClient.SORT_BY.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/client/OozieClient.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/client/WorkflowJob.Status.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/client/CoordinatorJob.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/client/SLAEvent.SlaAppType.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/client/package-use.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/client/BulkResponse.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/client/rest/JsonToBean.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/client/rest/package-tree.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/client/rest/RestConstants.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/client/rest/package-summary.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/client/rest/JsonTags.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/client/rest/JsonUtils.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/client/rest/class-use/JsonToBean.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/client/rest/class-use/RestConstants.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/client/rest/class-use/JsonTags.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/client/rest/class-use/JsonUtils.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/client/rest/package-frame.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/client/rest/package-use.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/client/JMSConnectionInfo.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/package-frame.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/package-use.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/cli/package-tree.html
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/docs/client/apidocs/org/apache/oozie/cli/OozieCLIException.html
 longer than 100 characters.
[WARNING] Entry:

Re: Review Request 54557: Change the OozieShareLibCLI to perform a final 'rename'

2016-12-13 Thread Yishan Yang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54557/
---

(Updated Dec. 13, 2016, 9:57 p.m.)


Review request for oozie and András Piros.


Changes
---

Address comments


Repository: oozie-git


Description
---

Change the OozieShareLibCLI to perform a final 'rename' to avoid a condition 
where 'half' of the sharelib could exist during sharelib installation, if Oozie 
attempts to run a workflow at the same time as the sharelib installation. Oozie 
will only start if the sharelib exists, but this by itself is not enough since 
the sharelib could be in an unusable state.


Diffs (updated)
-

  tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6 

Diff: https://reviews.apache.org/r/54557/diff/


Testing
---

$ mvn test -Dtest=TestOozieSharelibCLI

Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.497 sec

Results :

Tests run: 4, Failures: 0, Errors: 0, Skipped: 0


Thanks,

Yishan Yang

[jira] [Commented] (OOZIE-2629) Get OozieShareLibCLI to perform a final rename to destPath when creating a Sharelib

2016-12-13 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-2629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15746702#comment-15746702
 ] 

Hadoop QA commented on OOZIE-2629:
--

Testing JIRA OOZIE-2629

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:orange}0{color} There are [1] new bugs found in total that would be nice 
to have fixed.
.{color:green}+1{color} There are no new bugs found in [server].
.{color:green}+1{color} There are no new bugs found in [docs].
.{color:green}+1{color} There are no new bugs found in [sharelib/streaming].
.{color:green}+1{color} There are no new bugs found in [sharelib/distcp].
.{color:green}+1{color} There are no new bugs found in [sharelib/pig].
.{color:green}+1{color} There are no new bugs found in [sharelib/spark].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive2].
.{color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
.{color:green}+1{color} There are no new bugs found in [sharelib/oozie].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive].
.{color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
.{color:green}+1{color} There are no new bugs found in [core].
.{color:green}+1{color} There are no new bugs found in [examples].
.{color:green}+1{color} There are no new bugs found in 
[hadooplibs/hadoop-utils-2].
.{color:green}+1{color} There are no new bugs found in [client].
.{color:orange}0{color} There are [1] new bugs found in [tools] that would 
be nice to have fixed.
.You can find the FindBugs diff here: tools/findbugs-new.html
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1831
.Tests failed: 2
.Tests errors: 0

.The patch failed the following testcases:

.  
testBundleStatusCoordSubmitFails(org.apache.oozie.service.TestStatusTransitService)
.  
testActionKillCommandDate(org.apache.oozie.command.coord.TestCoordActionsKillXCommand)

.Tests failing with errors:
.  

{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/3510/

> Get OozieShareLibCLI to perform a final rename to destPath when creating a 
> Sharelib
> ---
>
> Key: OOZIE-2629
> URL: https://issues.apache.org/jira/browse/OOZIE-2629
> Project: Oozie
>  Issue Type: Improvement
>Affects Versions: 4.3.0
>Reporter: Yishan Yang
>Priority: Minor
> Fix For: trunk
>
> Attachments: OOZIE-2629-1.patch, OOZIE-2629-2.patch, 
> OOZIE-2629-3.patch, OOZIE-2629-4.patch
>
>
> Change the OozieShareLibCLI to perform a final 'rename' to avoid a condition 
> where 'half' of the sharelib could exist during sharelib installation, if 
> Oozie attempts to run a workflow at the same time as the sharelib 
> installation. Oozie will only start if the sharelib exists, but this by 
> itself is not enough since the sharelib could be in an unusable state.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] Subscription: Oozie Patch Available

2016-12-13 Thread jira 
Issue Subscription
Filter: Oozie Patch Available (86 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-2619  Make  Hive action defaults to match hive defaults when running from 
command line
https://issues-test.apache.org/jira/browse/OOZIE-2619
OOZIE-2618  Include hive-common in hive-beeline as compile component for 
hive-server2 (hive2 action)
https://issues-test.apache.org/jira/browse/OOZIE-2618
OOZIE-2617  Read properties files in action configurations
https://issues-test.apache.org/jira/browse/OOZIE-2617
OOZIE-2616  Add Tez profile for Hive and Pig sharelibs
https://issues-test.apache.org/jira/browse/OOZIE-2616
OOZIE-2606  Set spark.yarn.jars to fix Spark 2.0 with Oozie
https://issues-test.apache.org/jira/browse/OOZIE-2606
OOZIE-2589  CompletedActionXCommand priority param is of no effect
https://issues-test.apache.org/jira/browse/OOZIE-2589
OOZIE-2588  Support getting credentials for same cluster hcat when credentials 
config is empty
https://issues-test.apache.org/jira/browse/OOZIE-2588
OOZIE-2585  Disable or remove test case 
TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and 
testMemoryUsageAndSpeed
https://issues-test.apache.org/jira/browse/OOZIE-2585
OOZIE-2584  Eliminate Thread.sleep() calls in TestMemoryLocks
https://issues-test.apache.org/jira/browse/OOZIE-2584
OOZIE-2583  oozie throws EL Exception when reference variable name containing 
dot
https://issues-test.apache.org/jira/browse/OOZIE-2583
OOZIE-2582  Populating external child Ids for action failures
https://issues-test.apache.org/jira/browse/OOZIE-2582
OOZIE-2573  dataset url  which contains spaces can not be handled rightly
https://issues-test.apache.org/jira/browse/OOZIE-2573
OOZIE-2572  SLA DURATION miss not shown when job is running for longer than 
expected time
https://issues-test.apache.org/jira/browse/OOZIE-2572
OOZIE-2571  Add scala.binary.version Maven property so that Scala 2.11 can be 
used
https://issues-test.apache.org/jira/browse/OOZIE-2571
OOZIE-2569  Adding yarn-site, core-site, hdfs-site and mapred-site into spark 
launcher 
https://issues-test.apache.org/jira/browse/OOZIE-2569
OOZIE-2568  SSH  action pretends to retry automaticly when it failed
https://issues-test.apache.org/jira/browse/OOZIE-2568
OOZIE-2564  Create new log4j config for unit tests so that logged messages 
contain threads
https://issues-test.apache.org/jira/browse/OOZIE-2564
OOZIE-2552  Update ActiveMQ version for security and other fixes
https://issues-test.apache.org/jira/browse/OOZIE-2552
OOZIE-2538  Update HttpClient versions to close security vulnerabilities
https://issues-test.apache.org/jira/browse/OOZIE-2538
OOZIE-2534  Versioned action libs (similar to how SharedLibs works)
https://issues-test.apache.org/jira/browse/OOZIE-2534
OOZIE-2530  Remove jline from parent and sharelib pig pom
https://issues-test.apache.org/jira/browse/OOZIE-2530
OOZIE-2528  Print out environment variables in LauncherMapper
https://issues-test.apache.org/jira/browse/OOZIE-2528
OOZIE-2521  Filter options are case sensitive for jobtye=bundle
https://issues-test.apache.org/jira/browse/OOZIE-2521
OOZIE-2518  Oozie is creating many files and directories in /tmp per day
https://issues-test.apache.org/jira/browse/OOZIE-2518
OOZIE-2517  Add support for startCreatedTime and endCreatedTime filters for 
coord and bundles
https://issues-test.apache.org/jira/browse/OOZIE-2517
OOZIE-2513  log.scan.duration should not be used for error and audit logs
https://issues-test.apache.org/jira/browse/OOZIE-2513
OOZIE-2512  ShareLibservice returns incorrect path for jar
https://issues-test.apache.org/jira/browse/OOZIE-2512
OOZIE-2508  Documentation change for Coord action rerun [OOZIE-1735]
https://issues-test.apache.org/jira/browse/OOZIE-2508
OOZIE-2501  ZK reentrant lock doesn't work for few cases
https://issues-test.apache.org/jira/browse/OOZIE-2501
OOZIE-2500  -DtestJarSimple option mentioned in minioozie doc does not work
https://issues-test.apache.org/jira/browse/OOZIE-2500
OOZIE-2499  map-reduce launcher does not need distributed files, archives 
except  jar of input/outputformat class
https://issues-test.apache.org/jira/browse/OOZIE-2499
OOZIE-2498  Oozie CallerId configuration for downstream components 
https://issues-test.apache.org/jira/browse/OOZIE-2498
OOZIE-2495  change action status from  ErrorType.NON_TRANSIENT to TRANSIENT 
when SSH action occurs AUTH_FAILED occasionally
https://issues-test.apache.org/jira/browse/OOZIE-2495
OOZIE-2491  oozie acl cannot specify group,it does`t work
https://issues-test.apache.org/jira/browse/OOZIE-2491

[jira] Subscription: Oozie Patch Available

2016-12-13 Thread jira 
Issue Subscription
Filter: Oozie Patch Available (92 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-2757  Malformed xml is Spark action doc page
https://issues.apache.org/jira/browse/OOZIE-2757
OOZIE-2754  Document Ssh action failure if output is written to stdout/stderr 
upon login
https://issues.apache.org/jira/browse/OOZIE-2754
OOZIE-2751  LocalOozieClient is missing methods from OozieClient
https://issues.apache.org/jira/browse/OOZIE-2751
OOZIE-2750  Spelling errors in the log messages and exception messages
https://issues.apache.org/jira/browse/OOZIE-2750
OOZIE-2748  NPE in LauncherMapper.printArgs()
https://issues.apache.org/jira/browse/OOZIE-2748
OOZIE-2746  Several tests failure in TestV2ValidateServlet.java
https://issues.apache.org/jira/browse/OOZIE-2746
OOZIE-2739  Remove property expansion pattern from ShellMain's log4j properties 
content
https://issues.apache.org/jira/browse/OOZIE-2739
OOZIE-2736  Reduce the number of threads during test execution
https://issues.apache.org/jira/browse/OOZIE-2736
OOZIE-2733  change org.apache.hadoop.fs.permission.AccessControlException to 
org.apache.hadoop.security.AccessControlException
https://issues.apache.org/jira/browse/OOZIE-2733
OOZIE-2732  Remove login server example
https://issues.apache.org/jira/browse/OOZIE-2732
OOZIE-2728  Typo in Error E0405
https://issues.apache.org/jira/browse/OOZIE-2728
OOZIE-2727  OozieDBCLI creates temporary directories and do not delete them
https://issues.apache.org/jira/browse/OOZIE-2727
OOZIE-2726  Flaky test due to daylight saving changes
https://issues.apache.org/jira/browse/OOZIE-2726
OOZIE-2713  Provide an option to log out contents of the localized jar files
https://issues.apache.org/jira/browse/OOZIE-2713
OOZIE-2709  Log the substituted pig script by expanding macros
https://issues.apache.org/jira/browse/OOZIE-2709
OOZIE-2694  Add logging for FsActionExecutor 
https://issues.apache.org/jira/browse/OOZIE-2694
OOZIE-2693  SimpleHCatDependencyCache.removeMissingDependency can throw NPE
https://issues.apache.org/jira/browse/OOZIE-2693
OOZIE-2692  Oozie job submit doesn't report error message to user if there is 
any issue with job conf
https://issues.apache.org/jira/browse/OOZIE-2692
OOZIE-2691  Show workflow action retry count to UI
https://issues.apache.org/jira/browse/OOZIE-2691
OOZIE-2684  Bad database schema error for WF_ACTIONS table
https://issues.apache.org/jira/browse/OOZIE-2684
OOZIE-2681  fix javadoc to compile on JDK8 again
https://issues.apache.org/jira/browse/OOZIE-2681
OOZIE-2670  Upgrade Hbase to 1.2
https://issues.apache.org/jira/browse/OOZIE-2670
OOZIE-2668  Status update and recovery problems when coord action and its 
children not in sync
https://issues.apache.org/jira/browse/OOZIE-2668
OOZIE-2662  DB migration fails if DB is too big
https://issues.apache.org/jira/browse/OOZIE-2662
OOZIE-2654  Zookeeper dependent services should not depend on Connectionstate 
to be valid before cleaning up
https://issues.apache.org/jira/browse/OOZIE-2654
OOZIE-2652  Skip trash while deleting hive table partition
https://issues.apache.org/jira/browse/OOZIE-2652
OOZIE-2650  Retry coord start on database exceptions
https://issues.apache.org/jira/browse/OOZIE-2650
OOZIE-2644  Skip queuing Notification Commands when there's nothing to notify
https://issues.apache.org/jira/browse/OOZIE-2644
OOZIE-2641  rerunning an oozie coordinator job stucks on waiting
https://issues.apache.org/jira/browse/OOZIE-2641
OOZIE-2630  Oozie Coordinator EL Functions to get first day of the week/month
https://issues.apache.org/jira/browse/OOZIE-2630
OOZIE-2629  Get OozieShareLibCLI to perform a final rename to destPath when 
creating a Sharelib
https://issues.apache.org/jira/browse/OOZIE-2629
OOZIE-2619  Make  Hive action defaults to match hive defaults when running from 
command line
https://issues.apache.org/jira/browse/OOZIE-2619
OOZIE-2618  Include hive-common in hive-beeline as compile component for 
hive-server2 (hive2 action)
https://issues.apache.org/jira/browse/OOZIE-2618
OOZIE-2608  Comma in oozie.service.JPAService.jdbc.password value results in 
authentication error
https://issues.apache.org/jira/browse/OOZIE-2608
OOZIE-2585  Disable or remove test case 
TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and 
testMemoryUsageAndSpeed
https://issues.apache.org/jira/browse/OOZIE-2585
OOZIE-2573  dataset url  which contains spaces can not be handled rightly
https://issues.apache.org/jira/browse/OOZIE-2573
OOZIE-2568  SSH  action pretends to retry automaticly when it failed

[jira] [Commented] (OOZIE-2608) Comma in oozie.service.JPAService.jdbc.password value results in authentication error

2016-12-13 Thread Peter Cseh (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-2608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15744644#comment-15744644
 ] 

Peter Cseh commented on OOZIE-2608:
---

I can confirm that Oozie master won't start with the password {{oo*!zi,e}} 
before the patch and it works fine with the same password after the patch is 
applied on MySQL. 

> Comma in oozie.service.JPAService.jdbc.password value results in 
> authentication error
> -
>
> Key: OOZIE-2608
> URL: https://issues.apache.org/jira/browse/OOZIE-2608
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.2.0
> Environment: HDP 2.4 latest, Oozie 4.2.0.2.4.2.0-258
>Reporter: Nikolai Grigoriev
>Assignee: Peter Cseh
>Priority: Minor
> Attachments: OOZIE-2608-001.patch
>
>
> I have attempted to install Oozie with Postgres db backend via HDP 2.4. I was 
> able to test DB connection via Ambari, everything was correct. However, the 
> server failed to start with this error:
> {code}
> 2016-07-15 11:46:39,580  INFO URIHandlerService:520 - 
> SERVER[hadoop-master01.production.sociablelabs.net] Loaded default urihandler 
> org.apache.oozie.dependency.FSURIHandler
> 2016-07-15 11:46:40,323  INFO HadoopAccessorService:520 - 
> SERVER[hadoop-master01.production.sociablelabs.net] password key is 
> oozie.service.JPAService.jdbc.password
> 2016-07-15 11:46:40,331  INFO HadoopAccessorService:520 - 
> SERVER[hadoop-master01.production.sociablelabs.net] credential provider path 
> is null
> 2016-07-15 11:46:40,337  INFO HadoopAccessorService:520 - 
> SERVER[hadoop-master01.production.sociablelabs.net] password is null
> 2016-07-15 11:46:41,176 FATAL Services:514 - 
> SERVER[hadoop-master01.production.sociablelabs.net] Runtime Exception during 
> Services Load. Check your list of 'oozie.services' or 'o
> ozie.services.ext'
> 2016-07-15 11:46:41,194 FATAL Services:514 - 
> SERVER[hadoop-master01.production.sociablelabs.net] E0103: Could not load 
> service classes, Cannot create PoolableConnectionFactory (F
> ATAL: password authentication failed for user "oozie")
> org.apache.oozie.service.ServiceException: E0103: Could not load service 
> classes, Cannot create PoolableConnectionFactory (FATAL: password 
> authentication failed for user "oozie")
> at org.apache.oozie.service.Services.loadServices(Services.java:309)
> at org.apache.oozie.service.Services.init(Services.java:213)
> 
> Caused by: org.postgresql.util.PSQLException: FATAL: password authentication 
> failed for user "oozie"
> at 
> org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:291)
> at 
> org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108)
> at 
> org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)
> ...
> {code}
> Schema verification at startup worked fine and I could not find any bugs 
> related to it. So, I started getting suspicious about my password value, 
> which was like "pO,e43DpjK" (not exactly but it was a bunch of letters, 
> number and one comma). I suspected that the comma was probably interpreted as 
> a value list separator so I have changed the password removing the comma. 
> Server started flawlessly.
> I am not 100% sure if it is Ambari that should be escaping (or validating?) 
> the value or Oozie that fails to read the value as single string.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build is back to normal : oozie-trunk-find-patches-available #142622

2016-12-13 Thread Apache Jenkins Server 
See

[jira] [Commented] (OOZIE-1770) Create Oozie Application Master for YARN

2016-12-13 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-1770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15745395#comment-15745395
 ] 

Hadoop QA commented on OOZIE-1770:
--

Testing JIRA OOZIE-1770

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:red}-1{color} the patch contains 2 line(s) with trailing spaces
.{color:red}-1{color} the patch contains 34 line(s) longer than 132 
characters
.{color:green}+1{color} the patch does adds/modifies 44 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:red}-1 JAVADOC{color}
.{color:red}-1{color} the patch seems to introduce 2 new Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:red}-1{color} There are [23] new bugs found below threshold in total 
that must be fixed.
.{color:green}+1{color} There are no new bugs found in [server].
.{color:red}-1{color} There are [1] new bugs found below threshold in 
[client] that must be fixed.
.You can find the FindBugs diff here (look for the red and orange ones): 
client/findbugs-new.html
.The most important FindBugs errors are:
.At OozieCLI.java:[line 1794]: Useless object stored in variable options of 
method org.apache.oozie.cli.OozieCLI.slaCommand(CommandLine)
.{color:green}+1{color} There are no new bugs found in [docs].
.{color:red}-1{color} There are [1] new bugs found below threshold in 
[sharelib/hive] that must be fixed.
.You can find the FindBugs diff here (look for the red and orange ones): 
sharelib/hive/findbugs-new.html
.The most important FindBugs errors are:
.Dereferenced at HiveMain.java:[line 244]: Possible null pointer 
dereference in org.apache.oozie.action.hadoop.HiveMain.run(String[]) due to 
return value of called method
.Known null at HiveMain.java:[line 244]
.{color:red}-1{color} There are [1] new bugs found below threshold in 
[sharelib/spark] that must be fixed.
.You can find the FindBugs diff here (look for the red and orange ones): 
sharelib/spark/findbugs-new.html
.The most important FindBugs errors are:
.Dereferenced at SparkMain.java:[line 324]: Possible null pointer 
dereference in 
org.apache.oozie.action.hadoop.SparkMain.getMatchingFile(Pattern) due to return 
value of called method
.Known null at SparkMain.java:[line 324]
.{color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
.{color:red}-1{color} There are [1] new bugs found below threshold in 
[sharelib/hive2] that must be fixed.
.You can find the FindBugs diff here (look for the red and orange ones): 
sharelib/hive2/findbugs-new.html
.The most important FindBugs errors are:
.Dereferenced at Hive2Main.java:[line 151]: Possible null pointer 
dereference in org.apache.oozie.action.hadoop.Hive2Main.run(String[]) due to 
return value of called method
.Known null at Hive2Main.java:[line 151]
.{color:green}+1{color} There are no new bugs found in [sharelib/streaming].
.{color:green}+1{color} There are no new bugs found in [sharelib/pig].
.{color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
.{color:green}+1{color} There are no new bugs found in [sharelib/distcp].
.{color:red}-1{color} There are [6] new bugs found below threshold in 
[sharelib/oozie] that must be fixed.
.You can find the FindBugs diff here (look for the red and orange ones): 
sharelib/oozie/findbugs-new.html
.The most important FindBugs errors are:
.At HdfsOperations.java:[line 67]: 
org.apache.oozie.action.hadoop.HdfsOperations$1.run() makes inefficient use of 
keySet iterator instead of entrySet iterator
.At HdfsOperations.java:[line 110]: Found reliance on default encoding in 
org.apache.oozie.action.hadoop.HdfsOperations$3.run(): new 
java.io.OutputStreamWriter(OutputStream)
.At HdfsOperations.java:[line 124]: Found reliance on default encoding in 
org.apache.oozie.action.hadoop.HdfsOperations$4.run(): new 
java.io.InputStreamReader(InputStream)
.Dereferenced at LauncherMapper.java:[line 584]: Possible null pointer 
dereference in 
org.apache.oozie.action.hadoop.LauncherMapper.printContentsOfCurrentDir() due 
to return value of called method
.Possible null pointer dereference in 
org.apache.oozie.action.hadoop.LauncherMapper.printContentsOfCurrentDir() due 
to return value of called method: Known null at LauncherMapper.java:[line 583]
.Known null at LauncherMapper.java:[line 591]: Dereferenced at

Build failed in Jenkins: oozie-trunk-precommit-build #3509

2016-12-13 Thread Apache Jenkins Server 
See 

Changes:

[rkanter] OOZIE-2616 Add Tez profile for Hive and Pig sharelibs (poeppt via

--
[...truncated 23673 lines...]
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/oozie-sharelib-hive-4.4.0-SNAPSHOT.jar
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/oozie-sharelib-spark-4.4.0-SNAPSHOT.jar
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/hadoop-yarn-server-applicationhistoryservice-2.6.0.jar
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/hadoop-yarn-server-web-proxy-2.6.0.jar
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/oozie-sharelib-hcatalog-4.4.0-SNAPSHOT.jar
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/oozie-sharelib-hive2-4.4.0-SNAPSHOT.jar
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/oozie-sharelib-streaming-4.4.0-SNAPSHOT.jar
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/hadoop-yarn-server-resourcemanager-2.6.0.jar
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/geronimo-annotation_1.0_spec-1.1.1.jar
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/oozie-sharelib-distcp-4.4.0-SNAPSHOT.jar
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/geronimo-j2ee-management_1.1_spec-1.0.1.jar
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/org.eclipse.jgit-3.3.1.201403241930-r.jar
 longer than 100 characters.
[WARNING] Entry: 
oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/oozie-sharelib-sqoop-4.4.0-SNAPSHOT.jar
 longer than 100 characters.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie ZooKeeper Security Tests 4.4.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
oozie-zookeeper-security-tests ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-zookeeper-security-tests ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ 
oozie-zookeeper-security-tests ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-zookeeper-security-tests ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-zookeeper-security-tests ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ 
oozie-zookeeper-security-tests ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ 
oozie-zookeeper-security-tests ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-zookeeper-security-tests ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ 
oozie-zookeeper-security-tests ---
[INFO] Reading assembly descriptor: src/main/assemblies/empty.xml
[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Oozie Main .. SUCCESS [  1.619 s]
[INFO] Apache Oozie Client  SUCCESS [ 14.960 s]
[INFO] Apache Oozie Share Lib Oozie ... SUCCESS [  3.044 s]
[INFO] Apache Oozie Share Lib HCatalog  SUCCESS [  1.827 s]
[INFO]

[jira] [Updated] (OOZIE-1770) Create Oozie Application Master for YARN

2016-12-13 Thread Peter Cseh (JIRA) 

 [ 
https://issues.apache.org/jira/browse/OOZIE-1770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Cseh updated OOZIE-1770:
--
Attachment: OOZIE-1770-001.patch

Attaching a patch that produces a green build.


> Create Oozie Application Master for YARN
> 
>
> Key: OOZIE-1770
> URL: https://issues.apache.org/jira/browse/OOZIE-1770
> Project: Oozie
>  Issue Type: New Feature
>Reporter: Bowen Zhang
>Assignee: Peter Cseh
> Attachments: OOZIE-1770-001.patch, OYA Scoping Doc 001.pdf, 
> OozieYarnAM.pdf, Prelim OYA Scoping Doc 001.pdf, Prelim OYA Scoping Doc 
> 002.pdf, Prelim OYA Scoping Doc 003.pdf, oya-rm-screenshot.jpg, oya.patch
>
>
> After the first release of oozie on hadoop 2, it will be good if users can 
> set execution engine in oozie conf, be it YARN AM or traditional MR. We can 
> target this for post oozie 4.1 release.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (OOZIE-1770) Create Oozie Application Master for YARN

2016-12-13 Thread Peter Cseh (JIRA) 

 [ 
https://issues.apache.org/jira/browse/OOZIE-1770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Cseh reassigned OOZIE-1770:
-

Assignee: Peter Cseh  (was: Robert Kanter)

> Create Oozie Application Master for YARN
> 
>
> Key: OOZIE-1770
> URL: https://issues.apache.org/jira/browse/OOZIE-1770
> Project: Oozie
>  Issue Type: New Feature
>Reporter: Bowen Zhang
>Assignee: Peter Cseh
> Attachments: OYA Scoping Doc 001.pdf, OozieYarnAM.pdf, Prelim OYA 
> Scoping Doc 001.pdf, Prelim OYA Scoping Doc 002.pdf, Prelim OYA Scoping Doc 
> 003.pdf, oya-rm-screenshot.jpg, oya.patch
>
>
> After the first release of oozie on hadoop 2, it will be good if users can 
> set execution engine in oozie conf, be it YARN AM or traditional MR. We can 
> target this for post oozie 4.1 release.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-1770) Create Oozie Application Master for YARN

2016-12-13 Thread Peter Cseh (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-1770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15745307#comment-15745307
 ] 

Peter Cseh commented on OOZIE-1770:
---

The patch contains the work of [~pbacsko], [~asasvari], [~andras.piros] and 
[~gezapeti].
We would like to have OOZIE-2686 and OOZIE-2687 fixed soon, but I don't think 
it's blocking this merge.
We have several green Oozie builds with this patch, so the Oozie master will be 
usabe.
[~puru], [~rohini], [~jaydeepvishwakarma], [~rkanter], [~qwertymaniac], please 
try to review the change.
I've added a ReviewBoard link: https://reviews.apache.org/r/54705/


> Create Oozie Application Master for YARN
> 
>
> Key: OOZIE-1770
> URL: https://issues.apache.org/jira/browse/OOZIE-1770
> Project: Oozie
>  Issue Type: New Feature
>Reporter: Bowen Zhang
>Assignee: Peter Cseh
> Attachments: OOZIE-1770-001.patch, OYA Scoping Doc 001.pdf, 
> OozieYarnAM.pdf, Prelim OYA Scoping Doc 001.pdf, Prelim OYA Scoping Doc 
> 002.pdf, Prelim OYA Scoping Doc 003.pdf, oya-rm-screenshot.jpg, oya.patch
>
>
> After the first release of oozie on hadoop 2, it will be good if users can 
> set execution engine in oozie conf, be it YARN AM or traditional MR. We can 
> target this for post oozie 4.1 release.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (OOZIE-1770) Create Oozie Application Master for YARN

2016-12-13 Thread Peter Cseh (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-1770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15745307#comment-15745307
 ] 

Peter Cseh edited comment on OOZIE-1770 at 12/13/16 2:38 PM:
-

The patch contains the work of [~pbacsko], [~asasvari], [~andras.piros], 
[~rkanter] and [~gezapeti].
We would like to have OOZIE-2686 and OOZIE-2687 fixed soon, but I don't think 
it's blocking this merge. 
We have several green Oozie builds with this patch, so the Oozie master will be 
usabe.
The patch is huge already, we would like to finish up things in smaller patches.
[~puru], [~rohini], [~jaydeepvishwakarma], [~rkanter], [~qwertymaniac], please 
try to review the change.
I've added a ReviewBoard link: https://reviews.apache.org/r/54705/



was (Author: gezapeti):
The patch contains the work of [~pbacsko], [~asasvari], [~andras.piros] and 
[~gezapeti].
We would like to have OOZIE-2686 and OOZIE-2687 fixed soon, but I don't think 
it's blocking this merge.
We have several green Oozie builds with this patch, so the Oozie master will be 
usabe.
[~puru], [~rohini], [~jaydeepvishwakarma], [~rkanter], [~qwertymaniac], please 
try to review the change.
I've added a ReviewBoard link: https://reviews.apache.org/r/54705/


> Create Oozie Application Master for YARN
> 
>
> Key: OOZIE-1770
> URL: https://issues.apache.org/jira/browse/OOZIE-1770
> Project: Oozie
>  Issue Type: New Feature
>Reporter: Bowen Zhang
>Assignee: Peter Cseh
> Attachments: OOZIE-1770-001.patch, OYA Scoping Doc 001.pdf, 
> OozieYarnAM.pdf, Prelim OYA Scoping Doc 001.pdf, Prelim OYA Scoping Doc 
> 002.pdf, Prelim OYA Scoping Doc 003.pdf, oya-rm-screenshot.jpg, oya.patch
>
>
> After the first release of oozie on hadoop 2, it will be good if users can 
> set execution engine in oozie conf, be it YARN AM or traditional MR. We can 
> target this for post oozie 4.1 release.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (OOZIE-1770) Create Oozie Application Master for YARN

2016-12-13 Thread Peter Cseh (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-1770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15745307#comment-15745307
 ] 

Peter Cseh edited comment on OOZIE-1770 at 12/13/16 2:45 PM:
-

The patch contains the work of [~pbacsko], [~asasvari], [~andras.piros], 
[~rkanter] and [~gezapeti].
We would like to have OOZIE-2686 and OOZIE-2687 fixed soon, but I don't think 
it's blocking this merge. 
We have several green Oozie builds with this patch, so the Oozie master will be 
usabe.
The patch is huge already, we would like to finish up things in smaller patches.
[~puru], [~rohini], [~jaydeepvishwakarma], [~rkanter], [~qwertymaniac], please 
try to review the change.
I've added a ReviewBoard link: https://reviews.apache.org/r/54705/
This would also fix OOZIE-2761


was (Author: gezapeti):
The patch contains the work of [~pbacsko], [~asasvari], [~andras.piros], 
[~rkanter] and [~gezapeti].
We would like to have OOZIE-2686 and OOZIE-2687 fixed soon, but I don't think 
it's blocking this merge. 
We have several green Oozie builds with this patch, so the Oozie master will be 
usabe.
The patch is huge already, we would like to finish up things in smaller patches.
[~puru], [~rohini], [~jaydeepvishwakarma], [~rkanter], [~qwertymaniac], please 
try to review the change.
I've added a ReviewBoard link: https://reviews.apache.org/r/54705/


> Create Oozie Application Master for YARN
> 
>
> Key: OOZIE-1770
> URL: https://issues.apache.org/jira/browse/OOZIE-1770
> Project: Oozie
>  Issue Type: New Feature
>Reporter: Bowen Zhang
>Assignee: Peter Cseh
> Attachments: OOZIE-1770-001.patch, OYA Scoping Doc 001.pdf, 
> OozieYarnAM.pdf, Prelim OYA Scoping Doc 001.pdf, Prelim OYA Scoping Doc 
> 002.pdf, Prelim OYA Scoping Doc 003.pdf, oya-rm-screenshot.jpg, oya.patch
>
>
> After the first release of oozie on hadoop 2, it will be good if users can 
> set execution engine in oozie conf, be it YARN AM or traditional MR. We can 
> target this for post oozie 4.1 release.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

2016-12-13 Thread Peter Cseh 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/
---

Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.


Repository: oozie-git


Description
---

Replacing the launcher MR job with a launcher AM


Diffs
-

  client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
  client/src/main/java/org/apache/oozie/client/XOozieClient.java 
8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
  core/pom.xml 1dd104581ccab7976d788b674d2200e48a70c27f 
  core/src/main/java/org/apache/oozie/WorkflowJobBean.java 
55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
  core/src/main/java/org/apache/oozie/action/ActionExecutor.java 
1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
  core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java 
eadb47b07d157f45f086517ab41c4b9826f003f0 
  core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 
6fe22fbd2fca5d873d58ece60f20d0f131371475 
  core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 
78cd257a836707c1efeac13566d6285d42970545 
  core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 
27654741101d2056b25626fbbe345183eafd612d 
  core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 
801bfe6eb212b549888f0e9fb7df7b65bcf59393 
  core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 
36ad1df727e869b8051481ad969771e91bc6ae07 
  core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java 
ad2a71d641caa71efc098ad00bfbab3cec4d9506 
  core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 
307f565786ad77f5a35b5790bd5b052b04c6eac8 
  core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 
2aed936af75da38c433be5d202bdc01c2f612c76 
  core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 
8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
  core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
4beeb965a461e246479c3e8ad88f92ecafca13c1 
  core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 
9609fdc2f3440fb176b659ef16c2a23451724cf9 
  
core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 
55c937249ac35e14bd63dd0c1fa962ea636afa3a 
  core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 
581d3b3219eff882f961c8e495611e873c6a5a52 
  core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 
8b2dc16db80ad602911f1f64e60cfad41b198922 
  
core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java
 f2541265ca71fb7f3b053c59775ef5e23d0d8f94 
  core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java 
b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
  core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 
8e8d7d308b294efd91e3057e0d88ee06643ab2d4 
  core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 
6cee32aa8f5b8b36c9c7a98bb738f50b18b64624 
  core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java 
d2a2742ae418070a1cd0a889f14194224bc944a6 
  core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java 
cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
  
core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java
 9d413059e99bc676feb63e02a2f704048eb4a355 
  core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 
51b739e21c802e09f4110b740138f4ff5c341fe4 
  core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 
5377127736cf189bcf4aa3d62bbbeab6b3a29d52 
  core/src/main/java/org/apache/oozie/service/Services.java 
829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
  core/src/main/java/org/apache/oozie/service/ShareLibService.java 
b79bd37bb2b44348f220128334497d788dd922d4 
  core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 
612302173e0f485345558b0c96a8a75d2f43d238 
  core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
  core/src/main/java/org/apache/oozie/util/JobUtils.java 
a7a53b355d194b55959ede20c85d38e02294853d 
  core/src/main/resources/META-INF/persistence.xml 
edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
  core/src/main/resources/oozie-default.xml 
2d7650acee73af713e2e3ea70975e74d1866cfd4 
  core/src/main/resources/oozie-log4j.properties 
c86b30143f5f58135493d9e6bdba29f402a0b1fd 
  core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/action/hadoop/BlockingMapper.java 
PRE-CREATION 
  core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 
4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
  core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 
8f08ddd9fa077fc9526814fd25e7477850c51737 
  
core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java 
e8a140f330062c5ce8324112aae83b3d0ae1f772

[jira] [Commented] (OOZIE-2752) Separate WAR and embedded Jetty into different profiles

2016-12-13 Thread Attila Sasvari (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15745606#comment-15745606
 ] 

Attila Sasvari commented on OOZIE-2752:
---

Based on the execution time of {{mvn -o clean package assembly:single 
-DskipTests=true}} on my machine, I don't think there is real savings :
* if packaging type is {{jar}} in webapp
{noformat}
[INFO] Total time: 01:59 min
{noformat}

* if packaging type is {{war}} in webapp
{noformat}
[INFO] Total time: 02:04 min
{noformat}

(passing {{-o}} so that maven does not download dependencies)

Build time is dominated by Apache Oozie Core, Oozie Share Lib, Oozie Distro

> Separate WAR and embedded Jetty into different profiles
> ---
>
> Key: OOZIE-2752
> URL: https://issues.apache.org/jira/browse/OOZIE-2752
> Project: Oozie
>  Issue Type: Sub-task
>  Components: build
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Assignee: Attila Sasvari
> Fix For: 5.0.0
>
>
> Create a "war" profile to only build a WAR file, and update the default build 
> to only do the embedded Jetty option.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-1770) Create Oozie Application Master for YARN

2016-12-13 Thread Peter Bacsko (JIRA) 

[ 
https://issues.apache.org/jira/browse/OOZIE-1770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15745560#comment-15745560
 ] 

Peter Bacsko commented on OOZIE-1770:
-

Ouch... I'll address findbugs and formatting issues.

> Create Oozie Application Master for YARN
> 
>
> Key: OOZIE-1770
> URL: https://issues.apache.org/jira/browse/OOZIE-1770
> Project: Oozie
>  Issue Type: New Feature
>Reporter: Bowen Zhang
>Assignee: Peter Cseh
> Attachments: OOZIE-1770-001.patch, OYA Scoping Doc 001.pdf, 
> OozieYarnAM.pdf, Prelim OYA Scoping Doc 001.pdf, Prelim OYA Scoping Doc 
> 002.pdf, Prelim OYA Scoping Doc 003.pdf, oya-rm-screenshot.jpg, oya.patch
>
>
> After the first release of oozie on hadoop 2, it will be good if users can 
> set execution engine in oozie conf, be it YARN AM or traditional MR. We can 
> target this for post oozie 4.1 release.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (OOZIE-2752) Separate WAR and embedded Jetty into different profiles

2016-12-13 Thread Robert Kanter (JIRA) 

 [ 
https://issues.apache.org/jira/browse/OOZIE-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Kanter resolved OOZIE-2752.
--
Resolution: Not A Problem

Ok, then I think we may as well just always do both.  Thanks for checking.

> Separate WAR and embedded Jetty into different profiles
> ---
>
> Key: OOZIE-2752
> URL: https://issues.apache.org/jira/browse/OOZIE-2752
> Project: Oozie
>  Issue Type: Sub-task
>  Components: build
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Assignee: Attila Sasvari
> Fix For: 5.0.0
>
>
> Create a "war" profile to only build a WAR file, and update the default build 
> to only do the embedded Jetty option.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)