Jenkins build is back to normal : oozie-trunk-find-patches-available #136198

[Apache Jenkins Server]

See 

Build failed in Jenkins: oozie-trunk-find-patches-available #136197

[Apache Jenkins Server]

See 

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on H1 (yahoo-not-h2 Hadoop Mapreduce Hdfs Pig) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/oozie.git # timeout=10
Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/oozie.git
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/oozie.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision bd54452808335cbadc48d4aacd0bd23374bed928 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f bd54452808335cbadc48d4aacd0bd23374bed928
 > git rev-list bd54452808335cbadc48d4aacd0bd23374bed928 # timeout=10
[oozie-trunk-find-patches-available] $ /bin/bash 
/tmp/hudson3620212925226023422.sh
mkdir: cannot create directory 
':
 File exists
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
100  245k0  245k0 0   227k  0 --:--:--  0:00:01 --:--:--  227k
100  499k0  499k0 0   243k  0 --:--:--  0:00:02 --:--:--  243k
100  851k0  851k0 0   279k  0 --:--:--  0:00:03 --:--:--  279k
100 1202k0 1202k0 0   299k  0 --:--:--  0:00:04 --:--:--  299k
curl: (18) transfer closed with outstanding read data remaining
Could not retrieve available patches from JIRA
Build step 'Execute shell' marked build as failure

[jira] [Commented] (OOZIE-2666) Support embedding Jetty into Oozie

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15573451#comment-15573451
 ] 

Hadoop QA commented on OOZIE-2666:
--

Testing JIRA OOZIE-2666

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:red}-1{color} the patch contains 13 line(s) with tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 6 line(s) longer than 132 
characters
.{color:green}+1{color} the patch does adds/modifies 2 testcase(s)
{color:red}-1 RAT{color}
.{color:red}-1{color} the patch seems to introduce 3 new RAT warning(s)
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:red}-1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:red}-1{color} the patch seems to introduce 3 new javac warning(s)
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1821
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/3382/

> Support embedding Jetty into Oozie
> --
>
> Key: OOZIE-2666
> URL: https://issues.apache.org/jira/browse/OOZIE-2666
> Project: Oozie
>  Issue Type: Sub-task
>  Components: core
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2666.PATCH-01.patch, OOZIE-2666.PATCH-02.patch, 
> OOZIE-2666.PATCH-03.patch, OOZIE-2666.PATCH-05.patch, 
> OOZIE-2666.PATCH-06.patch, OOZIE-2666.PATCH-07.patch, 
> OOZIE-2666.PATCH-09.patch, OOZIE-2666.PATCH-10.patch, 
> OOZIE-2666.PATCH-11.patch, OOZIE-2666.PATCH-12.patch
>
>
> The purpose of this task is to allow Oozie to run as a standalone application 
> rather than being deployed to a container inside a WAR file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: oozie-trunk-precommit-build #3382

[Apache Jenkins Server]

See 

--
[...truncated 5832 lines...]
[INFO] Copying hive-shims-0.20S-1.2.0.jar to 

[INFO] Copying org.eclipse.jdt.core-3.8.2.v20130121.jar to 

[INFO] Copying quartz-2.1.7.jar to 

[INFO] Copying oozie-hadoop-auth-hadoop-2-4.4.0-SNAPSHOT.jar to 

[INFO] Copying mockito-all-1.10.19.jar to 

[INFO] Copying guice-servlet-3.0.jar to 

[INFO] Copying jung-algorithms-2.0.1.jar to 

[INFO] Copying libfb303-0.9.2.jar to 

[INFO] Copying stringtemplate-3.2.1.jar to 

[INFO] Copying stax-api-1.0-2.jar to 

[INFO] Copying hive-metastore-1.2.0.jar to 

[INFO] Copying jetty-util-9.2.19.v20160908.jar to 

[INFO] Copying calcite-core-1.2.0-incubating.jar to 

[INFO] Copying oozie-core-4.4.0-SNAPSHOT.jar to 

[INFO] Copying hive-service-1.2.0.jar to 

[INFO] Copying jetty-jndi-9.2.19.v20160908.jar to 

[INFO] Copying taglibs-standard-spec-1.2.1.jar to 

[INFO] Copying hadoop-yarn-client-2.4.0.jar to 

[INFO] Copying curator-framework-2.5.0.jar to 

[INFO] Copying servlet-api-2.5.jar to 

[INFO] Copying libthrift-0.9.2.jar to 

[INFO] Copying antlr-runtime-3.4.jar to 

[INFO] Copying geronimo-jms_1.1_spec-1.1.1.jar to 

[INFO] Copying ivy-2.4.0.jar to 

[INFO] Copying hadoop-yarn-common-2.4.0.jar to 

[INFO] Copying metrics-core-3.0.2.jar to 

[INFO] Copying jackson-core-asl-1.9.13.jar to 

[INFO] Copying commons-jexl-2.1.1.jar to 

[INFO] Copying commons-pool-1.5.4.jar to 

[INFO] Copying jaxb-impl-2.2.3-1.jar to 

[INFO] Copying jets3t-0.9.0.jar to 

Re: Review Request 52399: OOZIE-2666 Support embedding Jetty into Oozie

[Attila Sasvari]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52399/
---

(Updated Oct. 13, 2016, 9:53 p.m.)


Review request for oozie, András Piros, Peter Cseh, Peter Bacsko, and Robert 
Kanter.


Repository: oozie-git


Description
---

Embedding jetty into Oozie so that it can run as a standalone application. The 
changes also try to address OOZIE-2317 (i.e. Tomcat 6 is EOL).

New functionality
- New module (server) is added that sets up an embedded Jetty server and start 
Oozie services. Servlet mapping is done by reading web.xml of webapp at 
runtime. JSP is handled with custom code. Server version is not revealed in 
server repsonses.
- SSL protocols and ciphers can be configured via system properties and 
environment variables. Precedence: system properties, environment variables, 
default values
   
Changes in existing code
- Excluded jetty 6 dependencies from core and updated tests accordingly  

Packaging
- oozie.sh is modified so that it starts Oozie with embedded jetty by default. 
If someone would like to use tomcat for any reason, they can set an environment 
variable (e.g. OOZIE_USE_TOMCAT=1).

TODO:
- Add more tests
- Add more documentation
- Code cleanup + refactoring in packaging and core parts
- Maven clean up
- Allow to tune more Jetty settings (for example threadpool)
- More security measures (e.g. protect against clickjacking, CSRF, etc.)
- Update Oozie Documentation


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/util/Instrumentation.java 
99d64acec76a7f23a0d5b7fc98a2b9620b3b9a7d 
  core/src/main/resources/oozie-default.xml 
e71ebe3b7a85e6b23176ef30713af63847144498 
  distro/pom.xml c50572c57a376b28963d4e7da8ac7df777fe0480 
  distro/src/main/bin/oozie-jetty-server.sh PRE-CREATION 
  distro/src/main/bin/oozie-setup.sh 79b049bccceb2690f8a673a885a615c8d4d9578c 
  distro/src/main/bin/oozied.sh a869c3da177c863a068f2af45c7bca9d5cb771ac 
  pom.xml 83a1d4254787c2a810e9db37a5d1433b96e01986 
  server/pom.xml PRE-CREATION 
  server/src/main/assemblies/empty.xml PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java 
PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/HttpConfigurationWrapper.java 
PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/JspHandler.java PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/SSLServerConnectorFactory.java 
PRE-CREATION 
  server/src/main/resources/checkstyle-header.txt PRE-CREATION 
  server/src/main/resources/checkstyle.xml PRE-CREATION 
  server/src/test/java/org/apache/oozie/server/TestEmbeddedOozieServer.java 
PRE-CREATION 
  
server/src/test/java/org/apache/oozie/server/TestSSLServerConnectorFactory.java 
PRE-CREATION 
  src/main/assemblies/distro.xml 1ffbfd6d2ba33b390999e9094cbb336fbce45c21 

Diff: https://reviews.apache.org/r/52399/diff/


Testing
---

- Tested basic functionality by executing a workflow that uses the sample 
JavaAction
- without SSL - on a 2.4.0 pseudo Hadoop cluster
- SSL with Kerberos is using a test CDH cluster 
- Added new unit tests that check
- If oozie.ssl.enabled is not specified, server starts without SSL settings 
- If oozie.ssl.enabled is specified, server starts with SSL settings
- SSL protocols and ciphers can be configured via system properties and 
environment variables 
- Ran subset of tests using Hadoop-2 profile
- mvn clean package assembly:single   -DjavaVersion=1.8 -DtargetVersion=1.7 
 -Dtest=TestJavaActionExecutor  -Phadoop-2 -Dhadoop.version=2.4.0


Thanks,

Attila Sasvari

[jira] [Updated] (OOZIE-2666) Support embedding Jetty into Oozie

[Attila Sasvari (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Sasvari updated OOZIE-2666:
--
Attachment: OOZIE-2666.PATCH-12.patch

> Support embedding Jetty into Oozie
> --
>
> Key: OOZIE-2666
> URL: https://issues.apache.org/jira/browse/OOZIE-2666
> Project: Oozie
>  Issue Type: Sub-task
>  Components: core
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2666.PATCH-01.patch, OOZIE-2666.PATCH-02.patch, 
> OOZIE-2666.PATCH-03.patch, OOZIE-2666.PATCH-05.patch, 
> OOZIE-2666.PATCH-06.patch, OOZIE-2666.PATCH-07.patch, 
> OOZIE-2666.PATCH-09.patch, OOZIE-2666.PATCH-10.patch, 
> OOZIE-2666.PATCH-11.patch, OOZIE-2666.PATCH-12.patch
>
>
> The purpose of this task is to allow Oozie to run as a standalone application 
> rather than being deployed to a container inside a WAR file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 52737: OOZIE-2698 OYA: Refactor LauncherAM to make it more testable

[Attila Sasvari]

> On Oct. 13, 2016, 6:12 p.m., Attila Sasvari wrote:
> > sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java,
> >  line 103
> > 
> >
> > The second parameter of checkNotNull is incorrect, it is a String
> > that is used in the detail message of the exception:
> > 
> > ```
> > public static  T checkNotNull(T o, String name) throws 
> > NullPointerException {
> > if(o == null) {
> > throw new NullPointerException(name + " should not be 
> > null");
> > } else {
> > return o;
> > }
> > }
> > ```

false alarm. this checknotnull was from parquet...


- Attila


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52737/#review152545
---


On Oct. 11, 2016, 1:42 p.m., Peter Bacsko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52737/
> ---
> 
> (Updated Oct. 11, 2016, 1:42 p.m.)
> 
> 
> Review request for oozie, András Piros, Attila Sasvari, Peter Cseh, and 
> Robert Kanter.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> This review contains a suggested approach for refactoring LauncherAM on the 
> OYA branch.
> 
> The main idea is to inject all dependencies of the class via the constructor. 
> In some cases, this requires factory objects. Note that this is necessary 
> because in unit tests, because we don't want to create actual resources such 
> as UGI or async ResourceManager callback.
> 
> In the tests, we can instantiate this class with mocks. The testability is 
> greatly enhanced because various checks and verifications can be performed on 
> the mocks. Tests are currently not in this review (see 
> https://issues.apache.org/jira/browse/OOZIE-2685).
> 
> 
> Diffs
> -
> 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java 
> ed29299 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java
>  PRE-CREATION 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java
>  PRE-CREATION 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java 
> PRE-CREATION 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsAndLocalFSOperations.java
>  PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java 
> 85d78c6 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java
>  23648b8 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifierFactory.java
>  PRE-CREATION 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java
>  4a51d48 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java
>  PRE-CREATION 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/SequenceFileWriterFactory.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/52737/diff/
> 
> 
> Testing
> ---
> 
> Unit tests are being written right now.
> 
> 
> Thanks,
> 
> Peter Bacsko
> 
>

Re: Review Request 52227: OOZIE-2225 Add wild card filter for gathering jobs

[Sai Chirravuri]

> On Oct. 13, 2016, 4:50 a.m., Abhishek Bafna wrote:
> > core/src/main/java/org/apache/oozie/executor/jpa/WorkflowsJobGetJPAExecutor.java,
> >  line 279
> > 
> >
> > Can you make this to use 'else if' instead of 'if' conditions. You 
> > might need to change couple of 'if' conditions to 'else if' in this method.

It is possible to use more than one filter. Hence 'if' and not 'else if'. Am I 
missing something  ?


- Sai


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52227/#review152448
---


On Oct. 1, 2016, 12:29 a.m., Sai Chirravuri wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52227/
> ---
> 
> (Updated Oct. 1, 2016, 12:29 a.m.)
> 
> 
> Review request for oozie and Robert Kanter.
> 
> 
> Bugs: OOZIE-2225
> https://issues.apache.org/jira/browse/OOZIE-2225
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> commit 05c108f61f4c3da437d826b73251e92662a4518b
> Author: krish 
> Date:   Fri Sep 23 13:50:24 2016 -0700
> 
> OOZIE-2225 Add wild card filter for gathering jobs
> 
> :100644 100644 f1d0f2b... f10de36... M  
> client/src/main/java/org/apache/oozie/cli/OozieCLI.java
> :100644 100644 a882cab... 53a6ecf... M  
> client/src/main/java/org/apache/oozie/client/OozieClient.java
> :100644 100644 01aeddf... 7597142... M  
> core/src/main/java/org/apache/oozie/DagEngine.java
> :100644 100644 3c03267... 275aabf... M  
> core/src/main/java/org/apache/oozie/executor/jpa/WorkflowsJobGetJPAExecutor.java
> :100644 100644 fb1db29... 80f0756... M  
> core/src/main/java/org/apache/oozie/store/StoreStatusFilter.java
> :00 100644 000... dd1e943... A  
> core/src/main/java/org/apache/oozie/util/db/JPAUtils.java
> :100644 100644 0b1f644... 87fccd6... M  
> core/src/test/java/org/apache/oozie/executor/jpa/TestBundleJobInfoGetJPAExecutor.java
> :100644 100644 7d468f1... 4378286... M  
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobInfoGetJPAExecutor.java
> :100644 100644 ddb5506... 12cb16b... M  
> core/src/test/java/org/apache/oozie/executor/jpa/TestWorkflowsJobGetJPAExecutor.java
> :100644 100644 ea778bd... 2fed1f8... M  
> core/src/test/java/org/apache/oozie/test/XDataTestCase.java
> :100644 100644 567e260... 3daf38a... M  
> docs/src/site/twiki/DG_CommandLineTool.twiki
> :100644 100644 13e1691... a66e564... M  
> docs/src/site/twiki/WebServicesAPI.twiki
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 
> f1d0f2b7442e04e01bea93622cf378fc92934497 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java 
> a882cab95c4b75c3fc641fb8e5511357d482eeff 
>   core/src/main/java/org/apache/oozie/CoordinatorEngine.java 
> cec6347905233d9a5fe0eb0e2fa27336a6a967a9 
>   core/src/main/java/org/apache/oozie/DagEngine.java 
> 01aeddfc282261d9ad396565a9d3d30efec2dc59 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/WorkflowsJobGetJPAExecutor.java
>  3c032678d06641ae7bb2f2a9c0631948190ea662 
>   core/src/main/java/org/apache/oozie/store/StoreStatusFilter.java 
> fb1db29cdfe1dbe37c8e2176079f39f841232ee3 
>   core/src/main/java/org/apache/oozie/util/JobsFilterUtils.java 
> 18df836713918a2864566b56646f711dd84a5f97 
>   
> core/src/test/java/org/apache/oozie/executor/jpa/TestBundleJobInfoGetJPAExecutor.java
>  0b1f644c1ff74ce24b78eadf0ed7c0cbc87e93a3 
>   
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobInfoGetJPAExecutor.java
>  7d468f16981710d96473155df25b8de0d5ef322b 
>   
> core/src/test/java/org/apache/oozie/executor/jpa/TestWorkflowsJobGetJPAExecutor.java
>  ddb5506de90ae32d760657f8a867ab46599f1a86 
>   docs/src/site/twiki/DG_CommandLineTool.twiki 
> 567e260dd3e05c7619d957520ed49882b0cf37f2 
>   docs/src/site/twiki/WebServicesAPI.twiki 
> 13e1691c3497bed94957ef959e16e788d1196c67 
> 
> Diff: https://reviews.apache.org/r/52227/diff/
> 
> 
> Testing
> ---
> 
> CLI manual testing looks good and unit tests passed
> 
> 
> Thanks,
> 
> Sai Chirravuri
> 
>

[jira] [Commented] (OOZIE-2666) Support embedding Jetty into Oozie

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15572804#comment-15572804
 ] 

Hadoop QA commented on OOZIE-2666:
--

Testing JIRA OOZIE-2666

Cleaning local git workspace



{color:red}-1{color} Patch failed to apply to head of branch



> Support embedding Jetty into Oozie
> --
>
> Key: OOZIE-2666
> URL: https://issues.apache.org/jira/browse/OOZIE-2666
> Project: Oozie
>  Issue Type: Sub-task
>  Components: core
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2666.PATCH-01.patch, OOZIE-2666.PATCH-02.patch, 
> OOZIE-2666.PATCH-03.patch, OOZIE-2666.PATCH-05.patch, 
> OOZIE-2666.PATCH-06.patch, OOZIE-2666.PATCH-07.patch, 
> OOZIE-2666.PATCH-09.patch, OOZIE-2666.PATCH-10.patch, 
> OOZIE-2666.PATCH-11.patch
>
>
> The purpose of this task is to allow Oozie to run as a standalone application 
> rather than being deployed to a container inside a WAR file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: oozie-trunk-precommit-build #3381

[Apache Jenkins Server]

See 

--
[...truncated 1328 lines...]
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-docs 
---
[INFO] Compiling 1 source file to 

[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-docs ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-docs ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-docs ---
[INFO] Tests are skipped.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie WebApp 4.4.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ oozie-webapp ---
[INFO] Deleting 

[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-webapp 
---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-webapp ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-webapp 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-webapp ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-webapp ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-webapp ---
[INFO] Tests are skipped.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie Tools 4.4.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ oozie-tools ---
[INFO] Deleting 

[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-tools ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-tools ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-tools 
---
[INFO] Compiling 4 source files to 

[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-tools ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 11 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-tools ---
[INFO] Compiling 7 source files to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-tools ---
[INFO] Tests are skipped.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie MiniOozie 4.4.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ oozie-mini ---
[INFO] Deleting 

[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-mini ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[jira] [Commented] (OOZIE-2666) Support embedding Jetty into Oozie

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15572796#comment-15572796
 ] 

Hadoop QA commented on OOZIE-2666:
--

Testing JIRA OOZIE-2666

Cleaning local git workspace



{color:red}-1{color} Patch failed to apply to head of branch



> Support embedding Jetty into Oozie
> --
>
> Key: OOZIE-2666
> URL: https://issues.apache.org/jira/browse/OOZIE-2666
> Project: Oozie
>  Issue Type: Sub-task
>  Components: core
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2666.PATCH-01.patch, OOZIE-2666.PATCH-02.patch, 
> OOZIE-2666.PATCH-03.patch, OOZIE-2666.PATCH-05.patch, 
> OOZIE-2666.PATCH-06.patch, OOZIE-2666.PATCH-07.patch, 
> OOZIE-2666.PATCH-09.patch, OOZIE-2666.PATCH-10.patch, 
> OOZIE-2666.PATCH-11.patch
>
>
> The purpose of this task is to allow Oozie to run as a standalone application 
> rather than being deployed to a container inside a WAR file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: oozie-trunk-precommit-build #3380

[Apache Jenkins Server]

See 

--
[...truncated 1327 lines...]
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-docs 
---
[INFO] Compiling 1 source file to 

[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-docs ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-docs ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-docs ---
[INFO] Tests are skipped.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie WebApp 4.4.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ oozie-webapp ---
[INFO] Deleting 

[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-webapp 
---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-webapp ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-webapp 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-webapp ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-webapp ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-webapp ---
[INFO] Tests are skipped.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie Tools 4.4.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ oozie-tools ---
[INFO] Deleting 

[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-tools ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-tools ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-tools 
---
[INFO] Compiling 4 source files to 

[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-tools ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 11 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-tools ---
[INFO] Compiling 7 source files to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-tools ---
[INFO] Tests are skipped.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie MiniOozie 4.4.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ oozie-mini ---
[INFO] Deleting 

[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-mini ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[jira] [Updated] (OOZIE-2666) Support embedding Jetty into Oozie

[Attila Sasvari (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Sasvari updated OOZIE-2666:
--
Attachment: OOZIE-2666.PATCH-11.patch

> Support embedding Jetty into Oozie
> --
>
> Key: OOZIE-2666
> URL: https://issues.apache.org/jira/browse/OOZIE-2666
> Project: Oozie
>  Issue Type: Sub-task
>  Components: core
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2666.PATCH-01.patch, OOZIE-2666.PATCH-02.patch, 
> OOZIE-2666.PATCH-03.patch, OOZIE-2666.PATCH-05.patch, 
> OOZIE-2666.PATCH-06.patch, OOZIE-2666.PATCH-07.patch, 
> OOZIE-2666.PATCH-09.patch, OOZIE-2666.PATCH-10.patch, 
> OOZIE-2666.PATCH-11.patch
>
>
> The purpose of this task is to allow Oozie to run as a standalone application 
> rather than being deployed to a container inside a WAR file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 52399: OOZIE-2666 Support embedding Jetty into Oozie

[Attila Sasvari]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52399/
---

(Updated Oct. 13, 2016, 6:35 p.m.)


Review request for oozie, András Piros, Peter Cseh, Peter Bacsko, and Robert 
Kanter.


Repository: oozie-git


Description
---

Embedding jetty into Oozie so that it can run as a standalone application. The 
changes also try to address OOZIE-2317 (i.e. Tomcat 6 is EOL).

New functionality
- New module (server) is added that sets up an embedded Jetty server and start 
Oozie services. Servlet mapping is done by reading web.xml of webapp at 
runtime. JSP is handled with custom code. Server version is not revealed in 
server repsonses.
- SSL protocols and ciphers can be configured via system properties and 
environment variables. Precedence: system properties, environment variables, 
default values
   
Changes in existing code
- Excluded jetty 6 dependencies from core and updated tests accordingly  

Packaging
- oozie.sh is modified so that it starts Oozie with embedded jetty by default. 
If someone would like to use tomcat for any reason, they can set an environment 
variable (e.g. OOZIE_USE_TOMCAT=1).

TODO:
- Add more tests
- Add more documentation
- Code cleanup + refactoring in packaging and core parts
- Maven clean up
- Allow to tune more Jetty settings (for example threadpool)
- More security measures (e.g. protect against clickjacking, CSRF, etc.)
- Update Oozie Documentation


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/util/Instrumentation.java 
fa1e92a0f1fadfb66c5e66fea5f26f57579080e7 
  core/src/main/resources/oozie-default.xml 
e71ebe3b7a85e6b23176ef30713af63847144498 
  distro/pom.xml c50572c57a376b28963d4e7da8ac7df777fe0480 
  distro/src/main/bin/oozie-jetty-server.sh PRE-CREATION 
  distro/src/main/bin/oozie-setup.sh 79b049bccceb2690f8a673a885a615c8d4d9578c 
  distro/src/main/bin/oozied.sh a869c3da177c863a068f2af45c7bca9d5cb771ac 
  pom.xml 704a29f4e4805e3e08c2a547b2a375f6b1b2 
  server/pom.xml PRE-CREATION 
  server/src/main/assemblies/empty.xml PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java 
PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/HttpConfigurationWrapper.java 
PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/JspHandler.java PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/SSLServerConnectorFactory.java 
PRE-CREATION 
  server/src/main/resources/checkstyle-header.txt PRE-CREATION 
  server/src/main/resources/checkstyle.xml PRE-CREATION 
  server/src/test/java/org/apache/oozie/server/TestEmbeddedOozieServer.java 
PRE-CREATION 
  
server/src/test/java/org/apache/oozie/server/TestSSLServerConnectorFactory.java 
PRE-CREATION 
  src/main/assemblies/distro.xml 1ffbfd6d2ba33b390999e9094cbb336fbce45c21 

Diff: https://reviews.apache.org/r/52399/diff/


Testing
---

- Tested basic functionality by executing a workflow that uses the sample 
JavaAction
- without SSL - on a 2.4.0 pseudo Hadoop cluster
- SSL with Kerberos is using a test CDH cluster 
- Added new unit tests that check
- If oozie.ssl.enabled is not specified, server starts without SSL settings 
- If oozie.ssl.enabled is specified, server starts with SSL settings
- SSL protocols and ciphers can be configured via system properties and 
environment variables 
- Ran subset of tests using Hadoop-2 profile
- mvn clean package assembly:single   -DjavaVersion=1.8 -DtargetVersion=1.7 
 -Dtest=TestJavaActionExecutor  -Phadoop-2 -Dhadoop.version=2.4.0


Thanks,

Attila Sasvari

Jenkins build is back to normal : oozie-trunk-find-patches-available #136165

[Apache Jenkins Server]

See 

Re: Review Request 52399: OOZIE-2666 Support embedding Jetty into Oozie

[Attila Sasvari]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52399/
---

(Updated Oct. 13, 2016, 6:27 p.m.)


Review request for oozie, András Piros, Peter Cseh, Peter Bacsko, and Robert 
Kanter.


Changes
---

rebase


Repository: oozie-git


Description
---

Embedding jetty into Oozie so that it can run as a standalone application. The 
changes also try to address OOZIE-2317 (i.e. Tomcat 6 is EOL).

New functionality
- New module (server) is added that sets up an embedded Jetty server and start 
Oozie services. Servlet mapping is done by reading web.xml of webapp at 
runtime. JSP is handled with custom code. Server version is not revealed in 
server repsonses.
- SSL protocols and ciphers can be configured via system properties and 
environment variables. Precedence: system properties, environment variables, 
default values
   
Changes in existing code
- Excluded jetty 6 dependencies from core and updated tests accordingly  

Packaging
- oozie.sh is modified so that it starts Oozie with embedded jetty by default. 
If someone would like to use tomcat for any reason, they can set an environment 
variable (e.g. OOZIE_USE_TOMCAT=1).

TODO:
- Add more tests
- Add more documentation
- Code cleanup + refactoring in packaging and core parts
- Maven clean up
- Allow to tune more Jetty settings (for example threadpool)
- More security measures (e.g. protect against clickjacking, CSRF, etc.)
- Update Oozie Documentation


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/util/Instrumentation.java 
fa1e92a0f1fadfb66c5e66fea5f26f57579080e7 
  core/src/main/resources/oozie-default.xml 
e71ebe3b7a85e6b23176ef30713af63847144498 
  distro/pom.xml c50572c57a376b28963d4e7da8ac7df777fe0480 
  distro/src/main/bin/oozie-jetty-server.sh PRE-CREATION 
  distro/src/main/bin/oozie-setup.sh 79b049bccceb2690f8a673a885a615c8d4d9578c 
  distro/src/main/bin/oozied.sh a869c3da177c863a068f2af45c7bca9d5cb771ac 
  pom.xml 704a29f4e4805e3e08c2a547b2a375f6b1b2 
  server/pom.xml PRE-CREATION 
  server/src/main/assemblies/empty.xml PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java 
PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/HttpConfigurationWrapper.java 
PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/JspHandler.java PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/SSLServerConnectorFactory.java 
PRE-CREATION 
  server/src/main/resources/checkstyle-header.txt PRE-CREATION 
  server/src/main/resources/checkstyle.xml PRE-CREATION 
  server/src/test/java/org/apache/oozie/server/TestEmbeddedOozieServer.java 
PRE-CREATION 
  
server/src/test/java/org/apache/oozie/server/TestSSLServerConnectorFactory.java 
PRE-CREATION 
  src/main/assemblies/distro.xml 1ffbfd6d2ba33b390999e9094cbb336fbce45c21 

Diff: https://reviews.apache.org/r/52399/diff/


Testing
---

- Tested basic functionality by executing a workflow that uses the sample 
JavaAction
- without SSL - on a 2.4.0 pseudo Hadoop cluster
- SSL with Kerberos is using a test CDH cluster 
- Added new unit tests that check
- If oozie.ssl.enabled is not specified, server starts without SSL settings 
- If oozie.ssl.enabled is specified, server starts with SSL settings
- SSL protocols and ciphers can be configured via system properties and 
environment variables 
- Ran subset of tests using Hadoop-2 profile
- mvn clean package assembly:single   -DjavaVersion=1.8 -DtargetVersion=1.7 
 -Dtest=TestJavaActionExecutor  -Phadoop-2 -Dhadoop.version=2.4.0


Thanks,

Attila Sasvari

Re: Review Request 52737: OOZIE-2698 OYA: Refactor LauncherAM to make it more testable

[Attila Sasvari]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52737/#review152545
---




sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java 
(line 94)


The second parameter of checkNotNull is incorrect, it is a String
that is used in the detail message of the exception:

```
public static  T checkNotNull(T o, String name) throws 
NullPointerException {
if(o == null) {
throw new NullPointerException(name + " should not be null");
} else {
return o;
}
}
```


- Attila Sasvari


On Oct. 11, 2016, 1:42 p.m., Peter Bacsko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52737/
> ---
> 
> (Updated Oct. 11, 2016, 1:42 p.m.)
> 
> 
> Review request for oozie, András Piros, Attila Sasvari, Peter Cseh, and 
> Robert Kanter.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> This review contains a suggested approach for refactoring LauncherAM on the 
> OYA branch.
> 
> The main idea is to inject all dependencies of the class via the constructor. 
> In some cases, this requires factory objects. Note that this is necessary 
> because in unit tests, because we don't want to create actual resources such 
> as UGI or async ResourceManager callback.
> 
> In the tests, we can instantiate this class with mocks. The testability is 
> greatly enhanced because various checks and verifications can be performed on 
> the mocks. Tests are currently not in this review (see 
> https://issues.apache.org/jira/browse/OOZIE-2685).
> 
> 
> Diffs
> -
> 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java 
> ed29299 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java
>  PRE-CREATION 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java
>  PRE-CREATION 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java 
> PRE-CREATION 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsAndLocalFSOperations.java
>  PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java 
> 85d78c6 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java
>  23648b8 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifierFactory.java
>  PRE-CREATION 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java
>  4a51d48 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java
>  PRE-CREATION 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/SequenceFileWriterFactory.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/52737/diff/
> 
> 
> Testing
> ---
> 
> Unit tests are being written right now.
> 
> 
> Thanks,
> 
> Peter Bacsko
> 
>

[jira] [Updated] (OOZIE-2666) Support embedding Jetty into Oozie

[Attila Sasvari (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Sasvari updated OOZIE-2666:
--
Attachment: OOZIE-2666.PATCH-10.patch

rebase

> Support embedding Jetty into Oozie
> --
>
> Key: OOZIE-2666
> URL: https://issues.apache.org/jira/browse/OOZIE-2666
> Project: Oozie
>  Issue Type: Sub-task
>  Components: core
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2666.PATCH-01.patch, OOZIE-2666.PATCH-02.patch, 
> OOZIE-2666.PATCH-03.patch, OOZIE-2666.PATCH-05.patch, 
> OOZIE-2666.PATCH-06.patch, OOZIE-2666.PATCH-07.patch, 
> OOZIE-2666.PATCH-09.patch, OOZIE-2666.PATCH-10.patch
>
>
> The purpose of this task is to allow Oozie to run as a standalone application 
> rather than being deployed to a container inside a WAR file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: oozie-trunk-find-patches-available #136164

[Apache Jenkins Server]

See 

--
Started by timer
Started by timer
Started by timer
Started by timer
Started by timer
Started by timer
Started by timer
Started by timer
Started by timer
Started by timer
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on H1 (yahoo-not-h2 Hadoop Mapreduce Hdfs Pig) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/oozie.git # timeout=10
Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/oozie.git
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/oozie.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision bd54452808335cbadc48d4aacd0bd23374bed928 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f bd54452808335cbadc48d4aacd0bd23374bed928
 > git rev-list bd54452808335cbadc48d4aacd0bd23374bed928 # timeout=10
[oozie-trunk-find-patches-available] $ /bin/bash 
/tmp/hudson3237218287423097830.sh
mkdir: cannot create directory 
':
 File exists
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
curl: (22) The requested URL returned error: 400 Bad Request
Could not retrieve available patches from JIRA
Build step 'Execute shell' marked build as failure

[jira] [Commented] (OOZIE-2644) Skip queuing Notification Commands when there's nothing to notify

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15572660#comment-15572660
 ] 

Hadoop QA commented on OOZIE-2644:
--

Testing JIRA OOZIE-2644

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 2 line(s) longer than 132 
characters
.{color:green}+1{color} the patch does adds/modifies 5 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1823
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/3379/

> Skip queuing Notification Commands when there's nothing to notify
> -
>
> Key: OOZIE-2644
> URL: https://issues.apache.org/jira/browse/OOZIE-2644
> Project: Oozie
>  Issue Type: Improvement
>Affects Versions: 3.1.3
>Reporter: Robert Kanter
>Assignee: Azrael Seoeun
> Fix For: 5.0.0
>
> Attachments: OOZIE-2644.1.patch, OOZIE-2644.2.patch
>
>
> When you use the 
> [Workflow|https://oozie.apache.org/docs/4.2.0/WorkflowFunctionalSpec.html#a5_Workflow_Notifications]
>  or 
> [Coordinator|https://oozie.apache.org/docs/4.2.0/CoordinatorFunctionalSpec.html#a15._Coordinator_Notifications]
>  Notification features, Oozie can end up queuing up a lot of 
> {{WorkflowNotificationXCommand}} and {{CoordActionNotificationXCommand}}.  
> This happens even if there's no notification configured on the job (which I 
> imagine is most of the time); in this case, the {{execute}} method simply 
> does nothing.  This is wasteful and clogs the queue up.  
> We should change the code so that it doesn't queue up one of these Commands 
> unless there's actually a URL to notify.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 52824: OOZIE-2644 Skip queuing Notification Commands when there's nothing to notify

[Azrael Seoeun Park]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52824/
---

(Updated Oct. 13, 2016, 3:56 p.m.)


Review request for oozie and Robert Kanter.


Changes
---

Fix test failures.


Bugs: OOZIE-2644
https://issues.apache.org/jira/browse/OOZIE-2644


Repository: oozie-git


Description
---

Skip queing Notification Command if the notification url is not configured.

And this patch fix the NullPointerException when CoordActionNotification 
executed from CoordActionKillXCommand. This is caused the runConf of actionBean 
is not set.


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/CoordinatorActionBean.java b1be7c9 
  core/src/main/java/org/apache/oozie/CoordinatorJobBean.java 3a3120d 
  core/src/main/java/org/apache/oozie/command/NotificationXCommand.java 68359aa 
  core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f6 
  
core/src/main/java/org/apache/oozie/command/coord/CoordActionSkipXCommand.java 
ef0fc2d 
  
core/src/main/java/org/apache/oozie/command/coord/CoordActionStartXCommand.java 
45abe5a 
  
core/src/main/java/org/apache/oozie/command/coord/CoordActionTimeOutXCommand.java
 958194d 
  
core/src/main/java/org/apache/oozie/command/coord/CoordActionsKillXCommand.java 
d4c98c8 
  
core/src/main/java/org/apache/oozie/command/coord/CoordMaterializeTransitionXCommand.java
 f6c1782 
  core/src/main/java/org/apache/oozie/command/coord/CoordRerunXCommand.java 
f1509e6 
  core/src/main/java/org/apache/oozie/command/wf/ActionCheckXCommand.java 
d0551ff 
  core/src/main/java/org/apache/oozie/command/wf/ActionEndXCommand.java 740b8d3 
  core/src/main/java/org/apache/oozie/command/wf/ActionKillXCommand.java 
ac096cc 
  core/src/main/java/org/apache/oozie/command/wf/ActionStartXCommand.java 
edfac48 
  core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 836e5d4 
  core/src/main/java/org/apache/oozie/command/wf/KillXCommand.java 40b93d3 
  core/src/main/java/org/apache/oozie/command/wf/ResumeXCommand.java edbd767 
  core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java e95a60a 
  core/src/main/java/org/apache/oozie/command/wf/SuspendXCommand.java ef97990 
  core/src/main/java/org/apache/oozie/command/wf/WorkflowXCommand.java bc80dfe 
  
core/src/main/java/org/apache/oozie/executor/jpa/CoordActionQueryExecutor.java 
c0e6c19 
  core/src/main/java/org/apache/oozie/executor/jpa/CoordJobQueryExecutor.java 
432f075 
  core/src/main/java/org/apache/oozie/local/LocalOozie.java bf1b0db 
  core/src/main/resources/oozie-default.xml e71ebe3 
  
core/src/test/java/org/apache/oozie/command/coord/TestCoordActionNotificationXCommand.java
 8ca404f 
  
core/src/test/java/org/apache/oozie/command/coord/TestCoordActionsKillXCommand.java
 c9b2f28 
  
core/src/test/java/org/apache/oozie/command/wf/TestWorkflowNotificationXCommand.java
 ef342a4 
  
core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobQueryExecutor.java 
a0012fb 
  core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd 

Diff: https://reviews.apache.org/r/52824/diff/


Testing
---

Unit test.


Thanks,

Azrael Seoeun Park

[jira] [Updated] (OOZIE-2644) Skip queuing Notification Commands when there's nothing to notify

[Azrael Seoeun (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Azrael Seoeun updated OOZIE-2644:
-
Attachment: OOZIE-2644.2.patch

Fix test failures.

> Skip queuing Notification Commands when there's nothing to notify
> -
>
> Key: OOZIE-2644
> URL: https://issues.apache.org/jira/browse/OOZIE-2644
> Project: Oozie
>  Issue Type: Improvement
>Affects Versions: 3.1.3
>Reporter: Robert Kanter
>Assignee: Azrael Seoeun
> Fix For: 5.0.0
>
> Attachments: OOZIE-2644.1.patch, OOZIE-2644.2.patch
>
>
> When you use the 
> [Workflow|https://oozie.apache.org/docs/4.2.0/WorkflowFunctionalSpec.html#a5_Workflow_Notifications]
>  or 
> [Coordinator|https://oozie.apache.org/docs/4.2.0/CoordinatorFunctionalSpec.html#a15._Coordinator_Notifications]
>  Notification features, Oozie can end up queuing up a lot of 
> {{WorkflowNotificationXCommand}} and {{CoordActionNotificationXCommand}}.  
> This happens even if there's no notification configured on the job (which I 
> imagine is most of the time); in this case, the {{execute}} method simply 
> does nothing.  This is wasteful and clogs the queue up.  
> We should change the code so that it doesn't queue up one of these Commands 
> unless there's actually a URL to notify.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2666) Support embedding Jetty into Oozie

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15572269#comment-15572269
 ] 

Hadoop QA commented on OOZIE-2666:
--

Testing JIRA OOZIE-2666

Cleaning local git workspace



{color:red}-1{color} Patch failed to apply to head of branch



> Support embedding Jetty into Oozie
> --
>
> Key: OOZIE-2666
> URL: https://issues.apache.org/jira/browse/OOZIE-2666
> Project: Oozie
>  Issue Type: Sub-task
>  Components: core
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2666.PATCH-01.patch, OOZIE-2666.PATCH-02.patch, 
> OOZIE-2666.PATCH-03.patch, OOZIE-2666.PATCH-05.patch, 
> OOZIE-2666.PATCH-06.patch, OOZIE-2666.PATCH-07.patch, 
> OOZIE-2666.PATCH-09.patch
>
>
> The purpose of this task is to allow Oozie to run as a standalone application 
> rather than being deployed to a container inside a WAR file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: oozie-trunk-precommit-build #3378

[Apache Jenkins Server]

See 

--
[...truncated 1332 lines...]
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-docs 
---
[INFO] Compiling 1 source file to 

[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-docs ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-docs ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-docs ---
[INFO] Tests are skipped.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie WebApp 4.4.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ oozie-webapp ---
[INFO] Deleting 

[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-webapp 
---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-webapp ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-webapp 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-webapp ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-webapp ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-webapp ---
[INFO] Tests are skipped.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie Tools 4.4.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ oozie-tools ---
[INFO] Deleting 

[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-tools ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-tools ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-tools 
---
[INFO] Compiling 4 source files to 

[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-tools ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 11 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-tools ---
[INFO] Compiling 7 source files to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-tools ---
[INFO] Tests are skipped.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie MiniOozie 4.4.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ oozie-mini ---
[INFO] Deleting 

[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-mini ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[jira] [Updated] (OOZIE-2666) Support embedding Jetty into Oozie

[Attila Sasvari (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Sasvari updated OOZIE-2666:
--
Attachment: OOZIE-2666.PATCH-09.patch

> Support embedding Jetty into Oozie
> --
>
> Key: OOZIE-2666
> URL: https://issues.apache.org/jira/browse/OOZIE-2666
> Project: Oozie
>  Issue Type: Sub-task
>  Components: core
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2666.PATCH-01.patch, OOZIE-2666.PATCH-02.patch, 
> OOZIE-2666.PATCH-03.patch, OOZIE-2666.PATCH-05.patch, 
> OOZIE-2666.PATCH-06.patch, OOZIE-2666.PATCH-07.patch, 
> OOZIE-2666.PATCH-09.patch
>
>
> The purpose of this task is to allow Oozie to run as a standalone application 
> rather than being deployed to a container inside a WAR file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 52399: OOZIE-2666 Support embedding Jetty into Oozie

[Attila Sasvari]

> On Oct. 3, 2016, 4:27 p.m., Peter Bacsko wrote:
> > distro/src/main/bin/oozied.sh, line 115
> > 
> >
> > Is this TODO still relevant?

It is.


- Attila


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52399/#review151176
---


On Oct. 13, 2016, 2:17 p.m., Attila Sasvari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52399/
> ---
> 
> (Updated Oct. 13, 2016, 2:17 p.m.)
> 
> 
> Review request for oozie, András Piros, Peter Cseh, Peter Bacsko, and Robert 
> Kanter.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Embedding jetty into Oozie so that it can run as a standalone application. 
> The changes also try to address OOZIE-2317 (i.e. Tomcat 6 is EOL).
> 
> New functionality
> - New module (server) is added that sets up an embedded Jetty server and 
> start Oozie services. Servlet mapping is done by reading web.xml of webapp at 
> runtime. JSP is handled with custom code. Server version is not revealed in 
> server repsonses.
> - SSL protocols and ciphers can be configured via system properties and 
> environment variables. Precedence: system properties, environment variables, 
> default values
>
> Changes in existing code
> - Excluded jetty 6 dependencies from core and updated tests accordingly  
> 
> Packaging
> - oozie.sh is modified so that it starts Oozie with embedded jetty by 
> default. If someone would like to use tomcat for any reason, they can set an 
> environment variable (e.g. OOZIE_USE_TOMCAT=1).
> 
> TODO:
> - Add more tests
> - Add more documentation
> - Code cleanup + refactoring in packaging and core parts
> - Maven clean up
> - Allow to tune more Jetty settings (for example threadpool)
> - More security measures (e.g. protect against clickjacking, CSRF, etc.)
> - Update Oozie Documentation
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 
> fa1e92a0f1fadfb66c5e66fea5f26f57579080e7 
>   core/src/main/resources/oozie-default.xml 
> e71ebe3b7a85e6b23176ef30713af63847144498 
>   distro/pom.xml c50572c57a376b28963d4e7da8ac7df777fe0480 
>   distro/src/main/bin/oozie-jetty-server.sh PRE-CREATION 
>   distro/src/main/bin/oozie-setup.sh 79b049bccceb2690f8a673a885a615c8d4d9578c 
>   distro/src/main/bin/oozied.sh a869c3da177c863a068f2af45c7bca9d5cb771ac 
>   pom.xml 704a29f4e4805e3e08c2a547b2a375f6b1b2 
>   server/pom.xml PRE-CREATION 
>   server/src/main/assemblies/empty.xml PRE-CREATION 
>   server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java 
> PRE-CREATION 
>   server/src/main/java/org/apache/oozie/server/HttpConfigurationWrapper.java 
> PRE-CREATION 
>   server/src/main/java/org/apache/oozie/server/JspHandler.java PRE-CREATION 
>   server/src/main/java/org/apache/oozie/server/SSLServerConnectorFactory.java 
> PRE-CREATION 
>   server/src/main/resources/checkstyle-header.txt PRE-CREATION 
>   server/src/main/resources/checkstyle.xml PRE-CREATION 
>   server/src/test/java/org/apache/oozie/server/TestEmbeddedOozieServer.java 
> PRE-CREATION 
>   
> server/src/test/java/org/apache/oozie/server/TestSSLServerConnectorFactory.java
>  PRE-CREATION 
>   src/main/assemblies/distro.xml 1ffbfd6d2ba33b390999e9094cbb336fbce45c21 
> 
> Diff: https://reviews.apache.org/r/52399/diff/
> 
> 
> Testing
> ---
> 
> - Tested basic functionality by executing a workflow that uses the sample 
> JavaAction
> - without SSL - on a 2.4.0 pseudo Hadoop cluster
> - SSL with Kerberos is using a test CDH cluster 
> - Added new unit tests that check
> - If oozie.ssl.enabled is not specified, server starts without SSL 
> settings 
> - If oozie.ssl.enabled is specified, server starts with SSL settings
> - SSL protocols and ciphers can be configured via system properties and 
> environment variables 
> - Ran subset of tests using Hadoop-2 profile
> - mvn clean package assembly:single   -DjavaVersion=1.8 
> -DtargetVersion=1.7  -Dtest=TestJavaActionExecutor  -Phadoop-2 
> -Dhadoop.version=2.4.0
> 
> 
> Thanks,
> 
> Attila Sasvari
> 
>

Re: Review Request 52399: OOZIE-2666 Support embedding Jetty into Oozie

[Attila Sasvari]

> On Oct. 6, 2016, 11:04 p.m., Robert Kanter wrote:
> > distro/src/main/bin/oozie-jetty-server.sh, lines 40-61
> > 
> >
> > IMO, we should move as many of these as possible to oozie-default/site. 
> >  We had a bunch of these as JVM properties before because of Tomcat and 
> > other reasons, but it would be good to have them all in the same place as 
> > other config properties.  At the very least, the https/ssl ones (the other 
> > ones are probably out of scope for this JIRA).

Because the way Services and ConfigurationService classes are constructed we 
still need to pass some properties as system properties:

  jetty_opts="-Doozie.home.dir=${OOZIE_HOME}";
  jetty_opts="${jetty_opts} -Doozie.config.dir=${OOZIE_CONFIG}";
  jetty_opts="${jetty_opts} -Doozie.log.dir=${OOZIE_LOG}";
  jetty_opts="${jetty_opts} -Doozie.data.dir=${OOZIE_DATA}";
  jetty_opts="${jetty_opts} -Doozie.config.file=${OOZIE_CONFIG_FILE}";
  jetty_opts="${jetty_opts} -Doozie.log4j.file=${OOZIE_LOG4J_FILE}";
  jetty_opts="${jetty_opts} -Doozie.log4j.reload=${OOZIE_LOG4J_RELOAD}";

ConfigurationService is initialized by passing in a Services object (that is 
initialized)
However, oozie home needs to be known in Services init, so we cannot get it 
from ConfigurationService (that depends on Services).
Similarly Log4j properties are used in Services' init(). 

In my opinion, Services and ConfigurationService would benefit a lot from code 
clean up / refactoring. It is out of scope of this task I believe.


- Attila


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52399/#review151728
---


On Oct. 13, 2016, 2:17 p.m., Attila Sasvari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52399/
> ---
> 
> (Updated Oct. 13, 2016, 2:17 p.m.)
> 
> 
> Review request for oozie, András Piros, Peter Cseh, Peter Bacsko, and Robert 
> Kanter.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Embedding jetty into Oozie so that it can run as a standalone application. 
> The changes also try to address OOZIE-2317 (i.e. Tomcat 6 is EOL).
> 
> New functionality
> - New module (server) is added that sets up an embedded Jetty server and 
> start Oozie services. Servlet mapping is done by reading web.xml of webapp at 
> runtime. JSP is handled with custom code. Server version is not revealed in 
> server repsonses.
> - SSL protocols and ciphers can be configured via system properties and 
> environment variables. Precedence: system properties, environment variables, 
> default values
>
> Changes in existing code
> - Excluded jetty 6 dependencies from core and updated tests accordingly  
> 
> Packaging
> - oozie.sh is modified so that it starts Oozie with embedded jetty by 
> default. If someone would like to use tomcat for any reason, they can set an 
> environment variable (e.g. OOZIE_USE_TOMCAT=1).
> 
> TODO:
> - Add more tests
> - Add more documentation
> - Code cleanup + refactoring in packaging and core parts
> - Maven clean up
> - Allow to tune more Jetty settings (for example threadpool)
> - More security measures (e.g. protect against clickjacking, CSRF, etc.)
> - Update Oozie Documentation
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 
> fa1e92a0f1fadfb66c5e66fea5f26f57579080e7 
>   core/src/main/resources/oozie-default.xml 
> e71ebe3b7a85e6b23176ef30713af63847144498 
>   distro/pom.xml c50572c57a376b28963d4e7da8ac7df777fe0480 
>   distro/src/main/bin/oozie-jetty-server.sh PRE-CREATION 
>   distro/src/main/bin/oozie-setup.sh 79b049bccceb2690f8a673a885a615c8d4d9578c 
>   distro/src/main/bin/oozied.sh a869c3da177c863a068f2af45c7bca9d5cb771ac 
>   pom.xml 704a29f4e4805e3e08c2a547b2a375f6b1b2 
>   server/pom.xml PRE-CREATION 
>   server/src/main/assemblies/empty.xml PRE-CREATION 
>   server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java 
> PRE-CREATION 
>   server/src/main/java/org/apache/oozie/server/HttpConfigurationWrapper.java 
> PRE-CREATION 
>   server/src/main/java/org/apache/oozie/server/JspHandler.java PRE-CREATION 
>   server/src/main/java/org/apache/oozie/server/SSLServerConnectorFactory.java 
> PRE-CREATION 
>   server/src/main/resources/checkstyle-header.txt PRE-CREATION 
>   server/src/main/resources/checkstyle.xml PRE-CREATION 
>   server/src/test/java/org/apache/oozie/server/TestEmbeddedOozieServer.java 
> PRE-CREATION 
>   
> server/src/test/java/org/apache/oozie/server/TestSSLServerConnectorFactory.java
>  PRE-CREATION 
>   src/main/assemblies/distro.xml 1ffbfd6d2ba33b390999e9094cbb336fbce45c21 
> 
> Diff: https://reviews.apache.org/r/52399/diff/
> 
> 
> 

Re: Review Request 52399: OOZIE-2666 Support embedding Jetty into Oozie

[Attila Sasvari]

> On Oct. 6, 2016, 11:04 p.m., Robert Kanter wrote:
> > distro/src/main/bin/oozie-jetty-server.sh, line 1
> > 
> >
> > Please run all new/updated shell scripts through shellcheck 
> > (https://www.shellcheck.net/).  It finds various problems and best practice 
> > violations (kind of like findbugs).  You can even install it on your Mac 
> > with Homebrew or MacPorts.

done, 3 false positive warnings remained
  ${JAVA_BIN} ${JETTY_OPTS} org.apache.oozie.server.EmbeddedOozieServer >> 
"${JETTY_OUT}" 2>&1 &
  ^-- SC2086: Double quote to prevent globbing and word splitting.


In distro/src/main/bin/oozie-jetty-server.sh line 111:
  ${JAVA_BIN} ${JETTY_OPTS} org.apache.oozie.server.EmbeddedOozieServer
  ^-- SC2086: Double quote to prevent globbing and word splitting.


In distro/src/main/bin/oozie-jetty-server.sh line 140:
kill -0 "$(cat ${JETTY_PID_FILE})" >/dev/null 2>&1
   ^-- SC2086: Double quote to prevent globbing and word 
splitting.

if I double quote, I cannot start the server because system properties and 
others are not passed in an expected way to java.
It is also true for the existing oozie.d.sh script


- Attila


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52399/#review151728
---


On Oct. 13, 2016, 2:17 p.m., Attila Sasvari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52399/
> ---
> 
> (Updated Oct. 13, 2016, 2:17 p.m.)
> 
> 
> Review request for oozie, András Piros, Peter Cseh, Peter Bacsko, and Robert 
> Kanter.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Embedding jetty into Oozie so that it can run as a standalone application. 
> The changes also try to address OOZIE-2317 (i.e. Tomcat 6 is EOL).
> 
> New functionality
> - New module (server) is added that sets up an embedded Jetty server and 
> start Oozie services. Servlet mapping is done by reading web.xml of webapp at 
> runtime. JSP is handled with custom code. Server version is not revealed in 
> server repsonses.
> - SSL protocols and ciphers can be configured via system properties and 
> environment variables. Precedence: system properties, environment variables, 
> default values
>
> Changes in existing code
> - Excluded jetty 6 dependencies from core and updated tests accordingly  
> 
> Packaging
> - oozie.sh is modified so that it starts Oozie with embedded jetty by 
> default. If someone would like to use tomcat for any reason, they can set an 
> environment variable (e.g. OOZIE_USE_TOMCAT=1).
> 
> TODO:
> - Add more tests
> - Add more documentation
> - Code cleanup + refactoring in packaging and core parts
> - Maven clean up
> - Allow to tune more Jetty settings (for example threadpool)
> - More security measures (e.g. protect against clickjacking, CSRF, etc.)
> - Update Oozie Documentation
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 
> fa1e92a0f1fadfb66c5e66fea5f26f57579080e7 
>   core/src/main/resources/oozie-default.xml 
> e71ebe3b7a85e6b23176ef30713af63847144498 
>   distro/pom.xml c50572c57a376b28963d4e7da8ac7df777fe0480 
>   distro/src/main/bin/oozie-jetty-server.sh PRE-CREATION 
>   distro/src/main/bin/oozie-setup.sh 79b049bccceb2690f8a673a885a615c8d4d9578c 
>   distro/src/main/bin/oozied.sh a869c3da177c863a068f2af45c7bca9d5cb771ac 
>   pom.xml 704a29f4e4805e3e08c2a547b2a375f6b1b2 
>   server/pom.xml PRE-CREATION 
>   server/src/main/assemblies/empty.xml PRE-CREATION 
>   server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java 
> PRE-CREATION 
>   server/src/main/java/org/apache/oozie/server/HttpConfigurationWrapper.java 
> PRE-CREATION 
>   server/src/main/java/org/apache/oozie/server/JspHandler.java PRE-CREATION 
>   server/src/main/java/org/apache/oozie/server/SSLServerConnectorFactory.java 
> PRE-CREATION 
>   server/src/main/resources/checkstyle-header.txt PRE-CREATION 
>   server/src/main/resources/checkstyle.xml PRE-CREATION 
>   server/src/test/java/org/apache/oozie/server/TestEmbeddedOozieServer.java 
> PRE-CREATION 
>   
> server/src/test/java/org/apache/oozie/server/TestSSLServerConnectorFactory.java
>  PRE-CREATION 
>   src/main/assemblies/distro.xml 1ffbfd6d2ba33b390999e9094cbb336fbce45c21 
> 
> Diff: https://reviews.apache.org/r/52399/diff/
> 
> 
> Testing
> ---
> 
> - Tested basic functionality by executing a workflow that uses the sample 
> JavaAction
> - without SSL - on a 2.4.0 pseudo Hadoop cluster
> - SSL with Kerberos is using a test CDH cluster 
> - Added new unit tests that check
> - If oozie.ssl.enabled is not specified, server starts 

Re: Review Request 52399: OOZIE-2666 Support embedding Jetty into Oozie

[Attila Sasvari]

> On Oct. 3, 2016, 4:27 p.m., Peter Bacsko wrote:
> > server/pom.xml, line 26
> > 
> >
> > +1, I'd remove versions tags here

all other mvn properties have ".version" suffix


> On Oct. 3, 2016, 4:27 p.m., Peter Bacsko wrote:
> > server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java, 
> > lines 79-80
> > 
> >
> > Extract strings to constants like "oozie.http.port", "OOZIE_HTTP_PORT", 
> > etc

These strings are used only once. Therefore I think it is not necessary.


> On Oct. 3, 2016, 4:27 p.m., Peter Bacsko wrote:
> > server/src/main/java/org/apache/oozie/server/JspHandler.java, lines 51-60
> > 
> >
> > extract properties to constant

These strings are used only once. Therefore I think it is not necessary


> On Oct. 3, 2016, 4:27 p.m., Peter Bacsko wrote:
> > server/src/test/java/org/apache/oozie/server/TestEmbeddedOozieServer.java, 
> > line 54
> > 
> >
> > extract property

"oozie.ssl.enabled" is used only once. Therefore I think it is not necessary


- Attila


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52399/#review151176
---


On Oct. 13, 2016, 2:17 p.m., Attila Sasvari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52399/
> ---
> 
> (Updated Oct. 13, 2016, 2:17 p.m.)
> 
> 
> Review request for oozie, András Piros, Peter Cseh, Peter Bacsko, and Robert 
> Kanter.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Embedding jetty into Oozie so that it can run as a standalone application. 
> The changes also try to address OOZIE-2317 (i.e. Tomcat 6 is EOL).
> 
> New functionality
> - New module (server) is added that sets up an embedded Jetty server and 
> start Oozie services. Servlet mapping is done by reading web.xml of webapp at 
> runtime. JSP is handled with custom code. Server version is not revealed in 
> server repsonses.
> - SSL protocols and ciphers can be configured via system properties and 
> environment variables. Precedence: system properties, environment variables, 
> default values
>
> Changes in existing code
> - Excluded jetty 6 dependencies from core and updated tests accordingly  
> 
> Packaging
> - oozie.sh is modified so that it starts Oozie with embedded jetty by 
> default. If someone would like to use tomcat for any reason, they can set an 
> environment variable (e.g. OOZIE_USE_TOMCAT=1).
> 
> TODO:
> - Add more tests
> - Add more documentation
> - Code cleanup + refactoring in packaging and core parts
> - Maven clean up
> - Allow to tune more Jetty settings (for example threadpool)
> - More security measures (e.g. protect against clickjacking, CSRF, etc.)
> - Update Oozie Documentation
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 
> fa1e92a0f1fadfb66c5e66fea5f26f57579080e7 
>   core/src/main/resources/oozie-default.xml 
> e71ebe3b7a85e6b23176ef30713af63847144498 
>   distro/pom.xml c50572c57a376b28963d4e7da8ac7df777fe0480 
>   distro/src/main/bin/oozie-jetty-server.sh PRE-CREATION 
>   distro/src/main/bin/oozie-setup.sh 79b049bccceb2690f8a673a885a615c8d4d9578c 
>   distro/src/main/bin/oozied.sh a869c3da177c863a068f2af45c7bca9d5cb771ac 
>   pom.xml 704a29f4e4805e3e08c2a547b2a375f6b1b2 
>   server/pom.xml PRE-CREATION 
>   server/src/main/assemblies/empty.xml PRE-CREATION 
>   server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java 
> PRE-CREATION 
>   server/src/main/java/org/apache/oozie/server/HttpConfigurationWrapper.java 
> PRE-CREATION 
>   server/src/main/java/org/apache/oozie/server/JspHandler.java PRE-CREATION 
>   server/src/main/java/org/apache/oozie/server/SSLServerConnectorFactory.java 
> PRE-CREATION 
>   server/src/main/resources/checkstyle-header.txt PRE-CREATION 
>   server/src/main/resources/checkstyle.xml PRE-CREATION 
>   server/src/test/java/org/apache/oozie/server/TestEmbeddedOozieServer.java 
> PRE-CREATION 
>   
> server/src/test/java/org/apache/oozie/server/TestSSLServerConnectorFactory.java
>  PRE-CREATION 
>   src/main/assemblies/distro.xml 1ffbfd6d2ba33b390999e9094cbb336fbce45c21 
> 
> Diff: https://reviews.apache.org/r/52399/diff/
> 
> 
> Testing
> ---
> 
> - Tested basic functionality by executing a workflow that uses the sample 
> JavaAction
> - without SSL - on a 2.4.0 pseudo Hadoop cluster
> - SSL with Kerberos is using a test CDH cluster 
> - Added new unit tests that check
> - If 

Re: Review Request 52399: OOZIE-2666 Support embedding Jetty into Oozie

[Attila Sasvari]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52399/
---

(Updated Oct. 13, 2016, 2:17 p.m.)


Review request for oozie, András Piros, Peter Cseh, Peter Bacsko, and Robert 
Kanter.


Repository: oozie-git


Description
---

Embedding jetty into Oozie so that it can run as a standalone application. The 
changes also try to address OOZIE-2317 (i.e. Tomcat 6 is EOL).

New functionality
- New module (server) is added that sets up an embedded Jetty server and start 
Oozie services. Servlet mapping is done by reading web.xml of webapp at 
runtime. JSP is handled with custom code. Server version is not revealed in 
server repsonses.
- SSL protocols and ciphers can be configured via system properties and 
environment variables. Precedence: system properties, environment variables, 
default values
   
Changes in existing code
- Excluded jetty 6 dependencies from core and updated tests accordingly  

Packaging
- oozie.sh is modified so that it starts Oozie with embedded jetty by default. 
If someone would like to use tomcat for any reason, they can set an environment 
variable (e.g. OOZIE_USE_TOMCAT=1).

TODO:
- Add more tests
- Add more documentation
- Code cleanup + refactoring in packaging and core parts
- Maven clean up
- Allow to tune more Jetty settings (for example threadpool)
- More security measures (e.g. protect against clickjacking, CSRF, etc.)
- Update Oozie Documentation


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/util/Instrumentation.java 
fa1e92a0f1fadfb66c5e66fea5f26f57579080e7 
  core/src/main/resources/oozie-default.xml 
e71ebe3b7a85e6b23176ef30713af63847144498 
  distro/pom.xml c50572c57a376b28963d4e7da8ac7df777fe0480 
  distro/src/main/bin/oozie-jetty-server.sh PRE-CREATION 
  distro/src/main/bin/oozie-setup.sh 79b049bccceb2690f8a673a885a615c8d4d9578c 
  distro/src/main/bin/oozied.sh a869c3da177c863a068f2af45c7bca9d5cb771ac 
  pom.xml 704a29f4e4805e3e08c2a547b2a375f6b1b2 
  server/pom.xml PRE-CREATION 
  server/src/main/assemblies/empty.xml PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java 
PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/HttpConfigurationWrapper.java 
PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/JspHandler.java PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/SSLServerConnectorFactory.java 
PRE-CREATION 
  server/src/main/resources/checkstyle-header.txt PRE-CREATION 
  server/src/main/resources/checkstyle.xml PRE-CREATION 
  server/src/test/java/org/apache/oozie/server/TestEmbeddedOozieServer.java 
PRE-CREATION 
  
server/src/test/java/org/apache/oozie/server/TestSSLServerConnectorFactory.java 
PRE-CREATION 
  src/main/assemblies/distro.xml 1ffbfd6d2ba33b390999e9094cbb336fbce45c21 

Diff: https://reviews.apache.org/r/52399/diff/


Testing
---

- Tested basic functionality by executing a workflow that uses the sample 
JavaAction
- without SSL - on a 2.4.0 pseudo Hadoop cluster
- SSL with Kerberos is using a test CDH cluster 
- Added new unit tests that check
- If oozie.ssl.enabled is not specified, server starts without SSL settings 
- If oozie.ssl.enabled is specified, server starts with SSL settings
- SSL protocols and ciphers can be configured via system properties and 
environment variables 
- Ran subset of tests using Hadoop-2 profile
- mvn clean package assembly:single   -DjavaVersion=1.8 -DtargetVersion=1.7 
 -Dtest=TestJavaActionExecutor  -Phadoop-2 -Dhadoop.version=2.4.0


Thanks,

Attila Sasvari

[jira] [Commented] (OOZIE-2710) Oozie HCatalog example workflow fails

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15572036#comment-15572036
 ] 

Abhishek Bafna commented on OOZIE-2710:
---

Ping: [~rkanter], [~rohini].

> Oozie HCatalog example workflow fails
> -
>
> Key: OOZIE-2710
> URL: https://issues.apache.org/jira/browse/OOZIE-2710
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2710-00.patch
>
>
> Oozie HCatalog example workflow fails with the below exception:
> {noformat}
> 2016-10-12 10:33:37,260 [uber-SubtaskRunner] ERROR org.apache.pig.PigServer  
> - exception during parsing: Error during parsing. Could not resolve 
> org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., 
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
> Failed to parse: Pig script failed to parse: 
>  pig script failed to validate: 
> org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not 
> resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., 
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
>   at 
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:199)
>   at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1819)
>   at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1527)
>   at org.apache.pig.PigServer.parseAndBuild(PigServer.java:460)
>   at org.apache.pig.PigServer.executeBatch(PigServer.java:485)
>   at org.apache.pig.PigServer.executeBatch(PigServer.java:471)
>   at 
> org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:172)
>   at 
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:235)
>   at 
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:206)
>   at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
>   at org.apache.pig.Main.run(Main.java:501)
>   at org.apache.pig.PigRunner.run(PigRunner.java:49)
>   at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:290)
>   at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:233)
>   at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:60)
>   at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:73)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:497)
>   at 
> org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:232)
>   at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
>   at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
>   at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:380)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:301)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:187)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:230)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: 
>  pig script failed to validate: 
> org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not 
> resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., 
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
>   at 
> org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:1339)
>   at 
> org.apache.pig.parser.LogicalPlanBuilder.buildFuncSpec(LogicalPlanBuilder.java:1324)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.func_clause(LogicalPlanGenerator.java:5184)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3515)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1625)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
>   at 
> 

Re: Review Request 52399: OOZIE-2666 Support embedding Jetty into Oozie

[Attila Sasvari]

> On Oct. 6, 2016, 11:04 p.m., Robert Kanter wrote:
> > server/src/main/java/org/apache/oozie/server/JspHandler.java, line 51
> > 
> >
> > This typically points to /tmp right?  In a long running Oozie Server, 
> > might we run into problem where the OS cleans this up?

Yes, it is /tmp typically on a Linux system. On my Mac, however, it is 
/var/folders/9q/f8p_r6gj0wbck49_dc092q_mgp/T/embedded-jetty-jsp

Note: this is intended for temporary data. See 
https://github.com/jetty-project/embedded-jetty-jsp
"A sub directory in this temp directory will be automatically added by the JSP 
implementation for loading the compiled JSP classes."

So I do not think it will cause problems, it is only for compiling JSP classes. 
By default, /tmp is cleared upon every OS boot (assuming TMPTIME is set 0).

I tested what happens on my Mac, if I rm -rf 
/var/folders/9q/f8p_r6gj0wbck49_dc092q_mgp/T/embedded-jetty-jsp => it 
recreates the embedded-jetty-jsp directory. 


By the way, I followed jetty's reference example here: 
https://github.com/jetty-project/embedded-jetty-jsp/blob/master/src/main/java/org/eclipse/jetty/demo/Main.java#L135


- Attila


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52399/#review151728
---


On Oct. 13, 2016, 1:37 p.m., Attila Sasvari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52399/
> ---
> 
> (Updated Oct. 13, 2016, 1:37 p.m.)
> 
> 
> Review request for oozie, András Piros, Peter Cseh, Peter Bacsko, and Robert 
> Kanter.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Embedding jetty into Oozie so that it can run as a standalone application. 
> The changes also try to address OOZIE-2317 (i.e. Tomcat 6 is EOL).
> 
> New functionality
> - New module (server) is added that sets up an embedded Jetty server and 
> start Oozie services. Servlet mapping is done by reading web.xml of webapp at 
> runtime. JSP is handled with custom code. Server version is not revealed in 
> server repsonses.
> - SSL protocols and ciphers can be configured via system properties and 
> environment variables. Precedence: system properties, environment variables, 
> default values
>
> Changes in existing code
> - Excluded jetty 6 dependencies from core and updated tests accordingly  
> 
> Packaging
> - oozie.sh is modified so that it starts Oozie with embedded jetty by 
> default. If someone would like to use tomcat for any reason, they can set an 
> environment variable (e.g. OOZIE_USE_TOMCAT=1).
> 
> TODO:
> - Add more tests
> - Add more documentation
> - Code cleanup + refactoring in packaging and core parts
> - Maven clean up
> - Allow to tune more Jetty settings (for example threadpool)
> - More security measures (e.g. protect against clickjacking, CSRF, etc.)
> - Update Oozie Documentation
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 
> fa1e92a0f1fadfb66c5e66fea5f26f57579080e7 
>   core/src/main/resources/oozie-default.xml 
> e71ebe3b7a85e6b23176ef30713af63847144498 
>   distro/pom.xml c50572c57a376b28963d4e7da8ac7df777fe0480 
>   distro/src/main/bin/oozie-jetty-server.sh PRE-CREATION 
>   distro/src/main/bin/oozie-setup.sh 79b049bccceb2690f8a673a885a615c8d4d9578c 
>   distro/src/main/bin/oozied.sh a869c3da177c863a068f2af45c7bca9d5cb771ac 
>   pom.xml 704a29f4e4805e3e08c2a547b2a375f6b1b2 
>   server/pom.xml PRE-CREATION 
>   server/src/main/assemblies/empty.xml PRE-CREATION 
>   server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java 
> PRE-CREATION 
>   server/src/main/java/org/apache/oozie/server/HttpConfigurationWrapper.java 
> PRE-CREATION 
>   server/src/main/java/org/apache/oozie/server/JspHandler.java PRE-CREATION 
>   server/src/main/java/org/apache/oozie/server/SSLServerConnectorFactory.java 
> PRE-CREATION 
>   server/src/main/resources/checkstyle-header.txt PRE-CREATION 
>   server/src/main/resources/checkstyle.xml PRE-CREATION 
>   server/src/test/java/org/apache/oozie/server/TestEmbeddedOozieServer.java 
> PRE-CREATION 
>   
> server/src/test/java/org/apache/oozie/server/TestSSLServerConnectorFactory.java
>  PRE-CREATION 
>   src/main/assemblies/distro.xml 1ffbfd6d2ba33b390999e9094cbb336fbce45c21 
> 
> Diff: https://reviews.apache.org/r/52399/diff/
> 
> 
> Testing
> ---
> 
> - Tested basic functionality by executing a workflow that uses the sample 
> JavaAction
> - without SSL - on a 2.4.0 pseudo Hadoop cluster
> - SSL with Kerberos is using a test CDH cluster 
> - Added new unit tests that check
> - If oozie.ssl.enabled is not specified, server starts without SSL 
> settings 
> - 

Re: Review Request 52399: OOZIE-2666 Support embedding Jetty into Oozie

[Attila Sasvari]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52399/
---

(Updated Oct. 13, 2016, 1:37 p.m.)


Review request for oozie, András Piros, Peter Cseh, Peter Bacsko, and Robert 
Kanter.


Changes
---

- Remove admin port from jetty script
- Addressing review comments in tests


Repository: oozie-git


Description
---

Embedding jetty into Oozie so that it can run as a standalone application. The 
changes also try to address OOZIE-2317 (i.e. Tomcat 6 is EOL).

New functionality
- New module (server) is added that sets up an embedded Jetty server and start 
Oozie services. Servlet mapping is done by reading web.xml of webapp at 
runtime. JSP is handled with custom code. Server version is not revealed in 
server repsonses.
- SSL protocols and ciphers can be configured via system properties and 
environment variables. Precedence: system properties, environment variables, 
default values
   
Changes in existing code
- Excluded jetty 6 dependencies from core and updated tests accordingly  

Packaging
- oozie.sh is modified so that it starts Oozie with embedded jetty by default. 
If someone would like to use tomcat for any reason, they can set an environment 
variable (e.g. OOZIE_USE_TOMCAT=1).

TODO:
- Add more tests
- Add more documentation
- Code cleanup + refactoring in packaging and core parts
- Maven clean up
- Allow to tune more Jetty settings (for example threadpool)
- More security measures (e.g. protect against clickjacking, CSRF, etc.)
- Update Oozie Documentation


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/util/Instrumentation.java 
fa1e92a0f1fadfb66c5e66fea5f26f57579080e7 
  core/src/main/resources/oozie-default.xml 
e71ebe3b7a85e6b23176ef30713af63847144498 
  distro/pom.xml c50572c57a376b28963d4e7da8ac7df777fe0480 
  distro/src/main/bin/oozie-jetty-server.sh PRE-CREATION 
  distro/src/main/bin/oozie-setup.sh 79b049bccceb2690f8a673a885a615c8d4d9578c 
  distro/src/main/bin/oozied.sh a869c3da177c863a068f2af45c7bca9d5cb771ac 
  pom.xml 704a29f4e4805e3e08c2a547b2a375f6b1b2 
  server/pom.xml PRE-CREATION 
  server/src/main/assemblies/empty.xml PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/EmbeddedOozieServer.java 
PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/HttpConfigurationWrapper.java 
PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/JspHandler.java PRE-CREATION 
  server/src/main/java/org/apache/oozie/server/SSLServerConnectorFactory.java 
PRE-CREATION 
  server/src/main/resources/checkstyle-header.txt PRE-CREATION 
  server/src/main/resources/checkstyle.xml PRE-CREATION 
  server/src/test/java/org/apache/oozie/server/TestEmbeddedOozieServer.java 
PRE-CREATION 
  
server/src/test/java/org/apache/oozie/server/TestSSLServerConnectorFactory.java 
PRE-CREATION 
  src/main/assemblies/distro.xml 1ffbfd6d2ba33b390999e9094cbb336fbce45c21 

Diff: https://reviews.apache.org/r/52399/diff/


Testing
---

- Tested basic functionality by executing a workflow that uses the sample 
JavaAction
- without SSL - on a 2.4.0 pseudo Hadoop cluster
- SSL with Kerberos is using a test CDH cluster 
- Added new unit tests that check
- If oozie.ssl.enabled is not specified, server starts without SSL settings 
- If oozie.ssl.enabled is specified, server starts with SSL settings
- SSL protocols and ciphers can be configured via system properties and 
environment variables 
- Ran subset of tests using Hadoop-2 profile
- mvn clean package assembly:single   -DjavaVersion=1.8 -DtargetVersion=1.7 
 -Dtest=TestJavaActionExecutor  -Phadoop-2 -Dhadoop.version=2.4.0


Thanks,

Attila Sasvari

Jenkins build is back to normal : oozie-trunk-find-patches-available #136158

[Apache Jenkins Server]

See 

Build failed in Jenkins: oozie-trunk-find-patches-available #136157

[Apache Jenkins Server]

See 

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on H1 (yahoo-not-h2 Hadoop Mapreduce Hdfs Pig) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/oozie.git # timeout=10
Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/oozie.git
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/oozie.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision bd54452808335cbadc48d4aacd0bd23374bed928 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f bd54452808335cbadc48d4aacd0bd23374bed928
 > git rev-list bd54452808335cbadc48d4aacd0bd23374bed928 # timeout=10
[oozie-trunk-find-patches-available] $ /bin/bash 
/tmp/hudson324264346179145060.sh
mkdir: cannot create directory 
':
 File exists
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
100  148k0  148k0 0   172k  0 --:--:-- --:--:-- --:--:--  172k
100  440k0  440k0 0   240k  0 --:--:--  0:00:01 --:--:--  240k
100  693k0  693k0 0   247k  0 --:--:--  0:00:02 --:--:--  247k
100 1026k0 1026k0 0   270k  0 --:--:--  0:00:03 --:--:--  270k
100 1184k0 1184k0 0   237k  0 --:--:--  0:00:04 --:--:--  237k
100 1492k0 1492k0 0   256k  0 --:--:--  0:00:05 --:--:--  271k
100 1606k0 1606k0 0   260k  0 --:--:--  0:00:06 --:--:--  268k
curl: (18) transfer closed with outstanding read data remaining
Could not retrieve available patches from JIRA
Build step 'Execute shell' marked build as failure

[jira] [Commented] (OOZIE-1986) Add FindBugs report to pre-commit build

[Andras Piros (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15571743#comment-15571743
 ] 

Andras Piros commented on OOZIE-1986:
-

Got 2x Ship It! on [*ReviewBoard*|https://reviews.apache.org/r/52429/], should 
be fine.

> Add FindBugs report to pre-commit build
> ---
>
> Key: OOZIE-1986
> URL: https://issues.apache.org/jira/browse/OOZIE-1986
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
>Reporter: Robert Kanter
>Assignee: Andras Piros
> Fix For: 5.0.0
>
> Attachments: OOZIE-1986.001.patch, OOZIE-1986.002.patch, 
> OOZIE-1986.003.patch, OOZIE-1986.004.patch, OOZIE-1986.005.patch, 
> OOZIE-1986.006.patch, OOZIE-1986.007.patch, OOZIE-1986.008.patch, 
> OOZIE-1986.009.patch, summary.txt
>
>
> Now that we have OOZIE-1793, it will be really useful to have the pre-commit 
> build generate a findbugs report.  It should probably before and after the 
> patch and only complain it there are more after.  And it should link to the 
> report.  Ideally, this would do some kind of diff (like Hadoop does) so it's 
> easy to identify the new findbugs items.
> You can generate the findbugs reports by running:
> {noformat}
> mvn verify -DskipTests
> {noformat}
> This also runs checkstyle



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2644) Skip queuing Notification Commands when there's nothing to notify

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15571719#comment-15571719
 ] 

Hadoop QA commented on OOZIE-2644:
--

Testing JIRA OOZIE-2644

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 2 line(s) longer than 132 
characters
.{color:green}+1{color} the patch does adds/modifies 5 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1823
.Tests failed: 2
.Tests errors: 0

.The patch failed the following testcases:

.  
testSkipNotificationCommand(org.apache.oozie.command.coord.TestCoordActionNotificationXCommand)
.  
testQueueNotificationCommand(org.apache.oozie.command.coord.TestCoordActionNotificationXCommand)

{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/3377/

> Skip queuing Notification Commands when there's nothing to notify
> -
>
> Key: OOZIE-2644
> URL: https://issues.apache.org/jira/browse/OOZIE-2644
> Project: Oozie
>  Issue Type: Improvement
>Affects Versions: 3.1.3
>Reporter: Robert Kanter
>Assignee: Azrael Seoeun
> Fix For: 5.0.0
>
> Attachments: OOZIE-2644.1.patch
>
>
> When you use the 
> [Workflow|https://oozie.apache.org/docs/4.2.0/WorkflowFunctionalSpec.html#a5_Workflow_Notifications]
>  or 
> [Coordinator|https://oozie.apache.org/docs/4.2.0/CoordinatorFunctionalSpec.html#a15._Coordinator_Notifications]
>  Notification features, Oozie can end up queuing up a lot of 
> {{WorkflowNotificationXCommand}} and {{CoordActionNotificationXCommand}}.  
> This happens even if there's no notification configured on the job (which I 
> imagine is most of the time); in this case, the {{execute}} method simply 
> does nothing.  This is wasteful and clogs the queue up.  
> We should change the code so that it doesn't queue up one of these Commands 
> unless there's actually a URL to notify.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: oozie-trunk-precommit-build #3377

[Apache Jenkins Server]

See 

--
[...truncated 5965 lines...]
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-tools ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-tools ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-tools ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-tools ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-tools ---
[INFO] Reading assembly descriptor: ../src/main/assemblies/tools.xml
[WARNING] The following patterns were never triggered in this artifact 
exclusion filter:
o  '*:*:pom:*'

[INFO] Copying files to 

[WARNING] Assembly file: 

 is not a regular file (it may be a directory). It cannot be attached to the 
project build for installation or deployment.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie MiniOozie 4.4.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-mini ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-mini 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-mini ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-mini ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-mini ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-mini ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-mini ---
[INFO] Reading assembly descriptor: src/main/assemblies/empty.xml
[INFO] 
[INFO] 
[INFO] Building Apache Oozie Distro 4.4.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-distro 
---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-distro 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-distro ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-distro ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-distro ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-distro ---
[INFO] 
[INFO] --- maven-antrun-plugin:1.6:run 

Re: Review Request 52429: OOZIE-1986 Add FindBugs report to pre-commit build

[Attila Sasvari]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52429/#review152482
---


Ship it!




Ship It!

- Attila Sasvari


On Oct. 12, 2016, 4:08 p.m., András Piros wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52429/
> ---
> 
> (Updated Oct. 12, 2016, 4:08 p.m.)
> 
> 
> Review request for oozie and Robert Kanter.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> We go for that solution:
> 
> 1. create a new script as `bin/test-patch-11-findbugs-diff` so that it comes 
> right after compile
> 
> 2. `mvn clean verify -DskipTests` before and after applying the patch, and 
> saving each and every module's `target/findbugs/findbugsXml.xml` to a secure 
> folder
> 
> 3. call the `findbugs-diff` app for all the before / after FindBugs xml pairs
> 
> 4. analyze all the `findbugs-new.xml` entries for nodes with XPath 
> `/BugCollection/BugInstance` entries. Stop on any BugInstance w/ `priority` 
> and / or `rank` lower than threshold
> 
> 5. have the reports merged and be downloadable in any case in following 
> scenarios:
> 
> * -1 for bugs that are too serious to apply the patch with them
> * 0 for other new bugs
> * +1 if there is no new bug
> 
> 
> Diffs
> -
> 
>   bin/test-patch-11-findbugs-diff PRE-CREATION 
>   bin/test-patch-11-findbugs-diff-0.1.0-all.jar.md5 PRE-CREATION 
>   pom.xml 83a1d42 
> 
> Diff: https://reviews.apache.org/r/52429/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> András Piros
> 
>

Review Request 52824: OOZIE-2644 Skip queuing Notification Commands when there's nothing to notify

[Azrael Seoeun Park]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52824/
---

Review request for oozie and Robert Kanter.


Bugs: OOZIE-2644
https://issues.apache.org/jira/browse/OOZIE-2644


Repository: oozie-git


Description
---

Skip queing Notification Command if the notification url is not configured.

And this patch fix the NullPointerException when CoordActionNotification 
executed from CoordActionKillXCommand. This is caused the runConf of actionBean 
is not set.


Diffs
-

  core/src/main/java/org/apache/oozie/CoordinatorActionBean.java b1be7c9 
  core/src/main/java/org/apache/oozie/CoordinatorJobBean.java 3a3120d 
  core/src/main/java/org/apache/oozie/command/NotificationXCommand.java 68359aa 
  core/src/main/java/org/apache/oozie/command/TransitionXCommand.java 6024138 
  core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f6 
  
core/src/main/java/org/apache/oozie/command/coord/CoordActionSkipXCommand.java 
ef0fc2d 
  
core/src/main/java/org/apache/oozie/command/coord/CoordActionStartXCommand.java 
45abe5a 
  
core/src/main/java/org/apache/oozie/command/coord/CoordActionTimeOutXCommand.java
 958194d 
  
core/src/main/java/org/apache/oozie/command/coord/CoordActionsKillXCommand.java 
d4c98c8 
  
core/src/main/java/org/apache/oozie/command/coord/CoordMaterializeTransitionXCommand.java
 f6c1782 
  core/src/main/java/org/apache/oozie/command/coord/CoordRerunXCommand.java 
f1509e6 
  core/src/main/java/org/apache/oozie/command/wf/ActionCheckXCommand.java 
d0551ff 
  core/src/main/java/org/apache/oozie/command/wf/ActionEndXCommand.java 740b8d3 
  core/src/main/java/org/apache/oozie/command/wf/ActionKillXCommand.java 
ac096cc 
  core/src/main/java/org/apache/oozie/command/wf/ActionStartXCommand.java 
edfac48 
  core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 836e5d4 
  core/src/main/java/org/apache/oozie/command/wf/KillXCommand.java 40b93d3 
  core/src/main/java/org/apache/oozie/command/wf/ResumeXCommand.java edbd767 
  core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java e95a60a 
  core/src/main/java/org/apache/oozie/command/wf/SuspendXCommand.java ef97990 
  core/src/main/java/org/apache/oozie/command/wf/WorkflowXCommand.java bc80dfe 
  
core/src/main/java/org/apache/oozie/executor/jpa/CoordActionQueryExecutor.java 
c0e6c19 
  core/src/main/java/org/apache/oozie/executor/jpa/CoordJobQueryExecutor.java 
432f075 
  core/src/main/java/org/apache/oozie/local/LocalOozie.java bf1b0db 
  core/src/main/resources/oozie-default.xml e71ebe3 
  
core/src/test/java/org/apache/oozie/command/coord/TestCoordActionNotificationXCommand.java
 8ca404f 
  
core/src/test/java/org/apache/oozie/command/coord/TestCoordActionsKillXCommand.java
 c9b2f28 
  
core/src/test/java/org/apache/oozie/command/wf/TestWorkflowNotificationXCommand.java
 ef342a4 
  
core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobQueryExecutor.java 
a0012fb 
  core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd 

Diff: https://reviews.apache.org/r/52824/diff/


Testing
---

Unit test.


Thanks,

Azrael Seoeun Park

[jira] [Updated] (OOZIE-2644) Skip queuing Notification Commands when there's nothing to notify

[Azrael Seoeun (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Azrael Seoeun updated OOZIE-2644:
-
Attachment: OOZIE-2644.1.patch

The patch is addressed comments.

As of now, CallableQueueService increase one more count in queued group for 
instrumentation if CompositeCallable queued. So, if CompositeCallable which 
contain 2 callables is queued, it add 3 to queued count. I am not sure it is 
intended or not. If it is not intended, I can create separate Jira issue.

And this patch fix the NullPointerException when CoordActionNotification 
executed from CoordActionKillXCommand. This is caused the runConf of actionBean 
is not set.  I can create separate Jira issue if needed.

CoordActionNotificationXCommand.java
{noformat}
Configuration conf;
try {
conf = new XConfiguration(new 
StringReader(actionBean.getRunConf()));
}
catch (IOException e1) {
LOG.warn("Configuration parse error. read from DB :" + 
actionBean.getRunConf());
throw new CommandException(ErrorCode.E1005, e1.getMessage(), e1);
}
{noformat}

> Skip queuing Notification Commands when there's nothing to notify
> -
>
> Key: OOZIE-2644
> URL: https://issues.apache.org/jira/browse/OOZIE-2644
> Project: Oozie
>  Issue Type: Improvement
>Affects Versions: 3.1.3
>Reporter: Robert Kanter
>Assignee: Azrael Seoeun
> Fix For: 5.0.0
>
> Attachments: OOZIE-2644.1.patch
>
>
> When you use the 
> [Workflow|https://oozie.apache.org/docs/4.2.0/WorkflowFunctionalSpec.html#a5_Workflow_Notifications]
>  or 
> [Coordinator|https://oozie.apache.org/docs/4.2.0/CoordinatorFunctionalSpec.html#a15._Coordinator_Notifications]
>  Notification features, Oozie can end up queuing up a lot of 
> {{WorkflowNotificationXCommand}} and {{CoordActionNotificationXCommand}}.  
> This happens even if there's no notification configured on the job (which I 
> imagine is most of the time); in this case, the {{execute}} method simply 
> does nothing.  This is wasteful and clogs the queue up.  
> We should change the code so that it doesn't queue up one of these Commands 
> unless there's actually a URL to notify.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] Subscription: Oozie Patch Available

[jira]

Issue Subscription
Filter: Oozie Patch Available (86 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-2619  Make  Hive action defaults to match hive defaults when running from 
command line
https://issues-test.apache.org/jira/browse/OOZIE-2619
OOZIE-2618  Include hive-common in hive-beeline as compile component for 
hive-server2 (hive2 action)
https://issues-test.apache.org/jira/browse/OOZIE-2618
OOZIE-2617  Read properties files in action configurations
https://issues-test.apache.org/jira/browse/OOZIE-2617
OOZIE-2616  Add Tez profile for Hive and Pig sharelibs
https://issues-test.apache.org/jira/browse/OOZIE-2616
OOZIE-2606  Set spark.yarn.jars to fix Spark 2.0 with Oozie
https://issues-test.apache.org/jira/browse/OOZIE-2606
OOZIE-2589  CompletedActionXCommand priority param is of no effect
https://issues-test.apache.org/jira/browse/OOZIE-2589
OOZIE-2588  Support getting credentials for same cluster hcat when credentials 
config is empty
https://issues-test.apache.org/jira/browse/OOZIE-2588
OOZIE-2585  Disable or remove test case 
TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and 
testMemoryUsageAndSpeed
https://issues-test.apache.org/jira/browse/OOZIE-2585
OOZIE-2584  Eliminate Thread.sleep() calls in TestMemoryLocks
https://issues-test.apache.org/jira/browse/OOZIE-2584
OOZIE-2583  oozie throws EL Exception when reference variable name containing 
dot
https://issues-test.apache.org/jira/browse/OOZIE-2583
OOZIE-2582  Populating external child Ids for action failures
https://issues-test.apache.org/jira/browse/OOZIE-2582
OOZIE-2573  dataset url  which contains spaces can not be handled rightly
https://issues-test.apache.org/jira/browse/OOZIE-2573
OOZIE-2572  SLA DURATION miss not shown when job is running for longer than 
expected time
https://issues-test.apache.org/jira/browse/OOZIE-2572
OOZIE-2571  Add scala.binary.version Maven property so that Scala 2.11 can be 
used
https://issues-test.apache.org/jira/browse/OOZIE-2571
OOZIE-2569  Adding yarn-site, core-site, hdfs-site and mapred-site into spark 
launcher 
https://issues-test.apache.org/jira/browse/OOZIE-2569
OOZIE-2568  SSH  action pretends to retry automaticly when it failed
https://issues-test.apache.org/jira/browse/OOZIE-2568
OOZIE-2564  Create new log4j config for unit tests so that logged messages 
contain threads
https://issues-test.apache.org/jira/browse/OOZIE-2564
OOZIE-2552  Update ActiveMQ version for security and other fixes
https://issues-test.apache.org/jira/browse/OOZIE-2552
OOZIE-2538  Update HttpClient versions to close security vulnerabilities
https://issues-test.apache.org/jira/browse/OOZIE-2538
OOZIE-2534  Versioned action libs (similar to how SharedLibs works)
https://issues-test.apache.org/jira/browse/OOZIE-2534
OOZIE-2530  Remove jline from parent and sharelib pig pom
https://issues-test.apache.org/jira/browse/OOZIE-2530
OOZIE-2528  Print out environment variables in LauncherMapper
https://issues-test.apache.org/jira/browse/OOZIE-2528
OOZIE-2521  Filter options are case sensitive for jobtye=bundle
https://issues-test.apache.org/jira/browse/OOZIE-2521
OOZIE-2518  Oozie is creating many files and directories in /tmp per day
https://issues-test.apache.org/jira/browse/OOZIE-2518
OOZIE-2517  Add support for startCreatedTime and endCreatedTime filters for 
coord and bundles
https://issues-test.apache.org/jira/browse/OOZIE-2517
OOZIE-2513  log.scan.duration should not be used for error and audit logs
https://issues-test.apache.org/jira/browse/OOZIE-2513
OOZIE-2512  ShareLibservice returns incorrect path for jar
https://issues-test.apache.org/jira/browse/OOZIE-2512
OOZIE-2508  Documentation change for Coord action rerun [OOZIE-1735]
https://issues-test.apache.org/jira/browse/OOZIE-2508
OOZIE-2501  ZK reentrant lock doesn't work for few cases
https://issues-test.apache.org/jira/browse/OOZIE-2501
OOZIE-2500  -DtestJarSimple option mentioned in minioozie doc does not work
https://issues-test.apache.org/jira/browse/OOZIE-2500
OOZIE-2499  map-reduce launcher does not need distributed files, archives 
except  jar of input/outputformat class
https://issues-test.apache.org/jira/browse/OOZIE-2499
OOZIE-2498  Oozie CallerId configuration for downstream components 
https://issues-test.apache.org/jira/browse/OOZIE-2498
OOZIE-2495  change action status from  ErrorType.NON_TRANSIENT to TRANSIENT 
when SSH action occurs AUTH_FAILED occasionally
https://issues-test.apache.org/jira/browse/OOZIE-2495
OOZIE-2491  oozie acl cannot specify group,it does`t work
https://issues-test.apache.org/jira/browse/OOZIE-2491

[jira] Subscription: Oozie Patch Available

[jira]

Issue Subscription
Filter: Oozie Patch Available (77 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-2710  Oozie HCatalog example workflow fails
https://issues.apache.org/jira/browse/OOZIE-2710
OOZIE-2694  Add logging for FsActionExecutor 
https://issues.apache.org/jira/browse/OOZIE-2694
OOZIE-2693  SimpleHCatDependencyCache.removeMissingDependency can throw NPE
https://issues.apache.org/jira/browse/OOZIE-2693
OOZIE-2692  Oozie job submit doesn't report error message to user if there is 
any issue with job conf
https://issues.apache.org/jira/browse/OOZIE-2692
OOZIE-2691  Show workflow action retry count to UI
https://issues.apache.org/jira/browse/OOZIE-2691
OOZIE-2690  OOZIE NPE while executing kill()
https://issues.apache.org/jira/browse/OOZIE-2690
OOZIE-2684  Bad database schema error for WF_ACTIONS table
https://issues.apache.org/jira/browse/OOZIE-2684
OOZIE-2674  Improve oozie commads documentation
https://issues.apache.org/jira/browse/OOZIE-2674
OOZIE-2670  Upgrade Hbase to 1.2
https://issues.apache.org/jira/browse/OOZIE-2670
OOZIE-2668  Status update and recovery problems when coord action and its 
children not in sync
https://issues.apache.org/jira/browse/OOZIE-2668
OOZIE-2666  Support embedding Jetty into Oozie
https://issues.apache.org/jira/browse/OOZIE-2666
OOZIE-2662  DB migration fails if DB is too big
https://issues.apache.org/jira/browse/OOZIE-2662
OOZIE-2654  Zookeeper dependent services should not depend on Connectionstate 
to be valid before cleaning up
https://issues.apache.org/jira/browse/OOZIE-2654
OOZIE-2652  Skip trash while deleting hive table partition
https://issues.apache.org/jira/browse/OOZIE-2652
OOZIE-2650  Retry coord start on database exceptions
https://issues.apache.org/jira/browse/OOZIE-2650
OOZIE-2641  rerunning an oozie coordinator job stucks on waiting
https://issues.apache.org/jira/browse/OOZIE-2641
OOZIE-2630  Oozie Coordinator EL Functions to get first day of the week/month
https://issues.apache.org/jira/browse/OOZIE-2630
OOZIE-2619  Make  Hive action defaults to match hive defaults when running from 
command line
https://issues.apache.org/jira/browse/OOZIE-2619
OOZIE-2618  Include hive-common in hive-beeline as compile component for 
hive-server2 (hive2 action)
https://issues.apache.org/jira/browse/OOZIE-2618
OOZIE-2616  Add Tez profile for Hive and Pig sharelibs
https://issues.apache.org/jira/browse/OOZIE-2616
OOZIE-2585  Disable or remove test case 
TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and 
testMemoryUsageAndSpeed
https://issues.apache.org/jira/browse/OOZIE-2585
OOZIE-2573  dataset url  which contains spaces can not be handled rightly
https://issues.apache.org/jira/browse/OOZIE-2573
OOZIE-2568  SSH  action pretends to retry automaticly when it failed
https://issues.apache.org/jira/browse/OOZIE-2568
OOZIE-2564  Create new log4j config for unit tests so that logged messages 
contain threads
https://issues.apache.org/jira/browse/OOZIE-2564
OOZIE-2534  Versioned action libs (similar to how SharedLibs works)
https://issues.apache.org/jira/browse/OOZIE-2534
OOZIE-2528  Print out environment variables in LauncherMapper
https://issues.apache.org/jira/browse/OOZIE-2528
OOZIE-2518  Oozie is creating many files and directories in /tmp per day
https://issues.apache.org/jira/browse/OOZIE-2518
OOZIE-2513  log.scan.duration should not be used for error and audit logs
https://issues.apache.org/jira/browse/OOZIE-2513
OOZIE-2499  map-reduce launcher does not need distributed files, archives 
except  jar of input/outputformat class
https://issues.apache.org/jira/browse/OOZIE-2499
OOZIE-2495  change action status from  ErrorType.NON_TRANSIENT to TRANSIENT 
when SSH action occurs AUTH_FAILED occasionally
https://issues.apache.org/jira/browse/OOZIE-2495
OOZIE-2473  Connection pool for SMTP connection
https://issues.apache.org/jira/browse/OOZIE-2473
OOZIE-2461  Workflow, Coordinator and Bundle job querying should have last 
modified filter
https://issues.apache.org/jira/browse/OOZIE-2461
OOZIE-2458  'oozie-setup.sh sharelib create' should ensure uploaded jars are 
world readable
https://issues.apache.org/jira/browse/OOZIE-2458
OOZIE-2457  Oozie log parsing regex consume more than 90% cpu
https://issues.apache.org/jira/browse/OOZIE-2457
OOZIE-2427  With Kerberos enabled, the authentication failure for the 
javascript resources under /ext-2.2 
https://issues.apache.org/jira/browse/OOZIE-2427
OOZIE-2396  oozie compile with hadoop 2.7.1 failed
https://issues.apache.org/jira/browse/OOZIE-2396
OOZIE-2393  Allow table