Build failed in Jenkins: oozie-trunk-precommit-build #3950

[Apache Jenkins Server]

See 


--
[...truncated 330.46 KB...]
[INFO] Copying ivy-2.4.0.jar to 

[INFO] Copying jackson-core-asl-1.9.13.jar to 

[INFO] Copying commons-jexl-2.1.1.jar to 

[INFO] Copying oozie-sharelib-hcatalog-5.0.0-SNAPSHOT.jar to 

[INFO] Copying hadoop-mapreduce-client-jobclient-2.6.0.jar to 

[INFO] Copying gson-2.7.jar to 

[INFO] Copying oozie-core-5.0.0-SNAPSHOT.jar to 

[INFO] Copying commons-pool-1.5.4.jar to 

[INFO] Copying htrace-core-3.0.4.jar to 

[INFO] Copying jaxb-impl-2.2.3-1.jar to 

[INFO] Copying jets3t-0.9.0.jar to 

[INFO] Copying datanucleus-rdbms-3.2.9.jar to 

[INFO] Copying hive-common-1.2.0.jar to 

[INFO] Copying snappy-java-1.0.4.1.jar to 

[INFO] Copying javax.annotation-api-1.2.jar to 

[INFO] Copying commons-compiler-2.7.6.jar to 

[INFO] Copying ant-1.9.1.jar to 

[INFO] Copying hadoop-annotations-2.6.0.jar to 

[INFO] Copying janino-2.7.6.jar to 

[INFO] Copying activemq-client-5.13.3.jar to 

[INFO] Copying guice-3.0.jar to 

[INFO] Copying httpclient-4.3.6.jar to 

[INFO] Copying xz-1.0.jar to 

[INFO] Copying oozie-client-5.0.0-SNAPSHOT.jar to 

[INFO] Copying hadoop-mapreduce-client-common-2.6.0.jar to 

[INFO] Copying leveldbjni-all-1.8.jar to 

[INFO] Copying jetty-util-6.1.26.jar to 

[INFO] Copying velocity-1.5.jar to 

[INFO] Copying jackson-core-2.4.2.jar to 

[INFO] Copying apache-log4j-extras-1.2.17.jar to 

[INFO] Copying netty-3.7.0.Final.jar to 

[INFO] Copying 

[jira] [Commented] (OOZIE-2854) Oozie should handle transient database problems

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075310#comment-16075310
 ] 

Hadoop QA commented on OOZIE-2854:
--

Testing JIRA OOZIE-2854

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 5 line(s) longer than 132 
characters
.{color:green}+1{color} the patch does adds/modifies 12 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
.{color:red}WARNING{color}: the current HEAD has 1 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
{color:red}-1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:red}-1{color} the patch modifies 1 JPA file(s), persistence.xml or 
*-orm.xml
{color:green}+1 TESTS{color}
.Tests run: 1606
.Tests rerun: 3
.Tests failed at first run: 
org.apache.oozie.action.hadoop.TestHdfsOperations,
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/oozie-trunk-precommit-build/3950/

> Oozie should handle transient database problems
> ---
>
> Key: OOZIE-2854
> URL: https://issues.apache.org/jira/browse/OOZIE-2854
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Reporter: Peter Bacsko
>Assignee: Andras Piros
> Attachments: OOZIE-2854-001.patch, OOZIE-2854-002.patch, 
> OOZIE-2854-003.patch, OOZIE-2854-004.patch, OOZIE-2854-005.patch, 
> OOZIE-2854.006.patch, OOZIE-2854.007.patch, OOZIE-2854.008.patch, 
> OOZIE-2854-POC-001.patch
>
>
> There can be problems when Oozie cannot update the database properly. 
> Recently, we have experienced erratic behavior with two setups:
> * MySQL with the Galera cluster manager. Galera uses cluster-wide optimistic 
> locking which might cause a transaction to rollback if there are two or more 
> parallel transaction running and one of them cannot complete because of a 
> conflict.
> * MySQL with Percona XtraDB Cluster. If one of the MySQL instances is killed, 
> Oozie might get "Communications link failure" exception during the failover.
> The problem is that failed DB transactions later might cause a workflow 
> (which are started/re-started by RecoveryService) to get stuck. It's not 
> clear to us how this happens but it has to do with the fact that certain DB 
> updates are not executed.
> The solution is to use some sort of retry logic with exponential backoff if 
> the DB update fails. We could start with a 100ms wait time which is doubled 
> at every retry. The operation can be considered a failure if it still fails 
> after 10 attempts. These values could be configurable. We should discuss 
> initial values in the scope of this JIRA.
> Note that this solution is to handle *transient* failures. If the DB is down 
> for a longer period of time, we have to accept that the internal state of 
> Oozie is corrupted.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-2977) error: self-closing element not allowed

[Artem Ervits (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075181#comment-16075181
 ] 

Artem Ervits commented on OOZIE-2977:
-

not sure why tests are involved, my changes address javadoc comments only.

> error: self-closing element not allowed
> ---
>
> Key: OOZIE-2977
> URL: https://issues.apache.org/jira/browse/OOZIE-2977
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 4.3.0
>Reporter: Artem Ervits
>Assignee: Artem Ervits
>Priority: Trivial
> Fix For: 5.0.0
>
> Attachments: OOZIE-2977-0.patch
>
>
> compiling with Java 1.8, javadoc throws the following errors
> {noformat}
> /Users/aervits/NetBeansProjects/OOZIE-JAVADOC/oozie/core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java:464:
>  error: self-closing element not allowed
> /Users/aervits/NetBeansProjects/OOZIE-JAVADOC/oozie/core/src/main/java/org/apache/oozie/util/DateUtils.java:209:
>  error: self-closing element not allowed
> /Users/aervits/NetBeansProjects/OOZIE-JAVADOC/oozie/core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java:464:
>  error: self-closing element not allowed
> /Users/aervits/NetBeansProjects/OOZIE-JAVADOC/oozie/core/src/main/java/org/apache/oozie/util/DateUtils.java:209:
>  error: self-closing element not allowed
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-2969) Drop support for Java 1.7

[Artem Ervits (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075177#comment-16075177
 ] 

Artem Ervits commented on OOZIE-2969:
-

[~rkanter] please advise

> Drop support for Java 1.7
> -
>
> Key: OOZIE-2969
> URL: https://issues.apache.org/jira/browse/OOZIE-2969
> Project: Oozie
>  Issue Type: New Feature
>  Components: core
>Affects Versions: 4.3.0
>Reporter: Artem Ervits
>Assignee: Artem Ervits
> Fix For: trunk, 5.0.0
>
> Attachments: OOZIE-2969-0.patch, OOZIE-2969-0.patch, 
> OOZIE-2969-1.patch
>
>
>  I see the last jira to drop 1.6 support only resolved end of 2016 with 
> release of 4.3 but wondering if 5.0 is a good time to drop 1.7 support? 5.0 
> will support Hadoop 3 and minimum required version to support it is 1.8. 
> Happy to pick it up if there's interest.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-2969) Drop support for Java 1.7

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075154#comment-16075154
 ] 

Hadoop QA commented on OOZIE-2969:
--

Testing JIRA OOZIE-2969

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
.{color:red}WARNING{color}: the current HEAD has 1 Javadoc warning(s)
{color:red}-1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:red}-1{color} patch does not compile
.{color:red}-1{color} the patch seems to introduce 1 new javac warning(s)
{color:green}+1{color} There are no new bugs found in total.
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color} - patch does not compile, cannot run testcases
{color:red}-1 DISTRO{color}
.{color:red}-1{color} distro tarball fails with the patch


{color:red}*-1 Overall result, please check the reported -1(s)*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/oozie-trunk-precommit-build/3951/

> Drop support for Java 1.7
> -
>
> Key: OOZIE-2969
> URL: https://issues.apache.org/jira/browse/OOZIE-2969
> Project: Oozie
>  Issue Type: New Feature
>  Components: core
>Affects Versions: 4.3.0
>Reporter: Artem Ervits
>Assignee: Artem Ervits
> Fix For: trunk, 5.0.0
>
> Attachments: OOZIE-2969-0.patch, OOZIE-2969-0.patch, 
> OOZIE-2969-1.patch
>
>
>  I see the last jira to drop 1.6 support only resolved end of 2016 with 
> release of 4.3 but wondering if 5.0 is a good time to drop 1.7 support? 5.0 
> will support Hadoop 3 and minimum required version to support it is 1.8. 
> Happy to pick it up if there's interest.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Build failed in Jenkins: oozie-trunk-precommit-build #3951

[Apache Jenkins Server]

See 


--
[...truncated 99.47 KB...]
[INFO] This project has been banned from the build due to previous failures.
[INFO] 
[INFO] 
[INFO] 
[INFO] Skipping Apache Oozie Share Lib Sqoop
[INFO] This project has been banned from the build due to previous failures.
[INFO] 
[INFO] 
[INFO] 
[INFO] Skipping Apache Oozie Examples
[INFO] This project has been banned from the build due to previous failures.
[INFO] 
[INFO] 
[INFO] 
[INFO] Skipping Apache Oozie Share Lib Spark
[INFO] This project has been banned from the build due to previous failures.
[INFO] 
[INFO] 
[INFO] 
[INFO] Skipping Apache Oozie Share Lib
[INFO] This project has been banned from the build due to previous failures.
[INFO] 
[INFO] 
[INFO] 
[INFO] Skipping Apache Oozie Docs
[INFO] This project has been banned from the build due to previous failures.
[INFO] 
[INFO] 
[INFO] 
[INFO] Skipping Apache Oozie WebApp
[INFO] This project has been banned from the build due to previous failures.
[INFO] 
[INFO] 
[INFO] 
[INFO] Skipping Apache Oozie Tools
[INFO] This project has been banned from the build due to previous failures.
[INFO] 
[INFO] 
[INFO] 
[INFO] Skipping Apache Oozie MiniOozie
[INFO] This project has been banned from the build due to previous failures.
[INFO] 
[INFO] 
[INFO] 
[INFO] Skipping Apache Oozie Server
[INFO] This project has been banned from the build due to previous failures.
[INFO] 
[INFO] 
[INFO] 
[INFO] Skipping Apache Oozie Distro
[INFO] This project has been banned from the build due to previous failures.
[INFO] 
[INFO] 
[INFO] 
[INFO] Skipping Apache Oozie ZooKeeper Security Tests
[INFO] This project has been banned from the build due to previous failures.
[INFO] 
[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Oozie Main .. FAILURE [  0.311 s]
[INFO] Apache Oozie Client  SKIPPED
[INFO] Apache Oozie Share Lib Oozie ... SKIPPED
[INFO] Apache Oozie Share Lib HCatalog  SKIPPED
[INFO] Apache Oozie Share Lib Distcp .. SKIPPED
[INFO] Apache Oozie Core .. SKIPPED
[INFO] Apache Oozie Share Lib Streaming ... SKIPPED
[INFO] Apache Oozie Share Lib Pig . SKIPPED
[INFO] Apache Oozie Share Lib Hive  SKIPPED
[INFO] Apache Oozie Share Lib Hive 2 .. SKIPPED
[INFO] Apache Oozie Share Lib Sqoop ... SKIPPED
[INFO] Apache Oozie Examples .. SKIPPED
[INFO] Apache Oozie Share Lib Spark ... SKIPPED
[INFO] Apache Oozie Share Lib . SKIPPED
[INFO] Apache Oozie Docs .. SKIPPED
[INFO] Apache Oozie WebApp  SKIPPED
[INFO] Apache Oozie Tools . SKIPPED
[INFO] Apache Oozie MiniOozie . SKIPPED
[INFO] Apache Oozie Server  SKIPPED
[INFO] Apache Oozie Distro  SKIPPED
[INFO] Apache Oozie ZooKeeper Security Tests .. SKIPPED

[jira] [Updated] (OOZIE-2969) Drop support for Java 1.7

[Artem Ervits (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Artem Ervits updated OOZIE-2969:

Attachment: OOZIE-2969-1.patch

not sure why patch doesn't compile remotely, it works fine locally.

> Drop support for Java 1.7
> -
>
> Key: OOZIE-2969
> URL: https://issues.apache.org/jira/browse/OOZIE-2969
> Project: Oozie
>  Issue Type: New Feature
>  Components: core
>Affects Versions: 4.3.0
>Reporter: Artem Ervits
>Assignee: Artem Ervits
> Fix For: trunk, 5.0.0
>
> Attachments: OOZIE-2969-0.patch, OOZIE-2969-0.patch, 
> OOZIE-2969-1.patch
>
>
>  I see the last jira to drop 1.6 support only resolved end of 2016 with 
> release of 4.3 but wondering if 5.0 is a good time to drop 1.7 support? 5.0 
> will support Hadoop 3 and minimum required version to support it is 1.8. 
> Happy to pick it up if there's interest.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 60544: OOZIE-2854 Oozie should handle transient database problems

[András Piros]

> On July 4, 2017, 1:28 p.m., Peter Bacsko wrote:
> > core/src/main/java/org/apache/oozie/util/db/RetryAttemptCounter.java
> > Lines 41 (patched)
> > 
> >
> > Let's think about the naming.. we don't just count with this class, we 
> > also store state information. If you were not happy with NestedRetryUti, we 
> > can call it NestedRetryHandler or anyhing.

Renamed to `RetryAttemptState`, and renamed most of the methods inside. I don't 
like the words `Util(s)` and `Helper(s)` in variable or class names.


- András


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60544/#review179575
---


On July 5, 2017, 5:45 p.m., András Piros wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60544/
> ---
> 
> (Updated July 5, 2017, 5:45 p.m.)
> 
> 
> Review request for oozie, Attila Sasvari, Peter Cseh, and Peter Bacsko.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2854
> 
> 
> Diffs
> -
> 
>   core/pom.xml acddf349a89cf09a7fc4f384ebcaec56dfd0ab48 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/JsonBeanPersisterExecutor.java
>  PRE-CREATION 
>   core/src/main/java/org/apache/oozie/executor/jpa/QueryExecutor.java 
> 8d94c23e40d1281864db40e141b200ca207a6324 
>   core/src/main/java/org/apache/oozie/service/JPAService.java 
> 028381d3b72bcc3b8c2cd27cacb3e0ac6d48d146 
>   core/src/main/java/org/apache/oozie/sla/SLASummaryBean.java 
> cfe1522a4b1f89085eb29e7f1281c2abd631bdc2 
>   core/src/main/java/org/apache/oozie/store/WorkflowStore.java 
> c565e74893b863caef6c93015cfe38fe520d04ec 
>   core/src/main/java/org/apache/oozie/util/db/BasicDataSourceWrapper.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/db/DatabaseRetryPredicate.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/db/FailingConnectionWrapper.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/db/FailingHSQLDBDriverWrapper.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/db/FailingMySQLDriverWrapper.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/db/OperationRetryHandler.java 
> PRE-CREATION 
>   
> core/src/main/java/org/apache/oozie/util/db/PersistenceExceptionSubclassFilterRetryPredicate.java
>  PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/db/RetryAttemptState.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/db/RuntimeExceptionInjector.java 
> PRE-CREATION 
>   core/src/main/resources/META-INF/persistence.xml 
> bad9278597fcd4f93b4cc482afae8af14beaa922 
>   core/src/main/resources/oozie-default.xml 
> 5629a897b4a1834554a1a5f7a21984ea857ed133 
>   core/src/main/resources/oozie-log4j.properties 
> c065f3cd4c5a3df1308b69d7c16e8fcfa8796efc 
>   core/src/test/java/org/apache/oozie/command/SkipCommitFaultInjection.java  
>   core/src/test/java/org/apache/oozie/test/XTestCase.java 
> 161927ac8f1132b3080d2924844826fcc7b807a5 
>   
> core/src/test/java/org/apache/oozie/util/db/TestOozieDmlStatementPredicate.java
>  PRE-CREATION 
>   core/src/test/java/org/apache/oozie/util/db/TestOperationRetryHandler.java 
> PRE-CREATION 
>   
> core/src/test/java/org/apache/oozie/util/db/TestPersistenceExceptionSubclassFilterRetryPredicate.java
>  PRE-CREATION 
>   core/src/test/java/org/apache/oozie/util/db/TestRetryAttemptState.java 
> PRE-CREATION 
>   minitest/pom.xml 9515284bb5f32c279a93161c10e6571680e4f9fc 
>   
> minitest/src/test/java/org/apache/oozie/test/TestParallelJPAOperationRetries.java
>  PRE-CREATION 
>   minitest/src/test/java/org/apache/oozie/test/TestWorkflowRetries.java 
> PRE-CREATION 
>   minitest/src/test/java/org/apache/oozie/test/WorkflowTest.java 
> 2845f0af6efb9ef75fdbfcb326115c62e6fb3bdd 
>   minitest/src/test/resources/hsqldb-oozie-site.xml 
> fa5fe9c3185e973e8247d7bf10b126119d9c02c9 
>   minitest/src/test/resources/oozie-log4j.properties 
> c142d725140930bfa89cd2b163d0768a4c3a750a 
>   minitest/src/test/resources/parallel-fs-and-shell.xml PRE-CREATION 
>   minitest/src/test/resources/wf-test.xml 
> 20c4946862039a65c76ed7f49991345e90a694de 
>   pom.xml 16c5137d44d7db891da46f80adb51c85e4c1b214 
> 
> 
> Diff: https://reviews.apache.org/r/60544/diff/4/
> 
> 
> Testing
> ---
> 
> Tests covered in code:
> 
> Unit tests
> ==
> 
> * testing the retry handler, the retry predicate filter, and parallel calls 
> to JPA `EntityManager` (mostly Oozie database reads and writes) when 
> injecting failures
> 
> Integration tests
> =
> 
> * using the `MiniOozieTestCase` framework
> * fixing it so that also asynchronous workflow applications (the ones 

Re: Review Request 60544: OOZIE-2854 Oozie should handle transient database problems

[András Piros]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60544/
---

(Updated July 5, 2017, 5:45 p.m.)


Review request for oozie, Attila Sasvari, Peter Cseh, and Peter Bacsko.


Changes
---

Addressing latest review comments. Still need to document in TWiki.


Repository: oozie-git


Description
---

https://issues.apache.org/jira/browse/OOZIE-2854


Diffs (updated)
-

  core/pom.xml acddf349a89cf09a7fc4f384ebcaec56dfd0ab48 
  
core/src/main/java/org/apache/oozie/executor/jpa/JsonBeanPersisterExecutor.java 
PRE-CREATION 
  core/src/main/java/org/apache/oozie/executor/jpa/QueryExecutor.java 
8d94c23e40d1281864db40e141b200ca207a6324 
  core/src/main/java/org/apache/oozie/service/JPAService.java 
028381d3b72bcc3b8c2cd27cacb3e0ac6d48d146 
  core/src/main/java/org/apache/oozie/sla/SLASummaryBean.java 
cfe1522a4b1f89085eb29e7f1281c2abd631bdc2 
  core/src/main/java/org/apache/oozie/store/WorkflowStore.java 
c565e74893b863caef6c93015cfe38fe520d04ec 
  core/src/main/java/org/apache/oozie/util/db/BasicDataSourceWrapper.java 
PRE-CREATION 
  core/src/main/java/org/apache/oozie/util/db/DatabaseRetryPredicate.java 
PRE-CREATION 
  core/src/main/java/org/apache/oozie/util/db/FailingConnectionWrapper.java 
PRE-CREATION 
  core/src/main/java/org/apache/oozie/util/db/FailingHSQLDBDriverWrapper.java 
PRE-CREATION 
  core/src/main/java/org/apache/oozie/util/db/FailingMySQLDriverWrapper.java 
PRE-CREATION 
  core/src/main/java/org/apache/oozie/util/db/OperationRetryHandler.java 
PRE-CREATION 
  
core/src/main/java/org/apache/oozie/util/db/PersistenceExceptionSubclassFilterRetryPredicate.java
 PRE-CREATION 
  core/src/main/java/org/apache/oozie/util/db/RetryAttemptState.java 
PRE-CREATION 
  core/src/main/java/org/apache/oozie/util/db/RuntimeExceptionInjector.java 
PRE-CREATION 
  core/src/main/resources/META-INF/persistence.xml 
bad9278597fcd4f93b4cc482afae8af14beaa922 
  core/src/main/resources/oozie-default.xml 
5629a897b4a1834554a1a5f7a21984ea857ed133 
  core/src/main/resources/oozie-log4j.properties 
c065f3cd4c5a3df1308b69d7c16e8fcfa8796efc 
  core/src/test/java/org/apache/oozie/command/SkipCommitFaultInjection.java  
  core/src/test/java/org/apache/oozie/test/XTestCase.java 
161927ac8f1132b3080d2924844826fcc7b807a5 
  
core/src/test/java/org/apache/oozie/util/db/TestOozieDmlStatementPredicate.java 
PRE-CREATION 
  core/src/test/java/org/apache/oozie/util/db/TestOperationRetryHandler.java 
PRE-CREATION 
  
core/src/test/java/org/apache/oozie/util/db/TestPersistenceExceptionSubclassFilterRetryPredicate.java
 PRE-CREATION 
  core/src/test/java/org/apache/oozie/util/db/TestRetryAttemptState.java 
PRE-CREATION 
  minitest/pom.xml 9515284bb5f32c279a93161c10e6571680e4f9fc 
  
minitest/src/test/java/org/apache/oozie/test/TestParallelJPAOperationRetries.java
 PRE-CREATION 
  minitest/src/test/java/org/apache/oozie/test/TestWorkflowRetries.java 
PRE-CREATION 
  minitest/src/test/java/org/apache/oozie/test/WorkflowTest.java 
2845f0af6efb9ef75fdbfcb326115c62e6fb3bdd 
  minitest/src/test/resources/hsqldb-oozie-site.xml 
fa5fe9c3185e973e8247d7bf10b126119d9c02c9 
  minitest/src/test/resources/oozie-log4j.properties 
c142d725140930bfa89cd2b163d0768a4c3a750a 
  minitest/src/test/resources/parallel-fs-and-shell.xml PRE-CREATION 
  minitest/src/test/resources/wf-test.xml 
20c4946862039a65c76ed7f49991345e90a694de 
  pom.xml 16c5137d44d7db891da46f80adb51c85e4c1b214 


Diff: https://reviews.apache.org/r/60544/diff/4/

Changes: https://reviews.apache.org/r/60544/diff/3-4/


Testing
---

Tests covered in code:

Unit tests
==

* testing the retry handler, the retry predicate filter, and parallel calls to 
JPA `EntityManager` (mostly Oozie database reads and writes) when injecting 
failures

Integration tests
=

* using the `MiniOozieTestCase` framework
* fixing it so that also asynchronous workflow applications (the ones that use 
`CallableQueueService`) can be run
* following workflow scenarios:
* a very simple one consisting only of a `` and an `` node
* a more sophisticated one consisting of multiple synchronous `` nodes and 
a `` node
* the ultimate one consisting of a `` node, and two branches of an 
`` and an asynchronous `` nodes

Test cases run:
```
mvn clean test 
-Dtest=TestOperationRetryHandler,TestPersistenceExceptionSubclassFilterRetryPredicate,TestParallelJPAOperationRetries,TestWorkflow,TestWorkflowRetries,TestJPAService,TestRetryAttemptCounter
```

Functional and stress tests performed on a 4-node MySQL cluster. MySQL daemon 
has been stopped / killed / restarted several times. Also firewall rules have 
been modified temporarily to simulate network outages.


Thanks,

András Piros

[jira] [Updated] (OOZIE-2854) Oozie should handle transient database problems

[Andras Piros (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros updated OOZIE-2854:

Attachment: OOZIE-2854.008.patch

Addressing latest review comments. Still need to document in TWiki.

> Oozie should handle transient database problems
> ---
>
> Key: OOZIE-2854
> URL: https://issues.apache.org/jira/browse/OOZIE-2854
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Reporter: Peter Bacsko
>Assignee: Andras Piros
> Attachments: OOZIE-2854-001.patch, OOZIE-2854-002.patch, 
> OOZIE-2854-003.patch, OOZIE-2854-004.patch, OOZIE-2854-005.patch, 
> OOZIE-2854.006.patch, OOZIE-2854.007.patch, OOZIE-2854.008.patch, 
> OOZIE-2854-POC-001.patch
>
>
> There can be problems when Oozie cannot update the database properly. 
> Recently, we have experienced erratic behavior with two setups:
> * MySQL with the Galera cluster manager. Galera uses cluster-wide optimistic 
> locking which might cause a transaction to rollback if there are two or more 
> parallel transaction running and one of them cannot complete because of a 
> conflict.
> * MySQL with Percona XtraDB Cluster. If one of the MySQL instances is killed, 
> Oozie might get "Communications link failure" exception during the failover.
> The problem is that failed DB transactions later might cause a workflow 
> (which are started/re-started by RecoveryService) to get stuck. It's not 
> clear to us how this happens but it has to do with the fact that certain DB 
> updates are not executed.
> The solution is to use some sort of retry logic with exponential backoff if 
> the DB update fails. We could start with a 100ms wait time which is doubled 
> at every retry. The operation can be considered a failure if it still fails 
> after 10 attempts. These values could be configurable. We should discuss 
> initial values in the scope of this JIRA.
> Note that this solution is to handle *transient* failures. If the DB is down 
> for a longer period of time, we have to accept that the internal state of 
> Oozie is corrupted.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (OOZIE-2854) Oozie should handle transient database problems

[Andras Piros (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros updated OOZIE-2854:

Attachment: (was: OOZIE-2854.008.patch)

> Oozie should handle transient database problems
> ---
>
> Key: OOZIE-2854
> URL: https://issues.apache.org/jira/browse/OOZIE-2854
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Reporter: Peter Bacsko
>Assignee: Andras Piros
> Attachments: OOZIE-2854-001.patch, OOZIE-2854-002.patch, 
> OOZIE-2854-003.patch, OOZIE-2854-004.patch, OOZIE-2854-005.patch, 
> OOZIE-2854.006.patch, OOZIE-2854.007.patch, OOZIE-2854-POC-001.patch
>
>
> There can be problems when Oozie cannot update the database properly. 
> Recently, we have experienced erratic behavior with two setups:
> * MySQL with the Galera cluster manager. Galera uses cluster-wide optimistic 
> locking which might cause a transaction to rollback if there are two or more 
> parallel transaction running and one of them cannot complete because of a 
> conflict.
> * MySQL with Percona XtraDB Cluster. If one of the MySQL instances is killed, 
> Oozie might get "Communications link failure" exception during the failover.
> The problem is that failed DB transactions later might cause a workflow 
> (which are started/re-started by RecoveryService) to get stuck. It's not 
> clear to us how this happens but it has to do with the fact that certain DB 
> updates are not executed.
> The solution is to use some sort of retry logic with exponential backoff if 
> the DB update fails. We could start with a 100ms wait time which is doubled 
> at every retry. The operation can be considered a failure if it still fails 
> after 10 attempts. These values could be configurable. We should discuss 
> initial values in the scope of this JIRA.
> Note that this solution is to handle *transient* failures. If the DB is down 
> for a longer period of time, we have to accept that the internal state of 
> Oozie is corrupted.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-2854) Oozie should handle transient database problems

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16074978#comment-16074978
 ] 

Hadoop QA commented on OOZIE-2854:
--

Testing JIRA OOZIE-2854

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch does adds/modifies 3 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
.{color:red}WARNING{color}: the current HEAD has 1 Javadoc warning(s)
{color:red}-1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:red}-1{color} patch does not compile
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
{color:red}-1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:red}-1{color} the patch modifies 1 JPA file(s), persistence.xml or 
*-orm.xml
{color:red}-1 TESTS{color} - patch does not compile, cannot run testcases
{color:red}-1 DISTRO{color}
.{color:red}-1{color} distro tarball fails with the patch


{color:red}*-1 Overall result, please check the reported -1(s)*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/oozie-trunk-precommit-build/3949/

> Oozie should handle transient database problems
> ---
>
> Key: OOZIE-2854
> URL: https://issues.apache.org/jira/browse/OOZIE-2854
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Reporter: Peter Bacsko
>Assignee: Andras Piros
> Attachments: OOZIE-2854-001.patch, OOZIE-2854-002.patch, 
> OOZIE-2854-003.patch, OOZIE-2854-004.patch, OOZIE-2854-005.patch, 
> OOZIE-2854.006.patch, OOZIE-2854.007.patch, OOZIE-2854.008.patch, 
> OOZIE-2854-POC-001.patch
>
>
> There can be problems when Oozie cannot update the database properly. 
> Recently, we have experienced erratic behavior with two setups:
> * MySQL with the Galera cluster manager. Galera uses cluster-wide optimistic 
> locking which might cause a transaction to rollback if there are two or more 
> parallel transaction running and one of them cannot complete because of a 
> conflict.
> * MySQL with Percona XtraDB Cluster. If one of the MySQL instances is killed, 
> Oozie might get "Communications link failure" exception during the failover.
> The problem is that failed DB transactions later might cause a workflow 
> (which are started/re-started by RecoveryService) to get stuck. It's not 
> clear to us how this happens but it has to do with the fact that certain DB 
> updates are not executed.
> The solution is to use some sort of retry logic with exponential backoff if 
> the DB update fails. We could start with a 100ms wait time which is doubled 
> at every retry. The operation can be considered a failure if it still fails 
> after 10 attempts. These values could be configurable. We should discuss 
> initial values in the scope of this JIRA.
> Note that this solution is to handle *transient* failures. If the DB is down 
> for a longer period of time, we have to accept that the internal state of 
> Oozie is corrupted.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Build failed in Jenkins: oozie-trunk-precommit-build #3949

[Apache Jenkins Server]

See 


--
[...truncated 167.60 KB...]
[INFO] 
[INFO] --- maven-antrun-plugin:1.6:run (default) @ oozie-sharelib-hcatalog ---
[INFO] Executing tasks

main:
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-sharelib-hcatalog ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-sharelib-hcatalog ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ 
oozie-sharelib-hcatalog ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-sharelib-hcatalog 
---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-sharelib-hcatalog ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ 
oozie-sharelib-hcatalog ---
[INFO] Reading assembly descriptor: 
../../src/main/assemblies/partial-sharelib.xml
[WARNING] The following patterns were never triggered in this artifact 
exclusion filter:
o  'org.apache.oozie:oozie-hadoop-distcp:jar'

[INFO] Copying files to 

[INFO] 
[INFO] 
[INFO] Building Apache Oozie Share Lib Distcp 5.0.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
oozie-sharelib-distcp ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-sharelib-distcp ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ 
oozie-sharelib-distcp ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-dependency-plugin:2.4:build-classpath (gen-classpath) @ 
oozie-sharelib-distcp ---
[INFO] Wrote classpath file 
'
[INFO] 
[INFO] --- maven-antrun-plugin:1.6:run (default) @ oozie-sharelib-distcp ---
[INFO] Executing tasks

main:
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-sharelib-distcp ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-sharelib-distcp ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ 
oozie-sharelib-distcp ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-sharelib-distcp ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-sharelib-distcp ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ 
oozie-sharelib-distcp ---
[INFO] Reading assembly descriptor: 
../../src/main/assemblies/partial-sharelib.xml
[WARNING] The following patterns were never triggered in this artifact 
exclusion filter:
o  'org.apache.oozie:oozie-hadoop-distcp:jar'

[INFO] Copying files to 

[INFO] 
[INFO] 
[INFO] Building Apache Oozie Core 5.0.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-core ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-core ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 15 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- 

[jira] [Updated] (OOZIE-2854) Oozie should handle transient database problems

[Andras Piros (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros updated OOZIE-2854:

Attachment: OOZIE-2854.008.patch

Addressing latest review comments.

> Oozie should handle transient database problems
> ---
>
> Key: OOZIE-2854
> URL: https://issues.apache.org/jira/browse/OOZIE-2854
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Reporter: Peter Bacsko
>Assignee: Andras Piros
> Attachments: OOZIE-2854-001.patch, OOZIE-2854-002.patch, 
> OOZIE-2854-003.patch, OOZIE-2854-004.patch, OOZIE-2854-005.patch, 
> OOZIE-2854.006.patch, OOZIE-2854.007.patch, OOZIE-2854.008.patch, 
> OOZIE-2854-POC-001.patch
>
>
> There can be problems when Oozie cannot update the database properly. 
> Recently, we have experienced erratic behavior with two setups:
> * MySQL with the Galera cluster manager. Galera uses cluster-wide optimistic 
> locking which might cause a transaction to rollback if there are two or more 
> parallel transaction running and one of them cannot complete because of a 
> conflict.
> * MySQL with Percona XtraDB Cluster. If one of the MySQL instances is killed, 
> Oozie might get "Communications link failure" exception during the failover.
> The problem is that failed DB transactions later might cause a workflow 
> (which are started/re-started by RecoveryService) to get stuck. It's not 
> clear to us how this happens but it has to do with the fact that certain DB 
> updates are not executed.
> The solution is to use some sort of retry logic with exponential backoff if 
> the DB update fails. We could start with a 100ms wait time which is doubled 
> at every retry. The operation can be considered a failure if it still fails 
> after 10 attempts. These values could be configurable. We should discuss 
> initial values in the scope of this JIRA.
> Note that this solution is to handle *transient* failures. If the DB is down 
> for a longer period of time, we have to accept that the internal state of 
> Oozie is corrupted.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 60544: OOZIE-2854 Oozie should handle transient database problems

[András Piros]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60544/
---

(Updated July 5, 2017, 3:44 p.m.)


Review request for oozie, Attila Sasvari, Peter Cseh, and Peter Bacsko.


Changes
---

Addressing latest review comments.


Repository: oozie-git


Description
---

https://issues.apache.org/jira/browse/OOZIE-2854


Diffs (updated)
-

  core/pom.xml acddf349a89cf09a7fc4f384ebcaec56dfd0ab48 
  core/src/main/java/org/apache/oozie/executor/jpa/QueryExecutor.java 
8d94c23e40d1281864db40e141b200ca207a6324 
  core/src/main/java/org/apache/oozie/service/JPAService.java 
028381d3b72bcc3b8c2cd27cacb3e0ac6d48d146 
  core/src/main/java/org/apache/oozie/sla/SLASummaryBean.java 
cfe1522a4b1f89085eb29e7f1281c2abd631bdc2 
  core/src/main/java/org/apache/oozie/store/WorkflowStore.java 
c565e74893b863caef6c93015cfe38fe520d04ec 
  core/src/main/resources/META-INF/persistence.xml 
bad9278597fcd4f93b4cc482afae8af14beaa922 
  core/src/main/resources/oozie-default.xml 
5629a897b4a1834554a1a5f7a21984ea857ed133 
  core/src/main/resources/oozie-log4j.properties 
c065f3cd4c5a3df1308b69d7c16e8fcfa8796efc 
  core/src/test/java/org/apache/oozie/command/SkipCommitFaultInjection.java 
158fbfafd2ed4ebc741c4d33252d92e5bb77bb63 
  core/src/test/java/org/apache/oozie/test/XTestCase.java 
161927ac8f1132b3080d2924844826fcc7b807a5 
  minitest/pom.xml 9515284bb5f32c279a93161c10e6571680e4f9fc 
  minitest/src/test/java/org/apache/oozie/test/WorkflowTest.java 
2845f0af6efb9ef75fdbfcb326115c62e6fb3bdd 
  minitest/src/test/resources/hsqldb-oozie-site.xml 
fa5fe9c3185e973e8247d7bf10b126119d9c02c9 
  minitest/src/test/resources/oozie-log4j.properties 
c142d725140930bfa89cd2b163d0768a4c3a750a 
  minitest/src/test/resources/wf-test.xml 
20c4946862039a65c76ed7f49991345e90a694de 
  pom.xml 16c5137d44d7db891da46f80adb51c85e4c1b214 


Diff: https://reviews.apache.org/r/60544/diff/3/

Changes: https://reviews.apache.org/r/60544/diff/2-3/


Testing
---

Tests covered in code:

Unit tests
==

* testing the retry handler, the retry predicate filter, and parallel calls to 
JPA `EntityManager` (mostly Oozie database reads and writes) when injecting 
failures

Integration tests
=

* using the `MiniOozieTestCase` framework
* fixing it so that also asynchronous workflow applications (the ones that use 
`CallableQueueService`) can be run
* following workflow scenarios:
* a very simple one consisting only of a `` and an `` node
* a more sophisticated one consisting of multiple synchronous `` nodes and 
a `` node
* the ultimate one consisting of a `` node, and two branches of an 
`` and an asynchronous `` nodes

Test cases run:
```
mvn clean test 
-Dtest=TestOperationRetryHandler,TestPersistenceExceptionSubclassFilterRetryPredicate,TestParallelJPAOperationRetries,TestWorkflow,TestWorkflowRetries,TestJPAService,TestRetryAttemptCounter
```

Functional and stress tests performed on a 4-node MySQL cluster. MySQL daemon 
has been stopped / killed / restarted several times. Also firewall rules have 
been modified temporarily to simulate network outages.


Thanks,

András Piros

[jira] [Commented] (OOZIE-2371) Add docs for state transitions for WF Action states

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16074823#comment-16074823
 ] 

Hadoop QA commented on OOZIE-2371:
--

Testing JIRA OOZIE-2371

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
.{color:red}WARNING{color}: the current HEAD has 1 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1965
.Tests rerun: 3
.Tests failed at first run: 
org.apache.oozie.action.hadoop.TestHdfsOperations,
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/oozie-trunk-precommit-build/3948/

> Add docs for state transitions for WF Action states
> ---
>
> Key: OOZIE-2371
> URL: https://issues.apache.org/jira/browse/OOZIE-2371
> Project: Oozie
>  Issue Type: Sub-task
>  Components: docs
>Reporter: Robert Kanter
>Assignee: Daniel Becker
>Priority: Blocker
> Fix For: 5.0.0
>
> Attachments: OOZIE-2371-001.patch
>
>
> We have a list of Workflow Job state transitions here:
> http://oozie.apache.org/docs/4.2.0/WorkflowFunctionalSpec.html#a9_Workflow_Jobs_Lifecycle
> A list of Coordinator Job state transitions here:
> http://oozie.apache.org/docs/4.2.0/CoordinatorFunctionalSpec.html#a6.1.2._Coordinator_Job
> And a list of Coordinator Action state transitions here:
> http://oozie.apache.org/docs/4.2.0/CoordinatorFunctionalSpec.html#a6.1.3.2._Coordinator_Action_Status
> We don't have anything similar for Workflow Actions.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Build failed in Jenkins: oozie-trunk-precommit-build #3948

[Apache Jenkins Server]

See 


--
[...truncated 349.84 KB...]
[INFO] Copying ivy-2.4.0.jar to 

[INFO] Copying jackson-core-asl-1.9.13.jar to 

[INFO] Copying commons-jexl-2.1.1.jar to 

[INFO] Copying oozie-sharelib-hcatalog-5.0.0-SNAPSHOT.jar to 

[INFO] Copying hadoop-mapreduce-client-jobclient-2.6.0.jar to 

[INFO] Copying gson-2.7.jar to 

[INFO] Copying oozie-core-5.0.0-SNAPSHOT.jar to 

[INFO] Copying commons-pool-1.5.4.jar to 

[INFO] Copying htrace-core-3.0.4.jar to 

[INFO] Copying jaxb-impl-2.2.3-1.jar to 

[INFO] Copying jets3t-0.9.0.jar to 

[INFO] Copying datanucleus-rdbms-3.2.9.jar to 

[INFO] Copying hive-common-1.2.0.jar to 

[INFO] Copying snappy-java-1.0.4.1.jar to 

[INFO] Copying javax.annotation-api-1.2.jar to 

[INFO] Copying commons-compiler-2.7.6.jar to 

[INFO] Copying ant-1.9.1.jar to 

[INFO] Copying hadoop-annotations-2.6.0.jar to 

[INFO] Copying janino-2.7.6.jar to 

[INFO] Copying activemq-client-5.13.3.jar to 

[INFO] Copying guice-3.0.jar to 

[INFO] Copying httpclient-4.3.6.jar to 

[INFO] Copying xz-1.0.jar to 

[INFO] Copying oozie-client-5.0.0-SNAPSHOT.jar to 

[INFO] Copying hadoop-mapreduce-client-common-2.6.0.jar to 

[INFO] Copying leveldbjni-all-1.8.jar to 

[INFO] Copying jetty-util-6.1.26.jar to 

[INFO] Copying velocity-1.5.jar to 

[INFO] Copying jackson-core-2.4.2.jar to 

[INFO] Copying apache-log4j-extras-1.2.17.jar to 

[INFO] Copying netty-3.7.0.Final.jar to 

[INFO] Copying 

Re: Review Request 60544: OOZIE-2854 Oozie should handle transient database problems

[András Piros]

> On July 4, 2017, 1:16 p.m., Attila Sasvari wrote:
> > core/pom.xml
> > Lines 39 (patched)
> > 
> >
> > Do not add this dependency twice:
> > 
> > https://github.com/apache/oozie/blob/378f294c2cd92ac0d505201857f03746ce2e58ac/core/pom.xml#L743

Setting it to `compile` scope because we need `FailingMySQLDriverWrapper` to 
work.


> On July 4, 2017, 1:16 p.m., Attila Sasvari wrote:
> > core/src/main/java/org/apache/oozie/service/JPAService.java
> > Line 413 (original), 496 (patched)
> > 
> >
> > Use ``&& !updateQueryList.isEmpty()`` instead of size() >0

Using `CollectionUtils.isNotEmpty()`.


> On July 4, 2017, 1:16 p.m., Attila Sasvari wrote:
> > core/src/main/java/org/apache/oozie/sla/SLASummaryBean.java
> > Lines 71 (patched)
> > 
> >
> > This a new named query that it is only used in ``XTestCase``, and there 
> > is no check is performed on it in any tests; it is is used in 
> > ``cleanUpDBTablesInternal()``. In other words, no REST API requests are 
> > using it for any tasks.
> > 
> > As I see there are other named queries that are not used for meaningful 
> > purposes (i.e. they are only used in tests). It would be nice to go through 
> > all named queries and eliminate the "dead" ones. For example "GET_ACTIONS" 
> > is only used in tests.

Good idea, there is a lot more to do when speaking of constans / named queries 
/ other JPA primitives. Will fire up a separate JIRA.


> On July 4, 2017, 1:16 p.m., Attila Sasvari wrote:
> > core/src/main/java/org/apache/oozie/util/db/BasicDataSourceWrapper.java
> > Lines 41 (patched)
> > 
> >
> > If driverClassName is null, we can return earlier throwing a 
> > SQLNestedException with appropriate message. This way it will not be needed 
> > to check a second time driverClassName is null.

I would keep the fix in line w/ [the original DBCP 
one](https://github.com/apache/commons-dbcp/blob/DBCP_1_4_x_BRANCH/src/java/org/apache/commons/dbcp/BasicDataSource.java#L1588-L1660).


> On July 4, 2017, 1:16 p.m., Attila Sasvari wrote:
> > core/src/main/java/org/apache/oozie/util/db/BasicDataSourceWrapper.java
> > Lines 42 (patched)
> > 
> >
> > 1 try with multiple catch block should be enough

I would keep the fix in line w/ [the original DBCP 
one](https://github.com/apache/commons-dbcp/blob/DBCP_1_4_x_BRANCH/src/java/org/apache/commons/dbcp/BasicDataSource.java#L1588-L1660).


> On July 4, 2017, 1:16 p.m., Attila Sasvari wrote:
> > core/src/main/java/org/apache/oozie/util/db/FailingConnectionWrapper.java
> > Lines 347 (patched)
> > 
> >
> > nit: OozieDBCLI also uses the tablenames. We could extract these into a 
> > central place perhaps in a separate Jira?

Good idea, there is a lot more to do when speaking of constans / named queries 
/ other JPA primitives. Will fire up a separate JIRA.


> On July 4, 2017, 1:16 p.m., Attila Sasvari wrote:
> > core/src/main/java/org/apache/oozie/util/db/OperationRetryHandler.java
> > Lines 31 (patched)
> > 
> >
> > No need for this annotation; if  RETRY_ATTEMPT_COUNTER is package 
> > private, then TestOperationRetryHandler will access it. Or you can restrict 
> > visibility of RETRY_ATTEMPT_COUNTER to private.

Excerpt from the Javadoc of the class 
`com.google.common.annotations.VisibleForTesting`:
```
An annotation that indicates that the visibility of a type or member has been 
relaxed to make the code testable.
```

So the field `OperationRetryHandler.NESTED_RETRY_ATTEMPT_COUNTER` would be 
private, but for sake of testability its visibility is relaxed to package 
protected - hence the annotation.


> On July 4, 2017, 1:16 p.m., Attila Sasvari wrote:
> > minitest/src/test/java/org/apache/oozie/test/WorkflowTest.java
> > Lines 112 (patched)
> > 
> >
> > this is not necessary

>From an IDE(A) perspective, it's good to see why your test actually breaks. 
>And this `assumeFalse()` makes the root cause visible. From Maven, yes, 
>`JAVA_HOME` can be considered as always set, so no issues there.

Without adding the additional workflow property having `JAVA_HOME` as value, 
the test will fail in an IDE(A):
```
junit.framework.AssertionFailedError: 
Expected :SUCCEEDED
Actual   :RUNNING
 


at junit.framework.Assert.fail(Assert.java:57)
at junit.framework.Assert.failNotEquals(Assert.java:329)
at junit.framework.Assert.assertEquals(Assert.java:78)
at 

Re: Review Request 60544: OOZIE-2854 Oozie should handle transient database problems

[András Piros]

> On July 4, 2017, 1:02 p.m., Peter Bacsko wrote:
> > core/src/main/java/org/apache/oozie/service/JPAService.java
> > Line 272 (original), 329 (patched)
> > 
> >
> > I really don't like this code! It's been here for a long time, but it's 
> > very smelly. 
> > 
> > Should we remove this? Could pls check if we have tests that take 
> > advantage of this fault injection>

Unfortunately there are lots of tests based on `SkipCommitFaultInjection` 
system property:
```
TestActionFailover
TestBatchQueryExecutor
TestBundleJobsDeleteJPAExecutor
TestCoordActionsDeleteJPAExecutor
TestCoordJobsDeleteJPAExecutor
TestWorkflowJobsDeleteJPAExecutor
TestSLACalculationJPAExecutor
```
so I'm letting it stay :-(


> On July 4, 2017, 1:02 p.m., Peter Bacsko wrote:
> > core/src/main/java/org/apache/oozie/service/JPAService.java
> > Line 334 (original), 400 (patched)
> > 
> >
> > Check this injection usage

Unfortunately there are lots of tests based on `SkipCommitFaultInjection` 
system property:
```
TestActionFailover
TestBatchQueryExecutor
TestBundleJobsDeleteJPAExecutor
TestCoordActionsDeleteJPAExecutor
TestCoordJobsDeleteJPAExecutor
TestWorkflowJobsDeleteJPAExecutor
TestSLACalculationJPAExecutor
```
so I'm letting it stay :-(


> On July 4, 2017, 1:02 p.m., Peter Bacsko wrote:
> > core/src/main/java/org/apache/oozie/service/JPAService.java
> > Line 432 (original), 515 (patched)
> > 
> >
> > Just like above - check if this is worth keeping

Unfortunately there are lots of tests based on `SkipCommitFaultInjection` 
system property:
```
TestActionFailover
TestBatchQueryExecutor
TestBundleJobsDeleteJPAExecutor
TestCoordActionsDeleteJPAExecutor
TestCoordJobsDeleteJPAExecutor
TestWorkflowJobsDeleteJPAExecutor
TestSLACalculationJPAExecutor
```
so I'm letting it stay :-(


> On July 4, 2017, 1:02 p.m., Peter Bacsko wrote:
> > core/src/main/java/org/apache/oozie/util/db/RetryAttemptCounter.java
> > Lines 28 (patched)
> > 
> >
> > Let's rewrite this explanation and discuss f2f.
> 
> Peter Bacsko wrote:
> My idea:
> 
> "This class tracks nested OperationRetryHandler calls. Some JPAExecutor 
> implementations call other JPAExecutors. This results in two (or possibly 
> more) OperationRetryHandler.executeWithRetry() calls. If the innermost retry 
> handler has exhausted all attempts and re-throws the exception, then the 
> outer handler catches it and would re-start the JPA operation again. In order 
> to avoid this, RetryHandlers must communicate with each other on the same 
> thread by incrementing/decrementing the nesting level and signalling whether 
> the maximum number of attempts have been reached. 
> 
> We use thread locals because RetryHandlers might be called from different 
> threads in parallel. If the nesting level is 0, it's important to reset the 
> "retryAttemptsExhausted" back to false since this variable is re-used in the 
> thread pool."

Thanks, incorporated the class javadocs.


> On July 4, 2017, 1:02 p.m., Peter Bacsko wrote:
> > minitest/src/test/java/org/apache/oozie/test/TestParallelJPAOperationRetries.java
> > Lines 80 (patched)
> > 
> >
> > This assertion will not be evaluated by JUnit because it's called on a 
> > different thread. Instead modify a volatile boolean flag that something 
> > wasn't quite right.

Using a `private volatile Exception` here.


- András


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60544/#review179570
---


On July 3, 2017, 3:15 p.m., András Piros wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60544/
> ---
> 
> (Updated July 3, 2017, 3:15 p.m.)
> 
> 
> Review request for oozie, Attila Sasvari, Peter Cseh, and Peter Bacsko.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2854
> 
> 
> Diffs
> -
> 
>   core/pom.xml acddf349a89cf09a7fc4f384ebcaec56dfd0ab48 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/JsonBeanPersisterExecutor.java
>  PRE-CREATION 
>   core/src/main/java/org/apache/oozie/executor/jpa/QueryExecutor.java 
> 8d94c23e40d1281864db40e141b200ca207a6324 
>   core/src/main/java/org/apache/oozie/service/JPAService.java 
> 028381d3b72bcc3b8c2cd27cacb3e0ac6d48d146 
>   core/src/main/java/org/apache/oozie/sla/SLASummaryBean.java 
> 

[jira] [Commented] (OOZIE-2978) Remove code that handles Pig versions before 0.8

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16074715#comment-16074715
 ] 

Hadoop QA commented on OOZIE-2978:
--

Testing JIRA OOZIE-2978

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
.{color:red}WARNING{color}: the current HEAD has 1 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1964
.Tests rerun: 64
.Tests failed at first run: 
org.apache.oozie.action.hadoop.TestJavaActionExecutor,org.apache.oozie.action.hadoop.TestLauncherAM,
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/oozie-trunk-precommit-build/3947/

> Remove code that handles Pig versions before 0.8 
> -
>
> Key: OOZIE-2978
> URL: https://issues.apache.org/jira/browse/OOZIE-2978
> Project: Oozie
>  Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2978-01.patch, OOZIE-2978-02.patch, 
> OOZIE-2978-03.patch
>
>
> Oozie submits Pig jobs according to given Pig version. For example, Pig 
> versions before 0.8 are treated in the following way:
>  
> [Main.main(args)|https://github.com/apache/oozie/blob/678f1b2b2cc9a2456c7e3d9f693d2637f1f03257/sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java#L382]
> However, we dropped support for *Hadoop 1*, *Hadoop 0.23* support via 
> {{OOZIE-2316}}. I wonder whether submitting jobs using Pig version before 0.8 
> will work at all:
> - 
> https://pig.apache.org/releases.html#13+May%2C+2010%3A+release+0.7.0+available
>  says "This release works with Hadoop 0.20". .
> - Looking at [PIG-2125|https://issues.apache.org/jira/browse/PIG-2125], I 
> believe the answer is NO. 
> Therefore we can safely delete corresponding parts from the codebase. There 
> is also another class {{PigMainWithOldAPI}} that is only used by tests in 
> {{TestPigMainWithOldAPI}}. So we can remove these too.
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Build failed in Jenkins: oozie-trunk-precommit-build #3947

[Apache Jenkins Server]

See 


--
[...truncated 355.14 KB...]
[INFO] Copying jackson-core-asl-1.9.13.jar to 

[INFO] Copying commons-jexl-2.1.1.jar to 

[INFO] Copying oozie-sharelib-hcatalog-5.0.0-SNAPSHOT.jar to 

[INFO] Copying hadoop-mapreduce-client-jobclient-2.6.0.jar to 

[INFO] Copying gson-2.7.jar to 

[INFO] Copying oozie-core-5.0.0-SNAPSHOT.jar to 

[INFO] Copying commons-pool-1.5.4.jar to 

[INFO] Copying htrace-core-3.0.4.jar to 

[INFO] Copying jaxb-impl-2.2.3-1.jar to 

[INFO] Copying jets3t-0.9.0.jar to 

[INFO] Copying datanucleus-rdbms-3.2.9.jar to 

[INFO] Copying hive-common-1.2.0.jar to 

[INFO] Copying snappy-java-1.0.4.1.jar to 

[INFO] Copying javax.annotation-api-1.2.jar to 

[INFO] Copying commons-compiler-2.7.6.jar to 

[INFO] Copying ant-1.9.1.jar to 

[INFO] Copying hadoop-annotations-2.6.0.jar to 

[INFO] Copying janino-2.7.6.jar to 

[INFO] Copying activemq-client-5.13.3.jar to 

[INFO] Copying guice-3.0.jar to 

[INFO] Copying httpclient-4.3.6.jar to 

[INFO] Copying xz-1.0.jar to 

[INFO] Copying oozie-client-5.0.0-SNAPSHOT.jar to 

[INFO] Copying hadoop-mapreduce-client-common-2.6.0.jar to 

[INFO] Copying leveldbjni-all-1.8.jar to 

[INFO] Copying jetty-util-6.1.26.jar to 

[INFO] Copying velocity-1.5.jar to 

[INFO] Copying jackson-core-2.4.2.jar to 

[INFO] Copying apache-log4j-extras-1.2.17.jar to 

[INFO] Copying netty-3.7.0.Final.jar to 

[INFO] Copying xmlenc-0.52.jar to 

[INFO] 

Re: Review Request 60544: OOZIE-2854 Oozie should handle transient database problems

[Peter Bacsko]

> On júl. 4, 2017, 1:02 du, Peter Bacsko wrote:
> > core/src/main/java/org/apache/oozie/util/db/RetryAttemptCounter.java
> > Lines 28 (patched)
> > 
> >
> > Let's rewrite this explanation and discuss f2f.

My idea:

"This class tracks nested OperationRetryHandler calls. Some JPAExecutor 
implementations call other JPAExecutors. This results in two (or possibly more) 
OperationRetryHandler.executeWithRetry() calls. If the innermost retry handler 
has exhausted all attempts and re-throws the exception, then the outer handler 
catches it and would re-start the JPA operation again. In order to avoid this, 
RetryHandlers must communicate with each other on the same thread by 
incrementing/decrementing the nesting level and signalling whether the maximum 
number of attempts have been reached. 

We use thread locals because RetryHandlers might be called from different 
threads in parallel. If the nesting level is 0, it's important to reset the 
"retryAttemptsExhausted" back to false since this variable is re-used in the 
thread pool."


- Peter


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60544/#review179570
---


On júl. 3, 2017, 3:15 du, András Piros wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60544/
> ---
> 
> (Updated júl. 3, 2017, 3:15 du)
> 
> 
> Review request for oozie, Attila Sasvari, Peter Cseh, and Peter Bacsko.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2854
> 
> 
> Diffs
> -
> 
>   core/pom.xml acddf349a89cf09a7fc4f384ebcaec56dfd0ab48 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/JsonBeanPersisterExecutor.java
>  PRE-CREATION 
>   core/src/main/java/org/apache/oozie/executor/jpa/QueryExecutor.java 
> 8d94c23e40d1281864db40e141b200ca207a6324 
>   core/src/main/java/org/apache/oozie/service/JPAService.java 
> 028381d3b72bcc3b8c2cd27cacb3e0ac6d48d146 
>   core/src/main/java/org/apache/oozie/sla/SLASummaryBean.java 
> cfe1522a4b1f89085eb29e7f1281c2abd631bdc2 
>   core/src/main/java/org/apache/oozie/store/WorkflowStore.java 
> c565e74893b863caef6c93015cfe38fe520d04ec 
>   core/src/main/java/org/apache/oozie/util/db/BasicDataSourceWrapper.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/db/DatabaseRetryPredicate.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/db/FailingConnectionWrapper.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/db/FailingHSQLDBDriverWrapper.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/db/FailingMySQLDriverWrapper.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/db/OperationRetryHandler.java 
> PRE-CREATION 
>   
> core/src/main/java/org/apache/oozie/util/db/PersistenceExceptionSubclassFilterRetryPredicate.java
>  PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/db/RetryAttemptCounter.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/db/RuntimeExceptionInjector.java 
> PRE-CREATION 
>   core/src/main/resources/META-INF/persistence.xml 
> bad9278597fcd4f93b4cc482afae8af14beaa922 
>   core/src/main/resources/oozie-default.xml 
> c60a4581a84d4c67a1ac1cf3dfdc252b85ccd01c 
>   core/src/main/resources/oozie-log4j.properties 
> c065f3cd4c5a3df1308b69d7c16e8fcfa8796efc 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java 
> 161927ac8f1132b3080d2924844826fcc7b807a5 
>   
> core/src/test/java/org/apache/oozie/util/db/TestOozieDmlStatementPredicate.java
>  PRE-CREATION 
>   core/src/test/java/org/apache/oozie/util/db/TestOperationRetryHandler.java 
> PRE-CREATION 
>   
> core/src/test/java/org/apache/oozie/util/db/TestPersistenceExceptionSubclassFilterRetryPredicate.java
>  PRE-CREATION 
>   core/src/test/java/org/apache/oozie/util/db/TestRetryAttemptCounter.java 
> PRE-CREATION 
>   minitest/pom.xml 9515284bb5f32c279a93161c10e6571680e4f9fc 
>   
> minitest/src/test/java/org/apache/oozie/test/TestParallelJPAOperationRetries.java
>  PRE-CREATION 
>   minitest/src/test/java/org/apache/oozie/test/TestWorkflowRetries.java 
> PRE-CREATION 
>   minitest/src/test/java/org/apache/oozie/test/WorkflowTest.java 
> 2845f0af6efb9ef75fdbfcb326115c62e6fb3bdd 
>   minitest/src/test/resources/hsqldb-oozie-site.xml 
> fa5fe9c3185e973e8247d7bf10b126119d9c02c9 
>   minitest/src/test/resources/oozie-log4j.properties 
> c142d725140930bfa89cd2b163d0768a4c3a750a 
>   minitest/src/test/resources/parallel-fs-and-shell.xml PRE-CREATION 
>   minitest/src/test/resources/wf-test.xml 
> 20c4946862039a65c76ed7f49991345e90a694de 
>   pom.xml 16c5137d44d7db891da46f80adb51c85e4c1b214 
> 
> 
> Diff: 

[jira] [Comment Edited] (OOZIE-2371) Add docs for state transitions for WF Action states

[Daniel Becker (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16074572#comment-16074572
 ] 

Daniel Becker edited comment on OOZIE-2371 at 7/5/17 11:59 AM:
---

Hi!

I've added some documentation for workflow action state transitions.

However, there are some things that I'm not sure about, any help would be 
appreciated. [~rkanter] can you please address those?

I'm not sure what the state {{USER_RETRY}} means. It would be logical if the 
action entered this state when a user manually retries the action, for example 
after {{START_MANUAL}}, but from the code it seems that this state is entered 
automatically in the {{ActionXCommand#handleUserRetry}} method, which is called 
from {{ActionXCommand#handleError}}, for example when running the test 
{{TestWorkflowActionRetryInfoXCommand#testRetryConsoleUrl}}.

Another thing is what final states are accessible from {{END_RETRY}} and 
{{END_MANUAL}}. In the test {{TestActionErrors#testEndNonTransient}}, the 
action transitions from {{END_MANUAL}} to {{OK}} at the end, but this is a 
'test' action and the {{ActionExecutor}} is a {{ForTestingActionExecutor}}, 
which is not a "real", used-in-production {{ActionExecutor}}. The executor 
calls {{setEndData}} on the context which calls {{setEndData}} on the 
{{WorkflowActionBean}} instance that sets the status.
I decided to include the transition in the documentation because other 
{{ActionExecutor}} s' end methods do mainly the same thing (for example 
{{JavaActionExecutor}}), that is, set the action status according to the 
{{externalStatus}} string.
For the {{END_RETRY}} state, I wasn't able to find an instance of transition to 
a final state.

Thanks a lot,
Daniel


was (Author: daniel.becker):
Hi!

I've added some documentation for workflow action state transitions.
However, there are some things that I'm not sure about, any help would be 
appreciated.

I'm not sure what the state USER_RETRY means. It would be logical if the action 
entered this state when a user manually retries the action, for example after 
START_MANUAL, but from the code it seems that this state is entered 
automatically in the ActionXCommand#handleUserRetry method, which is called 
from ActionXCommand#handleError, for example when running the test 
TestWorkflowActionRetryInfoXCommand#testRetryConsoleUrl.

Another thing is what final states are accessible from END_RETRY and 
END_MANUAL. In the test TestActionErrors#testEndNonTransient, the action 
transitions from END_MANUAL to OK at the end, but this is a 'test' action and 
the ActionExecutor is a ForTestingActionExecutor, which is not a "real", 
used-in-production ActionExecutor. The executor calls setEndData on the context 
which calls setEndData on the WorkflowActionBean instance that sets the status.
I decided to include the transition in the documentation because other 
ActionExecutors' end methods do mainly the same thing (for example 
JavaActionExecutor), that is, set the action status according to the 
externalStatus string.
For the END_RETRY state, I wasn't able to find an instance of transition to a 
final state.

Thanks a lot,
Daniel

> Add docs for state transitions for WF Action states
> ---
>
> Key: OOZIE-2371
> URL: https://issues.apache.org/jira/browse/OOZIE-2371
> Project: Oozie
>  Issue Type: Sub-task
>  Components: docs
>Reporter: Robert Kanter
>Assignee: Daniel Becker
>Priority: Blocker
> Fix For: 5.0.0
>
> Attachments: OOZIE-2371-001.patch
>
>
> We have a list of Workflow Job state transitions here:
> http://oozie.apache.org/docs/4.2.0/WorkflowFunctionalSpec.html#a9_Workflow_Jobs_Lifecycle
> A list of Coordinator Job state transitions here:
> http://oozie.apache.org/docs/4.2.0/CoordinatorFunctionalSpec.html#a6.1.2._Coordinator_Job
> And a list of Coordinator Action state transitions here:
> http://oozie.apache.org/docs/4.2.0/CoordinatorFunctionalSpec.html#a6.1.3.2._Coordinator_Action_Status
> We don't have anything similar for Workflow Actions.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-2978) Remove code that handles Pig versions before 0.8

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16074608#comment-16074608
 ] 

Hadoop QA commented on OOZIE-2978:
--

Testing JIRA OOZIE-2978

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
.{color:red}WARNING{color}: the current HEAD has 1 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1965
.Tests rerun: 3
.Tests failed at first run: 
org.apache.oozie.action.hadoop.TestHdfsOperations,
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/oozie-trunk-precommit-build/3946/

> Remove code that handles Pig versions before 0.8 
> -
>
> Key: OOZIE-2978
> URL: https://issues.apache.org/jira/browse/OOZIE-2978
> Project: Oozie
>  Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2978-01.patch, OOZIE-2978-02.patch, 
> OOZIE-2978-03.patch
>
>
> Oozie submits Pig jobs according to given Pig version. For example, Pig 
> versions before 0.8 are treated in the following way:
>  
> [Main.main(args)|https://github.com/apache/oozie/blob/678f1b2b2cc9a2456c7e3d9f693d2637f1f03257/sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java#L382]
> However, we dropped support for *Hadoop 1*, *Hadoop 0.23* support via 
> {{OOZIE-2316}}. I wonder whether submitting jobs using Pig version before 0.8 
> will work at all:
> - 
> https://pig.apache.org/releases.html#13+May%2C+2010%3A+release+0.7.0+available
>  says "This release works with Hadoop 0.20". .
> - Looking at [PIG-2125|https://issues.apache.org/jira/browse/PIG-2125], I 
> believe the answer is NO. 
> Therefore we can safely delete corresponding parts from the codebase. There 
> is also another class {{PigMainWithOldAPI}} that is only used by tests in 
> {{TestPigMainWithOldAPI}}. So we can remove these too.
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Build failed in Jenkins: oozie-trunk-precommit-build #3946

[Apache Jenkins Server]

See 


Changes:

[pbacsko] OOZIE-2918 Delete LauncherMapper and its test (asasvari via pbacsko)

--
[...truncated 351.88 KB...]
[INFO] Copying ivy-2.4.0.jar to 

[INFO] Copying jackson-core-asl-1.9.13.jar to 

[INFO] Copying commons-jexl-2.1.1.jar to 

[INFO] Copying oozie-sharelib-hcatalog-5.0.0-SNAPSHOT.jar to 

[INFO] Copying hadoop-mapreduce-client-jobclient-2.6.0.jar to 

[INFO] Copying gson-2.7.jar to 

[INFO] Copying oozie-core-5.0.0-SNAPSHOT.jar to 

[INFO] Copying commons-pool-1.5.4.jar to 

[INFO] Copying htrace-core-3.0.4.jar to 

[INFO] Copying jaxb-impl-2.2.3-1.jar to 

[INFO] Copying jets3t-0.9.0.jar to 

[INFO] Copying datanucleus-rdbms-3.2.9.jar to 

[INFO] Copying hive-common-1.2.0.jar to 

[INFO] Copying snappy-java-1.0.4.1.jar to 

[INFO] Copying javax.annotation-api-1.2.jar to 

[INFO] Copying commons-compiler-2.7.6.jar to 

[INFO] Copying ant-1.9.1.jar to 

[INFO] Copying hadoop-annotations-2.6.0.jar to 

[INFO] Copying janino-2.7.6.jar to 

[INFO] Copying activemq-client-5.13.3.jar to 

[INFO] Copying guice-3.0.jar to 

[INFO] Copying httpclient-4.3.6.jar to 

[INFO] Copying xz-1.0.jar to 

[INFO] Copying oozie-client-5.0.0-SNAPSHOT.jar to 

[INFO] Copying hadoop-mapreduce-client-common-2.6.0.jar to 

[INFO] Copying leveldbjni-all-1.8.jar to 

[INFO] Copying jetty-util-6.1.26.jar to 

[INFO] Copying velocity-1.5.jar to 

[INFO] Copying jackson-core-2.4.2.jar to 

[INFO] Copying apache-log4j-extras-1.2.17.jar to 

[INFO] Copying netty-3.7.0.Final.jar to 

[jira] [Updated] (OOZIE-2371) Add docs for state transitions for WF Action states

[Daniel Becker (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Becker updated OOZIE-2371:
-
Attachment: OOZIE-2371-001.patch

Hi!

I've added some documentation for workflow action state transitions.
However, there are some things that I'm not sure about, any help would be 
appreciated.

I'm not sure what the state USER_RETRY means. It would be logical if the action 
entered this state when a user manually retries the action, for example after 
START_MANUAL, but from the code it seems that this state is entered 
automatically in the ActionXCommand#handleUserRetry method, which is called 
from ActionXCommand#handleError, for example when running the test 
TestWorkflowActionRetryInfoXCommand#testRetryConsoleUrl.

Another thing is what final states are accessible from END_RETRY and 
END_MANUAL. In the test TestActionErrors#testEndNonTransient, the action 
transitions from END_MANUAL to OK at the end, but this is a 'test' action and 
the ActionExecutor is a ForTestingActionExecutor, which is not a "real", 
used-in-production ActionExecutor. The executor calls setEndData on the context 
which calls setEndData on the WorkflowActionBean instance that sets the status.
I decided to include the transition in the documentation because other 
ActionExecutors' end methods do mainly the same thing (for example 
JavaActionExecutor), that is, set the action status according to the 
externalStatus string.
For the END_RETRY state, I wasn't able to find an instance of transition to a 
final state.

Thanks a lot,
Daniel

> Add docs for state transitions for WF Action states
> ---
>
> Key: OOZIE-2371
> URL: https://issues.apache.org/jira/browse/OOZIE-2371
> Project: Oozie
>  Issue Type: Sub-task
>  Components: docs
>Reporter: Robert Kanter
>Assignee: Daniel Becker
>Priority: Blocker
> Fix For: 5.0.0
>
> Attachments: OOZIE-2371-001.patch
>
>
> We have a list of Workflow Job state transitions here:
> http://oozie.apache.org/docs/4.2.0/WorkflowFunctionalSpec.html#a9_Workflow_Jobs_Lifecycle
> A list of Coordinator Job state transitions here:
> http://oozie.apache.org/docs/4.2.0/CoordinatorFunctionalSpec.html#a6.1.2._Coordinator_Job
> And a list of Coordinator Action state transitions here:
> http://oozie.apache.org/docs/4.2.0/CoordinatorFunctionalSpec.html#a6.1.3.2._Coordinator_Action_Status
> We don't have anything similar for Workflow Actions.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (OOZIE-2978) Remove code that handles Pig versions before 0.8

[Attila Sasvari (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Sasvari updated OOZIE-2978:
--
Attachment: OOZIE-2978-03.patch

[~pbacsko] thanks for the review! Please find new patch attached!

> Remove code that handles Pig versions before 0.8 
> -
>
> Key: OOZIE-2978
> URL: https://issues.apache.org/jira/browse/OOZIE-2978
> Project: Oozie
>  Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2978-01.patch, OOZIE-2978-02.patch, 
> OOZIE-2978-03.patch
>
>
> Oozie submits Pig jobs according to given Pig version. For example, Pig 
> versions before 0.8 are treated in the following way:
>  
> [Main.main(args)|https://github.com/apache/oozie/blob/678f1b2b2cc9a2456c7e3d9f693d2637f1f03257/sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java#L382]
> However, we dropped support for *Hadoop 1*, *Hadoop 0.23* support via 
> {{OOZIE-2316}}. I wonder whether submitting jobs using Pig version before 0.8 
> will work at all:
> - 
> https://pig.apache.org/releases.html#13+May%2C+2010%3A+release+0.7.0+available
>  says "This release works with Hadoop 0.20". .
> - Looking at [PIG-2125|https://issues.apache.org/jira/browse/PIG-2125], I 
> believe the answer is NO. 
> Therefore we can safely delete corresponding parts from the codebase. There 
> is also another class {{PigMainWithOldAPI}} that is only used by tests in 
> {{TestPigMainWithOldAPI}}. So we can remove these too.
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (OOZIE-2978) Remove code that handles Pig versions before 0.8

[Peter Bacsko (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16074538#comment-16074538
 ] 

Peter Bacsko edited comment on OOZIE-2978 at 7/5/17 10:35 AM:
--

1. Please remove the {{Class.forName()}} part which checks for {{PigRunner}}
2. {{PigMainWithOldApi.java}} (and related tests) should be deleted


was (Author: pbacsko):
1. Please remove the {{Class.forName()}} part which checks for {{PigRunner}}
2. {{PigMainWithOldApi. java}} (and related tests) should be deleted

> Remove code that handles Pig versions before 0.8 
> -
>
> Key: OOZIE-2978
> URL: https://issues.apache.org/jira/browse/OOZIE-2978
> Project: Oozie
>  Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2978-01.patch, OOZIE-2978-02.patch
>
>
> Oozie submits Pig jobs according to given Pig version. For example, Pig 
> versions before 0.8 are treated in the following way:
>  
> [Main.main(args)|https://github.com/apache/oozie/blob/678f1b2b2cc9a2456c7e3d9f693d2637f1f03257/sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java#L382]
> However, we dropped support for *Hadoop 1*, *Hadoop 0.23* support via 
> {{OOZIE-2316}}. I wonder whether submitting jobs using Pig version before 0.8 
> will work at all:
> - 
> https://pig.apache.org/releases.html#13+May%2C+2010%3A+release+0.7.0+available
>  says "This release works with Hadoop 0.20". .
> - Looking at [PIG-2125|https://issues.apache.org/jira/browse/PIG-2125], I 
> believe the answer is NO. 
> Therefore we can safely delete corresponding parts from the codebase. There 
> is also another class {{PigMainWithOldAPI}} that is only used by tests in 
> {{TestPigMainWithOldAPI}}. So we can remove these too.
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-2978) Remove code that handles Pig versions before 0.8

[Peter Bacsko (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16074538#comment-16074538
 ] 

Peter Bacsko commented on OOZIE-2978:
-

1. Please remove the Class.forName() part which checks for PigRunner
2. PigMainWithOldApi. java (and related tests) should be deleted

> Remove code that handles Pig versions before 0.8 
> -
>
> Key: OOZIE-2978
> URL: https://issues.apache.org/jira/browse/OOZIE-2978
> Project: Oozie
>  Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2978-01.patch, OOZIE-2978-02.patch
>
>
> Oozie submits Pig jobs according to given Pig version. For example, Pig 
> versions before 0.8 are treated in the following way:
>  
> [Main.main(args)|https://github.com/apache/oozie/blob/678f1b2b2cc9a2456c7e3d9f693d2637f1f03257/sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java#L382]
> However, we dropped support for *Hadoop 1*, *Hadoop 0.23* support via 
> {{OOZIE-2316}}. I wonder whether submitting jobs using Pig version before 0.8 
> will work at all:
> - 
> https://pig.apache.org/releases.html#13+May%2C+2010%3A+release+0.7.0+available
>  says "This release works with Hadoop 0.20". .
> - Looking at [PIG-2125|https://issues.apache.org/jira/browse/PIG-2125], I 
> believe the answer is NO. 
> Therefore we can safely delete corresponding parts from the codebase. There 
> is also another class {{PigMainWithOldAPI}} that is only used by tests in 
> {{TestPigMainWithOldAPI}}. So we can remove these too.
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (OOZIE-2978) Remove code that handles Pig versions before 0.8

[Peter Bacsko (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16074538#comment-16074538
 ] 

Peter Bacsko edited comment on OOZIE-2978 at 7/5/17 10:20 AM:
--

1. Please remove the {{Class.forName()}} part which checks for {{PigRunner}}
2. {{PigMainWithOldApi. java}} (and related tests) should be deleted


was (Author: pbacsko):
1. Please remove the Class.forName() part which checks for PigRunner
2. PigMainWithOldApi. java (and related tests) should be deleted

> Remove code that handles Pig versions before 0.8 
> -
>
> Key: OOZIE-2978
> URL: https://issues.apache.org/jira/browse/OOZIE-2978
> Project: Oozie
>  Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2978-01.patch, OOZIE-2978-02.patch
>
>
> Oozie submits Pig jobs according to given Pig version. For example, Pig 
> versions before 0.8 are treated in the following way:
>  
> [Main.main(args)|https://github.com/apache/oozie/blob/678f1b2b2cc9a2456c7e3d9f693d2637f1f03257/sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java#L382]
> However, we dropped support for *Hadoop 1*, *Hadoop 0.23* support via 
> {{OOZIE-2316}}. I wonder whether submitting jobs using Pig version before 0.8 
> will work at all:
> - 
> https://pig.apache.org/releases.html#13+May%2C+2010%3A+release+0.7.0+available
>  says "This release works with Hadoop 0.20". .
> - Looking at [PIG-2125|https://issues.apache.org/jira/browse/PIG-2125], I 
> believe the answer is NO. 
> Therefore we can safely delete corresponding parts from the codebase. There 
> is also another class {{PigMainWithOldAPI}} that is only used by tests in 
> {{TestPigMainWithOldAPI}}. So we can remove these too.
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-2978) Remove code that handles Pig versions before 0.8

[Attila Sasvari (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16074459#comment-16074459
 ] 

Attila Sasvari commented on OOZIE-2978:
---

Attached rebased patch

> Remove code that handles Pig versions before 0.8 
> -
>
> Key: OOZIE-2978
> URL: https://issues.apache.org/jira/browse/OOZIE-2978
> Project: Oozie
>  Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2978-01.patch, OOZIE-2978-02.patch
>
>
> Oozie submits Pig jobs according to given Pig version. For example, Pig 
> versions before 0.8 are treated in the following way:
>  
> [Main.main(args)|https://github.com/apache/oozie/blob/678f1b2b2cc9a2456c7e3d9f693d2637f1f03257/sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java#L382]
> However, we dropped support for *Hadoop 1*, *Hadoop 0.23* support via 
> {{OOZIE-2316}}. I wonder whether submitting jobs using Pig version before 0.8 
> will work at all:
> - 
> https://pig.apache.org/releases.html#13+May%2C+2010%3A+release+0.7.0+available
>  says "This release works with Hadoop 0.20". .
> - Looking at [PIG-2125|https://issues.apache.org/jira/browse/PIG-2125], I 
> believe the answer is NO. 
> Therefore we can safely delete corresponding parts from the codebase. There 
> is also another class {{PigMainWithOldAPI}} that is only used by tests in 
> {{TestPigMainWithOldAPI}}. So we can remove these too.
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (OOZIE-2978) Remove code that handles Pig versions before 0.8

[Attila Sasvari (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Sasvari updated OOZIE-2978:
--
Attachment: OOZIE-2978-02.patch

> Remove code that handles Pig versions before 0.8 
> -
>
> Key: OOZIE-2978
> URL: https://issues.apache.org/jira/browse/OOZIE-2978
> Project: Oozie
>  Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-2978-01.patch, OOZIE-2978-02.patch
>
>
> Oozie submits Pig jobs according to given Pig version. For example, Pig 
> versions before 0.8 are treated in the following way:
>  
> [Main.main(args)|https://github.com/apache/oozie/blob/678f1b2b2cc9a2456c7e3d9f693d2637f1f03257/sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java#L382]
> However, we dropped support for *Hadoop 1*, *Hadoop 0.23* support via 
> {{OOZIE-2316}}. I wonder whether submitting jobs using Pig version before 0.8 
> will work at all:
> - 
> https://pig.apache.org/releases.html#13+May%2C+2010%3A+release+0.7.0+available
>  says "This release works with Hadoop 0.20". .
> - Looking at [PIG-2125|https://issues.apache.org/jira/browse/PIG-2125], I 
> believe the answer is NO. 
> Therefore we can safely delete corresponding parts from the codebase. There 
> is also another class {{PigMainWithOldAPI}} that is only used by tests in 
> {{TestPigMainWithOldAPI}}. So we can remove these too.
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] Subscription: Oozie Patch Available

[jira]

Issue Subscription
Filter: Oozie Patch Available (109 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-2978  Remove code that handles Pig versions before 0.8 
https://issues.apache.org/jira/browse/OOZIE-2978
OOZIE-2977  error: self-closing element not allowed
https://issues.apache.org/jira/browse/OOZIE-2977
OOZIE-2975  code clean up in pig sharelib, replace Exception with more 
explicit, add try with resources, StringBuilder instead of StringBuffer
https://issues.apache.org/jira/browse/OOZIE-2975
OOZIE-2974  Change TestLiteWorkflowAppParser so that it works with Hadoop 3
https://issues.apache.org/jira/browse/OOZIE-2974
OOZIE-2969  Drop support for Java 1.7
https://issues.apache.org/jira/browse/OOZIE-2969
OOZIE-2964  getting at 
org.apache.maven.plugin.javadoc.AbstractJavadocMojo.executeJavadocCommandLine(AbstractJavadocMojo.java:5163)
https://issues.apache.org/jira/browse/OOZIE-2964
OOZIE-2963  getting error in build ArtifactNotFoundException: Could not find 
artifact org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.5-jhyde
https://issues.apache.org/jira/browse/OOZIE-2963
OOZIE-2962  bump maven-javadoc-plugin to 2.10.4
https://issues.apache.org/jira/browse/OOZIE-2962
OOZIE-2961  build contains multiple warnings for is not a parameter name
https://issues.apache.org/jira/browse/OOZIE-2961
OOZIE-2960  many warnings for @return tag has no arguments
https://issues.apache.org/jira/browse/OOZIE-2960
OOZIE-2957  Documentation states that starting a coordinator is possible
https://issues.apache.org/jira/browse/OOZIE-2957
OOZIE-2956  Fix Findbugs warnings related to reliance on default encoding in 
oozie-core
https://issues.apache.org/jira/browse/OOZIE-2956
OOZIE-2955  Fix Findbugs warnings related to reliance on default encoding in 
oozie-client
https://issues.apache.org/jira/browse/OOZIE-2955
OOZIE-2954  Fix Checkstyle issues in oozie-client
https://issues.apache.org/jira/browse/OOZIE-2954
OOZIE-2953  Fix Checkstyle issues in oozie-tools
https://issues.apache.org/jira/browse/OOZIE-2953
OOZIE-2952  Fix Findbugs warnings in oozie-sharelib-oozie
https://issues.apache.org/jira/browse/OOZIE-2952
OOZIE-2942  Fix Findbugs warnings in oozie-examples
https://issues.apache.org/jira/browse/OOZIE-2942
OOZIE-2937  Remove redundant groupId from the child pom's
https://issues.apache.org/jira/browse/OOZIE-2937
OOZIE-2934  Fix "Exceptional return value of java.io.File.mkdirs() ignored" 
Findbugs error in oozie-sharelib-spark
https://issues.apache.org/jira/browse/OOZIE-2934
OOZIE-2931  multiple warnings during build for "no @param for"
https://issues.apache.org/jira/browse/OOZIE-2931
OOZIE-2927  Append new line character for Hive2 query using query tag
https://issues.apache.org/jira/browse/OOZIE-2927
OOZIE-2916  Set a job name for the MR Action's child job
https://issues.apache.org/jira/browse/OOZIE-2916
OOZIE-2914  Consolidate Trim 
https://issues.apache.org/jira/browse/OOZIE-2914
OOZIE-2911  Re-add test testWfActionKillChildJob and adapt it to OYA
https://issues.apache.org/jira/browse/OOZIE-2911
OOZIE-2883  OOZIE throw the error "Missing 
[oozie.service.ProxyUserService.proxyuser.oozie.service.ProxyUserService.proxyuser.mr.groups]
 property"
https://issues.apache.org/jira/browse/OOZIE-2883
OOZIE-2877  Oozie Git Action
https://issues.apache.org/jira/browse/OOZIE-2877
OOZIE-2867  Timezone handling for Coordinators: emphasize "Continent/City" 
format
https://issues.apache.org/jira/browse/OOZIE-2867
OOZIE-2854  Oozie should handle transient database problems
https://issues.apache.org/jira/browse/OOZIE-2854
OOZIE-2834  ParameterVerifier logging non-useful warning for workflow definition
https://issues.apache.org/jira/browse/OOZIE-2834
OOZIE-2833  when using uber mode the regex pattern used in the 
extractHeapSizeMB method does not allow heap sizes specified in bytes.
https://issues.apache.org/jira/browse/OOZIE-2833
OOZIE-2829  Improve sharelib upload to accept multiple source folders
https://issues.apache.org/jira/browse/OOZIE-2829
OOZIE-2826  Falcon feed fails to aws s3; Oozie joda time version does not meet 
required jar version 2.2 or later
https://issues.apache.org/jira/browse/OOZIE-2826
OOZIE-2812  SparkConfigurationService should support loading configurations 
from multiple Spark versions
https://issues.apache.org/jira/browse/OOZIE-2812
OOZIE-2795  Create lib directory or symlink for Oozie CLI during packaging
https://issues.apache.org/jira/browse/OOZIE-2795
OOZIE-2791  ShareLib installation may fail on busy Hadoop clusters
https://issues.apache.org/jira/browse/OOZIE-2791
OOZIE-2784  Include WEEK as a