[jira] Subscription: Oozie Patch Available

[jira]

Issue Subscription
Filter: Oozie Patch Available (106 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-3148  Rerun Failing Tests through Maven surefire
https://issues.apache.org/jira/browse/OOZIE-3148
OOZIE-3147  Misleading documentation of oozie.service.PurgeService.purge.limit 
configuration property
https://issues.apache.org/jira/browse/OOZIE-3147
OOZIE-3135  Configure log4j2 in SqoopMain
https://issues.apache.org/jira/browse/OOZIE-3135
OOZIE-3121  bump all maven plugins to latest versions
https://issues.apache.org/jira/browse/OOZIE-3121
OOZIE-3113  Retry for ZK lock release
https://issues.apache.org/jira/browse/OOZIE-3113
OOZIE-3105  testJMXInstrumentation from the 
org.apache.oozie.util.TestMetricsInstrumentation class is flaky
https://issues.apache.org/jira/browse/OOZIE-3105
OOZIE-3094  fix for grammar mistake
https://issues.apache.org/jira/browse/OOZIE-3094
OOZIE-3091  Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: 
org/apache/avro/mapred/AvroWrapper"
https://issues.apache.org/jira/browse/OOZIE-3091
OOZIE-3085  Improve logging in ActionExecutors
https://issues.apache.org/jira/browse/OOZIE-3085
OOZIE-3083  Make improved version Info backward compatible
https://issues.apache.org/jira/browse/OOZIE-3083
OOZIE-3071  Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 
than Spark 2.2.0
https://issues.apache.org/jira/browse/OOZIE-3071
OOZIE-3063  Sanitizing variables that are part of openjpa.ConnectionProperties
https://issues.apache.org/jira/browse/OOZIE-3063
OOZIE-3062  Set HADOOP_CONF_DIR for spark action
https://issues.apache.org/jira/browse/OOZIE-3062
OOZIE-3002  address findbugs errors in client lib
https://issues.apache.org/jira/browse/OOZIE-3002
OOZIE-2975  code clean up in pig sharelib, replace Exception with more 
explicit, add try with resources, StringBuilder instead of StringBuffer
https://issues.apache.org/jira/browse/OOZIE-2975
OOZIE-2973  Make sure Oozie works with Hadoop 3 
https://issues.apache.org/jira/browse/OOZIE-2973
OOZIE-2957  Documentation states that starting a coordinator is possible
https://issues.apache.org/jira/browse/OOZIE-2957
OOZIE-2956  Fix Findbugs warnings related to reliance on default encoding in 
oozie-core
https://issues.apache.org/jira/browse/OOZIE-2956
OOZIE-2955  Fix Findbugs warnings related to reliance on default encoding in 
oozie-client
https://issues.apache.org/jira/browse/OOZIE-2955
OOZIE-2954  Fix Checkstyle issues in oozie-client
https://issues.apache.org/jira/browse/OOZIE-2954
OOZIE-2953  Fix Checkstyle issues in oozie-tools
https://issues.apache.org/jira/browse/OOZIE-2953
OOZIE-2952  Fix Findbugs warnings in oozie-sharelib-oozie
https://issues.apache.org/jira/browse/OOZIE-2952
OOZIE-2949  Escape quotes whitespaces in Sqoop  field
https://issues.apache.org/jira/browse/OOZIE-2949
OOZIE-2942  Fix Findbugs warnings in oozie-examples
https://issues.apache.org/jira/browse/OOZIE-2942
OOZIE-2937  Remove redundant groupId from the child pom's
https://issues.apache.org/jira/browse/OOZIE-2937
OOZIE-2934  Fix "Exceptional return value of java.io.File.mkdirs() ignored" 
Findbugs error in oozie-sharelib-spark
https://issues.apache.org/jira/browse/OOZIE-2934
OOZIE-2927  Append new line character for Hive2 query using query tag
https://issues.apache.org/jira/browse/OOZIE-2927
OOZIE-2914  Consolidate Trim 
https://issues.apache.org/jira/browse/OOZIE-2914
OOZIE-2883  OOZIE throw the error "Missing 
[oozie.service.ProxyUserService.proxyuser.oozie.service.ProxyUserService.proxyuser.mr.groups]
 property"
https://issues.apache.org/jira/browse/OOZIE-2883
OOZIE-2877  Oozie Git Action
https://issues.apache.org/jira/browse/OOZIE-2877
OOZIE-2867  Timezone handling for Coordinators: emphasize "Continent/City" 
format
https://issues.apache.org/jira/browse/OOZIE-2867
OOZIE-2834  ParameterVerifier logging non-useful warning for workflow definition
https://issues.apache.org/jira/browse/OOZIE-2834
OOZIE-2833  when using uber mode the regex pattern used in the 
extractHeapSizeMB method does not allow heap sizes specified in bytes.
https://issues.apache.org/jira/browse/OOZIE-2833
OOZIE-2829  Improve sharelib upload to accept multiple source folders
https://issues.apache.org/jira/browse/OOZIE-2829
OOZIE-2826  Falcon feed fails to aws s3; Oozie joda time version does not meet 
required jar version 2.2 or later
https://issues.apache.org/jira/browse/OOZIE-2826
OOZIE-2812  SparkConfigurationService should support loading configurations 
from multiple Spark versions
https://issues.apache.org/jira/browse/OOZIE-2812

[jira] Subscription: Oozie Patch Available

[jira]

Issue Subscription
Filter: Oozie Patch Available (111 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-3129  Fix test TestConfigurationService.testOozieConfig
https://issues-test.apache.org/jira/browse/OOZIE-3129
OOZIE-3127  Remove redundant check for user
https://issues-test.apache.org/jira/browse/OOZIE-3127
OOZIE-3126  Add option to allow list of users to access system config
https://issues-test.apache.org/jira/browse/OOZIE-3126
OOZIE-3121  bump all maven plugins to latest versions
https://issues-test.apache.org/jira/browse/OOZIE-3121
OOZIE-3118  fix for error: self-closing element not allowed
https://issues-test.apache.org/jira/browse/OOZIE-3118
OOZIE-3117  fix for warning: no @throws for
https://issues-test.apache.org/jira/browse/OOZIE-3117
OOZIE-3116  fix for warning: no description for @throws
https://issues-test.apache.org/jira/browse/OOZIE-3116
OOZIE-3115  fix for javadoc warning: empty  tag
https://issues-test.apache.org/jira/browse/OOZIE-3115
OOZIE-3113  Retry for ZK lock release
https://issues-test.apache.org/jira/browse/OOZIE-3113
OOZIE-3112  SparkConfigrationService overwrites properties provided via 
--properties-file option in SparkAction
https://issues-test.apache.org/jira/browse/OOZIE-3112
OOZIE-3105  testJMXInstrumentation from the 
org.apache.oozie.util.TestMetricsInstrumentation class is flaky
https://issues-test.apache.org/jira/browse/OOZIE-3105
OOZIE-3094  fix for grammar mistake
https://issues-test.apache.org/jira/browse/OOZIE-3094
OOZIE-3091  Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: 
org/apache/avro/mapred/AvroWrapper"
https://issues-test.apache.org/jira/browse/OOZIE-3091
OOZIE-3083  Make improved version Info backward compatible
https://issues-test.apache.org/jira/browse/OOZIE-3083
OOZIE-3071  Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 
than Spark 2.2.0
https://issues-test.apache.org/jira/browse/OOZIE-3071
OOZIE-3063  Sanitizing variables that are part of openjpa.ConnectionProperties
https://issues-test.apache.org/jira/browse/OOZIE-3063
OOZIE-3062  Set HADOOP_CONF_DIR for spark action
https://issues-test.apache.org/jira/browse/OOZIE-3062
OOZIE-3002  address findbugs errors in client lib
https://issues-test.apache.org/jira/browse/OOZIE-3002
OOZIE-2975  code clean up in pig sharelib, replace Exception with more 
explicit, add try with resources, StringBuilder instead of StringBuffer
https://issues-test.apache.org/jira/browse/OOZIE-2975
OOZIE-2973  Make sure Oozie works with Hadoop 3 
https://issues-test.apache.org/jira/browse/OOZIE-2973
OOZIE-2969  Drop support for Java 1.7
https://issues-test.apache.org/jira/browse/OOZIE-2969
OOZIE-2957  Documentation states that starting a coordinator is possible
https://issues-test.apache.org/jira/browse/OOZIE-2957
OOZIE-2956  Fix Findbugs warnings related to reliance on default encoding in 
oozie-core
https://issues-test.apache.org/jira/browse/OOZIE-2956
OOZIE-2955  Fix Findbugs warnings related to reliance on default encoding in 
oozie-client
https://issues-test.apache.org/jira/browse/OOZIE-2955
OOZIE-2954  Fix Checkstyle issues in oozie-client
https://issues-test.apache.org/jira/browse/OOZIE-2954
OOZIE-2953  Fix Checkstyle issues in oozie-tools
https://issues-test.apache.org/jira/browse/OOZIE-2953
OOZIE-2952  Fix Findbugs warnings in oozie-sharelib-oozie
https://issues-test.apache.org/jira/browse/OOZIE-2952
OOZIE-2949  Escape quotes whitespaces in Sqoop  field
https://issues-test.apache.org/jira/browse/OOZIE-2949
OOZIE-2942  Fix Findbugs warnings in oozie-examples
https://issues-test.apache.org/jira/browse/OOZIE-2942
OOZIE-2937  Remove redundant groupId from the child pom's
https://issues-test.apache.org/jira/browse/OOZIE-2937
OOZIE-2934  Fix "Exceptional return value of java.io.File.mkdirs() ignored" 
Findbugs error in oozie-sharelib-spark
https://issues-test.apache.org/jira/browse/OOZIE-2934
OOZIE-2927  Append new line character for Hive2 query using query tag
https://issues-test.apache.org/jira/browse/OOZIE-2927
OOZIE-2914  Consolidate Trim 
https://issues-test.apache.org/jira/browse/OOZIE-2914
OOZIE-2883  OOZIE throw the error "Missing 
[oozie.service.ProxyUserService.proxyuser.oozie.service.ProxyUserService.proxyuser.mr.groups]
 property"
https://issues-test.apache.org/jira/browse/OOZIE-2883
OOZIE-2877  Oozie Git Action
https://issues-test.apache.org/jira/browse/OOZIE-2877
OOZIE-2867  Timezone handling for Coordinators: emphasize "Continent/City" 
format
https://issues-test.apache.org/jira/browse/OOZIE-2867
OOZIE-2834  ParameterVerifier logging 

Re: [VOTE] Release Oozie 5.0.0-beta1 (candidate 0)

[Robert Kanter]

Thanks for putting this together.

Another important thing to note in the highlights that as part of moving
the launcher, we also dropped support for Hadoop 1.x and we added new 1.0
schemas.

+1

Here's what I did:
- Verified md5
- Verified signature (gpg)
- Built Oozie against Hadoop 2.7.2
- Ran a few of the examples, clicked around the UI and tried various CLI
commands

Here's some issues I ran into; none are blockers for 5b1:
- Filed OOZIE-3149 to delete work.log, which is really old (not a blocker)
- Filed OOZIE-3150 to update NOTICE.txt (I made this a blocker for 5.0.0)
- The examples are still the older schemas, which is fine for the beta, but
I made OOZIE-2814 a blocker for 5.0.0 because it will encourage users to
use the new schemas

- Robert


On Thu, Dec 28, 2017 at 1:23 PM, Peter Cseh  wrote:

> Hi Attila!
>
> It looks like I've messed up the extraction of the tar.gz and I was looking
> at empty folders :)
> Will validate the release tomorrow.
>
> Sorry for causing confusion!
> gp
>
> On Thu, Dec 28, 2017 at 2:08 PM, Attila Sasvari 
> wrote:
>
> > @Andras: many thanks. Please let me know if you notice anything strange
> > with hashes or run into other problems.
> >
> > @gp: thanks a lot. This is actually what the bin/create-release-artifact
> > script has generated. Comparing it with previous release candidate (4.3.0
> > rc1), file size of oozie-5.0.0-beta1.tar.gz looks correct to me
> >
> >   $ wget
> > https://dist.apache.org/repos/dist/dev/oozie/oozie-4.3.0-
> > rc1/oozie-4.3.0.tar.gz
> >   $ wget
> > http://people.apache.org/\~asasvari/oozie-5.0.0-beta1-
> > rc0/oozie-5.0.0-beta1.tar.gz
> >
> >   $ ls -lrt oozie-*tar.gz
> >   -rw-r--r--  1 asasvari  staff  2392205 Oct 25  2016 oozie-4.3.0.tar.gz
> >   -rw-r--r--  1 asasvari  staff  2539718 Dec 28 00:12
> > oozie-5.0.0-beta1.tar.gz
> >
> > oozie-5.0.0-beta1.tar.gz contains much more files (it contains all the
> > sources):
> >
> >   $  tar tvf oozie-5.0.0-beta1.tar.gz | grep "\.java" | head -5
> >   -rw-r--r--  0 asasvari wheel   10328 Dec 28 00:02
> > oozie-5.0.0-beta1/zookeeper-security-tests/src/test/java/
> > org/apache/oozie/util/TestZKUtilsWithSecurity.java
> >   -rw-r--r--  0 asasvari wheel7357 Dec 28 00:02
> > oozie-5.0.0-beta1/zookeeper-security-tests/src/test/java/
> > org/apache/oozie/test/ZKXTestCaseWithSecurity.java
> >   -rw-r--r--  0 asasvari wheel   32589 Dec 28 00:02
> > oozie-5.0.0-beta1/tools/src/test/java/org/apache/oozie/
> > tools/FakeConnection.java
> >   -rw-r--r--  0 asasvari wheel1921 Dec 28 00:02
> > oozie-5.0.0-beta1/tools/src/test/java/org/apache/oozie/
> > tools/FakeDriver.java
> >   -rw-r--r--  0 asasvari wheel2230 Dec 28 00:02
> > oozie-5.0.0-beta1/tools/src/test/java/org/apache/oozie/
> > tools/LauncherSecurityManager.java
> >
> > If you download the tarball and run bin/mkdistro.sh, it will probably
> fail
> > due to known flaky tests. Running tests with bin/mkdistro.sh
> > -Dsurefire.rerunFailingTestsCount=2, they shall pass (printing things
> > like [WARNING] Tests run: 1841, Failures: 0, Errors: 0, Skipped: 2,
> Flakes:
> > 2 ).
> >
> > - Attila
> >
> > On Thu, Dec 28, 2017 at 10:57 AM, Peter Cseh 
> > wrote:
> >
> > > Thanks Attila!
> > > The tar.gz does not contain the release, it only contains the pom
> files.
> > > The whole thing is 2.4 Mb, an Oozie was bigger the last time I've
> checked
> > > :)
> > >
> > > gp
> > >
> > > On Thu, Dec 28, 2017 at 12:53 AM, Attila Sasvári 
> > > wrote:
> > >
> > > > Hi there,
> > > >
> > > > I have created a build for Oozie 5.0.0-beta1, release candidate 0.
> > > >
> > > > This release introduces major changes:
> > > > - moving launcher from MapReduce AM to Oozie AM,
> > > > - switching from Tomcat to embedded Jetty,
> > > > - completely rewritten graph generator,
> > > > - local sharelib,
> > > > - JDK 8 support.
> > > >
> > > > Keys to verify the signature of the release artifact are available at
> > > >
> > > >  http://www.apache.org/dist/oozie/KEYS
> > > >
> > > > Please download, test, and try it out:
> > > >
> > > >   http://people.apache.org/~asasvari/oozie-5.0.0-beta1-rc0/
> > > >
> > > > The release, md5 signature, gpg signature, and rat report can all be
> > > found
> > > > at the above address.
> > > >
> > > > The vote is open until Thursday, January 04, 2018 at 12:00 CEST.
> > > >
> > > > Thanks,
> > > > Attila Sasvari
> > > >
> > >
> > >
> > >
> > > --
> > > Peter Cseh
> > > Software Engineer
> > > 
> > >
> >
> >
> >
> > --
> > --
> > Attila Sasvari
> > Software Engineer
> > 
> >
>
>
>
> --
> Peter Cseh
> Software Engineer
> 
>

[jira] [Commented] (OOZIE-2814) OYA: Update example workflows to newest schemas

[Robert Kanter (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16306578#comment-16306578
 ] 

Robert Kanter commented on OOZIE-2814:
--

Making this a blocker.  It'll help get users to start using the newer schema 
with all of the OYA changes.

> OYA: Update example workflows to newest schemas
> ---
>
> Key: OOZIE-2814
> URL: https://issues.apache.org/jira/browse/OOZIE-2814
> Project: Oozie
>  Issue Type: Sub-task
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Priority: Blocker
>
> OOZIE-2687 will add in a whole set of newer schemas that are more 
> Yarn-centric.  Most, if not all, examples currently use pretty old versions 
> of the schemas.  We should take this opportunity to update the examples to 
> the newest schemas added by OOZIE-2687, which should help demonstrate them.
> It may make sense to keep the older schemas too, to show that older workflows 
> still work.  We have some examples where we have two workflow XML files that 
> do the same thing in different ways.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (OOZIE-2814) OYA: Update example workflows to newest schemas

[Robert Kanter (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Kanter updated OOZIE-2814:
-
Priority: Blocker  (was: Critical)

> OYA: Update example workflows to newest schemas
> ---
>
> Key: OOZIE-2814
> URL: https://issues.apache.org/jira/browse/OOZIE-2814
> Project: Oozie
>  Issue Type: Sub-task
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Priority: Blocker
>
> OOZIE-2687 will add in a whole set of newer schemas that are more 
> Yarn-centric.  Most, if not all, examples currently use pretty old versions 
> of the schemas.  We should take this opportunity to update the examples to 
> the newest schemas added by OOZIE-2687, which should help demonstrate them.
> It may make sense to keep the older schemas too, to show that older workflows 
> still work.  We have some examples where we have two workflow XML files that 
> do the same thing in different ways.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (OOZIE-3150) Update NOTICE.txt

[Robert Kanter (JIRA)]

Robert Kanter created OOZIE-3150:


 Summary: Update NOTICE.txt
 Key: OOZIE-3150
 URL: https://issues.apache.org/jira/browse/OOZIE-3150
 Project: Oozie
  Issue Type: Bug
Affects Versions: 5.0.0b1
Reporter: Robert Kanter
Priority: Blocker
 Fix For: 5.0.0


NOTICE.txt currently contains:
{noformat}
===
NOTICE file for use with, and corresponding to Section 4 of,
the Apache License, Version 2.0,
in this case for the Oozie, Workflow Engine for Hadoop project
===

Copyright 2011 The Apache Software Foundation

This product includes software developed by The Apache Software
Foundation (http://www.apache.org/).

This product includes org.json (http://www.json.org/java/index.html),
Copyright (c) 2002 JSON.org

Detailed License information for all components can be found in the
documentation in the ooziedocs.war at index.html##LicenseInfo

Oozie source contains 3 binaries (wordcount-simple_Linux-amd64-64_h20,
wordcount-simple_Linux-i386-32_h20, wordcount-simple_Mac_OS_X-x86_64-64_h20) for
testing purposes, these binaries are compiled artifacts from Apache Hadoop.
{noformat}

A few problems:
- There is no ooziedocs.war
- Oozie doesn't contain any binaries AFAIK and hasn't for a long time
- We should see if there is anything else we need to add



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (OOZIE-3149) Delete work.log

[Robert Kanter (JIRA)]

Robert Kanter created OOZIE-3149:


 Summary: Delete work.log
 Key: OOZIE-3149
 URL: https://issues.apache.org/jira/browse/OOZIE-3149
 Project: Oozie
  Issue Type: Bug
Affects Versions: 5.0.0
Reporter: Robert Kanter
Priority: Trivial


The root dir has a file named {{work.log}} which appears to be an old version 
of {{release-log.txt}}, up to Oozie 3.0.1.  It should be safe to simply delete 
it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-3148) Rerun Failing Tests through Maven surefire

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16306428#comment-16306428
 ] 

Hadoop QA commented on OOZIE-3148:
--

Testing JIRA OOZIE-3148

Cleaning local git workspace



{color:red}-1{color} Patch failed to apply to head of branch



> Rerun Failing Tests through Maven surefire
> --
>
> Key: OOZIE-3148
> URL: https://issues.apache.org/jira/browse/OOZIE-3148
> Project: Oozie
>  Issue Type: Improvement
>  Components: tests
>Affects Versions: trunk
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Minor
> Fix For: 5.0.0
>
> Attachments: OOZIE-3148-01.patch, OOZIE-3148-02.patch, 
> OOZIE-3148-03.patch
>
>
> OOZIE-2817 introduced rerunning of failing test cases in pre-commit builds 
> (changed {{bin/test-patch-20-tests}} Bash script).  
> It is no longer necessary as OOZIE-3106 upgraded surefire-plugin to 2.20.1 
> and since of 2.19.1 version of the maven surefire plugin it is allowed 
> rerunning failing tests via {{-Dsurefire.rerunFailingTestsCount=2}} (see 
> http://maven.apache.org/surefire/maven-surefire-plugin/examples/rerun-failing-tests.html).
>  For example, a test passed in its second run will output on the screen:
> {code}
>   Run 1: ...
>   Run 2: PASS
> Then this test will be counted as a flaky test. The build will be successful, 
> but in the end of the summary of all tests run, the number of flaky tests 
> will be output on the screen, for example:
>   Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Flakes: 1
> {code}
> We could introduce this mechanism and update {{bin/test-patch-20-tests}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Failed: OOZIE-3148 PreCommit Build #296

[Apache Jenkins Server]

Jira: https://issues.apache.org/jira/browse/OOZIE-3148
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/296/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 683.56 KB...]
[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Oozie Main .. SUCCESS [  1.627 s]
[INFO] Apache Oozie Client  SUCCESS [ 17.030 s]
[INFO] Apache Oozie Share Lib Oozie ... SUCCESS [ 13.093 s]
[INFO] Apache Oozie Share Lib HCatalog  SUCCESS [  6.931 s]
[INFO] Apache Oozie Share Lib Distcp .. SUCCESS [  5.441 s]
[INFO] Apache Oozie Core .. SUCCESS [01:17 min]
[INFO] Apache Oozie Share Lib Streaming ... SUCCESS [  8.927 s]
[INFO] Apache Oozie Share Lib Pig . SUCCESS [  8.817 s]
[INFO] Apache Oozie Share Lib Hive  SUCCESS [  9.373 s]
[INFO] Apache Oozie Share Lib Hive 2 .. SUCCESS [ 10.377 s]
[INFO] Apache Oozie Share Lib Sqoop ... SUCCESS [  7.009 s]
[INFO] Apache Oozie Examples .. SUCCESS [  9.561 s]
[INFO] Apache Oozie Share Lib Spark ... SUCCESS [ 12.268 s]
[INFO] Apache Oozie Share Lib . SUCCESS [  0.024 s]
[INFO] Apache Oozie Docs .. SUCCESS [  3.482 s]
[INFO] Apache Oozie WebApp  SUCCESS [  3.340 s]
[INFO] Apache Oozie Tools . SUCCESS [ 14.990 s]
[INFO] Apache Oozie MiniOozie . SUCCESS [  2.714 s]
[INFO] Apache Oozie Server  SUCCESS [ 11.606 s]
[INFO] Apache Oozie Distro  SUCCESS [  2.104 s]
[INFO] Apache Oozie ZooKeeper Security Tests .. SUCCESS [  4.280 s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 03:51 min
[INFO] Finished at: 2017-12-29T18:06:07Z
[INFO] Final Memory: 650M/1935M
[INFO] 
[TRACE] FindBugs output in HEAD verified and saved
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO

Applying patch

Checking patch a/bin/test-patch-20-tests => b/bin/test-patch-20-tests...
error: a/bin/test-patch-20-tests: No such file or directory
Checking patch a/bin/test-patch-20-tests => b/bin/test-patch-20-tests...
error: a/bin/test-patch-20-tests: No such file or directory
Checking patch a/bin/test-patch-20-tests => b/bin/test-patch-20-tests...
error: a/bin/test-patch-20-tests: No such file or directory
Checking patch a/bin/test-patch-20-tests => b/bin/test-patch-20-tests...
error: a/bin/test-patch-20-tests: No such file or directory
Checking patch bin/test-patch-20-tests...
Checking patch bin/test-patch-20-tests...
error: while searching for:
}
###

parseArgs "$@"

case $OP in

error: patch failed: bin/test-patch-20-tests:81
error: bin/test-patch-20-tests: patch does not apply
Checking patch bin/test-patch-20-tests...
error: while searching for:
}
###

parseArgs "$@"

case $OP in

error: patch failed: bin/test-patch-20-tests:81
error: bin/test-patch-20-tests: patch does not apply
Checking patch bin/test-patch-20-tests...
error: while searching for:
}
###

parseArgs "$@"

case $OP in

error: patch failed: bin/test-patch-20-tests:81
error: bin/test-patch-20-tests: patch does not apply
Patch failed to apply to head of branch

  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  
0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  0 
00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
  4 3706k4  159k0 0   139k  0  0:00:26  0:00:01  0:00:25  
139k100 3706k  100 3706k0 0  2896k  0  0:00:01  0:00:01 --:--:-- 
25.6M
Adding comment to JIRA
Comment added.

test-patch exit code: 1

Build step 'Execute shell' marked build as failure
[description-setter] Description set: OOZIE-3148
Archiving artifacts
[Fast Archiver] Compressed 817.51 KB of artifacts by 66.5% relative to #284
Recording test results
ERROR: Step ?Publish JUnit test result 

[jira] [Updated] (OOZIE-3148) Rerun Failing Tests through Maven surefire

[Attila Sasvari (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Sasvari updated OOZIE-3148:
--
Attachment: OOZIE-3148-03.patch

> Rerun Failing Tests through Maven surefire
> --
>
> Key: OOZIE-3148
> URL: https://issues.apache.org/jira/browse/OOZIE-3148
> Project: Oozie
>  Issue Type: Improvement
>  Components: tests
>Affects Versions: trunk
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Minor
> Fix For: 5.0.0
>
> Attachments: OOZIE-3148-01.patch, OOZIE-3148-02.patch, 
> OOZIE-3148-03.patch
>
>
> OOZIE-2817 introduced rerunning of failing test cases in pre-commit builds 
> (changed {{bin/test-patch-20-tests}} Bash script).  
> It is no longer necessary as OOZIE-3106 upgraded surefire-plugin to 2.20.1 
> and since of 2.19.1 version of the maven surefire plugin it is allowed 
> rerunning failing tests via {{-Dsurefire.rerunFailingTestsCount=2}} (see 
> http://maven.apache.org/surefire/maven-surefire-plugin/examples/rerun-failing-tests.html).
>  For example, a test passed in its second run will output on the screen:
> {code}
>   Run 1: ...
>   Run 2: PASS
> Then this test will be counted as a flaky test. The build will be successful, 
> but in the end of the summary of all tests run, the number of flaky tests 
> will be output on the screen, for example:
>   Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Flakes: 1
> {code}
> We could introduce this mechanism and update {{bin/test-patch-20-tests}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (OOZIE-3148) Rerun Failing Tests through Maven surefire

[Attila Sasvari (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Sasvari updated OOZIE-3148:
--
Fix Version/s: (was: 5.0.0b1)
   5.0.0

> Rerun Failing Tests through Maven surefire
> --
>
> Key: OOZIE-3148
> URL: https://issues.apache.org/jira/browse/OOZIE-3148
> Project: Oozie
>  Issue Type: Improvement
>  Components: tests
>Affects Versions: trunk
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Minor
> Fix For: 5.0.0
>
> Attachments: OOZIE-3148-01.patch, OOZIE-3148-02.patch
>
>
> OOZIE-2817 introduced rerunning of failing test cases in pre-commit builds 
> (changed {{bin/test-patch-20-tests}} Bash script).  
> It is no longer necessary as OOZIE-3106 upgraded surefire-plugin to 2.20.1 
> and since of 2.19.1 version of the maven surefire plugin it is allowed 
> rerunning failing tests via {{-Dsurefire.rerunFailingTestsCount=2}} (see 
> http://maven.apache.org/surefire/maven-surefire-plugin/examples/rerun-failing-tests.html).
>  For example, a test passed in its second run will output on the screen:
> {code}
>   Run 1: ...
>   Run 2: PASS
> Then this test will be counted as a flaky test. The build will be successful, 
> but in the end of the summary of all tests run, the number of flaky tests 
> will be output on the screen, for example:
>   Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Flakes: 1
> {code}
> We could introduce this mechanism and update {{bin/test-patch-20-tests}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-3148) Rerun Failing Tests through Maven surefire

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16306394#comment-16306394
 ] 

Hadoop QA commented on OOZIE-3148:
--

Testing JIRA OOZIE-3148

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:red}-1{color} the patch contains 1 line(s) with trailing spaces
.{color:red}-1{color} the patch contains 2 line(s) longer than 132 
characters
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color} - patch does not compile, cannot run testcases
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/295/

> Rerun Failing Tests through Maven surefire
> --
>
> Key: OOZIE-3148
> URL: https://issues.apache.org/jira/browse/OOZIE-3148
> Project: Oozie
>  Issue Type: Improvement
>  Components: tests
>Affects Versions: trunk
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Minor
> Fix For: 5.0.0b1
>
> Attachments: OOZIE-3148-01.patch, OOZIE-3148-02.patch
>
>
> OOZIE-2817 introduced rerunning of failing test cases in pre-commit builds 
> (changed {{bin/test-patch-20-tests}} Bash script).  
> It is no longer necessary as OOZIE-3106 upgraded surefire-plugin to 2.20.1 
> and since of 2.19.1 version of the maven surefire plugin it is allowed 
> rerunning failing tests via {{-Dsurefire.rerunFailingTestsCount=2}} (see 
> http://maven.apache.org/surefire/maven-surefire-plugin/examples/rerun-failing-tests.html).
>  For example, a test passed in its second run will output on the screen:
> {code}
>   Run 1: ...
>   Run 2: PASS
> Then this test will be counted as a flaky test. The build will be successful, 
> but in the end of the summary of all tests run, the number of flaky tests 
> will be output on the screen, for example:
>   Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Flakes: 1
> {code}
> We could introduce this mechanism and update {{bin/test-patch-20-tests}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Failed: OOZIE-3148 PreCommit Build #295

[Apache Jenkins Server]

Jira: https://issues.apache.org/jira/browse/OOZIE-3148
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/295/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 1.56 MB...]
[DEBUG] There are no new bugs found in [sharelib/hcatalog].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/streaming].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/pig].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/sqoop].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/oozie].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [examples].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [client].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [core].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [tools].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [server].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2112 bytes
[TRACE] Full summary file size is 1248 bytes
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
 removed
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
 removed
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO


Testing JIRA OOZIE-3148

Cleaning local git workspace



+1 PATCH_APPLIES
+1 CLEAN
-1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
-1 the patch contains 1 line(s) with trailing spaces
-1 the patch contains 2 line(s) longer than 132 characters
+1 the patch does adds/modifies 1 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warnings
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/pig].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [examples].
 +1 There are no new bugs found in [client].
 +1 There are no new bugs found in [core].
 +1 There are no new bugs found in [tools].
 +1 There are no new bugs found in [server].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
-1 TESTS - patch does not compile, cannot run testcases
+1 DISTRO
+1 distro tarball builds with the patch 


-1 Overall result, please check the reported -1(s)


The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/295/

  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  
0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 
0100 3706k  100 3706k0 0  3219k  0  0:00:01  0:00:01 --:--:-- 5212k
Adding comment to JIRA
Comment added.

test-patch exit code: 1

Build step 'Execute shell' marked build as failure
[description-setter] Description set: OOZIE-3148
Archiving artifacts
[Fast Archiver] Compressed 1.71 MB of artifacts by 62.0% relative to #284
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[jira] [Updated] (OOZIE-3148) Rerun Failing Tests through Maven surefire

[Attila Sasvari (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Sasvari updated OOZIE-3148:
--
Attachment: OOZIE-3148-02.patch

> Rerun Failing Tests through Maven surefire
> --
>
> Key: OOZIE-3148
> URL: https://issues.apache.org/jira/browse/OOZIE-3148
> Project: Oozie
>  Issue Type: Improvement
>  Components: tests
>Affects Versions: trunk
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Minor
> Fix For: 5.0.0b1
>
> Attachments: OOZIE-3148-01.patch, OOZIE-3148-02.patch
>
>
> OOZIE-2817 introduced rerunning of failing test cases in pre-commit builds 
> (changed {{bin/test-patch-20-tests}} Bash script).  
> It is no longer necessary as OOZIE-3106 upgraded surefire-plugin to 2.20.1 
> and since of 2.19.1 version of the maven surefire plugin it is allowed 
> rerunning failing tests via {{-Dsurefire.rerunFailingTestsCount=2}} (see 
> http://maven.apache.org/surefire/maven-surefire-plugin/examples/rerun-failing-tests.html).
>  For example, a test passed in its second run will output on the screen:
> {code}
>   Run 1: ...
>   Run 2: PASS
> Then this test will be counted as a flaky test. The build will be successful, 
> but in the end of the summary of all tests run, the number of flaky tests 
> will be output on the screen, for example:
>   Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Flakes: 1
> {code}
> We could introduce this mechanism and update {{bin/test-patch-20-tests}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-1401) PurgeCommand should purge the workflow jobs w/o end_time

[Attila Sasvari (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16306319#comment-16306319
 ] 

Attila Sasvari commented on OOZIE-1401:
---

[~vaifer] thanks for spotting and reporting this issue. You are right that the 
mentioned queries do not return lastModificationTime, and if end_time is null, 
then those workflows can't be purged as there is no information when they have 
finished (but at least the purge service does not throw an NPE). An amendment 
patch should be fine I believe.


> PurgeCommand should purge the workflow jobs w/o end_time
> 
>
> Key: OOZIE-1401
> URL: https://issues.apache.org/jira/browse/OOZIE-1401
> Project: Oozie
>  Issue Type: Sub-task
>  Components: bundle, coordinator, workflow
>Affects Versions: trunk
>Reporter: Mona Chitnis
>Assignee: Attila Sasvari
> Fix For: 5.0.0b1
>
> Attachments: OOZIE-1401-001.patch
>
>
> Currently, {{PurgeXCommand}} logic is not working with those workflow jobs 
> with {{end_time=null}}. This command needs to take care of those jobs as 
> well. This happens in the case of long stuck jobs after Hadoop restarts or DB 
> failures. It could be done by checking {{last_modified_time}} instead, if 
> {{end_time}} is not available.
> The current query:
> {code:sql}
> select w from WorkflowJobBean w where w.endTimestamp < :endTime
> {code}
> There is also an issue when:
> * there is a parent workflow that has its {{end_time}} set
> * is otherwise eligible for {{PurgeXCommand}}: {{end_time}} is older than 
> configured number of days, and has {{status}} either {{KILLED}}, or 
> {{FAILED}}, or {{SUCCEEDED}}
> * has a child workflow that has the {{parent_id}} set to the {{id}} of the 
> parent workflow
> * child workflow has its {{end_time = NULL}}
> In this case, 
> [*{{PurgeXCommand#fetchTerminatedWorkflow()}}*|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/command/PurgeXCommand.java#L249]
>  throws a {{NullPointerException}} like this:
> {noformat}
> 2017-09-29 07:59:46,365 DEBUG org.apache.oozie.command.PurgeXCommand: 
> SERVER[host-10-17-101-90.coe.cloudera.com] USER[-] GROUP[-] TOKEN[-] APP[-] 
> JOB[-] ACTION[-] Purging workflows of long running coordinators is turned on
> 2017-09-29 07:59:46,371 DEBUG org.apache.oozie.command.PurgeXCommand: 
> SERVER[host-10-17-101-90.coe.cloudera.com] USER[-] GROUP[-] TOKEN[-] APP[-] 
> JOB[-] ACTION[-] Execute command [purge] key [null]
> 2017-09-29 07:59:46,371 INFO org.apache.oozie.command.PurgeXCommand: 
> SERVER[host-10-17-101-90.coe.cloudera.com] USER[-] GROUP[-] TOKEN[-] APP[-] 
> JOB[-] ACTION[-] STARTED Purge to purge Workflow Jobs older than [1] days, 
> Coordinator Jobs older than [1] days, and Bundlejobs older than [1] days.
> 2017-09-29 07:59:46,375 ERROR org.apache.oozie.command.PurgeXCommand: 
> SERVER[host-10-17-101-90.coe.cloudera.com] USER[-] GROUP[-] TOKEN[-] APP[-] 
> JOB[-] ACTION[-] Exception, 
> java.lang.NullPointerException
>   at 
> org.apache.oozie.command.PurgeXCommand.fetchTerminatedWorkflow(PurgeXCommand.java:249)
>   at 
> org.apache.oozie.command.PurgeXCommand.processWorkflowsHelper(PurgeXCommand.java:227)
>   at 
> org.apache.oozie.command.PurgeXCommand.processWorkflows(PurgeXCommand.java:199)
>   at 
> org.apache.oozie.command.PurgeXCommand.execute(PurgeXCommand.java:150)
>   at org.apache.oozie.command.PurgeXCommand.execute(PurgeXCommand.java:53)
>   at org.apache.oozie.command.XCommand.call(XCommand.java:286)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>   at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:178)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Failed: OOZIE-3148 PreCommit Build #294

[Apache Jenkins Server]

Jira: https://issues.apache.org/jira/browse/OOZIE-3148
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/294/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 1.56 MB...]
[DEBUG] There are no new bugs found in [sharelib/hcatalog].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/streaming].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/pig].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/sqoop].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/oozie].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [examples].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [client].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [core].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [tools].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [server].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2113 bytes
[TRACE] Full summary file size is 1248 bytes
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
 removed
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
 removed
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO


Testing JIRA OOZIE-3148

Cleaning local git workspace



+1 PATCH_APPLIES
+1 CLEAN
-1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
-1 the patch contains 3 line(s) longer than 132 characters
+1 the patch does adds/modifies 1 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warnings
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/pig].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [examples].
 +1 There are no new bugs found in [client].
 +1 There are no new bugs found in [core].
 +1 There are no new bugs found in [tools].
 +1 There are no new bugs found in [server].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
-1 TESTS - patch does not compile, cannot run testcases
+1 DISTRO
+1 distro tarball builds with the patch 


-1 Overall result, please check the reported -1(s)


The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/294/

  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  
0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 
0100 3706k  100 3706k0 0  2838k  0  0:00:01  0:00:01 --:--:-- 8057k
Adding comment to JIRA
Comment added.

test-patch exit code: 1

Build step 'Execute shell' marked build as failure
[description-setter] Description set: OOZIE-3148
Archiving artifacts
[Fast Archiver] Compressed 1.71 MB of artifacts by 62.0% relative to #284
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[jira] [Commented] (OOZIE-3148) Rerun Failing Tests through Maven surefire

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16306307#comment-16306307
 ] 

Hadoop QA commented on OOZIE-3148:
--

Testing JIRA OOZIE-3148

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 3 line(s) longer than 132 
characters
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color} - patch does not compile, cannot run testcases
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/294/

> Rerun Failing Tests through Maven surefire
> --
>
> Key: OOZIE-3148
> URL: https://issues.apache.org/jira/browse/OOZIE-3148
> Project: Oozie
>  Issue Type: Improvement
>  Components: tests
>Affects Versions: trunk
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Minor
> Fix For: 5.0.0b1
>
> Attachments: OOZIE-3148-01.patch
>
>
> OOZIE-2817 introduced rerunning of failing test cases in pre-commit builds 
> (changed {{bin/test-patch-20-tests}} Bash script).  
> It is no longer necessary as OOZIE-3106 upgraded surefire-plugin to 2.20.1 
> and since of 2.19.1 version of the maven surefire plugin it is allowed 
> rerunning failing tests via {{-Dsurefire.rerunFailingTestsCount=2}} (see 
> http://maven.apache.org/surefire/maven-surefire-plugin/examples/rerun-failing-tests.html).
>  For example, a test passed in its second run will output on the screen:
> {code}
>   Run 1: ...
>   Run 2: PASS
> Then this test will be counted as a flaky test. The build will be successful, 
> but in the end of the summary of all tests run, the number of flaky tests 
> will be output on the screen, for example:
>   Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Flakes: 1
> {code}
> We could introduce this mechanism and update {{bin/test-patch-20-tests}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-1401) PurgeCommand should purge the workflow jobs w/o end_time

[Sergey Svinarchuk (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16306288#comment-16306288
 ] 

Sergey Svinarchuk commented on OOZIE-1401:
--

This patch doesn't work because lastModificationTime always NULL. Need add 
lastModificationTime to GET_WORKFLOWS_BASIC_INFO_BY_PARENT_ID and 
GET_WORKFLOWS_BASIC_INFO_BY_COORD_PARENT_ID queries. I can create new patch for 
this ticket or open new issue.

> PurgeCommand should purge the workflow jobs w/o end_time
> 
>
> Key: OOZIE-1401
> URL: https://issues.apache.org/jira/browse/OOZIE-1401
> Project: Oozie
>  Issue Type: Sub-task
>  Components: bundle, coordinator, workflow
>Affects Versions: trunk
>Reporter: Mona Chitnis
>Assignee: Attila Sasvari
> Fix For: 5.0.0b1
>
> Attachments: OOZIE-1401-001.patch
>
>
> Currently, {{PurgeXCommand}} logic is not working with those workflow jobs 
> with {{end_time=null}}. This command needs to take care of those jobs as 
> well. This happens in the case of long stuck jobs after Hadoop restarts or DB 
> failures. It could be done by checking {{last_modified_time}} instead, if 
> {{end_time}} is not available.
> The current query:
> {code:sql}
> select w from WorkflowJobBean w where w.endTimestamp < :endTime
> {code}
> There is also an issue when:
> * there is a parent workflow that has its {{end_time}} set
> * is otherwise eligible for {{PurgeXCommand}}: {{end_time}} is older than 
> configured number of days, and has {{status}} either {{KILLED}}, or 
> {{FAILED}}, or {{SUCCEEDED}}
> * has a child workflow that has the {{parent_id}} set to the {{id}} of the 
> parent workflow
> * child workflow has its {{end_time = NULL}}
> In this case, 
> [*{{PurgeXCommand#fetchTerminatedWorkflow()}}*|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/command/PurgeXCommand.java#L249]
>  throws a {{NullPointerException}} like this:
> {noformat}
> 2017-09-29 07:59:46,365 DEBUG org.apache.oozie.command.PurgeXCommand: 
> SERVER[host-10-17-101-90.coe.cloudera.com] USER[-] GROUP[-] TOKEN[-] APP[-] 
> JOB[-] ACTION[-] Purging workflows of long running coordinators is turned on
> 2017-09-29 07:59:46,371 DEBUG org.apache.oozie.command.PurgeXCommand: 
> SERVER[host-10-17-101-90.coe.cloudera.com] USER[-] GROUP[-] TOKEN[-] APP[-] 
> JOB[-] ACTION[-] Execute command [purge] key [null]
> 2017-09-29 07:59:46,371 INFO org.apache.oozie.command.PurgeXCommand: 
> SERVER[host-10-17-101-90.coe.cloudera.com] USER[-] GROUP[-] TOKEN[-] APP[-] 
> JOB[-] ACTION[-] STARTED Purge to purge Workflow Jobs older than [1] days, 
> Coordinator Jobs older than [1] days, and Bundlejobs older than [1] days.
> 2017-09-29 07:59:46,375 ERROR org.apache.oozie.command.PurgeXCommand: 
> SERVER[host-10-17-101-90.coe.cloudera.com] USER[-] GROUP[-] TOKEN[-] APP[-] 
> JOB[-] ACTION[-] Exception, 
> java.lang.NullPointerException
>   at 
> org.apache.oozie.command.PurgeXCommand.fetchTerminatedWorkflow(PurgeXCommand.java:249)
>   at 
> org.apache.oozie.command.PurgeXCommand.processWorkflowsHelper(PurgeXCommand.java:227)
>   at 
> org.apache.oozie.command.PurgeXCommand.processWorkflows(PurgeXCommand.java:199)
>   at 
> org.apache.oozie.command.PurgeXCommand.execute(PurgeXCommand.java:150)
>   at org.apache.oozie.command.PurgeXCommand.execute(PurgeXCommand.java:53)
>   at org.apache.oozie.command.XCommand.call(XCommand.java:286)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>   at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:178)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (OOZIE-3148) Rerun Failing Tests through Maven surefire

[Attila Sasvari (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Sasvari updated OOZIE-3148:
--
Attachment: OOZIE-3148-01.patch

> Rerun Failing Tests through Maven surefire
> --
>
> Key: OOZIE-3148
> URL: https://issues.apache.org/jira/browse/OOZIE-3148
> Project: Oozie
>  Issue Type: Improvement
>  Components: tests
>Affects Versions: trunk
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Minor
> Fix For: 5.0.0b1
>
> Attachments: OOZIE-3148-01.patch
>
>
> OOZIE-2817 introduced rerunning of failing test cases in pre-commit builds 
> (changed {{bin/test-patch-20-tests}} Bash script).  
> It is no longer necessary as OOZIE-3106 upgraded surefire-plugin to 2.20.1 
> and since of 2.19.1 version of the maven surefire plugin it is allowed 
> rerunning failing tests via {{-Dsurefire.rerunFailingTestsCount=2}} (see 
> http://maven.apache.org/surefire/maven-surefire-plugin/examples/rerun-failing-tests.html).
>  For example, a test passed in its second run will output on the screen:
> {code}
>   Run 1: ...
>   Run 2: PASS
> Then this test will be counted as a flaky test. The build will be successful, 
> but in the end of the summary of all tests run, the number of flaky tests 
> will be output on the screen, for example:
>   Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Flakes: 1
> {code}
> We could introduce this mechanism and update {{bin/test-patch-20-tests}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (OOZIE-3148) Rerun Failing Tests through Maven surefire

[Attila Sasvari (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Sasvari reassigned OOZIE-3148:
-

Assignee: Attila Sasvari

> Rerun Failing Tests through Maven surefire
> --
>
> Key: OOZIE-3148
> URL: https://issues.apache.org/jira/browse/OOZIE-3148
> Project: Oozie
>  Issue Type: Improvement
>  Components: tests
>Affects Versions: trunk
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Minor
> Fix For: 5.0.0b1
>
> Attachments: OOZIE-3148-01.patch
>
>
> OOZIE-2817 introduced rerunning of failing test cases in pre-commit builds 
> (changed {{bin/test-patch-20-tests}} Bash script).  
> It is no longer necessary as OOZIE-3106 upgraded surefire-plugin to 2.20.1 
> and since of 2.19.1 version of the maven surefire plugin it is allowed 
> rerunning failing tests via {{-Dsurefire.rerunFailingTestsCount=2}} (see 
> http://maven.apache.org/surefire/maven-surefire-plugin/examples/rerun-failing-tests.html).
>  For example, a test passed in its second run will output on the screen:
> {code}
>   Run 1: ...
>   Run 2: PASS
> Then this test will be counted as a flaky test. The build will be successful, 
> but in the end of the summary of all tests run, the number of flaky tests 
> will be output on the screen, for example:
>   Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Flakes: 1
> {code}
> We could introduce this mechanism and update {{bin/test-patch-20-tests}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)