I took a look at the latest PreCommit job
<https://builds.apache.org/job/PreCommit-OOZIE-Build/288/consoleFull> and
it reported 55 rerun tests.
Tests rerun: 55
Tests failed at first run:
org.apache.oozie.action.hadoop.TestJavaActionExecutor,
However, looking through the actual output, I only see 1 test that failed
(and was rerun): TestJavaActionExecutor. testCredentialsSkip.
[INFO] Running org.apache.oozie.action.hadoop.TestJavaActionExecutor
[ERROR] Tests run: 55, Failures: 0, Errors: 1, Skipped: 0, Time
elapsed: 126.213 s <<< FAILURE! - in
org.apache.oozie.action.hadoop.TestJavaActionExecutor
[ERROR]
testCredentialsSkip(org.apache.oozie.action.hadoop.TestJavaActionExecutor)
Time elapsed: 0.532 s <<< ERROR!
org.apache.oozie.action.ActionExecutorException: JA020: Could not load
credentials of type [abc] with name [abcname]]; perhaps it was not
defined in oozie-site.xml?
at
org.apache.oozie.action.hadoop.TestJavaActionExecutor._testCredentialsSkip(TestJavaActionExecutor.java:1106)
at
org.apache.oozie.action.hadoop.TestJavaActionExecutor.testCredentialsSkip(TestJavaActionExecutor.java:1006)
In fact, the report only lists the one test class, not 55 of them. So I
think there's something wrong with our reporting here.
Anyway, typically, when we see test that succeed on their own but fail when
run all together, that means that (likely some other) test is not cleaning
up properly. This is unfortunately tricky to debug because it's hard to
figure out what the other test is. A long time ago, a big example of this
problem was not properly shutting down the Services singleton, so we'd have
duplicates and other issues.
For these specific issues, some hints:
# RUNNINGWITHERROR: This problem means that a yarn job in the mini cluster
has failed. To find out why, you should be able to dig out the app id from
the test output, and then find it's yarn logs somewhere (there's a
minicluster logs dir, but I forget where). That'll hopefully make it
obvious what's going on.
# Credentials: There's probably an oozie-site or Configuration class
leaking from somewhere or not properly cleaned up by a previous test or
setup by this test. The Credentials class is missing.
# TestJMSAccessorService: Sounds like something didn't get cleaned up.
One way that might be easier to fix this is to have the setUp methods
ensure that things are clean just in case. We actually have a number of
tests that do things like this too.
- Robert
On Thu, Dec 21, 2017 at 12:40 PM, Attila Sasvari <[email protected]>
wrote:
> Update:
> - bin/mkdistro.sh fails because there are test failures.
> [ERROR] Failures:
>
>
> [ERROR]
> TestCoordActionsKillXCommand.testActionKillCommandActionNumbers:96
> expected:<RUNNING> but was:<RUNNINGWITHERROR>
>
>
> [ERROR] Errors:
>
>
> [ERROR]
> TestJavaActionExecutor.testCredentialsSkip:1006->_testCredentialsSkip:1106
> ? ActionExecutor
>
> [ERROR] TestJMSAccessorService.testConnectionRetryExceptionListener:211
> ?
> InstanceAlreadyExists
>
>
> - if I run the tests separately, they pass. Looking at the latest precommit
> builds (https://builds.apache.org/job/PreCommit-OOZIE-Build), it turns out
> that a lot of tests had to be re-executed to get a +1 for the TESTS part.
> Problem is that the Oozie tests have impact on each other, and it looks
> like the execution order matters too.
> - uploaded work in progress (SNAPSHOT) artifacts here:
> http://people.apache.org/~asasvari/oozie-5.0.0-beta1-SNAPSHOT/
> - I plan to update the "How To Release" page as it contains some errors
> (e.g sftp shall be used to upload artifacts)
> - agreed with Artem that OOZIE-2231 will slip to 5.0.0
>
> Regards,
> - Attila
>
> On Mon, Dec 18, 2017 at 11:47 PM, Attila Sasvari <[email protected]>
> wrote:
>
> > Hi everyone,
> >
> > branch-5.0.0-beta1 has been created.
> >
> > Regards,
> > Attila
> >
> > On Mon, Dec 18, 2017 at 7:25 PM, Attila Sasvari <[email protected]>
> > wrote:
> >
> >> Thanks gp. I will follow the steps described on
> >> https://cwiki.apache.org/confluence/display/OOZIE/How+To+Release during
> >> the process.
> >>
> >> Next steps:
> >> - A new branch is about to be created from master.
> >>
> >> Artem, many thanks. I will review and commit that patch if everything is
> >> okay. I don't see any problem with including it in 5.0.0-beta1.
> >>
> >> On Mon, Dec 18, 2017 at 4:32 PM, Artem Ervits <[email protected]>
> >> wrote:
> >>
> >>> just uploaded patch for OOZIE-2231.
> >>>
> >>> On Mon, Dec 18, 2017 at 10:04 AM, Peter Cseh <[email protected]>
> >>> wrote:
> >>>
> >>> > Hey Attila,
> >>> >
> >>> > I won't be able to work on the release for a couple weeks now.
> >>> > Thanks for getting the release rolling!
> >>> >
> >>> > Cheers,
> >>> > gp
> >>> >
> >>> >
> >>> > On Mon, Dec 18, 2017 at 4:00 PM, Attila Sasvari <
> [email protected]
> >>> >
> >>> > wrote:
> >>> >
> >>> > > Hi everyone,
> >>> > >
> >>> > > I would like to create the release branch, branch-5.0.0-beta1
> >>> (following
> >>> > > Hadoop release versioning), earlier.
> >>> > >
> >>> > > Looking at https://issues.apache.org/jira/projects/OOZIE/versions/
> >>> > 12342048
> >>> > > there are 3 issues in progress (OOZIE-2231, OOZIE-2942, OOZIE-2974)
> >>> and 3
> >>> > > issues to do (OOZIE-2600, OOZIE-3093, OOZIE-1987). I will push
> those
> >>> out
> >>> > to
> >>> > > 5.0.0 if there are no objections.
> >>> > >
> >>> > > At the same time, I am volunteering to be the release manager if
> >>> Peter
> >>> > Cseh
> >>> > > does not mind.
> >>> > >
> >>> > > Regards,
> >>> > > Attila
> >>> > >
> >>> > > On Wed, Dec 6, 2017 at 8:28 PM, Robert Kanter <
> [email protected]>
> >>> > > wrote:
> >>> > >
> >>> > > > Sounds good to me!
> >>> > > >
> >>> > > > On Wed, Dec 6, 2017 at 5:17 AM, Andras Piros <
> >>> > [email protected]>
> >>> > > > wrote:
> >>> > > >
> >>> > > > > Good idea Gezapeti!
> >>> > > > >
> >>> > > > > Time to wrap things up towards a stable 5.0.0 - a release
> >>> candidate
> >>> > on
> >>> > > > > 5.0.0b1 is a good first step.
> >>> > > > >
> >>> > > > > Since other components that Oozie uses like Pig and Hive do not
> >>> > (fully)
> >>> > > > > support Hadoop 3, we have to wait with OOZIE-2973
> >>> > > > > <https://issues.apache.org/jira/browse/OOZIE-2973>.
> >>> > > > >
> >>> > > > > Thanks,
> >>> > > > >
> >>> > > > > Andras
> >>> > > > >
> >>> > > > > On Wed, Dec 6, 2017 at 1:50 PM, Peter Cseh <
> >>> [email protected]>
> >>> > > > wrote:
> >>> > > > >
> >>> > > > > > Hi everyone!
> >>> > > > > >
> >>> > > > > > Now that OOZIE-2969 <https://issues.apache.org/
> >>> > > jira/browse/OOZIE-2969>
> >>> > > > > is
> >>> > > > > > committed I'd like to start the process of creating the
> branch
> >>> for
> >>> > > > > 5.0.0b1
> >>> > > > > > and building a release from there.
> >>> > > > > > It's unfortunate that we won't be able to support Hadoop 3 in
> >>> the
> >>> > > beta
> >>> > > > > for
> >>> > > > > > reasons described in OOZIE-2973
> >>> > > > > > <https://issues.apache.org/jira/browse/OOZIE-2973>
> >>> > > > > > I don't see any more blockers for the beta1 and I hope we
> won't
> >>> > find
> >>> > > > > > hard-to-fix major issues so we can release Oozie 5.0.0 in
> early
> >>> > 2018.
> >>> > > > > > Please let me know if you have any suggestions.
> >>> > > > > > Thanks
> >>> > > > > > gp
> >>> > > > > >
> >>> > > > > > --
> >>> > > > > > Peter Cseh
> >>> > > > > > Software Engineer
> >>> > > > > > <http://www.cloudera.com>
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> > >
> >>> > >
> >>> > > --
> >>> > > --
> >>> > > Attila Sasvari
> >>> > > Software Engineer
> >>> > > <http://www.cloudera.com/>
> >>> > >
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > Peter Cseh
> >>> > Software Engineer
> >>> > <http://www.cloudera.com>
> >>> >
> >>>
> >>
> >>
> >>
> >> --
> >> --
> >> Attila Sasvari
> >> Software Engineer
> >> <http://www.cloudera.com/>
> >>
> >
> >
> >
> > --
> > --
> > Attila Sasvari
> > Software Engineer
> > <http://www.cloudera.com/>
> >
>
>
>
> --
> --
> Attila Sasvari
> Software Engineer
> <http://www.cloudera.com/>
>
