Can you give me the exact version of the Ubuntu VM?
Do you have a swapfile on the Ubuntu VM? The tests might currently allocate
up to 8 GB of memory, so maybe your VM is starting to swap and thats why
the test runs out of the timeout. (That would not explain the failure on
OSX)
I've tried it with a 3 GB (w/o swap file) 14.04 Ubuntu and only one test
failed.
On Sun, Jan 25, 2015 at 12:55 PM, Fabian Hueske <fhue...@gmail.com> wrote:
> My VM is configured with 6GB and the OS X host has 16GB. In both setups the
> error was identical (with and without cleared .m2).
> If the tests pass on a 3GB Travis host, I doubt that my errors are caused
> by lack of memory.
>
> 2015-01-25 12:28 GMT+01:00 Robert Metzger <rmetz...@apache.org>:
>
> > I also had errors when running the tests in an Ubuntu virtual machine,
> > caused by limited memory resources.
> >
> > Do you think its okay to assume that we have at least 3 GB of main memory
> > available for running the tests? Thats what we have with Travis. The YARN
> > tests are very memory intensive, because YARN itself has a bunch of
> > services (resource manager, 2 node managers) and we allocate some
> > containers there, which all need memory.
> >
> > The main problem is that the amount of memory per node manager is
> > hard-coded to 4GB. I've already opened an issue in the Hadoop project and
> > will fix the issue there...
> >
> > But I think I have some ideas how to work around the memory limitations
> for
> > now.
> >
> >
> >
> > On Sat, Jan 24, 2015 at 11:47 PM, Vasiliki Kalavri <
> > vasilikikala...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > "mvn clean verify" fails for me on Ubuntu with deleted .m2 repository.
> > > I'm getting the following:
> > >
> > > Results :
> > >
> > > Failed tests:
> > > YARNSessionFIFOITCase.setup:56->YarnTestBase.startYARNWithConfig:249
> > null
> > >
> > >
> > >
> >
> YARNSessionCapacitySchedulerITCase.setup:42->YarnTestBase.startYARNWithConfig:249
> > > null
> > >
> > > Tests run: 2, Failures: 2, Errors: 0, Skipped: 0
> > >
> > > -V.
> > >
> > > On 24 January 2015 at 23:03, Fabian Hueske <fhue...@gmail.com> wrote:
> > >
> > > > The build fails also after the .m2 repository was deleted.
> > > >
> > > > Does anybody else have this problem?
> > > >
> > > > 2015-01-24 21:31 GMT+01:00 Stephan Ewen <se...@apache.org>:
> > > >
> > > > > Is this reproducible on a machine when you delete the
> .m2/repository
> > > > > directory (local maven cache) ?
> > > > >
> > > > > (I currently cannot try that because I am behind a rather
> > low-bandwith
> > > > > connection and would take very long to re-download all dependency
> > > > > artifacts)
> > > > >
> > > > > On Sat, Jan 24, 2015 at 5:54 AM, Fabian Hueske <fhue...@gmail.com>
> > > > wrote:
> > > > >
> > > > > > I just tried to build ("mvn clean install") on a fresh Ubuntu VM.
> > > Fails
> > > > > > with the same exception as natively on MacOS.
> > > > > > Something strange is going on...
> > > > > >
> > > > > > 2015-01-24 11:19 GMT+01:00 Fabian Hueske <fhue...@gmail.com>:
> > > > > >
> > > > > > > Thanks Robert! Sounds indeed like an environment problem.
> > > > > > > Will run the tests again and send you the output.
> > > > > > >
> > > > > > > 2015-01-24 11:11 GMT+01:00 Robert Metzger <rmetz...@apache.org
> >:
> > > > > > >
> > > > > > >> Okay, the tests have finished on my local machine, and they
> > > passed.
> > > > So
> > > > > > it
> > > > > > >> looks like an environment specific issue.
> > > > > > >> Maybe the log helps me already to figure out whats the issue.
> > > > > > >> We should make sure that our tests are passing on all
> platforms
> > ;)
> > > > > > >>
> > > > > > >> On Sat, Jan 24, 2015 at 11:06 AM, Robert Metzger <
> > > > rmetz...@apache.org
> > > > > >
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >> > Hi,
> > > > > > >> >
> > > > > > >> > the tests are passing on travis. Maybe its a issue with your
> > > > > > >> environment.
> > > > > > >> > I'm currently running the tests on my machine as well, just
> to
> > > > make
> > > > > > >> sure.
> > > > > > >> > I haven't ran the tests on OS X, maybe that's causing the
> > > issues.
> > > > > > >> >
> > > > > > >> > Can you send me (privately) the full output of the tests?
> > > > > > >> >
> > > > > > >> > Best,
> > > > > > >> > Robert
> > > > > > >> >
> > > > > > >> >
> > > > > > >> >
> > > > > > >> > On Sat, Jan 24, 2015 at 11:00 AM, Fabian Hueske <
> > > > fhue...@gmail.com>
> > > > > > >> wrote:
> > > > > > >> >
> > > > > > >> >> Hi Henry,
> > > > > > >> >>
> > > > > > >> >> running "mvn -DskipTests clean install" before "mvn clean
> > > > install"
> > > > > > did
> > > > > > >> not
> > > > > > >> >> fix the build for me.
> > > > > > >> >> The failing tests are also integration tests (*ITCase)
> which
> > > are
> > > > > only
> > > > > > >> >> executed in Maven's verify phase which is not triggered if
> > you
> > > > run
> > > > > > "mvn
> > > > > > >> >> clean test".
> > > > > > >> >> If I run "mvn test" without "mvn install" it fails for me
> as
> > > well
> > > > > > with
> > > > > > >> the
> > > > > > >> >> error you posted.
> > > > > > >> >>
> > > > > > >> >> So there seem to be at least two build issues with the
> > current
> > > > > > master.
> > > > > > >> >>
> > > > > > >> >> 2015-01-24 1:47 GMT+01:00 Henry Saputra <
> > > henry.sapu...@gmail.com
> > > > >:
> > > > > > >> >>
> > > > > > >> >> > Hmm, I think there could be some weird dependencies to
> get
> > > the
> > > > > > Flink
> > > > > > >> >> > YARN uber jar.
> > > > > > >> >> >
> > > > > > >> >> > If you do "mvn clean install -DskipTests" then call "mvn
> > > test"
> > > > > all
> > > > > > >> the
> > > > > > >> >> > tests passed.
> > > > > > >> >> >
> > > > > > >> >> > But if you directly call "mvn clean test" then you see
> the
> > > > stack
> > > > > I
> > > > > > >> >> > have seen before.
> > > > > > >> >> >
> > > > > > >> >> > - Henry
> > > > > > >> >> >
> > > > > > >> >> >
> > > > > > >> >> > On Fri, Jan 23, 2015 at 3:35 PM, Henry Saputra <
> > > > > > >> henry.sapu...@gmail.com
> > > > > > >> >> >
> > > > > > >> >> > wrote:
> > > > > > >> >> > > Did not see that trace but do see this:
> > > > > > >> >> > >
> > > > > > >> >> > > -------------------------------------------------------
> > > > > > >> >> > >
> > > > > > >> >> > > T E S T S
> > > > > > >> >> > >
> > > > > > >> >> > > -------------------------------------------------------
> > > > > > >> >> > >
> > > > > > >> >> > > Running org.apache.flink.yarn.UtilsTest
> > > > > > >> >> > >
> > > > > > >> >> > > log4j:WARN No such property [append] in
> > > > > > >> >> org.apache.log4j.ConsoleAppender.
> > > > > > >> >> > >
> > > > > > >> >> > > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time
> > > > elapsed:
> > > > > > >> 0.476
> > > > > > >> >> > > sec <<< FAILURE! - in org.apache.flink.yarn.UtilsTest
> > > > > > >> >> > >
> > > > > > >> >> > > testUberjarLocator(org.apache.flink.yarn.UtilsTest)
> Time
> > > > > > elapsed:
> > > > > > >> >> > > 0.405 sec <<< FAILURE!
> > > > > > >> >> > >
> > > > > > >> >> > > java.lang.AssertionError: null
> > > > > > >> >> > >
> > > > > > >> >> > > at org.junit.Assert.fail(Assert.java:86)
> > > > > > >> >> > >
> > > > > > >> >> > > at org.junit.Assert.assertTrue(Assert.java:41)
> > > > > > >> >> > >
> > > > > > >> >> > > at org.junit.Assert.assertNotNull(Assert.java:621)
> > > > > > >> >> > >
> > > > > > >> >> > > at org.junit.Assert.assertNotNull(Assert.java:631)
> > > > > > >> >> > >
> > > > > > >> >> > > at
> > > > > > >> >>
> > > > >
> org.apache.flink.yarn.UtilsTest.testUberjarLocator(UtilsTest.java:32)
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > > Results :
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > > Failed tests:
> > > > > > >> >> > >
> > > > > > >> >> > > UtilsTest.testUberjarLocator:32 null
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > > - Henry
> > > > > > >> >> > >
> > > > > > >> >> > > On Fri, Jan 23, 2015 at 2:16 PM, Fabian Hueske <
> > > > > > fhue...@gmail.com>
> > > > > > >> >> > wrote:
> > > > > > >> >> > >> Hi all,
> > > > > > >> >> > >>
> > > > > > >> >> > >> I tried to build the current master (mvn clean
> install)
> > > and
> > > > > some
> > > > > > >> >> tests
> > > > > > >> >> > in
> > > > > > >> >> > >> the flink-yarn-tests module fail:
> > > > > > >> >> > >>
> > > > > > >> >> > >> Failed tests:
> > > > > > >> >> > >>
> > > > > > >> >> > >>
> > > > > > >> >> > >>
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> YARNSessionCapacitySchedulerITCase.testClientStartup:50->YarnTestBase.runWithArgs:314
> > > > > > >> >> > >> During the timeout period of 60 seconds the expected
> > > string
> > > > > did
> > > > > > >> not
> > > > > > >> >> > show up
> > > > > > >> >> > >>
> > > > > > >> >> > >>
> > > > > > >> >>
> > > > > >
> > YARNSessionCapacitySchedulerITCase>YarnTestBase.checkClusterEmpty:146
> > > > > > >> >> > >> There is at least one application on the cluster is
> not
> > > > > finished
> > > > > > >> >> > >>
> > > > > > >> >> > >>
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >>
> > > > > >
> > > >
> > YARNSessionFIFOITCase.perJobYarnCluster:184->YarnTestBase.runWithArgs:314
> > > > > > >> >> > >> During the timeout period of 60 seconds the expected
> > > string
> > > > > did
> > > > > > >> not
> > > > > > >> >> > show up
> > > > > > >> >> > >>
> > > > > > >> >> > >>
> > YARNSessionFIFOITCase>YarnTestBase.checkClusterEmpty:146
> > > > > There
> > > > > > >> is
> > > > > > >> >> at
> > > > > > >> >> > >> least one application on the cluster is not finished
> > > > > > >> >> > >>
> > > > > > >> >> > >>
> > YARNSessionFIFOITCase>YarnTestBase.checkClusterEmpty:146
> > > > > There
> > > > > > >> is
> > > > > > >> >> at
> > > > > > >> >> > >> least one application on the cluster is not finished
> > > > > > >> >> > >>
> > > > > > >> >> > >>
> > YARNSessionFIFOITCase>YarnTestBase.checkClusterEmpty:146
> > > > > There
> > > > > > >> is
> > > > > > >> >> at
> > > > > > >> >> > >> least one application on the cluster is not finished
> > > > > > >> >> > >>
> > > > > > >> >> > >>
> > YARNSessionFIFOITCase>YarnTestBase.checkClusterEmpty:146
> > > > > There
> > > > > > >> is
> > > > > > >> >> at
> > > > > > >> >> > >> least one application on the cluster is not finished
> > > > > > >> >> > >>
> > > > > > >> >> > >>
> > YARNSessionFIFOITCase>YarnTestBase.checkClusterEmpty:146
> > > > > There
> > > > > > >> is
> > > > > > >> >> at
> > > > > > >> >> > >> least one application on the cluster is not finished
> > > > > > >> >> > >>
> > > > > > >> >> > >>
> > YARNSessionFIFOITCase>YarnTestBase.checkClusterEmpty:146
> > > > > There
> > > > > > >> is
> > > > > > >> >> at
> > > > > > >> >> > >> least one application on the cluster is not finished
> > > > > > >> >> > >>
> > > > > > >> >> > >>
> > YARNSessionFIFOITCase>YarnTestBase.checkClusterEmpty:146
> > > > > There
> > > > > > >> is
> > > > > > >> >> at
> > > > > > >> >> > >> least one application on the cluster is not finished
> > > > > > >> >> > >>
> > > > > > >> >> > >>
> > > > > > >> >> > >> Tests run: 10, Failures: 10, Errors: 0, Skipped: 0
> > > > > > >> >> > >>
> > > > > > >> >> > >> Anybody else got this problem?
> > > > > > >> >> > >>
> > > > > > >> >> > >> Cheers, Fabian
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
