Re: Java compiler OOMs on Jenkins/Gradle

[Lukasz Cwik]

Try running without a daemon (use flag --no-daemon) to see if its an issue
with the gradle daemon you have been using isn't overloaded.

On Wed, May 30, 2018 at 5:11 PM Ankur Goenka  wrote:

> I am facing OOM while locally building the project using Gradle. Here is
> the scan https://scans.gradle.com/s/t3n42rw5666us
> The issue is happening from :rat task.
> Is this issue related?
>
> On Tue, May 1, 2018 at 4:40 PM Scott Wegner  wrote:
>
>> Sorry about the instability. We need to get the Gradle jobs tuned for our
>> Jenkins machines, and there's no way to test my configuration changes
>> without affecting all jobs :-/
>>
>> The changes I'm making are here: https://github.com/apache/beam/pull/5218
>>
>>
>> It seems that they're still not quite right: the intent is to allocate
>> half the memory to each job, and then divide it up by worker. But for 16
>> workers each is getting ~1GB, even though the machines have 100GB total. I
>> suspect I'm just calling the wrong API.
>>
>> On Tue, May 1, 2018 at 1:41 PM Eugene Kirpichov 
>> wrote:
>>
>>> Thanks! FWIW seems that my other Jenkins build is about to fail with the
>>> same issue
>>> https://builds.apache.org/job/beam_PreCommit_Java_GradleBuild/4806/ -
>>> "Expiring Daemon because JVM Tenured space is exhausted"
>>>
>>> On Tue, May 1, 2018 at 1:36 PM Lukasz Cwik  wrote:
>>>
 +sweg...@google.com who is currently messing around with tuning some
 Gradle flags related to the JVM and its memory usage.

 On Tue, May 1, 2018 at 1:34 PM Eugene Kirpichov 
 wrote:

> Hi,
>
> I've seen the same issue twice in a row on PR
> https://github.com/apache/beam/pull/4264 : the Java precommit fails
> with messages like:
>
> > Task :beam-sdks-java-core:compileTestJava
> An exception has occurred in the compiler ((version info not
> available)). Please file a bug against the Java compiler via the Java bug
> reporting page (http://bugreport.java.com) after checking the Bug
> Database (http://bugs.java.com) for duplicates. Include your program
> and the following diagnostic in your report. Thank you.
> java.lang.OutOfMemoryError: GC overhead limit exceeded
>
> Full build link:
> https://builds.apache.org/job/beam_PreCommit_Java_GradleBuild/4803/consoleFull
>
> Anybody know what's up with that? I thought we got new powerful
> Jenkins executors and we shouldn't be running out of memory? However, I 
> see
> that the build specifies* -Dorg.gradle.jvmargs=-Xmx512m* - this seems
> too small. Should we increase this?
>
> Thanks.
>
 --
>>
>>
>> Got feedback? http://go/swegner-feedback
>>
>

Re: Java compiler OOMs on Jenkins/Gradle

[Ankur Goenka]

I am facing OOM while locally building the project using Gradle. Here is
the scan https://scans.gradle.com/s/t3n42rw5666us
The issue is happening from :rat task.
Is this issue related?

On Tue, May 1, 2018 at 4:40 PM Scott Wegner  wrote:

> Sorry about the instability. We need to get the Gradle jobs tuned for our
> Jenkins machines, and there's no way to test my configuration changes
> without affecting all jobs :-/
>
> The changes I'm making are here: https://github.com/apache/beam/pull/5218
>
> It seems that they're still not quite right: the intent is to allocate
> half the memory to each job, and then divide it up by worker. But for 16
> workers each is getting ~1GB, even though the machines have 100GB total. I
> suspect I'm just calling the wrong API.
>
> On Tue, May 1, 2018 at 1:41 PM Eugene Kirpichov 
> wrote:
>
>> Thanks! FWIW seems that my other Jenkins build is about to fail with the
>> same issue
>> https://builds.apache.org/job/beam_PreCommit_Java_GradleBuild/4806/ -
>> "Expiring Daemon because JVM Tenured space is exhausted"
>>
>> On Tue, May 1, 2018 at 1:36 PM Lukasz Cwik  wrote:
>>
>>> +sweg...@google.com who is currently messing around with tuning some
>>> Gradle flags related to the JVM and its memory usage.
>>>
>>> On Tue, May 1, 2018 at 1:34 PM Eugene Kirpichov 
>>> wrote:
>>>
 Hi,

 I've seen the same issue twice in a row on PR
 https://github.com/apache/beam/pull/4264 : the Java precommit fails
 with messages like:

 > Task :beam-sdks-java-core:compileTestJava
 An exception has occurred in the compiler ((version info not
 available)). Please file a bug against the Java compiler via the Java bug
 reporting page (http://bugreport.java.com) after checking the Bug
 Database (http://bugs.java.com) for duplicates. Include your program
 and the following diagnostic in your report. Thank you.
 java.lang.OutOfMemoryError: GC overhead limit exceeded

 Full build link:
 https://builds.apache.org/job/beam_PreCommit_Java_GradleBuild/4803/consoleFull

 Anybody know what's up with that? I thought we got new powerful Jenkins
 executors and we shouldn't be running out of memory? However, I see that
 the build specifies* -Dorg.gradle.jvmargs=-Xmx512m* - this seems too
 small. Should we increase this?

 Thanks.

>>> --
>
>
> Got feedback? http://go/swegner-feedback
>

Re: Java compiler OOMs on Jenkins/Gradle

[Scott Wegner]

Sorry about the instability. We need to get the Gradle jobs tuned for our
Jenkins machines, and there's no way to test my configuration changes
without affecting all jobs :-/

The changes I'm making are here: https://github.com/apache/beam/pull/5218

It seems that they're still not quite right: the intent is to allocate half
the memory to each job, and then divide it up by worker. But for 16 workers
each is getting ~1GB, even though the machines have 100GB total. I suspect
I'm just calling the wrong API.

On Tue, May 1, 2018 at 1:41 PM Eugene Kirpichov 
wrote:

> Thanks! FWIW seems that my other Jenkins build is about to fail with the
> same issue
> https://builds.apache.org/job/beam_PreCommit_Java_GradleBuild/4806/ -
> "Expiring Daemon because JVM Tenured space is exhausted"
>
> On Tue, May 1, 2018 at 1:36 PM Lukasz Cwik  wrote:
>
>> +sweg...@google.com who is currently messing around with tuning some
>> Gradle flags related to the JVM and its memory usage.
>>
>> On Tue, May 1, 2018 at 1:34 PM Eugene Kirpichov 
>> wrote:
>>
>>> Hi,
>>>
>>> I've seen the same issue twice in a row on PR
>>> https://github.com/apache/beam/pull/4264 : the Java precommit fails
>>> with messages like:
>>>
>>> > Task :beam-sdks-java-core:compileTestJava
>>> An exception has occurred in the compiler ((version info not
>>> available)). Please file a bug against the Java compiler via the Java bug
>>> reporting page (http://bugreport.java.com) after checking the Bug
>>> Database (http://bugs.java.com) for duplicates. Include your program
>>> and the following diagnostic in your report. Thank you.
>>> java.lang.OutOfMemoryError: GC overhead limit exceeded
>>>
>>> Full build link:
>>> https://builds.apache.org/job/beam_PreCommit_Java_GradleBuild/4803/consoleFull
>>>
>>> Anybody know what's up with that? I thought we got new powerful Jenkins
>>> executors and we shouldn't be running out of memory? However, I see that
>>> the build specifies* -Dorg.gradle.jvmargs=-Xmx512m* - this seems too
>>> small. Should we increase this?
>>>
>>> Thanks.
>>>
>> --


Got feedback? http://go/swegner-feedback

Re: Java compiler OOMs on Jenkins/Gradle

[Eugene Kirpichov]

Thanks! FWIW seems that my other Jenkins build is about to fail with the
same issue
https://builds.apache.org/job/beam_PreCommit_Java_GradleBuild/4806/ -
"Expiring Daemon because JVM Tenured space is exhausted"

On Tue, May 1, 2018 at 1:36 PM Lukasz Cwik  wrote:

> +sweg...@google.com who is currently messing around with tuning some
> Gradle flags related to the JVM and its memory usage.
>
> On Tue, May 1, 2018 at 1:34 PM Eugene Kirpichov 
> wrote:
>
>> Hi,
>>
>> I've seen the same issue twice in a row on PR
>> https://github.com/apache/beam/pull/4264 : the Java precommit fails with
>> messages like:
>>
>> > Task :beam-sdks-java-core:compileTestJava
>> An exception has occurred in the compiler ((version info not available)).
>> Please file a bug against the Java compiler via the Java bug reporting page
>> (http://bugreport.java.com) after checking the Bug Database (
>> http://bugs.java.com) for duplicates. Include your program and the
>> following diagnostic in your report. Thank you.
>> java.lang.OutOfMemoryError: GC overhead limit exceeded
>>
>> Full build link:
>> https://builds.apache.org/job/beam_PreCommit_Java_GradleBuild/4803/consoleFull
>>
>> Anybody know what's up with that? I thought we got new powerful Jenkins
>> executors and we shouldn't be running out of memory? However, I see that
>> the build specifies* -Dorg.gradle.jvmargs=-Xmx512m* - this seems too
>> small. Should we increase this?
>>
>> Thanks.
>>
>

Re: Java compiler OOMs on Jenkins/Gradle

[Lukasz Cwik]

+sweg...@google.com who is currently messing around with tuning some Gradle
flags related to the JVM and its memory usage.

On Tue, May 1, 2018 at 1:34 PM Eugene Kirpichov 
wrote:

> Hi,
>
> I've seen the same issue twice in a row on PR
> https://github.com/apache/beam/pull/4264 : the Java precommit fails with
> messages like:
>
> > Task :beam-sdks-java-core:compileTestJava
> An exception has occurred in the compiler ((version info not available)).
> Please file a bug against the Java compiler via the Java bug reporting page
> (http://bugreport.java.com) after checking the Bug Database (
> http://bugs.java.com) for duplicates. Include your program and the
> following diagnostic in your report. Thank you.
> java.lang.OutOfMemoryError: GC overhead limit exceeded
>
> Full build link:
> https://builds.apache.org/job/beam_PreCommit_Java_GradleBuild/4803/consoleFull
>
> Anybody know what's up with that? I thought we got new powerful Jenkins
> executors and we shouldn't be running out of memory? However, I see that
> the build specifies* -Dorg.gradle.jvmargs=-Xmx512m* - this seems too
> small. Should we increase this?
>
> Thanks.
>

Java compiler OOMs on Jenkins/Gradle

[Eugene Kirpichov]

Hi,

I've seen the same issue twice in a row on PR
https://github.com/apache/beam/pull/4264 : the Java precommit fails with
messages like:

> Task :beam-sdks-java-core:compileTestJava
An exception has occurred in the compiler ((version info not available)).
Please file a bug against the Java compiler via the Java bug reporting page
(http://bugreport.java.com) after checking the Bug Database (
http://bugs.java.com) for duplicates. Include your program and the
following diagnostic in your report. Thank you.
java.lang.OutOfMemoryError: GC overhead limit exceeded

Full build link:
https://builds.apache.org/job/beam_PreCommit_Java_GradleBuild/4803/consoleFull

Anybody know what's up with that? I thought we got new powerful Jenkins
executors and we shouldn't be running out of memory? However, I see that
the build specifies* -Dorg.gradle.jvmargs=-Xmx512m* - this seems too small.
Should we increase this?

Thanks.