Re: OutOfMemoryError: Java heap space when using imapsync to transfer mail to James
Hello, https://github.com/apache/james-project/pull/1579 should fix the issue. Best regards. Benoit On 02/06/2023 10:21, Benoit TELLIER wrote: Hello Wojtek, I am taking a look into this, running a JPA james server localy, and synchronizing emails with a remote server, I also notice a memory leak. I opened an issue to track this: https://issues.apache.org/jira/browse/JAMES-3911 I suspect https://github.com/apache/james-project/pull/1574 could be related... Best regards On 02/06/2023 08:58, Wojtek wrote: Hi, I was attempting to transfer data from my old account to James and ran into an issue: "OutOfMemoryError: Java heap space". Upon closer inspection, James was using 4,7g virt / 2,3g res memory (system usage of the generated docker image was 7,6g with 0,1g free reported). This is a fairly simple setup of james-jpa backed by MySQL. Has anyone ran into similar problem? imap sync seemed to process messages one-by-one, so the load wasn't great. I'm currently collecting heapdump. Error stacktraces entry seems to point to MySQL/openJPA - one assumption would be that James itself can process incomming messages faster than the repository can process/store them? Isn't there a feature that would slow down processing incomming messages in thise case? 21:44:10.161 [WARN ] r.c.Exceptions - throwIfFatal detected a jvm fatal exception, which is thrown and logged below: java.lang.OutOfMemoryError: Java heap space at java.base/java.math.BigInteger.shiftRightImpl(Unknown Source) at java.base/java.math.BigInteger.shiftRight(Unknown Source) at java.base/sun.security.util.math.intpoly.IntegerPolynomial.setLimbsValuePositive(Unknown Source) at java.base/sun.security.util.math.intpoly.IntegerPolynomial.setLimbsValue(Unknown Source) at java.base/sun.security.util.math.intpoly.IntegerPolynomial$Element.setValue(Unknown Source) at java.base/sun.security.util.math.intpoly.IntegerPolynomial$Element.(Unknown Source) at java.base/sun.security.util.math.intpoly.IntegerPolynomial$ImmutableElement.(Unknown Source) at java.base/sun.security.util.math.intpoly.IntegerPolynomial.getElement(Unknown Source) at java.base/sun.security.util.math.intpoly.IntegerPolynomial.getElement(Unknown Source) at jdk.crypto.ec/sun.security.ec.XECOperations.encodedPointMultiply(Unknown Source) at jdk.crypto.ec/sun.security.ec.XDHKeyAgreement.engineDoPhase(Unknown Source) at java.base/javax.crypto.KeyAgreement.doPhase(Unknown Source) at java.base/sun.security.ssl.KAKeyDerivation.t13DeriveKey(Unknown Source) at java.base/sun.security.ssl.KAKeyDerivation.deriveKey(Unknown Source) at java.base/sun.security.ssl.ServerHello$T13ServerHelloConsumer.consume(Unknown Source) at java.base/sun.security.ssl.ServerHello$ServerHelloConsumer.onServerHello(Unknown Source) at java.base/sun.security.ssl.ServerHello$ServerHelloConsumer.consume(Unknown Source) at java.base/sun.security.ssl.SSLHandshake.consume(Unknown Source) at java.base/sun.security.ssl.HandshakeContext.dispatch(Unknown Source) at java.base/sun.security.ssl.HandshakeContext.dispatch(Unknown Source) at java.base/sun.security.ssl.TransportContext.dispatch(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at com.mysql.cj.protocol.ExportControlled.performTlsHandshake(ExportControlled.java:347) at com.mysql.cj.protocol.StandardSocketFactory.performTlsHandshake(StandardSocketFactory.java:194) at com.mysql.cj.protocol.a.NativeSocketConnection.performTlsHandshake(NativeSocketConnection.java:101) at com.mysql.cj.protocol.a.NativeProtocol.negotiateSSLConnection(NativeProtocol.java:308) at com.mysql.cj.protocol.a.NativeAuthenticationProvider.connect(NativeAuthenticationProvider.java:204) at com.mysql.cj.protocol.a.NativeProtocol.connect(NativeProtocol.java:1369) 21:44:14.037 [ERROR] r.c.s.Schedulers - Scheduler worker in group main failed with an uncaught exception java.lang.OutOfMemoryError: Java heap space at java.base/java.math.BigInteger.shiftRightImpl(Unknown Source) at java.base/java.math.BigInteger.shiftRight(Unknown Source) at java.base/sun.security.util.math.intpoly.IntegerPolynomial.setLimbsValuePositive(Unknown Source) at java.base/sun.security.util.math.intpoly.IntegerPolynomial.setLimbsValue(Unknown Source) at java.base/sun.security.util.math.intpoly.IntegerPolynomial$Element.setValue(Unknown Source) at
Re: OutOfMemoryError: Java heap space when using imapsync to transfer mail to James
Hello Wojtek, I am taking a look into this, running a JPA james server localy, and synchronizing emails with a remote server, I also notice a memory leak. I opened an issue to track this: https://issues.apache.org/jira/browse/JAMES-3911 I suspect https://github.com/apache/james-project/pull/1574 could be related... Best regards On 02/06/2023 08:58, Wojtek wrote: Hi, I was attempting to transfer data from my old account to James and ran into an issue: "OutOfMemoryError: Java heap space". Upon closer inspection, James was using 4,7g virt / 2,3g res memory (system usage of the generated docker image was 7,6g with 0,1g free reported). This is a fairly simple setup of james-jpa backed by MySQL. Has anyone ran into similar problem? imap sync seemed to process messages one-by-one, so the load wasn't great. I'm currently collecting heapdump. Error stacktraces entry seems to point to MySQL/openJPA - one assumption would be that James itself can process incomming messages faster than the repository can process/store them? Isn't there a feature that would slow down processing incomming messages in thise case? 21:44:10.161 [WARN ] r.c.Exceptions - throwIfFatal detected a jvm fatal exception, which is thrown and logged below: java.lang.OutOfMemoryError: Java heap space at java.base/java.math.BigInteger.shiftRightImpl(Unknown Source) at java.base/java.math.BigInteger.shiftRight(Unknown Source) at java.base/sun.security.util.math.intpoly.IntegerPolynomial.setLimbsValuePositive(Unknown Source) at java.base/sun.security.util.math.intpoly.IntegerPolynomial.setLimbsValue(Unknown Source) at java.base/sun.security.util.math.intpoly.IntegerPolynomial$Element.setValue(Unknown Source) at java.base/sun.security.util.math.intpoly.IntegerPolynomial$Element.(Unknown Source) at java.base/sun.security.util.math.intpoly.IntegerPolynomial$ImmutableElement.(Unknown Source) at java.base/sun.security.util.math.intpoly.IntegerPolynomial.getElement(Unknown Source) at java.base/sun.security.util.math.intpoly.IntegerPolynomial.getElement(Unknown Source) at jdk.crypto.ec/sun.security.ec.XECOperations.encodedPointMultiply(Unknown Source) at jdk.crypto.ec/sun.security.ec.XDHKeyAgreement.engineDoPhase(Unknown Source) at java.base/javax.crypto.KeyAgreement.doPhase(Unknown Source) at java.base/sun.security.ssl.KAKeyDerivation.t13DeriveKey(Unknown Source) at java.base/sun.security.ssl.KAKeyDerivation.deriveKey(Unknown Source) at java.base/sun.security.ssl.ServerHello$T13ServerHelloConsumer.consume(Unknown Source) at java.base/sun.security.ssl.ServerHello$ServerHelloConsumer.onServerHello(Unknown Source) at java.base/sun.security.ssl.ServerHello$ServerHelloConsumer.consume(Unknown Source) at java.base/sun.security.ssl.SSLHandshake.consume(Unknown Source) at java.base/sun.security.ssl.HandshakeContext.dispatch(Unknown Source) at java.base/sun.security.ssl.HandshakeContext.dispatch(Unknown Source) at java.base/sun.security.ssl.TransportContext.dispatch(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at com.mysql.cj.protocol.ExportControlled.performTlsHandshake(ExportControlled.java:347) at com.mysql.cj.protocol.StandardSocketFactory.performTlsHandshake(StandardSocketFactory.java:194) at com.mysql.cj.protocol.a.NativeSocketConnection.performTlsHandshake(NativeSocketConnection.java:101) at com.mysql.cj.protocol.a.NativeProtocol.negotiateSSLConnection(NativeProtocol.java:308) at com.mysql.cj.protocol.a.NativeAuthenticationProvider.connect(NativeAuthenticationProvider.java:204) at com.mysql.cj.protocol.a.NativeProtocol.connect(NativeProtocol.java:1369) 21:44:14.037 [ERROR] r.c.s.Schedulers - Scheduler worker in group main failed with an uncaught exception java.lang.OutOfMemoryError: Java heap space at java.base/java.math.BigInteger.shiftRightImpl(Unknown Source) at java.base/java.math.BigInteger.shiftRight(Unknown Source) at java.base/sun.security.util.math.intpoly.IntegerPolynomial.setLimbsValuePositive(Unknown Source) at java.base/sun.security.util.math.intpoly.IntegerPolynomial.setLimbsValue(Unknown Source) at java.base/sun.security.util.math.intpoly.IntegerPolynomial$Element.setValue(Unknown Source) at java.base/sun.security.util.math.intpoly.IntegerPolynomial$Element.(Unknown Source) at
