On Wed, Aug 22, 2018 at 6:17 PM, Tom Pantelis <tompante...@gmail.com> wrote:
>
>
> On Wed, Aug 22, 2018 at 4:57 PM, Jamo Luhrsen <jluhr...@gmail.com> wrote:
>
>> I am looking for some help/ideas. I am running three containers
>> on my laptop so I can test some cluster bugs locally. I am fine
>> with the netty based akka remoting (our default), but I've been
>> asked to reproduce a bug with the artery remoting.
>>
>> Every time I start my controllers with artery, they all die with:
>>
>> ERROR | opendaylight-cluster-data-akka.actor.default-dispatcher-56 |
>> ActorSystemImpl | 41 -
>> com.typesafe.akka.slf4j - 2.5.11 | Uncaught error from thread
>> [opendaylight-cluster-data-akka.remote.default-remote-dispatcher-7]: Di
>> rect buffer memory, shutting down JVM since
>> 'akka.jvm-exit-on-fatal-error' is enabled for ActorSystem[opendaylight-clust
>> er-data]
>> java.lang.OutOfMemoryError: Direct buffer memory
>> at java.nio.Bits.reserveMemory(Bits.java:695) ~[?:?]
>> at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
>> ~[?:?]
>> at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311) ~[?:?]
>> at
>> akka.remote.artery.EnvelopeBufferPool.acquire(EnvelopeBufferPool.scala:34)
>> ~[40:com.typesafe.akka.remote:2.5.11]
>> at akka.remote.artery.Encoder$$anon$2.onPush(Codecs.scala:93)
>> ~[40:com.typesafe.akka.remote:2.5.11]
>>
>>
>> I'm stuck in the mud on this one so far. I'm trying to tweak the
>> shared memory setting in the docker run command (to no avail).
>>
>> The container stats are nowhere near any kind of limit.
>>
>> It's not happening with netty based.
>>
>> I've tried both udp and tcp protocols with the same OOM death.
>>
>> appreciate any ideas or pointers. I'm literally just throwing mud against
>> the wall at this point hoping something will stick.
>>
>> TomP tells me it works on his laptop, but he's not using
>> docker/containers,
>> so that must be a clue.
>>
>
> The problem is this in configuration/factory/akka.conf:
>
> artery {
> advanced {
> maximum-frame-size = 1 GiB
> maximum-large-frame-size = 1 GiB
> }
> }
>
> I had set these originally a while ago when we were first looking at
> artery before I realized it was using direct memory. I forgot to remove
> them. I'll submit a patch to do that - in the mean time you can remove it
> locally.
>
I think it was loading fine on my system fine b/c I have single node and
the code that blows up gets executed when it needs to do remoting.
>
>
>
>>
>> Thanks,
>> JamO
>>
>
>
_______________________________________________
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev
