Thanks for the feedback Clebert! >I think we should use some sort of pool on messages. Reusing the big >pools before the messages are released. It could work! Just thinking loud; in order to implement this pool efficiently my advice is to not use a solution similar to the Netty pool, in which the object lifecycle could span between threads: Netty itself shows that it doesn't scale well for high throughput scenarios and risks to be error prone on leaking messages (it needs to implement a leak detector etc etc). I've already built some experiments around this approach if anyone is interested in the results, but could worth try it too!
Instead I'm proposing another form of pooling, that will provide the same effect of recycling memory but as a consequence and not as a mean: bounded queues between the message process stages. Having bounded concurrent queues will provide the pooling for free and a mean to measure the utilization between all the concurrent stages (eg incoming,encoding, journaling ,decoding, outgoing etc etc) recognizing when the system is overloaded in any of its part and letting the dev/user to tune it consequently. >One of the changes as part of my big PR was to not duplicate buffers >on every deliver.. that alone should release some pressure on GC for >cases where the message is delivered to more than one destination >(topic subscriptions). And it is a super plus to improve the throughput for sure (I've already benchmarked it, delivering outstanding improvements indeed ): bound the memory and the max latency is only an additional tool to exploit further this improved throughput, making it sustainable. >what would you change here? what would you use for such things? I would bound first (putting a bounded queue in front of) the stages that could be slowed down by external factors (like I/O speeds), that need to inform the previous ones of being overloaded and that could benefit by any form a batching: 1) the incoming/outcoming data phase of the connections (InVM or Netty) 2) the management of the journal (the write operations in particular) makes sense? -- View this message in context: http://activemq.2283324.n4.nabble.com/DISCUSS-OutOfDirectMemory-on-high-memory-pressure-4-NettyConnection-tp4722964p4722999.html Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
