[
https://issues.apache.org/jira/browse/IGNITE-6898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16251157#comment-16251157
]
Yakov Zhdanov commented on IGNITE-6898:
---------------------------------------
[~sbberkov] perNodeParallelOperations seems to be deprecated. Also, don't you
think that the same situation is possible if multiple clients do putAlls with
large key count at the same time?
Suggested solutions do not seem sufficient to me - literally they change
nothing.
I think that server node should drop data streamer request and send error
response to client to forcibly lower the limit for parallel requests at least
for peak load duration. Once load on server decreases - server sends flag to
client with some response and client puts the limit for parallel requests back.
How about that?
Yakov
> Datastreamers can lead to OOM on server side
> --------------------------------------------
>
> Key: IGNITE-6898
> URL: https://issues.apache.org/jira/browse/IGNITE-6898
> Project: Ignite
> Issue Type: Bug
> Security Level: Public(Viewable by anyone)
> Components: general
> Affects Versions: 2.1
> Reporter: Alexander Belyak
>
> If grid server node process many datastreamer in same time (from many
> clients, with many cache backups and persistence, i.e. if processing take
> some time) it can lead to OutOfMemoryError in server JVM. To fix we can:
> 1) specify buffer sized in bytes instead of entries
> 2) use pageMemory to store streamer buffers
> I get this problem on 16 server node grid with 45g heap each and 15 clients
> with 2 datastreamer each with this settings:
> autoFlushFrequency=0
> allowOverwrite=false
> perNodeParallelOperations=8
> perNodeBufferSize=10000
> Each client have 64g heap.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
