[
https://issues.apache.org/jira/browse/IGNITE-12543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054655#comment-17054655
]
Sunghan Suh edited comment on IGNITE-12543 at 3/9/20, 4:08 AM:
---------------------------------------------------------------
Hello, [~Pavlukhin].
I'm a leader of an open source development group in Samsung.
As [~redcomet] explained, we have been suffering from this issue in a mission
critical system.
I know limitations in the previous comment,
but the patch we have uploaded can fix our problem with ease and it doesn't
affect other codes.
Please review [GitHub Pull Request
#7403|https://github.com/apache/ignite/pull/7403] positively, and let me end
this issue gracefully.
Thank you for your times.
was (Author: sunghan.suh):
Hello, [~Pavlukhin].
I'm a leader of an open source development group in Samsung.
As [~redcomet] explained, we suffered from this issue in a mission critical
system.
I know limitations in the previous comment,
but the patch we have uploaded can fix our prolem with ease and it doesn't
affect other codes.
Please review [GitHub Pull Request
#7403|https://github.com/apache/ignite/pull/7403] positively, and let us end it
gracefully.
Thank you for your times.
> When put List<List<SomeObject>>, the data was increased much larger.
> --------------------------------------------------------------------
>
> Key: IGNITE-12543
> URL: https://issues.apache.org/jira/browse/IGNITE-12543
> Project: Ignite
> Issue Type: Bug
> Components: thin client
> Affects Versions: 2.6
> Reporter: LEE PYUNG BEOM
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I use Ignite 2.6 version of Java Thin Client.
>
> When I put data in the form List<List<SomeObject>>,
> The size of the original 200KB data was increased to 50MB when inquired by
> Ignite servers.
> On the Heap Dump, the list element was repeatedly accumulated, increasing the
> data size.
>
> When I checked org.apacheignite.internal.binary.BinaryWriterExImpl.java
> doWriteBinaryObject() method,
> {code:java}
> // org.apacheignite.internal.binary.BinaryWriterExImpl.java
> public void doWriteBinaryObject(@Nullable BinaryObjectImpl po) {
> if (po == null)
> out.writeByte(GridBinaryMarshaller.NULL);
> else {
> byte[] poArr = po.array();
> out.unsafeEnsure(1 + 4 + poArr.length +4);
> out.unsafeWriteByte(GridBinaryMarshaller.BINARY_OBJ);
> out.unsafeWriteInt(poArr.length);
> out.writeByteArray(poArr);
> out.unsafeWriteInt(po.start());
> }
> }
> {code}
>
> The current Ignite implementation for storing data in the form
> List<List<Some_Objectject>> is:
> In the Marshalling stage, for example, data the size of List(5
> members)<List(10 members)<Some_Object(size:200 KB)> is:
> As many as 10*5 of the list's elements are duplicated.
> If the above data contains five objects of 200KB size, ten by one,
> 50 iterations are stored and 200K*10**5 = 100MB of data is used for cache and
> transfer.
> As a result of this increase in data size, it is confirmed that the failure
> of OOM, GC, etc. is caused by occupying Heap memory.
> Unnecessarily redundant data is used for cache storage and network transport.
> When looking up cache data, only some of the data at the top is read based on
> file location information from the entire data, so that normal data is
> retrieved.
> The way we're implemented today is safe from basic behavior, but we're
> wasting memory and network unnecessarily using inefficient algorithms
> This can have very serious consequences. Please check.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
