Re: [I] [Question] When I serialize this object, enough space is allocated, but it still throws a java.lang.OutOfMemoryError: Java heap space [fury]</span></a></span> </h1> <p class="darkgray font13"> <span class="sender pipe"><a href="/search?l=commits@fury.apache.org&q=from:%22via+GitHub%22" rel="nofollow"><span itemprop="author" itemscope itemtype="http://schema.org/Person"><span itemprop="name">via GitHub</span></span></a></span> <span class="date"><a href="/search?l=commits@fury.apache.org&q=date:20241126" rel="nofollow">Tue, 26 Nov 2024 08:41:16 -0800</a></span> </p> </div> <div itemprop="articleBody" class="msgBody"> <!--X-Body-of-Message--> <pre> chaokunyang commented on issue #1958: URL: <a rel="nofollow" href="https://github.com/apache/fury/issues/1958#issuecomment-2501365172">https://github.com/apache/fury/issues/1958#issuecomment-2501365172</a></pre><pre> @a1342772 Fury supports zero-copy serialization of primitive array, there is no cost for serializing such objects, and of course no compression, the serialized size of array will be `n_elements * size_of(element_type)`. You could use zero-copy serialization by <a rel="nofollow" href="https://fury.apache.org/docs/guide/java_object_graph_guide#zero-copy-serialization:">https://fury.apache.org/docs/guide/java_object_graph_guide#zero-copy-serialization:</a> ```java import org.apache.fury.*; import org.apache.fury.config.*; import org.apache.fury.serializer.BufferObject; import org.apache.fury.memory.MemoryBuffer; import java.util.*; import java.util.stream.Collectors; public class ZeroCopyExample { // Note that fury instance should be reused instead of creation every time. static Fury fury = Fury.builder() .withLanguage(Language.JAVA) .build(); // mvn exec:java -Dexec.mainClass="io.ray.fury.examples.ZeroCopyExample" public static void main(String[] args) { List<Object> list = Arrays.asList("str", new byte[1000], new int[100], new double[100]); Collection<BufferObject> bufferObjects = new ArrayList<>(); byte[] bytes = fury.serialize(list, e -> !bufferObjects.add(e)); bufferObjects. .forEach(buf -> buf.writeTo(...)).collect(Collectors.toList()); System.out.println(fury.deserialize(bytes, buffers)); } } ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org </pre> </div> <div class="msgButtons margintopdouble"> <ul class="overflow"> <li class="msgButtonItems"><a class="button buttonleft " accesskey="p" href="msg06204.html">Previous message</a></li> <li class="msgButtonItems textaligncenter"><a class="button" accesskey="c" href="thrd11.html#06205">View by thread</a></li> <li class="msgButtonItems textaligncenter"><a class="button" accesskey="i" href="mail14.html#06205">View by date</a></li> <li class="msgButtonItems textalignright"><a class="button buttonright " accesskey="n" href="msg06206.html">Next message</a></li> </ul> </div> <a name="tslice"></a> <div class="tSliceList margintopdouble"> <ul class="icons monospace"> <li class="icons-email"><span class="subject"><a href="msg06195.html">[I] [Question] <title>Can MemoryBuffer achieve a smaller ...</a></span> <span class="sender italic">via GitHub</span></li> <li><ul> <li class="icons-email"><span class="subject"><a href="msg06196.html">Re: [I] [Question] <title>When I serialize this obje...</a></span> <span class="sender italic">via GitHub</span></li> <li class="icons-email"><span class="subject"><a href="msg06197.html">Re: [I] [Question] <title>When I serialize this obje...</a></span> <span class="sender italic">via GitHub</span></li> <li class="icons-email"><span class="subject"><a href="msg06198.html">Re: [I] [Question] <title>When I serialize this obje...</a></span> <span class="sender italic">via GitHub</span></li> <li class="icons-email"><span class="subject"><a href="msg06199.html">Re: [I] [Question] <title>When I serialize this obje...</a></span> <span class="sender italic">via GitHub</span></li> <li class="icons-email"><span class="subject"><a href="msg06200.html">Re: [I] [Question] <title>When I serialize this obje...</a></span> <span class="sender italic">via GitHub</span></li> <li class="icons-email"><span class="subject"><a href="msg06201.html">Re: [I] [Question] <title>When I serialize this obje...</a></span> <span class="sender italic">via GitHub</span></li> <li class="icons-email"><span class="subject"><a href="msg06202.html">Re: [I] [Question] <title>When I serialize this obje...</a></span> <span class="sender italic">via GitHub</span></li> <li class="icons-email"><span class="subject"><a href="msg06203.html">Re: [I] [Question] <title>When I serialize this obje...</a></span> <span class="sender italic">via GitHub</span></li> <li class="icons-email"><span class="subject"><a href="msg06204.html">Re: [I] [Question] <title>When I serialize this obje...</a></span> <span class="sender italic">via GitHub</span></li> <li class="icons-email tSliceCur"><span class="subject">Re: [I] [Question] <title>When I serialize this obje...</span> <span class="sender italic">via GitHub</span></li> </ul> </ul> </div> <div class="overflow msgActions margintopdouble"> <div class="msgReply" > <h2> Reply via email to </h2> <form method="POST" action="/mailto.php"> <input type="hidden" name="subject" value="Re: [I] [Question] <title>When I serialize this object, enough space is allocated, but it still throws a java.lang.OutOfMemoryError: Java heap space [fury]"> <input type="hidden" name="msgid" value="I_kwDOJcVKYc6gg9wC-5b770bff-8c24-480a-8bfb-a6e2b99b4e51@gitbox.apache.org"> <input type="hidden" name="relpath" value="commits@fury.apache.org/msg06205.html"> <input type="submit" value=" via GitHub "> </form> </div> </div> </div> <div class="aside" role="complementary"> <div class="logo"> <a href="/"><img src="/logo.png" width=247 height=88 alt="The Mail Archive"></a> </div> <form class="overflow" action="/search" method="get"> <input type="hidden" name="l" value="commits@fury.apache.org"> <label class="hidden" for="q">Search the site</label> <input class="submittext" type="text" id="q" name="q" placeholder="Search commits"> <input class="submitbutton" name="submit" type="image" src="/submit.png" alt="Submit"> </form> <div class="nav margintop" id="nav" role="navigation"> <ul class="icons font16"> <li class="icons-home"><a href="/">The Mail Archive home</a></li> <li class="icons-list"><a href="/commits@fury.apache.org/">commits - all messages</a></li> <li class="icons-about"><a href="/commits@fury.apache.org/info.html">commits - about the list</a></li> <li class="icons-expand"><a href="/search?l=commits@fury.apache.org&q=subject:%22Re%5C%3A+%5C%5BI%5C%5D+%5C%5BQuestion%5C%5D+%3Ctitle%3EWhen+I+serialize+this+object%2C+enough+space+is+allocated%2C+but+it+still+throws+a+java.lang.OutOfMemoryError%5C%3A+Java+heap+space+%5C%5Bfury%5C%5D%22&o=newest&f=1" title="e" id="e">Expand</a></li> <li class="icons-prev"><a href="msg06204.html" title="p">Previous message</a></li> <li class="icons-next"><a href="msg06206.html" title="n">Next message</a></li> </ul> </div> <div class="listlogo margintopdouble"> </div> <div class="margintopdouble"> </div> </div> </div> <div class="footer" role="contentinfo"> <ul> <li><a href="/">The Mail Archive home</a></li> <li><a href="/faq.html#newlist">Add your mailing list</a></li> <li><a href="/faq.html">FAQ</a></li> <li><a href="/faq.html#support">Support</a></li> <li><a href="/faq.html#privacy">Privacy</a></li> <li class="darkgray">I_kwDOJcVKYc6gg9wC-5b770bff-8c24-480a-8bfb-a6e2b99b4e51@gitbox.apache.org</li> </ul> </div> </body> </html> <script>(function(){function c(){var b=a.contentDocument||a.contentWindow.document;if(b){var d=b.createElement('script');d.innerHTML="window.__CF$cv$params={r:'9d17f89c086f5cd4',t:'MTc3MTY5NTAyMQ=='};var a=document.createElement('script');a.src='/cdn-cgi/challenge-platform/scripts/jsd/main.js';document.getElementsByTagName('head')[0].appendChild(a);";b.getElementsByTagName('head')[0].appendChild(d)}}if(document.body){var a=document.createElement('iframe');a.height=1;a.width=1;a.style.position='absolute';a.style.top=0;a.style.left=0;a.style.border='none';a.style.visibility='hidden';document.body.appendChild(a);if('loading'!==document.readyState)c();else if(window.addEventListener)document.addEventListener('DOMContentLoaded',c);else{var e=document.onreadystatechange||function(){};document.onreadystatechange=function(b){e(b);'loading'!==document.readyState&&(document.onreadystatechange=e,c())}}}})();</script>