https://bz.apache.org/bugzilla/show_bug.cgi?id=62328
--- Comment #2 from [email protected] --- The amounts of data we're processing don't go into millions of rows, but there's enough to blow out 8G heap with out of memory errors, with the heapdump showing an overwhelming majority of objects being of two types: org.apache.xmlbeans.impl.store.Xobj$ElementXobj and org.apache.xmlbeans.impl.store.Xobj$AttrXobj as well as TreeMap entries. We increased the heap size for production but obviously it will only take us so long as to the next out of memory due to increasing data volumes, which does happen at a slow-but-steady pace. And the servers don't have infinite amounts of RAM as we all know, even if they're virtuals. I don't think there's anything particularily complex about the spreadsheets that we're generating either, as described in the OP. None of the sheets has more than 30 columns, while row-wise the counts *maybe* go into the order of hundreds of thousands of rows max today, but only in the most busy sheet, while the majority will hover below 10% of that count. Having said the above, I have noticed a dramatic improvement in both memory footprint and processing time from using my proof of concept work on using the streaming api to generate these. And in my mind this is the right way to progress, but I need a way to overcome the limitations of the API, so any suggestions would be more than welcome. -- You are receiving this mail because: You are the assignee for the bug. --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
