On 20/01/2012 22:39, Matt Rogghe wrote:
Howdy all,
Looking simply for any tips when writing huge workbooks using the SXSSFWorkbook
implementation. I'm pulling data from a database and writing out a workbook.
In this particular case I'm writing upwards of 3+ million rows across 5 sheets.
Using the SXSSF implementation (MUCH better than XSSF obviously) I can nearly
complete the operation. Memory appears to quickly reach 2GB useage for the
process but performance degrades gradually until I get a
java.lang.OutOfMemoryError: GC overhead limit exceeded error.
Are there any quick wins I should be looking for? In the latest incarnation of
my code (of which there have been many) I am manually flushing 200 rows at a
time with the initial workbook constructor instantiated with the -1 (disable
auto flushing) flag.
I'm thinking maybe just up the Java memory footprint but... was hoping to keep
a more compact JVM using SXSSF.
Any thoughts?
Thanks,
-Matt
A few thoughts that might help:
Are you sure that you aren't loading your entire query resultset into
memory?
This is the default for JDBC, and ir;s very unhealthy with large resultsets.
SXSSF only streams the cell contents, are you doing anything else per
cell that might allocate memory?
The worst example would be creating a CellStyle per cell.
Can you publish the code you use to process each row?
Are you able to profile your code?
Jim
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
