[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 krzysztof.rzymkow...@gmail.com changed: What|Removed |Added Status|NEW |RESOLVED

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #26 from PJ Fanning --- POI 4.0.0 and poi-shared-strings 1.0.0 are now released. -- You are receiving this mail because: You are the assignee for the bug. - To

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #25 from PJ Fanning --- https://github.com/pjfanning/poi-shared-strings is an extension to poi that may help - will be released as soon as POI 4.0.0 is released (it relies on changes in POI 4.0.0). -- You are receiving this mail

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #24 from Thamodharan --- @Fanning, In below link, https://poi.apache.org/changes.html under History of Changes Version 4.0.0-SNAPSHOT (2018-08-??) The change 61832 / Github-85 is not involved. As you replied to marion, can we

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #23 from PJ Fanning --- There have been no changes made to POI for this. 1. If you use inline strings, the issue appears to be that Excel itself baulks at the large dataset (eg Comment 7 above). 2. Comment 16

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #22 from Marion --- This is still an issue. Can anyone from the Apache team confirm this issue will be resolved with the new release? -- You are receiving this mail because: You are the assignee

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #21 from Thamodharan --- Hi Fanning, I went through the pull request in github, really great. Is there a chance to expect complete code in upcoming version of apache poi ? -- You are receiving this mail

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #20 from PJ Fanning --- There is a Pull Request which uses temp files for constructing the shared strings table. https://github.com/apache/poi/pull/85 -- You are receiving this mail because: You are the

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #19 from Thamodharan --- My machine specs, Jboss server RAM 32 GB Heap size allocated to jboss 28 GB Harddisk 1 TB I am good to go with generating excel file upto a limit of 0.3 million X 200 rows with

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #18 from Thamodharan --- Hi mewalig, Kindly help us by explaining the calculation which you stated on your last reply. -- You are receiving this mail because: You are the assignee for the bug.

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #17 from Thamodharan --- Hi mewalig, Does 1mm what you mentioned refer to 0.1 million or 1 million rows ? Can you please explain this calculation [(256+32)/8*150*1mm = 5.4GB of memory] in detail, so that

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #16 from PJ Fanning --- I have the initial work done on a prototype version of SharedStringTable that can be used in SXSSF to reduce the memory footprint. The Pull Request is for discussion as opposed to

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #15 from mewa...@gmail.com --- I do not agree with this at all: "Some tasks just take a large amount of RAM, which is cheap these days, including the ability to "rent" time and space on any number of cloud provider platforms [...]".

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #14 from Thamodharan --- Though I adopt 64GB system RAM and 30GB to JBOSS heap space, I cant even produce two parallel workbook of such large size at a time. In case of adopting to write a patch, speed is

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #13 from Greg Woolsey --- At some point, you just have to allocate more heap, no way around it. Sounds like you might be there, needing to allow for a very large shared strings table. Some tasks just take a

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #12 from Thamodharan --- (In reply to Yegor Kozlov from comment #11) > try to construct SXSSFWorkbook in a different way: > > wb = new SXSSFWorkbook( > null, /* template workbook. pass null to start

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #11 from Yegor Kozlov --- try to construct SXSSFWorkbook in a different way: wb = new SXSSFWorkbook( null, /* template workbook. pass null to start with a blank workbook */ 1000, /* size of the sliding

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #10 from Thamodharan --- Hi Yegor Kozlov, Hope, you mean to set this limit 10K as mentioned below. SXSSFWorkbook wb = new SXSSFWorkbook(1); Though I set this, I face content issue while opening it.

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #9 from Yegor Kozlov --- In theory SXSSF can use a fixed-size cache of shared strings, e.g. we can allow SXSSF to put 10K distinct strings in the SST. If the cache is full then the string goes inline. The 10K limit

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #8 from Thamodharan --- Hi Greg Woolsey, SXSSF by default takes inline string, it can be changed to shared string for which some patch is available. Since Shared string takes the heap space memory, its

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #7 from Greg Woolsey --- I ran the attached code, and examined the generated workbook. All XML validates successfully with xmllint. The worksheet file, however, expands to > 5GB due to all the inlineStr

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #6 from Thamodharan --- (In reply to Yegor Kozlov from comment #5) > Can you please do one more test: generate a 500K rows x 150 columns > spreadsheet containing only numbers and see if it opens? Due to

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #5 from Yegor Kozlov --- Can you please do one more test: generate a 500K rows x 150 columns spreadsheet containing only numbers and see if it opens? Due to its streaming nature SXSSF stores strings differently than

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #4 from Thamodharan --- Created attachment 35571 --> https://bz.apache.org/bugzilla/attachment.cgi?id=35571=edit ExcelErrorAfterRepair -- You are receiving this mail because: You are the assignee for

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #3 from Thamodharan --- (In reply to Yegor Kozlov from comment #2) > (In reply to Thamodharan from comment #1) > > Created attachment 35566 [details] > > Excel 2013 - Alert Message - While opening such

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #2 from Yegor Kozlov --- (In reply to Thamodharan from comment #1) > Created attachment 35566 [details] > Excel 2013 - Alert Message - While opening such large file Is it a 32 or 64-bit Excel ? I wonder if it

[Bug 61832] Unable to create a excel of 500,000 rows and 150 column using SXSSF workbook

https://bz.apache.org/bugzilla/show_bug.cgi?id=61832 --- Comment #1 from Thamodharan --- Created attachment 35566 --> https://bz.apache.org/bugzilla/attachment.cgi?id=35566=edit Excel 2013 - Alert Message - While opening such large file -- You are receiving this