On Wednesday, May 9, 2012 6:21:39 PM UTC-3, Jedrin wrote: > > > > The heap contains all the objects created for the application.. In this > > case, it looks like your file is still too big > > > > > Error occurred during initialization of VM > > > Could not reserve enough space for object heap > > > JVM creation failed > > > > This means that you tried to allocate more than is available on the > machine > > > > Are you doing this for a single load, or will it be an application that > > will commonly receive large files? > > > > If it's the latter, I'd probably try to redesign the code you're using > to > > load the files. Sounds like this is part of a third party gem? If > that's > > the case, maybe they have some mechanism for handling larger files? > > > > -- > > Greg Akinshttp://twitter.com/akinsgre > > What I do is create a csv file from the database. I had some memory > problems there, but using active record find_in_batches() seemed to > solve that. > > The CSV file has 1.4 million rows. It gets created successfully. I > then use rubyzip gem to create a zip file that just contains that CSV > file. I just used examples I found from google searches on how to > create the zip file which are shown earlier up in the thread. I looked > at the class info on the web for rubyzip and didn't see an obvious way > to stream data into the zip file. Tomorrow I can look at perhaps some > other way to create a zip file using a different gem or some such .. > > As I mentioned in my previous reply and similar to the problem you had when creating the file: you're trying to load the whole thing.
There are two options for this: A) You stream the contents of your CSV file, reading by chunks into a ZipStream or B) You zip the file from outside Ruby (shelling out to gzip for example) -- Luis Lavena -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To view this discussion on the web visit https://groups.google.com/d/msg/rubyonrails-talk/-/mwyK5VTPabEJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
