Rohit, When your program is running look at the output of the top command to see which process is consuming the memory.
I'm not a Rubyist, but is that code just buffering all docs from the first database and then saving them in one call to bulk docs? HTH, Paul Davis On Tue, Apr 14, 2009 at 10:29 AM, Rohit Amarnath (JIRA) <[email protected]> wrote: > > [ > https://issues.apache.org/jira/browse/COUCHDB-325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12698786#action_12698786 > ] > > Rohit Amarnath commented on COUCHDB-325: > ---------------------------------------- > > Not sure, could you point me in the direction of some tool I could install > and run to give you that info? > > >> Memory Leak in 9.0? >> ------------------- >> >> Key: COUCHDB-325 >> URL: https://issues.apache.org/jira/browse/COUCHDB-325 >> Project: CouchDB >> Issue Type: Bug >> Affects Versions: 0.9 >> Environment: gentoo linux on c1.medium aws instance >> Reporter: Rohit Amarnath >> Priority: Minor >> Fix For: 0.9 >> >> >> I am using couchrest to transform a document from one database to another >> (millions of records). The memory usage continues to increase till all the >> memory is used. The process occasionally fails, but usually completes. I >> dont know enough about ruby/couchrest/couchdb to be able to tell you where >> the memory is increasing from - but if you give me some direction, I will >> be happy to take a look. >> If I comment out the save, it seems ok. >> Here is the code: >> -------------------------------- >> require 'rubygems' >> require 'couchrest' >> @db = CouchRest.database!("http://127.0.0.1:5984/xfpds_2008";) >> @db2 = CouchRest.database!("http://127.0.0.1:5984/fpds_2008";) >> @streamer = CouchRest::Streamer.new(@db) >> @streamer.view("_all_docs") do |row| >> begin >> doc = @db.get(row["id"]) >> # remove id so the new database gets fresh document >> doc.delete("_id") >> doc.delete("_rev") >> # change badgerfish notation to assign $ key value to parent >> doc["amounts"]["obligatedAmount"] = doc["amounts"]["obligatedAmount"]["$"] >> doc["amounts"]["baseAndAllOptionsValue"] = >> doc["amounts"]["baseAndAllOptionsValue"]["$"] >> doc["amounts"]["baseAndExercisedOptionsValue"] = >> doc["amounts"]["baseAndExercisedOptionsValue"]["$"] >> ...... A whole bunch of fields >> # save the document using bulk save >> response = �[email protected]_doc(doc,true) >> rescue >> # if streamer ends, save last few documents >> @db2.bulk_save >> end >> end > > -- > This message is automatically generated by JIRA. > - > You can reply to this email to add a comment to the issue online. > >
