Sorry for the delay... I was gone a week for US Thanksgiving holiday. You might look at Torque's sibling project called ddlutils, http://db.apache.org/ddlutils This uses the same basic XML format as Torque but specifically targeted at doing the data setup tasks that have been added into the torque core. FWIW, one of the V4.0 goals is to start using this project as the code behind these tasks.
Also, I while back I wrote a couple of Add-ons to Torque that allowed a method to import and export torque objects as XML. This consists of a generator addin that creates Betwixt maps for Torque data objects and a Runtime framework for using this to export and import data. It's was released as a SourceForge project: http://torque-addons.sourceforge.net/ If I remember right, one possible gotcha is that the Maven 2 plug-in (due to Maven 2 issues) can't control the build path like Ant and Maven 1. So the betwixt add-in may have to be merged with the Torque template jar and this substituted for the dependency. Not sure if this is right since I use ant for my builds and am not up on Maven. > -----Original Message----- > From: Ludwig Magnusson [mailto:[email protected]] > Sent: Tuesday, November 24, 2009 3:21 PM > To: 'Apache Torque Developers List' > Subject: RE: Extensive memory usage in torque-generator. Bug? > > In reply to this, I actually found the maven option to extend the amount > of > memory. > Now the VM can use 512Mb. I tried with my _actual_ data-xml file (which > was > created by torque:datadump) and that task succeeded but used 415Mb. My > computer has 4Gb of RAM but extending the allocated memory does not seem > like a solution. A database can grow much larger than the one we have. > We will take a look at the generator code and see if there is anything we > can find. (that would be the first place to look at right?) > /Ludwig > > > > -----Original Message----- > From: Thomas Fischer [mailto:[email protected]] > Sent: den 24 november 2009 21:09 > To: Apache Torque Developers List > Subject: RE: Extensive memory usage in torque-generator. Bug? > > Hm, on the one hand 65 MB RAM is not much on a modern computer. On the > other hand, it is much as compared to the data, as you said. > I am afraid that probably we do not have the manpower to dig into this; > memory usage is difficult to track down. If you find a reason for the > extensive memory usage and propose a solution, We'd be glad to > incorporate > it. > > For your immediate problem, try increasing the maven memory limits. (the > usual -Xmx -Xms options for the java vm). > > Thomas > > > > A few days ago I posted in the users list that I got a > > java.lang.OutOfMemoryError while trying to generate my data-sql. > > > > I did some stress testing on the generator, here is how it went: > > > > > > > > I used a simple table and tried to insert different number of rows, > ranging > > from 1 to 2000. > > > > This is my data.xml for n number of roles: > > > > <dataset name="all"> > > > > <Role Id="0" Name="Role_0"/> > > > > <Role Id="1" Name="Role_1"/> > > > > . > > > > <Role Id="n" Name="Role_n"/> > > > > </dataset> > > > > > > > > At the end of each run maven prints the amount of memory used (final > > memory), here are some values: > > > > > > > > 1 row - 11M/20M > > > > 100 rows - 15M/25M > > > > 1000 rows - 42M/63M > > > > 1500 rows - 58M/63M > > > > 2000 rows - fail 63M/63M > > > > > > > > The limit seems to be around 1500 rows. > > > > When that's the case, the input file is ca 60Kb and the output file is > ca > > 100Kb, so how can the generator use 58Mb of memory? > > > > Is this a bug? Known or unknown? > > > > /Ludwig > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] DukeCE Privacy Statement: Please be advised that this e-mail and any files transmitted with it are confidential communication or may otherwise be privileged or confidential and are intended solely for the individual or entity to whom they are addressed. If you are not the intended recipient you may not rely on the contents of this email or any attachments, and we ask that you please not read, copy or retransmit this communication, but reply to the sender and destroy the email, its contents, and all copies thereof immediately. Any unauthorized dissemination, distribution or copying of this communication is strictly prohibited. --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
