Hmm - I don't think this is going to work. I ran the script and it got killed after 51 of 700+ games, with 21000 TyperefGame relationship records. It was consuming over 1.5GB of RAM. I understand PDO is a memory hog, so may have to refactor using mysql_query() etc. :(
Any ideas are more than welcome :) Cheers, Dan. On May 7, 8:31 am, dan <[email protected]> wrote: > Hi all, > > I have a large XML file (140+MB) which I'm parsing with an sf Task. > > It's a complicated structure, and contains some interesting > relationships. > > The approach I'm using is to open the XML file with XMLReader, and > when I find a node of interest, convert that node to a SimpleXML > object, and then build my sf Records. > > Without going into too much detail, I have a few record types that are > of primary interest; Game, Handset, Category. > > Relating Games to Categories is easy, however Games and Handsets are > related via a Typeref, i.e. Games are published for certain Typerefs, > while each handset supports one or two different Typerefs. > > So I've also created tables for Typeref, TyperefGame and > TyperefHandset in order to manage the relationships. > > Here's my real question: > > As my Task parses each Game, it looks up each of the Typerefs (approx > 450 per game) associated with the game and creates the TyperefGame > relationship record. With 700+ games, that's about 315000 records. > > As you can imagine, this takes some time, and I'm looking for every > opportunity to speed it up. > > My first attempt was simply to create a new Doctrine Record for each > Typeref, setting the necessary attributes and then saving it. This was > slow. (around 450 save()s per game) > > My second attempt was to create a Typeref Doctrine Collection for each > game, and then do one save() on the collection once all Typerefs had > been parsed. I expected this to be faster that my first method, but if > anything, it appears slower. > > What should my third attempt be? > > Should I be building a single large raw SQL statement for each game- > load of Typerefs? > > Is there an even better way to approach this whole Task? > > Your thoughts and advice would be greatly appreciated. > > Cheers, > > Dan. > > -- > If you want to report a vulnerability issue on symfony, please send it to > security at symfony-project.com > > You received this message because you are subscribed to the Google > Groups "symfony users" group. > To post to this group, send email to [email protected] > To unsubscribe from this group, send email to > [email protected] > For more options, visit this group > athttp://groups.google.com/group/symfony-users?hl=en -- If you want to report a vulnerability issue on symfony, please send it to security at symfony-project.com You received this message because you are subscribed to the Google Groups "symfony users" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/symfony-users?hl=en
