That would be the simplest. Having something so inefficient just bugs me. :)
I'm using MSSQL, so limit() works. Would yield_per() help here, or is that for something different? Even if it didn't help local memory, but just kept the load on the DB server down, that would be good. On 3/16/16, Christopher Lee <[email protected]> wrote: > It sounds like you should just fire it up with the outer joins and watch > memory on the box. If it gets too high, or crashes entirely, then you can > look into different approaches. For example, you could keep the outer > joins, but paginate your query so that it is only pulling a subset of the > rows from your main table (but fully joining against the secondary > tables). Just one caveat... if you are using MySQL, then LIMIT and OFFSET > are not your friends; you'll want to find a different pagination mechanism. > > On Wed, Mar 16, 2016 at 10:29 AM, Jonathan Vanasco <[email protected]> > wrote: > >> We all inherit less-than-ideal situations. >> >> If this is running once a day and isn't impacting performance or other >> work, I wouldn't really worry about the huge join matrix. It sounds like >> the current solution is "good enough". In a few weeks or months you'll >> be >> better acquainted with SqlAlchemy and Sql in general and can revisit. >> >> In terms of your 15minute script: When you can use subqueries, filters >> and >> `load_only` for certain columns, your backend will generate a smaller >> matrix and there will be a less data transferred "over the wire". >> >> -- >> You received this message because you are subscribed to the Google Groups >> "sqlalchemy" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at https://groups.google.com/group/sqlalchemy. >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/sqlalchemy. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
