An ETL (an acronym I keep forgetting, then I look it up, then I say, "oh right I write three of those per week") is heavy on the flush side, assuming the "E" part is from spreadsheets or something like that. I made a few fairly vast improvements to flush() efficiency in 0.7, they're fun to watch in the logs.
On Jan 14, 2011, at 4:28 AM, dusans wrote: > So the C extensions would be good for someone that would use it for an > ETL with a lot of data? :) > > On 9 jan., 17:25, Michael Bayer <[email protected]> wrote: >> I remain a little nervous about the C extensions, not as much because the >> current code is unreliable, but because I'd like there to be a whole lot >> more C code here. I've used the existing C extensions quite a bit and done >> tons of profiling - they really don't account for the vast majority of time >> spent in large ORM operations, and they don't do much for a non-ORM app that >> emits a lot of statements. They just cut down on an app that is non-ORM and >> needs to fetch very large result sets. >> >> Basically I'd need a resource that can do lots more C code for me, if not >> myself since it just would take a huge block of extra time for me to get >> into it, before I'd feel good about the C code overall and confident that I >> can put out releases in a timely manner which would now require 100% >> reliable C code. >> >> The limitations of distutils are also troubling here. In my own work app, >> we use pip in conjunction with a Makefile and for the SQLAlchemy install the >> flag is on in the Makefile. In that regard the flag being off by default >> doesn't feel like that big a deal to me personally. >> >> On Jan 9, 2011, at 3:33 AM, Wichert Akkerman wrote: >> >> >> >> >> >> >> >>> Hi Mike, >> >>> On 1/9/11 00:14 , Michael Bayer wrote: >>>> The majority of my time is now spent developing 0.7, which is nearly ready >>>> for beta releases pending a few more little features I'd like to try to >>>> get in. 0.7 is really exciting with its new event API, lots of other nice >>>> touches and of course the most radical reduction in callcounts we've had >>>> in a few years. >> >>> Has been there a decision on enabling the C extensions by default in 0.7, >>> or is that still too controversial? >> >>> Regards, >>> Wichert. >> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "sqlalchemy" group. >>> To post to this group, send email to [email protected]. >>> To unsubscribe from this group, send email to >>> [email protected]. >>> For more options, visit this group >>> athttp://groups.google.com/group/sqlalchemy?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/sqlalchemy?hl=en. > -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
