Thomas That looks potentially useful - can you please give an example (on the webpage) of how to do a multi-row insertion? I typically need to add thousands of rows from existing spreadsheets - am using xlrd to read the data, but if DSE could speed up the write-to-database operation, that would be incredibly useful.
Thanks! On 6 January 2011 17:20, Thomas Weholt <[email protected]> wrote: > Just want to mention DSE which is especially made to solve problems > with mass insert and/updates in django: > > http://pypi.python.org/pypi/dse/0.3.0 > > Regards, > Thomas > > On Wed, Dec 29, 2010 at 1:22 PM, Derek <[email protected]> wrote: > > Thanks Craig - that explains it very clearly indeed ("textbook answer"!) > > > > On 27 December 2010 16:50, W. Craig Trader <[email protected]> > wrote: > >> > >> Derek ... > >> > >> Quoting from > http://docs.djangoproject.com/en/1.2/topics/db/transactions/: > >>> > >>> Django’s default behavior is to run with an open transaction which it > >>> commits automatically when any built-in, data-altering model function > is > >>> called. For example, if you call model.save() or model.delete(), the > change > >>> will be committed immediately. > >>> > >>> This is much like the auto-commit setting for most databases. As soon > as > >>> you perform an action that needs to write to the database, Django > produces > >>> the INSERT/UPDATE/DELETE statements and then does the COMMIT. There’s > no > >>> implicit ROLLBACK. > >> > >> Using the Django transaction API, you can manually control transactions. > >> In this case, Silva must be doing something like this: > >> > >> from django.db import transaction > >> > >> @transaction.commit_manually > >> def viewfunc(request): > >> > >> > >> > >> ... > >> # You can commit/rollback however and whenever you want > >> i = 0 > >> for x in data: > >> ... save something ... > >> i += 1 > >> if i % 1000: > >> > >> transaction.commit() > >> > >> # One last commit to persist the last batch of saves > >> transaction.commit() > >> > >> > >> > >> > >> I will also note that the transaction API can be (and should be) used > when > >> you're writing management commands. > >> > >> - Craig - > >> > >> On Mon, Dec 27, 2010 at 05:29, derek <[email protected]> wrote: > >>> > >>> Can you explain what you mean by "manual commitments" (preferably with > >>> a code example)? > >>> > >>> Thanks > >>> Derek > >>> > >>> On Dec 26, 4:38 am, Silva Paulo <[email protected]> wrote: > >>> > OK. I found a "solution". > >>> > Using manual commitments after lots of insertions (ex. >1000) makes > the > >>> > massive insertion very fast (at least for my needs). > >>> > > >>> > Thanks to all who answered. > >>> > > >>> > --- On Fri, 12/24/10, Silva Paulo <[email protected]> wrote: > >>> > > >>> > > >>> > > >>> > > >>> > > >>> > > >>> > > >>> > > From: Silva Paulo <[email protected]> > >>> > > Subject: Massive insertion of records > >>> > > To: "newbie" <[email protected]> > >>> > > Date: Friday, December 24, 2010, 6:26 PM > >>> > > I need to do a massive insertion of > >>> > > records in two tables "connectd" by "foreignkey". Is there a > >>> > > way, using the Django db API, to do it. e=Foo(...);e.save() > >>> > > seems too slow. > >>> > > >>> > > Thanks > >>> > > >>> > > > >>> > > >>> > > -- > >>> > > You received this message because you are subscribed to the > >>> > > Google Groups "Django users" group. > >>> > > To post to this group, send email to [email protected] > . > >>> > > To unsubscribe from this group, send email to > >>> > > [email protected]<django-users%[email protected]> > . > >>> > > For more options, visit this group > >>> > > athttp://groups.google.com/group/django-users?hl=en. > >>> > >>> -- > >>> You received this message because you are subscribed to the Google > Groups > >>> "Django users" group. > >>> To post to this group, send email to [email protected]. > >>> To unsubscribe from this group, send email to > >>> [email protected]<django-users%[email protected]> > . > >>> For more options, visit this group at > >>> http://groups.google.com/group/django-users?hl=en. > >>> > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups > >> "Django users" group. > >> To post to this group, send email to [email protected]. > >> To unsubscribe from this group, send email to > >> [email protected]<django-users%[email protected]> > . > >> For more options, visit this group at > >> http://groups.google.com/group/django-users?hl=en. > > > > -- > > You received this message because you are subscribed to the Google Groups > > "Django users" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]<django-users%[email protected]> > . > > For more options, visit this group at > > http://groups.google.com/group/django-users?hl=en. > > > > > > -- > Mvh/Best regards, > Thomas Weholt > http://www.weholt.org > > -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<django-users%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/django-users?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
