oh wow.. yes,
try this for starters (pseudocode):
new_rows = new List(500)
i = 0
for row in file:
new_row = RowType( somedata)
if (i < 500)
new_rows.add(new_row)
i++
else
i = 0
new_rows.put()
new_rows.clear()
new_rows.add(new_row)
i++
that should decrease the amount of puts you do into the datastore by a
factor of 2
On Apr 27, 1:21 pm, Geoff <[email protected]> wrote:
> Hi,
>
> I have an application which requires some relatively big inserts into
> the datastore. The amount of data isn't huge, but there are often>100,000
> rows in each file upload (and I could be doing about 500-1000
>
> per day). At the moment I'm doing something like
>
> for row in file:
> new_row = RowType( somedata )
> new_row.put()
>
> However this is way too slow (CPU-wise) for my needs; a 40,000 line
> file is taking about 4 CPU hours.
>
> Is there some obviously way of optimising the insert which I don't
> know about?
>
> Geoff.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to [email protected].
> To unsubscribe from this group, send email to
> [email protected].
> For more options, visit this group
> athttp://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
