Hi Giovanni,
My first comment is that the 1,000 entity limit has been removed.
My other comment is that all of the nested queries are probably not
going to perform well. Particularly with the size of your lists. Vi
and Vj appear to be the same query, so running the query once then
building a list of tuples, ie [(i1, j1), ..., (in, jn)] to loop over
querying would be a large improvement. However, this is still not
going to work well.
It looks like you may be trying to test for existence. If possible,
could you specify the key name of C to be something like "%d.%d" % (i,
j) so that you can compute C's key? If so you can reduce your number
of queries significantly. Use the above suggestion to fetch your V
entities. Loop over your entities building a list of keys, then you
can do a batch db.get() which will be much faster.
So maybe something like:
# grab as many Vs as possible
Vs = V.all().order('counter').fetch(1000)
c_keys = [] # to hold keys.
for vi in Vs:
for vj in Vs:
key_name = "%d.%d" % (vi.counter, vj.counter)
c_keys.append(db.Key.from_path('C', key_name))
Cs = db.get(c_keys)
for c in Cs:
if c: # need to be sure it exists.
value += c.value
Robert
On Thu, Jul 8, 2010 at 3:08 AM, Giovanni Novelli
<[email protected]> wrote:
> Hello,
> I'm relying upon Python runtime and I know about the 1000 records
> limit in GQL query recordsets.
>
> I have an entity V ordered with a progressive sharded counter like a
> vector v=[vi] with i in 1..n.
> I have an entity C that represents a matrix with entries Cij with
> i=1..n and j=1..n
> I want to implement a way to cycle on the matrix C considering
> submatrices of it with a limit of 1000 records each time.
>
> I'm far from this goal and actually I'm able to obtain Cij with the
> following code:
>
> Vi = db.GqlQuery("SELECT * FROM V ORDER BY counter ASC LIMIT 1000")
> for vi in Vi:
> Vj = db.GqlQuery("SELECT * FROM V ORDER BY counter ASC LIMIT 1000")
> for vj in Vj:
> query = "SELECT * FROM C WHERE i = " + str(vi.counter) + " AND j =
> " + str(vj.counter)
> Cij = db.GqlQuery(query)
> c = Cij.get()
> if Cij.count() == 1:
> value = c.value
> else:
> value = 0.0
>
> Any suggestion or link will be appreciated.
>
> Thanks
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to [email protected].
> To unsubscribe from this group, send email to
> [email protected].
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
