There are constantly posts about how slow it is to load data. Assuming 'tons' means thousands and thousands of entities, then from watching the groups it seems like it would be very hard to insert a ton of data after index building begins. Unless your planning to make index building slower.
On Fri, Sep 10, 2010 at 16:58, Ikai L (Google) <[email protected]> wrote: > Index building isn't done in what we call "user space", but that's certainly > one possibility. I think we're being a bit more intelligent about how we > schedule work, however, and it might be a bit more involved of a solution > than just creating multiple queues - for instance, you could effective > "game" the system by inserting a ton of data after the index building begins > (and we don't want to lock your datastore). We recognize that this is a > major pain point for developers and there's some work going on in this area > to make index building more fair all around. > > On Fri, Sep 10, 2010 at 1:25 PM, Jeff Schwartz <[email protected]> > wrote: >> >> I think the problem with 2 tasks queues is that they could still be >> hitting the same nodes on the network. Maybe just a better time slicing >> algorithm is what's needed. >> >> On Fri, Sep 10, 2010 at 3:36 PM, Nate Bauernfeind >> <[email protected]> wrote: >>> >>> Couldn't you create two task queue's? That should fix the starvation >>> problem. I suppose the trick would be to determine which queue to stick it >>> on. >>> >>> On Fri, Sep 10, 2010 at 1:15 PM, Ikai L (Google) <[email protected]> >>> wrote: >>>> >>>> I think one thing we can do better is to schedule some index builds for >>>> apps with no data ahead of apps with a lot of data, since they are >>>> essentially no-ops. Of course, one wonders if this means large index builds >>>> will be delayed forever? *rubs hands together for Computer Science fun* >>>> >>>> On Thu, Sep 9, 2010 at 7:11 PM, Jeff Schwartz <[email protected]> >>>> wrote: >>>>> >>>>> Soon after I posted the issue it was fixed. I don't know how or by whom >>>>> but somehow it got resolved. I waited a few hours for the index to build >>>>> even though there was no data to which the index was referring to. Thank >>>>> you, Ikai. >>>>> >>>>> On Thu, Sep 9, 2010 at 9:39 PM, Ikai L (Google) <[email protected]> >>>>> wrote: >>>>>> >>>>>> The indexes all look to be serving to me. >>>>>> >>>>>> On Thu, Sep 9, 2010 at 2:07 PM, Jeff Schwartz >>>>>> <[email protected]> wrote: >>>>>>> >>>>>>> I have 1 index that was building for quite a while though there was >>>>>>> no data associated with the index. When I clicked on the Datastore >>>>>>> Indexes >>>>>>> link in the admin console I am directed to a error page that says: >>>>>>> >>>>>>> Server Error (500) >>>>>>> >>>>>>> A server error has occurred. >>>>>>> >>>>>>> Return to Applications screen » >>>>>>> >>>>>>> My application's id is i-emote and its version is 1. >>>>>>> >>>>>>> Can anyone fix this for me and as soon as posible? Thanks in advance. >>>>>>> >>>>>>> Jeff >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Google App Engine" group. >>>>>>> To post to this group, send email to >>>>>>> [email protected]. >>>>>>> To unsubscribe from this group, send email to >>>>>>> [email protected]. >>>>>>> For more options, visit this group at >>>>>>> http://groups.google.com/group/google-appengine?hl=en. >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Ikai Lan >>>>>> Developer Programs Engineer, Google App Engine >>>>>> Blog: http://googleappengine.blogspot.com >>>>>> Twitter: http://twitter.com/app_engine >>>>>> Reddit: http://www.reddit.com/r/appengine >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Google App Engine" group. >>>>>> To post to this group, send email to >>>>>> [email protected]. >>>>>> To unsubscribe from this group, send email to >>>>>> [email protected]. >>>>>> For more options, visit this group at >>>>>> http://groups.google.com/group/google-appengine?hl=en. >>>>> >>>>> >>>>> >>>>> -- >>>>> -- >>>>> Jeff >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Google App Engine" group. >>>>> To post to this group, send email to [email protected]. >>>>> To unsubscribe from this group, send email to >>>>> [email protected]. >>>>> For more options, visit this group at >>>>> http://groups.google.com/group/google-appengine?hl=en. >>>> >>>> >>>> >>>> -- >>>> Ikai Lan >>>> Developer Programs Engineer, Google App Engine >>>> Blog: http://googleappengine.blogspot.com >>>> Twitter: http://twitter.com/app_engine >>>> Reddit: http://www.reddit.com/r/appengine >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Google App Engine" group. >>>> To post to this group, send email to [email protected]. >>>> To unsubscribe from this group, send email to >>>> [email protected]. >>>> For more options, visit this group at >>>> http://groups.google.com/group/google-appengine?hl=en. >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Google App Engine" group. >>> To post to this group, send email to [email protected]. >>> To unsubscribe from this group, send email to >>> [email protected]. >>> For more options, visit this group at >>> http://groups.google.com/group/google-appengine?hl=en. >> >> >> >> -- >> -- >> Jeff >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. > > > > -- > Ikai Lan > Developer Programs Engineer, Google App Engine > Blog: http://googleappengine.blogspot.com > Twitter: http://twitter.com/app_engine > Reddit: http://www.reddit.com/r/appengine > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
