Index building isn't done in what we call "user space", but that's certainly one possibility. I think we're being a bit more intelligent about how we schedule work, however, and it might be a bit more involved of a solution than just creating multiple queues - for instance, you could effective "game" the system by inserting a ton of data after the index building begins (and we don't want to lock your datastore). We recognize that this is a major pain point for developers and there's some work going on in this area to make index building more fair all around.
On Fri, Sep 10, 2010 at 1:25 PM, Jeff Schwartz <[email protected]>wrote: > I think the problem with 2 tasks queues is that they could still be hitting > the same nodes on the network. Maybe just a better time slicing algorithm is > what's needed. > > > On Fri, Sep 10, 2010 at 3:36 PM, Nate Bauernfeind < > [email protected]> wrote: > >> Couldn't you create two task queue's? That should fix the starvation >> problem. I suppose the trick would be to determine which queue to stick it >> on. >> >> >> On Fri, Sep 10, 2010 at 1:15 PM, Ikai L (Google) <[email protected]>wrote: >> >>> I think one thing we can do better is to schedule some index builds for >>> apps with no data ahead of apps with a lot of data, since they are >>> essentially no-ops. Of course, one wonders if this means large index builds >>> will be delayed forever? *rubs hands together for Computer Science fun* >>> >>> >>> On Thu, Sep 9, 2010 at 7:11 PM, Jeff Schwartz >>> <[email protected]>wrote: >>> >>>> Soon after I posted the issue it was fixed. I don't know how or by whom >>>> but somehow it got resolved. I waited a few hours for the index to build >>>> even though there was no data to which the index was referring to. Thank >>>> you, Ikai. >>>> >>>> On Thu, Sep 9, 2010 at 9:39 PM, Ikai L (Google) <[email protected]>wrote: >>>> >>>>> The indexes all look to be serving to me. >>>>> >>>>> On Thu, Sep 9, 2010 at 2:07 PM, Jeff Schwartz <[email protected] >>>>> > wrote: >>>>> >>>>>> I have 1 index that was building for quite a while though there was no >>>>>> data associated with the index. When I clicked on the Datastore Indexes >>>>>> link >>>>>> in the admin console I am directed to a error page that says: >>>>>> >>>>>> Server Error (500) >>>>>> >>>>>> A server error has occurred. >>>>>> >>>>>> Return to Applications screen ยป <http://appengine.google.com/> >>>>>> >>>>>> >>>>>> My application's id is i-emote and its version is 1. >>>>>> >>>>>> >>>>>> Can anyone fix this for me and as soon as posible? Thanks in advance. >>>>>> >>>>>> Jeff >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Google App Engine" group. >>>>>> To post to this group, send email to >>>>>> [email protected]. >>>>>> To unsubscribe from this group, send email to >>>>>> [email protected]<google-appengine%[email protected]> >>>>>> . >>>>>> For more options, visit this group at >>>>>> http://groups.google.com/group/google-appengine?hl=en. >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Ikai Lan >>>>> Developer Programs Engineer, Google App Engine >>>>> Blog: http://googleappengine.blogspot.com >>>>> Twitter: http://twitter.com/app_engine >>>>> Reddit: http://www.reddit.com/r/appengine >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Google App Engine" group. >>>>> To post to this group, send email to [email protected] >>>>> . >>>>> To unsubscribe from this group, send email to >>>>> [email protected]<google-appengine%[email protected]> >>>>> . >>>>> For more options, visit this group at >>>>> http://groups.google.com/group/google-appengine?hl=en. >>>>> >>>> >>>> >>>> >>>> -- >>>> -- >>>> >>>> Jeff >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Google App Engine" group. >>>> To post to this group, send email to [email protected]. >>>> To unsubscribe from this group, send email to >>>> [email protected]<google-appengine%[email protected]> >>>> . >>>> For more options, visit this group at >>>> http://groups.google.com/group/google-appengine?hl=en. >>>> >>> >>> >>> >>> -- >>> Ikai Lan >>> Developer Programs Engineer, Google App Engine >>> Blog: http://googleappengine.blogspot.com >>> Twitter: http://twitter.com/app_engine >>> Reddit: http://www.reddit.com/r/appengine >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Google App Engine" group. >>> To post to this group, send email to [email protected]. >>> To unsubscribe from this group, send email to >>> [email protected]<google-appengine%[email protected]> >>> . >>> For more options, visit this group at >>> http://groups.google.com/group/google-appengine?hl=en. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]<google-appengine%[email protected]> >> . >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > > > -- > -- > > Jeff > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<google-appengine%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
