Ikai, As Dinesh mentioned we had exported and imported data using remote api (using some weird query) to get this working. Unfortunately, batch export/import also did not work. So, some clarity related to this issue would help us avoid surprises in future: - Bulk export/import is also based on the same set of queries, which again does not return correct result. In cases such as these, when query itself fails, is there any other way to get data out? Is this not a single point of failure? Shouldnt the bulk export/import be run from inside the datacenter, directly on the application id/tenant without queries, without time out and 1000 items per query limitations? - Does this happen because of indexing failure during heavy entity writes? If so, are they not atomic? Why does the query return incorrect results, even when indexes are rebuilt from Datastore administration. Are they 2 different things? - As you can see from this thread, there has been no response for 2 weeks, when our application on production went down. Is there any escalation/support planned in future for production systems? At least, an acknowledgement of some form for us to try other alternatives.
Although bugs are understandable, this issue in a way, represents one of the inherent risks in the cloud: - data in servers that you do not have control on (to take backup or restore. Not really the usual "show me my data" stuff) - openness that hasnt yet reached maturity (atleast technically not working to resort to an alternative) So pls ensure that the fall back options are also reliable. thanks, mani On Dec 3, 2:44 pm, Dinesh Varadharajan <[email protected]> wrote: > Ikai, > Thanks for checking that out. We already exported and imported close > to 1 million records to make it work again. we left a sample set of > data for google to reproduce the issue. > > However we are concerned about this bug being fixed and if possible > when it was fixed, since this had been happening quite frequently last > month. > > On Dec 3, 12:00 am, "Ikai L (Google)" <[email protected]> wrote: > > > > > Dinesh, > > > I see the inconsistency. How long have you been writing to this dataset? My > > suspicion is that the indexes may have been updated incorrectly in a > > previous release due to a bug that we have since addressed. Unfortunately, > > the bug fix maynothave retroactively addressed the incorrectly updated > > indexes. > > > How large is the dataset? For small datasets, bulk exporting and importing > > will address the issue, but for large datasets, we'll have to look to an > > alternative solution. > > > On Tue, Dec 1, 2009 at 3:40 AM, Dinesh Varadharajan < > > > [email protected]> wrote: > > > Ikai, > > > Unfortunately we don't have dummy data to showcase this. I have a > > > reproducible case in production. > > > > The app id is os-dev.appspot.com. > > > > if you execute the query > > > > SELECT * FROM PrimaryData where ApplicationId = > > > 'Application_1652c875_be0f_11de_b4a5_a3c424aa5af6' and SheetMetadataId > > > = 'Sheet001' and Deleted=False > > > > it returns 8 records. > > > > and if you execute > > > > SELECT * FROM PrimaryData where ApplicationId = > > > 'Application_1652c875_be0f_11de_b4a5_a3c424aa5af6' and SheetMetadataId > > > = 'Sheet001' > > > > it should at least return 8 records(I am removing a condition). but > > > it returns only 4 records. > > > > Please let me know if you want to be added as developer to os-dev to > > > be able to access the datastore. > > > > Dinesh > > > > On Dec 1, 12:27 am, "Ikai L (Google)" <[email protected]> wrote: > > > > Prashant, do you have sample data you can provide? It's even better if > > > > it > > > > isn't real data. > > > > > On Sun, Nov 29, 2009 at 3:19 AM, Prashant <[email protected]> wrote: > > > > > what about this - I had a data store entity with known id, and JDOQL > > > simply > > > > > failed to retrieve it by id throwing JDOObjectNotFoundException. I > > > think > > > > > there is some major issue with datastore/indexes. > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > Groups > > > > > "Google App Engine" group. > > > > > To post to this group, send email to [email protected] > > > . > > > > > To unsubscribe from this group, send email to > > > > > [email protected]<google-appengine%2Bunsubscrib > > > > > [email protected]> > > > <google-appengine%[email protected]<google-appengine%252Bunsub > > > [email protected]> > > > > > > . > > > > > For more options, visit this group at > > > > >http://groups.google.com/group/google-appengine?hl=en. > > > > > -- > > > > Ikai Lan > > > > Developer Programs Engineer, Google App Engine > > > > -- > > > > You received this message because you are subscribed to the Google Groups > > > "Google App Engine" group. > > > To post to this group, send email to [email protected]. > > > To unsubscribe from this group, send email to > > > [email protected]<google-appengine%2Bunsubscrib > > > [email protected]> > > > . > > > For more options, visit this group at > > >http://groups.google.com/group/google-appengine?hl=en. > > > -- > > Ikai Lan > > Developer Programs Engineer, Google App Engine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
