Thanks Eli, I was trying to use the Key string for ordering and pagination. Guess I will have to look into creating a separate field for pagination. Thanks, Manny
On Sat, Feb 13, 2010 at 10:00 PM, Eli Jones <[email protected]> wrote: > No can do this: > > "An application should not rely on numeric IDs being assigned in > increasing order with the order of entity creation. This is generally the > case, but not guaranteed." > > > http://code.google.com/appengine/docs/python/datastore/keysandentitygroups.html > > On Sat, Feb 13, 2010 at 11:12 AM, Manny S <[email protected]> wrote: > >> Hi Tim, >> Thanks again Tim. A naive question - if I use the datastore generated keys >> and sort by the keys desc (for pagination) the key value of new data >> inserted will be greater than that of previously entered data (if I do a >> keytostring and a string compare?). >> >> In other words does the datastore generate keys in a order or at random. I >> did some experiments and I found that pagination works according to >> date/time of insert if I just sort keys by descending but just wanted to >> make sure. >> Regards, >> Manny >> >> On Sat, Feb 13, 2010 at 1:17 PM, Tim Hoffman <[email protected]> wrote: >> >>> Hi Manny >>> >>> Understood. >>> >>> You know you can let the system generate the entity identifiers for >>> you, and pagination will >>> just work (by __key__) any way. So you don't have to the key_name or >>> id at all >>> if you don't care what the key is. >>> >>> Rgds >>> >>> T >>> >>> On Feb 13, 1:00 am, Manny S <[email protected]> wrote: >>> > Thanks a bunch Tim for your inputs, >>> > >>> > My rationale for adding the date to the appstore generated key is to >>> make >>> > pagination easier. I would do pagination on the key and not add a >>> separate >>> > column for that. (Pagination by date alone will also not solve my >>> problem >>> > since it can have duplicates and hence couple it with the key) >>> > >>> > I understand fetching by keys is much faster. Though I don't see a >>> scenario >>> > where I would have to do that now I would like to architect my app >>> where >>> > that would be possible. However, I do not have anything unique in my >>> record >>> > with which I can set the key. It just contains city name, locality >>> details >>> > and a series of other fields all of which could have duplicates. Any >>> ideas >>> > as to how I can generate unique ids from these or any general pointers >>> > towards generating unique Ids from data where the data itself does not >>> have >>> > a unique field? >>> > >>> > Manny >>> > >>> > On Thu, Feb 11, 2010 at 4:49 PM, Tim Hoffman <[email protected]> >>> wrote: >>> > > Hi Manny >>> > >>> > > Do you really want to do that for a key. One if the big advantages >>> of >>> > > creating your own keys >>> > > is being able to explicitly get entities by key (1 or more with >>> > > db.get(list_of_keys) which is much >>> > > quicker than a gql or filter. Making your keys include dates mean >>> you >>> > > will be unlikely to >>> > > guess/know what the keys are in advance. >>> > >>> > > This of course may not be useful for what you are doing, but worth >>> > > keeping in mind. >>> > >>> > > Rgds >>> > >>> > > T >>> > >>> > > On Feb 11, 2:12 pm, Manny S <[email protected]> wrote: >>> > > > Hi Ikai, >>> > >>> > > > I did read the documentation and now I have my data structures in >>> place. >>> > > One >>> > > > thing I wanted to do and that was not clear from my previous post >>> was to >>> > > > append a app generated string (not unique) as a prefix to a >>> datastore >>> > > > generated key. For instance, I want to generate a key that has the >>> date >>> > > (of >>> > > > record creation) as a prefix to the datastore generated unique key. >>> Is >>> > > > there a way to do this? I do not want my application to generate >>> unique >>> > > Ids. >>> > >>> > > > From reading through the literature so far, I am guessing that will >>> not >>> > > be >>> > > > possible since the datastore keys are generated only at the time >>> when the >>> > > > objects are being made persistent. >>> > >>> > > > Manny >>> > >>> > > > On Wed, Feb 10, 2010 at 3:30 AM, Ikai L (Google) < >>> [email protected]> >>> > > wrote: >>> > >>> > > > > Have you read our documentation on KeyFactory? >>> > >>> > > > > >>> http://code.google.com/appengine/docs/java/datastore/relationships.html >>> > >>> > > > > < >>> > > >>> http://code.google.com/appengine/docs/java/datastore/relationships.html >>> > > >I'd >>> > > > > try to understand what's going on there. It sounds like you're >>> doing it >>> > > the >>> > > > > right way, but it's up to you to benchmark and find the best >>> approach >>> > > for >>> > > > > what works for you. The usage characteristics of your application >>> > > should >>> > > > > determine the way your store your data. >>> > >>> > > > > On Wed, Feb 3, 2010 at 3:42 AM, Manny S <[email protected]> >>> wrote: >>> > >>> > > > >> Ikai, >>> > > > >> Based on your inputs I created two data classes that have a >>> > > unidirectional >>> > > > >> one-to-one relationship >>> > > > >> Now, I have two data classes simpledata and detailscol. >>> > > > >> simpledata contains fields A, B, C (and a Key field) >>> > > > >> detailscol just contains field D. >>> > >>> > > > >> simpledata imports detailscol that contains field D (and a Key >>> field). >>> > > It >>> > > > >> also contains an accessor for the detailscol. >>> > > > >> Code: >>> > > > >> simpledata sdata = new simpledata(A,B,C); >>> > > > >> sdata.setKey(null); >>> > > > >> detailscol obj = new detailscol(D); >>> > > > >> sdata.setD(obj); >>> > >>> > > > >> The keys are generated by the application and then I make the >>> data >>> > > > >> persistent. >>> > >>> > > > >> Now, I display just the data in simpledata and if the user >>> clicks on a >>> > > > >> details link I get the data stored in detailscol >>> > > > >> To get to that data I just do >>> > >>> > > > >> detailscol d = sdata.getDetails(); >>> > >>> > > > >> Two questions: >>> > >>> > > > >> 1) Is this the right approach? >>> > >>> > > > >> 2) If I want to get the child data using just the parent keyhow >>> do I >>> > > go >>> > > > >> about it? >>> > >>> > > > >> E.g, user clicks details and I use some AJAX to redirect to a >>> > > different >>> > > > >> servlet with just parent key as a parameter (since I don't >>> access the >>> > > child >>> > > > >> object yet). I get the parent key using >>> > > > >> KeyFactory.keyToString(sdata.getKey()); >>> > >>> > > > >> Now, that I have the parent's key should I do a getObjectbyID on >>> the >>> > > > >> parent data again using this and then get the child using the >>> accessor >>> > > > >> method or is there a direct way to construct the child key and >>> get to >>> > > the >>> > > > >> child data. >>> > >>> > > > >> Due to the nature of my application I would like to have the key >>> > > generated >>> > > > >> automatically (using setKey(null)). >>> > >>> > > > >> Apologies for the confusion in advance :) >>> > >>> > > > >> Manny >>> > >>> > > > >> On Sat, Jan 30, 2010 at 12:16 AM, Ikai L (Google) < >>> [email protected] >>> > > >wrote: >>> > >>> > > > >>> Hi Manny, >>> > >>> > > > >>> A few things to first remember - App Engine's datastore is not >>> a >>> > > > >>> database, but a distributed key value store with additional >>> features. >>> > > Thus, >>> > > > >>> we should be careful not to frame our thinking in terms of >>> RDBMS >>> > > schemas. >>> > > > >>> For this reason, I like to avoid using database terminology >>> that can >>> > > > >>> confound the design process like "table" or "column". App >>> Engine >>> > > stores >>> > > > >>> objects serialized ("entities") and indexes on the values. It'd >>> be >>> > > similar >>> > > > >>> to an approach of creating a MySQL table with a String ID and a >>> blob >>> > > value, >>> > > > >>> storing serialized Objects in the blob column, or using >>> Memcache and >>> > > storing >>> > > > >>> JSON values. >>> > >>> > > > >>> When you retrieve a single value from the key value store, we >>> have to >>> > > > >>> retrieve everything at once. In most scenarios, unlike SQL >>> databases >>> > > you may >>> > > > >>> be used to, retrieving large binary or text data does not add >>> serious >>> > > > >>> overhead. Of course, this changes if you start storing data on >>> the >>> > > scale of >>> > > > >>> 1mb and are retrieving it unnecessarily. How large is the data >>> you >>> > > are >>> > > > >>> retrieving? >>> > >>> > > > >>> Here's the way I would model your scenario if I was positive >>> the >>> > > > >>> text/binary field had a 1:1 relationship with the parent class: >>> > >>> > > > >>> * on your main entity, define the properties. >>> > > > >>> * define a new entity with a text/binary field, and encode the >>> parent >>> > > key >>> > > > >>> information in this key such that generating the key for this >>> child >>> > > field is >>> > > > >>> very cheap. KeyFactory.stringToKey and KeyFactory.keyToString >>> are >>> > > crucial >>> > > > >>> here. Read more about them here: >>> > >>> > > >>> http://code.google.com/appengine/docs/java/javadoc/com/google/appengi.. >>> .. >>> > > > >>> You can call your child property "parent_id:additional_info" or >>> > > whatever >>> > > > >>> makes sense to you. >>> > >>> > > > >>> Robert's solution of using a child key is basically just a >>> variation >>> > > on >>> > > > >>> this, as parent key information is encoded in a child key. >>> > >>> > > > >>> A lot of this stuff can be a bit different to get used to. I >>> suggest >>> > > > >>> becoming familiar with keys and how they are used in App >>> Engine: >>> > >>> > > > >>> Basic documentation about relationships: >>> > >>> > > >>> http://code.google.com/appengine/docs/java/datastore/relationships.html >>> > > > >>> A more advanced article: >>> > > > >>> >>> http://code.google.com/appengine/articles/storage_breakdown.html >>> > >>> > > > >>> On Thu, Jan 28, 2010 at 10:28 PM, Manny S < >>> [email protected] >>> > > >wrote: >>> > >>> > > > >>>> Hi All, >>> > >>> > > > >>>> First off, thanks for your time. A quick noob question on the >>> right >>> > > way >>> > > > >>>> to model data. >>> > >>> > > > >>>> I have a table with four columns A,B,C, D. D - the fourth is >>> of >>> > > type >>> > > > >>>> text (contains quite a bit of data). >>> > >>> > > > >>>> I wanted to ensure that the contents of the details column 'D' >>> is >>> > > not >>> > > > >>>> fetched during a query. A sample scenario >>> > > > >>>> User does a search. Sees Columns A,B,C. If they need more >>> details >>> > > for >>> > > > >>>> that particular record Click on a link that fetches D for that >>> > > particular >>> > > > >>>> record. >>> > >>> > > > >>>> So I tried to do something like - Select A, B, C from >>> tablename. >>> > >>> > > > >>>> I found from the documentation that the GQL query returns full >>> data >>> > > > >>>> objects and so all queries start with SELECT *. Is this true >>> for >>> > > JDOQL on >>> > > > >>>> the datastore as well? Does this mean everytime I query the >>> data >>> > > store its >>> > > > >>>> going to return all columns consuming bandwidth? >>> > >>> > > > >>>> Also since I want the content of COlumn D to be fetched on >>> > > subsequent >>> > > > >>>> user action so should I instead create two tables one with >>> > >>> > > > >>>> ID_TB1, A, B, C >>> > >>> > > > >>>> and the other one with >>> > >>> > > > >>>> ID, ID_TB1, D? >>> > >>> > > > >>>> Manny >>> > >>> > > > >>>> -- >>> > > > >>>> You received this message because you are subscribed to the >>> Google >>> > > > >>>> Groups "Google App Engine" group. >>> > > > >>>> To post to this group, send email to >>> > > [email protected]. >>> > > > >>>> To unsubscribe from this group, send email to >>> > > > >>>> [email protected]<google-appengine%[email protected]> >>> <google-appengine%[email protected]<google-appengine%[email protected]> >>> > >>> > > <google-appengine%[email protected]<google-appengine%[email protected]> >>> <google-appengine%[email protected]<google-appengine%[email protected]> >>> > >>> > >>> > > > >>>> . >>> > > > >>>> For more options, visit this group at >>> > > > >>>>http://groups.google.com/group/google-appengine?hl=en. >>> > >>> > > > >>> -- >>> > > > >>> Ikai Lan >>> > > > >>> Developer Programs Engineer, Google App Engine >>> > > > >>>http://googleappengine.blogspot.com| >>> http://twitter.com/app_engine >>> > >>> > > > >>> -- >>> > > > >>> You received this message because you are subscribed to the >>> Google >>> > > > >>> Groups "Google App Engine" group. >>> > > > >>> To post to this group, send email to >>> > > [email protected]. >>> > > > >>> To unsubscribe from this group, send email to >>> > > > >>> [email protected]<google-appengine%[email protected]> >>> <google-appengine%[email protected]<google-appengine%[email protected]> >>> > >>> > > <google-appengine%[email protected]<google-appengine%[email protected]> >>> <google-appengine%[email protected]<google-appengine%[email protected]> >>> > >>> > >>> > > > >>> . >>> > > > >>> For more options, visit this group at >>> > > > >>>http://groups.google.com/group/google-appengine?hl=en. >>> > >>> > > > >> -- >>> > > > >> You received this message because you are subscribed to the >>> Google >>> > > Groups >>> > > > >> "Google App Engine" group. >>> > > > >> To post to this group, send >>> > >>> > ... >>> > >>> > read more ยป >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Google App Engine" group. >>> To post to this group, send email to [email protected]. >>> To unsubscribe from this group, send email to >>> [email protected]<google-appengine%[email protected]> >>> . >>> For more options, visit this group at >>> http://groups.google.com/group/google-appengine?hl=en. >>> >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]<google-appengine%[email protected]> >> . >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<google-appengine%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
