Hi, You would probably be better off not using exando. Instead use a db.Model with a text or blob property that you serialize your dynamic fields into and a list property (like you outline) for indexing those values. You still won't be able to do inequality queries on multiple properties because the list must have one value the satisfies all filters.
There will be one index write for each value in the list. Namespaces will not matter, but you can only query within one namespace. The total number of items in your index won't impact your query performance, unless you're returning more entities for each query. If you're talking about 1,000 index writes per entity, writes are going to be very expensive. The solution I suggested will avoid the two index writes per dynamic property you would get with exando, which should help your performance quite a lot. Robert On Wed, Jun 29, 2011 at 14:07, someone1 <[email protected]> wrote: > Hello All, > > I'm trying to figure out a clean way to make my Expando model's > searchable. I will have many user-defined fields that will need to be > more or less searchable. This means that I would have to "normalize" > my data (such as lower case all fields) so that the search is case > insensitive. > > My idea is to create a ComputedProperty: > > @db.ComputedProperty > def quick_search(self): > return [ "%s:%s" % (k,str(self.__getattr__(k)).lower()) for k > in self.dynamic_properties()] if self.dynamic_properties() else None > > I will update that to better handle list types and such, but it gets > the idea across. This way, I can just query that single property to > search my models. I can do multiple inequality filters on more than > one property this way too! > > I feel like this would be a great way to accomplish what I want, > versus creating a "shadow" custom field for every user-defined field > with the data normalized. > > With a setup like this, I wouldn't need all my dynamic properties to > be indexed. Is there a way I can disable auto-indexing the dynamic > properties? I don't need to sort by any field, just search. > > Is there a better approach to my problem? I don't know how well this > would "scale" up with 1000 or more dynamic properties. Running a few > test on app engine led me to some memory issues when displaying more > than 100 entries with 1000 dynamic entries (near 300MB!). Also, with > 1000 dynamic properties, 77 entries took 2610342cpu_ms > 2570465api_cpu_ms, which is a LOT. > > I also had a question on how the indexes would work across namespaces. > Say I have 100 users using my app, each user creates 50 unique dynamic > properties in their respective namespace for this model. Does that > mean when writing the model, it's considered to have 5000 indexes (one > for each unique dynamic property)? Or will each namespace be treated > differently? Disabling indexes on dynamic properties would definitely > help me on this front. > > Thanks in advanced! > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
