You know I thought about that, is it really better to open a file and read the contents then just querying the database?
Maybe for a table of my size, opening files would be faster, so definately a consideration. So considering this solution, how would you load the keywords from a local file on a disk? If I am not wrong, you have to just download it via http because there is no direct local file load right? If so, then won't that also compound the loading times of a local file? On Dec 18, 2:34 am, "Nick Johnson (Google)" <[email protected]> wrote: > Hi, > > > > On Thu, Dec 17, 2009 at 10:48 PM, killer barney <[email protected]> wrote: > > sorry, hopefully I can explain in better detail :) > > > I have a list of keywords in a table that's the size of thousands. It > > changes, but not very often, so the list doesn't have to be updated > > all that often. > > > When I receive a body of text, I want to highlight the keywords in the > > body of text. So I have to constantly compare the body of text to my > > keywords. Because of that, I think querying would be too large a hit > > on the database. So I have a working version where I store in > > memcache keywords sharded by letter, for example storing all keywords > > starting with the letter 'A', and so forth. This is a temporary fix > > that will work for now but will cause problems later as my keywords > > exceed 1000 per letter. Not only that, when the memcache gets > > disposed, the user will have to query to populate that memcache, which > > could take a really long time for that unlucky person who has to > > populate the memcache for everybody else. > > > So looking for a better version of what I'm currently doing. > > Why not simply load the keywords from a local file on disk? > > -Nick Johnson > > > > > > > On Dec 16, 5:12 am, "Nick Johnson (Google)" <[email protected]> > > wrote: > > > Hi, > > > > It's difficult to comment usefully without more information. Where do you > > > get your list from? How is it calculated? How large is it? > > > > -Nick Johnson > > > > On Tue, Dec 15, 2009 at 11:11 PM, killer barney <[email protected]> > > wrote: > > > > I have a question that perhaps someone can give me some insight on. > > > > > I have a list of thousands of keywords in my website that I am going > > > > to constantly need to query and iterate through. So I thought rather > > > > than querying for every keyword in the datastore everytime, I should > > > > store it in the memcache. But even this doesn't seem like a very good > > > > solution as the memcache gets eliminated and I'm going to have to > > > > somehow query the whole list and store it back into memcache everytime > > > > it gets disposed of. > > > > > I thought about having the list stored as sharded arrays into > > > > memcache, so when it does get disposed of, then I only have to update > > > > the keywords that start with "N", for example, but this still doesn't > > > > seem like the best solution. > > > > > Is there a better way to do this? > > > > > -- > > > > > You received this message because you are subscribed to the Google > > Groups > > > > "Google App Engine" group. > > > > To post to this group, send email to [email protected] > > . > > > > To unsubscribe from this group, send email to > > > > [email protected]<google-appengine%[email protected]><google-appengine%2Bunsubscrib > > [email protected]> > > > > . > > > > For more options, visit this group at > > > >http://groups.google.com/group/google-appengine?hl=en. > > > > -- > > > Nick Johnson, Developer Programs Engineer, App Engine > > > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration > > Number: > > > 368047 > > > > -- > > > Nick Johnson, Developer Programs Engineer, App Engine > > > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration > > Number: > > > 368047 > > > -- > > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]<google-appengine%[email protected]> > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. > > -- > Nick Johnson, Developer Programs Engineer, App Engine > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: > 368047 -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
