All, Beta 0.9 of "GIS in the Cloud" is now a little faster.
1) The app now uses Memcache to store the spatial index. We check to see if it is there before performing the spatial query with it. If it is not there, we create it from the serialized version on disk, then drop it in the cache. See http://code.google.com/p/giscloud/source/browse/trunk/giscloud/src/org/emcode/samples/giscloud/server/GisCloudServiceImpl.java#55 2) The load-on-startup option for the servlet is in web.xml, which inits() the servlet that handles the app, during which time the spatial index is created and put in the cache if not already there. These actions are logged so I can see when App Engine is spinning up my app. See http://code.google.com/p/giscloud/source/browse/trunk/giscloud/war/WEB-INF/web.xml#60 and the link above to the servlet. Toby, I am still getting that initial latency (less than 2 seconds) on my initial click -- not only after first going to the app after deployment (expected) but even coming to it after a few minutes of application inactivity. It seems App Engine is thinking about something -- like maybe where my app is. Subsequent clicks are milliseconds. Can you comment or offer any suggestions? On Sep 26, 11:04 pm, Stuart Moffatt <[email protected]> wrote: > Toby, > > I should have thought of it earlier, but you were exactly right in > suggesting the static var for the spatial index. First hit was 9760 > milliseconds (as App Engine was spinning up). Many subsequent hits were sub > 25 milliseconds. Some climbed up a little, but were still less than 3/4 > second. > > Will take a look at the load-on-startup option next, and figure out the best > way to share the index across different servlets. > > Stuart > > On Sat, Sep 26, 2009 at 9:39 PM, Stuart Moffatt > <[email protected]>wrote: > > > Toby, > > > Thanks for those tips. I will look into the load-on-startup option for the > > servlet, as well as caching the index locally on servlet init. And, allow > > for the spin up to handle new versions for a few tests. > > > Stuart > > > On Sat, Sep 26, 2009 at 9:34 PM, Toby Reyelts <[email protected]> wrote: > > >> Stuart, > >> When you were trying to get benchmarking numbers, are you sure you were > >> factoring out cases in which App Engine was creating new application > >> instances? > > >> In general, you have to be very careful about ensuring that you're making > >> apples-to-apples comparisons. For example, if you run the test several > >> times > >> from your local box, your operating system will have surely cached your > >> files off of disk. If you run the test several times against App Engine, > >> you > >> may end up hitting several different physical machines, all of which are > >> spinning up new instances of your aplication. > > >> One piece of advice I have been giving is to log from a servlet registered > >> as load-on-startup. In this way, you can see which requests involve a new > >> instance of your application. Every time you upload a new version of your > >> app, should expect this to happen several times, as App Engine has to spin > >> up new application instances to support your new version. > > >> Another question for you is if you really need to do this on every single > >> request. I assume your index is read-only, and it sounds relatively small. > >> Given that, it sounds like a perfect candidate for being read upon > >> initialization and cached locally, for example, in a static variable. > > >> On Sat, Sep 26, 2009 at 9:06 PM, Stuart Moffatt > >> <[email protected]>wrote: > > >>> All, > > >>> I am trying to implement spatial indexes via JTS on App Engine and > >>> have had some success. But, in order to make my spatial queries > >>> perform better, I should be using a spatial index. In an offline > >>> application, these indexes could be created once at runtime and used > >>> for the duration of the application. However, in an stateless online > >>> setting I would be recreating indexes every request. To circumvent > >>> that overhead, I decided to test how well App Engine would handle > >>> deserializing a spatial JTS index from the filesystem, and found > >>> performance to be pretty sad. > > >>> A description of the steps I took can be found here: > >>>http://code.google.com/p/giscloud/wiki/DeserializingPerformance > > >>> I am thinking there are a number of possible problem areas: > > >>> 1) App Engine does not like to read larger files from the file system. > >>> My serialized spatial index is 1.25MB. > > >>> 2) App Engine does not like to deserialize in memory (or perhaps my > >>> app's CPU threshold is being throttled when it starts to deserialize) > > >>> 3) My implementation of modifying the JTS STRtree spatial index to > >>> make it serializable is less than desirable. > > >>> The link above has comparable times for the routine to run in hosted > >>> development mode and in production. > > >>> Any thoughts are welcome. > > >>> Stuart --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~----------~----~----~----~------~----~------~--~---
