Our thoughts at MySpace were to parse and persist external resources on our end and the developer would simply opt in by supplying a standardized query parameter on the URI of the resource. I think we can understand why developers would want us to persist/cache some images/swfs for them and not others. We need a way for users to opt in individual files and while the pattern matching would work, the parameter just seems easier. I suppose for users that create apps on our developer site, we could publish the use of this parameter as a pattern match and still be compliant with the pattern matching detail of this proposal.
We'd like to obey caching headers, but would need to have a significant floor. We will be storing the parsed resources on Akamai and would have to run a job that updated these files and I don't like the messiness of that. I don't see developers having a huge issue versioning URIs as is typically done w/ JS libraries and update their gadget xml, so I'm not so concerned about this. I'm more concerned about the 90%+ developers that don't properly use cache control headers. Developers that are sophisticated enough to and have the need to invalidate these resources periodically are sophisticated enough to use URI versioning and update their gadget xml. Again, they also have the option of hosting the resource themselves and controlling any complicated/varying user-agent caching they have on the resource. We pre-process all markup when it is saved/published and the live version for installed users is retrieved already compressed to gzip (for users that don't already have it cached on their user-agent), so we don't have the same performance implications as shindig it sounds like. I assume that Shindig could store a lookup of cached urls and check/replace where it finds matches and where it does not, throw the work of fetching/caching to a background task and not rewrite those until it has, but I'll leave that up to the Shindig experts. Cc'ed Bryan Green, the main developer who has already finished most of our implementation, if someone would like to discuss. We're keen on this support so a general BIG +1 here. Could we separate the cache control portion of this proposal and move forward with resource parsing/persisting w/out it for now? Thx... ~Paul From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Louis Ryan Sent: Friday, May 16, 2008 4:47 PM To: [EMAIL PROTECTED] Cc: [email protected] Subject: Re: PROPOSAL : Rewriting links in generated and proxied content An initial implementation of this is now available as a patch for Shindig https://issues.apache.org/jira/browse/SHINDIG-276 On Wed, May 7, 2008 at 5:22 PM, Louis Ryan <[EMAIL PROTECTED]> wrote: 1. Yes 2. Yes, My sample implementation is now doing this 3. To some extent. Developers have a lot of control using the cache-control headers. Containers are likely to require a minimum value On Wed, May 7, 2008 at 4:40 PM, Graham Spencer <[EMAIL PROTECTED]> wrote: Thanks Louis. A few minor questions: [1] I assume DEFAULT should also include CSS files. It might make sense to base DEFAULT not on pattern matching targets but rather on the context (e.g. <IMG>, CSS, etc.). [2] Will CSS files be parsed to extract background images and such? [3] Should we give developers control over timeouts? --g On Tue, May 6, 2008 at 4:48 PM, Louis Ryan <[EMAIL PROTECTED]> wrote: Hi, Many containers offer the ability to rewrite links to use their proxy loading mechanism using gadgets.io.getProxyUrl() Typically the generated proxy URL can be expected to give better download performance for users of that particular container and also potentially reducing the load on gadget developer backends. Currently however there is no standard way for a gadget to have links in its generated markup be rewritten to use the proxy URL format. I would like to propose creating a new feature to allow for rewriting of links in generated gadget content Create a new standard gadget feature called proxy-rewriter which allows gadgets to control whether they want content re-writing enabled. Containers can choose to turn the feature on by default for all gadgets and gadgets can use this mechanism to opt-out. An 'include' param is used to control which URLs to rewrite: * ALL - All URLs in the content * DEFAULT - is recognized static file extension types such as .js, .png, .gif ... * NONE - disables the feature even if it is enabled by default by the container An 'include-pattern' and 'exclude-pattern' can be specified to implement more exact filtering rules. Patterns are applied to the URL to rewrite, excludes are processed after includes An 'apply-to' is used to specify the comma separated list of mime-types which the rewriter should recognize and rewrite. By default the list is text/html,text/xml,application/xml (suggestions welcome here) <Optional feature="proxy-rewriter"> <Param name="include">DEFAULT</Param> <Param name="include-pattern">.*\/mystaticcontent\/.*</Param> <Param name="exclude-pattern">.*\/mynonstaticcontent\/.*</Param> <Param name="apply-to">text/html</Param> </Optional> This feature will not only impact the content generated when the gadget is rendered but is also used to control whether any content fetched through makeRequest, Preload and proxied URLs is also rewritten. It is probably also worthwhile mentioning how re-writing a URL to be proxied impacts the caching behavior of the content. Containers will cache content fetched through the proxy, in general containers are likely to favor a simpler expires/max-age style cache control policy rather than the more complicated to implement and more latency sensitive Last-Modified/If-Modified-Since which is the default mechanism Apache and other webservers use when serving static files. Containers are free to make policy decisions about how to alter the cache-control headers of content fetched through their proxy. A sample policy might look like: - Pragma : no-cache & Cache-Control : no-cache,no-store are always respected and this content will never be cached by the proxy - Expires and Cache-Control : max-age are always respected. If expiration is shorter than one day and the URL is a recognized static file type then expiration is forced to 1 day minimum - ETag & Last-Modified are stripped if Expires or Cache-Control is set or the URL is a recognized static file type. Such a policy is attempting to balance the needs of sophisticated users of real If-Modified requests and the default configuration of common static file serving configurations. Thoughts? -Louis --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "OpenSocial and Gadgets Specification Discussion" group. To post to this group, send email to [EMAIL PROTECTED] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/opensocial-and-gadgets-spec?hl=en -~----------~----~----~----~------~----~------~--~---
