This relates more to implementation of the proxying server itself
rather than the gadget spec... I would like to see that the
proxying
be useful for people who do flash development. The proxying, if not
implemented with flash in mind, will be a security hole for flash
developers. We use crossdomain.xml as a security mechanism to
prevent
unwanted access and including *.gmodules.com, *.msappspace.com, or
*.hi5modules.com as an allowed domain in our crossdomain isn't
secure
if those domains proxy arbitrary urls. We may as well allow access
from all domains.
I see 2 ways we can get around the problem:
1) Create urls for proxied content using the original domain as
part
of the subdomain. Thus, something a url like
http://static.myminilife.com/someswf.swf
would be cached at static.myminilife.gmodules.com/... . Thus, we
can
allow the static.myminilife.gmodules.com specifically in the
crossdomain file. The proxying server would have to ensure that the
subdomain is the same as the url being proxied.
2) Do filtering of the request's http host against the proxied url
being asked for. With that in place, then I can create a CNAME
pointing one of our subdomains to the server doing the proxying.
Ie,
gmodules.myminilife.com points to gmodules.com. When I have a swf
at
static.myminilife.com/someswf.swf, I can replace that url with
gmodules.myminilife.com, which will retrieve the content from the
proxying server. Like above, The proxying server has to ensure that
the request's http host is the same as the content being proxied.
On May 28, 10:21 am, "Louis Ryan" <[EMAIL PROTECTED]> wrote:
Glad to hear it. Currently the implementation in Shindig is a
work in
progress though it already provides useful functionality. In
particular
because the configurable rewriting rules need to carry across a
chain
of
requests emanating from a gadget we need to carry the ability to
lookup
the
rule through that chain, this is something that is not yet
implemented.
Nor
for that matter is the ability to include or exclude URLs be regex
though
that will be added soon.
On Wed, May 28, 2008 at 1:12 AM, Kevin Brown <[EMAIL PROTECTED]>
wrote:
On Wed, May 28, 2008 at 12:39 AM, Paul Walker
<[EMAIL PROTECTED]>
wrote:
Great news. Without caching rules, we are definitely prepared
to
support this.
However, speaking of Shindig and the process of proposals…
since you
have
contributed the code to Shindig, is it just expected to be
approved
as a
proposal without voting?
No -- Shindig will always implement whatever the standard
specifies,
so if
the standard deviates from Louis' current proposal Shindig will
be
modified
to support it. Shindig is a useful test bed for ironing out
issues in
a
proposal before they are finalized, with enough users to easily
test
a wide
range of container deployment needs.
And on the specifics of what has been contributed to Shindig,
what
does
it actually do?
What Louis has contributed so far rewrites links as proposed on
this
thread. The content is examined for any links and those links are
rewritten
to go through a caching proxy. Several optimizations are done
to try
to
reduce http overhead, such as merging contiguous http requests
into
one.
We've deployed this to the Orkut sandbox as well to get some
real-world
feedback from developers. You can take a look at the code here:
http://svn.apache.org/viewvc/incubator/shindig/trunk/java/gadgets/src
...
Is this a specific to your provider? Where does the code you
contributed
actually persist the content and rewrite the URLs to?
Shindig's proxy provides a cache interface that users can
implement
to
store the data however they like. A default in memory LRU cache
is
provided.
Thanks,
Paul
*From:* [EMAIL PROTECTED] [mailto:
[EMAIL PROTECTED] *On Behalf Of
*Louis
Ryan
*Sent:* Thursday, May 22, 2008 2:11 PM
*To:* [EMAIL PROTECTED]
*Cc:* [EMAIL PROTECTED]; Bryan Green
*Subject:* Re: PROPOSAL : Rewriting links in generated and
proxied
content
Paul,
You certainly can use the include rule to specify the parameter
matching
implementation and just document that for your container. The
implementation
in shindig no longer cares about recognized file extensions but
rather uses
the referencing HTML tag (script, link, img & embed currently)
to
determine
whether rewriting should occur so the revised proposal looks
like....
<Optional feature="content-rewrite">
<Param name="include-tags">script,link,img,embed</Param>
<Param name="include-pattern">.*</Param>
<Param name="exclude-pattern"></Param>
</Optional>
Params are shown with their default values, defaults are used if
param is
omitted.
Exclude pattern overrides include pattern if exclude pattern is
defined.
The feature can be turned off by explicitly setting include-
tags to
an empty
list of exclude-pattern to '.*'
I didnt mean to imply that the cache-control rules are defined
by
this
feature, thats an implementation detail that containers are
free to
change
at their discretion. I provided the discussion to give folks
some
idea what
a typical container might do. Completely agree that good
developers
will
version content and that containers are free to enforce this
policy
by
setting cache-control to eons on rewritten content.
-Louis
On Fri, May 16, 2008 at 7:10 PM, Paul Walker
<[EMAIL PROTECTED]>
wrote:
Our thoughts at MySpace were to parse and persist external
resources
on
our end and the developer would simply opt in by supplying a
standardized
query parameter on the URI of the resource. I think we can
understand why
developers would want us to persist/cache some images/swfs for
them
and not
others. We need a way for users to opt in individual files and
while the
pattern matching would work, the parameter just seems easier. I
suppose for
users that create apps on our developer site, we could publish
the
use of
this parameter as a pattern match and still be compliant with
the
pattern
matching detail of this proposal.
We'd like to obey caching headers, but would need to have a
significant
floor. We will be storing the parsed resources on Akamai and
would
have to
run a job that updated these files and I don't like the
messiness of
that.
I don't see developers having a huge issue versioning URIs as is
typically
done w/ JS libraries and update their gadget xml, so I'm not so
concerned
about this. I'm more concerned about the 90%+ developers that
don't
properly use cache control headers. Developers that are
sophisticated
enough to and have the need to invalidate these resources
periodically are
sophisticated enough to use URI versioning and update their
gadget
xml.
Again, they also have the option of hosting the resource
themselves
and
controlling any complicated/varying user-agent caching they
have on
the
resource.
We pre-process all markup when it is saved/published and the
live
version
for installed users is retrieved already compressed to gzip (for
users that
don't already have it cached on their user-agent), so we don't
have
the same
performance implications as shindig it sounds like. I assume
that
Shindig
could store a lookup of cached urls and check/replace where it
finds
matches
and where it does not, throw the work of fetching/caching to a
background
task and not rewrite those until it has, but I'll leave that
up to
the
Shindig experts.
Cc'ed Bryan Green, the main developer who has already finished
most
of our
implementation, if someone would like to discuss. We're keen on
this
support so a general BIG +1 here. Could we separate the cache
control
portion of this proposal and move forward with resource
parsing/persisting
w/out it for now?
Thx…
~Paul
*From:* [EMAIL PROTECTED] [mailto:
[EMAIL PROTECTED] *On Behalf Of
*Louis
Ryan
*Sent:* Friday, May 16, 2008 4:47 PM
*To:* [EMAIL PROTECTED]
*Cc:* [EMAIL PROTECTED]
*Subject:* Re: PROPOSAL : Rewriting links in generated and
proxied
content
An initial implementation of this is now available as a patch
for
Shindig
https://issues.apache.org/jira/browse/SHINDIG-276
On Wed, May 7, 2008 at 5:22 PM, Louis Ryan <[EMAIL PROTECTED]>
wrote:
1. Yes
2. Yes, My sample implementation is now doing this
3. To some extent. Developers have a lot of control using the
cache-control headers. Containers are likely to require a
minimum
value
On Wed, May 7, 2008 at 4:40 PM, Graham Spencer <[EMAIL PROTECTED]>
wrote:
Thanks Louis. A few minor questions:
[1] I assume DEFAULT should also include CSS files. It might
make
sense to
base DEFAULT not on pattern matching targets but rather on the
context (e.g.
<IMG>, CSS, etc.).
[2] Will CSS files be parsed to extract background images and
such?
[3] Should we give developers control over timeouts?
--g
On Tue, May 6, 2008 at 4:48 PM, Louis Ryan <[EMAIL PROTECTED]>
wrote:
Hi,
Many containers offer the ability to rewrite links to use their
proxy
loading mechanism using *gadgets.io.getProxyUrl() *Typically the
generated proxy URL can be expected to give better download
performance for
users of that particular container and also potentially
reducing the
load on
gadget developer backends. Currently however there is no
standard
way for a
gadget to have links in its generated markup be rewritten to
use the
proxy
URL format. I would like to propose creating a new feature to
allow
for
rewriting of links in generated gadget content
Create a new standard gadget feature called *proxy-rewriter
*which
allows
gadgets to control whether they want content re-writing enabled.
Containers
can choose to turn the feature on by default for all gadgets and
gadgets can
use this mechanism to opt-out.
An 'include' param is used to control which URLs to rewrite:
- ALL - All URLs in the content
- DEFAULT - is recognized static file extension types such as
.js,
.png, .gif ...
- NONE - disables the feature even if it is enabled by
default by
the
container
An 'include-pattern' and 'exclude-pattern' can be specified to
implement
more exact filtering rules. Patterns are applied to the URL to
rewrite,
excludes are processed after includes
An 'apply-to' is used to specify the comma separated list of
mime-types
which the rewriter should recognize and rewrite. By default
the list
is
text/html,text/xml,application/xml (suggestions welcome here)
<Optional feature="proxy-rewriter">
<Param name="include">DEFAULT</Param>
<Param
name="include-pattern">.*\/mystaticcontent\/.*</Param>
<Param
name="exclude-pattern">.*\/mynonstaticcontent\/.*</Param>
<Param name="apply-to">text/html</Param>
</Optional>
This feature will not only impact the content generated when the
gadget is
rendered but is also used to control whether any content fetched
through
makeRequest, Preload and proxied URLs is also rewritten.
It is probably also
...
read more »
