Re: Best way to get automatic JavaScript minification in Restlet
Hello Tim,
thanks a lot for having entered an issue. This has been fixed (I've checked
with your code) a few minutes earlier, see
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447dsMessageId=2682395
.
The remaining size of a target buffer was not taken into account when
filling it.
Best regards,
Thierry Boileau
I just filed issue 1198 which has one file Java program that demonstrates
the problem.
http://restlet.tigris.org/issues/show_bug.cgi?id=1198
http://restlet.tigris.org/issues/show_bug.cgi?id=1198--tim
On Mon, Nov 8, 2010 at 9:03 PM, Tim Peierls t...@peierls.net wrote:
On Wed, Jul 28, 2010 at 6:27 PM, Tim Peierls t...@peierls.net wrote:
I wrote a Restlet Filter to minify application/x-javascript resources in
afterHandle (using JSMin now, but I'll probably switch to YUI Compressor).
As of 2.1-M1 (JSE edition), and using the internal connector only, I'm
getting a BufferOverflowException when I use this code in afterHandle:
Representation result =
new WriterRepresentation(mediaType, out.size()) {
public void write(Writer writer) throws IOException {
out.writeTo(writer);
}
};
response.setEntity(result);
where out is a CharArrayWriter that has been written to by a
YUICompressor instance and then flushed. It seems to only happen on fairly
large files, so it might be another manifestation of the
BufferOverflowException bug that was fixed back in January or so. I was
getting a different kind of exception in 2.0.x, something about writing to
closed socket, but it wasn't preventing things from working, whereas this
is.
As a workaround, I'm using this code:
Representation result =
new StringRepresentation(out.toString());
response.setEntity(result);
which means creating a copy of the entire minified file in memory, but
works with no warnings.
Another thing that might be worth mentioning is that the Filter this code
is part of is wrapped around a Directory with this code:
private Restlet getDirectoryOptionallyMinified(String path, boolean
minify) {
// XXX Can't figure out how to do this with Reference
construction,
// but it doesn't matter.
String ref = createClapReference(getClass().getPackage()) + / +
path;
Restlet directory = new Directory(getContext(), ref);
if (minify) {
Filter minified = new MinificationFilter();
minified.setNext(directory);
directory = minified;
}
return directory;
}
If no one sees any obvious gaffes in this code, I'll try to turn it into a
simple test program that illustrates the problem.
--tim
--
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447dsMessageId=2682449
Re: Best way to get automatic JavaScript minification in Restlet
I just filed issue 1198 which has one file Java program that demonstrates
the problem.
http://restlet.tigris.org/issues/show_bug.cgi?id=1198
http://restlet.tigris.org/issues/show_bug.cgi?id=1198--tim
On Mon, Nov 8, 2010 at 9:03 PM, Tim Peierls t...@peierls.net wrote:
On Wed, Jul 28, 2010 at 6:27 PM, Tim Peierls t...@peierls.net wrote:
I wrote a Restlet Filter to minify application/x-javascript resources in
afterHandle (using JSMin now, but I'll probably switch to YUI Compressor).
As of 2.1-M1 (JSE edition), and using the internal connector only, I'm
getting a BufferOverflowException when I use this code in afterHandle:
Representation result =
new WriterRepresentation(mediaType, out.size()) {
public void write(Writer writer) throws IOException {
out.writeTo(writer);
}
};
response.setEntity(result);
where out is a CharArrayWriter that has been written to by a
YUICompressor instance and then flushed. It seems to only happen on fairly
large files, so it might be another manifestation of the
BufferOverflowException bug that was fixed back in January or so. I was
getting a different kind of exception in 2.0.x, something about writing to
closed socket, but it wasn't preventing things from working, whereas this
is.
As a workaround, I'm using this code:
Representation result =
new StringRepresentation(out.toString());
response.setEntity(result);
which means creating a copy of the entire minified file in memory, but
works with no warnings.
Another thing that might be worth mentioning is that the Filter this code
is part of is wrapped around a Directory with this code:
private Restlet getDirectoryOptionallyMinified(String path, boolean
minify) {
// XXX Can't figure out how to do this with Reference construction,
// but it doesn't matter.
String ref = createClapReference(getClass().getPackage()) + / +
path;
Restlet directory = new Directory(getContext(), ref);
if (minify) {
Filter minified = new MinificationFilter();
minified.setNext(directory);
directory = minified;
}
return directory;
}
If no one sees any obvious gaffes in this code, I'll try to turn it into a
simple test program that illustrates the problem.
--tim
--
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447dsMessageId=2681369
Re: Best way to get automatic JavaScript minification in Restlet
On Wed, Jul 28, 2010 at 6:27 PM, Tim Peierls t...@peierls.net wrote:
I wrote a Restlet Filter to minify application/x-javascript resources in
afterHandle (using JSMin now, but I'll probably switch to YUI Compressor).
As of 2.1-M1 (JSE edition), and using the internal connector only, I'm
getting a BufferOverflowException when I use this code in afterHandle:
Representation result =
new WriterRepresentation(mediaType, out.size()) {
public void write(Writer writer) throws IOException {
out.writeTo(writer);
}
};
response.setEntity(result);
where out is a CharArrayWriter that has been written to by a YUICompressor
instance and then flushed. It seems to only happen on fairly large files, so
it might be another manifestation of the BufferOverflowException bug that
was fixed back in January or so. I was getting a different kind of exception
in 2.0.x, something about writing to closed socket, but it wasn't preventing
things from working, whereas this is.
As a workaround, I'm using this code:
Representation result =
new StringRepresentation(out.toString());
response.setEntity(result);
which means creating a copy of the entire minified file in memory, but works
with no warnings.
Another thing that might be worth mentioning is that the Filter this code is
part of is wrapped around a Directory with this code:
private Restlet getDirectoryOptionallyMinified(String path, boolean
minify) {
// XXX Can't figure out how to do this with Reference construction,
// but it doesn't matter.
String ref = createClapReference(getClass().getPackage()) + / +
path;
Restlet directory = new Directory(getContext(), ref);
if (minify) {
Filter minified = new MinificationFilter();
minified.setNext(directory);
directory = minified;
}
return directory;
}
If no one sees any obvious gaffes in this code, I'll try to turn it into a
simple test program that illustrates the problem.
--tim
--
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447dsMessageId=2680309
RE: Best way to get automatic JavaScript minification in Restlet
Hi all, This is a great use case that could be nicely solved by a combination of the TunnelService indeed, to define an encoding preference via a “min” extension, associated to an Encoding.JAVASCRIPT_MINIFICATION. This would update the Request#clientInfo#acceptedEncodings property. Then, the upcoming EncoderService could automatically wrap the response entity to minify if needed the JavaScript representations, using the EncoderFilter. To facilite the support of new encodings, we should support pluggable EncodingHelper that would allow new extensions to be discovered (YCompressor, JSMin, Zip, etc). I’ve added notes to this RFE: “Add EncoderService (similar to DecoderService)” http://restlet.tigris.org/issues/show_bug.cgi?id=208 Regarding caching, I think this should be treated as an orthogonal aspect across the board, based on response entities’ caching properties. The CachingService would have a pluggable storages, ideally the most requested entities should stay in memory while the less requested ones could land in a static directory, all configurable of course. “Add caching support” http://restlet.tigris.org/issues/show_bug.cgi?id=25 “Add caching support to Filter according to incoming Tag” http://restlet.tigris.org/issues/show_bug.cgi?id=772 All those features look really important to me at this point, so I’ve targeted them for 2.1 M1 and 2.1 M2. Best regards, Jerome -- Restlet ~ Founder and Technical Lead ~ http://www.restlet.org/ http://www.restlet.org Noelios Technologies ~ http://www.noelios.com/ http://www.noelios.com De : Thierry Boileau [mailto:thierry.boileau.noel...@gmail.com] Envoyé : vendredi 27 août 2010 16:21 À : discuss@restlet.tigris.org Objet : Re: Best way to get automatic JavaScript minification in Restlet Hello Tim, I'm sorry for my very late answer. You can also wrap the response's representation into a custom one that aims at applying the minification process. You can have a look at the Encoder filter, especially in the encode method which uses an EncodeRepresentation. Best regards, Thierry Boileau Perfect, thanks! --tim On Wed, Jul 28, 2010 at 7:43 PM, Tal Liron tal.li...@threecrickets.com wrote: I've handled it differently for Prudence: I create static unified/minifed files on-the-fly to be served via a Directory. If you request all.min.js, all .js files in that directory are unified (in alphabetical order) and minifed. The filter can detect if any of the source files has been updated, and the interval between such tests can be configured. This seems to me the best way to go because the case for serving static files has already been optimized. (Other static web servers/CDNs can easily be used instead of Restlet's Directory.) See the code in JavaScriptUnifyMinifyFilter (LGPL): http://code.google.com/p/prudence/source/browse/#svn/trunk/java/com.threecrickets.prudence/src/com/threecrickets/prudence/util%3Fstate%3Dclosed There's also a similar class for CSS. (There might be an issue with method and concurrent file locking on certain operating systems -- haven't done a lot of testing yet, but I'm sure that case would be easy to fix.) -Tal On 07/28/2010 05:27 PM, Tim Peierls wrote: I wrote a Restlet Filter to minify application/x-javascript resources in afterHandle (using JSMin now, but I'll probably switch to YUI Compressor). But is that all I can do with it? It feels as though this should be plugged into the TunnelService, with ways to take requests for foo.min.js, say, and return a minified foo.js, with caching and all those goodies, yet still return plain, unminified foo.js when it's requested. Feels like this must have been done already, but I can't find anything. Pointers? Advice? --tim -- http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447 http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447dsMessageId=2639913 dsMessageId=2639913 -- http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447dsMessageId=2653449
Re: Best way to get automatic JavaScript minification in Restlet
I agree that this is important! It's also quite a challenge to do caching right. In particular, you want to make sure that you are providing enough transparency to developers, choosing the right cache key per circumstance, etc. Most server frameworks that do dare to integrate caching do a poor job at it. From my experience, I would suggest to set a more cautious roadmap: perhaps a certain set of caching features for 2.1, with an eye towards more sophistication in a future version of Restlet. I would prefer that instead of configurable automation, we allow for a way for users to programatically cache, and just make sure that Restlet stays out of the way. I'll give an example: I've recently implemented caching of compressed (gzipped) output. I did not want to use the EncoderRepresentation, because it would encode per request. To accomplish my goal, I had to create a new kind of representation (ByteArrayRepresentation) that would allow the compressed data to pass through Restlet. It works great, but it's all a bit awkward, because I'm dealing with multiple representation classes. And so, I'm wondering if anyone has an idea for a representation API that would let me and other devs do the caching while maintaining more coherence. One thought -- A RepresentationIdentifier class, sitting between RepresentationInfo and Representation. What it adds to RepesentationInfo is some metadata about caching: cache keys, groups, partitions, backends, etc. I would then be able to return a RepresentationIdentifier from a @get method instead of a full-blown Representation, which would cause the CachingService to replace it with a real Representation, perhaps by calling another callback in my resource. If the cache entry is somehow invalid, a different route would have to happen to force my @get to generate a full Representation. I see the whole process working a little bit like negotiation (with getInfo and get). -Tal On 08/31/2010 05:35 AM, Jerome Louvel wrote: Hi all, This is a great use case that could be nicely solved by a combination of the TunnelService indeed, to define an encoding preference via a “min” extension, associated to an Encoding.JAVASCRIPT_MINIFICATION. This would update the Request#clientInfo#acceptedEncodings property. Then, the upcoming EncoderService could automatically wrap the response entity to minify if needed the JavaScript representations, using the EncoderFilter. To facilite the support of new encodings, we should support pluggable EncodingHelper that would allow new extensions to be discovered (YCompressor, JSMin, Zip, etc). I’ve added notes to this RFE: “Add EncoderService (similar to DecoderService)” http://restlet.tigris.org/issues/show_bug.cgi?id=208 Regarding caching, I think this should be treated as an orthogonal aspect across the board, based on response entities’ caching properties. The CachingService would have a pluggable storages, ideally the most requested entities should stay in memory while the less requested ones could land in a static directory, all configurable of course. “Add caching support” http://restlet.tigris.org/issues/show_bug.cgi?id=25 “Add caching support to Filter according to incoming Tag” http://restlet.tigris.org/issues/show_bug.cgi?id=772 All those features look really important to me at this point, so I’ve targeted them for 2.1 M1 and 2.1 M2. Best regards, Jerome -- Restlet ~ Founder and Technical Lead ~ http://www.restlet.o rg http://www.restlet.org/ Noelios Technologies ~ http://www.noelios.com http://www.noelios.com/ *De :* Thierry Boileau [mailto:thierry.boileau.noel...@gmail.com] *Envoyé :* vendredi 27 août 2010 16:21 *À :* discuss@restlet.tigris.org *Objet :* Re: Best way to get automatic JavaScript minification in Restlet Hello Tim, I'm sorry for my very late answer. You can also wrap the response's representation into a custom one that aims at applying the minification process. You can have a look at the Encoder filter, especially in the encode method which uses an EncodeRepresentation. Best regards, Thierry Boileau Perfect, thanks! --tim On Wed, Jul 28, 2010 at 7:43 PM, Tal Liron tal.li...@threecrickets.com mailto:tal.li...@threecrickets.com wrote: I've handled it differently for Prudence: I create static unified/minifed files on-the-fly to be served via a Directory. If you request all.min.js, all .js files in that directory are unified (in alphabetical order) and minifed. The filter can detect if any of the source files has been updated, and the interval between such tests can be configured. This seems to me the best way to go because the case for serving static files has already been optimized. (Other static web servers/CDNs can easily be used instead of Restlet's Directory.) See the code in JavaScriptUnifyMinifyFilter (LGPL): http://code.google.com/p
Re: Best way to get automatic JavaScript minification in Restlet
Hello Tim, I'm sorry for my very late answer. You can also wrap the response's representation into a custom one that aims at applying the minification process. You can have a look at the Encoder filter, especially in the encode method which uses an EncodeRepresentation. Best regards, Thierry Boileau Perfect, thanks! --tim On Wed, Jul 28, 2010 at 7:43 PM, Tal Liron tal.li...@threecrickets.comwrote: I've handled it differently for Prudence: I create static unified/minifed files on-the-fly to be served via a Directory. If you request all.min.js, all .js files in that directory are unified (in alphabetical order) and minifed. The filter can detect if any of the source files has been updated, and the interval between such tests can be configured. This seems to me the best way to go because the case for serving static files has already been optimized. (Other static web servers/CDNs can easily be used instead of Restlet's Directory.) See the code in JavaScriptUnifyMinifyFilter (LGPL): http://code.google.com/p/prudence/source/browse/#svn/trunk/java/com.threecrickets.prudence/src/com/threecrickets/prudence/util%3Fstate%3Dclosed There's also a similar class for CSS. (There might be an issue with method and concurrent file locking on certain operating systems -- haven't done a lot of testing yet, but I'm sure that case would be easy to fix.) -Tal On 07/28/2010 05:27 PM, Tim Peierls wrote: I wrote a Restlet Filter to minify application/x-javascript resources in afterHandle (using JSMin now, but I'll probably switch to YUI Compressor). But is that all I can do with it? It feels as though this should be plugged into the TunnelService, with ways to take requests for foo.min.js, say, and return a minified foo.js, with caching and all those goodies, yet still return plain, unminified foo.js when it's requested. Feels like this must have been done already, but I can't find anything. Pointers? Advice? --tim -- http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447dsMessageId=2639913 -- http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447dsMessageId=2652094
Best way to get automatic JavaScript minification in Restlet
I wrote a Restlet Filter to minify application/x-javascript resources in afterHandle (using JSMin now, but I'll probably switch to YUI Compressor). But is that all I can do with it? It feels as though this should be plugged into the TunnelService, with ways to take requests for foo.min.js, say, and return a minified foo.js, with caching and all those goodies, yet still return plain, unminified foo.js when it's requested. Feels like this must have been done already, but I can't find anything. Pointers? Advice? --tim -- http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447dsMessageId=2639902
Re: Best way to get automatic JavaScript minification in Restlet
I've handled it differently for Prudence: I create static unified/minifed files on-the-fly to be served via a Directory. If you request all.min.js, all .js files in that directory are unified (in alphabetical order) and minifed. The filter can detect if any of the source files has been updated, and the interval between such tests can be configured. This seems to me the best way to go because the case for serving static files has already been optimized. (Other static web servers/CDNs can easily be used instead of Restlet's Directory.) See the code in JavaScriptUnifyMinifyFilter (LGPL): http://code.google.com/p/prudence/source/browse/#svn/trunk/java/com.threecrickets.prudence/src/com/threecrickets/prudence/util%3Fstate%3Dclosed There's also a similar class for CSS. (There might be an issue with method and concurrent file locking on certain operating systems -- haven't done a lot of testing yet, but I'm sure that case would be easy to fix.) -Tal On 07/28/2010 05:27 PM, Tim Peierls wrote: I wrote a Restlet Filter to minify application/x-javascript resources in afterHandle (using JSMin now, but I'll probably switch to YUI Compressor). But is that all I can do with it? It feels as though this should be plugged into the TunnelService, with ways to take requests for foo.min.js, say, and return a minified foo.js, with caching and all those goodies, yet still return plain, unminified foo.js when it's requested. Feels like this must have been done already, but I can't find anything. Pointers? Advice? --tim -- http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447dsMessageId=2639913
Re: Best way to get automatic JavaScript minification in Restlet
Perfect, thanks! --tim On Wed, Jul 28, 2010 at 7:43 PM, Tal Liron tal.li...@threecrickets.comwrote: I've handled it differently for Prudence: I create static unified/minifed files on-the-fly to be served via a Directory. If you request all.min.js, all .js files in that directory are unified (in alphabetical order) and minifed. The filter can detect if any of the source files has been updated, and the interval between such tests can be configured. This seems to me the best way to go because the case for serving static files has already been optimized. (Other static web servers/CDNs can easily be used instead of Restlet's Directory.) See the code in JavaScriptUnifyMinifyFilter (LGPL): http://code.google.com/p/prudence/source/browse/#svn/trunk/java/com.threecrickets.prudence/src/com/threecrickets/prudence/util%3Fstate%3Dclosed There's also a similar class for CSS. (There might be an issue with method and concurrent file locking on certain operating systems -- haven't done a lot of testing yet, but I'm sure that case would be easy to fix.) -Tal On 07/28/2010 05:27 PM, Tim Peierls wrote: I wrote a Restlet Filter to minify application/x-javascript resources in afterHandle (using JSMin now, but I'll probably switch to YUI Compressor). But is that all I can do with it? It feels as though this should be plugged into the TunnelService, with ways to take requests for foo.min.js, say, and return a minified foo.js, with caching and all those goodies, yet still return plain, unminified foo.js when it's requested. Feels like this must have been done already, but I can't find anything. Pointers? Advice? --tim -- http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447dsMessageId=2639913 -- http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447dsMessageId=2639942
